The build commands above will use your local configuration, but pull
The build commands above will use your local configuration, but will pull
application code from the master branch of the application's repository. If you
application code from the master branch of the application's repository. If you
would like to use code from another branch/tag/hash, modify the ``*_VERSION``
would like to use code from another branch/tag/hash, modify the ``*_VERSION``
variable that lives in the ``ansible_overrides.yml`` file beside the
variable that lives in the ``ansible_overrides.yml`` file beside the
...
@@ -277,13 +283,15 @@ E-Commerce Service, you would modify ``ECOMMERCE_VERSION`` in
...
@@ -277,13 +283,15 @@ E-Commerce Service, you would modify ``ECOMMERCE_VERSION`` in
How do I create database dumps?
How do I create database dumps?
-------------------------------
-------------------------------
We use database dumps to speed up provisioning and generally spend less time running migrations. These dumps should be
We use database dumps to speed up provisioning and generally spend less time running migrations. These dumps should be
updated occasionally--when database migrations take a prolonged amount of time, or we want to incorporate changes that
updated occasionally - when database migrations take a prolonged amount of time *or* we want to incorporate changes that
require manual intervention. The process below details how to update the database dumps.
require manual intervention.
To update the database dumps:
1. Destroy and/or backup the data for your existing devstack so that you start with a clean slate.
1. Destroy and/or backup the data for your existing devstack so that you start with a clean slate.
2. Disable the loading of the existing database dumps during provisioning by commenting out any calls to ``load-db.sh``
2. Disable the loading of the existing database dumps during provisioning by commenting out any calls to ``load-db.sh``
in the provisioning scripts. This ensures that we start with a completely fresh database and incorporate any changes
in the provisioning scripts. This disabling ensures a start with a completely fresh database and incorporates any changes
that may have require some form of manual intervention for existing installations (e.g. drop/move tables).
that may have required some form of manual intervention for existing installations (e.g. drop/move tables).
3. Provision devstack with ``make provision``.
3. Provision devstack with ``make provision``.
4. Dump the databases and open a pull request with your updates:
4. Dump the databases and open a pull request with your updates:
...
@@ -308,8 +316,8 @@ Alternatively, you can discard and rebuild the entire database for all
...
@@ -308,8 +316,8 @@ Alternatively, you can discard and rebuild the entire database for all
devstack services by re-running ``make dev.provision`` or
devstack services by re-running ``make dev.provision`` or
``make dev.sync.provision`` as appropriate for your configuration. Note that
``make dev.sync.provision`` as appropriate for your configuration. Note that
if your branch has fallen significantly behind master, it may not include all
if your branch has fallen significantly behind master, it may not include all
of the migrations included in the database dump used by provisioning. In
of the migrations included in the database dump used by provisioning. In these
cases like this it's usually best to first rebase the branch onto master to
cases, it's usually best to first rebase the branch onto master to
get the missing migrations.
get the missing migrations.
...
@@ -358,7 +366,7 @@ starts, you have a few options:
...
@@ -358,7 +366,7 @@ starts, you have a few options:
reads ``...&& while true; do...`` could be changed to
reads ``...&& while true; do...`` could be changed to
``...&& pip install my-new-package && while true; do...``.
``...&& pip install my-new-package && while true; do...``.
* In order to work on locally pip-installed repos like edx-ora2, first clone
* In order to work on locally pip-installed repos like edx-ora2, first clone
them into ../src (relative to this directory). Then, inside your lms shell,
them into ``../src`` (relative to this directory). Then, inside your lms shell,
you can ``pip install -e /edx/src/edx-ora2``. If you want to keep this code
you can ``pip install -e /edx/src/edx-ora2``. If you want to keep this code
installed across stop/starts, modify ``docker-compose.yml`` as mentioned
installed across stop/starts, modify ``docker-compose.yml`` as mentioned
above.
above.
...
@@ -440,6 +448,7 @@ To detach from the container, you'll need to stop the container with:
...
@@ -440,6 +448,7 @@ To detach from the container, you'll need to stop the container with:
make stop
make stop
or a manual Docker command to bring down the container:
or a manual Docker command to bring down the container:
.. code:: sh
.. code:: sh
docker kill $(docker ps -a -q --filter="name=edx.devstack.<container name>")
docker kill $(docker ps -a -q --filter="name=edx.devstack.<container name>")
...
@@ -507,8 +516,7 @@ described above (Chrome is used by default).
...
@@ -507,8 +516,7 @@ described above (Chrome is used by default).
Troubleshooting: General Tips
Troubleshooting: General Tips
-----------------------------
-----------------------------
If you are having trouble with your containers there are a few general steps
If you are having trouble with your containers, this sections contains some troubleshooting tips.
you can take to try to resolve.
Check the logs
Check the logs
~~~~~~~~~~~~~~
~~~~~~~~~~~~~~
...
@@ -544,7 +552,7 @@ Clean the containers
...
@@ -544,7 +552,7 @@ Clean the containers
~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~
Sometimes containers end up in strange states and need to be rebuilt. Run
Sometimes containers end up in strange states and need to be rebuilt. Run
``make down`` to remove all containers and networks. This will NOT remove your
``make down`` to remove all containers and networks. This will **NOT** remove your
data volumes.
data volumes.
Start over
Start over
...
@@ -560,12 +568,13 @@ File ownership change
...
@@ -560,12 +568,13 @@ File ownership change
~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~
If you notice that the ownership of some (maybe all) files have changed and you
If you notice that the ownership of some (maybe all) files have changed and you
need to enter your root password when editing a file, that could be because you
need to enter your root password when editing a file, you might
have pulled changes the remote repository from within a container. While running
have pulled changes to the remote repository from within a container. While running
``git pull`` git changes the owner of the files that you pull to the user that runs
``git pull``, git changes the owner of the files that you pull to the user that runs
that command, and within a container that is the root user, hence git operations
that command. Within a container, that is the root user - so git operations
should be ran outside of the container.
should be ran outside of the container.
To fix this change the owner back to yourself outside of the container by running:
To fix this situation, change the owner back to yourself outside of the container by running:
.. code:: sh
.. code:: sh
...
@@ -574,9 +583,9 @@ To fix this change the owner back to yourself outside of the container by runnin
...
@@ -574,9 +583,9 @@ To fix this change the owner back to yourself outside of the container by runnin
Running LMS commands within a container
Running LMS commands within a container
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Most of the ``paver`` commands require a settings flag, which if omitted defaults to
Most of the ``paver`` commands require a settings flag. If omitted, the flag defaults to
``devstack`` which is the settings flag for vagrant-based devstack instances. Therefor
``devstack``, which is the settings flag for vagrant-based devstack instances.
if you run into issues running those command in a docker container you should append
So if you run into issues running ``paver`` commands in a docker container, you should append
the ``devstack_docker`` flag. For example:
the ``devstack_docker`` flag. For example:
.. code:: sh
.. code:: sh
...
@@ -589,6 +598,8 @@ Resource busy or locked
...
@@ -589,6 +598,8 @@ Resource busy or locked
While running ``make static`` within the ecommerce container you could get an error
While running ``make static`` within the ecommerce container you could get an error
saying:
saying:
.. code:: sh
Error: Error: EBUSY: resource busy or locked, rmdir '/edx/app/ecommerce/ecommerce/ecommerce/static/build/'
Error: Error: EBUSY: resource busy or locked, rmdir '/edx/app/ecommerce/ecommerce/ecommerce/static/build/'
To fix this, remove the directory manually outside of the container and run the command again.
To fix this, remove the directory manually outside of the container and run the command again.
...
@@ -596,10 +607,10 @@ To fix this, remove the directory manually outside of the container and run the
...
@@ -596,10 +607,10 @@ To fix this, remove the directory manually outside of the container and run the
No space left on device
No space left on device
~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
If you see the error "no space left on device" on a Mac, it means Docker has run
If you see the error ``no space left on device`` on a Mac, Docker has run
out of space in its Docker.qcow2 file.
out of space in its Docker.qcow2 file.
Here is an example error while running `make pull`:
Here is an example error while running ``make pull``:
.. code:: sh
.. code:: sh
...
@@ -608,15 +619,15 @@ Here is an example error while running `make pull`:
...
@@ -608,15 +619,15 @@ Here is an example error while running `make pull`:
ERROR: failed to register layer: Error processing tar file(exit status 1): write /edx/app/edxapp/edx-platform/.git/objects/pack/pack-4ff9873be2ca8ab77d4b0b302249676a37b3cd4b.pack: no space left on device
ERROR: failed to register layer: Error processing tar file(exit status 1): write /edx/app/edxapp/edx-platform/.git/objects/pack/pack-4ff9873be2ca8ab77d4b0b302249676a37b3cd4b.pack: no space left on device
make: *** [pull] Error 1
make: *** [pull] Error 1
You can clean up data by running `docker system prune`, but you will first want
You can clean up data by running ``docker system prune``, but you will first want
to run `make dev.up` so it doesn't delete stopped containers.
to run ``make dev.up`` so it doesn't delete stopped containers.
Or, you can run the following commands to clean up dangling images and volumes:
Or, you can run the following commands to clean up dangling images and volumes:
.. code:: sh
.. code:: sh
docker image prune -f
docker image prune -f
docker volume prune -f (Be careful, this will remove your persistent data!)
docker volume prune -f # (Be careful, this will remove your persistent data!)
No such file or directory
No such file or directory
~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~
...
@@ -626,12 +637,12 @@ While provisioning, some have seen the following error:
...
@@ -626,12 +637,12 @@ While provisioning, some have seen the following error:
.. code:: sh
.. code:: sh
...
...
cwd = os.getcwdu()
cwd = os.getcwdu()
OSError: [Errno 2] No such file or directory
OSError: [Errno 2] No such file or directory
make: *** [dev.provision.run] Error 1
make: *** [dev.provision.run] Error 1
Everyone who has seen this has gotten past it, but we don't have a surefire way
This issue can be worked around, but there's no guaranteed method to do so.
to do so. Rebooting and restarting Docker do *not* seem to correct the issue. It
Rebooting and restarting Docker does *not* seem to correct the issue. It
may be an issue that is exacerbated by our use of sync (which typically speeds
may be an issue that is exacerbated by our use of sync (which typically speeds
up the provisioning process on Mac), so you can try the following:
up the provisioning process on Mac), so you can try the following:
...
@@ -654,9 +665,9 @@ While provisioning, some have seen the following error:
...
@@ -654,9 +665,9 @@ While provisioning, some have seen the following error:
