Commit bcad51d8 by David Ormsbee

Partial revert of "Deprecate docker-sync."

This partially reverts commit 9b3b738a.

Further profiling showed that LMS/Studio startup times about tripled
when we dropped Docker Sync in favor of Docker's built in cached volume
mount system (going from 7-8s to anywhere betwen 22-28s). So we're
rolling back the part of that commit that removed Docker Sync and the
recommendations in the README.

However, we still want to keep the minimum version at 17.05 Edge, for
image building purposes. We also want to clarify that we did try using
Docker's built in cached volume support, so that people don't waste
time trying it again until Docker really implements "delegated" mode.
parent 05be0427
...@@ -30,6 +30,8 @@ dev.up: ## Bring up all services with host volumes ...@@ -30,6 +30,8 @@ dev.up: ## Bring up all services with host volumes
dev.sync.daemon.start: ## Start the docker-sycn daemon dev.sync.daemon.start: ## Start the docker-sycn daemon
docker-sync start docker-sync start
dev.sync.provision: | dev.sync.daemon.start dev.provision ## Provision with docker-sync enabled
dev.sync.requirements: ## Install requirements dev.sync.requirements: ## Install requirements
gem install docker-sync gem install docker-sync
......
...@@ -28,14 +28,23 @@ boot2docker) are not supported. ...@@ -28,14 +28,23 @@ boot2docker) are not supported.
`Docker for Windows`_ may work but has not been tested and is *not supported*. `Docker for Windows`_ may work but has not been tested and is *not supported*.
Docker Sync (Deprecated) Docker Sync
~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~
Our use of Docker Sync is now deprecated since Docker for Mac 17.04 introduced Docker for Mac has known filesystem issues that significantly decrease
`performance improvements for volume mounts`_. performance, paticularly for starting edx-platform (e.g. when you want to run a
test). In order to mitigate these issues, we use `Docker Sync`_ to synchronize
file data from the host machine to the containers.
If you are using macOS, please follow the `Docker Sync installation
instructions`_ before provisioning.
The performance improvements provided by `cached consistency mode for volume
mounts`_ introduced in Docker CE Edge 17.04 are still not good enough. It's
possible that the "delegated" consistency mode will be enough to no longer need
docker-sync, but this feature doesn't appear to have been fully implemented yet
(as of Docker 17.06.0-ce-rc2, "delegated" behaves the same as "cached").
Docker for Mac had known filesystem issues that significantly decreased
performance, and we were previously using `Docker Sync`_ to solve this issue.
Getting Started Getting Started
--------------- ---------------
...@@ -70,14 +79,17 @@ a minimum of 2 CPUs and 4GB of memory works well. ...@@ -70,14 +79,17 @@ a minimum of 2 CPUs and 4GB of memory works well.
the services directly via Django admin at the ``/admin/`` path, or login via the services directly via Django admin at the ``/admin/`` path, or login via
single sign-on at ``/login/``. single sign-on at ``/login/``.
Default Provision using docker-sync (recommended for macOS users)
.. code:: sh .. code:: sh
make dev.provision make dev.sync.provision
Default (non-macOS users)
For macOS users, we will no longer be supporting docker-sync. Ensure you .. code:: sh
have a new enough version of Docker for Mac as detailed in `Prerequisites`_.
make dev.provision
3. Start the services. This command will mount the repositories under the 3. Start the services. This command will mount the repositories under the
...@@ -85,13 +97,13 @@ a minimum of 2 CPUs and 4GB of memory works well. ...@@ -85,13 +97,13 @@ a minimum of 2 CPUs and 4GB of memory works well.
*Note: it may take up to 60 seconds for the LMS to start* *Note: it may take up to 60 seconds for the LMS to start*
Start using docker-sync (Deprecated for macOS users. See `Prerequisites`_.) Start using docker-sync (recommended for macOS users)
.. code:: sh .. code:: sh
make dev.sync.up make dev.sync.up
Default Default (non-macOS users)
.. code:: sh .. code:: sh
...@@ -361,9 +373,10 @@ Or, you can run the following commands to clean up dangling images and volumes: ...@@ -361,9 +373,10 @@ Or, you can run the following commands to clean up dangling images and volumes:
.. _Docker for Mac: https://docs.docker.com/docker-for-mac/ .. _Docker for Mac: https://docs.docker.com/docker-for-mac/
.. _Docker for Windows: https://docs.docker.com/docker-for-windows/ .. _Docker for Windows: https://docs.docker.com/docker-for-windows/
.. _Docker Sync: https://github.com/EugenMayer/docker-sync/wiki .. _Docker Sync: https://github.com/EugenMayer/docker-sync/wiki
.. _Docker Sync installation instructions: https://github.com/EugenMayer/docker-sync/wiki/1.-Installation
.. _cached consistency mode for volume mounts: https://docs.docker.com/docker-for-mac/osxfs-caching/
.. _configuring Docker for Mac: https://docs.docker.com/docker-for-mac/#/advanced .. _configuring Docker for Mac: https://docs.docker.com/docker-for-mac/#/advanced
.. _feature added in Docker 17.05: https://github.com/edx/configuration/pull/3864 .. _feature added in Docker 17.05: https://github.com/edx/configuration/pull/3864
.. _performance improvements for volume mounts: https://docs.docker.com/docker-for-mac/osxfs-caching/
.. _Pycharm Integration documentation: docs/pycharm_integration.rst .. _Pycharm Integration documentation: docs/pycharm_integration.rst
.. |Build Status| image:: https://travis-ci.org/edx/devstack.svg?branch=master .. |Build Status| image:: https://travis-ci.org/edx/devstack.svg?branch=master
:target: https://travis-ci.org/edx/devstack :target: https://travis-ci.org/edx/devstack
...@@ -35,7 +35,7 @@ use the following options: ...@@ -35,7 +35,7 @@ use the following options:
- Configuration files(s) - Configuration files(s)
- Docker Sync (Deprecated for macOS users. See the `README`_.) - Docker Sync (Mac)
- ``/LOCAL/PATH/TO/devstack/docker-compose.yml`` (e.g.~/edx/devstack/docker-compose.yml) - ``/LOCAL/PATH/TO/devstack/docker-compose.yml`` (e.g.~/edx/devstack/docker-compose.yml)
- ``/LOCAL/PATH/TO/devstack/docker-compose-sync.yml`` - ``/LOCAL/PATH/TO/devstack/docker-compose-sync.yml``
...@@ -67,15 +67,17 @@ use the following options: ...@@ -67,15 +67,17 @@ use the following options:
- PyCharm helpers path: Keep the default. - PyCharm helpers path: Keep the default.
**Important** Docker sync is deprecated. See the `README`_. If you are still
running with Docker Sync you will also need to first run ``docker-sync start`` **Important** If you are running with Docker Sync you will also need to first
to run sync in the background before running any servers. run ``docker-sync start`` to run sync in the background before running any servers.
**Note**: For lms and studio (edx-platform), it will take a long time to **Note**: For lms and studio (edx-platform), it will take a long time to
update skeletons (10 or more minutes). If you want to try a different update skeletons (10 or more minutes). If you want to try a different
set of configuration (compose) files, we recommend you create a new one set of configuration (compose) files, we recommend you create a new one
so you can easily switch back to old without this delay. so you can easily switch back to old without this delay.
**Warning**: When you change configuration files, the service dropdown gets **Warning**: When you change configuration files, the service dropdown gets
reset. Remember to restore to the IDA you wish to test. reset. Remember to restore to the IDA you wish to test.
...@@ -160,8 +162,7 @@ Currently not supported for PyCharm Development ...@@ -160,8 +162,7 @@ Currently not supported for PyCharm Development
- Debugging for JavaScript - Debugging for JavaScript
- Cython for fasterdebug - Cython for fasterdebug
.. _Django Server Run/Debug Configuration: https://www.jetbrains.com/help/pycharm/2017.1/run-debug-configuration-django-server.html
.. _PyCharm: https://www.jetbrains.com/pycharm/ .. _PyCharm: https://www.jetbrains.com/pycharm/
.. _PyCharm IDE setup: https://openedx.atlassian.net/wiki/display/ENG/PyCharm .. _PyCharm IDE setup: https://openedx.atlassian.net/wiki/display/ENG/PyCharm
.. _README: ../README.rst
.. _vendor documentation: https://www.jetbrains.com/help/pycharm/2017.1/configuring-remote-interpreters-via-docker-compose.html .. _vendor documentation: https://www.jetbrains.com/help/pycharm/2017.1/configuring-remote-interpreters-via-docker-compose.html
.. _Django Server Run/Debug Configuration: https://www.jetbrains.com/help/pycharm/2017.1/run-debug-configuration-django-server.html
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment