Skip to content

D
devstack
Commit c7a60973 by Clinton Blackburn
Options

Integrated edxapp 

The LMS and Studio (CMS) are now part of devstack!

ECOM-6560
parent d0dadc4e
Showing with 260 additions and 78 deletions
Makefile
...@@ -18,15 +18,12 @@ devstack.provision: ## Provision all services ...@@ -18,15 +18,12 @@ devstack.provision: ## Provision all services
devstack.reset: ## Remove all service containers devstack.reset: ## Remove all service containers
docker-compose down docker-compose down
devstack.start: loopback ## Start all services devstack.start: ## Start all services
docker-compose up docker-compose up
devstack.stop: ## Stop all services devstack.stop: ## Stop all services
docker-compose stop docker-compose stop
loopback: ## Create loopback alias
./loopback.sh
requirements: ## Install requirements requirements: ## Install requirements
pip install -r requirements.txt pip install -r requirements.txt
......
README.md
...@@ -6,7 +6,9 @@ This project is meant to replace the traditional Vagrant VM "devstack" with a mu ...@@ -6,7 +6,9 @@ This project is meant to replace the traditional Vagrant VM "devstack" with a mu
## Getting Started ## Getting Started
All of the services can be run by following the steps below. All of the services can be run by following the steps below. Note that since we are running many containers, you should
configure Docker with a sufficient amount of resources. Our testing found that [configuring Docker for Mac](https://docs.docker.com/docker-for-mac/#/advanced)
with 2 CPUs and 4GB of memory works well.
1. The Docker Compose file mounts a host volume for each service's executing code. The host directory is expected to be 1. The Docker Compose file mounts a host volume for each service's executing code. The host directory is expected to be
a sibling of this directory. For example, if this repo is cloned to `~/workspace/devstack`, host volumes will be a sibling of this directory. For example, if this repo is cloned to `~/workspace/devstack`, host volumes will be
...@@ -38,28 +40,3 @@ After the services have started, if you need shell access to one of the services ...@@ -38,28 +40,3 @@ After the services have started, if you need shell access to one of the services
``` ```
$ make devstack.open.discovery $ make devstack.open.discovery
``` ```
## Loopback Alias
Containers making requests to the LMS and Studio must communicate with ports exposed on the host system by the Vagrant VM. (This assumes that the LMS and Studio are still running in the traditional Vagrant-based devstack.)
This is fine on Linux, but doesn't work out of the box on OS X. Attempting to access localhost on a container will result
in talking to the Docker for Mac HyperKit VM, not the host machine.
While it's true that you can get this to work by accessing your Mac's external IP from your containers, this isn't ideal because
it won't work if you have no network access on your host. Your external IP also changes as you switch networks, meaning you'd have
to change the IP accessed by your containers every time you changed networks.
A better solution, borrowed from the [Docker forums](https://forums.docker.com/t/access-host-not-vm-from-inside-container/11747/10),
is to give your host a fixed address by creating a [loopback](http://askubuntu.com/questions/247625/what-is-the-loopback-device-and-how-do-i-use-it)
alias. This is done for you by the `devstack.start` target.
The result is a fixed IP which your containers can use to access ports on your host machine. Note that the underlying script uses `sudo`;
adding IP addresses requires root access. Also note that the alias will not survive a host reboot, which is why the `devstack.start` target
always attempts to set up the loopback for you.
Part of the loopback alias setup includes adding a line to the `/etc/hosts` file on your machine. If you want to stop using devstack, you can clean this up by opening your `/etc/hosts` file and removing this line:
```
10.254.254.254 docker.host
```
docker-compose.yml
...@@ -29,6 +29,14 @@ services: ...@@ -29,6 +29,14 @@ services:
# ports: # ports:
# - "11211:11211" # - "11211:11211"
mongo:
container_name: edx.devstack.mongo
image: mongo:2.6.5
# ports:
# - "27017:27017"
volumes:
- mongo_data:/data/db
mysql: mysql:
command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci
container_name: edx.devstack.mysql container_name: edx.devstack.mysql
...@@ -67,9 +75,6 @@ services: ...@@ -67,9 +75,6 @@ services:
environment: environment:
TEST_ELASTICSEARCH_URL: "http://edx.devstack.elasticsearch:9200" TEST_ELASTICSEARCH_URL: "http://edx.devstack.elasticsearch:9200"
ENABLE_DJANGO_TOOLBAR: 1 ENABLE_DJANGO_TOOLBAR: 1
extra_hosts:
# For server-to-server calls.
- "docker.host:10.254.254.254"
image: edxops/discovery:devstack image: edxops/discovery:devstack
ports: ports:
- "18381:18381" - "18381:18381"
...@@ -91,6 +96,20 @@ services: ...@@ -91,6 +96,20 @@ services:
volumes: volumes:
- ../ecommerce:/edx/app/ecommerce/ecommerce - ../ecommerce:/edx/app/ecommerce/ecommerce
edxapp:
command: /edx/app/edxapp/devstack.sh start
container_name: edx.devstack.edxapp
depends_on:
- mysql
- memcached
- mongo
image: edxops/edxapp:devstack
ports:
- "18000:18000"
- "18010:18010"
volumes:
- ../edx-platform:/edx/app/edxapp/edx-platform
programs: programs:
command: /edx/app/programs/devstack.sh start command: /edx/app/programs/devstack.sh start
container_name: edx.devstack.programs container_name: edx.devstack.programs
...@@ -107,4 +126,5 @@ services: ...@@ -107,4 +126,5 @@ services:
volumes: volumes:
elasticsearch_data: elasticsearch_data:
mongo_data:
mysql_data: mysql_data:
dump-db.sh 0 → 100644
#!/usr/bin/env bash
# Dump the specified database to a file of the same name.
#
# Example:
# $ dump-db edxapp
#
# This will dump the edxapp database to a file named exapp.sql.
if [ -z "$1" ]
then
echo "You must supply a database name!"
exit 1
fi
echo "Dumping the $1 database..."
docker exec -i edx.devstack.mysql mysqldump --skip-add-drop-table -B $1 > $1.sql
echo "Finished dumping the $1 database!"
ecommerce.sql 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
edxapp.sql 0 → 100644
This source diff could not be displayed because it is too large. You can view the blob instead.
edxapp_csmh.sql 0 → 100644
-- MySQL dump 10.13 Distrib 5.6.32, for Linux (x86_64)
--
-- Host: localhost Database: edxapp_csmh
-- ------------------------------------------------------
-- Server version 5.6.32
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Current Database: `edxapp_csmh`
--
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `edxapp_csmh` /*!40100 DEFAULT CHARACTER SET utf8 */;
USE `edxapp_csmh`;
--
-- Table structure for table `coursewarehistoryextended_studentmodulehistoryextended`
--
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `coursewarehistoryextended_studentmodulehistoryextended` (
`version` varchar(255) DEFAULT NULL,
`created` datetime(6) NOT NULL,
`state` longtext,
`grade` double DEFAULT NULL,
`max_grade` double DEFAULT NULL,
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`student_module_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `coursewarehistoryextended_studentmodulehistoryextended_2af72f10` (`version`),
KEY `coursewarehistoryextended_studentmodulehistoryextended_e2fa5388` (`created`),
KEY `coursewarehistoryextended_student_module_id_61b23a7a1dd27fe4_idx` (`student_module_id`)
) ENGINE=InnoDB AUTO_INCREMENT=10000 DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `coursewarehistoryextended_studentmodulehistoryextended`
--
LOCK TABLES `coursewarehistoryextended_studentmodulehistoryextended` WRITE;
/*!40000 ALTER TABLE `coursewarehistoryextended_studentmodulehistoryextended` DISABLE KEYS */;
/*!40000 ALTER TABLE `coursewarehistoryextended_studentmodulehistoryextended` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `django_migrations`
--
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `django_migrations` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`app` varchar(255) NOT NULL,
`name` varchar(255) NOT NULL,
`applied` datetime(6) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=212 DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `django_migrations`
--
LOCK TABLES `django_migrations` WRITE;
/*!40000 ALTER TABLE `django_migrations` DISABLE KEYS */;
INSERT INTO `django_migrations` VALUES (1,'contenttypes','0001_initial','2016-12-17 21:30:28.505194'),(2,'auth','0001_initial','2016-12-17 21:30:28.623739'),(3,'admin','0001_initial','2016-12-17 21:30:28.718110'),(4,'sites','0001_initial','2016-12-17 21:30:28.757525'),(5,'contenttypes','0002_remove_content_type_name','2016-12-17 21:30:28.927188'),(6,'api_admin','0001_initial','2016-12-17 21:30:29.051582'),(7,'api_admin','0002_auto_20160325_1604','2016-12-17 21:30:29.170420'),(8,'api_admin','0003_auto_20160404_1618','2016-12-17 21:30:29.515374'),(9,'api_admin','0004_auto_20160412_1506','2016-12-17 21:30:29.792546'),(10,'api_admin','0005_auto_20160414_1232','2016-12-17 21:30:29.895633'),(11,'api_admin','0006_catalog','2016-12-17 21:30:29.935659'),(12,'assessment','0001_initial','2016-12-17 21:30:31.459234'),(13,'assessment','0002_staffworkflow','2016-12-17 21:30:31.520779'),(14,'auth','0002_alter_permission_name_max_length','2016-12-17 21:30:31.616835'),(15,'auth','0003_alter_user_email_max_length','2016-12-17 21:30:31.733266'),(16,'auth','0004_alter_user_username_opts','2016-12-17 21:30:31.839695'),(17,'auth','0005_alter_user_last_login_null','2016-12-17 21:30:31.946886'),(18,'auth','0006_require_contenttypes_0002','2016-12-17 21:30:31.973266'),(19,'instructor_task','0001_initial','2016-12-17 21:30:32.070121'),(20,'certificates','0001_initial','2016-12-17 21:30:32.890502'),(21,'certificates','0002_data__certificatehtmlviewconfiguration_data','2016-12-17 21:30:32.955557'),(22,'certificates','0003_data__default_modes','2016-12-17 21:30:32.998844'),(23,'certificates','0004_certificategenerationhistory','2016-12-17 21:30:33.123344'),(24,'certificates','0005_auto_20151208_0801','2016-12-17 21:30:33.275835'),(25,'certificates','0006_certificatetemplateasset_asset_slug','2016-12-17 21:30:33.310157'),(26,'certificates','0007_certificateinvalidation','2016-12-17 21:30:33.444878'),(27,'badges','0001_initial','2016-12-17 21:30:33.685873'),(28,'badges','0002_data__migrate_assertions','2016-12-17 21:30:33.726406'),(29,'badges','0003_schema__add_event_configuration','2016-12-17 21:30:34.013221'),(30,'bookmarks','0001_initial','2016-12-17 21:30:34.475781'),(31,'branding','0001_initial','2016-12-17 21:30:34.793962'),(32,'course_groups','0001_initial','2016-12-17 21:30:36.088247'),(33,'bulk_email','0001_initial','2016-12-17 21:30:36.718426'),(34,'bulk_email','0002_data__load_course_email_template','2016-12-17 21:30:36.756662'),(35,'bulk_email','0003_config_model_feature_flag','2016-12-17 21:30:36.971127'),(36,'bulk_email','0004_add_email_targets','2016-12-17 21:30:37.662273'),(37,'bulk_email','0005_move_target_data','2016-12-17 21:30:37.706454'),(38,'catalog','0001_initial','2016-12-17 21:30:37.954117'),(39,'certificates','0008_schema__remove_badges','2016-12-17 21:30:38.443508'),(40,'commerce','0001_data__add_ecommerce_service_user','2016-12-17 21:30:38.499445'),(41,'commerce','0002_commerceconfiguration','2016-12-17 21:30:38.739545'),(42,'commerce','0003_auto_20160329_0709','2016-12-17 21:30:38.999318'),(43,'commerce','0004_auto_20160531_0950','2016-12-17 21:30:39.496923'),(44,'contentserver','0001_initial','2016-12-17 21:30:39.776248'),(45,'contentserver','0002_cdnuseragentsconfig','2016-12-17 21:30:40.061875'),(46,'cors_csrf','0001_initial','2016-12-17 21:30:40.308955'),(47,'course_action_state','0001_initial','2016-12-17 21:30:40.818977'),(48,'course_modes','0001_initial','2016-12-17 21:30:40.908209'),(49,'course_modes','0002_coursemode_expiration_datetime_is_explicit','2016-12-17 21:30:40.945208'),(50,'course_modes','0003_auto_20151113_1443','2016-12-17 21:30:41.000993'),(51,'course_modes','0004_auto_20151113_1457','2016-12-17 21:30:41.269098'),(52,'course_modes','0005_auto_20151217_0958','2016-12-17 21:30:41.326778'),(53,'course_modes','0006_auto_20160208_1407','2016-12-17 21:30:41.635635'),(54,'course_modes','0007_coursemode_bulk_sku','2016-12-17 21:30:41.674830'),(55,'course_overviews','0001_initial','2016-12-17 21:30:41.755403'),(56,'course_overviews','0002_add_course_catalog_fields','2016-12-17 21:30:41.939908'),(57,'course_overviews','0003_courseoverviewgeneratedhistory','2016-12-17 21:30:44.826858'),(58,'course_overviews','0004_courseoverview_org','2016-12-17 21:30:44.877007'),(59,'course_overviews','0005_delete_courseoverviewgeneratedhistory','2016-12-17 21:30:44.928707'),(60,'course_overviews','0006_courseoverviewimageset','2016-12-17 21:30:45.025353'),(61,'course_overviews','0007_courseoverviewimageconfig','2016-12-17 21:30:45.242695'),(62,'course_overviews','0008_remove_courseoverview_facebook_url','2016-12-17 21:30:45.257312'),(63,'course_overviews','0009_readd_facebook_url','2016-12-17 21:30:45.318911'),(64,'course_overviews','0010_auto_20160329_2317','2016-12-17 21:30:45.411705'),(65,'course_structures','0001_initial','2016-12-17 21:30:45.454214'),(66,'coursetalk','0001_initial','2016-12-17 21:30:45.691992'),(67,'coursetalk','0002_auto_20160325_0631','2016-12-17 21:30:45.884575'),(68,'courseware','0001_initial','2016-12-17 21:30:48.778894'),(69,'coursewarehistoryextended','0001_initial','2016-12-17 21:30:49.423427'),(70,'coursewarehistoryextended','0002_force_studentmodule_index','2016-12-17 21:30:49.784372'),(71,'credentials','0001_initial','2016-12-17 21:30:50.086610'),(72,'credentials','0002_auto_20160325_0631','2016-12-17 21:30:50.395102'),(73,'credit','0001_initial','2016-12-17 21:30:52.914182'),(74,'credit','0002_creditconfig','2016-12-17 21:30:53.351874'),(75,'credit','0003_auto_20160511_2227','2016-12-17 21:30:53.779850'),(76,'dark_lang','0001_initial','2016-12-17 21:30:54.257157'),(77,'dark_lang','0002_data__enable_on_install','2016-12-17 21:30:54.299218'),(78,'django_comment_common','0001_initial','2016-12-17 21:30:55.103249'),(79,'django_comment_common','0002_forumsconfig','2016-12-17 21:30:55.471448'),(80,'django_comment_common','0003_enable_forums','2016-12-17 21:30:55.509253'),(81,'django_comment_common','0004_auto_20161117_1209','2016-12-17 21:30:55.860866'),(82,'django_notify','0001_initial','2016-12-17 21:30:57.515091'),(83,'django_openid_auth','0001_initial','2016-12-17 21:30:57.978552'),(84,'oauth2','0001_initial','2016-12-17 21:31:00.319373'),(85,'edx_oauth2_provider','0001_initial','2016-12-17 21:31:00.949006'),(86,'edx_proctoring','0001_initial','2016-12-17 21:31:12.175564'),(87,'edx_proctoring','0002_proctoredexamstudentattempt_is_status_acknowledged','2016-12-17 21:31:13.157296'),(88,'edx_proctoring','0003_auto_20160101_0525','2016-12-17 21:31:15.129604'),(89,'edx_proctoring','0004_auto_20160201_0523','2016-12-17 21:31:16.130444'),(90,'edx_proctoring','0005_proctoredexam_hide_after_due','2016-12-17 21:31:17.113544'),(91,'edxval','0001_initial','2016-12-17 21:31:20.331618'),(92,'edxval','0002_data__default_profiles','2016-12-17 21:31:20.372665'),(93,'edxval','0003_coursevideo_is_hidden','2016-12-17 21:31:20.442999'),(94,'email_marketing','0001_initial','2016-12-17 21:31:20.805165'),(95,'email_marketing','0002_auto_20160623_1656','2016-12-17 21:31:24.227033'),(96,'email_marketing','0003_auto_20160715_1145','2016-12-17 21:31:26.533226'),(97,'embargo','0001_initial','2016-12-17 21:31:27.957969'),(98,'embargo','0002_data__add_countries','2016-12-17 21:31:28.263745'),(99,'enterprise','0001_initial','2016-12-17 21:31:28.957784'),(100,'enterprise','0002_enterprisecustomerbrandingconfiguration','2016-12-17 21:31:29.033374'),(101,'enterprise','0003_auto_20161104_0937','2016-12-17 21:31:31.101482'),(102,'enterprise','0004_auto_20161114_0434','2016-12-17 21:31:32.979171'),(103,'enterprise','0005_pendingenterprisecustomeruser','2016-12-17 21:31:33.939715'),(104,'enterprise','0006_auto_20161121_0241','2016-12-17 21:31:34.863432'),(105,'enterprise','0007_auto_20161109_1511','2016-12-17 21:31:36.779853'),(106,'enterprise','0008_auto_20161124_2355','2016-12-17 21:31:39.891187'),(107,'enterprise','0009_auto_20161130_1651','2016-12-17 21:31:46.026321'),(108,'external_auth','0001_initial','2016-12-17 21:31:48.247622'),(109,'grades','0001_initial','2016-12-17 21:31:48.439748'),(110,'grades','0002_rename_last_edited_field','2016-12-17 21:31:48.500681'),(111,'grades','0003_coursepersistentgradesflag_persistentgradesenabledflag','2016-12-17 21:31:50.765829'),(112,'grades','0004_visibleblocks_course_id','2016-12-17 21:31:50.872900'),(113,'grades','0005_multiple_course_flags','2016-12-17 21:31:52.007864'),(114,'grades','0006_persistent_course_grades','2016-12-17 21:31:52.096999'),(115,'grades','0007_add_passed_timestamp_column','2016-12-17 21:31:52.184338'),(116,'grades','0008_persistentsubsectiongrade_first_attempted','2016-12-17 21:31:52.239090'),(117,'lms_xblock','0001_initial','2016-12-17 21:31:55.865882'),(118,'microsite_configuration','0001_initial','2016-12-17 21:32:00.352651'),(119,'microsite_configuration','0002_auto_20160202_0228','2016-12-17 21:32:01.685077'),(120,'milestones','0001_initial','2016-12-17 21:32:02.357572'),(121,'milestones','0002_data__seed_relationship_types','2016-12-17 21:32:02.408134'),(122,'milestones','0003_coursecontentmilestone_requirements','2016-12-17 21:32:02.491874'),(123,'milestones','0004_auto_20151221_1445','2016-12-17 21:32:02.820002'),(124,'mobile_api','0001_initial','2016-12-17 21:32:03.575717'),(125,'mobile_api','0002_auto_20160406_0904','2016-12-17 21:32:03.684343'),(126,'notes','0001_initial','2016-12-17 21:32:04.599778'),(127,'oauth2','0002_auto_20160404_0813','2016-12-17 21:32:07.890890'),(128,'oauth2','0003_client_logout_uri','2016-12-17 21:32:08.977788'),(129,'oauth2','0004_add_index_on_grant_expires','2016-12-17 21:32:10.045008'),(130,'oauth2_provider','0001_initial','2016-12-17 21:32:14.596267'),(131,'oauth2_provider','0002_08_updates','2016-12-17 21:32:18.200314'),(132,'oauth_dispatch','0001_initial','2016-12-17 21:32:19.485230'),(133,'oauth_provider','0001_initial','2016-12-17 21:32:22.141621'),(134,'organizations','0001_initial','2016-12-17 21:32:22.298715'),(135,'problem_builder','0001_initial','2016-12-17 21:32:22.399239'),(136,'problem_builder','0002_auto_20160121_1525','2016-12-17 21:32:24.999678'),(137,'problem_builder','0003_auto_20161124_0755','2016-12-17 21:32:25.109541'),(138,'programs','0001_initial','2016-12-17 21:32:26.490436'),(139,'programs','0002_programsapiconfig_cache_ttl','2016-12-17 21:32:27.857435'),(140,'programs','0003_auto_20151120_1613','2016-12-17 21:32:34.236406'),(141,'programs','0004_programsapiconfig_enable_certification','2016-12-17 21:32:34.812108'),(142,'programs','0005_programsapiconfig_max_retries','2016-12-17 21:32:35.393887'),(143,'programs','0006_programsapiconfig_xseries_ad_enabled','2016-12-17 21:32:36.077073'),(144,'programs','0007_programsapiconfig_program_listing_enabled','2016-12-17 21:32:36.864433'),(145,'programs','0008_programsapiconfig_program_details_enabled','2016-12-17 21:32:37.620230'),(146,'programs','0009_programsapiconfig_marketing_path','2016-12-17 21:32:38.515400'),(147,'redirects','0001_initial','2016-12-17 21:32:39.643958'),(148,'rss_proxy','0001_initial','2016-12-17 21:32:39.704874'),(149,'self_paced','0001_initial','2016-12-17 21:32:40.875492'),(150,'sessions','0001_initial','2016-12-17 21:32:40.945170'),(151,'student','0001_initial','2016-12-17 21:33:21.977963'),(152,'shoppingcart','0001_initial','2016-12-17 21:33:50.995529'),(153,'shoppingcart','0002_auto_20151208_1034','2016-12-17 21:33:55.329721'),(154,'shoppingcart','0003_auto_20151217_0958','2016-12-17 21:33:59.912392'),(155,'site_configuration','0001_initial','2016-12-17 21:34:04.677398'),(156,'site_configuration','0002_auto_20160720_0231','2016-12-17 21:34:08.729573'),(157,'default','0001_initial','2016-12-17 21:34:16.891351'),(158,'default','0002_add_related_name','2016-12-17 21:34:17.664617'),(159,'default','0003_alter_email_max_length','2016-12-17 21:34:17.732051'),(160,'default','0004_auto_20160423_0400','2016-12-17 21:34:18.476339'),(161,'social_auth','0005_auto_20160727_2333','2016-12-17 21:34:18.549133'),(162,'splash','0001_initial','2016-12-17 21:34:19.396195'),(163,'static_replace','0001_initial','2016-12-17 21:34:20.198862'),(164,'static_replace','0002_assetexcludedextensionsconfig','2016-12-17 21:34:21.005088'),(165,'status','0001_initial','2016-12-17 21:34:22.770434'),(166,'student','0002_auto_20151208_1034','2016-12-17 21:34:24.545829'),(167,'student','0003_auto_20160516_0938','2016-12-17 21:34:27.160603'),(168,'student','0004_auto_20160531_1422','2016-12-17 21:34:28.693959'),(169,'student','0005_auto_20160531_1653','2016-12-17 21:34:30.308549'),(170,'student','0006_logoutviewconfiguration','2016-12-17 21:34:31.957635'),(171,'student','0007_registrationcookieconfiguration','2016-12-17 21:34:33.555787'),(172,'student','0008_auto_20161117_1209','2016-12-17 21:34:35.170434'),(173,'submissions','0001_initial','2016-12-17 21:34:35.740972'),(174,'submissions','0002_auto_20151119_0913','2016-12-17 21:34:36.022354'),(175,'submissions','0003_submission_status','2016-12-17 21:34:36.139134'),(176,'survey','0001_initial','2016-12-17 21:34:38.285443'),(177,'teams','0001_initial','2016-12-17 21:34:47.967805'),(178,'theming','0001_initial','2016-12-17 21:34:50.933332'),(179,'third_party_auth','0001_initial','2016-12-17 21:35:03.447321'),(180,'third_party_auth','0002_schema__provider_icon_image','2016-12-17 21:35:12.758783'),(181,'third_party_auth','0003_samlproviderconfig_debug_mode','2016-12-17 21:35:14.707968'),(182,'third_party_auth','0004_add_visible_field','2016-12-17 21:35:27.189553'),(183,'third_party_auth','0005_add_site_field','2016-12-17 21:35:37.943161'),(184,'track','0001_initial','2016-12-17 21:35:38.014287'),(185,'user_api','0001_initial','2016-12-17 21:35:53.143605'),(186,'util','0001_initial','2016-12-17 21:35:55.990480'),(187,'util','0002_data__default_rate_limit_config','2016-12-17 21:35:59.057428'),(188,'verified_track_content','0001_initial','2016-12-17 21:35:59.137003'),(189,'verified_track_content','0002_verifiedtrackcohortedcourse_verified_cohort_name','2016-12-17 21:35:59.232551'),(190,'verify_student','0001_initial','2016-12-17 21:36:13.458907'),(191,'verify_student','0002_auto_20151124_1024','2016-12-17 21:36:15.306038'),(192,'verify_student','0003_auto_20151113_1443','2016-12-17 21:36:17.179714'),(193,'wiki','0001_initial','2016-12-17 21:37:44.962002'),(194,'wiki','0002_remove_article_subscription','2016-12-17 21:37:45.100781'),(195,'wiki','0003_ip_address_conv','2016-12-17 21:38:02.005894'),(196,'wiki','0004_increase_slug_size','2016-12-17 21:38:07.575914'),(197,'workflow','0001_initial','2016-12-17 21:38:08.029907'),(198,'xblock_django','0001_initial','2016-12-17 21:38:14.509477'),(199,'xblock_django','0002_auto_20160204_0809','2016-12-17 21:38:19.652694'),(200,'xblock_django','0003_add_new_config_models','2016-12-17 21:38:38.326768'),(201,'xblock_django','0004_delete_xblock_disable_config','2016-12-17 21:38:42.344189'),(202,'social_auth','0001_initial','2016-12-17 21:38:42.365949'),(203,'social_auth','0003_alter_email_max_length','2016-12-17 21:38:42.382541'),(204,'social_auth','0002_add_related_name','2016-12-17 21:38:42.398242'),(205,'social_auth','0004_auto_20160423_0400','2016-12-17 21:38:42.415512'),(206,'contentstore','0001_initial','2016-12-17 21:39:54.291871'),(207,'course_creators','0001_initial','2016-12-17 21:39:54.385436'),(208,'tagging','0001_initial','2016-12-17 21:39:54.472126'),(209,'user_tasks','0001_initial','2016-12-17 21:39:55.400065'),(210,'user_tasks','0002_artifact_file_storage','2016-12-17 21:39:55.659426'),(211,'xblock_config','0001_initial','2016-12-17 21:39:55.979871');
/*!40000 ALTER TABLE `django_migrations` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
-- Dump completed on 2016-12-18 5:19:41
load-db.sh 0 → 100755
#!/usr/bin/env bash
# Load the specified database from a file of the same name.
#
# Example:
# $ load-db edxapp
#
# This will load the edxapp database from a file named exapp.sql.
if [ -z "$1" ]
then
echo "You must supply a database name!"
exit 1
fi
echo "Loading the $1 database..."
docker exec -i edx.devstack.mysql mysql -uroot $1 < $1.sql
echo "Finished loading the $1 database!"
loopback.sh deleted 100755 → 0
# It's not necessary to do this on Linux, so check that we're on OS X before continuing.
if [ "$(uname)" == "Darwin" ]; then
# This loopback alias allows access to OS X hosts from inside a container. 10.254.254.254
# is a private IP likely to be unused. As is, the alias will not survive a reboot.
# Borrowed from https://forums.docker.com/t/access-host-not-vm-from-inside-container/11747/10.
sudo ifconfig lo0 alias 10.254.254.254
# This will check for a "docker.host" entry in your hosts file. It's created if it doesn't
# exist, so you can use URLs like "http://docker.host:8000" in your service configuration.
if ! grep -q "docker.host" /etc/hosts; then
# Using tee to write to /etc/hosts because using >> to append isn't allowed, even as root.
# See http://stackoverflow.com/a/550808.
echo "10.254.254.254 docker.host" | sudo tee -a /etc/hosts > /dev/null
fi
elif [ "$(expr substr $(uname -s) 1 5)" == "Linux" ]; then
if ! grep -q "docker.host" /etc/hosts; then
echo "127.0.0.1 docker.host" | sudo tee -a /etc/hosts > /dev/null
fi
fi
mongo-provision.js 0 → 100644
conn = new Mongo();
users = [
{
'user': 'admin',
'pwd': 'password',
'roles': ['root'],
'database': 'admin'
},
{
'user': 'cs_comments_service',
'pwd': 'password',
'roles': ['readWrite'],
'database': 'cs_comments_service'
},
{
'user': 'edxapp',
'pwd': 'password',
'roles': ['readWrite'],
'database': 'edxapp'
}
];
for (var i = 0; i < users.length; i++) {
var user = users[i];
var username = user.user;
var db = conn.getDB(user.database);
delete user.database;
if (db.getUser(username) == null) {
db.createUser(user);
} else {
delete user.user;
db.updateUser(username, user);
}
}
provision.sh
#!/usr/bin/env bash #!/usr/bin/env bash
# This script will provision all of the services. Each service will be setup in the following manner:
#
# 1. Migrations run,
# 2. Tenants—as in multi-tenancy—setup,
# 3. Service users and OAuth clients setup in LMS,
# 4. Static assets compiled/collected.
RED='\033[0;31m' RED='\033[0;31m'
GREEN='\033[0;32m' GREEN='\033[0;32m'
YELLOW='\033[0;33m' YELLOW='\033[0;33m'
NC='\033[0m' # No Color NC='\033[0m' # No Color
# Bring the database online. # Bring the databases online.
docker-compose up -d mysql docker-compose up -d mysql mongo
n=0 # Ensure the MySQL server is online and usable
db_limit=5 echo "Waiting for MySQL"
db_sleep_time=2 until docker exec -i edx.devstack.mysql mysql -uroot -se "SELECT EXISTS(SELECT 1 FROM mysql.user WHERE user = 'root')" &> /dev/null
until [ $n -ge $db_limit ]
do do
docker exec -i edx.devstack.mysql mysql -uroot < provision.sql && echo -e "${GREEN}Service databases and users have been created.${NC}" && break printf "."
n=$[$n+1] sleep 1
echo -e "${YELLOW}Waiting ${db_sleep_time} seconds for MySQL to come online...${NC}"
sleep 2
done done
if [ $n -ge $db_limit ] # In the event of a fresh MySQL container, wait a few seconds for the server to restart
then # This can be removed once https://github.com/docker-library/mysql/issues/245 is resolved.
echo -e "${RED}Failed to create service databases and users!${NC}" sleep 20
exit 1
fi echo -e "MySQL ready"
echo -e "${GREEN}Creating databases and users...${NC}"
docker exec -i edx.devstack.mysql mysql -uroot mysql < provision.sql
docker exec -i edx.devstack.mongo mongo < mongo-provision.js
# Bring the rest of the services online # Bring the rest of the services online
docker-compose up -d docker-compose up -d
# Run migrations, and create superusers that can access the services without the need for single sign-on. # Load database dumps for the largest databases to save time
services=('credentials' 'discovery' 'ecommerce' 'programs') ./load-db.sh ecommerce
./load-db.sh edxapp
./load-db.sh edxapp_csmh
# Run edxapp migrations first since they are needed for the service users and OAuth clients
docker exec -t edx.devstack.edxapp bash -c 'source /edx/app/edxapp/edxapp_env && cd /edx/app/edxapp/edx-platform && paver update_db --settings devstack'
# Create a superuser for edxapp
docker exec -t edx.devstack.edxapp bash -c 'source /edx/app/edxapp/edxapp_env && python /edx/app/edxapp/edx-platform/manage.py lms --settings=devstack manage_user edx edx@example.com --superuser --staff'
docker exec -t edx.devstack.edxapp bash -c 'source /edx/app/edxapp/edxapp_env && echo "from django.contrib.auth import get_user_model; User = get_user_model(); user = User.objects.get(username=\"edx\"); user.set_password(\"edx\"); user.save()" | python /edx/app/edxapp/edx-platform/manage.py lms shell --settings=devstack' &
# We must fake an associative array for Bash 3 users
services=('credentials:18150' 'discovery:18381' 'ecommerce:18130' 'programs:18140')
for service in "${services[@]}" for service in "${services[@]}"
do do
echo -e "${GREEN}Running migrations for ${service}...${NC}" name=${service%%:*}
docker exec -t edx.devstack.${service} bash -c 'source /edx/app/$1/$1_env && cd /edx/app/$1/$1/ && make migrate' -- "$service" port=${service#*:}
echo -e "${GREEN}Creating super-user for ${service}...${NC}"
docker exec -t edx.devstack.${service} bash -c 'source /edx/app/$1/$1_env && echo "from django.contrib.auth import get_user_model; User = get_user_model(); User.objects.create_superuser(\"edx\", \"edx@example.com\", \"edx\") if not User.objects.filter(username=\"edx\").exists() else None" | python /edx/app/$1/$1/manage.py shell' -- "$service" echo -e "${GREEN}Running migrations for ${name}...${NC}"
docker exec -t edx.devstack.${name} bash -c 'source /edx/app/$1/$1_env && cd /edx/app/$1/$1/ && make migrate' -- "$name"
echo -e "${GREEN}Creating super-user for ${name}...${NC}"
docker exec -t edx.devstack.${name} bash -c 'source /edx/app/$1/$1_env && echo "from django.contrib.auth import get_user_model; User = get_user_model(); User.objects.create_superuser(\"edx\", \"edx@example.com\", \"edx\") if not User.objects.filter(username=\"edx\").exists() else None" | python /edx/app/$1/$1/manage.py shell' -- "$name" &
echo -e "${GREEN}Creating service user and OAuth client for ${name}...${NC}"
docker exec -t edx.devstack.edxapp bash -c 'source /edx/app/edxapp/edxapp_env && python /edx/app/edxapp/edx-platform/manage.py lms --settings=devstack manage_user $1_worker $1_worker@example.com --staff' -- "$name" &
docker exec -t edx.devstack.edxapp bash -c 'source /edx/app/edxapp/edxapp_env && python /edx/app/edxapp/edx-platform/manage.py lms --settings=devstack create_oauth2_client "http://localhost:$2" "http://localhost:$2/complete/edx-oidc/" confidential --client_name $1 --client_id "$1-key" --client_secret "$1-secret" --trusted --logout_uri "http://localhost:$2/logout/" --username $1_worker' -- "$name" "$port" &
done done
# TODO Create ecommerce tenant (ECOM-6564) # TODO Create ecommerce tenant (ECOM-6564)
# Use create_or_update_site (https://github.com/edx/ecommerce/blob/master/ecommerce/core/management/commands/create_or_update_site.py) # Use create_or_update_site (https://github.com/edx/ecommerce/blob/master/ecommerce/core/management/commands/create_or_update_site.py)
docker exec -t edx.devstack.ecommerce bash -c 'source /edx/app/ecommerce/ecommerce_env && python /edx/app/ecommerce/ecommerce/manage.py oscar_populate_countries' docker exec -t edx.devstack.ecommerce bash -c 'source /edx/app/ecommerce/ecommerce_env && python /edx/app/ecommerce/ecommerce/manage.py oscar_populate_countries' &
# TODO Create discovery tenant with correct credentials (ECOM-6565) # TODO Create discovery tenant with correct credentials (ECOM-6565)
docker exec -t edx.devstack.discovery bash -c 'source /edx/app/discovery/discovery_env && python /edx/app/discovery/discovery/manage.py create_or_update_partner --code edx --name edX --courses-api-url "http://edx.devstack.edxapp:18000/api/courses/v1/" --ecommerce-api-url "http://edx.devstack.ecommerce:18130/api/v2/" --organizations-api-url "http://edx.devstack.edxapp:18000/api/organizations/v0/" --programs-api-url "http://edx.devstack.programs:18140/api/v1/" --oidc-url-root "http://edx.devstack.edxapp:18000/oauth2" --oidc-key discovery-key --oidc-secret discovery-secret' & docker exec -t edx.devstack.discovery bash -c 'source /edx/app/discovery/discovery_env && python /edx/app/discovery/discovery/manage.py create_or_update_partner --code edx --name edX --courses-api-url "http://edx.devstack.edxapp:18000/api/courses/v1/" --ecommerce-api-url "http://edx.devstack.ecommerce:18130/api/v2/" --organizations-api-url "http://edx.devstack.edxapp:18000/api/organizations/v0/" --programs-api-url "http://edx.devstack.programs:18140/api/v1/" --oidc-url-root "http://edx.devstack.edxapp:18000/oauth2" --oidc-key discovery-key --oidc-secret discovery-secret' &
# TODO Create credentials tenant (ECOM-6566) # TODO Create credentials tenant (ECOM-6566)
# Compile static assets. We do this last since it takes the longest. # Compile static assets last since they are absolutely necessary for all services. This will allow developers to get
services=('credentials' 'discovery' 'ecommerce' 'programs') # started if they do not care about static assets
for service in "${services[@]}" for service in "${services[@]}"
do do
echo -e "${GREEN}Compiling static assets for ${service}...${NC}" name=${service%%:*}
docker exec -t edx.devstack.${service} bash -c 'source /edx/app/$1/$1_env && cd /edx/app/$1/$1/ && make static' -- "$service"
echo -e "${GREEN}Compiling static assets for ${name}...${NC}"
docker exec -t edx.devstack.${name} bash -c 'source /edx/app/$1/$1_env && cd /edx/app/$1/$1/ && make static' -- "$name" &
done done
# TODO Consider loading demo course/users via Ansible play?
# Save the longest for last...
docker exec -t edx.devstack.edxapp bash -c 'source /edx/app/edxapp/edxapp_env && cd /edx/app/edxapp/edx-platform && paver update_assets --settings devstack'
echo -e "${GREEN}Provisioning complete!${NC}" echo -e "${GREEN}Provisioning complete!${NC}"
provision.sql
...@@ -7,5 +7,12 @@ GRANT ALL ON discovery.* TO 'discov001'@'%' IDENTIFIED BY 'password'; ...@@ -7,5 +7,12 @@ GRANT ALL ON discovery.* TO 'discov001'@'%' IDENTIFIED BY 'password';
CREATE DATABASE IF NOT EXISTS ecommerce; CREATE DATABASE IF NOT EXISTS ecommerce;
GRANT ALL ON ecommerce.* TO 'ecomm001'@'%' IDENTIFIED BY 'password'; GRANT ALL ON ecommerce.* TO 'ecomm001'@'%' IDENTIFIED BY 'password';
CREATE DATABASE IF NOT EXISTS edxapp;
CREATE DATABASE IF NOT EXISTS edxapp_csmh;
GRANT ALL ON edxapp.* TO 'edxapp001'@'%' IDENTIFIED BY 'password';
GRANT ALL ON edxapp_csmh.* TO 'edxapp001'@'%';
CREATE DATABASE IF NOT EXISTS programs; CREATE DATABASE IF NOT EXISTS programs;
GRANT ALL ON programs.* TO 'programs001'@'%' IDENTIFIED BY 'password'; GRANT ALL ON programs.* TO 'programs001'@'%' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment