Skip to content

D
devstack
Commit 87acf413 by Renzo Lucioni
Options

Replace Python clone script with a Bash script 

Eliminates the need for a virtualenv.
parent ce95689c
Showing with 39 additions and 109 deletions
Makefile
...@@ -5,7 +5,7 @@ help: ## Display this help message ...@@ -5,7 +5,7 @@ help: ## Display this help message
@perl -nle'print $& if m{^[\.a-zA-Z_-]+:.*?## .*$$}' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m %-25s\033[0m %s\n", $$1, $$2}' @perl -nle'print $& if m{^[\.a-zA-Z_-]+:.*?## .*$$}' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m %-25s\033[0m %s\n", $$1, $$2}'
clone: ## Clone service repos clone: ## Clone service repos
./clone.py ./clone.sh
# TODO Print out help for this target. Even better if we can iterate over the services in docker-compose.yml, and # TODO Print out help for this target. Even better if we can iterate over the services in docker-compose.yml, and
# print the actual service names. # print the actual service names.
......
README.md
...@@ -23,50 +23,36 @@ All of the services can be run by following the steps below. Note that since we ...@@ -23,50 +23,36 @@ All of the services can be run by following the steps below. Note that since we
configure Docker with a sufficient amount of resources. Our testing found that [configuring Docker for Mac](https://docs.docker.com/docker-for-mac/#/advanced) configure Docker with a sufficient amount of resources. Our testing found that [configuring Docker for Mac](https://docs.docker.com/docker-for-mac/#/advanced)
with 2 CPUs and 4GB of memory works well. with 2 CPUs and 4GB of memory works well.
1. Create a Python 3 `virtualenv`. If you're using [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io 1. The Docker Compose file mounts a host volume for each service's executing code. The host directory is expected to be
), you can do this with:
```
$ mkvirtualenv devstack --python=$(which python3)
```
Source the virtualenv and install requirements:
```
$ workon devstack
(devstack)$ make requirements
```
2. The Docker Compose file mounts a host volume for each service's executing code. The host directory is expected to be
a sibling of this directory. For example, if this repo is cloned to `~/workspace/devstack`, host volumes will be a sibling of this directory. For example, if this repo is cloned to `~/workspace/devstack`, host volumes will be
expected in `~/workspace/course-discovery`, `~/workspace/ecommerce`, etc. These repos can be cloned with the command expected in `~/workspace/course-discovery`, `~/workspace/ecommerce`, etc. These repos can be cloned with the command
below. below.
``` ```
(devstack)$ make clone $ make clone
``` ```
3. Run the provision command, if you haven't already, to configure the various services with superusers (for 2. Run the provision command, if you haven't already, to configure the various services with superusers (for
development without the auth service) and tenants (for multi-tenancy). development without the auth service) and tenants (for multi-tenancy).
The username and password for the superusers are both "edx". You can access the services directly via Django admin The username and password for the superusers are both "edx". You can access the services directly via Django admin
at the `/admin/` path, or login via single sign-on at `/login/`. at the `/admin/` path, or login via single sign-on at `/login/`.
``` ```
(devstack)$ make devstack.provision $ make devstack.provision
``` ```
4. Start the services. 3. Start the services.
``` ```
(devstack)$ make devstack.start $ make devstack.start
``` ```
After the services have started, if you need shell access to one of the services, run `make devstack.open.<service>`. After the services have started, if you need shell access to one of the services, run `make devstack.open.<service>`.
For example to access the Catalog/Course Discovery Service, you can run: For example to access the Catalog/Course Discovery Service, you can run:
``` ```
(devstack)$ make devstack.open.discovery $ make devstack.open.discovery
``` ```
## Usernames and Passwords ## Usernames and Passwords
......
clone.py deleted 100755 → 0
#!/usr/bin/env python3
"""
Python 3 script for cloning Git repos housing edX services.
These repos are mounted as data volumes into their corresponding Docker
containers to facilitate development.
Repo clone URLs can be found in settings.yml. Repos are cloned to the directory
above the one housing this file.
"""
import concurrent.futures
import logging
from logging.config import dictConfig
import os
from os.path import join, abspath, dirname
import re
import subprocess
import yaml
# Configure logging.
dictConfig({
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'standard': {
'format': '%(asctime)s %(levelname)s %(process)d [%(filename)s:%(lineno)d] - %(message)s',
},
},
'handlers': {
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'standard',
},
},
'loggers': {
'': {
'handlers': ['console'],
'level': 'DEBUG',
'propagate': False
},
},
})
logger = logging.getLogger(__name__)
class Repo:
"""Utility class representing a Git repo."""
def __init__(self, clone_url):
self.clone_url = clone_url
match = re.match(r'.*edx/(?P<name>.*).git', self.clone_url)
self.name = match.group('name')
parent_path = dirname(dirname(abspath(__file__)))
self.path = join(parent_path, self.name)
def clone(self):
"""Clone the repo."""
subprocess.run(['git', 'clone', self.clone_url, self.path], check=True)
@property
def exists(self):
"""Determine if the repo is already checked out."""
return os.path.exists(self.path)
if __name__ == '__main__':
with open('settings.yml') as f:
settings = yaml.load(f)
with concurrent.futures.ThreadPoolExecutor() as executor:
repos = [Repo(clone_url) for clone_url in settings['repos']]
for repo in repos:
if repo.exists:
logger.info('Repo [{name}] found at [{path}].'.format(name=repo.name, path=repo.path))
else:
logger.info('Repo [{name}] not found. Cloning to [{path}].'.format(name=repo.name, path=repo.path))
executor.submit(repo.clone)
clone.sh 0 → 100755
#!/bin/bash
# Script for cloning Git repos housing edX services. These repos are mounted as
# data volumes into their corresponding Docker containers to facilitate development.
# Repos are cloned to the directory above the one housing this file.
repos=(
"https://github.com/edx/course-discovery.git"
"https://github.com/edx/credentials.git"
"https://github.com/edx/ecommerce.git"
"https://github.com/edx/edx-platform.git"
"https://github.com/edx/programs.git"
)
name_pattern=".*edx/(.*).git"
for repo in ${repos[*]}
do
# Use Bash's regex match operator to capture the name of the repo.
# Results of the match are saved to an array called $BASH_REMATCH.
[[ $repo =~ $name_pattern ]]
name="${BASH_REMATCH[1]}"
cd ..
if [ -d "$name" ]; then
printf "The [%s] repo is already checked out. Continuing.\n" $name
else
git clone $repo
fi
cd - &> /dev/null
done
settings.yml deleted 100644 → 0
repos:
- https://github.com/edx/course-discovery.git
- https://github.com/edx/credentials.git
- https://github.com/edx/ecommerce.git
- https://github.com/edx/programs.git
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment