Merge pull request #361 from edx/antoviaque/vagrant-rbenv

Add support for Vagrant install, and switch from RVM to rbenv

.gitattributes

+* -text

.gitignore

@@ -44,3 +44,4 @@ node_modules
 .prereqs_cache
 autodeploy.properties
 .ws_migrations_complete
+.vagrant/

README.md

@@ -2,8 +2,136 @@ This is the main edX platform which consists of LMS and Studio.
 
 See [code.edx.org](http://code.edx.org/) for other parts of the edX code base.
 
-Installation
-============
+Installation - The first time
+=============================
+
+The following instructions will help you to download and setup a virtual machine
+with a minimal amount of steps, using Vagrant. It is recommended for a first
+installation, as it will save you from many of the common pitfalls of the
+installation process.
+
+1. Make sure you have plenty of available disk space, >5GB
+2. Install Git: http://git-scm.com/downloads
+3. Install VirtualBox: https://www.virtualbox.org/wiki/Download_Old_Builds_4_2
+   (you need version 4.2.12, as later/earlier versions might not work well with 
+   Vagrant)
+4. Install Vagrant: http://www.vagrantup.com/ (Vagrant 1.2.2 or later)
+5. Open a terminal
+6. Download the project: `git clone git://github.com/edx/edx-platform.git`
+7. Enter the project directory: `cd edx-platform/`
+8. (Windows only) Run the commands to 
+   [deal with line endings and symlinks under Windows](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#dealing-with-line-endings-and-symlinks-under-windows)
+9. Start: `vagrant up`
+
+The last step might require your host machine's administrator password to setup NFS.
+
+Afterwards, it will download an image, install all the dependencies and configure
+the VM. It will take a while, go grab a coffee.
+
+Once completed, hopefully you should see a "Success!" message indicating that the
+installation went fine. (If not, refer to the 
+[troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting).)
+
+Note: by default, the VM will get the IP `192.168.20.40`. If you need to use a 
+different IP, you can edit the file `Vagrantfile`. If you have already started the 
+VM with `vagrant up`, see "Stopping and restarting the VM" below to take the change 
+into account.
+
+Accessing the VM
+----------------
+
+Once the installation is finished, to log into the virtual machine:
+
+```
+$ vagrant ssh
+```
+
+Note: This won't work from Windows, install install PuTTY from
+http://www.chiark.greenend.org.uk/%7Esgtatham/putty/download.html instead. Then
+connect to 127.0.0.1, port 2222, using vagrant/vagrant as a user/password.
+
+Using edX
+---------
+
+Once inside the VM, you can start Studio and LMS with the following commands
+(from the `/opt/edx/edx-platform` folder):
+
+Learning management system (LMS):
+
+```
+$ rake lms[cms.dev,0.0.0.0:8000]
+```
+
+Studio:
+
+```
+$ rake cms[dev,0.0.0.0:8001]
+```
+
+Once started, open the following URLs in your browser:
+
+* Learning management system (LMS): http://192.168.20.40:8000/
+* Studio (CMS): http://192.168.20.40:8001/
+
+You can develop by editing the files directly in the `edx-platform/` directory you
+downloaded before, you don't need to connect to the VM to edit them (the VM uses
+those files to run edX, mirroring the folder in `/opt/edx/edx-platform`).
+
+You may also want to create a super-user with:
+
+```
+$ rake django-admin["createsuperuser"]
+```
+
+Also note that if you register a new user through the web interface,
+the activiation email will be posted to your VM's terminal window (search for
+lines similar to):
+
+```
+Subject: Your account for edX Studio
+From: registration@edx.org
+```
+
+and find the activation URL for the account you've created.
+
+See the [Frequently Asked Questions](https://github.com/edx/edx-platform/wiki/Frequently-Asked-Questions)
+for more usage tips.
+
+Stopping & starting
+-------------------
+
+To stop the VM (from your `edx-platform/` directory):
+
+```
+$ vagrant halt
+```
+
+To restart:
+
+```
+$ vagrant up
+```
+
+or, to start without attempting to update the dependencies:
+
+```
+$ vagrant up --no-provision
+```
+
+Troubleshooting
+---------------
+
+If anything doesn't work as expected, see the 
+[troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting).
+
+Installation - Advanced
+=======================
+
+Note: The following installation instructions are for advanced users & developers
+who are familiar with setting up Python, Ruby & node.js virtual environments.
+Even if you know what you are doing, edX has a large code base with multiple
+dependencies, so you might still want to use the method described above the
+first time, as Vagrant helps avoiding issues due to the different environments.
 
 There is a `scripts/create-dev-env.sh` that will attempt to set up a development
 environment.

Vagrantfile

+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+Vagrant.configure("2") do |config|
+  config.vm.box = "precise32"
+  config.vm.box_url = "http://files.vagrantup.com/precise32.box"
+
+  config.vm.network :forwarded_port, guest: 8000, host: 9000
+  config.vm.network :forwarded_port, guest: 8001, host: 9001
+
+  # Create a private network, which allows host-only access to the machine
+  # using a specific IP.
+  config.vm.network :private_network, ip: "192.168.20.40"
+
+  nfs_setting = RUBY_PLATFORM =~ /darwin/ || RUBY_PLATFORM =~ /linux/
+  config.vm.synced_folder ".", "/opt/edx/edx-platform", id: "vagrant-root", :nfs => nfs_setting
+
+  # Make it so that network access from the vagrant guest is able to
+  # use SSH private keys that are present on the host without copying
+  # them into the VM.
+  config.ssh.forward_agent = true
+
+  config.vm.provider :virtualbox do |vb|
+    # Use VBoxManage to customize the VM. For example to change memory:
+    vb.customize ["modifyvm", :id, "--memory", "1024"]
+
+    # This setting makes it so that network access from inside the vagrant guest
+    # is able to resolve DNS using the hosts VPN connection.
+    vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
+  end
+
+  config.vm.provision :shell, :path => "scripts/vagrant-provisioning.sh"
+end

rakelib/workspace.rake

@@ -7,10 +7,11 @@ namespace :ws do
     task :migrate => MIGRATION_MARKER_DIR do
         Dir['ws_migrations/*'].select{|m| File.executable?(m)}.each do |migration|
             completion_file = File.join(MIGRATION_MARKER_DIR, File.basename(migration))
-            if ! File.exist?(completion_file)
+            is_excluded = File.basename(migration).start_with?("README")
+            if ! File.exist?(completion_file) && ! is_excluded
                 sh(migration)
                 File.write(completion_file, "")
             end
         end unless SKIP_MIGRATIONS
     end
-end
\ No newline at end of file
+end

scripts/install-system-req.sh

@@ -35,18 +35,13 @@ case `uname -s` in
             squeeze|wheezy|jessie|maya|lisa|olivia|nadia|natty|oneiric|precise|quantal|raring)
                 output "Installing Debian family requirements"
 
-                # DEBIAN_FRONTEND=noninteractive is required for silent mysql-server installation
-                export DEBIAN_FRONTEND=noninteractive
-
                 # add repositories
                 cat $APT_REPOS_FILE | xargs -n 1 sudo add-apt-repository -y
-                sudo apt-get -y update
-                sudo apt-get -y install gfortran
-                sudo apt-get -y install graphviz libgraphviz-dev graphviz-dev
-                sudo apt-get -y install libatlas-dev libblas-dev 
-                sudo apt-get -y install ruby-rvm
+                sudo apt-get -yq update
+                sudo DEBIAN_FRONTEND=noninteractive apt-get -yq install gfortran graphviz \
+                            libgraphviz-dev graphviz-dev libatlas-dev libblas-dev
                 # install packages listed in APT_PKGS_FILE
-                cat $APT_PKGS_FILE | xargs sudo apt-get -y install
+                cat $APT_PKGS_FILE | xargs sudo DEBIAN_FRONTEND=noninteractive apt-get -yq install
                 ;;
             *)
                 error "Unsupported distribution - $distro"

scripts/vagrant-provisioning.sh

+#!/bin/bash -e
+#
+# Copyright (C) 2013 edX <info@edx.org>
+#
+# Authors: Xavier Antoviaque <xavier@antoviaque.org>
+#          David Baumgold <david@davidbaumgold.com>
+#
+# This software's license gives you freedom; you can copy, convey,
+# propagate, redistribute and/or modify this program under the terms of
+# the GNU Affero General Public License (AGPL) as published by the Free
+# Software Foundation (FSF), either version 3 of the License, or (at your
+# option) any later version of the AGPL published by the FSF.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero
+# General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program in a file in the toplevel directory called
+# "AGPLv3".  If not, see <http://www.gnu.org/licenses/>.
+
+
+###############################################################################
+
+# vagrant-provisioning.sh:
+#
+# Script to setup base environment on Vagrant, based on `precise32` image
+# Runs ./scripts/create-dev-env.sh for the actual setup
+#
+# This script is ran by `$ vagrant up`, see the README for more explanations
+
+
+# APT - Packages ##############################################################
+
+apt-get update
+apt-get install -y python-software-properties vim
+
+
+# Curl - No progress bar ######################################################
+
+[[ -f ~vagrant/.curlrc ]] || echo "silent show-error" > ~vagrant/.curlrc
+chown vagrant.vagrant ~vagrant/.curlrc
+
+
+# SSH - Known hosts ###########################################################
+
+# Github
+([[ -f ~vagrant/.ssh/known_hosts ]] && grep "zBX7bKA= ssh" ~vagrant/.ssh/known_hosts) || {
+    echo "|1|4DtBcMsTM4zgl/jTS7h3ZkmS/Vc=|XkRnn2xEhr8ixOxeskJAzBX7bKA= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==" >> ~vagrant/.ssh/known_hosts
+}
+([[ -f ~vagrant/.ssh/known_hosts ]] && grep "jO3J5bvw= ssh" ~vagrant/.ssh/known_hosts) || {
+    echo "|1|9rANf/qOAPgKH/TXpGuZCAgGxMs=|x9VYWEDI8kiotbhhNXqjO3J5bvw= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==" >> ~vagrant/.ssh/known_hosts
+}
+chown vagrant.vagrant ~vagrant/.ssh/known_hosts
+
+
+# edX - Development environment ###############################################
+
+# Node modules require a filesystem with symlinks (Windows support)
+mkdir -p /opt/edx/node_modules /opt/edx/edx-platform/node_modules
+mount -o bind /opt/edx/node_modules /opt/edx/edx-platform/node_modules
+
+# Force rechecking all prerequisites (could have been fetched outside of the VM)
+rm -rf /opt/edx/edx-platform/.prereqs_cache
+
+# Permissions
+chown vagrant.vagrant /opt/edx /opt/edx/node_modules /opt/edx/edx-platform/node_modules
+
+# For convenience with `vagrant ssh`, the `edx-platform` virtualenv is always
+# loaded after the first run, so we need to deactivate that behavior to run
+# `create-dev-env.sh`.
+[[ -f ~vagrant/.bash_profile ]] && {
+    mv ~vagrant/.bash_profile ~vagrant/.bash_profile.bak
+}
+sudo -u vagrant -i bash -c "cd /opt/edx/edx-platform && PROJECT_HOME=/opt/edx ./scripts/create-dev-env.sh -ynq"
+
+# Load .bashrc ################################################################
+([[ -f ~vagrant/.bash_profile ]] && grep ".bashrc" ~vagrant/.bash_profile) || {
+    echo -e "\n. /home/vagrant/.bashrc\n" >> ~vagrant/.bash_profile
+}
+
+
+# Virtualenv - Always load ####################################################
+
+([[ -f ~vagrant/.bash_profile ]] && grep "edx-platform/bin/activate" ~vagrant/.bash_profile) || {
+    echo -e "\n. /home/vagrant/.virtualenvs/edx-platform/bin/activate\n" >> ~vagrant/.bash_profile
+}
+
+
+# Directory ###################################################################
+
+grep "cd /opt/edx/edx-platform" ~vagrant/.bash_profile || {
+    echo -e "\ncd /opt/edx/edx-platform\n" >> ~vagrant/.bash_profile
+}
+
+
+# End #########################################################################
+
+cat << EOF
+==============================================================================
+Success!
+==============================================================================
+
+Now, from the virtual machine (connect with "vagrant ssh" if vagrant didn't
+log you in already), you can start Studio & LMS with the following commands:
+
+- Learning management system (LMS):
+    $ rake lms[cms.dev,0.0.0.0:8000]
+
+    =>  http://192.168.20.40:8000/
+
+- Studio:
+    $ rake cms[dev,0.0.0.0:8001]
+
+    => http://192.168.20.40:8001/
+
+
+See the README for details.
+
+
+EOF
+
+