Open edX DevStack: A Toolkit for the Future of Learning Software
This article was written by team member Fox Piacenti.
If you’ve had the opportunity to take a course on edx.org, you might wonder how you can contribute to the Open edX platform. The Open edX project is massive – involving many interdependent pieces working together. Deploying the Open edX platform takes a great deal of expertise. However, you can run your own copy for development easily with the “DevStack.” The DevStack sets up all the major components of an Open edX installation and gives you the tools to make your own contributions to Open edX.
Following the DevStack repository’s README results in a working copy of Open edX. Curiously, it creates the repositories not in the DevStack directory, but the one above it:
fox@masamune[~/projects/openedx/devstack] (master)$ ll ../ total 0 drwxr-xr-x 19 fox admin 608 Oct 24 13:33 ./ drwxrwxr-x 28 root admin 896 Oct 24 13:29 ../ drwxr-xr-x 44 fox admin 1408 Oct 24 13:30 course-discovery/ drwxr-xr-x 40 fox admin 1280 Oct 24 13:30 credentials/ drwxr-xr-x 30 fox admin 960 Oct 24 13:30 cs_comments_service/ drwxr-xr-x 69 fox admin 2208 Oct 24 13:30 devstack/ drwxr-xr-x 36 fox admin 1152 Oct 24 13:30 ecommerce/ drwxr-xr-x 26 fox admin 832 Oct 24 13:33 edx-analytics-pipeline/ drwxr-xr-x 25 fox admin 800 Oct 24 13:30 edx-e2e-tests/ drwxr-xr-x 26 fox admin 832 Oct 24 13:31 edx-notes-api/ drwxr-xr-x 61 fox admin 1952 Oct 24 13:33 edx-platform/ drwxr-xr-x 25 fox admin 800 Oct 24 13:33 frontend-app-course-authoring/ drwxr-xr-x 23 fox admin 736 Oct 24 13:33 frontend-app-gradebook/ drwxr-xr-x 25 fox admin 800 Oct 24 13:33 frontend-app-learning/ drwxr-xr-x 25 fox admin 800 Oct 24 13:33 frontend-app-library-authoring/ drwxr-xr-x 27 fox admin 864 Oct 24 13:33 frontend-app-program-console/ drwxr-xr-x 27 fox admin 864 Oct 24 13:33 frontend-app-publisher/ drwxr-xr-x 29 fox admin 928 Oct 24 13:33 registrar/ drwxr-xr-x 22 fox admin 704 Oct 24 13:33 xqueue/
This allows the DevStack to act as one repository among the whole group. However, this may cause a problem if you weren’t expecting to create directories in your parent directory. So we recommend you create a project directory and then clone the DevStack there, so that you can keep your work area clean.
Finding your Bearings
Each of the directories contains its own project. For example,
edx-platform is the Studio and the Learning Management System (LMS). They share a lot of code and so work as one project together. In another example,
cs_comments_service is the forum backend – it handles the course discussions.
Each project runs in its own Docker container, orchestrated by Docker Compose. Docker Compose puts all of the DevStack containers together on a virtual network, with ports exposed to your machine. For instance, you can visit Studio by opening your browser and going to http://localhost:18010, or to http://localhost:18000 for the LMS. Note that the LMS takes considerably longer to start up, and may not be running for up to a minute after you start the Open edX DevStack.
One quirk here is that the LMS and the Studio each have their own docker container, but they both load their code from the
edx-platform repository. Don’t forget updating that Studio’s code also updates the LMS.
For most of this article, we will focus on the LMS and Studio, as they are the most commonly worked on projects in the DevStack.
Updating the LMS and Studio
The majority of the work done on the DevStack involves
edx-platform in some way – and thus the LMS and Studio containers are involved. If you switch branches, you must update your Python requirements and your database.
Let’s say you want to pull the latest changes from master. You go into the
edx-platform repository and then pull:
$ git pull From github.com:edx/edx-platform * branch master -> FETCH_HEAD Updating 6f69fd7dc2..8cef028429 Fast-forward …
The updated code likely has changed the dependencies. Switch back to the
Then, enter the LMS.
Once inside, we want to run:
Once the database is updated, start the process over again with Studio. Exit the shell with
And then enter the Studio shell with:
You can then run the same paver commands as before. Your LMS and Studio should now be ready!
Working with Static/Frontend Assets
Once inside, run:
The frontend files will then be built and will watch for any changes, automatically rebuilding when they are found.
Any changes you make to the platform’s logic should always be verified with new tests. Likewise, you should run existing tests to verify you’ve not broken anything with your updates. To run unit tests, once more enter the shell of the system you wish to test from the
To run every test imaginable (we don’t recommend this unless you have a lot of time on your hands):
If you wish to run all Python unit tests:
To run specific Python unit tests:
paver test_system -s lms -t path/to/file
path/to/file is a path to a file or directory that contains tests you want to run.
You may have noticed that the paver command is used to perform several actions. Well, we’ve only scratched the surface of what it can do. Most of the commands that we haven’t covered won’t be useful to the average developer, but may come in handy from time to time. You can get a full listing of all commands paver supports by running:
One thing that paver does NOT do is run the Django shell for you. You are likely aware that the LMS and Studio are very complex Django projects. Django comes with a REPL that allows you to run code directly. This makes it easy to look up database information without using SQL, for instance. You can perform any number of tasks with the full power of a contextualized Python shell.
To run the LMS Django shell, from within the LMS container, run:
./manage.py lms shell
Keen eyes will notice that this is almost the normal Django shell command. However, for all management commands in edx-platform, you must specify either lms or cms, since they both initialize different environments.
In fact, many other Django management commands are available. Most of the useful ones are invoked by paver as required, but you can list them directly like so:
This will list all commands, which you can then investigate further. For instance, to get help for the migrate command, run:
./manage.py lms migrate --help
Getting oriented around the Open edX DevStack can be overwhelming at times, but with a handful of commands you can make your way around the project and contribute your own code to the Open edX platform.
Of course, if you like, you can always call upon us to assist you with Open edX projects. Come drop us a line and let us know what you’re interested in doing. We’ll let you know how we can help!