Continuously Deploying Django with GitHub Actions

By anders pearson 12 May 2019

[Edit 2019-09-21: Updated with the new YAML syntax]

A couple years ago, I wrote a post covering how I set up a continuous deployment pipeline for my personal Django apps using Docker and Jenkins.

A lot has changed since then so I thought it was about time I updated. Especially since this weekend I switched from Jenkins to GitHub Actions.

So, a quick disclaimer before I go on: GitHub Actions is currently in public beta. They have been clear that it is not yet considered stable for production and comes with no warranty. It could change and break my setup at any time. They also haven’t released any information about what it will cost when it is fully out. These are my personal apps so I’m fine with all of that. I’ll try to remember to update this post when the beta is over, but in the meantime, use it at your own risk.

My old post covered the basic setup and, honestly, the general approach hasn’t changed much; I’ve just been swapping out the pieces. It’s worth reading the old post for more details, but I’ll recap the basic approach here so the rest of this post can just cover the new GitHub Actions stuff.

  • I’m still packaging Django apps up in Docker images. My Dockerfile has changed, but these days there are a million articles on how to run Django in Docker, so I won’t cover those changes.
  • I have three application servers where the Django apps run. Each app gets deployed to at least two of them so there is redundancy. This means that a server can fail or go down for maintenance without the site going down. For some of them, there are also Celery Worker and Celery Beat services running to handle offline tasks. There’s an nginx proxy in front of that setup proxying back to the individual Django apps. Consul, consul-template, and registrator are used to dynamically adjust the proxying setup so everything is handled smoothly when a server goes down or when an application instance is added or removed. This part still works basically the same so see the old post if you want more details on how that works.
  • My servers are all managed with Salt, including the production settings and secrets. The Docker containers get those settings in environment variables. Those variables are put in place by Salt and handed to the containers via a couple shell scripts. Again, that part is all unchanged and you can get more details in the old post. What’s relevant here is that there is a docker-runner script on each server that will run an application’s container with all of the production settings injected and let me basically do manage.py commands in those containers. So docker-runner myapp migrate is equivalent to starting up the thraxil/myapp:$TAG container (where $TAG is specified in a designated file and is generally a git hash) with production settings and running manage.py migrate in the container.
  • Static files are hosted on Amazon S3 and CloudFront. Again, this is pretty typical for Django deployments and is better covered elsewhere. All you need to know for my setup is that manage.py collectstatic and manage.py compress need to be called during the deployment to push the latest static files up.

The old approach to deployment was that I ran Jenkins server alongside my app servers. When there was a push to the master branch on GitHub, it would trigger a build there, which would do the following:

  • Build a new docker image for the app. I like to include running unit tests as part of the Dockerfile. That ensures that the tests pass in the exact environment with all the same dependencies as the code will have in production.
  • Tag that image and push it to the Docker Hub.
  • Do a docker pull of that exact tag on all of the app servers.
  • Write the tag out to the right place on those servers so docker-runner will use that version of the image.
  • On one of the servers, run docker-runner myapp migrate, docker-runner myapp collectstatic, and docker-runner myapp compress to handle database migrations and static files.
  • One by one, restart the processes on the app servers.

Jenkins used to run those steps with a fairly small shell script described in the old post. At some point, I converted that to a “proper” Jenkins pipeline specified in a Jenkinsfile. I never wrote about that, and I’m replacing it now, but it’s up here if you are curious. It did some things better than the the shell script version and made for a much nicer overall experience in the Jenkins web interface, but mostly it made me glad that I don’t often have to code in Groovy.

Jenkins worked OK for me but it’s always been a pretty awkward part of the stack and not a lot of fun to run and keep updated.

So when Github Actions came out and I got access to the public beta, I decided to see if I could replace my Jenkins setup.

What I came up with is seems to work pretty well.

Actions for a project are stored as code in a .github directory in your project. There’s a nice web UI for editing actions, but it’s worth looking at the code. My old blog post covered the deployment of my antisocial feed reader app, so for consistency, let’s look at how the GitHub Actions setup looks for it.

I have two workflows. The first just sets up and runs Jessie Frazelle’s branch-cleanup-action GitHub action which keeps things tidy by deleting merged branches. She has a great blog post on how it works that helped me start to get my head around Actions.

The main deploy workflow, in .github/workflows/deploy.yml starts with:

on:
  push:
        branches: master
name: deploy
jobs:
  buildDockerImage:
    name: Build docker image
    runs-on: ubuntu-latest
    steps:

The first step (actually technically two steps) that runs is:

    - uses: actions/checkout@master
    - name: Build docker image
      uses: actions/docker/cli@master
      with:
        args: build -t thraxil/antisocial:${{ github.sha }} .

That builds the docker image, tagged with the git SHA, which GitHub Actions conveniently exposes in the github.sha variable. As I mentioned before, with my Dockerfiles, the unit tests run during the build, so this step also serves as a good check on PRs.

The next two are fairly self-explanatory:

    - name: docker login
      uses: actions/docker/login@master
      env:
        DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
        DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
    - name: docker push
      uses: actions/docker/cli@master
      with:
        args: push thraxil/antisocial:${{ github.sha }}

They log us into the Docker Hub and push the docker image there. The only new bit is that the login step uses the secrets field to grant itself access to some secret settings that are stored in the GitHub project settings.

The bulk of the deploy work (that I had to do, at least) is in the next stanza:

    - name: deploy
      uses: thraxil/django-deploy-action@master
      env:
        APP: antisocial
        KNOWN_HOSTS: ${{ secrets.KNOWN_HOSTS }}
        PRIVATE_KEY: ${{ secrets.PRIVATE_KEY }}
        SSH_USER: anders
        WEB_HOSTS: ${{ secrets.WEB_HOSTS }}
        CELERY_HOSTS: ${{ secrets.CELERY_HOSTS }}
        BEAT_HOSTS: ${{ secrets.BEAT_HOSTS }}

That uses a custom action that I placed in its own repo so I could easily re-use it across my projects.

The great thing about GitHub Actions is that they are just Docker containers that get some specific environment variables and shared directories set up and run how you need them. GitHub Actions will find the Dockerfile in there, build it (if it hasn’t already cached), and run it in the appropriate environment.

That means that it’s easy to package up pretty much any common deployment tool you can think of as a GitHub Action (if someone else hasn’t already done it) or just put your own together if you know how to make a Docker image and write a little shell script.

If I were doing this from scratch or if it were a more complicated deployment process, I’d probably grab or build an Ansible action and do the rest of the deployment that way. In my case though, it’s only a couple steps and I pretty much already had a shell script written (see the old post). So I just made an action that pretty much runs that script in a container, with some tweaks to make it work in the new environment.

The Dockerfile is pretty minimal. Just builds off debian:stable-slim (which GitHub highly recommends to keep images small and using a common base to maximize caching), installs openssh-client (the only package we need that isn’t already there) and drops in a small script.

That script should look familiar from the old blog post. It just does a few additional things to deal with the GitHub environment variables and setting up a valid SSH config. All of the variables that it needs are either set via env in the workflow config above, or they are stored in the project settings as secrets (SSH private key, etc.)

Finally, since I like to use Sentry to track exceptions and Sentry does a better job if you tell it when you deploy new code, I use a community published action to publish a new sentry release for the project:

    - name: sentry release
      uses: juankaram/sentry-release@master
      env:
        ENVIRONMENT: production
        SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
        SENTRY_ORG: thraxil
        SENTRY_PROJECT: antisocial

That’s basically it. I’m mostly pleased with the setup. It took me a few hours to figure out all the pieces and work through some stupid bugs (my own) to get it working how I wanted, but now it’s pretty solid.

Like I said earlier, I really like that it works by just stringing together Docker containers. That means there’s never any question about when GitHub Actions will support some tool. If you can stick it in a Docker image, you can use it. Configuration is straightforward once you’ve spent some time with it (and the Web UI is surprisingly usable and powerful).

I feel like I’m only scratching the surface of what it’s capable of (I’m not even running anything in parallel). The more interesting uses will be less of this “traditional” kind of deployment pipeline and will take better advantage of the Actions’ direct access to the rest of GitHub’s APIs. Right now it feels like the community is still figuring out what those possibilities are and I’m excited to see what patterns emerge.

Tags: django docker devops continuous deployment github actions