Native Scheduler: Performing Database Migrations with Ephemeral Containers

Database migrations are an important part of any software upgrade, and having a strategy to manage
data migrations in enterprise deployments is important. One common pattern to manage database migrations is to run an ephemeral container along with container events to run migration tasks.

In this example, I will take a very simply python stack based on the Django framework and Postgres, and show how to run database migrations. No knowledge of python is necessary to understand this.

Database Component

Let’s begin by creating a container in the Replicated based on the public Postgres Docker image from
Dockerhub. This snippet will expose Postgres on port 5432 on the host, and define an event that
instructs Replicated to poll port 5432 on the host until a connection can be established. We are
going to use this event as a trigger to run the migration.

- name: db
  containers:
  - source: public
    image_name: postgres
    version: 9.5
    env_vars:
    - name: POSTGRES_DB
      value: pythonapp
    - name: POSTGRES_USER
      value: 'some_user'
    - name: POSTGRES_PASSWORD
      value: 'some_password'
    volumes:
    - host_path: /data/postgresql/data
      container_path: /var/lib/postgresql/data
    ports:
    - private_port: "5432"
      public_port: "5432"
      port_type: tcp
    publish_events:
    - name: Postgres started and waiting for connections
      trigger: port-listen
      data: "5432"
      subscriptions:
      - component: db-migration
        container: pythonapp
        action: start

Migration Component

Next, we will run our “app” component, but overriding the CMD in with python manage.py db upgrade,
which is how a migration is performed in a Django app. This should be replaced with rake db:migrate
or anything appropriate for your stack.

We define this container with ephemeral: true to indicate that this container is expected to exit.
Without this flag, Replicated will detect that the container exited, and will flag the entire app
as “Stopped” in the dashboard.

Finally, we listen for this container to stop, and fire an event to start the app container.

- name: db-migration
  containers:
  - source: replicated
    image_name: pythonapp
    version: 1.4.2
    ephemeral: true
    env_vars:
      - name: DB_URL
        value: "postgresql://pythonapp:{{repl ConfigOption \"postgres_pw\"}}@{{repl NodePrivateIPAddress \"db\" \"postgres\" }}:5432/pythonapp"
    cmd: '["python", "manage.py", "db", "upgrade"]'
    publish_events:
    - name: db migration complete
      trigger: container-stop
      subscriptions:
      - component: python-app
        container: pythonapp
        action: start

App Component

The app component is a standard Django app. We assume that it can start normally using a built in
CMD or ENTRYPOINT in the Dockerfile. We aren’t overriding the CMD this time, so the container
will start serving the web site.

- name: python-app
  containers:
  - source: replicated
    image_name: pythonapp
    version: 1.4.2
    env_vars:
      - name: DB_URL
        value: "postgresql://pythonapp:{{repl ConfigOption \"postgres_pw\"}}@{{repl NodePrivateIPAddress \"db\" \"postgres\" }}:5432/pythonapp"

If this release contains a required migration and the migration is not going to be present in future
releases, you can mark this release as required when promoting it.

For more on sequencing the startup of your application take a look at this article.

Download entire Replicated YML.