Upgrade Releases With Edeliver

Written by Pete Corey on Jan 23, 2017.

Last week we saw that using edeliver could drastically simplify our Elixir release process. Now that we can build and deploy our initial releases, we should investigate how edeliver can help us build, manage, and deploy upgrade releases.

To get a better idea of how upgrade releases work with edeliver, let’s make a change to our original application, build an upgrade release from that change, and then deploy that change to our production server.

Configuring edeliver

Before we start building our upgrade release, we’ll need to make a configuration change to Distillery.

Recently, Distillery changed where it stores its release artifacts. If left unchanged, this new output directory would break our edeliver release process:

==> Upgrading prod from 0.0.1 to 0.0.1+82e2ed7
==> Upgrade from 0.0.1 to 0.0.1+82e2ed7 failed:
  0.0.1 does not exist at _build/prod/rel/prod/releases/0.0.1

Thankfully, we can change where these artifacts are stored with Distillery’s output_dir configuration option.

Let’s edit rel/config.exs and set output_dir to the rel/<application name> directory:


set output_dir: "rel/hello_edeliver"

Now Distillery will store our release artifacts in rel/hello_edeliver, which is where edeliver expects to find them.

Making Our Change

Let’s make a small change to our application. We’ll update our index.html.eex and add a line indicating that we’ve upgraded the application.


<p>We've upgraded!</p>

At this point, we can either manually upgrade our application’s version in our mix.exs file, or we can let edeliver manage our versioning for us.

Let’s keep things as hands-off as possible and let edeliver manage our application’s version.

At this point, we can commit our changes and move on to building our upgrade.


git commit -am "Added upgrade message."

Building and Deploying Our Upgrade

Elixir upgrade releases are always built off on a previous release. Distillery will generate a patch that changes only what is different between the old and new version of the application.

With that in mind, we need to know what version of our application is currently running in production. We can find this out using edeliver’s version mix task:


mix edeliver version production

In our case, we can see that we’re running version 0.0.1 of our application in production.

Knowing that, we can generate our upgrade release. Upgrade releases are built with the build upgrade mix task:


mix edeliver build upgrade --with=0.0.1 --auto-version=git-revision

We’re using the --with flag to point to our previous 0.0.1 release. We’re also using the --auto-version flag to tell edeliver to automatically handle the versioning of our application.

This command will generate an upgrade release with a version similar to 0.0.1+82e2ed7. You can see that the git-revision option appended the current commit’s SHA to the end of the original release version.

For more information about upgrade flags and automatic versioning, be sure the check out the edeliver documentation and wiki.


Once our upgrade release has been built, deploying it to our production environment is a simple task:


mix edeliver deploy upgrade to production

That’s it! Our change was applied instantly to our running application with zero downtime.

If we check the version of our application running in production, we should see 0.0.1+82e2ed7:


mix edeliver version production

And lastly, when we view our application in the browser, we should see our upgrade message in the DOM.

Success!

Final Thoughts

After getting over a few configuration hurdles, using edeliver to build and deploy an upgrade release was a breeze.

Being able to seamlessly deploy an upgrade to any number of hosts with zero downtime is an incredibly powerful tool, and edeliver gives a polished interface for just this.

I’m looking forward to using edeliver more in the future.

Simplifying Elixir Releases With Edeliver

Written by Pete Corey on Jan 16, 2017.

In our previous two posts, we’ve built releases manually with Distillery, and deployed an upgrade release to an existing Elixir application. As we’ve seen, this was a very hands-on process.

Thankfully, there is another tool that we can use to simplify our release process. In this post, we’ll dive into using edeliver to build and release Elixir applications.

Our Example Application

To help walk ourselves through the process of using edeliver, let’s create a new Phoenix application that we can deploy to a “production server”.

Let’s start by creating a new, basic Phoenix project:


mix phoenix.new hello_edeliver --no-ecto

We’ll be deploying this application as is, so let’s move onto the process of installing and configuring our release tools.

First, we’ll add dependencies on edeliver, and distillery, and add edeliver to our projects list of applications:


def application do
  [...,
   applications: [..., :edeliver]]
end

defp deps do
  [
   ...
   {:edeliver, "~> 1.4.0"},
   {:distillery, ">= 0.8.0", warn_missing: false}]
end

Note: edeliver is a high-level tool designed to orchestrate the creation, deployment, and management of Elixir releases. Under the covers, it can use Distillery, exrm, relx, or rebar to build release bundles.

Our application won’t use any “production secrets”, so to simplify this introduction to edeliver, let’s remove our dependency on the config/prod.secrets.exs file by commenting out its import_config line in config/prod.exs:


# import_config "prod.secrets.exs"

Finally, while we’re in config/prod.exs, let’s add a few vital configuration options to our HelloEdeliver endpoint:


config :hello_edeliver, HelloEdeliver.Endpoint,
  http: [port: {:system, "PORT"}],
  url: [host: "...", port: {:system, "PORT"}],
  server: true,
  root: ".",
  version: Mix.Project.config[:version],
  cache_static_manifest: "priv/static/manifest.json"

And with that, our example application should be ready for release.

Before we move on, let’s fire up our development server with mix phoenix.server and make sure that everything looks as we’d expect.

Configuring Distillery and edeliver

Now that our application is ready, we need to spend some time configuring our deployment tools.

First, let’s create our Distillery configuration file (rel/config.exs) with the release.init mix task:


mix release.init

The default configuration options provided by release.init should be fine for our first deployment.

Next, let’s take a look at our edeliver configuration file (.deliver/config). We’ll update the provided fields and point edeliver to a remote “build host” and “production host”:


APP="hello_edeliver"

BUILD_HOST="ec2-52-87-163-123.compute-1.amazonaws.com"
BUILD_USER="ec2-user"
BUILD_AT="/home/ec2-user/hello_edeliver/builds"

PRODUCTION_HOSTS="ec2-54-172-3-38.compute-1.amazonaws.com"
PRODUCTION_USER="ec2-user"
DELIVER_TO="/home/ec2-user"

At this point, we could add any number of production hosts, or add another environment entirely, such as a staging environment.


Because we’re building a Phoenix application, we’ll need to build and digest all of our static assets before building our release.

We’re building our release bundle on our remote build host, so we’ll need to instruct edeliver to do these things for us. Thankfully, edeliver comes with a host of pre and post hooks that we can use to accomplish this task.

Let’s add a “pre compile” hook to build (npm install, npm run deploy) and digest (mix phoenix.digest) our static assets:


pre_erlang_clean_compile() {
  status "Installing NPM dependencies"
  __sync_remote "
    [ -f ~/.profile ] && source ~/.profile
    set -e

    cd '$BUILD_AT'
    npm install $SILENCE
  "

  status "Building static files"
  __sync_remote "
    [ -f ~/.profile ] && source ~/.profile
    set -e

    cd '$BUILD_AT'
    mkdir -p priv/static
    npm run deploy $SILENCE
  "

  status "Running phoenix.digest"
  __sync_remote "
    [ -f ~/.profile ] && source ~/.profile
    set -e

    cd '$BUILD_AT'
    APP='$APP' MIX_ENV='$TARGET_MIX_ENV' $MIX_CMD phoenix.digest $SILENCE
  "
}

By default, edeliver will store all release tarballs in the .deliver/releases folder. Let’s preemptively exclude this folder from revision control:


echo ".deliver/releases/" >> .gitignore

The last thing we need to do to prepare for our first release is to commit all of our changes!

Preparing Our Hosts

When using edeliver, we’ll always have at least two different types of remote environments.

The first is our build host. This is where edeliver actually builds our release. In our case, it’s running a variation of a mix release command. As we saw in a previous article, it’s important that the build host be nearly identical to the production host in terms or architectures, etc…

Our build host has several dependencies. We’ll need to install Git, Erlang & Elixir, and Node. Thankfully, we’ll only need to provision a single build host.

The second type of host is our deployment target. These machines will be running our application in environments such as staging or production. We can have any number of these hosts living in the wild.

Our deployment machines have no dependencies. We don’t even need to install Erlang - our release will bring it along for us.

However, we do need to set any required environment variables on these hosts. Out of the box, a Phoenix application needs a PORT value. We can export this from our ~/.profile:


export PORT=4000

If you’re using Amazon EC2 hosts, you’ll most likely authenticate with your remote hosts by passing along a *.pem file when you establish your SSH connection.

While edeliver doesn’t explicitly support this kind of authentication, adding a Host/IndentityFile entry in your ~/.ssh/config file for each of your remote hosts will authorize edeliver to communicate with these hosts:

Host ec2-foo.compute-1.amazonaws.com
  IdentityFile ~/my_identity_file.pem

Host ec2-bar.compute-1.amazonaws.com
  IdentityFile ~/my_identity_file.pem

Making Our Release

Once our application is ready, our deployment tools are configured, and our hosts are provisioned, we can build and deploy our release.

The first step is to instruct edeliver to build our release on our build host:


mix edeliver build release

Success! If everything went well, we should find our newly built release tarball in our release store (.deliver/releases).

Next, let’s push our initial release up to our production server:


mix edeliver deploy release to production

Another success! We’ve deployed our initial release to our production environment.

Now let’s fire up the Phoenix application in production:


mix edeliver start production

We can check that everything went well by using edeliver to ping our application:


mix edeliver ping production

If our application is up and running, we should receive a pong reply to our ping.

At this point, we should be able to navigate to our production host and find our application running at the port we’ve specified.

Final Thoughts

From my limited experience, edeliver is a fantastic tool.

While it does require up-front work (more work than using Distillery on its own), that work is purely up-front. Once you’ve provisioned your build host and set up your edeliver configuration file, building and deploying releases to any number of hosts is smooth sailing.

I’m excited to work more with edeliver. Expect an upcoming post on deploying hot upgrade releases to see how the process compares with just using Distillery.

Upgrade Releases With Distillery

Written by Pete Corey on Jan 9, 2017.

Now that we’ve deployed our first Elixir application using Distillery, let’s dive into the process of building and deploying a “hot upgrade” for our application!

We’ll move through the process of making a change to our application, using Distillery to build our upgrade, and then deploying the upgrade with zero downtime.

Making Our Changes

With the goal of keeping things simple, let’s make a small change to our application’s index.html.eex file. We’ll add a new <h3> tag that shows we’ve upgraded our application:


<div class="jumbotron">
  <h2><%= gettext "Welcome to %{name}", name: "Phoenix!" %></h2>
  <h3>Version 0.0.2!</h3>
  ...
</div>

Next, we’ll need to upgrade our project’s version in our mix.exs file:


def project do
  [app: :hello_distillery,
   version: "0.0.2",
   ...

It’s important to upgrade our project’s version so distillery can correctly build an upgrade/downgrade patch for our application.

Problems With Our Build

And with that, we should be ready to build our upgrade release. The process for building an upgrade is very similar to that of building a normal release:


MIX_ENV=prod mix do compile, phoenix.digest, release --env=prod --upgrade

Unfortunately, due to how we configured our project in the last article, this upgrade build will fail:


==> Assembling release..
==> Building release hello_distillery:0.0.2 using environment prod
==> Failed to build release:
    Hot upgrades will fail when include_erts: false is set,
    you need to set include_erts to true or a path if you plan to use them!

Because we installed Erlang on our production machine, we set include_erts to false in our release configuration file, indicating that we didn’t want to include the Erlang runtime in our final build.

Distillery is complaining that when building upgrade releases, include_erts either needs to be true (which will include the Erlang runtime installed on our development machine in the release), or a path pointing to the Erlang runtime we want to include in the release.

A Tale of Two Erlangs

My development machine is a Macbook, and my production machine is an Amazon Linux EC2 instance. This means that including my development machine’s version of Erlang in the release is not an option.

This means that we’ll have to copy the instance of Erlang we installed on our production server onto our development server and point to it with include_erts.

The Distillery walkthrough touches in this in the “Deploying Your Release” section:

If you are deploying to a different OS or architecture than the build machine, you should either set include_erts: false or include_erts: "path/to/cross/compiled/erts".

The latter will require that you have built/installed Erlang on the target platform, and copied the contents of the Erlang lib directory somewhere on your build machine, then provided the path to that directory to include_erts.

Following this advice, we’ll copy the Erlang runtime from our production server into a folder on our development machine (~/al-erlang):


scp -i ~/hello_distillery.pem -r \
    ec2-user@ec2...amazonaws.com:/usr/local/lib/erlang \
    /Users/pcorey/al-erlang

Now we can change our include_erts to point to our newly downloaded ~/al-erlang directory:


set include_erts: "/Users/pcorey/al-erlang"

And finally, we can build our upgrade release:


MIX_ENV=prod mix do compile, phoenix.digest, release --env=prod --upgrade

Hot Deploying Our Upgrade

Now that we’ve successfully built our upgrade release, we can deploy it to our production server.

The first thing we’ll need to do is ssh into our production server and create a 0.0.2 folder in our application’s releases directory:


mkdir ~/releases/0.0.2/

Next, we’ll hop back over to our development server and copy our newly built release tarball into that new directory:


scp -i /hello_distillery.pem \
    _build/prod/rel/hello_distillery/releases/0.0.2/hello_distillery.tar.gz \
    ec2-user@ec2-...amazonaws.com:/home/ec2-user/releases/0.0.2/ \
    hello_distillery.tar.gz

Lastly, we’ll switch back to our production server and run the upgrade command, passing in the new version of our application that we just uploaded:


./bin/hello_distillery upgrade 0.0.2

If everything went well, we should be able to refresh our application in the browser and see our "Version 0.0.2!" message!

Final Thoughts

Minus a few burs and rough edges, Distillery is a fantastic tool for building and deploying Elixir/Phoenix applications.

I imagine that all of the sticking points I encountered can be smoothed out with a combination of well-designed build scripts and building releases on a machine with the same architecture as the production environment.

Looking to the future, it looks like edeliver does exactly that. Expect to see an article in the next few weeks about simplifying the deployment process with edeliver!