Video: create custom infrastructure images with Packer

January 30, 2018 - by Alexandra White

There are a number of ways to deploy custom applications on Triton. Triton provides multiple hardware virtual machine (HVM) and infrastructure images (just run triton images to see what we offer) to meet all of your various application needs.

Although those images can be deployed into containers which can be customized individually, that extra work can be cumbersome and difficult to replicate. With Packer by Hashicorp, it's quick and easy to automate the customization of a Triton image which you can easily deploy into multiple instances and update as needed.

Since originally writing about this topic in November 2017, I've learned a number of new best practices. In this post, I'll share a video overview for creating a Packer image, as well as the step-by-step instructions for deploying said image on Triton. If you want a longer, more in-depth video experience, sign up to watch our on-demand webinar.

If you haven't already, be sure to sign up for Triton and set up the triton CLI tool.

Watch to learn

Watch our video to see how simple it is to create custom infrastructure images with Packer on Triton.

What is Packer?

Packer by Hashicorp

Packer makes it simple to automate the creation of any type of machine image from a single configuration file. With the help of scripts to install and configure software within your image, Packer allows you to run multiple containers without needing to tweak each individual one.

Packer is platform-agnostic. That means if you've got existing Packer templates which build images for other cloud providers, it's easy to adapt them to build those same images on Triton.

There are a number of terms to know for Packer, but here are the highlights:

  • Template: templates are configuration files for Packer images.
  • Builders: builders create machine images for individual platforms. Triton is a builder, using CloudAPI to create the image. The builder launches a temporary VM based on the template, runs any provisioning necessary, creates a reusable image, and then destroys the VM. This builder does not manage images; you must use or delete it with CloudAPI outside of Packer.
  • Provisioners: provisioners install and configure software within a running machine prior to the machine becoming a static image. They perform the work which customizes Triton images to contain software including installing packages, creating users, and downloading application code.
  • Artifacts: artifacts are the result of a single build, including a set of IDs or files which represent the final machine image. Every builder produces a single artifact. For the Triton builder, the artifact is the new image ID.

You can read more about taking your Packer images and adding application infrastructure with Terraform to further automate your deployments.

Installing Packer

Before you can create an image with Packer, you must install it. You have a few options:

The unzipped, downloaded file should live in Unix systems in the ~/packer directory or /usr/local/packer. For Windows, the location of the downloaded file doesn't matter.

Afterwards, you must set the PATH for your system.

Set PATH for macOS or Linux

Edit your bash profile (which may look like .bash_profile or .bashrc) to add the PATH and other environment variables, just as you've done for CloudAPI environment variables.

Add the following content:

export PATH=$PATH:/usr/local/packer

If you've installed Packer in a different directory, you must modify /usr/local/packer to reflect the correct information.

Set PATH for Windows

You can set the PATH by going to the Control Panel -> System -> Advanced System Settings. Under Environment Variables, scroll until you find PATH. Edit accordingly and be sure to include a semicolon at the end of any previously set paths. For example:

c:\path\to\example1;c:\path\to\packer

Create the template

We'll be creating a template for a web application that I've created, dubbed the Happiness Randomizer. In that repository, you'll also find folders related to the other blog posts in this Packer and Terraform series.

Before adding Packer, you'll need to either download and unzip the Happiness Randomizer or fork the application on GitHub. This repository already has all of the completed files necessary to build an image right away.

I'll tell you step-by-step how to add the two files necessary to create a Packer image: a shell script to modify the file structure of an Nginx image and a JSON configuration file.

If you'd like to start from scratch following these instructions, remove the directories.sh and happy-image.json files from your application folder.

Adding the shell script

Get into the directory of your local Happy Randomizer application. You can use whatever text-editor you're most comfortable with to create and edit the shell script and the following configuration files. Let's call our script directories.sh:

$ cd ~/happy-randomizer/
$ touch directories.sh

Our shell script is going to create the directories which match our application, for CSS, JavaScript, and other resources.

mkdir -p /usr/share/nginx/html/css
mkdir -p /usr/share/nginx/html/js
mkdir -p /usr/share/nginx/html/resources
mkdir -p /usr/share/nginx/html/resources/images

That's it. Those four lines create the necessary directories. We'll talk more about why this is important in our later step on provisioning.

Adding a Packer configuration file

The configuration file will determine the type of image we're building, written in JSON. Let's call ours happy-image.json.

$ touch happy-image.json

The contents of our file will begin with Triton environment variables for your account, followed by the builders and provisioners.

NOTE: Though it is possible to proceed without setting up environment variables by replacing the contents with the corresponding information, we do not advise you do so. It is a best practice to store all important keys locally instead of tying it to your application files.

Variables are particularly useful when it comes to sensitive information, such as your account login and SSH key fingerprint. These are the same variables used for CloudAPI and other Triton tools. To access these variables, it's important to set the Triton environment before building your image.

{
  "variables": {
      "triton_url": "{{env `SDC_URL`}}",
      "triton_account": "{{env `SDC_ACCOUNT`}}",
      "triton_key_id": "{{env `SDC_KEY_ID`}}"
  },

We'll be referencing those variables within our builder. As defined above, the Triton builder will build an image with the help of CloudAPI. In our builder, we will define the keys to accessing Triton (those above variables), SSH access, the source image on which to build our application, details about our new image.

Since the Happiness Randomizer is a web-based application, Nginx will be a great source image along with the smallest (and cheapest) available package, g4-highcpu-128M. To view all available Triton images, execute triton images.

To get the "source_machine_image" (i.e. the image ID), execute triton image get along with the image name and grep to get the specific piece of information we're after.

$ triton image get nginx | grep "id"
"id": "2d7ec6d2-f100-11e5-84d7-77c57246a64a"

Our custom image will be called happy_randomizer.

"builders": [
   {
      "type": "triton",
      "triton_url": "{{user `triton_url`}}",
      "triton_account": "{{user `triton_account`}}",
      "triton_key_id": "{{user `triton_key_id`}}",

      "source_machine_name": "nginx",
      "source_machine_package": "g4-highcpu-128M",
      "source_machine_image": "2d7ec6d2-f100-11e5-84d7-77c57246a64a",

      "ssh_username": "root",

      "image_name": "happy_randomizer",
      "image_version": "1.0.0",
      "image_tags": {
         "Project": "Happy-Randomizer"
      }
   }
],

NOTE: For your SSH key to be usable, it must be available through the ssh-agent.

Once our builder is in place, we can add provisioners to customize it. We'll be using two types of provisioners: shell and file.

The shell provisioner will call to the shell script, which ensures that the Nginx image matches the setup of our local application.

The file provisioners upload all of our necessary application materials. This can be either a single file, such as index.html, or an entire directory, such as css/.

"provisioners": [
   {
      "type": "shell",
      "script": "directories.sh"
   },
   {
      "type": "file",
      "source": "favicon.ico",
      "destination": "/usr/share/nginx/html/"
   },
   {
      "type": "file",
      "source": "index.html",
      "destination": "/usr/share/nginx/html/"
   },
   {
      "type": "file",
      "source": "css/",
      "destination": "/usr/share/nginx/html/css/"
   },
   {
      "type": "file",
      "source": "js/",
      "destination": "/usr/share/nginx/html/js/"
   },
   {
      "type": "file",
      "source": "resources/",
      "destination": "/usr/share/nginx/html/resources/"
   }
]
}

All together this creates a single JSON file to build your Packer image.

View the full Packer configuration file on GitHub.

Build the image

Once the template is completed, you're ready to build the image. But before we build it, it's important to validate the template and ensure that the JSON syntax and configuration values are correct.

Run packer validate with the name of the Packer template:

$ packer validate happy-image.json
Template validated successfully.

NOTE: If the template validation was not successful, that means there is an error in your configuration file. The validate command should tell you where to find the error.

If you receive the success notice, you can move on to build the Happiness Randomizer image. To do so, we'll be executing packer build with the name of our template file:

$ packer build happy-image.json
triton output will be in this color.

==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
==> triton: Provisioning with shell script: directories.sh
==> triton: Uploading favicon.ico => /usr/share/nginx/html/
==> triton: Uploading index.html => /usr/share/nginx/html/
==> triton: Uploading css/ => /usr/share/nginx/html/css/
==> triton: Uploading js/ => /usr/share/nginx/html/js/
==> triton: Uploading resources/ => /usr/share/nginx/html/resources/
==> triton: Stopping source machine (6163c9e1-0ee6-eccb-c8bb-9f4369c73bb0)...
==> triton: Waiting for source machine to stop (6163c9e1-0ee6-eccb-c8bb-9f4369c73bb0)...
==> triton: Creating image from source machine...
==> triton: Waiting for image to become available...
==> triton: Deleting source machine...
==> triton: Waiting for source machine to be deleted...
Build 'triton' finished.

==> Builds finished. The artifacts of successful builds are:
--> triton: Image was created: c7da3619-5b1d-4fc7-bde6-503a3f8450b1

At the end of the output, you'll see a list of the artifacts from your build. For us, that's a single Triton image with the ID c7da3619-5b1d-4fc7-bde6-503a3f8450b1. Check out your newly created image:

$ triton images | grep "happy_randomizer"
c7da3619  happy_randomizer          1.0.0       I      linux    lx-dataset    2018-01-29

The happy_randomizer image is now available for you to deploy a container.

Deploy the container

After the image has been built on Triton, you can quickly spin up a new container with the help of triton. For more on how to use the creation command, read our documentation on instance creation.

$ triton instance create -w --name=happy-randomizer happy_randomizer g4-highcpu-128M
Creating instance happy-randomizer (9181422e-3e14-4409-a1fb-ed23d9067769, happy_randomizer@1.0.0)
Created instance happy-randomizer (9181422e-3e14-4409-a1fb-ed23d9067769) in 38s

Let's check out our new application and make sure everything is running as expected. To do so, we'll need the IP address:

$ triton inst ip happy-randomizer
165.225.165.140

Wrapping up

Now that you've created an infrastructure image, you're ready to start deploying custom applications. Read our blog posts on Terraform to further automate that process: