Autopilot Pattern Node.js in Docker

September 01, 2016 - by Wyatt Lyon Preul

The Autopilot Pattern makes it easy to build applications that can automatically configure themselves when you deploy them (making it easy to bring up copies of the application for testing and development), and reconfigure themselves when you scale them up and down with traffic or roll out updates. Today we're introducing two Node.js modules to make it even easier to build Node.js apps that use the Autopilot Pattern.

These two modules, consulite and piloted, are installable using npm and provide common functionality used by Autopilot Pattern applications, including interaction with Consul and reloading the configuration as the application topology changes. These modules are not required to implement the Autopilot Pattern, they just make doing so easier.

Let's look at these two modules in the context of our Autopilot Pattern workshop application.

ContainerPilot with Node.js

Before demonstrating the usefulness of the new modules, lets look at how ContainerPilot integrates with a Node.js application running in Docker. The following figure illustrates the points in an application's lifecycle that typically will need to be configured with ContainerPilot.

application lifecycle with ContainerPilot

As illustrated, the ContainerPilot configuration commonly used will send a SIGHUP signal to the Node.js application. This is done whenever there is a change to a backend that the application depends on. While ContainerPilot will notify the application about these dependent backend changes, it will not send the application the host and port information about these backends. This is a task left to the application developer to decide how to implement. The following figure demonstrates the typical flow of data between an application, ContainerPilot, and a service registry like Consul.

application flow with ContainerPilot and consul

The modules that will be demonstrated below are designed to solve the common problem illustrated above, which is the handling of the onChange event and the loading of the backend addresses from Consul. The onChange event occurs whenever there is a backend service change. This can be caused by a backend suddenly becoming unhealthy or from new backend addresses being registered with Consul. Either way, ContainerPilot will notify the Node.js application by executing the command thats configured.

Keeping a list of healthy backend addresses within the application that depends on them is an important aspect of the Autopilot Pattern. One of the implications of this design choice is that an application is less likely to make requests to a backend if the backend isn't healthy, as the application will know ahead of time of the health. It also means that load balancers are less useful to the deployment of an application and its backend services.

Consulite

One of the wonderful benefits of the Autopilot Pattern with the aid of ContainerPilot is that you don't need to register your application with a service discovery registrar. ContainerPilot manages that common task for you, as well as notifying your application when there are disruptions or changes to any services that your application is dependent on.

While you don't need to be concerned with the service registration and subscribing to notifications from the registrar, you will need to know how to talk to the registrar to locate the host and port of the services you depend on. Consulite helps in this regard by making it trivial to get the service information your application depends on. As the name denotes, the registry that consulite knows how to communicate with is Consul.

Retrieving registry entries

In order to help demonstrate the usefulness and simplicity of consulite lets look at an example. The example is borrowed from the Autopilot workshop, in particular the sales service.

const Consulite = require('consulite');
const Wreck = require('wreck');

Consulite.getService('customers', (err, host) => {
  if (err) {
    return console.error(err);
  }

  Wreck.get(`http://${host.address}:${host.port}/data`, (err, res, customers) => {
    // Handle error or respond with relevant customer data
  });
});

In the example above, if there aren't any cached addresses for the customers service then a request will be made to Consul to get all healthy service addresses. Typically there will be multiple backend addresses for a particular service. In order to help with this common setup, consulite will round-robin the list of addresses and return the next one. The logic for the round-robin currently starts with the first address that hasn't been consumed then the last address that was consumed. These features result in a module that helps reduce the burden of maintaining a list of addresses and communicating with Consul.

Handling registry changes

Consulite eases the burden associated with maintaining a cached list of healthy services and then updating the list when there is a service change. Previously, it was demonstrated that ContainerPilot is usually configured with a Node.js application to send a SIGHUP signal in order to notify it when a dependent service changes. Below is an example of handling the signal together with consulite in order to refresh the local cache of services in order to handle the configured onChange behavior in ContainerPilot.

process.on('SIGHUP', () => {
  Consulite.refreshService('customers', (err) => {
    if (err) {
      console.error(err);
    }
  });
});

Even though consulite makes it simpler for an application developer to adopt ContainerPilot, the responsibility of handling the process event as well as populating the initial list of service addresses still remains. To solve this other common problem we created piloted.

Piloted

Instead of expecting an application developer to implement the handling of the SIGHUP event and the initial loading of service addresses, we decided to do that work with the piloted module. All of the backends and services that an application depends on are already maintained in the containerpilot.json configuration file. Therefore, an application developer can relay this configuration to piloted and it will load and maintain a healthy list of backend address information for the application. Below is an example of how to send piloted the configuration details and retrieve address information for the 'customers' service.

Usage

const Piloted = require('piloted');
const Wreck = require('wreck');
const ContainerPilot = require('/etc/containerpilot.json');

Piloted.config(ContainerPilot, () => {
  const host = Piloted('customers');

  Wreck.get(`http://${host.address}:${host.port}/data`, (err, res, customers) => {
    // Handle error or respond with relevant customer data
  });
});

In the above example the ContainerPilot configuration file is loaded into piloted. After this, piloted will maintain a local cache of all of the healthy service addresses that an application depends on. In the event that there is a change to one of these services, piloted will update the list with the latest information from Consul.

Another benefit to this design is that modules running in the same process will have access to the application wide service addresses. Therefore, you only need to configure piloted once.

Summary

As you have seen, building Node.js applications that follow the Autopilot Pattern is becoming easier. If you would like to see the full working example application that the code snippets are borrowed from please look through the Autopilot Workshop. As always, we are looking for feedback and input, therefore please submit issues to any of our repositories.