A Bit More About the New Joyent Cloud API

January 03, 2012 - by joyentjames

As part of SmartDataCenter 6.5, we've introduced something I'm pretty excited about sharing, which is an open signing protocol for HTTP. We're using it in what we call CloudAPI, which is the REST API that powers our customer portal, and is exposed to you to create your own applications. In this post I'll go over what the signing protocol is and why you should care, how SmartDataCenter is using it, and some ideas we have in mind for the future.

If you're not an RFC nerd, HTTP defines a "pluggable" framework for authentication so that any number of authentication mechanisms can be used, and goes on to define Basic Authentication. Basic Authentication is nothing more than sending your username and password across the wire, so it's not ideal from a security perspective. Following Basic Authentication, HTTP also goes on to define Digest Authentication, which involves sending a hashed form of your password across the wire, but it's hard to use and doesn't see much use these days.

However, for the best security properties, you really want to be leveraging public key cryptography to authenticate. HTTPS (well, really SSL) has a mode of allowing this called "client-auth"; basically your client has a certificate and private key, and in addition to validating the server's certificate, the server asks you (the client) to present your certificate. Sounds great, and does do what we want from a security perspective. However, it's plagued by poor usability in browsers, is more or less a programming nightmare (especially with "custom" CAs), and can incur significant server-side cost, since it's usually a load balancer (i.e., HTTPS termination point) that is handling the cryptography validation. All of this means that we want the benefits of public key crypto without the burden of client-auth SSL.

To solve this problem with SmartDataCenter, we're introducing the HTTP Signature authentication scheme, which is already released as open source. The HTTP Signature scheme allows for digital signatures to be leveraged as your credentials, and does not imply any particular key management scheme. It should be suitable for just about any REST API out there, so we're certainly hoping if you're an API vendor you take a look at it, but rather than jump into the technical details of the spec itself, I thought it would be interesting to show you some CLI candy, so lets walk through some examples.

Some Signing Candy

First, assuming you're a node.js user (and if you're not, you should be!), use npm to install the smartdc toolkit:

$ npm install smartdc -g

The smartdc module installs the CloudAPI SDK for node.js, which includes a full command line interface (CLI), and we're going to show off some of its features here.

To "configure" your CLI, just run the setup command:

$ sdc-setup https://us-west-1.api.joyentcloud.com

Username (login): (james) james

Password:

The following keys exist in SmartDataCenter:

 [1] id_rsa

Would you like to use an existing key? (yes) yes

If you set these environment variables, your life will be easier:

export SDC_CLI_URL=https://us-west-1.api.joyentcloud.com

export SDC_CLI_ACCOUNT=james

export SDC_CLI_KEY_ID=id_rsa

You'll additionally want to set SDC_CLI_IDENTITY to the full path location of the ssh

key on your system (try export SDC_CLI_IDENTITY=/Users/james/.ssh/id_rsa).

In my case, sdc-setup found an existing ssh key in SmartDataCenter, and I simply selected that; if I didn't already have one "on record" sdc-setup would have asked me to upload a new one. At any rate, sdc-setup gave me some environment variables to set, so I just put them in my ~/.bashrc file (sorry Windows users, smartdc's CLI isn't there yet).

Note that there's no more use of your password, and assuming you did set up those environment variables then you can simply run:

$ sdc-listmachines

[

{

"id": "30e8d834-7210-44f7-be58-7179396bfe20",

"name": "foo",

"type": "smartmachine",

"state": "running",

"dataset": "sdc:sdc:smartos:1.3.18",

"ips": [

"10.88.88.50"

],

"memory": 128,

"disk": 5120,

"metadata": {},

"created": "2011-09-15T04:12:53+00:00",

"updated": "2011-09-15T04:14:35+00:00"

}

]

The CLI just authenticated to SmartDataCenter using my SSH key. That means for interacting with SmartDataCenter you don't need to maintain any "extra" keys; in my case I can then go ahead and do this:

$ ssh -i ~/.ssh/id_rsa admin@10.88.88.50

Last login: Thu Sep 15 04:17:52 2011 from 10.88.88.1

[admin@foo ~]$ exit

While not technically part of the CLI, check this out:

$ sdc-createkey ~/.ssh/id_dsa.pub

{

"name": "id_dsa.pub",

"key": "ssh-dss AAAAB3Nza... james@Howl.local",

"created": "2011-09-15T04:19:10+00:00",

"updated": "2011-09-15T04:19:10+00:00"

}

$ ssh -i ~/.ssh/id_dsa admin@10.88.88.50

Last login: Thu Sep 15 04:18:05 2011 from 10.88.88.1

[admin@foo ~]$ exit

Oh yeah, SmartDataCenter has "live" key rotation of your SSH keys in running SmartMachines. Finally for the "candy" section, here's "one more thing":

$ ssh-add

Identity added: /Users/james/.ssh/id_rsa (/Users/james/.ssh/id_rsa)

Identity added: /Users/james/.ssh/id_dsa (/Users/james/.ssh/id_dsa)

$ ssh admin@10.88.88.50

Last login: Thu Sep 15 04:19:13 2011 from 10.88.88.1

[admin@foo ~]$ export SDC_CLI_URL=https://us-west-1.api.joyentcloud.com

[admin@foo ~]$ export SDC_CLI_ACCOUNT=mcavage

[admin@foo ~]$ export SDC_CLI_KEY_ID=id_rsa

[admin@foo ~]$ sdc-listmachines

[

{

"id": "30e8d834-7210-44f7-be58-7179396bfe20",

"name": "foo",

"type": "smartmachine",

"state": "running",

"dataset": "sdc:sdc:smartos:1.3.18",

"ips": [

"10.88.88.50"

],

"memory": 128,

"disk": 5120,

"metadata": {},

"created": "2011-09-15T04:12:53+00:00",

"updated": "2011-09-15T04:14:35+00:00"

}

]

Wow! So what we just did is use the OpenSSH ssh-agent program to stash our keys on your local host and the smartdc CLI is slick enough to interact with that (if it's available) instead of looking for a private key on the filesystem. Using that mechanism, you can actually SSH from box to box, without worrying about copying your keys around. Now you've actually got single sign on (SSO) happening between your hosts and SSO to the CloudAPI.

Some Technical Details

HTTP Signature is actually really straightforward, and if you've ever looked at OAuth, or WRAP, et al, this should be both familiar and much simpler. It's as simple as pointing out that you just have to sign the HTTP Date header, which accomplishes a few things:

  • We can securely authenticate you using your SSH private key.
  • Guarantees there at least some entropy in the string to be signed.
  • Allows us to prevent replay attacks on a coarse level by expiring requests after 5m.

As an example, suppose you had an HTTP request that looked like this:

GET /my/machines HTTP/1.1

Host: us-west-1.api.joyentcloud.com

Date: Mon, 12 Sep 2011 23:05:42 GMT

Accept: application/json

X-Api-Version: ~6.5

That's the "ListMachines" call availalble in CloudAPI. To add HTTP Signature authentication to that call, we'd add an Authorization header that looks like this:

Authorization: Signature keyId="/james/keys/id_rsa",algorithm="rsa-sha256" <signature>

Where <signature> is:

<Base64(rsa(sha256($Date)))>

Signature is the HTTP authentication scheme. keyId is telling the server what key identifier to validate the signature with. HTTP Signature does not imply any sort of key management scheme so it can work in the broadest possible scenarios, but SmartDataCenter uses the resource path to an SSH key under your account (more on this soon). algorithm tells the server how the client signed the request, and here we're using rsa-sha256. Finally <signature> would be the base64-encoded value of an RSA digital signature.

That's pretty much all there is to it at a high level for how this works.

What's next?

Clearly, we'd like to see this authentication mechanism get picked up; it should be simple and flexible enough to be used in a lot of places. While we've released an SDK for this in node.js, other languages like ruby and python would benefit from an implementation. Hopefully this quick primer got you excited about some of the innovation happening in SmartDataCenter, and we'd love to hear what you think.

Sign up Now For
Instant Cloud Access

Get Started