Using AWS KMS to manage secrets in your Infrastructure

The new KMS service provides HSM-style key management that is both inexpensive and easy to use via a web service API. First, we'll look at what KMS is and how you can use it to manage encryption keys. Then, we'll look at credstash, a simple system that uses KMS and DynamoDB to safely store, distribute, and manage credentials in the cloud.

What is KMS?

Basic functionality

Other than the excellent GenerateRandom API call (which you should check out for seeding your PRNGs), KMS is composed of a set of API operations for creating, managing, and using a relatively small set of encryption keys, called Customer Master Keys (here, "master keys"). There are a bunch of operations for managing grants and policy around who can use which keys for what operations, but the fundamental operations in KMS are CreateKey, Encrypt, and Decrypt. CreateKey will generate a key in the KMS service that will never leave the KMS service. Once you create a key in KMS, you can disable it, you can set permissions on who can use it, you can alias it, but you cannot export it. In order to use the keys for cryptography, you use the Encrypt and Decrypt API calls. This is the core security value proposition in KMS: no one can run off with the keys.

In fact, this is the same model that is used by expensive Hardware Security Modules (HSMs): you generate a key in the device; once it's generated, it never leaves. Instead, you send the data to encrypt/decrypt to the device and say "encrypt this blob with key foo," and the HSM returns the resulting ciphertext or plaintext.

In the case of KMS, this is done using the Encrypt API operation. You pass the service the handle of the KMS master key that you want to use for encryption, along with up to 4KB of data to encrypt. You get back a blob containing ciphertext and a key reference, which can later be passed to KMS's Decrypt operation, which will return the plaintext. Again, there are many useful operations for managing key audit, policy, and grants. But the service really boils down to creating a key, then using it to encrypt and decrypt 4KB blobs of data.

4KB?! How am I supposed to use KMS for encryption?

There are lots of things that we might want to encrypt that are larger than 4KB. In fact, the new Relational Database Service (RDS) Encryption uses KMS to manage the keys used to encrypt entire databases! In order to encrypt arbitrary data and still keep our keys safe in KMS, we use a technique called Envelope Encryption.

Here's how it works:

1. Locally generate a random encryption key (or use the excellent GenerateDataKey operation). We will call this your data key.

2. Use the data key to encrypt your data.

3. Use KMS to encrypt your data key with one of your master keys. This is called key wrapping. The encrypted data key is now a "wrapped key."

4. Discard the plaintext datakey.

You can now safely store the encrypted data and the wrapped key. You can even store them next to each other in a database, on your filesystem, etc. Note: this is not "encraption" (the practice of storing a key next to the data that it protects), because without access to the master key that wraps the data key, the data key is useless. It is an opaque blob.

To decrypt data, you simply:

1. Fetch the wrapped data key and the encrypted data.

2. Use KMS to decrypt the wrapped data key.

3. Use the decrypted data key to decrypt the encrypted data.

It should now be obvious why KMS refers to the its keys as "Master Keys"–they are not used to encrypt data, but are instead used to encrypt keys that encrypt data. A single master key can protect many keys, and, in fact, every independent datum in your system can have its own unique data key.

Now that we understand how to use KMS to encrypt things, let's look at a practical example using KMS and DynamoDB to manage and distribute credentials to systems.

Credstash: using KMS and DynamoDB to manage credentials

Software systems often need access to some shared credential. For example, your web application needs access to a database password or an API key for a third party service. CredStash is a very simple, easy-to-use credential management and distribution system that uses:

• AWS Key Management System (KMS) for key wrapping and master key storage, and

• DynamoDB for credential storage and sharing.

Check out the code at https://github.com/fugue/credstash and follow the directions to set up credstash. You will end up with a master key in KMS and a DynamoDB table to hold encrypted credentials and wrapped data keys.

Whenever you want to store/share a credential, such as a database password, you simply run

$ credstash put [credential-name] [credential-value]

For example, credstash put myapp.db.prod supersecretpassword1234.

Credstash will:

• go to the KMS and generate a unique data encryption key, which is wrapped by the master key;

• use the data encryption key to encrypt the credential value; and

• store the encrypted credential, along with the wrapped (encrypted) data encryption key in the credential store in DynamoDB.

When you want to fetch a credential, perhaps as part of the bootstrap process on your web-server, you simply do

$ credstash get [credential-name]

For example, export DBPASSWORD=$(credstash get myapp.db.prod).

When you run get, credstash will:

• go and fetch the encrypted credential and the wrapped encryption key from the credential store (DynamoDB);

• send the wrapped encryption key to KMS, where it is decrypted with the master key;

• use the decrypted data encryption key to decrypt the credential; and

• print the credential to stdout, so you can use it in scripts or assign environment variables to it.

The README file in the credstash repo has lots of additional information and some notes about the actual operational security of this setup, so you should check it out to learn more.

Wrapping up and Learning More

KMS is a great new service that makes it easier than ever to safely store and manage secrets across your infrastructure. When used with other AWS services, like DynamoDB, you can add more security and control than before to high-churn immutable infrastructure (i.e., what Fugue creates).

You can learn lots more about KMS and access detailed API docs at:

• http://aws.amazon.com/kms/

• http://docs.aws.amazon.com/kms/latest/developerguide/overview.html

• http://docs.aws.amazon.com/kms/latest/APIReference/Welcome.html

And check out credstash at https://github.com/fugue/credstash.