OpenSSL the Ansible vault... using PBKDF2

Ansible is a popular open-source software platform for configuring and managing computers. It helps sysadmins to provision new servers in a reliable and repeatable way, and helps developers who want to push their code as fast as possible. It takes scripts (playbooks) as input, which a lot of people can and do share with each other. The beauty of open source. Playbooks can contain sensitive data like passwords and SSL keys - stuff that you don't want to share, or incidentally upload to GitHub.

Last year Ansible added a tool to its arsenal to easily encrypt structured datafiles (containing sensitive data), called Ansible Vault. You can specify a key or keyfile when running a playbook, which decrypts the data on-the-fly. Encrypted data can still be edited

I love it when people make it easier to use encryption. The easier it becomes, the more people will use it, the safer everybody will be.

Another beauty of open source is that you can inspect the code. And modify it! I wanted to be able to encrypt and decrypt the data where/when you cannot use Ansible vault, by using other tools and languages like OpenSSL and Bash script.

Under the hood Ansible vault uses open and known encryption standards, through the Python PyCrypto library. All information is encrypted using the AES cipher in counter (CTR) mode with a key size of 256 bits [1].

A user enters a password, which, together with a salt, serves as input for PBKDF2, a well-known keystretching function. Here, SHA 256 is being used as hash-based pseudorandom function for PBKDF2.

After 10000 iterations PBKDF2 spews out three 'keys': One 256 bit key to encrypt the actual data, one 256 bit key to create a keyed hash (again using SHA 256) over the data, and one 128 bit nonce (initialization vector) [2].

The 10000 iterations is a nice big number to make it harder for attackers to bruteforce the key.

When you use Ansible vault on a plaintext file the contents are overwritten with the encrypted contents. The first line contains a header of three fields separated by colons: A header containing the magic string $ANSIBLE_VAULT, the Vault version number (as of this writing 1.1), and the cipher being used (AES256). The next huge string (truncated on 80 columns) contains a hexadecimal representation of three variables. Each of those variables is also hexadecimal encoded [3]. The three values are a salt, the HMAC and the encrypted data.

To be able to decrypt this data using OpenSSL you first need to stretch a password using PBKDF2 using 10000 iterations and SHA 256. As the stock version of OpenSSL doesn't have this functionality exposed (yet) in the application I created a patch (loosely inspired on an earlier rejected OpenSSL patch). It lets you choose the number of iterations, the pseudo random function and the number of keys. Using this patch you can compile openssl, and use it to create two 256 bit keys and a 128 bit initialization vector from a given salt and password. That's almost all that's needed to convert Ansible vault encrypted files into plaintext, and vice versa: AES-256-CTR and SHA 256 are already supported.

Using the following command, three keys can be derived from password "password" and salt "salt":

openssl enc -pbkdf2 -pass pass:password -S 73616c74 -md sha256 -c 10000 -dklen 80

Almost... all that's missing is a Bash script wrapping everything nicely up. Until next time!

You can find the PBKDF2 patch for openssl on my GitHub repository.