Locked Vault

During a recent project, I needed to test against an implementation of HashiCorp's Vault [1]. If you haven't come across Vault before, it's the industry leader for cloud-native secrets and certificate management that, by design, also plays very nicely with software in the cloud, such as Kubernetes. According to the Vault website, you can safely keep and fine-tune access for users and applications for "tokens, passwords, certificates, encryption keys … and other sensitive data using a UI, CLI, or HTTP API."

Working on security in DevOps teams, I had seen and used Vault, built by Terraform, on two or three different Amazon Web Services (AWS) estates but had never set up a lab myself on a local machine for testing. I wanted a simple solution that was slick and quickly reproducible, and, because it was for a lab environment, it didn't have to support high availability (HA). Thankfully, the clever Vault provides what is called dev server mode for just that requirement.

In this article, I deploy the venerable Vault locally to offer a centralized, flexible, and highly secure place to keep secrets and certificates; then, I'll run through a few of its core features to get to grips with the basics.

Is It Safe?

I'll dive straight into getting Vault up and running. I'm using Linux Mint 19, which sits atop Ubuntu 18.04. The set up should be mostly the same configuration, however, if you use another Linux distribution.

First, I'll navigate to the binary download page [2] and choose Linux 64-bit . I'm going to become the root user in this terminal and in a second terminal a little later on. After clicking the Download button, a ZIP file starts downloading. In the following command, you can see the version I'm using as I decompress the file:

$ unzip vault_1.3.2_linux_amd64.zip

The next thing to do is move the resulting vault binary file to the user path:

$ mv vault /usr/local/bin

To see if it works, I enter:

$ vault --version
Vault v1.3.2

One nice touch is that the vault command can use autocompletion, just like the Bash shell. To switch that feature on, enter,

$ vault -autocomplete-install

which will drop the line complete -C /usr/local/bin/vault vault into your .bashrc file. Close and reopen your terminal to load that up, or in your home directory run

$ . .bashrc

to load that file into your shell again. Try hitting the Tab key with the vault prompt on your command line now, and you'll see a number of options, which is a great way of checking syntax when you're learning Vault.

Under Lock and Key

As mentioned, I'm going to create a dev server for the purposes of running a lab environment. The Vault docs warn you that it shouldn't be used in production, isn't very secure, stores loads of things in memory, and automatically initializes (or "unseals") the strong locking or "sealing" mechanism at install time. You will also lose your secrets when you kill the terminal in which the dev server launches. Consider yourself suitably warned! For more information, read the "Concepts" box.

All commands that you send from the vault binary (think of it as a client) are encrypted by TLS as they travel over to the Vault server. To fire the dev server up, use the command:

$ vault server -dev

The uppermost part of the abbreviated output in Listing 1 shows what you need to know. Note that you will need to open another terminal to continue with other commands from this point on, because the server will keep that window open as it runs in the foreground.

==> Vault server configuration:
        Api Address: http://127.0.0.1:8200
                Cgo: disabled
    Cluster Address: https://127.0.0.1:8201
         Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
          Log Level: info
              Mlock: supported: true, enabled: false
      Recovery Mode: false
            Storage: inmem
            Version: Vault v1.3.2
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
You may need to set the following environment variable:
  $ export VAULT_ADDR='http://127.0.0.1:8200'
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: PrAbjZWj2NYYnV6Lh/FRpf5Nu5f2E5fwZZf95JQiLdo=
Root Token: s.pKGs7KObPOp0Cx2ybiUWuEIW
Development mode should NOT be used in production installations!

After opening up a second terminal, you can now interact with the Vault server. For good measure, I'll offer the new terminal the VAULT_ADDR environment variable with the command

$ export VAULT_ADDR="http://127.0.0.1:8200"

so that the shell can access the correct IP address and TCP port for the server.

The next step is using an environment variable for the root token, copied from the dev server start-up output in the first terminal, within the second terminal:

$ export VAULT_DEV_ROOT_TOKEN_ID="s.pKGs7KObPOp0Cx2ybiUWuEIW"

From the second terminal, you can check that the client – the vault binary that provides its command-line interface (CLI)) – can connect to the server (Listing 2). The output shows a successful response, which means that your client to server comms are working as hoped.

$ vault status
Key            Value
---            -----
Seal Type      shamir
Initialized    true
Sealed         false
Total Shares   1
Threshold      1
Version        1.3.2
Cluster Name   vault-cluster-e5878f26
Cluster ID     54ef0ecc-5f31-0926-e904-1bd94bc152ae
HA Enabled     false

Lockdown, Lockdown

With a working Vault instance, I will now look at a few ways to make use of it. The first, most obvious thing to try is saving a secret to the secured vault. This secret could be any bit of data – not just a password:

$ vault kv put secret/chrisbinnie liverpool=best
Key              Value
---              -----
created_time     2020-03-13T12:56:24.857347664Z
deletion_time    n/a
destroyed        false
version          1

That output looks promising; Vault has responded as expected. If that doesn't work correctly for you for some reason, then you've probably not exported the two environment variables correctly, so check them. Take note that the secret/chrisbinnie part in the command above relates to the path where the secret is stored. The "Secrets Engines" box has more information to help explain how the path is configured. As you might expect, the liverpool=best element is the key-value pair.

Now that you know how to write a secret, I'll try to retrieve one with a get command (Listing 3). You can verify that you are seeing the correct secret by the timestamp in the command's output; also, you can see the key-value pair data under the Data section.

$ vault kv get secret/chrisbinnie
====== Metadata ======
Key            Value
---            -----
created_time   2020-03-13T12:56:24.857347664Z
deletion_time  n/a
destroyed      false
version        1
====== Data ======
Key         Value
---         -----
liverpool   best

Simply replace the get in the command in Listing 3 with delete to remove a secret. Note that it's possible to write multiple secrets to the same path. At an uber-basic level, that's how secrets are written, retrieved, and deleted in Vault.

However, another very powerful feature concerns dynamic secrets [6]. According to the docs: "Dynamic secrets do not exist until they are read, so there is no risk of someone stealing them … dynamic secrets can be revoked immediately after use, minimizing the amount of time the secret existed."

This sophisticated functionality is perfect for an environment such as AWS, wherein an IAM role is given access via a role with a policy attached to it. Vault then temporarily offers access to permitted resources with the use of dynamic credentials that can be revoked or set to expire within customized time periods. Additionally, ephemeral, single-use tokens are powerful.