Depending on a company's security paradigm, the admin team members share a joint account or assign admin rights to personal accounts. In the first case, each member must know the account's shared password. This sometimes leads to unfortunate situations when an established employee leaves the team or even the company. In the second case, the administrators handle all their daily tasks with a user account that has rights far above normal levels.

Additionally, the use of cloud infrastructure along with dynamic resource management is geared to the actual needs of the respective services. In high-utilization situations, instances added at short notice carry out the necessary calculations. User accounts for databases or APIs in such dynamic environments typically share secret keys for encrypting communication or access to shared filesystems. Each password is then stored in a widely available configuration file that is used when setting up the instance for configuration. Alternatively, the shared admin password is used to log in and the remaining configurations are semiautomatic.

In both scenarios, different instances use the same secrets to perform the work to be done. On the one hand, this leads to accountability issues for the work performed – for example, traceability: Who logged into the administrative account on a system and when? If cloud instances have the same access credentials for database access, it is impossible to determine unambiguously which instance is responsible for errors or malicious actions. On the other hand, existing structures can change after distributing the secrets. This results in new problems: What happens to the old password? Do you need to equip all systems with a new password or is it sufficient to do this for future instances?

Installation and Configuration

HashiCorp [1] is known in the field of dynamic services, mainly for its Vagrant and Packer tools. HashiCorp Vault [2] gives you access to shared resources and services, cryptographic keys, and dynamic access to user accounts. Vault is developed as an open source client-server application, primarily in the Go programming language. Installing it from the GitHub source is a trouble-free process; the vendor provides ready-to-run packages for popular operating systems. After the installation, the program should be located in the execution path to simplify its use.

Vault manages the secrets in a separate server process. This can be controlled with the well-documented HTTP API. The Vault client mediates between command-line arguments and the API on the server. However, before you can safely store the first secret, you first need to configure the server. I am assuming a central service on a publicly accessible server system. Access rules in the packet filter may increase security. However, you should choose these such that access to a shared cloud infrastructure is possible. A configured subdomain (e.g., vault.example.com ) allows for easy configuration of the client. You need to secure the connection between the Vault client and server by SSL; for this, you need a certificate for the relevant subdomain.

If you have set up the subdomain and the SSL certificate is available, initial configuration of the server can take place. You create this in the specially developed HashiCorp Configuration Language (HCL). Because HCL is compatible with JSON, the format is quickly accessible. The vault.example.com.config configuration file might look like Listing 1.

backend "file" {
  path = "/opt/vault/store.db"
}
listener "tcp" {
  address = "0.0.0.0:8200"
  tls_cert_file = "/opt/vault/server.pem"
  tls_key_file = "/opt/vault/key.pem"
  tls_min_version = "tls12"
}

First, configure the server's back end, the place where Vault safely stores secrets. Here, you can choose different targets that can be used by Amazon S3 with a MySQL database or the filesystem up to the memory, as needed. For my first attempt, I am choosing the back-end file, which stores the file on the local hard disk.

The only currently supported listener configuration is TCP. In this configuration block, address configures the local address and port to be used by the server. The tls_cert_file and tls_cert_key options define the location of the SSL certificate, as well as the associated secret key. To optimize privacy, you should not allow TLS versions before 1.2. To do this, use the tls_min_version option with the tls12 value.

You are probably wondering whether storing secrets in a file on the local hard drive or on Amazon S3 can be secure. The answer is yes, because Vault stores the secrets entrusted to it in encrypted form. The key is stored in system memory. To ensure this and to avoid swapping the password out to a hard disk, Vault uses the Linux mlock kernel function, which is reserved for root, so you need to launch the Vault server with the following command:

$ sudo vault server --config vault.example.com.config

Setting Up the Server

After the first launch of the server, you must prepare it for use. This is done with the same Vault program you used to launch the server. For convenience, it makes sense to store the address of the Vault server in an environment variable. On Linux, you can do this for the current terminal session as follows:

$ export VAULT_ADDR="https://vault.example.com:8200"

The following command ensures that you are able to access the server and that it has not been previously initialized:

$ vault status

The result of the status request should be * server is not yet initialized . Now you can initialize the server. During this process, Vault generates the key used to protect the secrets. Since the key is never written to the hard disk, you have to input it each time you reboot the server to unlock the vault. Yet another feature increases the security of the Vault server: The key itself is never shown after initialization. Vault uses Shamir's Secret Sharing procedure developed by Adi Shamir.

Vault divides the key used in the default configuration into five subkeys, which together describe the master key. After restarting the server, Vault requires at least three of the five subkeys to generate the master key again. Therefore, responsibility for the master key can be split between all members of a team. You can initialize the vault with the command:

$ vault init

In addition to the five generated subkeys, the dialog also shows the root token (Figure 1), which authorizes your actions on the server. Distribute the subkey and make a note of the displayed token.

Figure 1: The subkey and root token after initialization. Of the five keys, only three are needed to open the vault.

If you now check the status of the vault, you will receive information about the state, which is still sealed, as indicated by sealed: true displayed in the status output. The next step is to open the vault. As previously mentioned, three subkeys are necessary. Vault prompts you to enter your subkey after you run the command:

$ vault unseal

If the required subkeys are made available by commanding unseal multiple times, Vault generates the master key, and the state of the vault is now marked as unsealed . From this point, you can store and retrieve secrets. Before using Vault, you need to know which existing back ends are used for storage, authentication, and auditing.

Customization via Vault Back Ends

Vault lets you adapt to individual conditions with different back ends. In doing so, vault distinguishes between storage, authentication, and auditing back ends.

Storage back ends manage the secrets, the dynamic generation of time-limited access, and the corresponding access to it. You need to integrate the required back ends with the vault before you can use them. A back end is similar to a virtual filesystem. The supported features are read, write, and delete. Secrets are stored using paths. One secret can be stored in each path. You can control access to individual paths using rule sets or policies. An example of this is shown later. First, the following command shows the currently integrated storage back ends:

$ vault mounts

When you first start, the generic, system, and cubbyhole back ends are already mounted. The generic back end is storage for arbitrary secrets. The system back end stores the server's configuration settings. These also include the previously mentioned policies for access control of the vault's seal state. In the cubbyhole back end, Vault manages the secrets for each user token. Access is not regulated using policies, but with authentication against the server. You can store secrets that you do not want to share in this storage area (Figure 2).

Figure 2: Access to the secret is impossible without authorization.

To learn more about the use of the server, first store a simple password with the integrated generic back end under secret/ and then read it right back:

$ vault write /secret/secret value="secret"
$ vault read /secret/secret

It is important to note here that although Vault encrypts the value stored in value, the path remains unencrypted. You should also avoid using the path for storing confidential information. For an overview of the paths used, simply use the command:

$ vault list /secret

The list shows the paths and folders used at the requested secret folder level. In this case, secrets and subfolders can have the same name. As with the generic back end, you can also access the cubbyhole back end. However, the difference is that the command list actually only shows the paths used.

In addition to those already mentioned, other back ends have different purposes, including, for example, back ends for dynamic access to SSH or MySQL servers, which are particularly useful in cloud environments. Before setting up these two, I'll take a closer look at the authentication and audit back ends in Vault.