Vault

Overview

Kylo uses HashiCorp Vault to securely store secret information, such as user credentials to databases, S3 buckets etc. For detailed information about Vault review HashiCorp Vault Documentation, while here we will show how Kylo integrates with Vault. Kylo integration with Vault is implemented as a Kylo plugin. This allows to easily replace Vault with any other technology. It is a required plugin unless replaced with other implementation. Plugin can be found at kylo-services/plugins/kylo-catalog-credential-vault-<version>.jar.

Installation

If you already use Vault in your organisation you can skip to Kylo Configuration, otherwise use either of two options to install Vault with our scripts:

After installation you can find Vault at following default locations:

  • /opt/vault/current installation directory
  • /opt/vault/data encrypted data storage location
  • /opt/vault/current/conf configuration location
  • /var/log/vault vault log and vault unseal log

Access to all of these directories should be limited to security personnel. Note that Kylo installation scripts adhere to this security recommendation and install Vault with least permissive privileges where only “vault” user is allowed access to Vault.

Service

Service can be found at /etc/init.d/vault. You can use following commands to interact:

# to start and automatically unseal
service vault start

# to start sealed and unseal manually
service vault run

service vault status
service vault stop
service vault restart

Vault Configuration

By default configuration is stored in /opt/vault/current/conf. This directory contains

  • vault.conf
  • vault.init
  • vault-cert.pem
  • vault-key.pem
  • ca-cert.pem

vault.conf

Configures Vault parameters, such as storage locations, transport protocols, memory lock etc. For details see HashiCorp Vault Configuration. Kylo installation scripts configure Vault with self-signed SSL certificates and with memory lock turned off, because memory lock is not supported on all operating systems. For production environments it is recommended to turn memory lock on, e.g. disable_mlock=false. If Vault doesn’t start with Failed to lock memory: cannot allocate memory set disable_mlock=true. Consider installing Vault on another OS if memory lock is not supported on your OS.

vault.init

Contains unseal keys and root key generated by standard vault operator init output. Vault Service uses this file to automatically unseal Vault when started with service vault start. For maximum security unseal keys should be securely distributed to designated security personnel and this file should be securely destroyed with shred after Vault installation. If this file is destroyed Vault Service will not be able to automatically unseal Vault. In this case use service vault run to start Vault sealed and unseal Vault manually.

vault-key.pem

Vault private key which is used to setup SSL. This file is referenced by vault.conf

vault-cert.pem

Vault certificate. Used by Vault to identify itself. Signed by CA certificate ca-cert.pem. Imported to Kylo’s truststore /opt/kylo/ssl/kylo-vault-truststore.jks. Referenced by vault.conf

ca-cert.pem

CA certificate which signed Vault certificate vault-cert.pem. CA certificate is used to initialise and unseal Vault. Referenced by init.sh, setup.sh and unseal.sh scripts in /opt/vault/current/conf directory.

Kylo Configuration

Find following Vault properties in kylo-services/conf/application.properties:

Connection

Property Default Value
vault.scheme https
vault.host localhost
vault.port 8200

Path in Vault

Kylo needs to write data to Vault. Update this property to change where data is stored. Kylo token or certificate used for authentication with Vault will need to have read and write access to this path.

Property Default Value
vault.root secrets/kylo

Authentication

By default Kylo is configured to use client certificate authentication with Vault, but it also supports simple token authentication. Only one of these authentication methods should be used at a time. Certificate authentication will take precedence over token authentication when both are configured. Note that both token and cert need to have read and write access to path defined by vault.root property mentioned previously. Access to token or keystore implies access to secrets stored in Vault by Kylo, therefore care should be taken to limit access to either of these items to only required personnel. SSL configuration is located outside of kylo-services directory in /opt/kylo/ssl to avoid it being overwritten when Kylo is upgraded.

Property Default Value
vault.keyStoreDirectory /opt/kylo/ssl
vault.keyStoreName kylo-vault-keystore.jks
vault.keyStorePassword no default value
vault.trustStoreDirectory /opt/kylo/ssl
vault.trustStoreName kylo-vault-truststore.jks
vault.trustStorePassword no default value
vault.token no default value

Vault Operations

Setup client environment:

vi /opt/vault/current/bin/env.sh
    #! /bin/sh
    export VAULT_TOKEN=<insert root token here from /opt/vault/current/conf/vault.init>
    export VAULT_ADDR=https://localhost:8200
    export VAULT_CAPATH="/opt/vault/current/conf/ca-cert.pem"
source env.sh

Once client environment is set up, here are most common Vault operations you can do, refer to HashiCorp Vault CLI for full list:

# List keys on path
/opt/vault/current/bin/vault list secret
/opt/vault/current/bin/vault list secret/kylo
/opt/vault/current/bin/vault list secret/kylo/catalog
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource/<data-source-name>
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource/<data-source-name>/users
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource/<data-source-name>/groups

# Read values of keys
/opt/vault/current/bin/vault read secret/kylo/catalog/datasource/<data-source-name>/defaults
/opt/vault/current/bin/vault read secret/kylo/catalog/datasource/<data-source-name>/users/<user-name>
/opt/vault/current/bin/vault read secret/kylo/catalog/datasource/<data-source-name>/groups/<group-name>

# Write secrets
/opt/vault/current/bin/vault write secret/kylo/catalog/datasource/<data-source-name>/users/<user-name> options=@user-options.json
/opt/vault/current/bin/vault write secret/kylo/catalog/datasource/<data-source-name>/groups/<group-name> options=@group-options.json index=1