Vault¶
Overview¶
Kylo uses HashiCorp Vault to securely store secret information, such as user credentials to databases, S3 buckets etc.
For detailed information about Vault review HashiCorp Vault Documentation, while here we will show how Kylo integrates with Vault.
Kylo integration with Vault is implemented as a Kylo plugin. This allows to easily replace Vault with any other technology.
It is a required plugin unless replaced with other implementation. Plugin can be found at kylo-services/plugins/kylo-catalog-credential-vault-<version>.jar
.
Installation¶
If you already use Vault in your organisation you can skip to Kylo Configuration, otherwise use either of two options to install Vault with our scripts:
- Setup Wizard Deployment Guide for local development and single node development
- Manual Vault Installation in Manual Deployment Guide for test and production environment
After installation you can find Vault at following default locations:
/opt/vault/current
installation directory/opt/vault/data
encrypted data storage location/opt/vault/current/conf
configuration location/var/log/vault
vault log and vault unseal log
Access to all of these directories should be limited to security personnel. Note that Kylo installation scripts adhere to this security recommendation and install Vault with least permissive privileges where only “vault” user is allowed access to Vault.
Service¶
Service can be found at /etc/init.d/vault
.
You can use following commands to interact:
# to start and automatically unseal service vault start # to start sealed and unseal manually service vault run service vault status service vault stop service vault restart
Vault Configuration¶
By default configuration is stored in /opt/vault/current/conf
. This directory contains
vault.conf
vault.init
vault-cert.pem
vault-key.pem
ca-cert.pem
vault.conf¶
Configures Vault parameters, such as storage locations, transport protocols, memory lock etc. For details see HashiCorp Vault Configuration.
Kylo installation scripts configure Vault with self-signed SSL certificates and with memory lock turned off, because memory lock is not supported on all operating systems.
For production environments it is recommended to turn memory lock on, e.g. disable_mlock=false
. If Vault doesn’t start with
Failed to lock memory: cannot allocate memory
set disable_mlock=true
.
Consider installing Vault on another OS if memory lock is not supported on your OS.
vault.init¶
Contains unseal keys and root key generated by standard vault operator init
output.
Vault Service uses this file to automatically unseal Vault when started with service vault start
. For maximum security unseal keys should be securely distributed to
designated security personnel and this file should be securely destroyed with shred
after Vault installation. If this file is destroyed Vault Service will not be able to automatically
unseal Vault. In this case use service vault run
to start Vault sealed and unseal Vault manually.
vault-key.pem¶
Vault private key which is used to setup SSL. This file is referenced by vault.conf
vault-cert.pem¶
Vault certificate. Used by Vault to identify itself. Signed by CA certificate ca-cert.pem
.
Imported to Kylo’s truststore /opt/kylo/ssl/kylo-vault-truststore.jks
. Referenced by vault.conf
ca-cert.pem¶
CA certificate which signed Vault certificate vault-cert.pem
. CA certificate is used to initialise and unseal Vault.
Referenced by init.sh
, setup.sh
and unseal.sh
scripts in /opt/vault/current/conf
directory.
Kylo Configuration¶
Find following Vault properties in kylo-services/conf/application.properties
:
Connection¶
Property | Default Value |
---|---|
vault.scheme | https |
vault.host | localhost |
vault.port | 8200 |
Path in Vault¶
Kylo needs to write data to Vault. Update this property to change where data is stored. Kylo token or certificate used for authentication with Vault will need to have read and write access to this path.
Property | Default Value |
---|---|
vault.root | secrets/kylo |
Authentication¶
By default Kylo is configured to use client certificate authentication with Vault, but it also supports simple token authentication.
Only one of these authentication methods should be used at a time. Certificate authentication will take precedence over token authentication when both are configured.
Note that both token and cert need to have read and write access to path defined by vault.root
property mentioned previously.
Access to token or keystore implies access to secrets stored in Vault by Kylo, therefore care should be taken to limit access to either of these items to only required personnel.
SSL configuration is located outside of kylo-services
directory in /opt/kylo/ssl
to avoid it being overwritten when Kylo is upgraded.
Property | Default Value |
---|---|
vault.keyStoreDirectory | /opt/kylo/ssl |
vault.keyStoreName | kylo-vault-keystore.jks |
vault.keyStorePassword | no default value |
vault.trustStoreDirectory | /opt/kylo/ssl |
vault.trustStoreName | kylo-vault-truststore.jks |
vault.trustStorePassword | no default value |
vault.token | no default value |
Vault Operations¶
Setup client environment:
vi /opt/vault/current/bin/env.sh
#! /bin/sh
export VAULT_TOKEN=<insert root token here from /opt/vault/current/conf/vault.init>
export VAULT_ADDR=https://localhost:8200
export VAULT_CAPATH="/opt/vault/current/conf/ca-cert.pem"
source env.sh
Once client environment is set up, here are most common Vault operations you can do, refer to HashiCorp Vault CLI for full list:
# List keys on path
/opt/vault/current/bin/vault list secret
/opt/vault/current/bin/vault list secret/kylo
/opt/vault/current/bin/vault list secret/kylo/catalog
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource/<data-source-name>
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource/<data-source-name>/users
/opt/vault/current/bin/vault list secret/kylo/catalog/datasource/<data-source-name>/groups
# Read values of keys
/opt/vault/current/bin/vault read secret/kylo/catalog/datasource/<data-source-name>/defaults
/opt/vault/current/bin/vault read secret/kylo/catalog/datasource/<data-source-name>/users/<user-name>
/opt/vault/current/bin/vault read secret/kylo/catalog/datasource/<data-source-name>/groups/<group-name>
# Write secrets
/opt/vault/current/bin/vault write secret/kylo/catalog/datasource/<data-source-name>/users/<user-name> options=@user-options.json
/opt/vault/current/bin/vault write secret/kylo/catalog/datasource/<data-source-name>/groups/<group-name> options=@group-options.json index=1