HashiCorp Vault

Set up Luna HSM

Set up HashiCorp Vault

Verify that the HSM is set up, initialized, provisioned, and ready for deployment.

This integration has been verified on the RHEL platform.

Refer to Luna HSM documentation for detailed steps on creating NTLS connection, initializing the partitions, and assigning various user roles.

Create a partition on the HSM that will be later used by HashiCorp Vault. If you are using a Luna Network HSM, register a client for the system and assign the client to the partition to create an NTLS connection. Initialize the Crypto Officer and Crypto User roles for the registered partition.

Run the following command to verify that the partition has been successfully registered and configured:

/usr/safenet/lunaclient/bin/lunacm

You should see the following output:

lunacm (64-bit) v10.2.0-111. Copyright (c) 2020 SafeNet. All rights reserved.
Available HSMs:
Slot Id ->              0 
Label ->                HashiCorp Vault
Serial Number ->        1280780175943
Model ->                LunaSA 7.4.0 
Firmware Version ->     7.4.0
Configuration ->        Luna User Partition With SO (PW) Key Export With Cloning Mode 
Slot Description ->     Net Token Slot
FM HW Status ->         FM Ready
Current Slot Id: 0

Enable partition policies 22 and 23 to allow activation and auto-activation, in case you are using PED-authenticated HSMs.

Refer to Luna HSM documentation for detailed steps on creating NTLS connection, initializing the partitions, and assigning various user roles.

Transfer the downloaded .zip file to your client workstation using pscp, scp, or other secure means

This integration has been certified on the RHEL platform.

Extract the .zip file into a directory on your client workstation.

Extract or untar the appropriate client package for your operating system. Do not extract to a new subdirectory; place the files in the client install directory.

tar -xvf cvclient-min.tar

Run the setenv script to create a new configuration file containing information required by the Luna Cloud HSM service.

source ./setenv

To add the configuration to an already installed UC client, use the –addcloudhsm option when running the setenv script.

Run the LunaCM utility and verify that the Cloud HSM service is listed.

If your organization requires non-FIPS algorithms for your operations, ensure that the Allow non-FIPS approved algorithms check box is checked. For more information, refer to Supported Mechanisms.

Download the HashiCorp Vault package.

Unzip the downloaded package in the working directory on your host machine. HashiCorp Vault operates as a single binary named vault.

Add the current working directory to the PATH so that vault is executable from any location.

Open a new terminal session and confirm the installation by checking the availability of the Vault binary.

Enable PKCS#11 seal

Configure entropy augmentation

Start HashiCorp Vault

Initialize HashiCorp Vault

Log in to HashiCorp Vault

Use secrets engine

Enable entropy augmentation

Rotate HashiCorp Vault keys

Create a configuration file named config.json for HashiCorp Vault.

Inside config.json, define the PKCS#11 seal along with storage and listener configurations:

# PKCS11 seal
seal "pkcs11" {
lib = "path to cryptoki library"
slot = "slot number"
pin = "partition password"
key_label = "HashiCorp"
hmac_key_label = "HashiCorp_hmac"
generate_key = "true"
} 
storage "file" {
path = "/tmp/vault"
}

# Addresses and ports on which Vault will respond to requests
listener "tcp" {
address     = "127.0.0.1:8200"
tls_disable = "true"
}
ui = true

Add a conditional block to include license path if you are using Vault version 1.10 or above:

# Conditional license path inclusion for Vault 1.10 and above:
template {
  data = <

This step is only required if you are using Vault version 1.10 or above.
{: .note }




###### Define RSA_PKCS_OEAP Mechanism




Define the `RSA_PKCS_OEAP` mechanism for master key wrapping in the `config.json` file, as illustrated in this example:

# PKCS11 seal
seal "pkcs11" {
lib = "path to cryptoki library"
slot = "slot number"
pin = "partition password"
key_label = "rsa_oaep_key"
mechanism = "0x0009"
rsa_oaep_hash = "sha256"
generate_key = "true"
}

storage "file" {
path = "/tmp/vault"
}

# Addresses and ports on which Vault will respond to requests
listener "tcp" {
address     = "127.0.0.1:8200"
tls_disable = "true"
}
ui = true


Ensure to replace placeholders like `"path to cryptoki library"` and `"slot number"` with the appropriate values for your environment.
{: .note }

Launch a new terminal session and export the Vault address.

export VAULT_ADDR='http://127.0.0.1:8200'

Check the version of Vault.

./vault --version

Check the status of the Vault.

./vault status

The command returns the following output:

Key                     Value
---                     -----
Recovery Seal Type      pkcs11
Initialized             false
Sealed                  true
Total Recovery Shares   0
Threshold               0
Unseal Progress         0/0
Unseal Nonce            n/a
Version                 n/a
HSM Enabled             false

Set the client timeout. This ensures that if Vault initialization takes longer than expected, an error will be displayed. Run the following command:

export VAULT_CLIENT_TIMEOUT=180

This command sets the timeout to 180 seconds (3 minutes). You must execute this command to ensure proper handling of initialization errors.

Initialize the Vault.

./vault operator init -recover-shares=1 -recovery=threshold=1

This command initializes the Vault with recovery shares and threshold set to 1. Adjust the values of -recover-shares and -recovery-threshold based on your specific requirements.

Securely safeguard the recovery key and initial root token values generated during this process by copying and storing them in a highly secure location. Refer to the logs in the initial terminal where the Vault server is running for important information:

2018-11-16T06:15:48.859-0500 [INFO] core: loaded wrapping token key 
2018-11-16T06:15:48.860-0500 [INFO] core: successfully mounted backend: type=kv path=secret/ 
…  
2018-11-16T06:15:48.952-0500 [INFO] core: root token generated  
…  
2018-11-16T06:15:49.031-0500 [INFO] core: vault is unsealed  
2018-11-16T06:15:49.032-0500 [INFO] core: post-unseal setup starting  
2018-11-16T06:15:49.153-0500 [INFO] core: loaded wrapping token key  
…  
2018-11-16T06:15:49.157-0500 [INFO] core: successfully unsealed with stored key(s): stored_keys_used=1 
2018-11-16T06:15:49.157-0500 [INFO] expiration: lease restore complete

Verify the keys generated on the partition by executing the following command in lunacm:

partition contents

Run the vault login command.

./vault login VAULT-TOKEN

Here, VAULT-TOKEN is the initial root token generated during Vault initialization.

Ensure that the Vault server has completed initialization and is now in an auto-unsealed state.

vault status

Access the Vault console through your web browser using the URL http://127.0.0.1:8200/ui, if necessary, as the UI feature is enabled in the config.json file.

Sign in to the Vault console using the initial root token generated during the Vault initialization process.

Execute the following command to view the secrets:

./vault secrets list

Enable the kv engine.

./vault secrets enable version=1 kv

Store secret data carefully.

./vault kv put kv/my-secret my-value=s3cr3t

Retrieve and display the stored secret value.

./vault kv get kv/my-secret

The output will resemble the following example:

====== Data ======
Key Value
--- -----
my-value    s3cr3t

This completes the integration of HashiCorp Vault with Luna HSM.

Enable transit secrets engine with external entropy source.

./vault secrets enable -external-entropy-access transit

List the enabled secrets engine with the -detailed flag. Observe that the External Entropy Access is set to true for transit/.

Use the transit secrets engine to encrypt sensitive data, leveraging the HSM as its external entropy source. Create a new encryption key named orders.

./vault write -f transit/keys/orders

Send a base64-encoded string to be encrypted by Vault.

./vault write transit/encrypt/orders plaintext=$(base64 <<< "4111 1111 1111 1111")

The output will resemble the following example:

Key   Value
---   -----
ciphertext    vault:v1:ZXKU9Yc8+BefMCkPJVUksh5y0NlTymeToyTKl7NzdE5I4CpRtcjjPnUsvVmwPpQ/

Verify that ciphertext can be decrypted.

./vault write transit/decrypt/orders \ciphertext="vault:v1:ZXKU9Yc8+BefMCkPJVUksh5y0NlTymeToyTKl7NzdE5I4CpRtcjjPnUsvVmwPpQ/"

The output will resemble the following example:

Key  Value
---  -----
plaintext    NDExMSAxMTExIDExMTEgMTExMQo=

Decode to get the original plaintext string.

base64 --decode <<< NDExMSAxMTExIDExMTEgMTExMQo=

The output will be:

4111 1111 1111 1111

When the external entropy access is enabled, the connectivity to the HSM is required. If the HSM becomes unreachable for any reason, the transit secrets engine returns an error and data cannot be encrypted or decrypted until the HSM connection gets restored.

Stop the Vault server from the terminal, if running.

Update the config.json file as follows:

#Entropy
entropy "seal" {
mode = "augmentation"
}
# PKCS11 seal
seal "pkcs11" {
lib = "path to cryptoki library"
slot = "slot number"
pin = "partition password"
default_key_label="HashiCorp"
key_label = "HashiCorp_rot"
default_hmac_key_label = "HashiCorp_hmac"
hmac_key_label = "HashiCorp_hmac_rot"
generate_key = "true"
}
storage "file" {
path = "/tmp/vault"
}

Start the Vault server using the updated configuration file.

./vault server -config config.json

Launch a new terminal session and execute the following command:

export VAULT_ADDR='\http://127.0.0.1:8200'

Verify the Vault status and list the vault secrets.

./vault secrets list

Verify the keys generated in the partition by executing the partition contents command in lunacm utility on host.

partition contents

Set up Docker

Set up Docker-Compose

Enable host to use Luna HSM in container

Create a Docker container

Set up Vault in container

Log in to Vault on host

Use secrets engine on host

Use Vault from one container to another

Connect to the host system as root or as a user with administrative privileges.

Create a working directory to place all the necessary binaries and files required for the integration.

mkdir /opt/luna-docker
cd /opt/luna-docker

Transfer the Luna minimal client, Vault binary, Luna configuration file, and Luna client/server certificates into the current working directory.

You can use any supported version of Luna minimal client and Vault binary as per your requirement. All other files are copied from the workstation where you have already registered a Luna partition and created an NTL connection.

Update the Luna configuration file Chrystoki.conf in the current directory and edit the following sections of the file to make it compatible with Luna minimal client library paths.

Chrystoki2 = {
  LibUNIX = /usr/safenet/lunaclient/libs/64/libCryptoki2.so;
  LibUNIX64 = /usr/safenet/lunaclient/libs/64/libCryptoki2_64.so;
}

Remove the Secure Trusted Channel section from the Chrystoki.conf file.

Secure Trusted Channel = {
ClientTokenLib = /usr/safenet/lunaclient/libs/64/libSoftToken.so;
SoftTokenDir = /usr/safenet/lunaclient/configData/token;
ClientIdentitiesDir = /usr/safenet/lunaclient/data/client_identities;
PartitionIdentitiesDir = /usr/safenet/lunaclient/data/partition_identities;
}

Create a HashiCorp Vault configuration file config.json for enabling the PKCS11 seal in the current working directory, as indicated here:

# PKCS11 seal
seal "pkcs11" {
lib = "/usr/safenet/lunaclient/libs/64/libCryptoki2_64.so"
slot = "0"
pin = "userpin1"
key_label = "HashiCorpRSA"
hmac_key_label = "HashiCorpHMac"
generate_key = "true"
mechanism = "0x0009"
rsa_oaep_hash = "sha256"
}
storage "file" {
path = "/tmp/vault"
}
# Addresses and ports on which Vault will respond to requests
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = "true"
}
# Entropy
entropy "seal" {
mode = "augmentation"
}
disable_mlock = true
ui = true

Create a shell script vault_start.sh containing minimum commands to up and run HashiCorp Vault inside the Docker container. Keep the script in current working directory, as indicated here:

#! /bin/bash
# REMOVING VAULT DIRECTORY IF ALREADY PRESENT
rm -rf /tmp/vault > /dev/null
# GO TO / DIRECTORY WHERE VAULT BINARY IS PRESENT
cd /hashi_vault
# START VAULT IN THE BACKGROUND
./vault server -config config.json &
# SLEEPING FOR 10 SECONDS, FOR VAULT TO START UP AND LOAD
sleep 10s
# SETTING THE VAULT_ADDR PARAMETER
export VAULT_ADDR='http://0.0.0.0:8200'
# LISTING VAULT STATUS
./vault status
# INITIALIZING THE VAULT
./vault operator init -recovery-shares=1 -recovery-threshold=1
# SLEEPING FOR 10 SECONDS, FOR VAULT TO GET INITIALIZED
sleep 10s
# LISTING VAULT STATUS
./vault status
# RESTARTING THE VAULT
kill -9 `pgrep vault`
sleep 10s
./vault server -config config.json
sleep 10s

Create a Dockerfile in the current directory that will create a Docker image containing Vault and the associated resources for Luna minimal client to communicate with the Luna HSM. Check the file names and paths for all the resources.

### luna-docker image
FROM centos:centos7 as vault-server
WORKDIR /hashi_vault

COPY vault_1.6.2+ent.hsm_linux_amd64.zip /tmp/
RUN yum install -y unzip
RUN unzip /tmp/vault_1.6.2+ent.hsm_linux_amd64.zip
COPY config.json /hashi_vault/config.json
COPY vault_start.sh /vault_start.sh

COPY LunaClient-Minimal-10.4.0-417.x86_64.tar /tmp/
RUN mkdir -p /usr/safenet/lunaclient
RUN mkdir -p /usr/safenet/lunaclient/cert
RUN mkdir -p /usr/safenet/lunaclient/cert/client
RUN mkdir -p /usr/safenet/lunaclient/cert/server
RUN tar -xvf /tmp/LunaClient-Minimal-10.4.0-417.x86_64.tar --strip 1 -C /usr/safenet/lunaclient
RUN cp /usr/safenet/lunaclient/openssl.cnf /usr/safenet/lunaclient/bin
ENV ChrystokiConfigurationPath=/etc
COPY Chrystoki.conf /etc/Chrystoki.conf 
COPY CAFile.pem /usr/safenet/lunaclient/cert/server
COPY dockerclientKey.pem /usr/safenet/lunaclient/cert/client
COPY dockerclient.pem /usr/safenet/lunaclient/cert/client
RUN chmod +x /vault_start.sh
ENTRYPOINT ["/vault_start.sh"]

Generate a Docker image named vault-server by executing the docker build command with the specified Dockerfile. Run the following command in the terminal:

docker build . -t vault-server

Look for a confirmation message, indicating the successful creation of the vault-server image.

Verify if your newly created image is visible among other images using the docker images command.

Create a docker-compose.yaml file that includes the following information:

version: "3"
volumes:
  vault:
services:
  HashiCorpVaultContainer:
    container_name: hashicorp_vault_container
    image: vault-server:latest
    ports:
      - 8200:8200
    volumes:
      - vault:/hashi_vault

Create and start the container using the docker-compose up command:

docker-compose up 

After a successful initialization of Vault, the output will reflect the status and key information from the initialization process.

Open a new terminal and use the following command to verify that the Docker container, named hashicorp_vault_container is running:

docker ps -a

Run the docker logs command to retrieve the root token required for logging into the Vault.

docker logs hashicorp_vault_container > logs.txt

You will find the Initial Root Token in the generated logs.txt file.

Create a soft link for the vault binary, mounted from container to host.

ln -sf /var/lib/docker/volumes/luna-docker_vault/_data/vault

Set the VAULT_ADDR variable on host to begin using the vault service on host system.

export VAULT_ADDR='http://0.0.0.0:8200'

Log in to the Vault.

./vault login VAULT-TOKEN

Here, VAULT-TOKEN is the initial root token generated during Vault initialization.

Verify that the Vault server is initialized and auto-unsealed.

Execute the command below to activate the KV secrets engine. Upon successful completion, a confirmation message will be presented.

./vault secrets enable -version=1 kv

Use the following command to view the stored secrets:

./vault secrets list

Safely store arbitrary secret data.

./vault kv put kv/my-secret my-value=s3cr3t

Upon successful completion, you'll receive the confirmation: Success! Data written to: kv/my-secret.

Access the secret value:

./vault kv get kv/my-secret

You now possess the capability to run Vault within an integrated container environment alongside Luna HSM, ensuring smooth and secure operations. Additionally, you can mount the Vault binary on the host system independently, eliminating the need for the Luna Client. This grants autonomy in managing and utilizing Vault's capabilities on the host. This flexibility extends to mounting the Vault binary from the host to the container for any application that requires Vault functionality in encrypting/decrypting secrets.

Use either of the following commands to create another container without installing Luna Client and Vault binary. You've already mounted the Vault binary from a container to host and the same binary will now be mounted in the new container.

docker run -it --name vault-service -v /opt/luna-docker/vault:/usr/bin/vault --net=host centos:centos7

or

docker run -it --name vault-service -v /opt/luna-docker/vault:/usr/bin/vault --env VAULT_ADDR=http://0.0.0.0:8200 --net=host centos:centos7

This is just an example to run the vault service in multiple containers without installing the Luna Client and Vault. You can use it as a reference and use the Vault integrated with Luna HSM in multiple containers.

Export the VAULT_ADDR variable in a new container.

export VAULT_ADDR='http://0.0.0.0:8200'

Skip this step if you have already set the VAULT_ADDR environment while executing the docker run command in step 1.

Log in to the Vault in new container.

vault login 

Write arbitrary secret data from the new container.
{: .numbered }


vault kv put kv/my-secret my-value=C0ntainer


Upon successful completion of the task, you'll see the following error message: 

Success! Data written to: kv/my-secret


Display the secret value by executing the following command in the new container:
{: .numbered }

vault kv get kv/my-secret


This completes the integration of HashiCorp Vault running inside a Docker container with Luna HSM.

Configure managed key on Luna HSM with auto-unseal disabled

Configure managed key on Luna HSM with auto-unseal enabled

Generate managed key within Luna HSM

Validate PKI functionality by using managed key generated on Luna HSM

Create a Vault configuration file named managed-key-config.hcl with the following contents.

kms_library "pkcs11" {
  name = "luna"
  library = "/usr/safenet/lunaclient/lib/libCryptoki2_64.so"
}

storage "file" {
  path    = "./vault/data"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_disable = "true"
}

disable_mlock = true
license_path = "License.txt"

api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"
ui = true

Save the Vault license in a file named License.txt and provide the license file path in license_path.

Start the Vault using the configuration file.

vault server -config=managed-key-config.hcl

Open a new terminal window and export the following variables.

export VAULT_ADDR="http://127.0.0.1:8200" export PIN=[partition_co_password] export LABEL=[partition_label]

Initialize the Vault if you are running it for the first time.

Make note of the unseal key and root token as they will be used later.

vault operator init -key-shares=1 -key-threshold=1

Unseal the Vault using the unseal key that is generated when Vault is initialized.

vault operator unseal 

Log in to the Vault. The Vault is now configured to use Luna HSM for managed key.

vault login 

Create a Vault configuration file named hsm-config.hcl with the following contents.

seal "pkcs11" {
  lib = "/usr/safenet/lunaclient/lib/libCryptoki2_64.so"
  slot = "slot_id"
  pin = "partition_co_password"
  key_label="hsm-mkek-key1"
  hmac_key_label="hsm-hmac-key1"
  generate_key ="true"
  mechanism=0x1 
}

kms_library "pkcs11" {
  name="luna"
  library = "/usr/safenet/lunaclient/lib/libCryptoki2_64.so"
}

storage "file" {
  path    = "./vault/hsm-data"
}

listener "tcp" {

address     = "0.0.0.0:8200"
  tls_disable = "true"
}

disable_mlock = true
license_path = "License.txt"

api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"
ui = true

Save the Vault license in a file named License.txt. The license file path must be provided in the license_path. Start the Vault using the following configuration file:

vault server -config=hsm-config.hcl

Open a new terminal window and export the following variables.

export VAULT_ADDR="http://127.0.0.1:8200"
export PIN=[partition_co_password]
export LABEL=[partition_label]

Initialize the Vault if you are running it for the first time.

Make note of the root token that will be use later.

vault operator init -recovery-shares=1 -recovery-threshold=1

Log in to Vault. Vault is now configured to use Luna HSM for generating managed key.

vault login 

Create a RSA-2048 key manually on Luna HSM.

/usr/safenet/lunaclient/bin/cmu generatekeypair -modulusBits=2048 -publicExponent=65537 -label=test-hsm-key -sign=1 -verify=1 -encrypt=1 -decrypt=1 -wrap=1 -unwrap=1 -id=c50f7b86372b441ba77cb6f8598f1e35

Execute the following command using either key_label or key_id, depending on how you manually created the key.

USING KEY_LABEL

vault write /sys/managed-keys/pkcs11/hsm-key2 library=luna token_label=$LABEL pin=$PIN key_label="test-hsm-key" allow_generate_key=false allow_store_key=false mechanism=0x0001 key_bits=2048 any_mount=false

You only need to use either key_label or key_id in the config.

USING KEY_ID

vault write /sys/managed-keys/pkcs11/hsm-key3 library=luna token_label=$LABEL pin=$PIN key_id=c50f7b86372b441ba77cb6f8598f1e35 allow_generate_key=false allow_store_key=false mechanism=0x0001 key_bits=2048 any_mount=false

Verify that the configuration has been written to Vault.

vault list /sys/managed-keys/pkcs11

Verify the key configuration by test signing some data, where hsm-key1 is the key name defined in the vault write command.

vault write -f /sys/managed-keys/pkcs11/hsm-key1/test/sign

Create a managed key for the root CA.

vault write /sys/managed-keys/pkcs11/hsm-key-root library=luna token_label=$LABEL pin=$PIN key_label="hsm-key-root" allow_generate_key=true allow_store_key=true mechanism=0x0001 key_bits=2048 any_mount=false 

Create a managed key for the intermediate CA.

vault write /sys/managed-keys/pkcs11/hsm-key-int library=luna token_label=$LABEL pin=$PIN key_label="hsm-key-int" allow_generate_key=true allow_store_key=true mechanism=0x0001 key_bits=2048 any_mount=false

Enable PKI secrets engine for the root.

vault secrets enable -path=pki -allowed-managed-keys=hsm-key-root pki

Enable intermediate secrets engine.

vault secrets enable -path=pki_int -allowed-managed-keys=hsm-key-int pki 

Create a root CA certificate with a managed key on Luna HSM.

vault write -field=certificate pki/root/generate/kms managed_key_name=hsm-key-root common_name=example.com ttl=8760h > /tmp/CA_cert.crt

Verify the certificate.

cat /tmp/CA_cert.crt

Create an Intermediate CA CSR with the managed key on Luna HSM.

vault write -format=json pki_int/intermediate/generate/kms managed_key_name=hsm-key-int common_name="example.com" | jq -r '.data.csr' > /tmp/pki_intermediate.csr

Verify the CSR.

cat /tmp/pki_intermediate.csr

Sign the intermediate CA CSR with managed root CA.

vault write -format=json pki/root/sign-intermediate csr=@/tmp/pki_intermediate.csr format=pem_bundle ttl="43800h" | jq -r '.data.certificate' > /tmp/intermediate.cert.pem

Create an intermediate CA certificate in managed key.

vault write pki_int/intermediate/set-signed certificate=@/tmp/intermediate.cert.pem

Create a role to issue a leaf certificate from the intermediate key.

vault write pki_int/roles/example-dot-com allowed_domains="example.com" allow_subdomains=true max_ttl="720h"

Issue leaf certificate.

vault write -format=json pki_int/issue/example-dot-com common_name="test.example.com" ttl="24h"