Configuration Concepts
The following concepts are important to understand for success with CTE for Kubernetes.
For more information on Kubernetes, consult the Kubernetes documentation.
CipherTrust Manager CTE for Kubernetes Storage Groups
CTE for Kubernetes configuration is accomplished with CipherTrust Managers(CM) through K8s Storage Groups. Security administrators define which combination of Kubernetes Storage Classes and Namespaces are allowed to register to CM. Once a K8s Storage Group is defined, the user can create a GuardPolicy that can be used by the CTE-CSI volume to create, view, edit, and delete Kubernetes (K8s) storage groups on the K8s Storage Groups page of the CipherTrust Manager GUI.
Note
A single node in a Kubernetes cluster can register itself multiple times to different K8s Storage Groups based on different Storage Class and Namespace usage patterns. This allows for separation of security context based on the security needs of the Kubernetes namespace that the application pod will be running in.
- See Managing Kubernetes Storage Groups and Clients for more information about creating storage groups in CipherTrust Manager.
Using CipherTrust Manager with CTE for Kubernetes
On the CipherTrust Manager, to setup your Kubernetes environment:
-
Create a Kubernetes Client.
-
Create a Kubernetes storage group, which includes a client profile which contains a Guard Policy. The Guard Policy must contain:
-
Supported access controls
-
Resource sets
-
Process sets
-
Signature sets
-
User sets with UIDs
-
-
Register the Kubernetes Client
Registration is the process of configuring a Kubernetes (Kubernetes) client with CipherTrust Manager. This process creates SSL certificates for further communication between the CipherTrust Manager and the Kubernetes client.
After registration, the Kubernetes client can communicate with the CipherTrust Manager. All of the Guard Policies applied to the Kubernetes storage group are automatically added to the Kubernetes client. The client configuration is then built for the Kubernetes client (exactly like a CTE client) and sent to the CipherTrust Manager.
After successful registration, the Kubernetes client appears on the Kubernetes Clients page of the CipherTrust Manager GUI. The client status becomes Healthy.
CTE for Kubernetes CSI driver
Protecting Persistent volumes in a Kubernetes Cluster
Unlike traditional CSI drivers, the CTE for Kubernetes CSI driver (CTE-CSI) does not store persistent data. CTE-CSI relies on other Persistent Volumes(PV) in the cluster for actual data storage. The CTE-CSI driver provides a layer on top of those PVs that adds encryption and data protection. This encryption is transparent to the Pod that wants to use a CTE-CSI volume. Each CTE Persistent Volume Claim (PVC) must be backed by a filesystem-capable PV that is claimed by a PVC in the same namespace.
Each CTE PVC must be configured to use a GuardPolicy, configured in CM, that will provide the required access control and key rules.
When protecting a PV, you must:
-
Choose a PVC in the cluster to protect and to act as your data source.
-
Create a CTE-CSI Storage Class with a parameter to connect to CM.
-
Create a CTE-CSI PVC with a GuardPolicy and a data source for PVC parameters.
-
Add the CTE-CSI PVC to a Pod as a regular volume.
Supported Persistent volumes
CTE Kubernetes is expected to work well with most persistent volumes types with file system semantics. Raw volumes with no filesystem are not supported at this time.
For more information, see Types of Persistent Volumes
The following table lists the unsupported PVC volumes:
Storage Type | Supported | Full Name |
---|---|---|
configMap | No | ConfigMap |
downwardAPI | No | Downward API |
emptyDir | No | EmptyDir |
fc | No | Fibre Channel (FC) storage |
flocker | No | Flocker storage |
gitRepo | No | GIT repository |
hostPath | No | HostPath volume |
projected | No | Projected volume |
quobyte | No | Quobyte volume |
secret | No | Secrets |
storageOS | No | StorageOS volume |
subPath | No | Subpath volume |
CTE service pods
CTE for Kubernetes is implemented as a Kubernetes CSI driver. A typical CTE-CSI driver deployment will contain one cte-csi-node-xxxx for every worker node in the cluster. These pods are the main workers for CTE for Kubernetes and perform tasks such as volume attachment, CTE agent registration and volume encryption within the cluster node. The cluster will also contain at least one cte-csi-controller-xxxx which is in charge of dynamic provisioning of CTE-CSI volumes. These two types of service pods run in the kube-system namespace and are able to serve volumes across different namespaces in the cluster.
Warning
It is important to never restart a node's cte-csi-node-xxx pod if this node still has CTE-CSI volumes attached to it from other application pods. Killing a busy CTE-CSI pod will result in application Pods losing their connection to the encryption service. This will render the volume on those pods inaccessible, and can result in data loss. When this happens, the application Pod must be restarted to restore proper functionality.
CTE for Kubernetes registration
CTE for Kubernetes dynamically registers the agent when a CTE-CSI volume attachment request is performed on a node. CTE will use the configuration parameters of the StorageClass, as well as the Namespace of the Pod, requesting the volume to attempt registration. Registration only succeeds if the Storage Class name and the Pod's Namespace matches that of the Storage Group defined in CipherTrust Manager. The CTE-CSI driver will automatically deregister itself for a given StorageClass/Namespace combination once all Pods that were consuming a particular registration in the node have been evicted from the node. The amount of time that the CTE-CSI driver waits from last eviction to deregistration can be configured through the StorageClass registration_period parameter
Note
Registration and deregistration to CM is dynamic, based on pod usage within the nodes in the cluster. This can quickly exhaust registration token client capacity. Therefore, it is recommended that the user define a CipherTrust Manager registration token with a large client capacity.
Warning
Kubernetes will always retry and mount a Pod's volume in the event of a failure. If that failure is due to a registration failure (for example, non-matching StorageClass/Namespace parameters in the K8s Storage Group), retry attempts by Kubernetes will quickly exhaust registration token client capacity.
CTE-CSI Storage Class
For CTE for K8s, as soon as you create a pod to protect a cte-csi volume, cte-csi is using a storage class. Once this storage class is created, it automatically creates a K8s client and registers it with CipherTrust Manager.
When first creating Storage Class objects, Administrators set the name and other parameters for a class in a yaml file. Note that objects cannot be updated once they are created.
Important standard Kubernetes specification definitions
The following list defines the parts of the yaml file:
-
metadata.name: Name of the StorageClass. This name MUST match the StorageClass name specified in the CM K8s Storage Group.
-
provisioner: Determines what volume plugin is used for provisioning Persistent Volumes. This field must be specified. For CTE for Kubernetes, it is always defined as csi.cte.cpl.thalesgroup.com
-
reclaimPolicy: Determines what to do with the PersistentVolume once its claim has been deleted. CTE PVs do not hold any data. They act solely as configuration placeholders for real PVs. The policy MUST be set to Delete as they are not usable outside of the context of its PVC that created it.
-
allowVolumeExpansion: CTE for Kubernetes does not store data so these volumes are not expandable. This does not prevent the source unprotected volume from being expandable. This MUST be set to false
CTE-CSI Storage Class parameters
-
key_manager_addr: Domain name or IP address of the CipherTrust Manager. This is a required field.
-
k8_storage_group: Name of the CipherTrust Manager K8s Storage Group. This is a required field.
-
registration_token_secret: Kubernetes Secret with CM registration token. The CM registration token must be saved into the Kubernetes secret with as a base64 encoded string. This is a required field.
-
registration_period: Time in minutes to wait before deregistering from the CipherTrust Manager once all volumes on a node have been unguarded. Parameter must be added as a string integer value. Default "10" minute. This field is optional.
Example
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: <CHANGEME to K8s Storage Class>
provisioner: csi.cte.cpl.thalesgroup.com
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: false
parameters:
# Domain name or IP address of the CipherTrust Manager (Required)
key_manager_addr: <CHANGEME to your CM ADDR>
# Name of the CipherTrust Manager K8s Storage Group. (Required)
k8_storage_group: <CHANGEME to your K8s Storage Group name>
# Kubernetes Secret with CM registration token (Required)
registration_token_secret: <CHANGEME to K8s secret>
# Time in minutes to wait before unregistering from the CipherTrust Manager
# once all volumes have been unguarded. Parameter must be added as a string
# integer value. Default "10" minute. (Optional)
registration_period: "10"
CTE Persistent Volume Claim
Important standard Kubernetes spec definitions
The following list defines the parts of the yaml file:
-
metadata.name: Name of the CTE-CSI PVC. Use this PVC name in your Pod definition instead of the unprotected source PVC.
-
metadata.annotations.csi.cte.cpl.thalesgroup.com/policy: CTE for Kubernetes GuardPolicy name. This GuardPolicy is located on the CipherTrust Manager and should match a policy name available on the storage class for this PVC. This parameter is required.
-
metadata.annotations.csi.cte.cpl.thalesgroup.com/source_pvc: Name of the unprotected source PVC that will be protected by this CTE-PVC. This parameter is required.
-
spec.storageClassName: CTE-CSI StorageClass name. This parameter is required
-
spec.resources.requests.storage: Kubernetes PV capacity request. This parameter is required by Kubernetes but ignored by CTE-CSI as capacity is dictated by the source PVC.
Example
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: cte-claim
annotations:
# CTE for Kubernetes GuardPolicy name. This GuardPolicy is located on the
# CipherTrust Manager and should match a policy name available on the
# storage class for this PVC. (Required)
csi.cte.cpl.thalesgroup.com/policy: <CHANGEME CTE for Kubernetes policy>
# Name of the unprotected source PVC that will be protected by this CTE-PVC.
# (Required)
csi.cte.cpl.thalesgroup.com/source_pvc: <CHANGEME to a regular PVC to protect>
spec:
storageClassName: <CHANGEME to the CTE-CSI StorageClass>
accessModes:
- ReadWriteMany
resources:
requests:
# This parameter is required by Kubernetes but ignored by CTE-CSI.
storage: 1Ki