Your suggested change has been received. Thank you.

close

Suggest A Change

https://thales.na.market.dpondemand.io/docs/dpod/services/kmo….

Thales Logo

All

  • All
back

Deployment Methods

Deploying CTE for Kubernetes in an Air-Gapped OpenShift Container Platform Cluster using CTE for Kubernetes Operator

search

Deploying CTE for Kubernetes in an Air-Gapped OpenShift Container Platform Cluster using CTE for Kubernetes Operator

Overview

This section documents the mandatory steps to deploy CTE for Kubernetes Operator and CTE for Kubernetes on an OCP cluster that is not connected to the internet. Once your OCP cluster is setup and configured properly, you can then mirror the CTE for Kubernetes Operator Catalog, CTE for Kubernetes images, and other images on which CTE for Kubernetes is dependent.

Note

The following information is valid for CTE for Kubernetes Operator v1.5.9 and subsequent versions. It is not valid with any previous version.

Prerequisites

You must have a docker-v2 compliant image registry that is set up properly, running, and is accessible from the internal network used by your OCP cluster. The illustrative commands in this document use the registry setup internally at ocp-discon.lab.ocp.lan:5000. This registry is accessible on the OCP private network.

Assumptions for this Document

The Bastion/Service/Helper node of the OCP cluster has access to the internet and the OCP cluster.

If this is not true, then you must manually mirror the CTE for Kubernetes Operator Catalog onto a system that has access to internet. The catalog may include other related images in the form of a file archive from docker.io (or their respective locations). This archive must then be copied to the Bastion/Service/Helper node.

Mirror the CTE for Kubernetes Catalog

"

  1. Configure imageset-configuration-ctek8soperator.yaml. Refer to Appendix: Sample Script for a sample ImageSetConfiguration file for CTE for Kubernetes Operator and related images.

  2. Mirror the CTE for Kubernetes Operator Catalog, and other related images, to a file on a disk.

  3. Run the oc-mirror command, type:

    oc-mirror --config imageset-configs/imageset-config-ctek8soperator.yaml file:///root/ocp-mirror/ocp-release/ctek8soperator-images  2>&1 | tee ctek8soperator-mirror.log

    Expected output:

    ---> info: Mirroring completed in 3m58.31s (11.52MB/s)
---> Creating archive /root/ocp-mirror/ocp-release/ctek8soperator-images/mirror_seq1_000000.tar

  4. Disable internet access

    Note

    Not applicable if creating the mirror archive through another server. If so, manually copy the archive.

  5. Upload the images, that were just mirrored to the file archive, to your internal docker-v2 compliant registry:

    oc-mirror --from=/root/ocp-mirror/ocp-release/ctek8soperator-images/mirror_seq1_000000.tar docker://ocp-discon.lab.ocp.lan:5000

    Expected output:

    ---> Rendering catalog image "ocp-discon.lab.ocp.lan:5000/shetland/ciphertrust-transparent-encryption-kubernetes-operator-catalog:v1.2.5" with file-based catalog
---> Writing image mapping to oc-mirror-workspace/results-1737970204/mapping.txt
---> Writing CatalogSource manifests to oc-mirror-workspace/results-1737970204
---> Writing ICSP manifests to oc-mirror-workspace/results-1737970204

Edit the deploy scripts -- Mandatory

  1. Edit the file oc-mirror-workspace/results-1737970204/imageContentSourcePolicy.yaml and change the Spec.Metadata.Name:

    metadata:
name: cte-operator-generic-0
:
:
metadata:
labels:
    operators.openshift.org/catalog: "true"
name: cte-operator-0

  2. Apply imageContentSourcePolicy.yaml.

    cat oc-mirror-workspace/results-1737970204/imageContentSourcePolicy.yaml

    Response:

    imagecontentsourcepolicy.operator.openshift.io/cte-operator-generic-0 created
imagecontentsourcepolicy.operator.openshift.io/cte-operator-0 created

  3. Open the following file for editing: 

    oc-mirror-workspace/results-1737970204/catalogSource-cs-ciphertrust-transparent-encryption-kubernetes-operator-catal.yaml

  4. Append the following 5 lines to the file:

    displayName: CTE-K8s Operator Catalog
publisher: Thales Group
updateStrategy:
registryPoll:
interval: 24h

    Response:

    apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: ctek8soperator-catalog
namespace: openshift-marketplace
spec:
image: ocp-discon.lab.ocp.lan:5000/public-repos/ciphertrust-transparent-encryption-kubernetes-operator-catalog:v1.2.5
sourceType: grpc
displayName: CTE-K8s Operator Catalog
publisher: Thales Group
updateStrategy:
    registryPoll:
    interval:   24h

  5. Apply the CatalogSource manifest:

    oc apply -f oc-mirror-workspace/results-1741799472/catalogSource-cs-ciphertrust-transparent-encryption-kubernetes-operator-catal.yaml

    Response:

    catalogsource.operators.coreos.com/ctek8soperator-catalog created

  6. Verify the catalog resource:

    oc get catsrc -n openshift-marketplace

    Response:

    NAME                     DISPLAY                    TYPE   PUBLISHER      AGE
ctek8soperator-catalog   CTE-K8s Operator Catalog   grpc   Thales Group   5s

  7. Verify the pods:

    oc get pods -n openshift-marketplace

    Response:

    NAME                                    READY   STATUS    RESTARTS   AGE
ctek8soperator-catalog-9wv58            1/1     Running   0          27s

Download and configure deploy scripts -- Mandatory

Before Deployment, you must follow the steps in this section. It is mandatory.

  1. Clone the deploy scripts from https://github.com/thalescpl-io/ciphertrust-transparent-encryption-kubernetes.git

  2. Edit this file:

    vi deploy/kubernetes/x.y.z/values.yaml

  3. Replace the registry name for each of the images with the corresponding URL in the file oc-mirror-workspace/results-nnnnnnn/mapping.txt.

  4. Execute the detemplatize_manifest.sh script to create a set of files for use in creating a configMap. Type:

    ./detemplatize_manifest.sh

    This script:

    • Picks up manifest files from deploy/kubernetes/*/templates

    • Uses the values.yaml file from the respective chart version to get values for fields templatized in the manifests.

    • Creates a set of detemplatized manifest files for creating a custom ConfigMap for CTE-K8s deployment

    In case further changes are required, beyond what can be customized via values.yaml (logLevel, Image URL, APIServerThrottle.burst, APIServerThrottle.qps), like change tolerations, change nodeAffinity, for controller pod or nodeserver pod, do it now directly on the custom manifest files generated.

  5. Create a ConfigMap with the customized manifests, type:

    oc create configmap ctecustomconfig -n <operator namespace> --from-file=./custom_manifests/X.Y.Z/

  6. Verify that the ConfigMap was created, type:

    oc get configmap ctecustomconfig -n <operator namespace>

  7. Edit this file:

    vi deploy/kubernetes/x.y.z/operator-deploy/ctek8soperator-subscription.yaml

    Change the Spec.source and Spec.sourceNamespace to match the values in oc-mirror-workspace/results-1741799472/catalogSources-ciphertrust-transparent-encryption-kubernetes-operatorcatal.yaml that you changed in the section above.

  8. Edit the deploy/kubernetes/X.Y.Z/operator-deploy/deploy.sh

  9. Comment out the line containing certified-operators. The difference should looks like the following:

    diff --git a/deploy/kubernetes/1.6.0/operator-deploy/deploy.sh b/deploy/kubernetes/1.6.0/operator-deploy/deploy.sh

    Response:

    diff --git a/deploy/kubernetes/1.6.0/operator-deploy/deploy.sh b/deploy/kubernetes/1.6.0/operator-deploy/deploy.sh
index b556bc3..3462943 100755
--- a/deploy/kubernetes/1.6.0/operator-deploy/deploy.sh
+++ b/deploy/kubernetes/1.6.0/operator-deploy/deploy.sh
@@ -211,7 +211,7 @@ deploy_cte_csi()
{
    if [ ${IS_OCP} -eq 1 ]; then
        VALIDATE=""
        # sed -i s/"^  source: .*"/"  source: certified-operators"/g ${DEPLOY_SCRIPT_PATH}/ctek8soperator-subscription.yaml
        # sed -i s/"^  sourceNamespace: .*"/"  sourceNamespace: openshift-marketplace"/g ${DEPLOY_SCRIPT_PATH}/ctek8soperator-subscription.yaml
    else
        # some fields in the manifests for Openshift are not yet supported on Kubernetes.
}

Deploy CTE for Kubernetes Operator

  • Deploy CTE for Kubernetes Operator with the deploy.sh script.

    ./deploy.sh

    Response

    Starting the cte-csi containers.
Automatic detection of CRI socket is enabled
Using default CRI socket path /run/crio/crio.sock for container runtime cri-o
Using CRISocket path: /run/crio/crio.sock
--------------------------------------------------------------------------
CipherTrust Transparent Encryption for Kubernetes Operator will be deployed in namespace: kube-system
CipherTrust Transparent Encryption for Kubernetes will be deployed in namespace: kube-system
--------------------------------------------------------------------------
serviceaccount/cte-csi-controller created
clusterrole.rbac.authorization.k8s.io/cte-csi-controller-ac created
clusterrolebinding.rbac.authorization.k8s.io/cte-csi-controller-binding created
serviceaccount/cte-csi-node created
clusterrole.rbac.authorization.k8s.io/cte-csi-node-ac created
clusterrolebinding.rbac.authorization.k8s.io/cte-csi-node-binding created
operatorgroup.operators.coreos.com/ctek8soperator-og created
subscription.operators.coreos.com/ctek8soperator-sub created
Waiting for InstallPlan to be instantiated and approved..........
Successfully installed CipherTrust Transparent Encryption for Kubernetes Operator
Deploying CipherTrust Transparent Encryption for Kubernetes
Waiting for the ctek8soperator CRD to get registered with K8s..
Successfully registered ctek8soperator CRD with K8s
Waiting for ctek8soperator controller manager to get ready........
ctek8soperator controller manager is ready
ctek8soperator.cte-k8s-operator.csi.cte.cpl.thalesgroup.com/ctek8soperator created
Waiting for CipherTrust Transparent Encryption for Kubernetes...........
==========================================================================================
CipherTrust Transparent Encryption for Kubernetes Operator deployed in namespace kube-system
CipherTrust Transparent Encryption for Kubernetes in namespace kube-system
cte-csi-controller-6dd49c67f4-gf4x9                    0/5     ContainerCreating   0          1s
cte-csi-node-2wj4s                                     0/4     ContainerCreating   0          1s
cte-csi-node-bjntf                                     0/4     ContainerCreating   0          1s
cte-csi-node-mp4xv                                     0/4     ContainerCreating   0          1s
cte-csi-node-r9ldw                                     0/4     ContainerCreating   0          1s
cte-csi-node-w7gzf                                     0/4     ContainerCreating   0          1s
==========================================================================================

Appendix: Sample Script

 
kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
storageConfig:
registry:
    imageURL: ocp-discon.lab.ocp.lan:5000/cteoperator/operator-metadata
    skipTLS: false
mirror:
  operators:
  - catalog : docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes-operator-catalog:v1.2.5
    packages:
      - name: cte-k8s-operator
      defaultChannel: stable
      channels:
      - name: stable
  additionalImages:
    - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:<current_version.build>
    - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:<current_version>-latest
    - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:<current_version.build>
    - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:<current_version>-latest
    - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:<current_version.build>
    - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:<current_version>-latest
    - name: registry.k8s.io/pause:3.9
    - name: registry.k8s.io/sig-storage/csi-provisioner:v4.0.0
    - name: k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.0.1
    - name: k8s.gcr.io/sig-storage/csi-attacher:v3.3.0
    - name: registry.k8s.io/sig-storage/csi-snapshotter:v6.3.3
    - name: registry.k8s.io/sig-storage/csi-resizer:v1.8.0

Example of Additional Images

  - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:1.6.0.27
  - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:1.6.0-latest
  - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:1.5.0.37
  - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:1.5.0-latest
  - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:1.4.0.33
  - name: docker.io/thalesciphertrust/ciphertrust-transparent-encryption-kubernetes:1.4.0-latest

On this page