System Upgrade/Downgrade

You can upgrade/downgrade your CipherTrust Manager firmware to and from 2.18.x by securely downloading and applying a new/older system archive file.

Specific upgrade steps vary depending on whether the CipherTrust Manager is clustered or unclustered.

Downgrade is only supported on unclustered appliances.

System Upgrade for an Unclustered Appliance

Refer to Cluster Upgrade for details on upgrading a CipherTrust Manager which is part of a cluster of devices.

For 2.18.x, we tested upgrade from 2.17.0, 2.16.1, and 2.15.0.

You require ksadmin level access with an SSH key.

1. Obtain the signed archive file for the upgrade from the Support Portal. The file has the format ks_upgrade_<major.minor.patch+build_number>.tar.gz.gpg.

2. On CipherTrust Manager create and download a backup with corresponding backup key, in case there are any problems.

   Note

   Upgrades keep all the data and may migrate the data and configuration. Therefore, as a precaution, it is recommended to take a backup before upgrading.
   Consult the backups page for details on forward compatibility for backups. Restoring a newer backup to an older version is never supported.

3. Transfer the archive file to the CipherTrust Manager. You require the private SSH key associated with the ksadmin account. The command and supported private key format depends on the operating system you are transferring the archive file from.

   Source Operating System	Example Command Syntax	Required Private Key Format
   Linux	scp -i <path_to_private_SSH_key> <archive_file_name> ksadmin@<ip>:.	OpenSSH format
   Windows	pscp -i <path_to_private_SSH_key> <archive_file_name> ksadmin@<ip>:.	PuTTY PPK format

4. ssh into the CipherTrust Manager as ksadmin and ensure there is sufficient disk space available. The required disk space is 5 times the size of the upgrade file. For example, if the upgrade file is 5 GB, 25 GB of free space is required. Use df -h / to view available space.

5. Run the following command:

   sudo /opt/keysecure/ks_upgrade.sh -f <archive_file_path>

   Here, <archive_file_path> specifies the CipherTrust Manager path to the signed archive file.

   The signature of the archive file is verified and the upgrade is applied.

6. Reboot the appliance when prompted. The following output indicates the upgrade has completed:

   Upgrade succeeded
   Reboot the instance to finish the upgrade
   

   Note

   If you have lost the SSH session, you can access upgrade logs in /home/ksadmin/. Upgrade log files have the prefix ks_upgrade. Search for "Upgrade succeeded" to confirm the upgrade completed, and that you must reboot. Take care to verify the upgrade has completed before rebooting. You must always reboot to fully apply the upgrade.

7. Ensure the CipherTrust Manager services have started. From the ksadmin session, run systemctl status keysecure. Alternatively, you can visit the CipherTrust Manager web console or attempt to connect with the ksctl CLI.

8. If you need to verify the new version, look for the version displayed in CipherTrust Manager banner in the SSH session on reboot, as shown in the screenshot. Alternatively, log in the web console UI and click the information icon at the top right to display version information.

   

9. Re-download and re-install ksctl to access commands and settings introduced in the new CipherTrust Manager software version.

   Note

   Previous versions of ksctl still work with new CipherTrust Manager software for existing features and settings.

System Downgrade

An unclustered CipherTrust Manager can be downgraded to the previous minor version. For release-specific upgrade/downgrade information, refer to the release notes for your release.

Downgrades perform a CipherTrust Manager reset, which wipes all CipherTrust Manager data except the backup files that already exist.

As well, the PCI HSM drivers on k570 models, and base operating system packages are not changed during downgrade.

To downgrade your CipherTrust Manager

1. SSH into the CipherTrust Manager as "ksadmin".

2. Downgrade the CipherTrust Manager:

   $ sudo /opt/keysecure/ks_downgrade.sh -f <~/filename>
   

Usage: ks_downgrade.sh -f <FILE> [-o]

*  `-f`: Path to the signed CipherTrust Manager installer file.

*  `-o`: Clustered node cannot be downgraded. Use this flag to override this behavior.

Cluster Upgrade

A cluster can be upgraded in two ways:

• Online In-place Cluster Upgrade

• Using the Cluster Remove/Rebuild Method

The following table indicates the differences between the two ways:

Prerequisites

• You require ksadmin level access with an SSH key.

• Obtain the signed archive file for the upgrade from the Support Portal. The file has the format ks_upgrade_<major.minor.patch+build_number>.tar.gz.gpg.

Online In-place Cluster Upgrade

Be aware of the following considerations when performing an in-place cluster upgrade:

• The node being upgraded will be inaccessible during the upgrade. This may be as long at 10 or more minutes. Clients must be able to handle this outage.

• There will be a brief period of time (under 30 seconds) where the database will be locked while upgrading the first node. This affects all nodes at the same time, and some nodes may give error responses during this time.

• All nodes in the cluster should be upgraded as soon as possible - nodes running different version of the firmware will behave differently, potentially causing problems with applications.

• We strongly recommend consulting the Upgrade Paths if you are starting from two or more minor versions below the target version.

To perform an online in-place cluster upgrade

1. Before doing any upgrade operation, ensure that you have a backup, and that you have downloaded the backup and associated backup key.

2. Ensure all nodes in the cluster are up and operating normally. Resolve any issues (like removing any obsolete nodes) before performing the upgrade.

3. Perform a system upgrade on each node, one at a time, and one minor version at a time.

   1. Transfer the archive file to the CipherTrust Manager. You require the private SSH key associated with the ksadmin account. The command and supported private key format depends on the operating system you are transferring the archive file from.

      Source Operating System	Example Command Syntax	Required Private Key Format
      Linux	scp -i <path_to_private_SSH_key> <archive_file_name> ksadmin@<ip>:.	OpenSSH format
      Windows	pscp -i <path_to_private_SSH_key> <archive_file_name> ksadmin@<ip>:.	PuTTY PPK format

   2. SSH into the CipherTrust Manager as ksadmin and ensure there sufficient disk space available. The required disk space is 5 times the size of the upgrade file. For example, if the upgrade file is 5 GB, 25 GB of free space is required. Use df -h / to view available space.

   3. Run the following command to upgrade the device.

      $ sudo /opt/keysecure/ks_upgrade.sh -f <archive_file_path>
      

      Here, <archive_file_path> specifies the CipherTrust Manager path to the signed archive file.

      The signature of the archive file is verified and the upgrade is applied.

   4. Reboot the appliance as prompted. The following output indicates the upgrade has completed:

      Upgrade succeeded
      Reboot the instance to finish the upgrade
      

      Note

      If you have lost the SSH session, you can access upgrade logs in /home/ksadmin/. Upgrade log files have the prefix ks_upgrade. Search for "Upgrade succeeded" to confirm the upgrade completed, and that you must reboot. Take care to verify the upgrade has completed before rebooting. You must always reboot to fully apply the upgrade.

   5. Ensure the upgrade of each node is complete and that the node is operating normally, before proceeding to the next node.

      From the ksadmin session, run systemctl status keysecure to ensure that the CipherTrust Manager services have started. Alternatively, you can visit the CipherTrust Manager web console or attempt to connect with the ksctl CLI.

      Note

      When updating the first node in a cluster, the cluster nodes may briefly experience slower than usual response times. This occurs because the shared database schema for the cluster is updated with the first node.

   6. If you need to verify the new version, look for the version displayed in CipherTrust Manager banner in the SSH session on reboot, as shown in the screenshot. Alternatively, log in the web console UI and click the information icon at the top right to display version information.

      

4. Repeat step 3 for each minor version.

5. Re-download and re-install ksctl to access commands and settings introduced in the new CipherTrust Manager software version.

   Note

   Previous versions of ksctl still work with new CipherTrust Manager software for existing features and settings.

Upgrade Using the Cluster Remove/Rebuild Method

Supported upgrade paths are the same as for system upgrade of an unclustered appliance.

Prerequisite for Cluster Remove/Rebuild Method

Plan for downtime when clients cannot make requests to the cluster and manually block client access to the CipherTrust Manager IP addresses/hostnames in your network before starting.

To Perform an Upgrade Using Cluster Remove/Rebuild Method

1. On one of the cluster nodes, create and download a backup with corresponding backup key, in case there are any problems.

2. Remove all nodes from the cluster except one.

3. Delete the cluster configuration on the remaining node.

   $ ksctl cluster delete
   

4. Perform the upgrade on that remaining node.

   1. Transfer the archive file to the CipherTrust Manager. You require the private SSH key associated with the ksadmin account. The command and supported private key format depends on the operating system you are transferring the archive file from.

      Source Operating System	Example Command Syntax	Required Private Key Format
      Linux	scp -i <path_to_private_SSH_key> <archive_file_name> ksadmin@<ip>:.	OpenSSH format
      Windows	pscp -i <path_to_private_SSH_key> <archive_file_name> ksadmin@<ip>:.	PuTTY PPK format

   2. SSH into the CipherTrust Manager as ksadmin and ensure there sufficient disk space available. The required disk space is 5 times the size of the upgrade file. For example, if the upgrade file is 5 GB, 25 GB of free space is required. Use df -h / to view available space.

   3. Run the following command to upgrade the device.

      $ sudo /opt/keysecure/ks_upgrade.sh -f <archive_file_path>
      

      Here, <archive_file_path> specifies the CipherTrust Manager path to the signed archive file.

      The signature of the archive file is verified and the upgrade is applied.

   4. Reboot the appliance as prompted. The following output indicates the upgrade has completed:

      Upgrade succeeded
      Reboot the instance to finish the upgrade
      

      Note

      If you have lost the SSH session, you can access upgrade logs in /home/ksadmin/. Upgrade log files have the prefix ks_upgrade. Search for "Upgrade succeeded" to confirm the upgrade completed, and that you must reboot. Take care to verify the upgrade has completed before rebooting. You must always reboot to fully apply the upgrade.

   5. From the ksadmin session, run systemctl status keysecure to ensure that the CipherTrust Manager services have started. Alternatively, you can visit the CipherTrust Manager web console or attempt to connect with the ksctl CLI.

   6. If you need to verify the new version, look for the version displayed in CipherTrust Manager banner in the SSH session on reboot, as shown in the screenshot. Alternatively, log in the web console UI and click the information icon at the top right to display version information.

      

5. After the appliance reboots, re-build the cluster by creating a new cluster on this node.

6. Perform the upgrade on all other removed nodes, as outlined in step 4.

   Note

   If a previously used node is to be re-used, the cluster must first be deleted from that system.

7. Join new instances to the cluster.

8. Re-establish client access to the cluster.

9. Re-download and re-install ksctl to access commands and settings introduced in the new CipherTrust Manager software version.

   Note

   Previous versions of ksctl still work with new CipherTrust Manager software for existing features and settings.

Upgrade Troubleshooting

The following section lists some commonly encountered errors during upgrade and their solutions.

Occasional Error for Devices Upgraded from NextGen KeySecure 1.6 or 1.7

CipherTrust Manager appliances that at one time ran NextGen KeySecure software versions 1.6 or 1.7 occasionally fail to upgrade on future versions. The error reported is E: Unable to locate package docker-compose-plugin.

Solution: Contact customer support to resolve this state.