Managing Cluster Members
These procedures will allow you to add new members to an existing cluster, remove a functioning member, or delete a non-functioning member from the cluster configuration:
>Adding a New Member to an Existing Cluster
>Moving a Member to a Different Affinity Group
>Promoting a Member to Primary
>Removing a Functioning Member From a Cluster
>Deleting an Unrecoverable Member From a Cluster
NOTE Thales recommends Luna Appliance Software 7.8.3 with cluster package 1.0.3, Luna HSM Firmware 7.8.2,
Creating a Cluster
The following procedures will allow you to create a new cluster on the Luna Network HSM 7 appliance, and authorize the local member on the appliance to create objects on the cluster, using REST API calls or LunaSH.
NOTE This procedure applies to the first cluster member only, or to a cluster of one appliance. The member on which the cluster is created is the primary member, and this affects the behavior of the cluster as a whole. Refer to The Primary Member for more information. If you want to join the appliance to an existing cluster, see Adding a New Member to an Existing Cluster.
REST API
Prerequisites
>If you are creating a cluster on a multifactor quorum-authenticated HSM, you require a local or remote Luna PED. Refer to Multifactor Quorum Authentication for setup procedures.
To create a cluster and authorize the first member
1.Create a cluster on the appliance. You must specify the HSM SO password (or "password": "" and present the HSM SO PED key) and a label for the new cluster.
{
"authentication": [
{
"type": "SO",
"password": "IamtheHSMSO"
}
],
"clusterLabel": "myNewCluster"
}
2.A default member is created on the cluster with the name LNH-<HSM_SN>. Display a list of members on the cluster to find its Member ID.
GET /api/clusters/{clusterID}/members
3.Authorize the member to create objects on the cluster's keyrings by specifying the application partition, and CO password (or "password": "" and present the CO PED key).
PUT /api/clusters/{clusterID}/members/{memberID}
{
"action": "authorize",
"authentication": [
{
"type": "CO",
"password": "IamtheCO",
"partitionLabel": "clusterpar"
}
]
}
TIP Manual member authorization is required only:
>to authorize the first member of a cluster
>when recovering a cluster with a single member that has failed
>when manually reinstating the first member of a cluster where all members failed
In a cluster where at least one member remains active and connected, rejoining members automatically take their authorization from that member, and manual re-authorization is not needed.
You can now create new keyrings on the cluster (see Managing Keyrings).
4.You must run the provided LNHClientRegistration script on any Luna HSM Client computers that will query this member to create or use keyrings on the cluster. See Cluster-Client Connections for the procedure.
LunaSH
Prerequisites
>If you are creating a cluster on a multifactor quorum-authenticated HSM, you require a local or remote Luna PED. Refer to Multifactor Quorum Authentication for setup procedures.
To create a cluster and authorize the first member
1.Create a cluster on the appliance. You must specify the HSM SO password/PED key and a label for the new cluster. Do not create more than one cluster.
lunash:> cluster create -label <label> [-password <HSMSO_password>]
2.A default member is created on the cluster with the name LNH-<HSM_SN>. Display a list of members on the cluster to find its Member ID.
lunash:> cluster member list -clusterid <UUID>
3.Authorize the member to create objects on the cluster's keyrings by specifying the application partition, CO password/PED key, and Member ID.
lunash:> cluster member authorize -partition <label> -memberid <string> [-copassword <password>]
TIP Manual member authorization is required only:
>to authorize the first member of a cluster
>when recovering a cluster with a single member that has failed
>when manually reinstating the first member of a cluster where all members failed
In a cluster where at least one member remains active and connected, rejoining members automatically take their authorization from that member, and manual re-authorization is not needed.
You can now create new keyrings on the cluster (see Managing Keyrings).
4.You must run the provided LNHClientRegistration script on any Luna HSM Client computers that will query this member to create or use keyrings on the cluster. See Cluster-Client Connections for the procedure.
Adding a New Member to an Existing Cluster
Use this procedure to connect a Luna Network HSM 7 appliance to an existing cluster created on another appliance using REST API or LunaSH.
REST API
Prerequisites
>On Appliance A (where the cluster is created):
•The appliance must be configured for clustering (see Configuring the Luna Network HSM 7 for Clustering).
•A cluster must be created on the appliance, and the appliance must be authorized as a member (see Creating a Cluster).
•Thales recommends that new members join the primary cluster member. If there are already multiple members in the cluster, and you are not sure which is currently the primary member, use (the primary member returns "primarynode": true):
GET /api/clusters/{clusterID}/members/{memberID}
>On Appliance B (that will join the existing cluster):
•The appliance must be configured for clustering (see Configuring the Luna Network HSM 7 for Clustering).
NOTE If a cluster already exists on appliance B, it will not be allowed to join appliance A's cluster.
>Both appliances must be accessible to each other over the network. All members of a cluster must be able to communicate bi-directionally with all other members.
>If you are joining a cluster on a multifactor quorum-authenticated HSM, you require a local or remote Luna PED. Refer to Multifactor Quorum Authentication for setup procedures.
>Both appliances should have NTP configured, or the appliance clocks must be synchronized manually; if the appliance times are out of sync by more than 30 seconds, the new member will not be allowed to join the cluster:
Using NTP (actionid: synchronize): POST /api/lunasa/ntp/actions/{actionid}
Manually: PUT /api/lunasa/time
CAUTION! Ensure that you add only one member to the cluster at a time. Undesired behavior can result from adding more than one member simultaneously.
To join an existing cluster
1.On appliance A, get the cluster ID you wish to join.
Return:
{
"data": [
{
"clusterLabel": "myCluster",
"clusterUUID": "528b9b72-c305-4dd1-bebc-a4ca24f8c28b"
}
]
}
2.On appliance B, join appliance A's cluster by specifying its cluster ID, IP address, admin user password, the label of the partition on appliance B that will authorize appliance B as a member, and that partition's CO password (or "password": "" and present the CO PED key).
{
"authentication": [
{
"type": "CO",
"password": "IamtheCO",
"partitionLabel": "clusterpar"
}
],
"remoteClusterIpAddress": "127.0.0.1",
"remoteAdminPassword": "Iamtheadmin",
"action": "join"
}
NOTE This operation may take a few minutes, even though a success message is received right away.
New members joining an existing cluster are authorized automatically.
3.Confirm that both cluster members are now listed. If they are not, wait a few minutes and try again -- the join process may still be underway.
GET /api/clusters/{clusterID}/members
4.You must run the provided LNHClientRegistration script on any Luna HSM Client computers that will query this member to create or use keyrings on the cluster. See Cluster-Client Connections for the procedure.
LunaSH
Prerequisites
>On Appliance A (where the cluster is created):
•The appliance must be configured for clustering (see Configuring the Luna Network HSM 7 for Clustering).
•A cluster must be created on the appliance, and the appliance must be authorized as a member (see Creating a Cluster).
•Thales recommends that new members join the primary cluster member. If there are already multiple members in the cluster, and you are not sure which is currently the primary member, use (the primary member is marked with a "P"):
lunash:> cluster member list -clusterid <UUID>
>On Appliance B (that will join the existing cluster):
•The appliance must be configured for clustering (see Configuring the Luna Network HSM 7 for Clustering).
NOTE If a cluster already exists on appliance B, it will not be allowed to join appliance A's cluster.
>Both appliances must be accessible to each other over the network. All members of a cluster must be able to communicate bi-directionally with all other members.
>If you are joining a cluster on a multifactor quorum-authenticated HSM, you require a local or remote Luna PED. Refer to Multifactor Quorum Authentication for setup procedures.
>Both appliances should have NTP configured, or the appliance clocks must be synchronized manually; if the appliance times are out of sync by more than 30 seconds, the new member will not be allowed to join the cluster. See Setting the System Date and Time.
CAUTION! Ensure that you add only one member to the cluster at a time. Undesired behavior can result from adding more than one member simultaneously.
To join an existing cluster
1.On appliance A, get the cluster ID you wish to join.
lunash:> cluster list
2.On appliance B, join appliance A's cluster by specifying its cluster ID, IP address, admin user password, the label of the partition on appliance B that will authorize appliance B as a member, and that partition's CO password/PED key.
lunash:> cluster join -clusterid <string> -clusteripaddress <ipaddress> -remotepassword <adminpassword> -partition <name> [-copassword <password>] [-force]
3.Confirm that both cluster members are now listed. The new member is listed as unauthorized. An asterisk indicates the local member.
lunash:> cluster member list -clusterid <UUID>
New members joining an existing cluster are authorized automatically.
4.You must run the provided LNHClientRegistration script on any Luna HSM Client computers that will query this member to create or use keyrings on the cluster. See Cluster-Client Connections for the procedure.
Moving a Member to a Different Affinity Group
Luna Network HSM 7s within a cluster can be added to an affinity group. Operations from a connected client application are load-balanced between members of the same group only. This allows you to use the other members, which can be at a remote location with greater latency, as backup or standby members for a specific client. If all members of a client's preferred group are unavailable, operations then fail over to other members of the cluster. The state of keyrings and objects stored on them is always synchronized across all members of the cluster, regardless of group. You can create up to 64 affinity groups in a cluster.
New members are added to the local group by default and can be moved to another group using REST API or LunaSH.
REST API
To move a member to another group
Move the member to the new group, specifying the group name. If the group does not already exist, a new group is created with the specified name.
PUT /api/clusters/{clusterID}/members/{memberID}
{
"action": "config",
"group": "newgroup"
}
LunaSH
To move a member to another group
1.Move the member to the new group, specifying the group name. If the group does not already exist, a new group is created with the specified name.
lunash:> cluster member config -memberid <UUID> -group <name>
2.[Optional] Check the new group configuration.
lunash:> cluster group list
Promoting a Member to Primary
This procedure allows you to promote any member of the cluster to be the primary member (see The Primary Member for more information) using REST API or LunaSH. You can promote any active cluster member to primary from any other active member.
REST API
Prerequisites
>The member being promoted must have read-write privileges on the cluster. If you are promoting another member because the original primary member is unrecoverable, you must first delete the original primary from the cluster before you can promote your desired member (see Deleting an Unrecoverable Member From a Cluster).
To promote a member to primary
1.Check that the member you wish to promote has read-write privileges ("mode": "Read-Write").
GET /api/clusters/{clusterID}/members/{memberID}
2.Promote the member to primary.
PUT /api/clusters/{clusterID}/members/{memberID}
{
"action": "config",
"primary": true
}
NOTE If you have automatic scheduled backups configured, you must disable and re-enable backups after promoting a new primary member. You do not need to re-configure the schedule.
LunaSH
Prerequisites
>The member being promoted must have read-write privileges on the cluster. If you are promoting another member because the original primary member is unrecoverable, you must first delete the original primary from the cluster before you can promote your desired member (see Deleting an Unrecoverable Member From a Cluster).
To promote a member to primary
1.Ensure the member you wish to promote has read-write privileges (Mode: Read-Write).
lunash:> cluster member show -memberid <UUID>
2.Promote the member to primary.
lunash:> cluster member config -memberid <UUID> -primary
3.[Optional] Check the new primary status (Primary: True).
lunash:> cluster member show -memberid <UUID>
NOTE If you have automatic scheduled backups configured, you must disable and re-enable backups after promoting a new primary member. You do not need to re-configure the schedule.
lunash:> cluster backup disable
lunash:> cluster backup enable -backuppassword <password>
Removing a Functioning Member From a Cluster
This procedure allows you to remove a functioning member from the cluster using REST API or LunaSH. This could be done for scheduled maintenance or updates, and assumes that the member is accessible and functioning properly within the cluster. This operation must be done on the member appliance that is leaving the cluster.
CAUTION! Ensure that all other members are online before you proceed. If another member is offline, it will not be updated with the new cluster configuration, and its cluster service may need to be factory reset as a result.
REST API
Prerequisites
>If you are removing a member from a multifactor quorum-authenticated HSM cluster, you require a local or remote Luna PED. Refer to Multifactor Quorum Authentication for setup procedures.
To remove a member from the cluster
1.Check that the member you wish to remove can communicate with the other cluster members ("visibleToServicingNode": true).
GET /api/clusters/{clusterID}/members/{memberID}
2.Remove the member from the cluster, specifying the label of the partition containing the cluster SMK and the CO password (or "password": "" and present the CO PED key).
PATCH /api/clusters/{clusterID}/members/{memberID}
{
"authentication": [
{
"type": "CO",
"password": "IamtheCO",
"partitionLabel": "clusterpar"
}
],
"action": "leave"
}
LunaSH
Prerequisites
>If you are removing a member from a multifactor quorum-authenticated HSM cluster, you require a local or remote PED key. Refer to Multifactor Quorum Authentication for setup procedures.
To remove a member from the cluster
1.Check that the member you wish to remove is the local member (Local: True) and can communicate with the other cluster members (Visible: True).
lunash:> cluster member show -memberid <UUID>
2.Remove the member from the cluster, specifying the local partition and its Crypto Officer password/PED key.
lunash:> cluster leave -memberid <UUID> -partition <label> [-copassword <password>]
Deleting an Unrecoverable Member From a Cluster
This procedure allows you to delete a member from the cluster that has become unrecoverable and cannot be accessed via SSH or REST API. This action removes references to the unrecoverable member from the other members of the cluster. This operation must be done on another cluster member using REST API or LunaSH.
CAUTION! Ensure that all members except those being deleted are online before you proceed. If another member is offline, it will not be updated with the new cluster configuration, and its cluster service may need to be factory reset as a result.
REST API
Prerequisites
>If you are deleting a member from a multifactor quorum-authenticated HSM cluster, you require a local or remote PED key. Refer to Multifactor Quorum Authentication for setup procedures.
To delete an unrecoverable member from the cluster
1.Check that the member you wish to remove is not visible to the other cluster members ("visibleToServicingNode": false).
GET /api/clusters/{clusterID}/members/{memberID}
2.Delete the member from the cluster, specifying the Crypto Officer password (or "password": "" and present the CO PED key) for the local cluster partition.
DELETE /api/clusters/{clusterID}/members/{memberID}
{
"authentication":
[{
"type": "CO",
"password": "IamtheCO"
}]
}
NOTE If the member you just removed was the primary member, another member is automatically promoted to primary and read-write cluster function resumes. If you prefer to use a different member as primary, see Promoting a Member to Primary.
When you are able to re-establish an SSH, REST, or serial connection to the deleted member appliance, you must factory reset its cluster service before rejoining it to the cluster (serviceid: cluster, actionid: factoryreset):
POST /api/lunasa/services/{serviceid}/actions/{actionid}
lunash:> sysconf config factoryReset -service cluster
LunaSH
Prerequisites
>If you are deleting a member from a multifactor quorum-authenticated HSM cluster, you require a local or remote PED key. Refer to Multifactor Quorum Authentication for setup procedures.
To delete an unrecoverable member from the cluster
1.Check that the member you wish to remove is not visible to the other cluster members (Visible: False).
lunash:> cluster member show -memberid <UUID>
2.Delete the member from the cluster, specifying the Crypto Officer password/PED key for the local cluster partition.
lunash:> cluster member delete -memberid <UUID> [-copassword <password>]
NOTE If the member you just removed was the primary member, another member is automatically promoted to primary and read-write cluster function resumes. If you prefer to use a different member as primary, see Promoting a Member to Primary.
When you are able to re-establish an SSH, REST, or serial connection to the deleted member appliance, you must factory reset its cluster service before rejoining it to the cluster (serviceid: cluster, actionid: factoryreset):
POST /api/lunasa/services/{serviceid}/actions/{actionid}
lunash:> sysconf config factoryReset -service cluster
