Monitoring HA Status

Rapid HA group status checking

To check the status of HA group members, the old PKCS#11 Extensions function CA_GetHAState has been superseded by CA_GetCurrentHaState(), which is invoked programmatically and is demonstrated by CKDemo option 49, as in the following example.

Environment Requirements

When an HA group member becomes unavailable, the CA_GetCurrentHaState() function (once called) detects its unavailability within 3 seconds (in optimal conditions, depending on various factors including network hazards). For performance near the speed that was verified in testing, your operational environment must embody good IT conditions (network, CPU, RAM, storage) and reasonable perturbations (<=20% of optimal conditions). When those conditions are met, CA_GetCurrentHaState() can be repeated quickly and continually for ongoing monitoring. Requires Luna HSM Client 10.7.0 or newer.

When a previously unavailable HA group member becomes available again, the recovery process must detect the member and then confirm that it can be used for PKCS#11 operations. The recovery process is affected by several configuration file parameters, including AutoReconnectInterval and statusTimeout; if the status takes longer than the time specified by statusTimeout, the API terminates and reports the status of all members fetched so far.

Example of HA Current Status check using CKDemo

Assume that the group (12 members in this example) is already created and the Crypto Officer is logged in.

1.Initially, all members are connected and working. Check the status.

(TITLE) menu titles, (99 or FULL) Full Help, (NONE) No help, (0 or EXIT) Quit


Status: Doing great, no errors (CKR_OK)
Enter your choice : 49

Slots available:
        slot#0 - Net Token Slot
        slot#1 - Net Token Slot
        slot#2 - Net Token Slot
        slot#3 - Net Token Slot
        slot#4 - Net Token Slot
        slot#5 - Net Token Slot
        slot#6 - Net Token Slot
        slot#7 - Net Token Slot
        slot#8 - Net Token Slot
        slot#9 - Net Token Slot
        slot#10 - Net Token Slot
        slot#11 - Net Token Slot
        slot#12 - Net Token Slot
        slot#19 - HA Virtual Card Slot
Select a slot (last selected slot = 19): 19

HA group 11327020333032 status:

     HSM 1327020333032   - CKR_OK
     HSM 1327024989635   - CKR_OK
     HSM 1378778575417   - CKR_OK
     HSM 1378780903721   - CKR_OK
     HSM 1305890956073   - CKR_OK
     HSM 1305921224055   - CKR_OK
     HSM 1372948497179   - CKR_OK
     HSM 1459759386390   - CKR_OK
     HSM 1238656463702   - CKR_OK
     HSM 1485871338183   - CKR_OK
     HSM 1358801709927   - CKR_OK
     HSM 1259264300119   - CKR_OK
     HSM 1382217483713   - CKR_OK
Status: Doing great, no errors (CKR_OK)

2.Some members are disconnected or disabled, and the scan is repeated.

Enter your choice : 49

Slots available:
        slot#0 - Net Token Slot
        slot#1 - Net Token Slot
        slot#2 - Net Token Slot
        slot#3 - Net Token Slot
        slot#4 - Net Token Slot
        slot#5 - Net Token Slot
        slot#6 - Net Token Slot
        slot#7 - Net Token Slot
        slot#8 - Net Token Slot
        slot#9 - Net Token Slot
        slot#10 - Net Token Slot
        slot#11 - Net Token Slot
        slot#12 - Net Token Slot
        slot#19 - HA Virtual Card Slot
Select a slot (last selected slot = 19): 19

HA group 11327020333032 status:

     HSM 1327020333032   - CKR_OK
     HSM 1327024989635   - CKR_OK
     HSM 1378778575417   - CKR_TOKEN_NOT_PRESENT
     HSM 1378780903721   - CKR_TOKEN_NOT_PRESENT
     HSM 1305890956073   - CKR_OK
     HSM 1305921224055   - CKR_OK
     HSM 1372948497179   - CKR_OK
     HSM 1459759386390   - CKR_TOKEN_NOT_PRESENT
     HSM 1238656463702   - CKR_TOKEN_NOT_PRESENT
     HSM 1485871338183   - CKR_TOKEN_NOT_PRESENT
     HSM 1358801709927   - CKR_TOKEN_NOT_PRESENT
     HSM 1259264300119   - CKR_OK
     HSM 1382217483713   - CKR_TOKEN_NOT_PRESENT
Status: Doing great, no errors (CKR_OK)

The HA group remains in operation, with reduced functional members, with non-functional members identified.

Assume that the connection, routing, or other problems are corrected, as indicated by log entries...

Mon Nov 27 16:56:28 2023 : [14686] HA group: 11327020333032 recovery attempt #3 succeeded for member: 1378778575417
Mon Nov 27 16:56:31 2023 : [14686] HA group: 11327020333032 recovery attempt #3 succeeded for member: 1378780903721
Mon Nov 27 16:56:33 2023 : [14686] HA group: 11327020333032 recovery attempt #3 succeeded for member: 1459759386390
Mon Nov 27 16:56:35 2023 : [14686] HA group: 11327020333032 recovery attempt #3 succeeded for member: 1238656463702
Mon Nov 27 16:56:38 2023 : [14686] HA group: 11327020333032 recovery attempt #3 succeeded for member: 1485871338183
Mon Nov 27 16:56:41 2023 : [14686] HA group: 11327020333032 recovery attempt #3 succeeded for member: 1358801709927
Mon Nov 27 16:56:48 2023 : [14686] HA group: 11327020333032 recovery attempt #5 succeeded for member: 1382217483713

3.Check status again.

NOTE   Checking could have been performed every few seconds while corrective actions were in progress.

Enter your choice : 49

Slots available:
        slot#0 - Net Token Slot
        slot#1 - Net Token Slot
        slot#2 - Net Token Slot
        slot#3 - Net Token Slot
        slot#4 - Net Token Slot
        slot#5 - Net Token Slot
        slot#6 - Net Token Slot
        slot#7 - Net Token Slot
        slot#8 - Net Token Slot
        slot#9 - Net Token Slot
        slot#10 - Net Token Slot
        slot#11 - Net Token Slot
        slot#12 - Net Token Slot
        slot#19 - HA Virtual Card Slot
Select a slot (last selected slot = 19): 19

HA group 11327020333032 status:

     HSM 1327020333032   - CKR_OK
     HSM 1327024989635   - CKR_OK
     HSM 1378778575417   - CKR_OK
     HSM 1378780903721   - CKR_OK
     HSM 1305890956073   - CKR_OK
     HSM 1305921224055   - CKR_OK
     HSM 1372948497179   - CKR_OK
     HSM 1459759386390   - CKR_OK
     HSM 1238656463702   - CKR_OK
     HSM 1485871338183   - CKR_OK
     HSM 1358801709927   - CKR_OK
     HSM 1259264300119   - CKR_OK
     HSM 1382217483713   - CKR_OK
Status: Doing great, no errors (CKR_OK)

(TITLE) menu titles, (99 or FULL) Full Help, (NONE) No help, (0 or EXIT) Quit


Status: Doing great, no errors (CKR_OK)
Enter your choice :

And the log entries for that action say:

Mon Nov 27 16:56:59 2023 : [14686] HA group: 11327020333032 Initializing HA State API
Mon Nov 27 16:56:59 2023 : [14686] HA group: 11327020333032 Retrieved Current HA Status