Load Balancing

The default behavior of the client library is to attempt to load-balance the application’s cryptographic requests across each active member of an HA group. Any standby members in the HA group are not used to perform cryptographic operations, and are therefore not part of the load-balancing scheme (see Standby Members).

The top-level algorithm is a round-robin scheme that is modified to favor the least busy device in the set. As each new command is processed, the SafeNet Luna HSM client looks at how many commands it has scheduled on every device in the group. If all devices have an equal number of outstanding commands, the new command is scheduled on the next device in the list – creating a round-robin behavior. However, if the devices have a different number of commands outstanding on them, the new command is scheduled on the device with the fewest commands queued – creating a least-busy behavior. This modified round-robin has the advantage of biasing load away from any device currently performing a lengthy command. In addition to this least-busy bias, the type of command also affects the scheduling algorithm, as follows:

>Single-part (stateless) cryptographic operations are load-balanced.

>Multi-part (stateful) commands that involve cryptographic operations are load-balanced.

>Multi-part (stateful) commands that involve information retrieval are not load-balanced. Multi-part operations carry over several individual commands. The cost of distributing the commands to different HA group members is generally greater than the benefit. For this reason, multi-part commands that involve information retrieval are all targeted at one member. Multi-part operations are typically not used, or are infrequent actions, so most applications are not affected by this restriction.

> Key management commands are not load-balanced. Key management commands affect the state of the keys stored in the HSM. As such, these commands are targeted at all HSMs in the group. That is, the command is performed on the primary HSM and then the result is replicated to all members in the HA group. Key management operations are also an infrequent occurrence for most applications .

It is important to understand that the least-busy algorithm uses the number of commands outstanding on each device as the indication of its busyness. When an application performs a repeated command set, this method works very well. When the pattern is interrupted, however, the type of command can have an impact. For example, when the HSM is performing signing and an atypical asymmetric key generation request is issued, some number of the application’s signing commands are scheduled on the same device (behind the key generation). Commands queued behind the key generation therefore have a large latency driven by the key generation. However, the least-busy characteristic automatically schedules more commands to other devices in the HA group, minimizing the impact of the key generation.

It is also important to note that the load-balancing algorithm operates independently in each application process. Multiple processes on the same client or on different clients do not share their “busyness” information while making their scheduling choice. In most cases this is reasonable, but some mixed use cases might cause certain applications to hog the HSMs.

Finally, when an HA group is shared across many servers, different initial members can be selected while the HA group is being defined on each server. The member first assigned to each group becomes the primary. This approach optimizes an HA group to distribute the key management and/or multi-part cryptographic operation load more equally.

In summary, the load-balancing scheme used by SafeNet is a combination of round-robin and least-busy for most operations. However, as required, the algorithm adapts to various conditions and use cases so it might not always emulate a round-robin approach.

Example

When the client makes a request on a virtual HA slot, the request goes to the first member in the HA group, as listed in the Chrystoki.conf file (Linux/UNIX) or Crystoki.ini file (Windows), unless it is busy. A member is busy if it has not yet responded to the most recent request that was sent to it. If the primary member is busy, the client sends the request to the next non-busy member of the HA Group.

If you add network latency, or if you increase the key-size, or if you interleave other crypto operations, then performance may drop for individual active members as they become busier.

If you have any group members set to "Standby" status, then they do not contribute to group performance, even if the client can saturate the active members.