The Crypto Traffic Controller (CTC) service on the Luna Network HSM 7 is an optional feature that allows you to regulate the bandwidth that is consumed by each client. A client here refers to any machine communicating with the Luna Network HSM 7 through one of its network interfaces. Named classes (defined by you) predetermine sets of minimum and maximum bandwidth parameters. Clients are then assigned, one or more, to appropriate classes. This feature helps to address the noisy neighbor problem, wherein one or more clients, connected to a network interface of the Luna Network HSM 7, dominate the bandwidth thus restricting others. A lack of control over the network bandwidth allocation could result in failure to meet QoS constraints established for your operations, or mandated by your industry niche.
By default, any client that is not explicitly assigned to a named class (for a given interface) falls into the "default" class and is therefore not bound by any bandwidth limitations. See Default class.
If you already have an overview of problematic high usage, by time-frame or by other activity patterns, you can quickly impose general constraints to limit the worst offenders, and then fine-tune at leisure.
Monitoring the volumes of bandwidth usage provides the factual information you need
>when applying constraints on excess bandwidth usage for some clients,
>when applying assurance of minimum bandwidth availability for other clients,
>when negotiating with your users/clients, such as for adjustments of contracts, or
>when tracking respective usage to guide decisions around additional resources and their allocation.
The CTC service is added to the service and status commands, and the commands for implementing and managing CTC are at sysconf ctc.
Bandwidth sharing
The service is referred-to as "crypto" traffic control, because whether you reserve an interface for administrative traffic, or allow it on any interface, client traffic resulting from cryptographic operations by the HSM is expected to greatly exceed the occasional administrative exchange, and therefore will be the bulk of any contention among users for bandwidth. If a connection is being used for SSH, that device would be visible under the "measurement show" output (see below), though only if such traffic is exchanged during the measurement period. Typically SSH pings the client at intervals to keep a session alive, but this is unlikely to be a high-frequency exchange compared with client crypto traffic.
CTC guarantees a specific level of bandwidth, defined as a class minimum amount. CTC controls egress traffic. Allotment of ingress traffic bandwidth is not addressed.
CTC gives you tools:
>to measure average bandwidth-usage values per connected client during the time span of interest,
>to measure peak bandwidth usage by connected clients during that time span,
>to configure relative weights, that we call classes, of a client's entitlement out of the total supported bandwidth (for an individual interface eth0, eth1, eth2, eth3, bond0, bond1),
>to assign specific clients to become members of a class, sharing that defined minimum and maximum entitlement of the total bandwidth provided by the particular interface.
Different clients can be assigned to different classes for a given interface.
The class provides instructions to the underlying system to treat the members as fairly as possible. Together with other classes, CTC might ensure that members of a class X are each able to access at least the minimum bandwidth defined for class X, by borrowing unused capacity from other classes. So, any capacity of (say) class Y and class Z that exceeded the minimums defined for those classes might be borrowed to provide class X with at least the minimum processing of its packets. CTC will continue to allot and reassign bandwidth, according to your settings, as long as there is any capacity to do so. Bandwidth could also be borrowed from Default class, as any Clients not explicitly assigned to named classes are unprotected.
For a class to be effective for a client, it must be applied against an interface that is being used for egress traffic to the client. See Impact of route metric on CTC and Impact of default route on CTC
Default class
The default class is a special case. Clients of the Luna Network HSM 7 fall into the default class if they are not assigned to a defined (named) class. Clients in the default class have no pre-imposed restrictions. This means that even if CTC is active, if no clients are assigned to a named class, then they are all in the default class and bandwidth usage is free-for-all. That would be the initial stage when you might be measuring to see what the actual bandwidth usage is, before deciding how to determine the weighting for named classes that you will create.
If you have a clear idea of how all clients are using the HSM, and a given network interface in particular, then create a class that restricts the offending "noisy neighbor" to reasonable bandwidth usage, and just leave the other clients in default class.
However, to be prudently conservative, if, for example, service-level-agreements promise availability of bandwidth to some or all clients, or if there is doubt about what demands some other clients might make, then in addition to the class for the high-volume user(s), consider also creating a class or classes for the other users, so that all are under constraints, and all have minimum-bandwidth allotments to rely on, to avoid surprises for everyone.
What is the basic approach?
With CTC classes you are instructing the system how to address egress-traffic packet handling for members of the class, relative to other users. You can approach the use of CTC in whatever way works for you (within the limits) but the original intent is that:
>You have some number of clients that are using a particular interface on your Luna Network HSM 7.
>Most of those clients have steady small-to-moderate bandwidth usage - the timely availability of that much bandwidth is important to their functioning, but the nature of their usage is that they generally don't demand more; they are consistent.
>Then, you have (for example) one or two clients
•that might have spikes of very high usage, or
•their overall constant demand is growing
(these are the noisy neighbors), and they are putting pressure on the smaller-usage clients, sometimes crowding them out.
>So, you address the issue by
•verifying the bandwidth usage of your clients during a representative time to gain insight into normal and problematic usage
•creating a class that applies to the clients that are heavy or erratic users, ensuring them of a suitable high minimum amount of bandwidth, but capping their maximum usage, to prevent excess usage that crowds out the other clients;
•creating a class or classes for the lower-volume users, with suitable minimum and maximum settings, to ensure that everyone is treated fairly when bandwidth becomes tight.
All of this takes place within the context of the total bandwidth that the given device supports.
That is:
>for the big-usage clients, -min is important to them, because they obviously need a lot of bandwidth; -max for the big users curtails their high usage enough to let smaller users have some needed bandwidth (as determined by the next point)
>for the smaller steady-usage clients, -min is most important because they need consistency of access to bandwidth, but -max is less important; they are unlikely to require that much more than they normally use.
The CTC service can provide insight and control for best management of bandwidth usage among your clients.
CTC is a service
CTC is an appliance service, added to the Luna Network HSM 7 beginning with appliance software version 7.8.3. An appliance admin or operator role can control the state of the service with the usual service commands, service start, service stop, and service restart. CTC measures and controls egress traffic from the Luna Network HSM 7 and does not address inbound traffic.
This section discusses the use and behavior of CTC in the context of other networking commands.
Executing any of the following network commands restarts CTC. Network-level changes affect the interface state, which flushes the underlying traffic rules. The restart of CTC service reinstates your rules. When the CTC service has been started, then as long as the Luna Network HSM 7 continues running, the CTC service automatically resumes after any of the silent restarts invoked by these commands:
network dns delete searchdomain
An implication of this behavior is that the restart (due to those network commands) also immediately launches any new CTC rules you might have added, without waiting for you to explicitly issue service restart ctc. That is worth keeping in mind if you had any reason to delay your new CTC rules coming into force.
Separately, the sysconf ctc enable command does include a start of CTC service at the time it is run, but the intended use of sysconf ctc enable is only to include CTC among services that are relaunched at reboot or system power-up. We recommend that you use the service start and service stop commands for CTC while you are setting up classes and clients, and use sysconf ctc enable one time, when you are satisfied that your configuration is working as you want it. Thereafter, use service start ctc and service stop ctc whenever you want to make adjustments during normal operation.
NOTE After making CTC configuration changes, restart the service, to have those changes become operational.
For other network commands, noteworthy behavior includes:
>If you run network interface bonding enable, then CTC is disabled over the slave interfaces upon successful command execution. However, in case of failure of bonding enable, CTC is restarted on the slave interfaces, if it was already running. You must explicitly start CTC on the bonded interface if needed.
>If you run network interface bonding disable, then CTC is disabled for the bond, upon successful command execution. In case of failure of bonding disable, CTC is restarted on the bond, if it was already running.
>Restarting the network service ( service restart network ) stops CTC and you must restart it ( service start ctc ). For the same reason, it is not advisable to restart network service while measurement is running.
NOTE After factory reset of network services, it is normal for NTLS service to be still running. However, the CTC service stops (if it was running). Since none of the devices has a gateway, after reconfiguring of eth0, CTC is still inactive. Run sysconf ctc enable to resume CTC operation.
Included in backup of services configuration
CTC service configuration is included in the information that is backed-up and restored with the sysconf config backup and sysconf config restore commands.
Measurement
Traffic measurement requires that CTC service is running, and is controlled by sysconf ctc measurement enable, sysconf ctc measurement disable, and sysconf ctc measurement show commands. Two values are important to the traffic measurement aspect of CTC:
| Parameter | Which command | Description |
|---|---|---|
| -interval | sysconf ctc measurement enable | Sets the time between consecutive measurements, when CTC measurement is enabled. |
| -duration | sysconf ctc measurement show | Uses the duration value to calculate the starting point from which the minimum/maximum and average bit rates for clients are to be calculated, for display by the 'show' command. The -duration value determines how far back in time, before the present moment (the moment that the show command is launched), the starting point for measurement collection should be set, for display. In other words, it's the span of recent time over which you want to collect and summarize measurements of bandwidth-usage, per interface. |
Measurements are stored as log entries in memory, so sysconf ctc measurement show can look back in time as far as such entries are available, or as far back as the span of time indicated by -duration, whichever is less.
NOTE If you restart/reboot the Luna Network HSM 7 appliance, while using CTC measurement, then existing measurement records are lost. If you have an intentional restart coming up, consider running sysconf ctc measurement show first, to capture bandwidth usage information that is about to be deleted.
Two options for starting CTC
Issue the service start -ctc command to have CTC service running for current session only. With that launch method, the service stops and does not resume if the appliance is restarted or the power is cycled.
For resilient, persistent CTC service, start the service with sysconf ctc enable command, in which case the service resumes after restarts or power cycles, and requires that you issue sysconf ctc disable command when you want the service to no longer resume automatically after reboot. But see above about the CTC service does not automatically restart after a service restart network command has been run.
The intended approach is to initially run sysconf ctc enable to establish ongoing CTC, and then use service start -ctc and service stop -ctc commands if needed for short-term adjustments, because using the enable command performs additional background operations that are not needed every time the service is started or stopped.
Ongoing measurement (started with sysconf ctc measurement enable) allows you to quantify how well you are meeting Service Level Agreements with your clients, and helps suggest the scope of changes that might be necessary, before action becomes urgent.
Records
Measurements are recorded in appliance memory, while the service is active. The records reside in memory only, and do not go to syslog, as is done for other services. For example, if you set an -interval of 5, then every 5 seconds, a traffic measurement entry would be added to memory. That goes on as long as measurement is active. Recording is independent of the sysconf ctc measurement show command, that tells CTC to go back -duration seconds from the moment the sysconf ctc measurement show command is run, to gather however many measurements have been taken over that timespan, for display in the command output.
NOTE The archive of CTC measurement records is maintained in memory and is rotated when more space is needed. Do not count on older records being available, as they could have been rotated out. If a restart is performed, the records are wiped entirely.
Caveats
The memory available for CTC measurement records is part of appliance system memory and therefore finite. If the measurement records begin to approach the limit of reserved memory, CTC begins overwriting earliest entries.
The number of records being generated depends on:
>the interval you set - obviously the smaller the interval, the more frequent the addition of records to CTC memory
>the number of clients that connect and generate traffic.
The memory usage and availability are dynamic, so we cannot provide specific numbers. Therefore, if you set a very short interval, against an interface that is seeing high usage, and let measurement run for days, the earliest records might be lost. Calculations are made against available records at the time the sysconf ctc measurement show command is run.
Who can access CTC?
The CTC commands are accessible by the built-in appliance admin and operator users, or by any custom-named administrative users that you create with roles that are given explicit access to the CTC commands. See Appliance Users and Roles. and Creating Custom Appliance Roles. View/show commands can be run by users with the monitor role privileges.
How to use CTC to measure and manage client usage of HSM appliance communication bandwidth
Recommended approach
>Perform a measurement first, to assess the situation.
>Find the egress interface for a given client. See Impact of route metric on CTC and Impact of default route on CTC
>Create a class over that interface, with initial low and high (min and max) bandwidth boundaries.
>Assign the client to the class.
>Perform further measurements and adjust the class parameters, or create additional classes for other clients or groups of clients until the bandwidth usage by all clients is suitably managed.
Impact of route metric on CTC
The metric for each route in a routing table is an estimate of the cost associated with using that route in terms of link speed, hop count, or time delay. When the client is on the same subnet as the interface through which traffic is entering the Luna Network HSM 7, then, the client is served through the interface with the lowest metric.
For example, say that a client (at address 192.168.143.92) is assigned to ETH1 (192.168.143.1) and ETH2 (192.168.142.20) has the lowest metric value. Then, the egress traffic for this client flows out of ETH2, and this occurs even if ETH0 is set as a default route. The result is that there is no restriction for this client on ETH2. and this client can have unlimited bandwidth. You would need to create a class over the actual egress-traffic route and assign the client to that class to control its usage of that interface.
You can see the current metric values applied to each interface in the output of the network show command.
If needed, you can adjust metric values with network route metric, but consult with your network administrator.
Impact of default route on CTC
When the client is on a different subnet from the interface through which traffic is entering, then, the client is served through the interface configured as the default route.
For example, say that a client (192.168.79.157) is assigned to ETH2 (192.168.142.20), and ETH0 (192.168.143.184) is the default route. Then, the egress traffic for this client flows out of ETH0. For such clients, you would need to create classes over that interface (the default route).
Preconditions
Partitions are already created and configured.
Clients are already registered and connected.
Operations by your client applications against respective partitions are proceeding normally.
To perform ongoing measurement of bandwidth usage to track the success of your class definitions and client assignments
1.Enable CTC to take a measurement, every <interval>.
sysconf ctc measurement enable-interval <how often measurements are made>
(This automatically restarts the service)
2.Once measurement has been going on long enough to have a useful accumulation of measurements, view the usage summary over a recent time period.
sysconf ctc measurement show -duration <time span to view and summarize measurements>
3.Measurement continues until you explicitly disable. That is, you can leave it running and making entries indefinitely, until you run:
sysconf ctc measurement disable
To manage bandwidth usage by clients on a network interface
1.Configure classes. (This can be done without starting CTC service. The service uses any defined classes to determine action applied to assigned clients, and does not perform any management of Default clients [clients not assigned to a defined class].)
sysconf ctc class define -class <name of new class> -interface <eth?|bond?> -minimum <minimum_bandwidth> [-maximum <maximum_bandwidth>]
CAUTION! Avoid setting CTC limits so low that normal services like NTLS cannot function.
2.Assign clients to classes, by reference to the -ip address from which each client connects, and the -interface/device they use to connect.
sysconf ctc class assign -interface <eth#> -class <classname> -ip <ip address of client>
TIP This is where you would check the routing table portion of the network show output (while keeping in mind Impact of route metric on CTC and Impact of default route on CTC) to determine the interface to which this class should be assigned.
3.Start the CTC service.
Use sysconf ctc enableif CTC service has not previously been enabled, or if you have since run sysconf ctc disable.
Otherwise, use service start ctc, if CTC has been enabled and not disabled.
NOTE Changes made to CTC configuration come into effect the next time CTC service is restarted. This gives you control over when changes become effective.
4.Keep track of classes you have created, according to the interface to which they apply.
sysconf ctc class show -interface <ethX or bondX>
5.Keep track of clients you have created, and the classes to which they have been assigned.
sysconf ctc client show -interface <ethX or bondX>
Example
A partition exists and is registered with a client. We could start by checking the CTC situation.
[local_host] lunash:>sysconf ctc client show No clients are configured Command Result : 0 (Success) [local_host] lunash:>sysconf ctc class show No class configured. Command Result : 0 (Success) [local_host] lunash:>service status ctc ctc@eth0 is inactive ctc@eth1 is inactive ctc@eth2 is inactive ctc@eth3 is inactive CTC is inactive on all interfaces Command Result : 0 (Success)
Create (define) a class and assign a client to that class.
[local_host] lunash:>sysconf ctc class define -class Test@0 -interface eth0 -min 10 -max 20
Command Result : 0 (Success)
[local_host] lunash:>sysconf ctc class assign -class Test@0 -interface eth0 -ip 192.168.143.48
Command Result : 0 (Success)
[local_host] lunash:>sysconf ctc class show
eth0:
class min max
---------------------------------------------
Test@0 10kbit 20kbit
---------------------------------------------
Command Result : 0 (Success)
[local_host] lunash:>sysconf ctc client show
eth0:
client class
------------------------------
192.168.143.48 Test@0
------------------------------
Command Result : 0 (Success)
Start the CTC service.
[local_host] lunash:>service start ctc Please be patient while the operation is running... Starting ctc@eth0: [ OK ] Starting ctc@eth1: [ OK ] Starting ctc@eth2: [ OK ] Starting ctc@eth3: [ OK ] Command Result : 0 (Success)
Enable CTC measurement.
[local_host] lunash:>sysconf ctc measurement enable -interval 5 Command Result : 0 (Success)
On the client, we generate some traffic for this example.
[root@AA3578 bin]# ./multitoken -mode rsasigver -slots 0
multitoken (64-bit) v10.6.0-402. Copyright (c) 2023 Thales Group. All rights reserved.
Warning: packet size not specified. Using default packet size of 16.
Warning: Key size not specified. Using default key size of 2048.
Initializing library...Finished Initializing
...done.
Do you wish to continue?
Enter 'y' or 'n': y
Constructing thread objects.
Logging in to tokens...
slot 0...
Enter password: ********
Serial Number 1335066958556
Please wait, creating test threads.
Test threads created successfully. Press ENTER to terminate testing.
RSA sign/verify 2048-bit : (packet size = 16 bytes)
Using token objects.
Logged in as Crypto Officer.
+ operations/second | elapsed
0, 0 | total average | time (secs)
------ | ------- ---------- | ------------
2.0 | 2.0 2.100* | 70
Waiting for threads to terminate.
Back at the Luna Network HSM 7 we can take a look at what CTC has recorded.
[local_host] lunash:>sysconf ctc measurement show -duration 60 Measurement status: enabled eth0: client min max avg class -------------------------------------------------------------------------------------- 192.168.143.48 6400 8552 7004 Test@0 -------------------------------------------------------------------------------------- eth1: client min max avg class -------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------- eth2: client min max avg class -------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------- eth3: client min max avg class -------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------- Command Result : 0 (Success)
If necessary, modify the class by repeating the sysconf ctc class define command for the same class name and interface, but with a different -min or -max value. This is equivalent to an edit/overwrite of the relevant parameters of the existing class. The change goes into effect as soon as the CTC service is restarted.
