Audit Logging

Each event that occurs on the HSM can be recorded in the HSM event log, allowing you to audit your HSM usage. The HSM event log is viewable and configurable only by the audit user role. This audit role is disabled by default and must be explicitly enabled.

This chapter describes how to use audit logging to provide security audits of HSM activity. It contains the following sections:

>Audit Logging General Advice and Recommendations

>Logging In as Auditor

>Configuring and Using Audit Logging

>Remote Audit Logging

>Changing the Auditor Credentials

>Audit Log Categories and HSM Events

>Audit Log Troubleshooting

Audit Logging Features

The following list summarizes the functionality of the audit logging feature:

>Log entries originate from the Luna Network HSM 7 (cryptographic module - the feature is implemented via HSM firmware (rather than in the library), for maximum security.

>Log origin is assured.

>Logs and individual records can be validated by any Luna Network HSM 7 that is a member of the same domain.

>Audit Logging can be performed on password-authenticated and multifactor quorum-authenticated (both FIPS 140-3 level 3) configurations, but these configurations may not validate each other's logs - see the "same domain" requirement, above.

NOTE   The "same domain" requirement still applies, but with the introduction of Extended Domain Management [ see Universal Cloning and Domain Planning ] (Partition Policy 44) in firmware version 7.8.0, you can change/add domains, such that the verifying HSM could be given the same domain as the original logging HSM.

Thus, from Luna HSM firmware version 7.8.0 onward, it is possible

for a Multifactor Quorum HSM log to be validated by a Password-authenticated HSM
or

for a Password-authenticated HSM log to be validated by a Multifactor Quorum HSM.

>Each entry includes the following:

When the event occurred

Who initiated the event (the authenticated entity)

What the event was

The result of the logging event (success, error, etc.)

>Multiple categories of audit logging are supported, configured by the audit role.

>Audit management is a separate role - the role creation does not require the presence or co-operation of the Luna Network HSM 7 SO.

>The category of audit logging is configurable by (and only by) the audit role.

>Audit log integrity is ensured against the following:

Truncation - erasing part of a log record

Modification - modifying a log record

Deletion - erasing of the entire log record

Addition - writing of a fake log record

>The following critical events are logged unconditionally, regardless of the state of the audit role (initialized or not):

Tamper

Decommission

Zeroization

SO creation

Audit role creation

Types of events included in the logs

The events that are included in the log is configurable by the audit role. The types of events that can be logged include the following:

>log access attempts (logins)

>log HSM management (init/reset/etc)

>key management events (key create/delete)

>asymmetric key usage (sig/ver)

>first asymmetric key usage only (sig/ver)

>symmetric key usage (enc/dec)

>first symmetric key usage only (enc/dec)

>log messages from CA_LogExternal

>log events relating to log configuration

Each of these events can be logged if they fail, succeed, or both.

Event log storage

When the HSM logs an event, the log is stored on the HSM. The audit user cannot view these log entries. Before a log can be viewed, it must be rotated. Log rotation saves the log entries on the HSM to the HSM appliance, where they can be viewed. Log records are HMACed using an audit log secret to ensure their authenticity. The audit log secret is unique to the HSM where the log was created, and is required to view the HSM event logs. The secret can be exported, allowing you to view and verify the logs on another HSM.

TIP   Log entries are stored in the cryptographic module (HSM) until they are rotated off. Log entries are not rotated out of the cryptographic module until the audit user is initialized and audit logging is configured. By default, even if there is no audit user or configuration, the cryptographic module logs unconditional events within its own memory, like:

>zeroize

>decommission

>hardware tamper

>card removal

>etc.

If the crypto module internal space ever fills completely with log records,

>whether slowly from unconditional logs, or

>quickly from more voluble high-volume event recording,

...the HSM / cryptographic module would stop all operations that were not audit init and audit config. The HSM would resume providing service only after the audit user cleared the logs.

To avoid that ever happening, configure audit logging to organize log parameters and handling, being sure to set sufficient frequency of rotation for the volume of record generation that you enable.

Best practice is to:

>initialize the audit role as soon as the HSM is first powered on for production audit init

>configure the log storage path on the external file system, along with the types of events to log, the rotation interval, etc. audit config

>then, initialize the HSM Security Officer (this helps ensure that all messages, demanded by your auditing authority, are captured) hsm init

>then, proceed with partition initialization and usage with your application(s)

>then, revisit audit log configuration at regular intervals to tune the balance between

desired message types,

volume of audited actions normally performed (*),

and so on

>when you change behavior of the crypto module or change the types of events to audit, be sure to revisit also the rotation interval.

[* Example, you might always want to record the generation of keys, but if usage of those keys is very high-volume (like in some signature use-cases), and thus would generate a high volume of log entries, it might be permissible, and prudent, to log only first-use of any key. Check with the relevant authority.]

Audit log locations in the HSM appliance

When viewing HSM appliance information (like status disk), you might see mention of two log-file locations.

>The folder /var/audit receives the HSM audit logs only. This is for HSM events and cryptographic operations. No information about host system events is logged here.

>The folder /var/log/audit is for the appliance operating system (host system) audit logs. No information about cryptographic operations is logged here.

Event logging impacts HSM performance

Each audit log record generated requires HSM resources. Configuring event logging to record most, or all, events may have an impact on HSM performance. You may need to adjust your logging configuration to provide adequate logging without significantly affecting performance. By default, only critical events are logged, imposing virtually no load on the HSM.

Audit limitations and Controlled tamper recovery state

The following conditions apply when HSM Policy "48: Do controlled tamper recovery" is enabled (default setting).

>Auditor (the Audit role) cannot verify the integrity of audit logs until after recovery from tamper.

>Auditor cannot be initialized when the HSM is in controlled tamper recovery state.

>Existing Audit role can login when in controlled tamper recovery state.

>Existing Audit role cannot make audit config changes when in controlled tamper recovery state.

>Existing Audit role cannot export the audit secret when in controlled tamper recovery state.

The Audit Role

The audit logging function is controlled by two roles on Luna Network HSM 7, that must be used together:

>The "audit" appliance account (use SSH or PuTTy to log in as "audit", instead of "admin", or "operator", or "monitor", etc.)

>The "audit" HSM account (accessible only if you have logged into the appliance as "audit"; this account must be initialized)

On Luna Network HSM 7, the audit logging is managed by an audit user (an appliance system role), in combination with the HSM audit role, through a set of LunaSH commands. The audit user can perform only the audit-logging related tasks and self-related tasks. Other HSM appliance users, such as admin, operator, and monitor, have no access to the audit logging commands.

A default appliance (LunaSH) audit user is automatically created, but must be enabled. Upon first login, the audit user is asked to change their password. That appliance audit user would need to initialize the HSM audit role first, before being able to administer the audit logging. The Luna Network HSM 7 admin user can create more audit users when necessary.

To simplify configuration,

>The maximum log file size is capped at 50 MB.

>The log path is kept internal.

>The rotation offset is set at 0.

Audit User on the Appliance

The appliance audit user is a standard user account on Luna Network HSM 7, with default password "PASSWORD" (without the quotation marks). By default, the appliance audit user is disabled. Therefore, you must enable it in LunaSH before it becomes available. See user enable for the command syntax.

Audit Role on the HSM

A Luna Network HSM 7 Audit role allows complete separation of Audit responsibilities from the HSM Security Officer (HSM SO), the Crypto Officer(or User), and other HSM roles. If the Audit role is initialized, the HSM and Partition SOs are prevented from working with the log files, and auditors are unable to perform administrative tasks on the HSM. As a general rule, the Audit role should be created before the HSM Security Officer role, to ensure that all important HSM operations (including those that occur during initialization), are captured.

Use the LunaSH command audit init to initialize the audit role, as described in audit init.

Password-authenticated HSMs

For Luna Network HSM 7s with Password Authentication, the auditor role logs into the HSM to perform their activities using a password. After initializing the Audit role on a password-authenticated HSM, log in as the Auditor and set the domain (see role setdomain). This step is required before setting logging parameters or the log filepath, or importing/exporting audit logs.

Multifactor Quorum-authenticated HSMs

For Luna Network HSM 7s with multifactor quorum authentication, the auditor role logs into the HSM to perform their activities using the Audit (white) PED key.

Role Initialization

Creating the Audit role (and imprinting the white PED key for multifactor quorum-authenticated HSMs) does not require the presence or cooperation of the HSM SO.

Appliance Audit User Available Commands

The Audit role has a limited set of operations available to it, on the HSM, as reflected in the reduced command set available to the "audit" user when logged in to the shell (LunaSH).

login as: audit
audit@192.20.11.78's password:
Last login: Fri Mar 31 09:37:53 2020 from 10.124.0.31

Luna SA 7.7.0 Command Line Shell - Copyright (c) 2001-2020 SafeNet, Inc. All rights reserved.

lunash:>help

The following top-level commands are available:

 Name                 (short)    Description
 --------------------------------------------------------------------------------
 help                 he         Get Help
 exit                 e          Exit Luna Shell
 hsm                  hs         > Hsm
 audit                a          > Audit
 my                   m          > My
 network              n          > Network

Audit Log Secret

The HSM creates a log secret unique to the HSM, computed during the first initialization after manufacture. The log secret resides in flash memory (permanent, non-volatile memory), and is used to create log records that are sent to a log file. Later, the log secret is used to prove that a log record originated from a legitimate HSM and has not been tampered with.

Log Secret and Log Verification

The 256-bit log secret which is used to compute the HMACs is stored in the parameter area on the HSM. It is set the first time an event is logged. It can be exported from one HSM to another so that a particular sequence of log messages can be verified by the other HSM. Conversely, it can be imported from other HSMs for verification purpose.

To accomplish cross-HSM verification, the HSM generates a key-cloning vector (KCV, a.k.a. the Domain key) for the audit role when it is initialized. The KCV can then be used to encrypt the log secret for export to the HOST.

To verify a log that was generated on another HSM, assuming it is in the same domain, we simply import the wrapped secret, which the HSM subsequently decrypts; any records that are submitted to the host for verification will use this secret thereafter.

When the HSM exports the secret, it calculates a 32-bit checksum which is appended to the secret before it is encrypted with the KCV.

When the HSM imports the wrapped secret, it is decrypted, and the 32-bit checksum is calculated over the decrypted secret. If this doesn’t match the decrypted checksum, then the secret that the HSM is trying to import comes from a system on a different domain, and an error is returned.

To verify a log generated on another HSM, in the same domain, the host passes to the target HSM the wrapped secret, which the target HSM subsequently decrypts; any records submitted to the target HSM for verification use this secret thereafter.

Importing a log secret from another HSM does not overwrite the target log secret because the operation writes the foreign log secret only to a separate parameter area for the wrapped log secret.

CAUTION!   Once an HSM has imported a wrapped log secret from another HSM, it must export and then re-import its own log secret in order to verify its own logs again.

Audit Log Records

A log record consists of two fields – the log message and the HMAC for the previous record. When the HSM creates a log record, it uses the log secret to compute the SHA256-HMAC of all data contained in that log message, plus the HMAC of the previous log entry. The HMAC is stored in HSM flash memory. The log message is then transmitted, along with the HMAC of the previous record, to the host. The host has a logging daemon to receive and store the log data on the host hard drive.

For the first log message ever returned from the HSM to the host there is no previous record and, therefore, no HMAC in flash. In this case, the previous HMAC is set to zero and the first HMAC is computed over the first log message concatenated with 32 zero-bytes. The first record in the log file then consists of the first log message plus 32 zero-bytes. The second record consists of the second message plus HMAC1 = HMAC (message1 || 0x0000). This results in the organization shown below.

MSG 1 HMAC 0
  . . .
MSG n-1 HMAC n-2
MSG n HMAC n-1
. . .  
MSG n+m HMAC n+m-1
MSG n+m+1 HMAC n+m
. . .  
MSG end HMAC n+m-1

 

Recent HMAC in NVRAM HMAC end

To verify a sequence of m log records which is a subset of the complete log, starting at index n, the host must submit the data illustrated above. The HSM calculates the HMAC for each record the same way as it did when the record was originally generated, and compares this HMAC to the value it received. If all of the calculated HMACs match the received HMACs, then the entire sequence verifies. If an HMAC doesn’t match, then the associated record and all following records can be considered suspect. Because the HMAC of each message depends on the HMAC of the previous one, inserting or altering messages would cause the calculated HMAC to be invalid.

The HSM always stores the HMAC of the most-recently generated log message in flash memory. When checking truncation, the host would send the newest record in its log to the HSM; and, the HSM would compute the HMAC and compare it to the one in flash. If it does not match, then truncation has occurred.

Audit Log Message Format

Each message is a fixed-length, comma delimited, and newline-terminated string. The table below shows the width and meaning of the fields in a message.

Offset Length (Chars) Description
0 10 Sequence number
10 1 Comma
11 17 Timestamp
28 1 Comma
29 256 Message text, interpreted from raw data   
285 1 Comma
286 64 HMAC of previous record as ASCII-HEX   
350 1 Comma   
351 96 Data for this record as ASCII-HEX (raw data)   
447 1 Newline '\n'

The raw data for the message is stored in ASCII-HEX form, along with a human-readable version. Although this format makes the messages larger, it simplifies the verification process, as the HSM expects to receive raw data records.

Example

The following example shows a sample log record. It is separated into multiple lines for readability even though it is a single record. Some white spaces are also omitted.

38,12/08/13 15:30:50,session 1 Access 2147483651:22621 operation LUNA_CREATE_CONTAINER
returned LUNA_RET_SM_UNKNOWN_TOSM_STATE(0x00300014) (using PIN (entry=LUNA_ENTRY_DATA_AREA)),
29C51014B6F131EC67CF48734101BBE301335C25F43EDF8828745C40755ABE25,
2600001003600B00EA552950140030005D580000030000800100000000000000000000000000000000000000

The sequence number is “38”. The time is “12/08/13 15:30:50”.

The log message is “session 1 Access 2147483651:22621 operation LUNA_CREATE_CONTAINER returned LUNA_RET_SM_UNKNOWN_TOSM_STATE(0x00300014) (using PIN (entry=LUNA_ENTRY_DATA_AREA))”.

In the message text, the “who” is the session identified by “session 1 Access 2147483651:22621” (the application is identified by the access ID major = 2147483651, minor = 22621).

The “what” is “LUNA_CREATE_CONTAINER”.

The operation status is “LUNA_RET_SM_UNKNOWN_TOSM_STATE(0x00300014)”.

The HMAC of previous record is “29C51014B6F131EC67CF48734101BBE301335C25F43EDF8828745C40755ABE25”.

The remainder is the raw data for this record as ASCII-HEX.

>The “who” is LunaSH session “session 1 Access 2147483651:22621”
(identified by the lunash access ID major = 2147483651, minor = 22621).

>The “what” is “LUNA_CREATE_CONTAINER”.

>The operation status is “LUNA_RET_SM_UNKNOWN_TOSM_STATE(0x00300014)”.

Timestamping

The HSM has an internal real-time clock (RTC). The RTC does not have a relevant time value until it is synchronized with the HOST system time. Because the HSM and the host time could drift apart over time, periodic re-synchronization is necessary. Only an authenticated Auditor is allowed to synchronize the time.

Time Reported in Log

When you perform audit show, you might see a variance of a few seconds between the reported HSM time and the Host time. Any difference up to five seconds should be considered normal, as the HSM reads new values from its internal clock on a five-second interval. So, typically, Host time would show as slightly ahead.

Log Capacity

The Luna Network HSM 7 appliance has approximately 220 GB of HDD capacity available for storing Audit logs. If you are logging everything the HSM does, this space can fill up completely over time if the Auditor does not periodically export logs off the appliance.

LOG FULL condition

If you receive CKR_LOG_FULL, the log capacity has been reached, and all HSM operations will stop. This is to prevent the HSM from performing unlogged operations. In this condition, most HSM commands will not work; only commands that allow the Auditor to log in, clear the log storage, set the logging configuration, or reset the HSM to factory conditions are permitted.

See Copying Log Files Off the Appliance for details of this recovery procedure.

Configuration Persists Unless Factory Reset is Performed

Audit logging configuration is not removed or reset upon HSM re-initialization or a tamper event. Factory reset or HSM decommission will remove the Audit user and configuration. Logs must be cleared by specific command. Therefore, if your security regime requires decommission at end-of-life, or prior to shipping an HSM, then explicit clearing of HSM logs should be part of that procedure.

This is by design, as part of separation of roles in the HSM. When the Audit role exists, the HSM SO cannot modify the logging configuration, and therefore cannot hide any activity from auditors.

NTLS is stopped but log still records LUNA_OPEN_SESSION/LUNA_CLOSE_SESSION messages

LUNA_OPEN_SESSION and LUNA_CLOSE_SESSION messages continue to appear in the audit logs, even though NTLS is stopped and applications cannot connect.

This is expected: inside the Luna Network HSM 7 appliance, a system state-of-health monitor routinely calls "hsm show", to ensure that the HSM is still functioning. Those calls trigger audit log messages.