Operations
This section provides information on operations that the DDC administrator performs on the CipherTrust Manager.
Reconfiguring DDC agents
In some situations, for example, if the hostname or IP address of the CipherTrust Manager appliance changes, Agents' connection with DDC must be reconfigured with the new hostname or IP address.
Reconfiguring DDC agents on Windows
To reconfigure a DDC Agent:
Log on to the host machine as administrator.
Open Enterprise Recon Configuration Tool (er2_config_cmd.exe).
By default, the tool is available at
C:\Program Files (x86)\Ground Labs\Enterprise Recon 2\.In the Master server IP address or host name field, specify the new hostname or IP address of the CipherTrust Manager.
Click Test Connection. A message stating "Connectivity test is successful" confirms successful reconfiguration.
Click Finish.
Reconfiguring DDC agents on Debian
To reconfigure a DDC Agent:
Log on to the host machine as a user with root privileges.
Reconfigure connection with DDC on the CipherTrust Manager appliance.
sudo er2-config -i <hostname|ip_address>Here,
<hostname|ip_address>represents the new IP address or hostname of the CipherTrust Manager appliance.Restart the Agent service. Configuration settings will be effective after the Agent restarts.
sudo /etc/init.d/er2-agent -restart
Reconfiguring DDC agents on RHEL
To reconfigure a DDC Agent:
Log on to the host machine as a user with root privileges.
Reconfigure connection with DDC on the CipherTrust Manager appliance.
er2-config -i <hostname|ip_address>Here,
<hostname|ip_address>represents the new IP address or hostname of the CipherTrust Manager appliance.Restart the Agent service. Configuration settings will be effective after the Agent restarts.
sudo /etc/init.d/er2-agent restart
Tuning scan settings
You can customize system parameters for all the DDC Agents by using the ksctl tool. The following system parameters can be modified this way:
Agent Memory (in MB)
CPU used
Throughput
The ksctl command to use to this end is ksctl ddc settings scan modify.
Command syntax:
ksctl ddc settings scan modify [flags]
Flags:
| Flag | Usage |
|---|---|
--agent-memory int | Setting for the maximum memory usage that the scanner service can use on the agent host, in MB. Default value: 1024. |
-h, --help | Help on the command's usage |
--jsonfile string | JSON format to create resources in DDC endpoints. |
--max-scan-throughput int | Max I/O rate the scanner service will use to read data from the data store, in MBps. Set to 0 for unlimited. Default value is 0. |
--scan-cpu string | CPU priority set for the agent used in the scan. The possible values are 'low' and 'normal'. Default value is 'low'. |
Global Flags:
| Flag | Usage |
|---|---|
--configfile string | Full path and name to a file that contains the configuration parameters (optional). |
--connection string | The friendly name of the server you want to authenticate against. (default "local_account") |
--domain string | The CipherTrust Manager domain that the command will operate in. Can be used only with user/password and not with token. By default the command will operate in the root domain or the domain the user is logged-in. |
--jwt string | The JSON Web Token (JWT). It is the access token that can be passed instead of user/password (optional). The command ksctl tokens create creates a JWT. |
--nosslverify | Do not verify the certificate for SSL/HTTPS authentication (not recommended). |
--password string | CipherTrust Manager server user password. Do not use this flag to enter the password (masked) from terminal. |
--respfmt string | Response Output format (json is the only supported value at present, optional) (default "json") |
--timeout int | Timeout in seconds for TCP connection attempts. |
--token string | The refresh token returned from the login command to be passed instead of user/password (optional). The command ksctl login creates a token and writes it to the config file. |
--url string | CipherTrust Manager server URL. |
--user string | CipherTrust Manager server user name. |
-v, --verbose | Provide verbose output while executing command (optional). |
Examples:
ksctl ddc settings scan modify --agent-memory 1024 --scan-cpu normal --max-scan-throughput 2
ksctl ddc settings scan modify --jsonfile scansettings.json
Warning
The ksctl ddc settings scan command will be deprecated in the coming release.
Restarting DDC agents
Restarting agents on Windows
To restart a DDC Agent, run the following commands:
net stop "Enterprise Recon 2 Agent (<ARCH>)"
net start "Enterprise Recon 2 Agent (<ARCH>)"
Here, <ARCH> represents the Windows architecture - x32 or x64.
Restarting agents on Debian
To restart a DDC Agent, run:
sudo /etc/init.d/er2-agent restart
Alternatively, restart the Agent service by stopping it and again starting it manually. Run the following commands:
sudo /etc/init.d/er2-agent stop
sudo /etc/init.d/er2-agent start
Restarting agents on RHEL
To restart a DDC Agent, run:
sudo /etc/init.d/er2-agent restart
Alternatively, restart the Agent service by stopping it and again starting it manually. Run the following commands:
sudo /etc/init.d/er2-agent stop
sudo /etc/init.d/er2-agent start
Renewing DDC ML agent certificate
DDC ML agents are linked with CipherTrust Manager either through client certificates or server certificates for web interface. Certificate renewal is necessary under the following scenarios:
When client certificate is near expiry
When signer CA of client certificate is renewed or updated in CipherTrust Manager
When server certificate is renewed or updated
Note
Only local CAs are supported for issuing certificate for agents. See Certificate Authority to learn more about local CA certificates.
Renewing client certificate
Client certificate is renewed when the DDC ML agent certificate is about to expire or when its signing CA (Certificate Authority) is modified in the CipherTrust Manager. The DDC ML agent automatically handles renewal of client certificates.
Agent certificate or its signing CA is near expiry
When the agent certificate (with a valid signing CA) is about to expire, the agent initiates the renewal process sixty days before the expiry. It requests the CipherTrust Manager to issue new certificates. The new certificates are automatically downloaded and saved on the agent host machine.
If the agent certificate’s signing CA is about to expire, the agent requests CipherTrust Manager to issue a new certificate. This process continues until the expiration or update of the agent certificate's signing CA.
When the signing CA is renewed or updated, CipherTrust Manager sends renewal notifications to the agent, as described in the next section. However, if the signing CA is expired, you must re-install the agent to reestablish communication with CipherTrust Manager.
Signing CA of agent certificate is updated
In CipherTrust Manager, each DDC client is associated with a Client Profile. Further, each Client Profile has a CA associated with it, which acts as the signing CA for the agent certificate.
When the signing CA is updated or renewed, CipherTrust Manager sends renewal notification to all the clients associated with the Client Profile.
Upon receiving the notification, DDC ML agents set a flag in their configuration file and within the next 12 hours, the agent certificates are renewed.
When certificate renewal is successful, a message Updated Agent and signer CA certificates is added in the log files and the agent continues to function normally.
Note
If the agent is down or unavailable, CipherTrust Manager will continue to send notifications of CA changes until the agent is accessible again.
In the event of agent certificate expiring unexpectedly, you need to reinstall the agent to restore communication with CipherTrust Manager.
Renewing server certificate
When a server certificate or one of its signing CAs is renewed or updated, a notification is sent to all the agents registered with the CipherTrust Manager for renewal.
You can renew server certificate either through the CipherTrust Manager API or GUI.
See Get certificate for NAE, KMIP, and WEB interfaces to issue web interface certificate and use them during renewing.
Add the new certificate to the list of upcoming sever certificates using the below API.
put /v1/configs/interfaces/web/renewal-certificateParameter Description certificateSpecify the certificate chain having the primary certificate, the CA, and its private key. formatSpecify the certificate format. Restart the web interface services using the below API.
post /v1/system/services/restartParameter Description delaySpecify the delay vale. Default value is 5 seconds. servicesSpecify the interface service you want to restart. Enter 'web' as the parameter value. After upcoming renewal certificate is updated successfully, following events occur at the agent side:
Active agents receive notification about the update from CipherTrust Manager and updates the trust store with the new certificate chain.
Note
If the agent is down or unavailable, CipherTrust Manager continues to send notifications until the agent is accessible again.
Old and new certificates are stored on the agent side at
$INSTALL_DIR/Certificates/trustStore.pemand$INSTALL_DIR/Certificates/trustStore.0locations, respectively.Agent log displays the Server CA certificate renewal completed successfully message.
Agent trusts both old and new certificates until the former one expires.
Fetch the upcoming renewal certificate using the below API.
get /v1/configs/interfaces/web/renewal-certificateThe API returns the public key and CA chain of the upcoming renewal certificate in the PEM format.
When the current web interface certificate is about to expire, apply the new certificate using the below API.
post /v1/configs/interfaces/{interface}/renewal-certificate/applyNote
If the new certificate is not applied manually, it will be automatically applied on expiration of the current certificate.
Upload the upcoming renewal certificate
Go to Admin Settings > Interfaces.
Click the ellipses icon (...) next to the web interface.
Select Renewal Certificate Options.
Click Upload/Generate to upload a new server certificate.
Click OK.
Select an upload option.
File Upload: Use this option to upload the certificate file.
Text: Use this option to paste the signed certificate and its private key content.
Select PEM as Format.
Click Upload Certificate.
Go to Admin Settings > Services and click the Restart button next to web services.
After the upcoming renewal certificate is uploaded successfully, following events occur at the agent side:
Active agents receive notification of this update from the CipherTrust Manager and updates the trust store with the new certificate chain.
Note
If the agent is down or unavailable, CipherTrust Manager continues to send notifications until the agent is accessible again.
Old and new certificates are stored on the agent side at
$INSTALL_DIR/Certificates/trustStore.pemand$INSTALL_DIR/Certificates/trustStore.0locations, respectively.Agent log displays the Server CA certificate renewal completed successfully message
Agent trusts both old and new server certificates until the former one expires.
Apply the upcoming renewal certificate
When the current interface certificate is about to expire, you need to apply the newly uploaded upcoming renewal certificate for the interface.
Click the ellipses icon (...) next to the web interface.
Click Renewal Certificate Options.
Select Apply and click Ok to apply the upcoming renewal server certificate.
The existing server certificate is replaced with the newly uploaded server certificate.
Note
If the new certificate is not applied manually, it will be automatically applied on expiry of the current certificate.
DDC ML agent configuration parameters
The configuration parameters are available in the <INSTALL_DIR>/conf/ddc_agent.conf file.
Performance log settings
| Parameters | Default | Description |
|---|---|---|
ddc_agent_extraction_log_threshold | None | This is the elapsed time threshold in milliseconds to log text extraction call. A call will be logged if the elapsed time is equal to or greater than the threshold. |
ddc_agent_prediction_log_threshold | None | This is the elapsed time threshold in milliseconds to log ML model prediction call. A call will be logged if the elapsed time is equal to or greater than the threshold. |
ddc_agent_archive_log | None | This is the boolean flag on whether to log messages for archive extraction. |
Request configuration settings
| Parameters | Default | Description |
|---|---|---|
ddc_prediction_worker | (host cpu number) / 4 | This is the number of threads to call the ML model prediction interface concurrently. |
ddc_prediction_queue | 20 | This is the maximum number of texts that can be queued for prediction. |
ddc_prediction_batch_memory | 1 megabytes | ML model has prediction call to accept an array of texts, and this parameter just hints the maximum memory size of the text array in bytes. |
ddc_prediction_batch_number | 10 | ML model has prediction call to accept an array of texts, and this parameter specifies the maximum number of texts in the array. |
ddc_parse_worker | (host cpu number) / 4 | This is the number of threads to call Apache Tika parser interface concurrently. |
ddc_parse_queue | 20 | This is the maximum number of files can be queued for parsing. |
ddc_parser_chunk_size | 10 megabytes | When the ML Agent calls Apache Tika to parse a big file, it can receive result texts chunk by chunk and call the ML model prediction interface with those chunks. This parameter specifies the chunk size in bytes. |
ddc_prediction_report_threshold | 100 | This is the frequency for the ML agent to report prediction progress. The default value is 100, which means the ML agent reports progress when every 100 number of files are processed and predicted. |
ddc_scan_report_threshold | 10000 | This is the frequency for the ML agent to report progress of scan file list generation. The default value is 10000, which means the ML agent reports progress when every 10000 files are written into the scan file list. |
ddc_java_options | No default | This is the JVM options list separated by comma, for example -Xmx8g, -XX:+UseG1GC. |
Model driver
| Parameters | Default | Description |
|---|---|---|
ddc_ml_log_level | info | Represent the log level for the model driver. Possible values: info, warn, or error. |
TDPaaS retention policy
A retention policy allows you to store data on the cloud TDP (TDPaaS) for a definite amount of time. This feature is useful if there is a need to delete the stored information after a definite amount of time to comply with some regulations and policies.
After the retention policy is enabled, information removal is automatically handled by DDC through scrub jobs. The scrub jobs operate automatically at regular intervals, and you can view the complete details on the Operations tab. The scrubbing process clears the scan executions and reports from the cloud storage and CipherTrust Manager database, ensuring they don't show up in the list of executions.
After the scrub job is completed, scrubbed data is no longer accessible for report generation or scan executions, and it can't be restored.
Note
A retention policy is only available when TDPaaS is provisioned. Refer to TDPaaS Deployment Guide to learn how to provision TDPaaS.
Key features
Enable/Disable Flexibility: Activate or deactivate the Retention Policy at any time.
Automatic Scrubbing Jobs: Scrubbing jobs operate automatically at regular intervals to effectively manage data.
Retention Period: Flexibility to select the retention duration in the range of 3 to 60 months. If the retention period exceeds 60 months, it's recommended to avoid enabling the retention policy.
Data Management: Scrubbing job eliminates scan executions and reports from the cloud storage and the CipherTrust Manager database, ensuring they are no longer visible in the execution list.
View Job Details: Access information about scrub jobs, including:
Execution ID - Displays the execution ID of the scrub job.
Execution Started - Displays the timestamp when scrub job execution started.
Duration - Displays the duration of the scrub job execution.
Status - Displays the current status.
Space Freed - Displays the amount of space freed from cloud storage and CipherTrust Manager databases in MBs.
Retention Period - Displays the retention period in months.
Filtering Options: Filter scrub job data based on the status of job executions and chosen date ranges.
Enabling retention policy
Log on to CipherTrust Manager.
Go to Settings > Cloud Management.
Select the Retention Policy tab.
Turn on the Retention Policy toggle.
Enter Retention Period in the range of 3 to 60 months. By default, it is set to 12 months.
Click Apply.
Note
If you enable retention policy and switch back to on-premises TDP, then retention policy doesn't disable automatically but remains ineffective.
Viewing Scrub job executions
After enabling the retention policy:
Go to the Operations tab.
This tab displays all scrub job executions up to the current date. Scrub jobs run automatically at a reasonable frequency.
Use the date filter to narrow down job executions to a specific date range.
Use the status filter to filter job executions as per Running, Completed, or Failed statuses.
