Scheduler
Note
This feature is not included in Community Edition and requires a valid Virtual CipherTrust Manager license. To activate your instance with a trial evaluation, or a term or perpetual license, see Licensing.
The CipherTrust Manager can periodically run key rotation and database backup jobs in the background. These jobs can be:
Configured using the scheduler
Scheduled by creating a job configuration
A job configuration defines what job is run and when. Job configurations can be read, listed, updated, and deleted. Jobs are run after job configurations are created. The jobs that were run can also be listed.
In a clustered environment, by default the scheduled job can run on any cluster node. There is an option in the API and CLI to restrict which nodes can run the scheduled job.
This section provides examples of scheduler configurations using the ksctl utility. Refer to the ksctl or API documentation for details.
Default Scheduler
The system generated scheduler (system_auto_key_rotation) is disabled by default. When this scheduler is enabled, it runs every day at midnight, and rotates the keys with a rotation date less than the current time. To enable the scheduler, run:
ksctl scheduler configs modify key-rotation --id <ID> --disabled=false
Example
ksctl scheduler configs modify key-rotation --id e08708b9-c6f0-49f2-bbea-64bd6ee80779 --disabled=false
Time Specification
The time that a job runs is described in the run-at parameter using the cron expression format: "* * * * *"
These five values indicate when the job should be executed. They are in order of minute, hour, day of month, month, and day of week.
The following table lists the accepted values:
| Field Name | Mandatory | Allowed Values | Allowed Special Characters |
|---|---|---|---|
| Minute | Yes | 0-59 | * / , - |
| Hour | Yes | 0-23 | * / , - |
| Day of month | Yes | 1-31 | * / , -? |
| Month | Yes | 1-12 or JAN-DEC | * / , - |
| Day of week | Yes | 0-6 or SUN-SAT | * / , -? |
Note
Month and Day of week field values are case insensitive. For example, "SUN", "Sun", and "sun" are equally accepted.
Note
Scheduler itself is a generic service, so it provides granularity of a minute. A job configuration should take the nature of operation information into consideration. For example, rotating the same set of multiple keys and backing up a database every minute are highly unlikely use cases.
Allowed Special Characters
The following table describes the use of special characters in the cron expression format:
| Special Character | Description |
|---|---|
| Asterisk * | Indicate that the cron expression will match for all values of the field. For example, using an asterisk in the 5th field (month) would indicate every month. |
| Slash / | Describe increments of ranges. For example, 3-59/15 in the 1st field (minutes) would indicate the 3rd minute of the hour and every 15 minutes thereafter. |
| Comma , | Separate items of a list. For example, using "MON,WED,FRI" in the 5th field (day of week) would mean Mondays, Wednesdays, and Fridays. |
| Hyphen - | Define ranges. For example, 9-17 would indicate every hour between 9am and 5pm inclusive, MON-WED would indicate Monday through Wednesday, and JAN-JUN would indicate every month from January through June. |
| Question mark ? | Use instead of "*" for leaving either day-of-month or day-of-week blank. "?" means no specific value. For example, to run the job every 10th day of the month and not limit the job to a particular day of the week: "* * 10 * ?". |
Creating Job Configurations
To create a job configuration, run the command ksctl scheduler configs create <operation> --name <config-name> --run-at "<cron-expression-format>".
Here,
<operation>: Operation to schedule. The options are:backup: Create a job configuration for backup operation. Refer to Scheduling Backups more details. {key-rotation: Create a job configuration for key rotation. Refer to Scheduling Key Rotation more details.
<config-name>: Name for the job configuration.<cron-expression-format>: Time when the job runs. The format must have five fields. If the number of fields is not equal to five, the format becomes invalid. Refer to time specification for details.<scp-flag>: If sets totruethen it securely transfers system backup from CipherTrust Manager to the external servers.<scp-connection>: Sets the connection type as SCP.
Example - backup scheduler:
ksctl scheduler configs create backup --name ab-backup-config --run-at "0 12 * * 0" --do-scp true --conn "scp-connection"
{
"id": "fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a",
"uri": "kylo:kylo:scheduler:job_configs:fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-12T06:06:15.110530931Z",
"name": "ab-backup-config",
"updatedAt": "2020-04-12T06:06:15.110530931Z",
"description": "",
"operation": "database_backup",
"run_at": "0 12 * * 0",
"run_on": "any",
"disabled": false
"do_scp": true
"connection": "scp-connection"
}
Example - key rotation scheduler:
ksctl scheduler configs create key-rotation --name ab-key-rot-config --run-at "45 11 * * 0"
{
"id": "20c5263d-e296-4994-9129-b71aff49f4b3",
"uri": "kylo:kylo:scheduler:job_configs:20c5263d-e296-4994-9129-b71aff49f4b3",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-12T06:07:40.05471195Z",
"name": "ab-key-rot-config",
"updatedAt": "2020-04-12T06:07:40.05471195Z",
"description": "",
"operation": "key_rotation",
"run_at": "45 11 * * 0",
"run_on": "any",
"disabled": false
}
Viewing Existing Job Configurations
To view the list of job configurations, run the command ksctl scheduler configs list.
Example:
ksctl scheduler configs list
{
"skip": 0,
"limit": 10,
"total": 2,
"resources": [
{
"id": "20c5263d-e296-4994-9129-b71aff49f4b3",
"uri": "kylo:kylo:scheduler:job_configs:20c5263d-e296-4994-9129-b71aff49f4b3",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-12T06:07:40.054712Z",
"name": "ab-key-rot-config",
"updatedAt": "2020-04-12T06:07:40.054712Z",
"description": "",
"operation": "key_rotation",
"run_at": "45 11 * * 0",
"run_on": "any",
"disabled": false
},
{
"id": "fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a",
"uri": "kylo:kylo:scheduler:job_configs:fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-12T06:06:15.110531Z",
"name": "ab-backup-config",
"updatedAt": "2020-04-12T06:06:15.110531Z",
"description": "",
"operation": "database_backup",
"run_at": "0 12 * * 0",
"run_on": "any",
"disabled": false
}
]
}
The output shows existing job configurations.
Viewing Details of Job Configurations
To view the details of a job configuration, run the command ksctl scheduler configs get --id "configID". Here, <configID> represents the ID of the configuration.
Example
ksctl scheduler configs get --id "fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a"
{
"id": "fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a",
"uri": "kylo:kylo:scheduler:job_configs:fd0c27b8-a502-47cc-b6bc-4c1c6fbbe89a",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-12T06:06:15.110531Z",
"name": "ab-backup-config",
"updatedAt": "2020-04-12T06:06:15.110531Z",
"description": "",
"operation": "database_backup",
"run_at": "0 12 * * 0",
"run_on": "any",
"disabled": false
}
Modifying Job Configurations
Existing job configurations can be modified when needed. To modify a job configuration, run the command: ksctl scheduler configs modify <operation> --id "<configID>" <args>
Here,
<operation>: Operation can bebackuporkey-rotation.<configID>: ID of the job configuration to be modified.<args>: Parameters to modify.
Example:
ksctl scheduler configs modify key-rotation --id "20c5263d-e296-4994-9129-b71aff49f4b3" --run-at "50 11 * * 0"
{
"id": "20c5263d-e296-4994-9129-b71aff49f4b3",
"uri": "kylo:kylo:scheduler:job_configs:20c5263d-e296-4994-9129-b71aff49f4b3",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-12T06:07:40.054712Z",
"name": "ab-key-rot-config",
"updatedAt": "2020-04-12T06:14:57.23658004Z",
"description": "",
"operation": "key_rotation",
"run_at": "50 11 * * 0",
"run_on": "any",
"disabled": false
}
Manually Running Job Configurations
A scheduled configuration can be run manually when needed. To manually run a job configuration, run the command: ksctl scheduler configs run-now --id "<configID>"
Here, <configID> represents the ID of the configuration. The command creates a new job and waits for the job to be posted on the job queue. The "job_id" is returned if the job is posted within about 30 seconds.
Example:
ksctl scheduler configs run-now --id "e90cb2df-7df5-487a-ab5e-2c0ead7405c4"
{
"job_id": "1e05dfb0-a5f9-48f6-afdf-406c368d2847",
"job_config_id": "e90cb2df-7df5-487a-ab5e-2c0ead7405c4"
}
Deleting Job Configurations
To delete a job configuration, run the command ksctl scheduler configs delete --id "<configID>". Here, <configID> is the ID of the job configuration to be deleted. The specified job configuration is deleted without any message.
After a configuration is deleted, it will no longer run jobs automatically.
Job Runs
A job is created and run automatically when a scheduler configuration is run manually (using run-now) or automatically at a schedule defined in the job configuration. You can view the results of jobs executed in the past, including status and error count.
Viewing Job Runs
To view the list of already run jobs for the logged in account, run the command ksctl scheduler jobs list.
Example:
ksctl scheduler jobs list
{
"skip": 0,
"limit": 10,
"total": 2,
"resources": [
{
"id": "5ce37ebd-f43f-4a97-bbaf-d6a7416074a7",
"uri": "kylo:kylo:scheduler:jobs:5ce37ebd-f43f-4a97-bbaf-d6a7416074a7",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-15T02:25:01.749924Z",
"name": "rotate AES keys weekly",
"updatedAt": "2020-04-15T02:25:14.003393Z",
"job_config_id": "cbf85c25-08e9-4932-93da-a1042aafafdd",
"description": "This is to rotate AES keys every Sunday, at 02:00",
"operation": "key_rotation",
"job_config_params": {
"meta": {
"naeState": "Restricted"
},
"query": {
"algorithm": "AES"
},
"deactivate_replaced_key": 30
},
"enqueued_at": "2020-04-15T02:25:01.749791Z",
"status": "completed",
"processing_node": "local-node",
"details": {
"job_id": "5ce37ebd-f43f-4a97-bbaf-d6a7416074a7",
"job_config_id": "cbf85c25-08e9-4932-93da-a1042aafafdd",
"name": "rotate AES keys weekly",
"operation": "key_rotation",
"status": "completed",
"percent_done": 100,
"details": {
num_rotated": 1
}
}
},
{
"id": "5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1",
"uri": "kylo:kylo:scheduler:jobs:5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-15T02:09:02.740656Z",
"name": "rotate AES keys",
"updatedAt": "2020-04-15T02:09:13.978049Z",
"job_config_id": "61bf1c6f-f1b7-4ef9-b097-af8304647ecf",
"description": "This is to rotate AES keys",
"operation": "key_rotation",
"job_config_params": {
"meta": {
"naeState": "Restricted"
},
"query": {
"algorithm": "AES"
},
"deactivate_replaced_key": 30
},
"enqueued_at": "2020-04-15T02:09:02.740554Z",
"status": "completed",
"processing_node": "local-node",
"details": {
"job_id": "5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1",
"job_config_id": "61bf1c6f-f1b7-4ef9-b097-af8304647ecf",
"name": "rotate AES keys",
"operation": "key_rotation",
"status": "completed",
"percent_done": 100,
"details": {
"num_rotated": 1
}
}
}
]
}
The output shows that two jobs have run successfully.
Viewing Details of a Particular Job Run
To view the details of a particular job run for the logged in account, run the command ksctl scheduler jobs get --id. Here, --id is the ID of the job run you are searching for.
Example:
ksctl scheduler jobs get --id "5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1"
{
"id": "5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1",
"uri": "kylo:kylo:scheduler:jobs:5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1",
"account": "kylo:kylo:admin:accounts:kylo",
"application": "ncryptify:gemalto:admin:apps:kylo",
"devAccount": "ncryptify:gemalto:admin:accounts:gemalto",
"createdAt": "2020-04-15T02:09:02.740656Z",
"name": "rotate AES keys",
"updatedAt": "2020-04-15T02:09:13.978049Z",
"job_config_id": "61bf1c6f-f1b7-4ef9-b097-af8304647ecf",
"description": "This is to rotate AES keys",
"operation": "key_rotation",
"job_config_params": {
"meta": {
"naeState": "Restricted"
},
"query": {
"algorithm": "AES"
},
"deactivate_replaced_key": 30
},
"enqueued_at": "2020-04-15T02:09:02.740554Z",
"status": "completed",
"processing_node": "local-node",
"details": {
"job_id": "5f2cf5fb-94ef-419a-9cf6-4f4045d43dc1",
"job_config_id": "61bf1c6f-f1b7-4ef9-b097-af8304647ecf",
"name": "rotate AES keys",
"operation": "key_rotation",
"status": "completed",
"percent_done": 100,
"details": {
"num_rotated": 1
}
}
}
Deleting a Job
To delete a job, run the command ksctl scheduler jobs delete --id. This command stops the job (if it is running) and deletes it without any confirmation message.
