Performance of CipherTrust Manager Cluster
This document provides performance statistics for various operations on a CipherTrust Manager cluster when subjected to different connector loads. Understanding these metrics can help customers anticipate resource consumption on their CipherTrust Manager cluster under similar workloads.
Understanding the Performance Statistics Tables
The following sections present performance statistics using several common table formats. Here's a brief overview of what each table type generally represents:
Request Statistics: This table typically details the number of requests made for specific operations, the number of failures, average, minimum, and maximum response times (often in milliseconds), the average size of the data transferred (in bytes), the number of requests per second (RPS), and failures per second. An aggregated summary across all operations in that section is usually included.
Response Time Statistics: This table provides a more granular view of system responsiveness by showing percentile data (for example, 50th, 60th, 70th, 80th, 90th, 95th, 99th, and 100th percentile) for the response times of the operations detailed in the "Request Statistics" table. An aggregated summary is also typically included.
Failures Statistics: When present, this table lists the specific errors encountered during the tests (for example, HTTP error codes and messages) and the number of times each error occurred. It provides context for any non-zero values in the "# Fails" column of the "Request Statistics" table.
Exceptions Statistics: This table, when included, lists any software exceptions that occurred during the tests. It typically shows the count of each exception, the error message, a detailed traceback (the sequence of calls that led to the error), and the node(s) where the exception occurred. This information is crucial for diagnosing underlying software issues.
The following connector and operation types are covered.
KMIP Clients
Symmetric/Asymmetric Key Creation
The following tables present performance statistics for KMIP clients performing symmetric (AES) and asymmetric (RSA) key creation, as well as key retrieval operations on a CipherTrust Manager cluster.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
POST | Create AES Key | 1123 | 0 | 818 | 209 | 4176 | 0 | 6.1 | 0.0 |
POST | Create RSA Key | 1147 | 0 | 820 | 180 | 4218 | 0 | 6.3 | 0.0 |
GET | Get Key by ID | 5333 | 0 | 554 | 93 | 3057 | 1866 | 29.1 | 0.0 |
GET | Get Key by another Node | 3074 | 0 | 598 | 100 | 3037 | 1142 | 16.8 | 0.0 |
Aggregated | 10677 | 0 | 623 | 93 | 4218 | 1261 | 58.3 | 0.0 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
POST | Create AES Key | 570 | 630 | 700 | 880 | 1600 | 2700 | 3900 | 4200 |
POST | Create RSA Key | 570 | 620 | 720 | 880 | 1600 | 2600 | 3900 | 4200 |
GET | Get Key by ID | 370 | 410 | 460 | 540 | 1500 | 1900 | 2200 | 3100 |
GET | Get Key by another Node | 390 | 430 | 480 | 560 | 1800 | 2000 | 2200 | 3000 |
Aggregated | 410 | 460 | 520 | 690 | 1600 | 2000 | 2900 | 4200 |
Charts
REST Clients
Symmetric/Asymmetric Key Creation
The following tables present performance statistics for REST API clients performing symmetric (AES) and asymmetric (RSA) key creation, as well as key retrieval operations on a CipherTrust Manager cluster.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
POST | Create AES Key | 515 | 0 | 9492 | 57 | 77498 | 1090 | 2.9 | 0.0 |
POST | Create RSA Key | 487 | 0 | 16409 | 532 | 67990 | 2642 | 2.7 | 0.0 |
GET | Get Key by ID | 2153 | 0 | 9360 | 7 | 123749 | 1387 | 12.0 | 0.0 |
GET | Get Key by another Node | 1211 | 0 | 68 | 8 | 623 | 2642 | 6.7 | 0.0 |
Aggregated | 4366 | 0 | 7585 | 7 | 123749 | 1840 | 24.3 | 0.0 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
POST | Create AES Key | 5400 | 7900 | 11000 | 14000 | 24000 | 37000 | 58000 | 77000 |
POST | Create RSA Key | 13000 | 16000 | 19000 | 26000 | 33000 | 43000 | 57000 | 68000 |
GET | Get Key by ID | 97 | 240 | 8800 | 18000 | 34000 | 46000 | 74000 | 124000 |
GET | Get Key by another Node | 34 | 46 | 62 | 100 | 170 | 230 | 470 | 620 |
Aggregated | 120 | 530 | 6700 | 14000 | 26000 | 39000 | 64000 | 124000 |
Charts
Azure CRUD Key Operations
The following tables present performance statistics for CCKM (Cloud Key Manager) operations involving Azure, specifically focusing on Create, Read (Get), Update, and Delete (CRUD) operations for keys, including soft and hard deletes, and backup deletion.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
POST | Create Key | 108 | 0 | 2888 | 1106 | 16318 | 2300 | 0.4 | 0.0 |
POST | Delete Backup | 98 | 0 | 438 | 282 | 1168 | 15 | 0.3 | 0.0 |
GET | Get Key | 108 | 0 | 29 | 13 | 50 | 2329 | 0.4 | 0.0 |
POST | Hard Delete Key | 113 | 14 | 1056 | 696 | 9972 | 2097 | 0.4 | 0.0 |
POST | Soft Delete Key | 100 | 0 | 1022 | 675 | 6348 | 2283 | 0.3 | 0.0 |
PATCH | Update Key | 101 | 0 | 1293 | 786 | 17662 | 2266 | 0.3 | 0.0 |
Aggregated | 628 | 14 | 1131 | 13 | 17662 | 1904 | 2.1 | 0.0 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
POST | Create Key | 1800 | 1900 | 2000 | 2300 | 8300 | 8900 | 16000 | 16000 |
POST | Delete Backup | 340 | 410 | 480 | 580 | 720 | 760 | 1200 | 1200 |
GET | Get Key | 28 | 40 | 42 | 44 | 45 | 48 | 51 | 51 |
POST | Hard Delete Key | 920 | 990 | 1000 | 1100 | 1200 | 1400 | 2800 | 10000 |
POST | Soft Delete Key | 930 | 990 | 1100 | 1200 | 1300 | 1400 | 6300 | 6300 |
PATCH | Update Key | 1100 | 1200 | 1200 | 1300 | 1400 | 1500 | 2000 | 18000 |
Aggregated | 900 | 1000 | 1200 | 1400 | 1800 | 2100 | 10000 | 18000 |
Failures Statistics
Method | Name | Error | Occurrences |
---|---|---|---|
POST | Hard Delete Key | 409 Client Error: Conflict for url: Hard Delete Key | 14 |
Charts
Azure Key Rotation Sync/Refresh
The following tables present performance statistics for CCKM operations related to Azure key management, focusing on key rotation, synchronization, and refresh actions.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 16 | 0 | 15 | 6 | 27 | 859 | 0.1 | 0.0 |
POST | Rotate Key | 213 | 197 | 1484 | 6 | 24056 | 157 | 0.7 | 0.7 |
POST | Sync Key | 118 | 0 | 346 | 261 | 1323 | 543 | 0.4 | 0.0 |
Aggregated | 347 | 197 | 1029 | 6 | 24056 | 321 | 1.2 | 0.7 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 18 | 19 | 20 | 20 | 22 | 27 | 27 | 27 |
POST | Rotate Key | 19 | 20 | 20 | 21 | 25 | 18000 | 24000 | 24000 |
POST | Sync Key | 300 | 310 | 310 | 360 | 470 | 560 | 1200 | 1300 |
Aggregated | 21 | 26 | 270 | 300 | 410 | 1200 | 21000 | 24000 |
Failures Statistics
Method | Name | Error | Occurrences |
---|---|---|---|
POST | Rotate Key | 409 Client Error: Conflict for url: Rotate Key | 197 |
Exceptions Statistics
Count | Message | Traceback | Nodes |
---|---|---|---|
1 | list index out of range | File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 340, in run self.execute_next_task() File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 373, in execute_next_task self.execute_task(self._task_queue.pop(0)) File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 487, in execute_task task(self.user).run() File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 325, in run self.on_start() File "/root/cte_client_dir/vaqa/Engine/lib/../../testcases/kylo/rest/performance/cckm/azure/locust/azure_key_rotate_sync_perf.py", line 123, in on_start self.vault_id = self.vault_id_list[self.vault_index] | local |
Charts
AWS Key Rotation Sync/Refresh
The following tables present performance statistics for CCKM operations related to AWS key management, focusing on key rotation, synchronization, and refresh actions.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 16 | 0 | 20 | 7 | 28 | 848 | 0.1 | 0.0 |
POST | Rotate Key | 367 | 351 | 913 | 7 | 24099 | 159 | 1.2 | 1.2 |
POST | Sync Key | 147 | 96 | 2668 | 265 | 14870 | 322 | 0.5 | 0.3 |
Aggregated | 530 | 447 | 1373 | 7 | 24099 | 225 | 1.8 | 1.5 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 22 | 23 | 24 | 25 | 28 | 28 | 28 | 28 |
POST | Rotate Key | 20 | 21 | 22 | 24 | 27 | 180 | 24000 | 24000 |
POST | Sync Key | 1200 | 1500 | 1800 | 2500 | 11000 | 12000 | 14000 | 15000 |
Aggregated | 22 | 25 | 280 | 760 | 2200 | 11000 | 24000 | 24000 |
Failures Statistics
Method | Name | Error | Occurrences |
---|---|---|---|
POST | Rotate Key | 409 Client Error: Conflict for url: Rotate Key | 351 |
POST | Sync Key | 409 Client Error: Conflict for url: Sync Key | 96 |
Charts
GCP Key Rotation Sync/Refresh
The following tables present performance statistics for CCKM operations related to Google Cloud Platform (GCP) key management, focusing on key rotation, synchronization, and refresh actions.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 14 | 0 | 17 | 7 | 21 | 846 | 0.0 | 0.0 |
POST | Rotate Key | 199 | 185 | 1393 | 7 | 24054 | 157 | 0.7 | 0.6 |
POST | Sync Key | 78 | 77 | 327 | 254 | 1183 | 249 | 0.3 | 0.3 |
Aggregated | 291 | 262 | 1041 | 7 | 24054 | 215 | 1.0 | 0.9 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 19 | 19 | 20 | 20 | 20 | 22 | 22 | 22 |
POST | Rotate Key | 20 | 20 | 20 | 21 | 34 | 18000 | 24000 | 24000 |
POST | Sync Key | 300 | 300 | 310 | 310 | 400 | 450 | 1200 | 1200 |
Aggregated | 20 | 22 | 260 | 300 | 310 | 1200 | 24000 | 24000 |
Failures Statistics
Method | Name | Error | Occurrences |
---|---|---|---|
POST | Rotate Key | 409 Client Error: Conflict for url: Rotate Key | 185 |
POST | Sync Key | 409 Client Error: Conflict for url: Sync Key | 77 |
Exceptions Statistics
Count | Message | Traceback | Nodes |
---|---|---|---|
4 | list index out of range | File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 340, in run self.execute_next_task() File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 373, in execute_next_task self.execute_task(self._task_queue.pop(0)) File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 487, in execute_task task(self.user).run() File "/usr/local/lib/python3.8/dist-packages/locust/user/task.py", line 325, in run self.on_start() File "/root/cte_client_dir/vaqa/Engine/lib/../../testcases/kylo/rest/performance/cckm/gcp/locust/gcp_keys_rotate_sync_perf.py", line 132, in on_start self.key_ring_id = self.key_rings[self.key_ring_index] | local |
Charts
Oracle Key Rotation Sync/Refresh
The following tables present performance statistics for CCKM operations related to Oracle Cloud Infrastructure (OCI) key management, focusing on key rotation, synchronization, and refresh actions.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 16 | 0 | 15 | 5 | 22 | 849 | 0.1 | 0.0 |
POST | Rotate Key | 212 | 196 | 1462 | 6 | 24064 | 157 | 0.7 | 0.7 |
POST | Sync Key | 266 | 262 | 873 | 641 | 2244 | 249 | 0.9 | 0.9 |
Aggregated | 494 | 458 | 1098 | 5 | 24064 | 229 | 1.6 | 1.5 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 18 | 19 | 20 | 20 | 21 | 23 | 23 | 23 |
POST | Rotate Key | 19 | 20 | 21 | 21 | 31 | 18000 | 24000 | 24000 |
POST | Sync Key | 830 | 850 | 880 | 930 | 1100 | 1200 | 2000 | 2200 |
Aggregated | 710 | 790 | 830 | 880 | 1000 | 1400 | 24000 | 24000 |
Failures Statistics
Method | Name | Error | Occurrences |
---|---|---|---|
POST | Rotate Key | 409 Client Error: Conflict for url: Rotate Key | 196 |
POST | Sync Key | 409 Client Error: Conflict for url: Sync Key | 262 |
Charts
SAP Key Rotation Sync/Refresh
The following tables present performance statistics for CCKM operations related to SAP key management, focusing on key rotation, synchronization, and refresh actions.
Request Statistics
Method | Name | # Requests | # Fails | Average (ms) | Min (ms) | Max (ms) | Average size (bytes) | RPS | Failures/s |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 16 | 0 | 12 | 7 | 25 | 849 | 0.1 | 0.0 |
POST | Rotate Key | 122 | 106 | 2675 | 6 | 24060 | 153 | 0.4 | 0.4 |
POST | Sync Key | 385 | 376 | 559 | 259 | 13904 | 174 | 1.3 | 1.3 |
Aggregated | 523 | 482 | 1036 | 6 | 24060 | 189 | 1.7 | 1.6 |
Response Time Statistics
Method | Name | 50%ile (ms) | 60%ile (ms) | 70%ile (ms) | 80%ile (ms) | 90%ile (ms) | 95%ile (ms) | 99%ile (ms) | 100%ile (ms) |
---|---|---|---|---|---|---|---|---|---|
GET | Rotate Key | 9 | 9 | 18 | 18 | 19 | 25 | 25 | 25 |
POST | Rotate Key | 19 | 20 | 21 | 23 | 18000 | 21000 | 24000 | 24000 |
POST | Sync Key | 310 | 320 | 400 | 470 | 610 | 790 | 7100 | 14000 |
Aggregated | 310 | 310 | 350 | 460 | 620 | 1800 | 21000 | 24000 |
Failures Statistics
Method | Name | Error | Occurrences |
---|---|---|---|
POST | Rotate Key | 409 Client Error: Conflict for url: Rotate Key | 105 |
POST | Rotate Key | 401 Client Error: Unauthorized for url: Rotate Key | 1 |
POST | Sync Key | 500 Server Error: Internal Server Error for url: Sync Key | 376 |