Performance Summary
This section provides the performance summary of the Google Workspace CSE service managed through CCKM on the CipherTrust Manager server. The information on this page describes:
What is the configuration (CPU, RAM, and Disk) of each server?
Where are the clients located in relation to the servers?
What are the minimum network requirements for reliable connectivity?
How many requests per second (RPS) are handled in different scenarios?
Test Setup
This section describes the test environment in which the performance of the CCKM Google Workspace CSE service has been tested. The performance numbers provided in this section are based on this setup.
Note
Actual performance numbers in your environment can be different. The results can vary based on factors such as how and where the CipherTrust Manager is deployed, the CipherTrust Manager resources, the location of clients, the network connectivity, and how the traffic is load-balanced.
CipherTrust Manager
Virtual CipherTrust Manager k170v deployed to
Google Cloud
Amazon Web Services (AWS)
Physical Appliance k570
Client
k6 (an open-source load testing tool) is used as a client to simulate traffic between the Google Workspace CSE and the CipherTrust Manager.
The k6 client is deployed on the Google Cloud in the us-central1-a
region.
Network Requirements
The following ports must be open for CipherTrust Manager/CCKM communication:
Type | Protocol | Port Range |
---|---|---|
SSH | TCP | 22 |
HTTPS | TCP | 443 |
PostgreSQL (for Cluster) | TCP | 5432 |
Requests Per Second
The following sections list the wrap requests per second, approximate latency, and the number of virtual users for different deployment scenarios.
Note
To provide a good user experience, Google recommends a maximum latency of 200 ms (for 99% of the requests). Therefore, the performance numbers on this page are based on a latency of around 200 ms.
Google Cloud
Server Location | Client Location |
---|---|
us-central1-a | us-central1-a |
Simulated the wrap
requests for Google Workspace CSE on the CipherTrust Manager deployed on Google Cloud Platform using the k6 tool. The following table shows the handled number of requests per second (RPS), with approximate latency, and the number of virtual users for different data samples on a standalone CipherTrust Manager and a two-node CipherTrust Manager cluster connected with a load balancer.
Click a tab to view performance numbers based on two different specifications.
System Volume | Memory | CPUs | NICs |
---|---|---|---|
50 GB | 16 GB | 4 | 1 |
Click a tab to view performance numbers for a standalone CipherTrust Manager or a two-node cluster with a load balancer.
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 65.34 | 15.71 |
20 | 66.57 | 31.05 |
30 | 69.61 | 46.61 |
40 | 71.13 | 61.9 |
50 | 80.44 | 76.71 |
60 | 119.31 | 89.59 |
70 | 119.84 | 104.31 |
80 | 178.42 | 112.54 |
90 | 269.76 | 117.03 |
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 69.55 | 15.61 |
20 | 66.45 | 31.3 |
30 | 68.02 | 46.38 |
40 | 69.09 | 62.14 |
50 | 71.13 | 77.42 |
60 | 76.55 | 92.04 |
70 | 79.22 | 107.21 |
80 | 128.66 | 118.32 |
90 | 115.47 | 134.5 |
100 | 96.17 | 151.39 |
110 | 109.88 | 164 |
120 | 130.95 | 176.85 |
130 | 140.25 | 189.6 |
140 | 189.29 | 198.03 |
150 | 231.87 | 205.03 |
Comparison Graphs
Specification 1: Standalone vs Two-Node Cluster with Load Balancer
System Volume | Memory | CPUs | NICs |
---|---|---|---|
50 GB | 64 GB | 8 | 1 |
Click a tab to view performance numbers for a standalone CipherTrust Manager or a two-node cluster with a load balancer.
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 78.16 | 14.89 |
20 | 76.14 | 30.58 |
30 | 78.09 | 45.61 |
40 | 77.73 | 60.73 |
50 | 78.17 | 76.35 |
60 | 79.50 | 90.83 |
70 | 81.83 | 106.42 |
80 | 88.93 | 120.26 |
90 | 86.51 | 134.79 |
100 | 89.79 | 150.23 |
110 | 94.87 | 164.41 |
120 | 110.48 | 177.5 |
130 | 108.17 | 191.41 |
140 | 130.7 | 202.44 |
150 | 132.99 | 216.24 |
160 | 147.72 | 228.45 |
170 | 199.9 | 231.91 |
180 | 222.12 | 241.67 |
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 66.11 | 15.63 |
20 | 62.26 | 31.25 |
30 | 61.38 | 46.72 |
40 | 63.06 | 62.13 |
50 | 63.2 | 77.52 |
60 | 63.81 | 93.09 |
70 | 64.08 | 108.82 |
80 | 67.6 | 124.32 |
90 | 68.56 | 139.64 |
100 | 68.23 | 154.2 |
110 | 69.00 | 170.6 |
120 | 71.88 | 185.07 |
130 | 71.47 | 200.31 |
140 | 73.06 | 215.42 |
150 | 74.61 | 230.29 |
160 | 75.06 | 245.8 |
170 | 78.08 | 260.04 |
180 | 77.85 | 276.29 |
190 | 80.45 | 290.22 |
200 | 87.16 | 304.36 |
210 | 88.49 | 319.56 |
220 | 93.45 | 332.61 |
230 | 93.14 | 347.45 |
240 | 101.26 | 360.64 |
250 | 106.16 | 373 |
260 | 112.68 | 387.33 |
270 | 112.72 | 400.57 |
280 | 120.5 | 413.48 |
290 | 138.9 | 422.29 |
300 | 158.41 | 430.16 |
310 | 172.73 | 438.65 |
320 | 173.43 | 450.89 |
330 | 190.55 | 459.62 |
340 | 207.67 | 468.35 |
Comparison Graphs
Specification 2: Standalone vs Two-Node Cluster with Load Balancer
AWS Cloud
Server Location | Client Location |
---|---|
us-east-1b | us-central1-a |
Simulated the wrap
requests for Google Workspace CSE on the CipherTrust Manager deployed on AWS cloud using the k6 tool. The following table shows the handled number of requests per second (RPS), with approximate latency, and the number of virtual users for different data samples on a standalone CipherTrust Manager and a two-node CipherTrust Manager cluster connected with a load balancer.
Click a tab to view performance numbers based on two different specifications.
System Volume | Memory | CPUs | NICs |
---|---|---|---|
50 GB | 16 GB | 4 | 1 |
Click a tab to view performance numbers for a standalone CipherTrust Manager or a two-node cluster with a load balancer.
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 49.58 | 16.03 |
20 | 47.56 | 31.97 |
30 | 47.48 | 48.08 |
40 | 50.11 | 63.68 |
50 | 51.43 | 79.52 |
60 | 54.08 | 95.39 |
70 | 62.57 | 110.33 |
80 | 67.65 | 126.25 |
90 | 90.55 | 138.89 |
100 | 99.06 | 152.43 |
110 | 157.68 | 159.96 |
120 | 246.52 | 165.1 |
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 51.94 | 16.05 |
20 | 50.03 | 32.05 |
30 | 49.89 | 47.93 |
40 | 52.49 | 63.77 |
50 | 51.05 | 79.23 |
60 | 52.7 | 95.6 |
70 | 54.48 | 110.8 |
80 | 53.42 | 126.96 |
90 | 54.3 | 142.96 |
100 | 55.11 | 158.46 |
110 | 56.33 | 174.33 |
120 | 59.18 | 189.59 |
130 | 60.66 | 204.9 |
140 | 64.91 | 219.63 |
150 | 68.59 | 233.94 |
160 | 73.84 | 249.03 |
170 | 88.03 | 261.59 |
180 | 94.24 | 274.27 |
190 | 113.54 | 284.72 |
200 | 144.69 | 293.23 |
210 | 160.23 | 304.81 |
220 | 215.11 | 306.44 |
Comparison Graphs
Specification 1: Standalone vs Two-Node Cluster with Load Balancer
System Volume | Memory | CPUs | NICs |
---|---|---|---|
50 GB | 64 GB | 8 | 1 |
Click a tab to view performance numbers for a standalone CipherTrust Manager or a two-node cluster with a load balancer.
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 43.97 | 16.21 |
20 | 43.01 | 32.51 |
30 | 42.93 | 48.42 |
40 | 56.53 | 63.45 |
50 | 42.66 | 80.83 |
60 | 50.9 | 95.36 |
70 | 45.54 | 112.43 |
80 | 47.02 | 128.36 |
90 | 65.12 | 142.32 |
100 | 68.99 | 156.6 |
110 | 68.81 | 170.76 |
120 | 79.2 | 185.46 |
130 | 70.51 | 204.54 |
140 | 100.1 | 214.89 |
150 | 131.71 | 225.22 |
160 | 125.13 | 241.78 |
170 | 84.51 | 262.35 |
180 | 104.14 | 276.44 |
190 | 99.66 | 290.61 |
200 | 127.36 | 301.65 |
210 | 136.48 | 313.56 |
220 | 143.68 | 324.27 |
230 | 245.09 | 318.67 |
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 45.03 | 16.08 |
20 | 44.06 | 32.07 |
30 | 44.24 | 48.08 |
40 | 44.07 | 64.46 |
50 | 44.45 | 80.16 |
60 | 44.94 | 96.24 |
70 | 46.61 | 111.98 |
80 | 45.69 | 127.84 |
90 | 45.87 | 144.04 |
100 | 45.62 | 160.35 |
110 | 46.03 | 175.59 |
120 | 46.11 | 191.64 |
130 | 47.04 | 207.6 |
140 | 48.32 | 222.59 |
150 | 48.6 | 239.82 |
160 | 46.87 | 256.42 |
170 | 50.69 | 270.98 |
180 | 52.53 | 285.45 |
190 | 53.97 | 301.68 |
200 | 51.31 | 317.32 |
210 | 54.71 | 332.79 |
220 | 55.66 | 349.72 |
230 | 52.44 | 366.51 |
240 | 54.63 | 381.84 |
250 | 56.22 | 396.67 |
260 | 67.06 | 407.14 |
270 | 74.4 | 420.76 |
280 | 78.71 | 433.79 |
290 | 74.36 | 450.47 |
300 | 73.85 | 468.84 |
310 | 75.46 | 483.42 |
320 | 75.96 | 499.24 |
330 | 79.52 | 513.18 |
340 | 88.56 | 524.6 |
350 | 77.46 | 543.71 |
360 | 84 | 554.31 |
370 | 95.23 | 565 |
380 | 100.92 | 576.25 |
390 | 102.75 | 589.21 |
400 | 103.8 | 605.46 |
410 | 124.6 | 608.42 |
420 | 132.51 | 620.63 |
430 | 141.22 | 628.96 |
440 | 166.33 | 629.27 |
450 | 191.44 | 629.58 |
460 | 216.55 | 629.89 |
Comparison Graphs
Specification 2: Standalone vs Two-Node Cluster with Load Balancer
Physical Appliance
Server Location | Client Location |
---|---|
San Jose | us-central1-a |
CipherTrust Manager Configuration
System Volume | Memory | CPUs | NICs |
---|---|---|---|
2 TB | 16 GB | 1 with 4 Cores | 1 |
Simulated the wrap
requests for Google Workspace CSE on the CipherTrust Manager deployed on a physical appliance using the k6 tool. The following table shows the handled number of requests per second (RPS), with approximate latency, and the number of virtual users for different data sample for three runs.
Users | Latency (in ms) | Requests/Second |
---|---|---|
10 | 85.72 | 15.3 |
20 | 98.705 | 30.165 |
30 | 111.69 | 45.03 |
40 | 135.3 | 58.74 |
50 | 158.91 | 72.45 |
60 | 177.04 | 85.06 |
70 | 195.17 | 97.67 |
80 | 229.715 | 108.8 |
90 | 264.26 | 119.93 |
Comparison Graphs
Physical Appliance vs AWS Cloud vs Google Cloud
Recommendations
Assumption
Each user has 3 documents open and is editing them resulting in an autosave every 30 seconds, that is, 3/30=0.1 transactions per second (tps) per user.
Number of users = (throughput for around 200 ms latency)/0.1
Google Cloud
Response time compliance of around 200 ms was met for a maximum throughput of 112.54 operations per second with a standalone CipherTrust Manager k170v instance with 4 CPUs and 16 GB RAM.
The approximate number of users this configuration can handle is 1125.
Response time compliance of around 200 ms was met for a maximum throughput of 198.03 operations per second with a two-node CipherTrust Manager k170v cluster (each node with 4 CPUs and 16 GB RAM) connected with a Google Cloud load balancer.
The approximate number of users this configuration can handle is 1980.
Response time compliance of around 200 ms was met for a maximum throughput of 231.91 operations per second with a standalone CipherTrust Manager k470v instance with 8 CPUs and 64 GB RAM.
The approximate number of users this configuration can handle is 2319.
Response time compliance of around 200 ms was met for a maximum throughput of 459.62 operations per second with a two-node CipherTrust Manager k470v cluster (each node with 8 CPUs and 64 GB RAM) connected with a Google Cloud load balancer.
The approximate number of users this configuration can handle is 4596.
AWS Cloud
Response time compliance of around 200 ms was met for a maximum throughput of 159.96 operations per second with a standalone CipherTrust Manager k170v instance with 4 CPUs and 16 GB RAM.
The approximate number of users this configuration can handle is 1600.
Response time compliance of around 200 ms was met for a maximum throughput of 304.81 operations per second with a two-node CipherTrust Manager k170v cluster (each node with 4 CPUs and 16 GB RAM) connected with an AWS load balancer.
The approximate number of users this configuration can handle is 3048.
Response time compliance of around 200 ms was met for a maximum throughput of 324.27 operations per second with a standalone CipherTrust Manager k470v instance with 8 CPUs and 64 GB RAM.
The approximate number of users this configuration can handle is 3243.
Response time compliance of around 200 ms was met for a maximum throughput of 629.58 operations per second with a two-node CipherTrust Manager k470v cluster (each node with 8 CPUs and 64 GB RAM) connected with an AWS load balancer.
The approximate number of users this configuration can handle is 6296.
Conclusion
The number of users and throughput almost doubles up on moving from CipherTrust Manager k170v to k470v. Moreover, adding an additional node to the cluster also doubles up the throughput. Overall, a performance gain of 400 percent is achieved by moving from a standalone CipherTrust Manager k170v to a two-node CipherTrust Manager k470v cluster.
Moreover, an increase in virtual users by roughly 140 percent is observed over CipherTrust Manager version 2.6.