CSEG API for Cloud Storage Services

Google Cloud Storage

Google Cloud Storage is an Internet service to store data in the Google's cloud. Google Cloud Storage allows world-wide storing and retrieval of any amount of data and at any time. It provides a simple programming interface which enables the developers to take advantage of Google's own reliable and fast networking infrastructure to perform data operations in a secure and cost effective manner.

If expansion needs arise, developers can benefit from the scalability provided by Google's infrastructure. Google Cloud Storage provides a range of programming languages to choose from when creating applications. These languages are supported by client libraries that allow applications to communicate with Google Cloud Storage. The libraries take care of the HTTP protocol details when using the Google Cloud Storage APIs.

To use the Google Cloud Storage service, users need to activate Google Cloud Storage, create unique buckets to store data (objects), upload data, and control access.

Prerequisite

Before implementing this integration, make sure to activate the Google Cloud Storage service. Activating the Google Cloud Storage service requires a Google account and a Google APIs console project. At least one project must exist before the Google Cloud Storage service can be activated.

Activating Google Cloud Storage Service

After creating a project, the Google Cloud Storage service needs to be activated.

To activate the Google Cloud Storage service:

1. Log on to the Google APIs console (if needed). In case of multiple projects, select the project to activate Google Cloud Storage for.

2. Click the Service Accounts tab and create a service account for the project.

3. Create a key for the service account in P12 format. On clicking Create button, the file containing the private key gets downloaded (in P12 format). This certificate is used for Google Cloud Storage Service.

4. Upload the p12 certificate to CSEG using the CertCreate request mentioned below.

CertCreate

This service uploads user account certificate for cloud authentication on CSEG. This is a POST request.

URL: :///safenetcloud/storage/getTmpCertFilePath

Input Parameters

• certFile – Upload certificate file downloaded from Google Cloud Storage in P12 format
• service_account_email – service account email id for the project
• cloudName – googlecloud
• bucketName – name of the bucket in which file is to be uploaded.
• cloudCertName – (optional), name of the certificate. If not provided, the system creates the certificate file with the service_account_email, or else it will create the certificate by the name provided in cloudCertName.

Response

XML

<?xml version="1.0" encoding="UTF-8" ?>
<CertificateFileResponse>
<description>test-412@crucial-cabinet-140617.iam.gserviceaccount.com</description>
</CertificateFileResponse>

JSON

{
"CertificateFileResponse": {
"description":
"test-412@crucial-cabinet-140617.iam.gserviceaccount.com"
}
}

With CSEG integration, the CertCreate process for Google Cloud Storage is as follows:

1. User provides the certFile, service_account_email and cloudCertName (optional) to CSEG.

2. CSEG stores cert file on server and renames it with cloudCertName (if provided) else with service_account_email.

Upload File

This service uploads the file onto the Google Cloud Storage server and uses http POST method for it.

URL: :///safenetcloud/storage/cloud_upload

Input Parameters

• service_account_email – Service account email id for the project.

• ksUserName – Key Manager user name.

• ksUserPassword – Key Manager password.

• keyName – Name of the AES-256 Key to be used as data-encryption-key for client-side encryption or to be used as key-encryption-key for wrapping the data-encryption-key (provided by Google cloud) for server-side encryption (CSEK-customer supplied encryption key).

• algorithm – AES algorithm.
  Possible values are: AES, AES/CBC/NoPadding, AES/CBC/PKCS5Padding, AES/ECB/NoPadding, AES/ECB PKCS5Padding. AES uses ECB/PKCS5Padding if not mentioned explicitly. For server-side encryption, only AES is to be used.

• target – Name by which the file is to be uploaded.

• file – File to be uploaded (attach the file with this request).

• cloudName – googlecloud (use the cloud name the way it is provided here).

• bucketName – Name of the bucket in which the file is to be uploaded.

• isClientSide – (optional), true for client-side encryption else false. Default value is true. This parameter determines if the encryption is to be performed at client-side (Thales) or server-side (Google cloud).

• canKeyRotate – (optional), true or false. Default value is true. This parameter enables rotation of encryption key (for server-side encryptions) for future operations. The client-side encryption operation ignores this parameter.

• cloudCertName – (optional), Name of the certificate. If not provided, the system finds the certificate with the service account Email Id name, or otherwise it will find the certificate by the name provided in cloudCertName.

• certAlias – (optional)

• certPassword – (optional)

Response

XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<FileUploadResponse>
<description>testfile.txt is uploaded as testfile successfully</description>
</FileUploadResponse>

JSON

{
"FileUploadResponse": {
"description": "testfile.txt is uploaded as testfile successfully"
}
}

The following figure describes encrypting a file and uploading it to Google Cloud Storage (client-side encryption).

With CSEG, the upload file process for Google Cloud Storage is as follows for the client-side encryption:

1. User provides the key name, data, and Google client secrets (Service account email and cloud access certificate) to CSEG.

2. CSEG sends Google client secrets to retrieve access token from Google.

3. Core solution sends the data and the encryption key name to the Encryption client.

4. Encryption client connects to Key Manager, locates the key name on Key Manager, and fetches the key.

5. Encryption client encrypts the data using the encryption key.

6. Encryption client sends the encrypted data to core solution.

7. CSEG uploads the encrypted data to Google Cloud Storage under a specified bucket.

With CSEG, the upload file process for Google Cloud Storage is as follows for the server-side encryption:

1. User provides the key name, file, and Google client secrets (Service account email and cloud access certificate) to CSEG.

2. CSEG connects to Key Manager, locates the key name on Key Manager, and fetches the key.

3. Google generates a random data-encryption-key using which the file is encrypted and stored in the specified bucket.

4. The data-encryption-key is then wrapped using the key (key-encryption-key) provided by CSEG.

Rotate Key

This service rotates the key-encryption-key, which was used to wrap the data-encryption-key for the server-side encryption. The service uses the http POST method.

URL: :///safenetcloud/storage/cloud_rotateKey

Input Parameters

• service_account_email – Service account email id for the project.

• ksUserName – Key Manager user name.

• ksUserPassword – Key Manager user password.

• oldKeyName – Name of the old AES-256 key used for wrapping the data-encryption-key.

• newkeyName – Name of the new AES-256 key to be used for wrapping the data-encryption-key.

• fileName – name of the file used during the upload request.

• newFileName – (optional), new name to be used for the file. If new file name is not provided old file name will be used.

• algorithm – AES

• cloudName – googlecloud (use the cloud name the way it is provided here.)

• bucketName – Name of the bucket that has the file.

• newBucketName – (optional), new bucket name where the file is to be stored. If new bucket name is not provided old bucket will continue to be used.

• cloudCertName – (optional), Name of the certificate. If not provided, the system finds the certificate with the service account Email ID name or else it will find the certificate by the name provided in cloudCertName.

• certAlias – (optional)

• certPassword – (optional)

Response

JSON

{
"String": "rotation is successful"
}

• If the request fails, corresponding error message is displayed.

• If a file is uploaded using server-side encryption and canKeyRotate as false, and user is required to rotate the encryption key. In such case, download the file and upload it again with canKeyRotate as true.

Download File

This service downloads the file from the Google Cloud Storage server and uses http POST method for it.

URL: :///safenetcloud/storage/cloud_downloadFile

Input Parameters

• service_account_email – Service account email id for the project.

• ksUserName – Key Manager user name.

• ksUserPassword – Key Manager user password.

• keyName – Name of the AES-256 Key that had been used for uploading the file.

• algorithm – AES algorithm, the value mentioned here should be same as the one used for uploading the file.

• Possible values are: AES, AES/CBC/NoPadding, AES/CBC/PKCS5Padding, AES/ECB/NoPadding, AES/ECB/PKCS5Padding. AES uses ECB/PKCS5Padding if not mentioned explicitly. For server-side encryption, only AES is to be used.

• target – Name of the file that you want to download. The name provided here should be same as one used during uploading the file.

• cloudName – googlecloud (use the cloud name the way it is provided here.)

• bucketName – Name of the bucket from which the file is to be downloaded.

• isClientSide – (optional), true or false. Default value is true. This parameter indicates if the encryption was performed at client-side (Thales) or server-side (Google cloud). Provide the value true or false as used while uploading the file. For example, if isClientSide parameter was set as true while uploading the file, use the value true while downloading the file else the download will fail.

• cloudCertName – (optional), Name of the certificate. If not provided, the system finds the certificate with the service account Email ID name or else it will find the certificate by the name provided in cloudCertName.

• certAlias – (optional)

• certPassword – (optional)

Response

File gets downloaded. In case the download fails, the corresponding error message is displayed.

The following figure describes how a file is downloaded from Google Cloud Storage and its decryption (client-side decryption).

With this integration, the download file process for Google Cloud Storage is as follows for the client-side encryption:

1. User provides the key name and Google client secrets to CSEG.

2. CSEG retrieves data from Google Cloud Storage.

3. CSEG checks metadata for Isencrypted flag. If the flag is not found or set to false, follow step 7 else follow steps 4, 5 and 6 in sequence before step 7.

4. Core solution sends the data and the key name to the Encryption client.

5. Encryption client connects to Key Manager, locates the key name on Key Manager, and fetches the key.

6. Encryption client decrypts the data using the key received from Key Manager.

7. Encryption client sends the decrypted data to core solution.

8. CSEG sends the decrypted data to the user.

With CSEG, the download file process from Google Cloud Storage is as follows for the server-side encryption:

1. User provides the key name (key-encryption-key) and Google client secrets to CSEG.

2. CSEG connects to Key Manager, locates the key name on Key Manager, and fetches the key.

3. CSEG connects to Google Cloud Storage.

4. Google Cloud Storage unwraps the data-encryption-key using the key-encryption-key.

5. Google Cloud Storage decrypts the file from the specified bucket using the data-encryption-key.

6. Google Cloud Storage provides the file to the CSEG.

DeleteRequest

This service deletes the file from the Google Cloud Storage and use DELETE method for it.

URL: :///safenetcloud/storage/cloud_deleteFile

Input Parameters

• target – path of the file (in a bucket), which is to be deleted.

• service_account_email – service account email id for the project.

• bucketName – name of the Google cloud bucket.

• cloudCertName – (optional), name of the certificate. If not provided, the system find the certificate with the service account Email ID name, or else it will find the certificate by the name provided in cloudCertName.

• cloudName – googlecloud

Response

XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<FileDeleteResponse>
<description>/here/tester.txt</description>
</FileDeleteResponse>

JSON

{
"FileDeleteResponse": {
"description": "/here/tester.txt"
}
}

With this integration, the delete request process for Google Cloud Storage is as follows:

1. User provides bucketName, target (complete file path in bucket), service_account_email and cloudCertName (optional).

2. CSEG searches Google client secrets with cloudCertName, (if provided), on server; else it searches with service_account_email.

3. CSEG makes delete object request on Google Cloud Storage for target file name.

ListFiles

This service lists all the files available in the given bucket in request format. This is a GET request.

URL: :///safenetcloud/storage/cloud_folderInfo?cloudName=googlecloud

Input Parameters

• bucketName – name of the google cloud bucket.

• service_account_email – service account email id for the project.

• cloudName – googlecloud

• cloudCertName – (optional), name of the certificate. If not provided, the system finds the certificate with the service account Email ID name, or else it will find the certificate by the name provided in cloudCertName.

Response

XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<FilesInBucket>
<file>this_is_test.txt</file>
</FilesInBucket>

JSON

{
"FilesInBucket": {
"file": ["this_is_test.txt"]
}
}

With this integration, the list file process for Google Cloud Storage is as follows:

1. User provides bucketName, service_account_email and cloudCertName (optional).

2. CSEG searches Google client secrets with cloudCertName, (if provided), on server; else it searches with service_account_email.

3. CSEG retrieves list of objects on Google Cloud Storage for bucketName.

4. CSEG sends the list to user.

EncryptFiles

This service encrypts the un-encrypted files available in the given bucket. This is a POST request.

The files already encrypted without using CSEG WebService API will also get further encrypted.

Input Parameters

• service_account_email – service account email id for the project.

• ksUserName – Key Manager username.

• ksUserPassword – Key Manager password.

• keyName – name of the AES-256 Key.

• algorithm – AES algorithm.
  Possible values are: AES, AES/CBC/NoPadding, AES/CBC/PKCS5Padding, AES/ECB/NoPadding, AES/ECB/PKCS5Padding. AES uses ECB/PKCS5Padding if not mentioned explicitly. For server-side encryption, only AES is to be used.

• cloudName –googlecloud

• bucketName – name of the Google cloud bucket.

• isClientSide – (optional), true for client-side encryption else false. Default value is true. This parameter determines if the encryption is to be performed at client-side (Thales) or server-side (Google cloud).

• canKeyRotate – (optional), true or false. Default value is true. This parameter enables rotation of encryption key (for server-side encryptions) for future operations. The client-side encryption operation ignores this parameter.

• cloudCertName – (optional), name of the certificate. If not provided, the system finds the certificate with the service_account_email, or else it will find the certificate by the name provided in cloudCertName.

• certAlias – (optional)

• certPassword – (optional)

Response

XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<FilesEncryptedList>
<successfulList>
<file>tester.txt</file>
</successfulList>
<unsuccessfulList>
<file>this_is_test.txt</file>
</unsuccessfulList>
</FilesEncryptedList>

JSON

{
"FilesEncryptedList": {
"successfulList": {
"file": ["tester.txt"]
},
"unsuccessfulList": {

"file": ["this_is_test.txt"] } } }

With this integration, the encrypt file process for Google Cloud Storage is as follows (for client-side encryption):

1. User provides bucketName, service_account_email, key, transformation and cloudCertName (optional).

2. CSEG searches Google client secrets with cloudCertName, (if provided), on server; else it searches with service_account_email.

3. CSEG retrieves list of objects on Google Cloud Storage for bucketName.

4. CSEG parses the list and check metadata of each object for IsEncrypted flag, if flag found and set to true, it skips that object and moves to next.

5. If IsEncrypted flag not found or set to false, CSEG downloads object’s data.

6. Core solution sends the data and the key name to the Encryption client.

7. Encryption client connects to Key Manager, locates the key name on Key Manager, and fetches the key.

8. Encryption client encrypts the data using the key received from Key Manager.

9. Encryption client sends the encrypted data to core solution.

10. CSEG uploads this data on Google Cloud Storage. And add its entry in the successful files list.

11. If it fails to encrypt/upload any data, it adds its entry into unsuccessful files list.

12. CSEG returns both of the lists to user.

With this integration, the encrypt file process for Google Cloud Storage is as follows (for server-side encryption):

1. User provides bucketName, service_account_email, key, transformation and cloudCertName (optional).

2. CSEG searches Google client secrets with cloudCertName, (if provided), on server; else it searches with service_account_email.

3. CSEG retrieves list of objects on Google Cloud Storage for bucketName.

4. CSEG parses the list and check metadata of each object for IsEncrypted flag, if flag found and set to true, it skips that object and moves to next.

5. CSEG connects to Key Manager, locates the key name on Key Manager, and fetches the key.

6. If IsEncrypted flag not found or set to false, CSEG downloads object’s data.

7. CSEG sends these object’s data to Google Cloud Storage again.

8. Google generates a random data-encryption-key using which the files corresponding to the object’s data are encrypted and stored in the specified bucket.

9. The data-encryption-key is then wrapped using the key (key-encryption-key) provided by CSEG.