Adding a New DataNode to a CTE-protected HDFS

Use the following procedure to add a new DataNode to a CTE-protected HDFS. If not followed, HDFS encrypted files could be exposed in cleartext.

1. Install the HDFS client on the host. This option is available in Ambari when adding a new DataNode to the cluster.

2. Add the new node to the CipherTrust Manager database and make sure that host/client settings of the new node is the same as existing nodes in the cluster. See the CTE Installation for Hadoop chapter in the CTE Installation and Configuration Guide.

3. Install CTE on the new node, register to CipherTrust Manager, and run config-hadoop.sh to prepare the libraries. See the Configuring Hadoop to use CTE section in the CTE Installation and Configuration Guide.

4. Make sure that the data directories (specified in dfs.datanode.data.dir property) exist on the new node. They must have the same permission and ownership as the other existing nodes in the cluster. If necessary, create them.

5. Add the host/client to the HDFS Host Group that is guarding the cluster. This is important: Do not rely on the DataNode to create the data directories as the data replication can occur before the GuardPoints are in effect.

6. Add the DataNode service to the new node. Again this option is available through Ambari.

7. If using Kerberos, check that the keytab files are created correctly.

8. Start the DataNode service on the new node.

9. Execute some hdfs dfs shell commands to ensure that encryption/decryption of data works correctly.

Install CTE on the Cluster Nodes before Ambari Installs Hadoop

If CTE is already installed on the cluster nodes before Ambari installs the Hadoop software, Ambari can mistakenly pick up the .sec directory in configuration steps to store the HDFS data. Make sure the following properties do not contain the .sec directory:

• DataNode data directory

• NameNode data directory

• Secondary NameNode checkpoint directory

• Zookeeper directory

• yarn.nodemanager.local-dirs

• yarn.nodemanager.log-dirs

• yarn.timeline-service.leveldb-timeline-store.path

• yarn.timeline-service.leveldb-state-store.path