Please Note:

Adding Data Stores

Local Data Stores

Local Data Stores, i.e. local storage and local memory are standard scan locations. To add a local data store use the Add Data Store wizard to add a local data store.

1. Select Store Type

In the Select Store Type screen of the wizard select Local Storage in the * Select Data Store Category.
From the Select Local Storage Type drop-down list, select Local Storage.
Select Type shows types of data storage. By default, the drop-down list shows all types of data stores. When a category is selected under Select Data Store Category, the label Select Type is changed to reflect the selection. For example, for Local Storage, the label becomes Select Local Storage Type.
Click Next to go on to the Configure Connection screen.

2. Configure Connection

The Configure Connection screen is displayed.
Specify Hostname/IP of the machine where the local data store resides. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Note
Local data stores need a DDC Agent installed on the same host.
Click Next to go to the General Info screen.

3. General Info

Configure the General Info part per the information in General Info.
Click Next to go to the Add Tags & Access Control screen.

4. Add Tags & Access Control

Configure the Tags & Access Control par per the information in Tags & Access Control.
Click Save. The newly created data store appears on the Data Stores page. By default, data stores are displayed in alphabetic order by name. Depending on the number of entries per page, you might need to navigate to other pages to view the newly created data store.

Network Data Stores

DDC supports two types of Network Storage types as data stores: Linux Network File Share (NFS) and Windows share (SMB/CIFS).

Note

SMB/CIFS is supported for Windows only. Currently, the SMB implementation on Linux (Samba) is not supported. Also, we cannot guarantee that NFS type data stores on MAC will work properly.

To create a Windows Network Storage data store:

Use a Windows Proxy Agent.
Ensure that the target storage is accessible from the Proxy agent host.

To create a Linux Network Storage data store:

Use a Linux Proxy Agent.
The target storage path must be mounted on the Proxy agent host.

Note

For both types of these data stores, the credentials to access the target storage must have the minimum permissions required to scan it. Bear in mind that data discovery or scanning of data requires read access.

1. Select Store Type

In the Select Store Type screen of the wizard select Network Storage in the Select Data Store Category.
From the Select Network Storage Type drop-down list select:
- SMB/CIFS Share - for a Linux Data Store.
- NFS Share - for a Windows Data Store.
Click Next to go on to the Configure Connection screen.

2. Configure Connection

In the Configure Connection screen of the wizard, provide the following configuration details for your data store:

Linux Data Store

Hostname/IP - a valid hostname, IP address, or URI of the data store.
Share Path - a valid NFS path, it must begin with a slash (“/”). The path must be set to the mount path on the Proxy host.
Agent Hostname/IP - a valid hostname, IP address, or URI of the host where the DDC agent resides.
Mount Point (On Proxy Agent) - the mount path on the Proxy host (for the Share Path above). See also "Mounting an NFS Share".

Click Next to go to the General Info screen.

Windows Data Store

Hostname/IP - a valid hostname, IP address, or URI of the data store.
Share Name - a valid Windows share name. These characters are not allowed in the Share Name: =*?,<>|;:+[]"/\
Caution
Do not confuse the Share Name with the Network Path. In Windows, the Share Name is typically set in the Advanced Sharing settings in the folder sharing properties.
Credentials - provide a valid username and password. Use the appropriate user name format for the target Windows hosts credentials:
- <domain\username> - target host resides in the same Active Directory domain as the Windows proxy agent.
- <target_hostname\username> - target host does not reside in the same Active Directory domain as the Windows proxy agent.
Tip
DNS / DNS reverse resolution may increase the time to scan. Make sure that you optimize your DNS resolution or modify the agent's hosts file to skip the external DNS resolution as indicated in this technical note.

Click Next to go to the General Info screen.

3. General Info

Configure the General Info part per the information in General Info.
Click Next to go to the Add Tags & Access Control screen.

4. Add Tags & Access Control

Configure the Tags & Access Control par per the information in Tags & Access Control.
Click Save. The newly created data store appears on the Data Stores page. By default, data stores are displayed in alphabetic order by name. Depending on the number of entries per page, you might need to navigate to other pages to view the newly created data store.

Database Data Stores

The tables in the PosgresQL database must have a Primary Key (PK), otherwise the scan results may be incomplete.
PostgreSQL by default blocks remote connections to the PostgreSQL server, so you have to configure it to allow remote connections. For instructions, see Allowing Remote Connections to PostgreSQL Server.
To connect to Microsoft SQL DDC requires the ODBC drivers to be installed in the same environment as the DDC agent. If DDC cannot find a suitable agent, make sure that these drivers are installed. If necessary, upgrade them to the latest available version. Thus, if your MSSQL Server is configured with TLS 1.2 only, install the ODBC Driver 17 (or newer) for MSSQL Server.
Before adding an Oracle database, make sure that you have the schema name or the database and service name to hand. For information on how to get this information, see Obtaining the Oracle Configuration Details.

Use the Add Data Store wizard to add a database type data store. Adding a database data store involves the steps described in the following sections.

1. Select Store Type

In the Select Store Type screen of the wizard select Database in the Select Data Store Category.
From the Select Database Type drop-down list select:
- IBM DB2: Select to add an IBM DB2 database.
- Oracle: Select to add an Oracle database
- Microsoft SQL: Select to add a Microsoft SQL database.
- PostgreSQL: Select to add a PostgreSQL database.
- SAP HANA: Select to add a SAP HANA database.
- MySQL: Select to add a MySQL database.
- MongoDB: Select to add a Mongo DB database.
Click Next to go on to the Configure Connection screen.

2. Configure Connection

In the Configure Connection screen of the wizard, provide the following configuration details for your data store:

IBM DB2

Note

DB2: Windows Agent built-in drivers are required to connect to a DB2 data store.

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for IBM DB2 is 50000.
In the Database field, specify the name of the database service.
In the Authentication part, specify valid user credentials, User and Password.

Oracle

Note

Windows and Linux Agent built-in drivers are required to connect to an Oracle data store.

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for Oracle is 1521.
In the Database field, specify the name of the database service.
Use a schema nameSCHEMA or a database name and service nameDB(SERVICE_NAME=XXX). For example:
* Schema name: HR
* Database name and service name: MYDB(SERVICE_NAME=XE)
Note
If you are using Oracle 12x, or if the Oracle database displays a TNS: protocol adapter error, you must specify a database and service name in the Database field. For example: HR(SERVICE_NAME=XE)
In the Authentication part, specify valid user credentials, User and Password.

Microsoft SQL

Note

Windows host ODBC drivers are sufficient to connect to a MS SQL data store. ODBC Drivers version 17 are required to support TLS 1.2 connections.

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for Microsoft SQL is 1433.
In the Database field, specify the name of the database service.
In the Authentication part, specify valid user credentials, User and Password.

PostgreSQL

Note

Windows and Linux agent built-in drivers are required to connect to a PostgreSQL data store. The built-in driver does not support password authentication with 'scram-sha-256' method.

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for PostgreSQL MongoDB is 5432.
In the Database field, specify the name of the database service.
In the Authentication part, specify valid user credentials, User and Password.

SAP HANA

Note

Windows Agent built-in drivers are required to connect to a SAP Hana data store. If the Agent host has SAP HANA ODBC drivers installed, the Agent will use those drivers instead of its built-in drivers.

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for SAP HANA is 30015.
In the Database field, specify the name of the database service.
In the Authentication part, specify valid user credentials, User and Password.

MySQL

Note

Windows and Linux Agent built-in drivers are required to connect to a MySQL data store. The built-in driver does not support password authentication with 'caching_sha2_password' method.

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for MySQL is 3306.
In the Authentication part, specify valid user credentials, User and Password.

MongoDB

Specify Hostname/IP of the database server. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Specify Port of the database server. The port must be a number between 1 and 65535. The default port for MongoDB is 27017.
In the Authentication Database field, specify the name of the database service.
User and Password - specify the Username, password and authentication database in the following manner:
- Username: <authentication_database>/<user_name>
  Example: pgdb1/user1
- Password: <password>
  Example: myPassword123

Click Next to go to the General Info screen.

3. General Info

Configure the General Info part per the information in General Info.
Click Next to go to the Add Tags & Access Control screen.

4. Add Tags & Access Control

Configure the Tags & Access Control par per the information in Tags & Access Control.
Click Save. The newly created data store appears on the Data Stores page. By default, data stores are displayed in alphabetic order by name. Depending on the number of entries per page, you might need to navigate to other pages to view the newly created data store.

Allowing Remote Connections to PostgreSQL Server

PostgreSQL by default blocks all connections that are not from the PostgreSQL database server itself. This means that to scan a PostgreSQL database, the Agent must either be installed on the PostgreSQL database server itself (not recommended), or the PostgreSQL server must be configured to allow remote connections.

To configure a PostgreSQL server to allow remote connections:

On the PostgreSQL database server, locate the pg_hba.conf configuration file. On a Unix-based server, the file is usually found in the /var/lib/postgresql/data directory.
Open pg_hba.conf in a text editor, as root.
Add the following to the end of the file:
```
# Syntax:
# host <database_name> <postgresql_user_name> <agent_host_address> <auth-method>
host all all all md5
```
The above configuration allows any remote client to connect to the PostgreSQL server if a correct user name and password is provided. For a more secure configuration, use configuration statements that are specific to a database, user or IP address. For example:
Note
host database_A scan_user 172.17.0.0/24 md5
Open the postgresql.conf file and modify the Connections and Authentication section.
You should change the #listen_addresses = 'localhost' line to this:
```
listen_addresses = '*'
```
Tip
You can also use a specific IP address of the PostgreSQL server to listen on, instead of the global *.
Save the file and restart the PostgreSQL service.

Obtaining the Oracle Configuration Details

To find the schema for the current user you can run this query:
```
SELECT SYS_CONTEXT('USERENV','CURRENT_SCHEMA') FROM DUAL;
```

To find the schema (or owner) for a particular table, you can run:

SELECT DISTINCT OWNER, OBJECT_NAME FROM DBA_OBJECTS 
WHERE OBJECT_TYPE = 'TABLE' 
AND OBJECT_NAME = '[your table]';

To find all tables for a particular schema (or owner), you can run:

SELECT DISTINCT OWNER, OBJECT_NAME FROM DBA_OBJECTS
WHERE OBJECT_TYPE = 'TABLE'
AND OWNER = '[your schema]';

To get the information about the service name contact your Oracle database administrator.

Big Data Stores

DDC supports two types of Big Data data stores:

Hadoop Cluster
Teradata (Teradata 14.10.00.02 and above)

Hadoop Cluster Considerations

Nodes where data blocks distributed by HDFS are stored are called DataNodes. DataNodes are treated as “slaves” in a Hadoop cluster.
A node that maintains the index of directories and files and manages data blocks stored on DataNodes is called a NameNode. A NameNode is treated as “master” in a Hadoop cluster.

Teradata Considerations

Teradata data stores require Teradata Tools and Utilities 16.10.xx to be installed on the Agent. These utilities are also mandatory:
- ODBC Driver for Teradata
- FastExport
You may have to restart the Agent after the installation.
A scan of a Teradata data store may create temporary tables named erecon_fexp_<YYYYMMDDHHMMSS><PID><RANDOM>. Do not remove these tables while the scan is in progress. They are automatically removed when a scan completes. If a scan fails or is interrupted by an error, the temporary tables may remain in the database. In this case, it is safe to delete the temporary tables.

Use the Add Data Store wizard to add a big data type data store. Adding a Big Data data store involves the following steps:

1. Select Store Type

In the Select Store Type screen of the wizard select Big Data in the Select Data Store Category.
From the Select Database Type drop-down list select Hadoop Cluster or Teradata.
Click Next to go on to the Configure Connection screen.

2. Configure Connection

Hadoop Cluster

Hostname/IP - Specify Hostname/IP of the Hadoop cluster's active NameNode. Specify a valid hostname, IP address, or Uniform Resource Identifier (URI). The hostname must be longer than two characters. This is a mandatory field.
Port - Default 8020. This is a mandatory field.

Click Next to go to the General Info screen.

Teradata

Hostname - Specify a valid Hostname of the Teradata server. The hostname must be longer than two characters. This is a mandatory field.
Port - Default 1025. This is a mandatory field.
User - The name of the Teradata user.
Due to known Teradata limitations DDC cannot use the following internal Teradata users to scan:
Warning
DBC, tdwm, LockLogShredder, External_AP, TDPUSER, SysAdmin, SystemFe, TDMaps, Crashdumps, Sys_Calendar, viewpoint, console.
Password - The password of the Teradata user.

Click Next to go to the General Info screen.

3. General Info

Configure the General Info part per the information in General Info.
Click Next to go to the Add Tags & Access Control screen.

4. Add Tags & Access Control

Configure the Tags & Access Control par per the information in Tags & Access Control.
Click Save. The newly created data store appears on the Data Stores page. By default, data stores are displayed in alphabetic order by name. Depending on the number of entries per page, you might need to navigate to other pages to view the newly created data store.

Cloud Data Stores

DDC supports these types of Cloud storages as data stores:

AWS S3 - AWS (Amazon Web Services).
Azure Blobs - Microsoft Azure Blobs (used to store unstructured text and binary data).
Azure Table - lets programs store structured text in partitioned collections of entities that are accessed by partition key and primary key.
Office 365 Sharepoint Online - Sharepoint Online is a document management and storage system delivered as part of Microsoft Online Services suite.
Office 365 Exchange Online - Exchange Online is Exchange Server delivered as a cloud service hosted by Microsoft.
Note
Before adding any Cloud data store, make sure that you have the required user credentials handy.

Use the Add Data Store wizard to add a big data type data store. Adding a Big Data data store involves the following steps:

1. Select Store Type

In the Select Store Type screen of the wizard select Cloud in the Select Data Store Category.
From the Select Database Type drop-down list select:
- AWS S3
- Azure Blobs
- Azure Table
- Office 365: Sharepoint Online
- Office 365: Exchange Online
Click Next to go on to the Configure Connection screen.