Appendix

Scan Filters

This section provides you with information on scan filters in various data stores, syntax details and examples of their usage. ("N/S" stands for "not supported". Numbers used in the parameters section are provided as examples.)

Files

LocalStorage - Linux

Exclude location by prefix

• root directory: / (Supported but not recommended)

• 1 folder:
  /folder1 - Exclude all the folders starting by /folder1 like /folder1, /folder12, /folder123 and their content
  /folder1/ - Exclude all the folders starting by the name /folder1/ and their content like /folder1/file.txt, or /folder1/myfile.txt.

• more folders:
  /folder1/folder2/ - Exclude all the folders starting by /folder1 /folder2/ like /folder1/folder2/file3.txt, or /folder1/folder2/folder3/myfile.txt

• files:
  folder1/file.txt - Exclude all the files with name file and extension txt and located in a folder with name folder1

Exclude location by sufix

• root directory: N/S

• 1 folder:
  myfolder - Exclude all the directories with that name and their content. It is equivalent to */myfolder
  myfolder/*.txt - Exclude all the txt files in the folders with name myfolder
  

• more folders:
  /folder1/folder2/ - Exclude the folder with name folder2 located in parent folder named folder1

• files:
  myfolder/*.pdf - Exclude files with extension pdf: myfolder/*.pdf
  myfolder/otherDir/myfile.txt - Exclude a specific file: myfolder/otherDir/myfile.txt
  myfolder/subfolder/*.txt - Exclude all txt files located in that folder and subfolder. Subfolders can be used to improve the filtering.

Exclude locations by expression

• root directory: / (Supported but not recommended)

• 1 folder:
  */folder1/* - Exclude all folders named folder1 and their content

• more folders:
  /folder1/folder - Exclude all folders in folder 1 whose name is folder (?) like /folder1/folder (1), /folder1/folder (2) or /folder1/folder (a))

• files:
  *file.txt - Exclude the elements matching with the expression like /myfolder/file.txt, /myfolder/myfile.txt, /myfolder/otherfolder/file.txt, /myfolder/otherfolder/mysensitivefile.txt
  */myfile.txt - Exclude the elements matching with the expression like /myfolder/myfile.txt or /myfolder/otherfolder/myfile.txt (but NOT /myfolder/file.txt or /myfolder/otherfolder/mysensitivefile.txt)

Include locations within modification date

parameters:

• toDate: "2021-05-30"
• fromDate: "2021-08-01"

Include files whose modification date is between that dates

Include locations modified recently

parameters:

• days: 5 - Include files in the 5 previous days modified, even though there is a different filter excluding them

Exclude locations greater than file size

parameters:

• size: 20 (size in MB) - Exclude the files whose size in MB is 20 or greater than 20 MB

LocalStorage - Windows

Exclude location by prefix

• root directory: C:\ - Exclude the volume C:\

• 1 folder:
  *\my_folder - Exclude all the folders with name my_folder and their content

• more folders:
  *\my_folder\my_subfolder - Exclude all subfolders with name my_subfolder located in a folder with name my_folder and their content

• files:
  C:\mydir\ - Exclude all files located in C:\mydir\ that begins with name file like file1.txt, file2.pdf*, etc.

Exclude location by sufix

• root directory: C:\ (Supported but not recommended)

• 1 folder:
  my_folder - Exclude all the folders with name my_folder and their content

• more folders:
  my_folder\my_subfolder - Exclude all the folders with name my_subfolder located in a folder with name my_folder

• files:
  .txt - Exclude all files with extension txt

Exclude locations by expression

• root directory: C:\ - Exclude a whole volume C:\*

• 1 folder:
  *\my_folder - Exclude all folders named my_folder an their content

• more folders:
  my_folder\my_subfolder - Exclude all the folders with name my_subfolder located in a folder with name my_folder

• files:
  *.txt - Exclude all files with extension .txt

Include locations within modification date

parameters:

• toDate: "2021-05-30"
• fromDate: "2021-08-01"

Include files whose modification date is between that dates

Include locations modified recently

parameters:

• days: 5 - Include files in the 5 previous days modified, even though there is a different filter excluding them

Exclude locations greater than file size

parameters:

• size: 20 (size in MB) - Exclude the files whose size in MB is 20 or greater than 20 MB

SMB

Exclude location by prefix

• root directory: N/S

• 1 folder:
  <sharename>\\sambafolder or *\sambafolder

• more folders:
  *folder\subfolder

• files:
  *\sambafolder\file.txt

Exclude location by sufix

• root directory: N/S

• 1 folder:
  myfolder - Exclude folder called :
  \\samba
  \\exclude_suffix

• more folders:
  myfolder\mysubfolder

• files:
  myfolder\mysubfolder\file.txt

Exclude locations by expression

• root directory: N/S

• 1 folder:
  *my?folder - Exclude folder called \samba\my_folder, also works for my-folder, etc

• more folders:
  *my_folder\my_subfolder

• files:
  *my_folder\my_subfolder

Include locations within modification date

parameters:

• toDate: "2021-05-30"
• fromDate: "2021-08-01"

Include locations modified recently

parameters:

• days: 5

Exclude locations greater than file size

parameters:

• size: 20 (size in MB)

NFS

Exclude location by prefix

• root directory: N/S

• 1 folder: /mnt/nfs/myfolder or *myfolder - Exclude folder with name myfolder and all the subfolder contained
  

  Note

  The mount point for NFS file system in this example is:/mnt/nfs

• more folders: /mnt/nfs/myfolder/mysubfolder or *myfolder/mysubfolder

• files:
  /myfolder/myfile.txt - Exclude txt file with that name located in that folder

Exclude location by sufix

• root directory: N/S

• 1 folder:
  myfolder - Exclude folder named myfolder

• more folders:
  myfolder/mysubfolder - Exclude folder myfolder/mysubfolder

• files:
  myfolder/myfile.xlsx - Exclude that file in myfolder with that extension

Exclude locations by expression

• root directory: *

• 1 folder:
  *my?folder - Exclude folder called my_folder, my-folder, etc

• more folders:
  myfolder/mysubfolder - Exclude folder myfolder/mysubfolder

• files:
  myfolder/myfile.xlsx - Exclude that file in myfolder with that extension

Include locations within modification date

parameters:

• toDate: "2021-05-30"
• fromDate: "2021-08-01"

Include locations modified recently

parameters:

• days: 5

Exclude locations greater than file size

parameters:

• size: 20 (size in MB)

Hadoop

Exclude location by prefix

• root directory: N/S

• 1 folder:
  */my_data - Exclude all the directories with name my_data and their content

• more folders:
  */my_data/subdir - Exclude all directories with name subdir wich parent directoriy is named my_data

• files:
  /my_data/file.ppt - Exclude file with that name and extension located in the directory my_data

Exclude location by sufix

• root directory: N/S

• 1 folder:
  mydir - Exclude all directories with name mydir, my_directory, etc, and their content
  mydir/ - Exclude all directories with name mydir and their content

• more folders:
  mydir/subdir - Exclude all directories with name subdir, subdirectory, etc, and their content which are located in a directory with name mydir
  mydir/subdir/ - Exclude directory with name subdir and their content which is located in a directory with name mydir

• files:
  *.pptx - Exclude all files with extension pptx

Exclude locations by expression

• root directory: N/S

• 1 folder:
  */mydir - Exclude all directories with name mydir and their content

• more folders:
  */mydir/subdir - Exclude directory with name subdir and their content which is located in a directory with name mydir

• files:
  */mydir/My_file.pptx - Exclude the file with that name and extension located in a directory with name mydir

Include locations within modification date

parameters:

• toDate: "2021-05-30"
• fromDate: "2021-08-01"

Include files whose modification date is between that dates

Include locations modified recently

parameters:

• days: 5 - Include files in the 5 previous days modified, even though there is a different filter excluding them

Exclude locations greater than file size

parameters:

• size: 20 (size in MB) - Exclude the files whose size in MB is 20 or greater than 20 MB

AWS S3

Exclude location by prefix

• root directory: Bucket (Sample: asrm-ddcqa*)

• 1 folder: myfolder

• more folders: myfolder/mysubfolder

• files: -

Exclude location by sufix

• root directory: Bucket

• 1 folder: Folder

• more folders: -

• files: -

Exclude locations by expression

• root directory: Bucket

• 1 folder: Folder

• more folders: -

• files: -

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

Databases

Oracle

Exclude location by prefix

• Schema: N/S

• Table: HR(SERVICE_NAME=XE):1521/my (You have to include the database, SID and port in the format <database>(SERVICE_NAME=<sid>):<port>/<table-prefix>)

• Column: N/S

Exclude location by sufix

• Schema: N/S

• Table: user (Just add the last characters of the name of the table. * is not needed.)

• Column: N/S

Exclude locations by expression

• Schema: N/S

• Table: HR(SERVICE_NAME=XE):1521/sensitive_expression_data (Add the table at the end of the string: <database>(SERVICE_NAME=<sid>):<port>/<my-table>)

• Column: N/S

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

IBM DB2

Exclude location by prefix

• Database: testdb:50000 (Supported but not meaningful to use)

• Schema: testdb:50000/DB2IN - Exclude schemas that beggin with DB2IN located in that database

• Table: testdb:50000/DB2INST1/SENSITIVE_DATA - Exclude tables that begin with SENSITIVE_DATA located in that database/schema

• Column: testdb:50000/DB2INST1/SENSITIVE_DATA/first_ - Exclude columns that begin with first_ located in that database/schema/table

Exclude location by sufix

• Database: testdb:50000 (Supported but not meaningful to use)

• Schema: _MYSCHEMA - Exclude a schema. * not needed.

• Table: MYTABLE - Exclude a table. * not needed.

• Column: _name - Exclude a column. * not needed.

Exclude locations by expression

• Database: testdb:50000* (Supported but not meaningful to use)

• Schema: testdb:50000/OTHER_SCHEMA/*" - Exclude a schema and its content

• Table: testdb:50000/DB2INST1/*DATA_EX*" - Exclude all tables that contain DATA_EX in their names

• Column: testdb:50000/DB2INST1/DATA_EXPRESSION/*first_name*" Exclude all columns that contain firs_name in its name

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

Microsoft SQL

The asterisk (*) means that the filter matches anything. There is no limitation as to adding the asterisk before a schema or not. For example, the filter can be effective with these two expressions:
️ testddc:1433/testschema
️ */testschema

Exclude location by prefix

• Database: testddc:1433

  Note

  Supported but not useful (the database name is specified when the datastore is instanciated so no meaning then to filter out the whole db).

• Schema: testddc:1433/testschema

  Note

  The schema is aways relative to what is called the catalog (ie the database name and the port).
  The same filter can be written also as */testschema but using testschema only is not effective as a prefix.

• Table: testddc:1433/testschema/mytable

  Note

  The entire complete path should be specified or optionally omitted using a *. A filter like */mytable has the side effect of being all the tables whose name is mytable being filtered - no matter which schema they belong to.

• Column: N/S

Exclude location by sufix

• Database: testddc:1433

  Note

  Supported but not useful (the database name is specified when the datastore is instantiated so no meaning then to filter out the whole db).

• Schema: testddc:1433/testschema (As it is a suffix, testschema also works with the same effect)

• Table: testddc:1433/testschema/mytable

  Note

  If the table name is unique in all schemas mytable is enough to filter.
  If different schemas contain the same tablename then caution should be used.

• Column: N/S

Exclude locations by expression

• Database: testddc:1433

  Note

  Supported but not useful (the database name is specified when the datastore is instantiated so there is no point filtering out the whole db).

• Schema: testddc:1433/testschema

  Note

  Tthe schema is aways relative to what is called the catalog (i.e. the database name and the port). The same filter can be written also as /testschema but testschema* however it is not effective as a prefix.

• Table: testddc:1433/testschema/mytable

  Note

  The entire complete path should be specified or optionally omitted using a *. A filter like */mytable has the side effect of having all the tables whose name is mytable filtered, no matter which schema they belong to.

• Column: N/S

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

Azure tables

Exclude location by prefix

• Database: N/S
• Schema: N/S
• Table: table (Trailing '*' is mandatory to match row_count)
• Column: N/S

Exclude location by sufix

• Database: N/S
• Schema: N/S
• Table: table* (Trailing '*' is mandatory to match row_count)
• Column: N/S

Exclude locations by expression

• Database: N/S
• Schema: N/S
• Table: table* (Trailing '*' is mandatory to match row_count)
• Column: N/S

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

MySQL

Exclude location by prefix

• Database: database

  Note

  It is not mandatory to include * at the end of the database.
  It will ignore the database called database.

• Schema: N/S

  Note

  MySQL only contains the database and tables.
  In some cases, for MySQL, database are also refered as schema.
  MySQL scan path is database/table.

• Table: database/table or *table

  Note

  It is mandatory to include * at the beginning of the table name.
  Replace the trailing * with `database/` if you only want to exclude a table from a specific database.

• Column: *testddc/sensitive_data/email (It's mandatory to include * at the beginning.)

Exclude location by sufix

• Database: database*

  Note

  It is mandatory to include * at the end of the database.
  You can replace * with `/table` to only ignore a specific table from that database.

• Schema: N/S

• Table: database/table or table

  Note

  It will ignore all tables ending with the word table.
  Include the full path if you wish to ignore only a table in a specific database.

• Column: testddc/sensitive_data/email

Exclude locations by expression

• Database: database* (It is mandatory to include * at the end of the database.)

• Schema: N/S

• Table: database/table or *table (It is mandatory to include * at the beginning of the table name.)

• Column: *testddc/sensitive_data/email

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

postgreSQL

Exclude location by prefix

• Database: database (e.g. hr)

• Schema: datbase:port/schema (e.g. hr:5432/prod for excluding the schemas that have the prod prefix or hr:5432/* for excluding all schemas from hr db scan locations)

• Table: datbase:port/schema/table (e.g. hr:5432/hr/prod)

• Column: *datbase:port/schema/table/column* (e.g. *hr:5432/hr/prod/EMAIL*)

  Note

  It is mandatory to include * at the beginning and end.

Exclude location by sufix

• Database: database* (e.g. hr*)

  Note

  It is mandatory to include * at the end of the database.

• Schema: *schema* (e.g. *prod*)

  Note

  It is mandatory to include * at the beginning and end.
  Along with schema it is also excluding table having suffix prod.

• Table: datbase:port/schema/table (e.g. hr:5432/hr/prod)

• Column: datbase:port/schema/table/column (e.g. hr:5432/hr/prod/EMAIL)

Exclude locations by expression

• Database: database* (e.g. hr*)

  Note

  It is mandatory to include * at the end.

• Schema: datbase:port/schema/* (e.g. hr:5432/prod/*)

  Note

  It is mandatory to include * at the end.

• Table: datbase:port/schema/table (e.g. hr:5432/hr/prod)

• Column: *datbase:port/schema/table/column* (e.g. *hr:5432/prod/prod/EMAIL*)

  Note

  It is mandatory to include * at the beginning and end.

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

NoSQL

Azure blobs

Exclude location by prefix

• Account: N/S

• Container: ddctest* or ddctest

• Blob: ddctest/reuters/reut2-012.sgm or *reut2-012.sgm

Exclude location by sufix

• Account: N/S

• Container: *my_data*

  ️ */my_data - N/S.
  ️ *my_data - N/S
  *️ */my_data*/ - N/S.
  

  Note

  ️ *my_data* - Supported, but along with the Container it excludes the Blob also having suffix my_data* and in report filter type is "Exclude location by expression" not by suffix.

  Warning

  Container is allowed for exclude_suffix, but you should be careful when using it.

• Blob:

  1. my_data
  2. my_data*
  3. ddctestblob/reut2*

Exclude locations by expression

• Account: N/S

• Container: ddctestdata* or *test* (It is mandatory to include * at the end of container name.)

• Blob:

  1. *reut2*
  2. *.txt
  3. ddctestblob/reut2*
  4. *dctestblob/*.txt
  5. */reut2* (It is mandatory to give a * at the start if you want to exclude a specific blob like reut2-012.sgm would be *reut2-012.sgm)

Include locations within modification date

parameters:

• toDate: "2021-08-04"
• fromDate: "2021-09-24"

Date format is in "YYYY-MM-DD".
toDate and fromDate are inclusive for files that are being scanned.

Include locations modified recently

parameters:

• date: 2

Exclude locations greater than file size

parameters:

• size: 1 (Size in MB)

MongoDB

Exclude location by prefix

• Database: database (It is not mandatory to include * at the end of the database.)

• Collection: database/colletion* or *collection*

  Note

  It is mandatory to include * at the beginning and end of the schema.
  Leading * matches database while trailing * match the _id.
  _id is primary key in mongoDB. ER2 creates an object for each document/record/ row instead of each table.

• Fields: N/S

Exclude location by sufix

• Database: database* (It is mandatory to include * at the end of the database.)

• Collection: database/colletion* or collection*

  Note

  It is mandatory to include * at the end of the database.
  The trailing * matches _id.

• Fields: N/S

Exclude locations by expression

• Databse: database* (It is mandatory to include * at the end of the database.)

• Collection: database/collection* or *collection* (It is mandatory to include * at the beginning and end of the schema.)

• Fields: N/S

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

SAP Hana

Exclude location by prefix

• Database: database

• Schema:

  1. database:port/schema_name
  2. database:port/*

  Note

  It is mandatory to include port if it is different than the default one.
  We can replace schema_name with * to exclude all schema's.

• Table:

  1. database:port/schema_name/table
  2. database:port/*/table
  3. database:port/*/*

  Note

  You can replace schema_name with * , to exclude the table from all schemas. Also we can replace the table name with *, to exclude all tables.

• Column: *database:port/schema/table/column* (It is mandatory to include * at the beginning and end.)

Exclude location by sufix

• Database: database* (It is mandatory to include * at the end of the database.)

• Schema: -

• Table:

  1. table
  2. database:port/schema/*table

  Note

  It will ignore all tables ending with the word table.
  Include the full path if you wish to ignore only a table in a specific schema and database.

• Column:

  1. column
  2. table/column
  3. database:port/schema/table/column

  Note

  "Pipe" (|) can be used to exclude multiple columns.

Exclude locations by expression

• Database: *database* (It is mandatory to add * at the beginning and ending and then only the filter type in the report is considered as "exclude_expression".)

• Schema: database:port/schema/* (It is mandatory to include * at the end.)

• Table: database:port/schema/table_expression (e.g. HXE:39041/*/*cop?)

• Column:

  1. *column*
  2. *table/column*
  3. *schema/*/column*
  4. *database:port/schema/table/column*

  Note

  To exclude the column location for a specific table, you have to add the table name before the column.
  To exclude from a specific schema you have to add the schema as shown in example 3.
  It is mandatory to include * at the beginning and end.

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

Mail

G-Mail

Exclude location by prefix

• User: datastorecicduser

• Account: - (User and Account seem to be the same thing)

• Folder/Label: datastorecicduser/inbox or *inbox

  Note

  Use lower case for folder names.
  Labels are not supported as label id is required to filter.
  The second option would scan the inbox of every user.
  Question: I am filtering Gmail scan by labels - why am I not getting expected results? Answer: For the default system labels, Gmail creates some folders that do not match the label name. Please look at Gmail documentation to learn the right labels to filter data objects in a Gmail scan.

• File:

  1. datastorecicduser/inbox/mydata filter test data/Mon, 23 Aug 2021 10:54:40 +0530/50-contacts.csv or
  2. datastorecicduser/inbox/mydata filter test data/*/50-contacts.csv or
  3. datastorecicduser/inbox/test filter test data or
  4. datastorecicduser/Label_4800715244570918733/test filter test data/ or
  5. datastorecicduser/*/mydata filter test data/

  Note

  The second example would be recommened to the user to avoid manully checking email's date and time and converitng it to the required format.
  The third option is used if you want to scan a specific email and all its content.
  As mentioned in the previous comment, label filters need label id as mentioned in the fourth example.
  The user can replace label id with '*', as shown in the fifth example, if he is scanning a specific label and wants to filter a specific email or file.
  The fourth and fifth example can be used similarly for exclude_suffix and exclude_expression.

Exclude location by sufix

• User: datastorecicduser* (Trailing * matches remaining path.)

• Account: -

• Folder/Label: datastorecicduser/inbox* or *inbox*

  Note

  Question: I am filtering Gmail scan by labels - why am I not getting the expected results? Answer: For the default system labels, Gmail creates some folders that do not match the label name. Please look at Gmail documentation to learn the right labels to filter data objects in a Gmail scan.

• File:

  • datastorecicduser/inbox/mydata filter test data/Mon, 23 Aug 2021 10:54:40 +0530/50-contacts.csv or
  • datastorecicduser/inbox/mydata filter test data/*/50-contacts.csv or
  • datastorecicduser/inbox/mydata filter test data*

Exclude locations by expression

• User: datastorecicduser*

• Account: -

• Folder/Label: datastorecicduser/inbox* or *inbox*

  Note

  Question: I am filtering Gmail scan by labels - why am I not getting the expected results? Answer: For the default system labels, Gmail creates some folders that do not match the label name. Please look at Gmail documentation to learn the right labels to filter data objects in a Gmail scan.

• File:

  • datastorecicduser/inbox/mydata filter test data/Mon, 23 Aug 2021 10:54:40 +0530/50-contacts.csv or
  • datastorecicduser/inbox/mydata filter test data/*/50-contacts.csv or
  • datastorecicduser/inbox/mydata filter test data*

Include locations within modification date

parameters:

• toDate: "2021-08-04"
• fromDate: "2021-08-24"

Date format is in "YYYY-MM-DD".
toDate and fromDate are inclusive for files that are being scanned.
The files matching include_date_range do not have to match include_recent.

Include locations modified recently

parameters:

• days: 2

Days should be between 1 to 99 (both inclusive).
The files matching include_recent do not have to match include_data_range.

Exclude locations greater than file size

N/A

Others

Teradata

Exclude location by prefix

• Schema: teradata (You can replace * with a schema to exclude all schemas.)
• Table: teradata/sensitive_data or */sensitive_data (You can include * to exclude table from all schemas.)
• Column: *teradata/sensitive_data/EMAIL (It is mandatory to include * at the beginning.)

Exclude location by sufix

• Schema: teradata*
• Table: teradata/sensitive*

• Column:

  1. EMAIL
  2. EMAIL | MAC_ADDR
  3. teradata/EMAIL|MAC_ADDR
  4. teradata/sensitive_data/EMAIL

Exclude locations by expression

• Schema: *teradata* or teradata* or teradata

• Table:

  • teradata/sensitive_data_replica
  • */sensitive_data_replica

• Column:

  • *EMAIL*
  • *teradata/*/EMAIL*

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

Exchange Online

Exclude location by prefix

• Group: All Users

• User/Account:

  1. All Users/sample@sjcpl.onmicrosoft.com or
  2. *sample@sjcpl.onmicrosoft.com

  Note

  The second option would filter out "sample@sjcpl.onmicrosoft.com" user data objects from every group.

• Folder: All Users/sample@sjcpl.onmicrosoft.com/inbox or *inbox

  Note

  Folder name is case-sensitive.
  The second option would filter out inbox data objects of every user and group.

• Attachment:

  1. All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/2021-02-22T06:40:18Z/maildir-a.zip or
  2. All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/*/maildir-a.zip or
  3. maildir-a.zip or
  4. All Users/sample@sjcpl.onmicrosoft.com/folder_name/subject or
  5. All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/2021-02-22T06:40:18Z or
  6. *subject

  The second example would be recommended to the user to avoid manually checking mail's date and time and converting it to required format.
  The third option would filter out data objects with attachment maildir-a.zip.
  The fourth option is used if you want to filter out a specific mail and all its content with a corresponding subject name.
  

  Note

  The fifth and sixth option would filter out data objects with given timestamp and subject name.

Exclude location by sufix

• Group: All Users* (You have to use trailing * to exclude given location.)

• User/Account: All Users/sample@sjcpl.onmicrosoft.com* or *sample@sjcpl.onmicrosoft.com*

• Folder: All Users/sample@sjcpl.onmicrosoft.com/inbox* or *inbox*

• Attachment:

  • All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/2021-02-22T06:40:18Z/maildir-a.zip* or
  • All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/*/maildir-a.zip* or
  • *maildir-a.zip*

Exclude locations by expression

• Group: All Users* (You have to use trailing * to exclude a given location.)

• User/Account: All Users/sample@sjcpl.onmicrosoft.com* or *sample@sjcpl.onmicrosoft.com*

• Folder: All Users/sample@sjcpl.onmicrosoft.com/inbox* or *inbox*

• Attachment:

  • All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/2021-02-22T06:40:18Z/maildir-a.zip* or
  • All Users/sample@sjcpl.onmicrosoft.com/Inbox/Mail a/*/maildir-a.zip* or
  • *maildir-a.zip*

Include locations within modification date

N/S

Include locations modified recently

N/S

Exclude locations greater than file size

N/S

Sharepoint Online

N/S

G-Drive

Exclude location by prefix

• User: datastorecicduser (This is for a user with email address datastorecicduser@ddc-thalescpl.com)

• Folder: datastorecicduser/my drive/some/folder (The path should be in lower case.)

• File: datastorecicduser/my drive/some/folder/file.ext

Exclude location by sufix

• User: datastorecicduser* (Trailing * matches the remaining path.)

• Folder: datastorecicduser/my drive/some/folder* or *folder* (The second example with exclude folder present inside every user's drive.)

• File: datastorecicduser/my drive/some/folder/file.ext* or *file.ext*

  Note

  Trailing * is mandatory even for absolute path.
  The second example with exclude file.ext present inside every user's drive.

Exclude locations by expression

• User: datastorecicduser*

• Folder: datastorecicduser/my drive/some/folder* or *folder*

• File: datastorecicduser/my drive/some/folder/file.ext* or *file.ext*

Include locations within modification date

parameters:

• toDate: "2021-08-18"
• fromDate: "2021-08-19"

Date format is in "YYYY-MM-DD".
toDate and fromDate are inclusive for files that are being scanned.
The files matching include_date_range do not have to match .include_recent.

Include locations modified recently

parameters:

• days: 2

Days should be between 1 to 99 (both inclusive).
The files matching include_recent do not have to match .include_data_range.

Exclude locations greater than file size

parameters:

• size: 2 (Size is in MB)

Information Types

Supported Formats

Files

Office files

WORD

• Legacy: Legacy filename extensions denote binary Microsoft Word formatting that became outdated with the release of Microsoft Office 2007. Although the latest version of Microsoft Word can still open them, they are no longer developed. Legacy filename extensions include:

  • .doc – Legacy Word document; Microsoft Office refers to them as "Microsoft Word 97 – 2003 Document"
  • .dot – Legacy Word templates; officially designated "Microsoft Word 97 – 2003 Template"
  • .wbk – Legacy Word document backup; referred as "Microsoft Word Backup Document"

• OOXML: Office Open XML (OOXML) format was introduced with Microsoft Office 2007 and became the default format of Microsoft Word ever since. Pertaining file extensions include:

  • .docx – Word document
  • .docm – Word macro-enabled document; same as docx, but may contain macros and scripts
  • .dotx – Word template
  • .dotm – Word macro-enabled template; same as dotx, but may contain macros and scripts
  • .docb – Word binary document introduced in Microsoft Office 2007

EXCEL

• Legacy: Legacy filename extensions denote binary Microsoft Excel formats that became outdated with the release of Microsoft Office 2007. Although the latest version of Microsoft Excel can still open them, they are no longer developed. Legacy filename extensions include:

  • .xls – Legacy Excel worksheets; officially designated "Microsoft Excel 97-2003 Worksheet"
  • .xlt – Legacy Excel templates; officially designated "Microsoft Excel 97-2003 Template"
  • .xlm – Legacy Excel macro

• OOXML: Office Open XML (OOXML) format was introduced with Microsoft Office 2007 and became the default format of Microsoft Excel ever since. Excel-related file extensions of this format include:

  • .xlsx – Excel workbook
  • .xlsm – Excel macro-enabled workbook; same as xlsx but may contain macros and scripts
  • .xltx – Excel template
  • .xltm – Excel macro-enabled template; same as xltx but may contain macros and scripts

POWERPOINT

• Legacy:

  • .ppt – Legacy PowerPoint presentation
  • .pot – Legacy PowerPoint template
  • .pps – Legacy PowerPoint slideshow

• OOXML:

  • .pptx – PowerPoint presentation
  • .pptm – PowerPoint macro-enabled presentation
  • .potx – PowerPoint template
  • .potm – PowerPoint macro-enabled template
  • .ppam – PowerPoint add-in
  • .ppsx – PowerPoint slideshow
  • .ppsm – PowerPoint macro-enabled slideshow
  • .sldx – PowerPoint slide
  • .sldm – PowerPoint macro-enabled slide

ACCESS

• Legacy:

  • .ade – Protected Access Data Project (not supported in 2013)
  • .adp - Access Data Project (not supported in 2013)
  • .mdb - Access Database (2003 and earlier)
  • .cdb - Access Database (Pocket Access for Windows CE)
  • .mda - Access Database, used for addins (Access 2, 95, 97), previously used for workgroups (Access 2)
  • .mdt - Access Add-in Data (2003 and earlier)
  • .mdf - Access (SQL Server) detached database (2000)
  • .mde - Protected Access Database, with compiled VBA and macros (2003 and earlier)
  • .ldb - Access lock files (associated with .mdb)

• Available formats since Access 2007:

  • .accdb – The file extension for the new Office Access 2007 file format. This takes the place of the MDB file extension
  • .accde – The file extension for Office Access 2007 files that are in "execute only" mode. ACCDE files have all Visual Basic for Applications (VBA) source code hidden. A user of an ACCDE file can only execute VBA code, but not view or modify it. ACCDE takes the place of the MDE file extension
  • .accdt – The file extension for Access Database Templates
  • .accdr – is a new file extension that enables you to open a database in runtime mode. By simply changing a database's file extension from .accdb to .accdr, you can create a "locked-down" version of your Office Access database. You can change the file extension back to .accdb to restore full functionality

OUTLOOK

• .pst - Outlook
• .ost - Outlook
• .msg - Outlook
• .dbx - Outlook

OTHER

• .pub – a Microsoft Publisher publication
• .xps – a XML-based document format used for printing (on Windows Vista and later) and preserving documents

Databases

• Microsoft SQL
• Oracle
• DB2
• PostgresQL

Big Data

• Hadoop

Configuration Backup

You can back up and restore the DDC configuration by using the Backup/Restore functionality available in CipherTrust Manager. Such a backup will include the following elements:

• Data Stores

• Branch Locations

• Classification Profiles

• Infotypes

• Report definitions

This backup will not include the information about the scan executions.

Creating/Restoring the Configuration Backup

To create or restore a backup of your DDC configuration:

1. Log in to CipherTrust Manager.

2. Click the Admin Settings link on the dashboard.

3. Select Backups from the sidebar on the left. This will display the Backups screen.

4. To create a backup of your DDC configuration, click the Create Backup button.

5. To restore your DDC configuration from a backup, click the Upload Backup button.

For more details refer to these sections of the CipherTrust Manager documentation:

• CipherTrust Manager Backup

• CipherTrust Manager Scheduling Backups

• CipherTrust Manager Scheduler

Configuration Backup Limitations

• The configuration backup references the DDC Active Node. Restoring the backup to a different CipherTrust Manager cluster leaves DDC referencing an invalid node, and therefore without any valid active node.

• The configuration backup contains the definition of the DDC resources (such as the Scan or Data Store definitions). Restoring from a backup that does not contain a certain resource (for example, a Custom Classification Profile) or a resource version after a scan had been completed causes a TDP scan execution data to point to an invalid resource identifier.

  If you generate a report that points to the missing resource you may display incomplete data (such as not being able to display the resource name) and/or fail.

Creating/Restoring Backup of Scan Executions

To back up or restore the your Data Discovery and Classification scan executions data you need to access the DDC data stored in Hadoop. For details, refer to the Thales Data Platform Hadoop Backup section in the Thales Data Platform Administrator Guide.

Mounting an NFS Share

To mount an NFS share on a Proxy agent, run this command as root:

sudo mount <nfs-server-hostname|nfs-server-ipaddress>:</target/directory/share-name>