Using Text Qualifier Characters in Input Files

A text qualifier character can be used to mark the start and end of a column field in a file that is input to the BDT utility. This is useful when the field value itself contains a character that would normally act as a field delimiter. Use the qualifier to indicate to BDT that the data following the delimiter is part of the same field, and not the beginning of the next field.

Any printable ASCII character can be specified as a text qualifier, except for the backslash (\) character. Also note that the qualifier character cannot be the same as the delimiter character.

For example, in the following input file, the comma is the delimiter character, but it is also part of the address field. To let BDT know that the text after the comma is also part of the same field, the value must be enclosed with a text qualifier.

Example of data before adding the text qualifier:

Firstname, Lastname, Address
James,Brown,1745 E. New St., Apt. 22,Chicago
Scott,Smith,400 South Warren Street,Trenton

After adding the text qualifier:

James,Brown,"1745 E. New St., Apt. 22,Chicago"
Scott,Smith,"400 South Warren Street,Trenton"

In most cases, the input file for BDT is created by exporting data from a database. Databases provide the option to specify a character to be used as a qualifier in exported data.

To add text qualifier characters:

1. When exporting data from the database, specify the qualifier using the technique that is supported by the database.

2. In the policy file, add the qualifier parameter to the top of the file, before the column specifications begin. If the qualifier is a double quote character, escape it in the policy file with a backslash.

Example configuration in policy file, including the double quote character (") as a qualifier:

"source":{
        "@type":"CSV_FILE",
        "filepath":"input.csv",
        "delimiter":",",
        "qualifier":"\"",
        "unescapeInput":true,
        "columnCount":"12",
        "hasHeaderRow":true
    }