Your suggested change has been received. Thank you.

close

Suggest A Change

https://thales.na.market.dpondemand.io/docs/dpod/services/kmo….

Thales Logo

All

  • All
back

Monitoring Data Transformation

Using dataxform_status Files

search

Please Note:

Using dataxform_status Files

Several dataxform_status_* files are used to run dataxform. One file you generate. The other files are generated by dataxform to track the dataxform session. Each dataxform_status file contains specialized records of the last or current dataxform session. The files are placed in /var/log/vormetric by default, but you can specify an alternate location using the --dir_recovery argument to dataxform.

The files are:

dataxform_status-_gp
dataxform_status_alt-_gp
dataxform_status_error-_gp
dataxform_status_skip-_gp

where, gp is the full path to the GuardPoint. The slash (/) and backslash () path delimiters are replaced with underscores (_).

# pwd
/var/log/vormetric
# ls dataxform_status*
dataxform_status_alt-_opt_apps_lib_dx1
dataxform_status_error-_opt_apps_lib_dx1
dataxform_status_skip-_opt_apps_lib_dx1
dataxform_status-_opt_apps_lib_dx1
dataxform_status-_opt_apps_lib_dx2_aa_dir

where /opt/apps/lib/dx1 and /opt/apps/lib/dx2/aa_dir are the actual GuardPoint paths.

The format and use of each file are described below.

dataxform_status-_gp and dataxform_status_alt-_gp are used together to monitor data file processing. These status files indicate the threads that are started, the threads that are running, their status, and sequence.

For a dataxform failure or interruption, the dataxform_status-_gp and dataxform_status_alt-_gp files can be used to guesstimate the files that have been completed and the files that still must be transformed. Some manual verification will be needed to verify precisely the files that have and have not been transformed. You can use this information to create a file list and manually resume the dataxform session, or, you can just leave these status files in place and resume automatic dataxform. Automatic dataxform will determine where to resume.

The structure of the dataxform_status-_gp and dataxform_status_alt-_gp files is:

version=n
status=stat
operation=action
current=file
n in-progress files
seqno=n
hmac=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

where:

  • version=n is an internally used version number and can be ignored.

  • status is the current dataxform processing status. Status can be done, in-progress, stopped, or undone. An in-progress status indicates that dataxform is still processing data files. Do not work in, or access, the GuardPoint while dataxform is still in-progress. Otherwise, you risk corrupting data. A stopped status indicates that the last dataxform session was interrupted before it could complete. The done status means that dataxform completed. Note that, though dataxform completes, not all files are necessarily transformed. Check the log files for data files that have not been transformed. undone indicates that rekey from a file list had been performed.

  • operation=action is the operation specified on the dataxform command line. It can be either rekey or rekey_list. Automatic dataxform is implicitly configured to operate with rekey.

  • current=file is the full path name of the file currently being processed. When status is done, the current parameter is blank. In every other case, current will be set to some value, such as the default unset value -1.

  • n in-progress files is the total number of files being concurrently processed by dataxform. Each file is transformed as a separate sub-process, or thread. n is the value of the --thd paramerter passed to the dataxform command, and it can be between 1 and 32. The default number of threads is 8 or the number of CPUs, whichever is less. For specifics on the -thd parameter option, see dataxform Examples and Full Command Syntax.

  • seqno=n is the sequence number. There are two status files, dataxform_status-_gp and dataxform_status_alt-_gp. The dataxform utility writes status information to one file. When a sub-process completes and a new data file is opened for transformation, the current status file is closed, and the other status file is opened with the new data file as the current=file file. The seqno number increments each time dataxform begins to process the next data file. If a dataxform session fails or is interrupted, check the seqno number. Use the status file with the higher seqno number to determine the files that were being processed when dataxform stopped. The status file with the lower seqno number indicates the last fully processed data file in the current=file parameter. The status file with the higher seqno number indicates the last opened data file in the current=file parameter. Most likely the last opened data file was not successfully transformed.

  • hmac=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is a check value used to ensure the integrity of the status files, and it can be ignored.

By default, the log files are generated by dataxform during manual and automatic data transformation and placed in /var/log/vormetric on Linux systems. An alternate location can be specified using the --dir_recovery argument to dataxform.

A successful dataxform status file looks like:

# cat ./dataxform_status-_opt_apps_lib_dx2_aa_dir
version=5
status=done
operation=rekey
current=0 in-progress files
seqno=2004
hmac=6BI53B320C455A45E013AC21F353CE0BBCF159440B2B32F3F3
000FBB0BA7B3C5267D32D60385C786FEDA8C59D57F1C44
5588A9FCAB56CC642E8B2E5601D09CA7E9F6AFFA886ADR6
2A8FE559CF
C89B9F55G
3E65F4
I8FD574

Note that the status is done, there is no current file, and there are 0 files being processed. The seqno number is the number of files in the GuardPoint. There is no corresponding dataxform_status_alt-_gp file because that file is deleted when dataxform completes successfully. Also, look for a dataxform_status_skip-_gp file to see what files in the GuardPoint, if any, were detected but not transformed.

The dataxform_status-_gp and dataxform_status_alt-_gp files for a dataxform session are shown below. Some things to note:

  • The number that precedes every file path in the status file is the offset into the dataxform file list. It is used to quickly locate entries if dataxform needs to be restarted. This number of can be ignored.

  • The dataxform utility was running 8 sub-processes, as shown in the 8 in-process files field. The 8 files being processed are shown beneath this entry.

  • The seqno field indicates that dataxform_status_alt-_Guard is the true snapshot of what dataxform was doing while the files were captured or dataxform failed because it has the higher sequence number. The other status file, dataxform_status-_Guard, is older because it has the lower seqno number.

    This means that all of the files shown in dataxform_status_alt-_Guard were being processed when dataxform stopped.

  • The final line with the prefix hmac= is a checksum field. This is used to verify that the file content is intact should the transform be interrupted and need to be restarted.

    All data in the status files after this line are 'noise'. This occurs for efficiency reasons. The dataxform program writes the status files using a simple block write of all the status information into the existing file, overwriting any entries already present. If, as often happens, the existing status file is larger than the data being written, then some of the stale entries may be visible after the final checksum line. The additional effort of removing this stale data for each file transformation would add a significant additional amount of time to the overall transformation, so this 'noise' is simply left and can be ignored.

    # cat dataxform_status-_Guard
version=5
status=in-progress
operation=rekey
current=48986 /Guard/dir_47/boot/grub/stage2
8 in-progress files
48881 /Guard/dir_47/boot/grub/reiserfs_stage1_5
48936 /Guard/dir_47/boot/grub/fat_stage1_5
48333 /Guard/dir_47/boot/initramfs-2.6.32-71.el6.x86_64.img
48758 /Guard/dir_47/boot/vmlinuz-2.6.32-71.el6.x86_64
46899 /Guard/dir_5/boot/initramfs-2.6.32-71.el6.x86_64.img
48821 /Guard/dir_47/boot/config-2.6.32-71.el6.x86_64
48986 /Guard/dir_47/boot/grub/stage2
47316 /Guard/dir_5/boot/vmlinuz-2.6.32-71.el6.x86_64
seqno=738
hmac=2ED53B320C455A45E013AC21F353CE0BBCF159440B2B32F3F3
000FBB0AD9B3C5267D32D60385C786FEDA8C59D57F1C52
7788A9FCAB56CC642E8B2E5601D09CA7E9F6AFFA886ADE1
2A8FE559FC
C89B9F55E
3E65F5
D8FD544

# cat dataxform_status_alt-_Guard
version=5
status=in-progress
operation=rekey
current=49031 /Guard/dir_47/boot/grub/ufs2_stage1_5
8 in-progress files
49031 /Guard/dir_47/boot/grub/ufs2_stage1_5
48936 /Guard/dir_47/boot/grub/fat_stage1_5
48333 /Guard/dir_47/boot/initramfs-2.6.32-71.el6.x86_64.img
48758 /Guard/dir_47/boot/vmlinuz-2.6.32-71.el6.x86_64
46899 /Guard/dir_5/boot/initramfs-2.6.32-71.el6.x86_64.img
48821 /Guard/dir_47/boot/config-2.6.32-71.el6.x86_64
48986 /Guard/dir_47/boot/grub/stage2
47316 /Guard/dir_5/boot/vmlinuz-2.6.32-71.el6.x86_64
seqno=739
hmac=E113351BDB7497AAD150DEC57953DB0E57401404F564733E71
02A612158F65D7ACC2E1D9A75CB94ED91751397CDFD1CF
EE3483B58DFD6DD4DC722D
0CD47F89890DFA572DE7F3E4E9F
4
0A2
E78F4
F8
7B
DC8D603

On this page