Data copy options
Do verify: Verification on correctness of copied files when no existing checksum is available. An extra file reading step (e.g. verify read) will be executed to verify the files are correctly written to the selected destination or destinations. Checksum will be calculated from the verify read and will be compared to the checksum calculated from source file read. If the checksums match, it means the files are copied correctly and the copied files are identical to the source files. If the checksums are not matched, it indicates something wrong happened during copying (e.g. bad sectors on destination drive or IO abnormal due to the bad slot or malfunctioning cable) and thus the copied file being verified is NOT the same as its source file. In that case, error message will be given in red color and a copy failure dialog will be displayed when the copying is done.
Do verify is optional but highly recommended when using Data Copy to create an original copy of the mass memory Source. Without this check data corruption caused by abnormal IO may be missed by the user. The corruption will then only be noticed during ingest where processing will fail and the user has to go through tedious trouble-shooting to find and remove affected data. It is therefore worth to accept the extra processing time as it will make all following processing steps more reliable.
Due to extra file reading, the data copy process will be slower when "Do verify" is active. Approximately data copy will be ~1.3 times slower if one destination is selected and ~1.6 times slower if two destinations are selected. Information on verify read (number of files, transferred data amount and rate) is written to detail information windows.
Write checksum: Checksum info is written as a *.sha1 file alongside the copied files in the Destination directory.
Skip copy checksum file: Applicable when the Source dataset has checksum files available. Checksum files in the Source will not be copied to the Destination.
Verify with checksum file: Applicable when the Source dataset has checksum files available. Verification of copied data happens against already existing checksum files. Checksum calculated during file copy read operation will be compared with the content of the checksum from Source. If the checksums are not matched, it indicates that the file has been modified (either from manual edit or file corrupting) or something wrong happened during copy operation. In that case, error message will be given in red color and a copy failure dialog will be displayed when the copying is done.
Check MM set consistency and completeness before copying: If enabled, MM set will be validated for consistent raw data folder structure, Also the number of MMs will be checked to match the disk count stored in the disk setup file to make sure that the MM set is complete. If not fulfilled, a warning is given to the user.
Check available free space on output device: If enabled, a pre-check is done on available disk space. If not sufficient, a warning and log message is given.
Allowable combined total of errors and fatals: error threshold before the copy will be stopped due to abnormal IO. It should be set to the sum of Source retries and Destination retries, so that all retry attempts can occur. When the threshold is reached, the copy process will stop and user will be notified in the Data Copy dialog.
Source retries: Setting is used for read thread to control the number of retries when a read error happens (either errors on open a file or loading a buffer due to various reasons)
0 = no retry
positive number (n) = n retries
-1 = unlimited retries
Destination retries: Setting is used for write thread to control the number of retries when a write error happens (either errors on create a file or dumping a buffer due to various reasons)
0 = no retry
positive number (n) = n retries
-1 = unlimited retries