Distributed HxMap deployment

Once the HTCondor installation is complete for all machines of the processing cluster, the user may proceed to install and configure HxMap for use in distributed processing.

A shared software installation is utilized to simplify the deployment task.

Install the License Server

One machine will serve as central license server for all Submitters and Nodes in the cluster. As discussed in P1 of the HTCondor Installation this could be the Cluster Manager, but it is not required. The machine chosen as the license server should however satisfy all requirements described for the Cluster Manager.

1. Do a full CLM (clm_package_xxx.exe) installation on the license server. The package will install the CLM for Floating Licenses application.

2. Launch the Client License Manager application. Activate your EID’s and verify them by viewing the installed licenses.

Create the Shared HxMap Installation

To create the shared HxMap installation, perform the following steps while logged into the Cluster Manager.

• On a Server OS, turn off UAC from Start > Settings > Control Panel > User Account

• In order to get rid of a "Open File Security Warning" during software installation from trusted sources like your local host, go to the IE -> Internet Options->Security->TrustedSites->Sites and add your local machine. However this differs depending on the Windows OS (Windows 7, Server2008) where and if you have to add the qualified name of the domain, e.g. file://*.hostname.domain.

  An alternative way to add local shares is to add the machine to the "Local Internet“ at the Security tab. For further help please refer to:http://support.tabs3.com/main/R11124.htm

• Map Network drives as the processing nodes will see it (e.g. let us assume SW_DRIVE is "S:\" and DATA_DRIVE is "Y:")

• Do a full HxMap installation (with GeoData and Prerequisites) into the software file share created in P8 above (e.g, to "S:\Hexagon\HxMap\Workflow Manager")

• Edit the “HxMap.ini” file and specify the shared path location of the following configuration items:

  • Global calibration repository:

    CODE

    [Calibrations]
    ; default location for calibrations
    location=c:/Users/public/cam

  • Geodata coordinate systems:

    CODE

    [Mapping]
    ...
    location=$etc_folder/geodetic

  • Geodata Global DEM:

    CODE

    [GTopo]
    ...
    datafile=$etc_folder/dem/gtopoDem_ell.jptf

• Edit the "HxMap_condor_runner_WINDOWS_X86_64.bat" batch file using the information from P6:

  • The “HxMap_condor_runner_WINDOW_X86_64.bat” is the ‘job runner’ passed by HTCondor to the cluster node for every processing job. It is used to prepare the environment for the HxMap software to run under the HTCondor service.

  • “Mapping Share Drives” section. Populate USER, PASSWORD, SW_DRIVE, and DATA_DRIVE. More than one DATA_DRIVE can be specified if needed. Be sure to add an additional entry “call :MapNetowrkDrive” for each DATA_DRIVE which is added.

    CODE

    echo Map share drives:
    REM Watch out for the order of the arguments: [Drive] [NetworkSharePath] [Username] [Password]
    REM Hint: The arguments for user name and password (as a unit) are optional.
    REM set USER=user
    REM set PASSWORD=pwd
    REM set SW_DRIVE=W:
    REM set DATA_DRIVE=Y:
    call :MapNetworkDrive %SW_DRIVE% \\sw_host\sw_share %USER% %PASSWORD%
    call :MapNetworkDrive %DATA_DRIVE% \\datahost\data_share %USER% %PASSWORD%

  • “Network Path Section”. In case your Shared Installation path is not as shown above, change this entry accordingly.

    CODE

    REM Full path to a network location containing workflow manager installation and settings.
    set "NETWORK_PATH=%SW_DRIVE%\Hexagon\HxMap\Workflow Manager"

Submitter preparation

Submitters will run HxMap Workflow Manager from the Shared Installation prepared above. The following steps are required to prepare the Submitter.

• Map Network drives as the processing nodes will see it (For example, let us assume SW_DRIVE is "S:\" and DATA_DRIVE is "Y:")

• Make sure the required HxMap prerequisites are installed on the system, otherwise add them (Microsoft runtime, which can be found under the "prereq\" folder of the install media)

• Do a full CLM (clm_package_xxx.exe) installation. The package will install the CLM for Floating Licenses

  • Define the License Server and verify the submitter is acquiring the licenses.

• Install the HxMap DMCSupport application, if you intend to use the submitter also as processing node or wish to ingest DMCIII data locally.

• Create shortcuts on the desktop to "Workflow Manager.exe" available in the Shared Folder (e.g, "S:\Hexagon\HxMap\Workflow Manager")

Start HxMap WFM from the desktop shortcut to confirm it works from the shared installation.

Its recommended for a new submitter to start a small ingest/ rectification job on the LOCAL machine to confirm everything is working and the data can be mapped correctly, prior to submitting a large cluster processing.

Processing Nodes preparation

The procedure for the installation on the processing nodes is quite identical to the procedure described for the submitter.

It is not required to Map Network drives directly on the Nodes, as the “Map share drives” of the "HxMap_condor_runner_WINDOWS_X86_64.bat" will perform this step for each processing job.

• Make sure the required HxMap prerequisites are installed on the system, otherwise add them (Microsoft runtime, which can be found under the "prereq\" folder of the install media)

• Do a full CLM (clm_package_xxx.exe) installation. The package will install the CLM for Floating Licenses

  • Define the License Server and verify the processing node is acquiring the licenses.

• Install the HxMap DMC Support application, if you intend to ingest DMCIII data

Full HTCondor Test

• Login to the submitter machine.

• Start HxMap and launch a local job (e.g. product generation) first to verify overall functionality. If it doesn’t execute successfully, use the local log files to find out why. It's most likely a drive mapping or HxMap.ini problem.

• In a 2^nd run, submit the same job to run on the cluster.

• Run "condor_q" from the command line on the submitter. You should see a list of jobs either in "queued" or "idle" state.

• Within a few minutes, the job status should change to “running”. Run time should count up and status should move to done resp. completed once it’s finalized.

• Submit more jobs and the other nodes should pick these up and run.

• If jobs run and stop very quickly (status “Held”), it indicates an issue with drive mapping, file permission, or file not found.

• To find out, why a job has gone in “Held” status

  • Run "condor_q -analyze -long" to see why jobs are not running or stopping

  • Look into the HxMap job log files (*.log, *.err, *.out) to identify the root cause of the problem

If the reason for failure cannot be identified, please zip up the .log, .err, and .out files from the HxMap_Temp folder on the submission machine and sending them to HxMap support.

Deploying more than one HxMap version

Rarely you may wish to operate more than one HxMap version at the same time in your cluster environment. Perform the following steps if this is needed.

• Log into the Cluster Manager.

  • Run the HxMap Full Installer and (optionally) the Geodata Installer.

  • For the Install location, choose a unique path from the one described above, e.g. "S:\Hexagon\HxMap\Workflow Manager 4.2"

  • Edit the “HxMap.ini” and the "HxMap_condor_runner_WINDOWS_X86_64.bat" file entries described above being sure to reflect the unique installation path chosen.

• Log into the Submitter.

  • Create shortcuts on the desktop to "Workflow Manager.exe" available in the Shared Folder of the unique installation (e.g, "S:\Hexagon\HxMap\Workflow Manager 4.2")

  • Start HxMap WFM from the desktop shortcut to confirm it works from the shared installation.

Each time Workflow Manager runs it utilizes the files and configurations of the folder from which it was launched. When processing is submitted to the Cluster, the "HxMap_condor_runner_WINDOWS_X86_64.bat" and the HTCondor job files will be copied to a unique sub-directory of the directory specified in the “HxMap.ini” file’s “condor_temp_dir” variable. In this way multiple Cluster jobs can be submitted from different HxMap versions on the same Submitter.