Application Settings

The application settings for HxMap are persisted in "<InstallationPath>\HxMap.ini". Various settings in the file are grouped in sections. The settings file is divided into multiple sections that are grouped based on the frequency of customization. At a minimum, for WFM to work correctly please ensure that the following settings exist and have reasonable values. Note that newer installations of WFM will maintain customized settings whenever possible.

Part 1: Settings you likely want to customize

Licensing

The server keyword needs to point to a valid license server or "localhost" for locally installed licenses. Note that license server is set by the user at installation time and is overwritten each time the user sets a new license server.

[Licensing]
 ; Specify the license server used for HxMap licenses.
 ; Must use IP address or host name
 ; If all licenses are installed on the client use localhost
 server = localhost

Calibration

HxMap’s global calibration repository is the central location to store calibrations for sensor systems. It is necessary to load a valid calibration for each system into the repository prior to working with its data in HxMap. The repository should configured in a centralized location for Local and Cluster processing.

[Calibrations]
; default location for calibrations
location=T:/ref_data/cal

The repository is organized in subdirectories by sensor system family, and then by sensor system instance, then by calibration version.

• Sensor system family: RCD, DMC, CM (CityMapper), SPL, TM, CNM (ContentMapper), CTM (CountryMapper), CSTM (CoastalMapper)

• Sensor system instance: the sensor system type and serial number

  • Sensor system types are: RCD, DMCIII, DMC4, CM, CM2, SPL, TM, TM2, TM3, CNM, CTM, CSTM

  • Examples: CM2_96601, CSTM_ALB003

• Calibration version: v000 to v999

  • a calibration version contains both the system calibration (.hexsystem) and the system’s sensor calibrations (.hexsensor)

Putting the above together in an example, the complete path CoastalMapper ALB003 calibration version 10 within the repository would be: CSTM\CSTM_ALB003\v010.

Lidar Calibration Tool and Accept Triangulation Results with Create Project Calibration both create new system calibrations in an intermediate location. These intermediate calibrations maybe used in Ingest as Project Calibration to verify results, then added to the global repository as a new version for frequent use.

QC Issue types

Define "IssueType" in the INI to extend at each deployment as required. By default, the INI file will have the common ones like "Sensor Problem", "Cloud", "Turbulence" etc.

[QCViewer]
IssueTypes = Sensor Problem/Cloud/Turbulence/Image Quality/Shadows/Fog/Haze/Smoke/Flooding/Accuracy

DEM Editor

Enable the DEM Editor view in Product Generation perspective.

[ProductGeneration]
dem_editor_enable=true

Part 2: Settings you may want to customize

Product Templates

HxMap separates product templates into system and user types. System product templates store the system definitions for Ingest and Overview generation. User product templates store new product definitions created by the User through the Product Generation template editors. User’s may wish to customize the user_location to a central repository on the network.

[ProductTemplates]
; Location of the product templates used for product generations which are global
system_location=$etc_folder/producttemplates/system
user_location=$etc_folder/producttemplates/user

Global Elevation

The Global Elevation entry points to an elevation source used by many parts of HxMap, e.g. calculation of footprints, virtual mosaics, etc. The default elevation source, which is also provided in the Global Elevation Installer, is a low resolution version (32 arc second) of ALOS World 3D. See this page for more information on ALOS World 3D. The Global Elevation installer also allows to download the original ALOS World 3D high resolution (1 arc-second) data for individual areas based on the United Nations Geoscheme.

[GlobalElevation]
; location and filename for the global HxElevation database file
; default is $etc_folder/GlobalElevation/aw3d30_global_low_res.hexes
datafile=C:/Program Files/Hexagon/GeoData/GlobalElevation/aw3d30_global_low_res.hexes

Background layer

An Open GeoSpatial Consortium(OGC) compliant Web Mapping Service(WMS) or Web Mapping Tiled Service(WMTS) could be used as a background layer instead of the default Open Street Map(OSM). The configuration for secured or unsecured WMS/WMTS server is configured in the [BaseLayer] section of the INI file.

[BaseLayer]
 ; descriptive name to be used as a layer name
 Name = "OSM"
 ; full OSM/WMS/WMTS URL including ports
 URL =  https://tiles.cache.services.hxgncontent.com/OSM/3857/%l/%x/%y.png
 ; web service type (OSM/WMS/WMTS)
 Service = OSM
 ; CRS to use
 CRS = EPSG:3857
 ; comma delimited list of layers to be rendered [optional]
 ;Layers =
 ; data format [optional]
 ; e.g. "image/gif", "image/jpeg", "image/png", "image/tiff"
 ;Format =
 ; username for secured services [optional]
 ;User =
 ; clear text password for the given user [optional]
 ;Password =
 ; style to send to the server [optional]
 ;Style =

Metadata

If you do not plan to generate xml Metadata for your projects you can set the default for the product generation wizard to None.

To do so modify the following parameter to 0:

[ProductGeneration]
MetadataType=0

Cluster Job Report

Enable Cluster Job Report to generate detailed and summary job metrics for each Cluster job processed.

[General]
create_job_report=true

HxMap Bathy

For ingest of CoastalMapper data HxMap needs to know the path to the take processor application in HxMap Bathy. By default this is configured to the same path HxMap Bathy installs to. In case of a shared install for HTCondor cluster, update the path to the full shared drive path of TakeProcessorTheia.exe

[HxMapBathy]
; Path to the executable required for processing Bathy raw data
; TakeProcessor=C:/Program Files/Hexagon/HxMapBathy/TakeProcessorTheia.exe

Part 3: Settings requiring in-depth knowledge

Condor Logs

The command line options captured in RSP files, the output, error  and job monitor logs are written to a temporary directory configured in the HxMap INI. Every time a job is submitted to the HTCondor cluster, a new set of related log files are created for each job and its dependencies. The system will create this directory on the first job submitted if doesn't exist already. 

[General]
;skip_stats_check=false 
; Whether we ignore failures creating/updating metainfo files. By defualt we will
;ignore_meta_info_failure=true 

; Whether to use compression for the minifications or not. False by default
;compress_minifications=false
; compression algorithm to use (if compress_minifications is true). Either LZW or DEFLATE (default)
;compression_algorithm=DEFLATE 

; Where to put the HTCondor temp files (.dag, .sub, .rtk, .log, .out, .err, etc)
condor_temp_dir=C:/HXMAP_Temp

2015-03-23 11:37:30,964 [0x0000209c] FATAL workflowmanager - Failed to run due to error:

2015-03-23 11:37:30,964 [0x0000209c] FATAL workflowmanager - condor_submit_dag did not start correctly: 2

Ingest

There are a few options available to modify the default Ingest behavior.

For MFC and RCD ingest, its possible to split image ingest into multiple jobs per strip to achieve better job parallelization for cluster utilization. As each splitting increases overhead of job startup by a small amount, its recommended to not reduce below 50 image per job. The parameter is ignored for DMCIII ingest due to the function of RPP+ during ingest. One take corresponds to one event from flight planning; images from all cameras triggered at that event are processed together

; Number of takes used to break up ingest image processing step for a strip. -1 means no break up and process the whole strip as a job.
; NumberOfTakes=-1

In case the user intends to use the data processed by the Ingest step directly in a 3^rd party application, as it is possible for RCD30 based systems, there exists an option to create an ASCII file with the exterior orientation information directly during Ingest. The exterior orientation information will be delivered in UTM coordinates referenced to WGS84.

; flag to indicate export of EOP files at ingest for 3rd party applications
; export_eop = false

HxMap requires minifications to be available with all ingested imagery when running any part of the workflow (Triangulation, Blockwide Radiometry, Product Generation, etc). By default the minifications are embedded into the 1 to 1 images created during ingest. Users can optionally configure ingest to support storing the minifications externally, which can be useful to reduce disk storage required for archival of processed data e.g. to remove external minifications and only archive 1 to 1 images at the end of a project.

; flag to indicate external minifications for MFC150 imagery ingest if true, embedded minifications by default
;external_minifications = false

Product generation

In order to optimal use available hardware resources in a distributes processing environment, it may be recommended to specify the amount of data handled by an individual processing node within a job. Per default a job during product generation includes a complete strip, but could be broken down as well to a specific number of images.

; Number of Images per image generation job
; NumberOfImages=-1

By default Image Index View loads only 100 images to have good performance on a range of workstations. The limit can be adjusted up or down to fine tune for individual hardware.

;number of images that are loaded into the image index viewer
;ImageIndexViewerImageLimit=100

Automatic Tie Point Matching

HxMap's APM can run based on two matching engines, which are per default dependent on the system type.

[APM]
; Choose between AREA_BASED and FEATURE_BASED for any of the sensors types listed below
;RCD=AREA_BASED
;RCD_OBLIQUE=FEATURE_BASED
;DMC=AREA_BASED
;MFC=AREA_BASED
;ContentMapper=FEATURE_BASED
;CountryMapper=FEATURE_BASED
;CityMapper_2=FEATURE_BASED
;TerrainMapper_2=AREA_BASED

In cases where the Area based APM engine is used, there exists several settings to fine tune the approach. HxMap's Area based APM assumes that exterior orientation parameter exist in a quality resulting from GNSS\INS post-processing and camera misalignment parameter are properly given. The default search algorithms starts on a medium minification level and have a limited search range for fast processing. For some deployments the Misalignment may have to be updated from time to time due to various reasons, an APM run in order to determine a Misalignment would need modified settings in order to produce a reasonable number of good quality points. The following table shows items to fine tune in priority order:

[APM]
...
; AREA based engine options
; inline_init_master_search_size = 400
; interest_window_size = 11
; interest_circle_cutoff = 0.75
; inline_rms_cutoff_per_pixel = 1.5
; inline_expected_parallax = 4
; inline_parallax_cutoff = 6
; transfer_template_size = 15
; transfer_expected_parallax = 15
; transfer_parallax_cutoff = 20
transfer_template_size = 21
transfer_expected_parallax = 60
transfer_parallax_cutoff = 80

MultiframeRadiometry

This section includes settings and thresholds used to calculate atmospheric corrections during the Ingest step.

A user can activate or deactivate this process step:

multiframe_processing = true

Multiframe radiometry statistics and tie point collection are split during Ingest and Blockwide Radiometry to improve parallelization during Cluster processing. Default splitting values can be adapted to tune job size for cluster resources. Statistics collection is split by number of images and tie point collection by number of jobs.

max_number_of_images_per_job = 10000
number_of_tie_points_collections_jobs = 10

PointCloudWriter

This section allows the user to specify the temporary directory which is used to store intermediate files when processing Lidar data. 

[PointCloudWriter]
; The TemporaryDir should have enough space available to store roughly the same size as the data being exported. 
; By default, it is a unique directory on C drive on Windows platform.
;TemporaryDir=

Show Perspectives

This section allows one to enable and disable individual perspectives to customize the deployment of the HxMap Workflow Manager to different users within the organization. Future versions of the HxMap WFM will add other perspectives as they are released to the general public.

[ShowPerspectives]
Session = true
QCViewer = true
Triangulation = true
ProductGeneration = true
Mosaic = true
Editor3D = true
Demo= true

Split LiDAR Take Ingest

Lidar takes may be split based on a fixed time interval to parallelize the ingest processing step.

The default time intervals are:

• Hyperion (all versions): 500 seconds

• Single photo lidar (all versions): 200 seconds

The default values can be adjusted by adding the following HxMap INI entries. Reducing the default time interval must be carefully evaluated; too small of a time interval may have negative impacts on performance and data quality.

[HyperionIngest]
ProcessTimeInterval=500

[SPLIngest]
ProcessTimeInterval=200