HTCondor Installation
Download HTCondor
HxMap has been tested so far with HTCondor 8.x.x and 9.x.x series releases. Note that for 9.x.x series releases additional security configurations are required. We recommend to select releases listed in the Long Term Support (LTS) releases of HTCondor.
Download the HTCondor installer for the chosen release version here.
Installation Prerequisites
ID | Topic | Description |
|---|---|---|
P1 | Manager | Use Windows Server Edition (Windows 2016 recommended) to use as the cluster manager. This machine may also be used as the shared installation location, from which all submitters and cluster nodes will access the HxMap software. This machine must have reliable network connectivity and be planned for high availability without frequent reboot. This is also an ideal machine for the license server. For our examples we will call this machine cluster-manager.leica.com |
P2 | Submitter | Use Windows Desktop/Server Edition (Window 10 x64 (preferred), Windows Server 2016/ 2012 R2) for submission nodes. Please have a list of the machine names and IP addresses. For our example, we shall call these machines cluster-submitter.leica.com |
P3 | Node | Use Windows Desktop/Server Edition (Window 10 x64 (preferred), Windows Server 2016/ 2012 R2) for processing nodes. Please have a list of these machine names and IP addresses. For our example, we shall call these machines cluster-node1.leica.com |
P4 | Authentication | All machines must be part of an active directory. This is important to eliminate authentication issues. |
P5 | DNS | Proper DNS service functioning on the network. |
P6 | Network Shares | A list of all data shares (both drive letters and UNC names) that the production environment uses and needs to be accessible to the cluster. A single user account and password that has read/write permissions to all the above shared folders is required. We strongly suggest using DFS (Distributed File System), to simplify this to a single DFS root share to be mapped by all production users. |
P7 | Remote Administration | For the ease of setup and maintenance all machines in the cluster should have RDP (Remote Desktop Protocol) turned on. |
P8 | Software Share | On the cluster manager machine, setup a share called "HxMap". Set the permissions so that the account in P6 can read/ write from this share. If you are using DFS you can mount this share as a DFS leaf with a name such as “cluster_software” |
P9 | Obtain HTCondor | Download the latest stable HTCondor version from the HTCondor website (https://research.cs.wisc.edu/htcondor/tarball/ ) and store it in the share created in P8. |
P10 | Administrator Privilege | Have an account with administrator rights on all cluster machines to use for installation of HTCondor. |
P11 | Firewall | Firewall / Virus Scan should be TURNED OFF or include corresponding exceptions for HTCondor Daemons/processes and Ports. |
IT infrastructure testing
DNS testing | |
|---|---|
T1 | From the machine identified as the cluster manager open a command line window and confirm that it can ping the submitters and worker nodes by DNS name: i.e.: ping cluster-node1.leica.com Response should be the IP address. Repeat for all worker nodes and all submitters and ensure the name resolves to the correct IP! |
T2 | Login to worker nodes and confirm they have correct DNS resolution for the cluster manager: i.e.: ping cluster-manager.leica.com Response should be the IP address for the manager machine. Repeat for all the worker nodes! |
T3 | Login to submitters and confirm they have correct DNS resolution for the cluster manager: i.e.: ping cluster-manager.leica.com Response should be the IP address for the manager machine. Repeat for all the submitters! |
We are now confident that the DNS resolution is correct. If you had any errors above or timeouts getting a response, there is no point in continuing as HTCondor WILL NOT work reliably. Please work with your IT department to resolve these issues before continuing.
HTCondor Installation
Installing and properly configuring HTCondor requires basic understanding of IT infrastructure. Working together with your IT department is highly recommended.
The procedures for the installation of HTCondor on the Cluster Manager / Submitter / Nodes are very similar and only a handful of the configuration parameters are different as specified below.
Install HTCondor using the installer matching the operating system and architecture
Pick the appropriate role for the system in the HTCondor installer dialog:
Role | recommended Installation |
|---|---|
Cluster Manager | Do not run jobs on this machine |
Submitter | Submit Jobs to HTCondor Pool / Do not run jobs on this machine |
Node | Always run jobs and never suspend them |
Once installation is completed, check the variable DAEMON_LIST in the "condor_config" file to verify the expected daemons (services) are listed. Also check, that the corresponding daemon processes have started and are visible in the Windows task manager. The typical setup is listed in the table below:
Role | Typical Configuration | Processes to Verify |
|---|---|---|
Cluster Manager | DAEMON_LIST = MASTER COLLECTOR NEGOTIATOR | condor_master, condor_collector, condor_negotiator |
Submitter | DAEMON_LIST = MASTER SCHEDD | condor_master, condor_schedd |
Node | DAEMON_LIST = MASTER STARTD | condor_master, condor_startd |
Check in the condor_config file, that the machines are configured with the required Read/ Write permission for your network:
CODEALLOW_WRITE = * ALLOW_READ = $(CONDOR_HOST), $(IP_ADDRESS)Further preparation of the condor_config file:
Role | Configuration | Comment |
|---|---|---|
Node |
CODE
| By default HxMap will use all cores, to support single core (useful for non threaded applications) and whole machine jobs on the same node. Please reference the HTCondor Knowledge base at https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=WholeMachineSlots |
Once the condor_config is setup as described above, send the changes to the Cluster Manager by restarting the HTCondor service on the machine with command “condor_restart”.
Use command “condor_status” to check the number of Slots.
After editing the condor_config file there should be only one machine listed per node
you should see a list of the worker nodes already installed in the cluster.
On the submitter machine: Type “condor_q” and see if HTCondor returns the empty queue
On the submitter machine: Use the “condor_store_cred add” command to add the “user name” and “password” using the account created in P6.
If “condor_status” doesn’t show the expected number of nodes, STOP and validate that the Firewall is Turned OFF and the network configuration is correct by looking at the HTCondor logs.