HTCondor at Washington and Lee University

HTCondor¹⁾ is a high-throughput computing system that is made available to Washington and Lee students and faculty by the Computer Science department. It is administered by Garrett Koller under the supervision of Dr. Joshua Stough.

HTCondor is a high-throughput computing system. Similar to a high-performance computing system, which is the most typical kind of supercomputer, a “high-throughput computing system” is designed to process very large amounts of data with distributed computational resources. Since such programs are typically limited with respect to reading and writing the data, high performance is needed in the realm of data access and manipulation rather than in processing speed. In order to accomplish this, the system provides an efficient way to store and access data while providing computational resources that allow the data to be processed by many machines at once if the data permits.²⁾

Use HTCondorView to check the status of the HTCondor cluster at any given moment. You can even see which users are submitting the most jobs to HTCondor.

Check out this spreadsheet to see HTCondor's benchmarking statistics, including how many CPU and GPU cores are on the cluster.

When HTCondor isn't busy running user jobs, it intelligently runs what are called “Backfill” jobs. This means that when the computers in the HTCondor cluster aren't busy and HTCondor doesn't have any jobs to run, HTCondor participates in volunteer computing projects. Right now, HTCondor is participating in the MilkyWay@home Project, which is a project created and supported by Rensselaer Polytechnic Institute to model the 3D structures in our galaxy.

We currently use HTCondor version 7.8.1. The HTCondor documentation for version 7.8.1 can be found here. You can find the official documentation for other versions of HTCondor on the official HTCondor Project site.