HTCondor at Washington and Lee University

HTCondor1) is a high-throughput computing system that is made available to Washington and Lee students and faculty by the Computer Science department. It is administered by Garrett Koller under the supervision of Dr. Joshua Stough.

What is HTCondor?

HTCondor is a high-throughput computing system. Similar to a high-performance computing system, which is the most typical kind of supercomputer, a “high-throughput computing system” is designed to process very large amounts of data with distributed computational resources. Since such programs are typically limited with respect to reading and writing the data, high performance is needed in the realm of data access and manipulation rather than in processing speed. In order to accomplish this, the system provides an efficient way to store and access data while providing computational resources that allow the data to be processed by many machines at once if the data permits.2)

How is HTCondor Used at W&L?

HTCondor's Status

Use HTCondorView to check the status of the HTCondor cluster at any given moment. You can even see which users are submitting the most jobs to HTCondor.

Benchmarking Statistics

Check out this spreadsheet to see HTCondor's benchmarking statistics, including how many CPU and GPU cores are on the cluster.

Mapping the Universe

When HTCondor isn't busy running user jobs, it intelligently runs what are called “Backfill” jobs. This means that when the computers in the HTCondor cluster aren't busy and HTCondor doesn't have any jobs to run, HTCondor participates in volunteer computing projects. Right now, HTCondor is participating in the MilkyWay@home Project, which is a project created and supported by Rensselaer Polytechnic Institute to model the 3D structures in our galaxy.

HTCondor Documentation

We currently use HTCondor version 7.8.1. The HTCondor documentation for version 7.8.1 can be found here. You can find the official documentation for other versions of HTCondor on the official HTCondor Project site.

1)
formerly known as Condor
2)
Ideally, the data is organized in such a way that it can be processed in many independent segments that do not rely on other segments in order to make progress.
condor/home.txt · Last modified: 2013/10/06 23:11 by garrettheath4
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0