TSRI - IT Services

The garibaldi LINUX Compute Cluster

TSRI has a cluster of 1200 2.33 GHz Intel dual core XEON-EMT processors and 620 3.4 GHz Intel XEON-EMT processors for a total of 3020 CPUs. Each server has a 72 GByte local disk and all systems share a 20 TeraByte Lustre distributed filesystem. Jobs running on our LINUX clusters are managed by PBS. An additional server, garibaldi, with two Intel dual core XEON-EMT CPUs serves as an interactive frontend where you can compile and test your code and/or test your runs before submission to PBS. An account on the cluster is available to anyone with a valid TSRI email account. To request an account, email wyoung@scripps.edu

Available software

If an application that you need is missing, please request that it be installed.

Software package	Owner	Location	Description
Amber	case	/garibaldi/people-b/applications/amber9	molecular simulation (EM, MD, etc.)
ARP	weber	/garibaldi/people-b/applications/arp_6.0	Xray structures
auto3dem	weber	/garibaldi/people-b/applications/auto3dem_v2.0	EM 3D reconstruction
autodock	garrett	/garibaldi/people-b/applications/autodock	docking, virtual screening
BLAST	weber	/garibaldi/people-b/applications/blast-2.2.10	sequence searching
ClustalW	weber	/garibaldi/people-b/applications/clustalw1.83	sequence alignment
CCP4	weber	/garibaldi/people-b/applications/ccp4-6.0.2	Xray structures
CNS	weber	/garibaldi/people-b/applications/cns_solve_1.2	Xray and NMR structures
Cyana	weber	/garibaldi/people-b/applications/cyana-2.1	NMR structures
em3dr2	weber	/garibaldi/people-b/applications/pft3dr	EM 3D reconstruction
EMAN	wyoung	/garibaldi/people-b/applications/EMAN	EM 3D reconstruction
epmr	weber	/garibaldi/people-b/applications/epmr	Xray structure determination
Gaussian 03	weber	/garibaldi/people-b/applications/g03	quantum chemistry
Gamess	weber	/garibaldi/people-b/applications/gamess	quantum chemistry
HMMER	weber	/garibaldi/people-b/applications/hmmer-2.3.2	sequence comparison
icm	abagyan	/garibaldi/people-b/applications/icm	molecular modelling
MEME	weber	/garibaldi/people-b/applications/meme.3.0.4	sequence motif analysis
modelfree	weber	/garibaldi/people-b/applications/modelfree	NMR relaxation analysis
modeller8	weber	/garibaldi/people-b/applications/modeller8v2	protein homology modelling
mpich2	weber	/garibaldi/people-b/applications/mpich2	MPI multiprocessing
NAMD	weber	/garibaldi/people-b/applications/namd	MD
Orca	weber	/garibaldi/people-b/applications/orca_amd64_2.5.20	quantum chemistry
openmpi	weber	/garibaldi/people-b/applications/openmpi-1.1.4	MPI multiprocessing
phaser	weber	/garibaldi/people-b/applications/phaser-1.3.1	crystal structure determination
Phenix	weber	/garibaldi/people-b/applications/phenix-1.24.1b	crystal structure determination
rosetta	weber	/garibaldi/people-b/applications/rosetta	structure prediction
sasref, gasbor, bunch	weber	/garibaldi/people-b/applications/sax	small angle Xray scattering
shelx	weber	/garibaldi/people-b/applications/shelx	crystal structure determination
solve	weber	/garibaldi/people-b/applications/solve-2.13	crystal structure determination

Compilation issues

The garibaldi cluster has both the Intel suite of compilers and the GNU toolkit.
Intel compiler setup:

if ( ${?prompt}) then
        source /opt/intel/fce/9.1.033/bin/ifortvars.csh
        source /opt/intel/cce/9.1.039/bin/iccvars.csh
endif

Use the following Intel compiler flags for best performance:
Start with -fast (currently equivalent to -ipo -O3 -no-prec-div -static -xP)
If this doesn't work, try individual combinations of -xT -O3 -ipo in this order. Always link statically with -static.
Use -O2 and/or -ip if your code breaks with the above heavy optimization, but still try to use -xT or -xP to take advantage of SSE2 and SSE3 extensions.
Remember: The executables will only run on 64bit Intel EM64T CPUs!
Some more hints for EM64T CPUs: C++ Fortran and Performance Optimization Quick Guide

Checkpoint/restart

Some applications offer built-in checkpoint and restart support. These are listed below.

Application CPR facility

Amber of unknown quality

Gaussian 03 make sure checkpoint file gets saved by naming it explicitly

Jaguar from file restart.in or jobname.**.in