Supercomputing:

The garibaldi LINUX Compute Cluster

The Institute maintains a large 64-bit LINUX cluster, garibaldi, on the La Jolla campus to support support scientific computation and data processing. The garibaldi cluster consists of: 192 2.66 GHz Intel quad core E5430 XEON-EMT processors, 192 2.26 GHz Intel quad core E5520 XEON-EMT processors, and 1200 2.33 GHz Intel dual core 5140 XEON-EMT processors for a total of 3936 CPUs. Each server has a 72 GByte or a 146 GByte local disk and all systems share a 30 TeraByte Lustre distributed filesystem. Jobs running on our LINUX clusters are managed by PBS. You can check the current status of the garibaldi cluster here or view historical usage trends here. An additional server, garibaldi, with two Intel dual core XEON-EMT CPUs serves as an interactive frontend where you can compile and test your code and/or test your runs before submission to PBS. An account on the cluster is available to anyone with a valid TSRI email account. To request an account, email wyoung@scripps.edu

Available software

If an application that you need is missing, please request that it be installed.

Software package	Location	Description
Amber	/lustre/people/applications/amber10	molecular simulation
	/lustre/people/applications/amber9	(EM, MD, etc.)
	/lustre/people/applications/amber8
ARP	/lustre/people/applications/arp_6.0	Xray structures
auto3dem	/lustre/people/applications/auto3dem_v2.0	EM 3D reconstruction
autodock	/lustre/people/applications/autodock	docking, virtual screening
BLAST	/lustre/people/applications/blast-2.2.10	sequence searching
ClustalW	/lustre/people/applications/clustalw1.83	sequence alignment
CCP4	/lustre/people/applications/ccp4-6.0.2	Xray structures
CNS	/lustre/people/applications/cns_solve_1.2	Xray and NMR structures
Cyana	/lustre/people/applications/cyana-2.1	NMR structures
em3dr2	/lustre/people/applications/pft3dr	EM 3D reconstruction
EMAN	/lustre/people/applications/EMAN	EM 3D reconstruction
epmr	/lustre/people/applications/epmr	Xray structure determination
Gaussian 03	/lustre/people/applications/g03	quantum chemistry
Gamess	/lustre/people/applications/gamess	quantum chemistry
HMMER	/lustre/people/applications/hmmer-2.3.2	sequence comparison
MEME	/lustre/people/applications/meme.3.0.4	sequence motif analysis
modelfree	/lustre/people/applications/modelfree	NMR relaxation analysis
modeller8	/lustre/people/applications/modeller8v2	protein homology modelling
mpich2	/lustre/people/applications/mpich2	MPI multiprocessing
NAMD	/lustre/people/applications/namd	MD
Orca	/lustre/people/applications/orca_amd64_2.5.20	quantum chemistry
openmpi	/lustre/people/applications/openmpi-1.1.2	MPI multiprocessing
	/lustre/people/applications/openmpi-1.1.4
	/lustre/people/applications/openmpi-1.2.2
	/lustre/people/applications/openmpi-1.3.3
phaser	/lustre/people/applications/phaser-1.3.1	crystal structure determination
Phenix	/lustre/people/applications/phenix-1.24.1b	crystal structure determination
rosetta	/lustre/people/applications/rosetta	structure prediction
sasref, gasbor, bunch	/lustre/people/applications/sax	small angle Xray scattering
shelx	/lustre/people/applications/shelx	crystal structure determination
solve	/lustre/people/applications/solve-2.13	crystal structure determination

Compilation issues

The garibaldi cluster has both the Intel suite of compilers and the GNU toolkit.
Intel compiler setup:

if ( ${?prompt}) then
        source /opt/intel/fce/9.1.033/bin/ifortvars.csh
        source /opt/intel/cce/9.1.039/bin/iccvars.csh
endif

Use the following Intel compiler flags for best performance:
Start with -fast (currently equivalent to -ipo -O3 -no-prec-div -static -xP)
If this doesn't work, try individual combinations of -xT -O3 -ipo in this order. Always link statically with -static.
Use -O2 and/or -ip if your code breaks with the above heavy optimization, but still try to use -xT or -xP to take advantage of SSE2 and SSE3 extensions.
Remember: The executables will only run on 64bit Intel EM64T CPUs!
Some more hints for EM64T CPUs: C++ Fortran and Performance Optimization Quick Guide

Checkpoint/restart

Some applications offer built-in checkpoint and restart support. These are listed below.

Application CPR facility

Amber of unknown quality

Gaussian 03 make sure checkpoint file gets saved by naming it explicitly

Jaguar from file restart.in or jobname.**.in