Cluster Resources
Hardware specifications and limits
This page summarizes the available hardware and resource limits.
Hardware overview
Compute nodes (12x)
Each of the 12 compute nodes (node01–node12) is identical:
| Resource | Spec |
|---|---|
| CPU | AMD EPYC 9655P (Zen 5, Turin) |
| Cores | 96 |
| Threads per core | 2 |
| Base clock | 2.6 GHz |
| L3 cache | 384 MB |
| RAM | 1152 GB DDR5-5600 ECC (24x 48GB) |
GPU node (1x)
The GPU node (gnode01) has two CPUs and two GPUs:
| Resource | Spec |
|---|---|
| GPUs | 2x NVIDIA H200 (141 GB HBM3e each, PCIe) |
| CPUs | 2x AMD EPYC 9555 (Zen 5, Turin) |
| Cores | 128 (2x 64) |
| Threads per core | 2 |
| Base clock | 3.2 GHz |
| L3 cache | 512 MB (2x 256 MB) |
| RAM | 3072 GB DDR5-4800 ECC (48x 64GB) |
Head node
| Resource | Spec |
|---|---|
| CPU | AMD EPYC 7443 (Zen 3, Milan) |
| Cores | 24 |
| Base clock | 2.85 GHz |
| L3 cache | 128 MB |
| RAM | 1024 GB DDR4-3200 ECC |
The head node is for login, package installation, and job submission. Do not run computations here.
Partitions
sinfo # View current partition status| Partition | Nodes | Default | Max time |
|---|---|---|---|
compute |
12 (node01-12) | Yes | 20 days |
gpu |
1 (gnode01) | No | 20 days |
Quality of Service (QoS)
| QoS | Max time | Limits | Priority |
|---|---|---|---|
interactive |
1 day | 2 jobs, 192 CPUs | highest |
short |
1 hour | – | high |
medium |
1 day | – | medium |
long |
7 days | – | low |
extended |
20 days | 1 job | lowest |
The interactive QoS is automatically applied for salloc sessions.
Storage
| Path | Type | Purpose |
|---|---|---|
/srv/home/<user> |
NFS (NVMe RAID5) | Home directory, scripts, active project data |
/srv/data |
NFS (shared) | Shared datasets |
/mnt/sas |
NFS (HDD array) | Long-term storage, archives, inactive projects |
/srv/home lean
/srv/home is backed by fast NVMe storage with limited capacity. Move inactive projects and large datasets you don’t actively need to /mnt/sas to keep space available for everyone. /mnt/sas is slower (spinning disks) but has substantially more capacity.
Archive storage (/mnt/sas)
The /mnt/sas directory is intended for large datasets and archival storage – data that you don’t need immediate access to but want to keep available on the cluster.
| Path | Purpose |
|---|---|
/mnt/sas/users/<username> |
Your personal archive space |
/mnt/sas/groups/<group> |
Shared data for your research group |
/mnt/sas/scratch |
Temporary workspace (may be cleaned periodically) |
The cluster storage is not backed up. Any deleted data cannot be restored.
For important data that needs proper backup and long-term retention, use your project folders on the BIPS network share managed by IT. The cluster is meant for active computation and temporary storage, not as a primary archive.
Useful commands
# Cluster status
sinfo
# Your running jobs
squeue --me
# Your past jobs
sacct --starttime=today
# Detailed job info
scontrol show job <jobid>
# QoS limits
sacctmgr show qos format=name,maxwall,maxjobspu
# Node details
scontrol show node <nodename>