Description

The Delta CPU resource comprises 124 dual-socket compute nodes for general purpose computation across a broad range of domains able to benefit from the scalar and multi-core performance provided by the CPUs, such as appropriately scaled weather and climate, hydrodynamics, astrophysics, and engineering modeling and simulation, and other domains using algorithms not yet adapted for the GPU. Each Delta CPU node is configured with 2 AMD EPYC 7763 (“Milan”) processors with 64-cores/socket (128-cores/node) at 2.45GHz and 256GB of DDR4-3200 RAM. An 800GB, NVMe solid-state disk is available for use as local scratch space during job execution. All Delta CPU compute nodes are interconnected to each other and to the Delta storage resource by a 100 Gb/sec HPE Slingshot network fabric.

Resource ID
741
Global Resource ID
delta-cpu.ncsa.access-ci.org
Resource Type
Compute
Latest Status
production
Latest Status Begin
Latest Status End
Project Affiliation
ACCESS
Organization Name
National Center for Supercomputing Applications
RP Description

Delta CPU is a CPU-only compute cluster consisting of 132 dual-socket AMD EPYC 7763 “Milan” nodes with 128 CPU cores, 256 GB of RAM, and 0.74 TB of local storage per node. It is particularly well suited for CPU-intensive and parallel workloads that do not require GPU acceleration, and is often used for weather and climate modelling, hydrodynamics, astrophysics, and engineering simulation. It includes a great deal of scientific computing, parallel programming, and data-analysis software.

Storage Text

Delta CPU provides Home, Projects, Work HDD, Work NVMe, and Node-local storage. The work file systems are shared with DeltaAI and should be used for job I/O; home directories are separate, and home-directory snapshots are not backups. For details, see [Delta File Systems].

Jobs Information

You can run jobs at different sizes and durations on Delta CPU. The following lists the different queues that you can submit to, describing how many nodes you get, how long you can run, the type of resources you get, and the average wait time.

Delta uses the Slurm scheduler. If you do not request a wall-clock time or memory amount, Delta defaults to 30 minutes and 1,000 MB of memory per core. Submit batch jobs with sbatch; use srun to run a command or start an interactive job; and use salloc to reserve resources for multiple commands. For instructions and Delta-specific examples, see [Delta Batch Jobs] for sbatch and [Delta Interactive Jobs] for srun and salloc.

Compute resources are also available through [Delta Open OnDemand] applications such as JupyterLab, VS Code Code Server, and noVNC Desktop.

Track your work with the usual Slurm commands: squeue -u $USER shows the status of your queued and running jobs, scontrol show job followed by a job ID shows the detail for one job, sinfo -a summarizes all Delta partitions and their node states, and scancel followed by a job ID cancels a queued job or terminates a running one.

Delta CPU jobs should use cpu for standard production jobs, cpu-preempt for checkpointing-capable jobs that can tolerate interruption, and cpu-interactive for short interactive sessions.

Wall-time limits and node counts for each partition are in the table below.

Node sharing is the default. To request exclusive use of a CPU node, request all consumable resources for the node or use --exclusive --mem=0. CPU and GPU allocations use separate local charge accounts, so submit CPU jobs with the appropriate Delta CPU account.

For complete job guidance, see [Delta Running Jobs]. Example scripts for serial, MPI, OpenMP, hybrid, and other job types are available in [Delta Sample Scripts].

Storage Filesystems
Directory
Home
File System Path
$HOME
Quota size
100
Quota inode amount
750000
Purge Policy
Never purged
Backup Policy
Snapshots only, retained 30 days
Notes
Software, scripts, and job files; not intended for job I/O. Snapshots live on the same hardware as the primary copy and are not a backup - back up anything you cannot lose.
Directory
Projects
File System Path
/projects/<project>
Quota size
500
Quota inode amount
300000
Purge Policy
Never purged
Backup Policy
Not backed up
Notes
Shared project data, software, and results. Up to 25 TB by allocation request.
Directory
Work HDD
File System Path
/work/hdd
Quota size
1000
Quota inode amount
500000
Purge Policy
Never purged
Backup Policy
Not backed up
Notes
Active computational data and job I/O. Up to 100 TB by allocation request. Shared with DeltaAI.
Directory
Work NVMe
File System Path
/work/nvme/<project>
Purge Policy
Never purged
Backup Policy
Not backed up
Notes
Available upon request. Use quota to check assigned capacity and inode limits. Best for frequent small-file I/O. Shared with DeltaAI.
Directory
Node-local
File System Path
/tmp
Purge Policy
Purged at job end
Backup Policy
Not backed up
Notes
Approximately 740 GB per CPU node; no enforced quota. Capacity may be shared by concurrent jobs. Use for fast small-file I/O and copy results before the job ends.
Queue Specifications
Queue Name
cpu
Purpose
Standard production batch work - the queue to use unless you need one of the others
CPU Type
2x AMD EPYC 7763 "Milan"
GPU Type
None
GPU Count
0
GPU vRAM
0
CPU Count
128
Node RAM
256
Queue Name
cpu-preempt
Purpose
Checkpointing-capable CPU jobs that can tolerate interruption and loss of unsaved progress
CPU Type
2x AMD EPYC 7763 "Milan"
GPU Type
None
GPU Count
0
GPU vRAM
0
CPU Count
128
Node RAM
256
Queue Name
cpu-interactive
Purpose
Short interactive CPU sessions for development, testing, debugging, compiling, and other interactive work; maximum 4 nodes per job
CPU Type
2x AMD EPYC 7763 "Milan"
GPU Type
None
GPU Count
0
GPU vRAM
0
CPU Count
128
Node RAM
256