Description

Bridges-2 combines high-performance computing (HPC), high performance artificial intelligence (HPAI), and large-scale data management to support simulation and modeling, data analytics, community data, and complex workflows. Bridges-2 Accelerated GPU (GPU) nodes are optimized for scalable artificial intelligence (AI; deep learning). They are also available for accelerated simulation and modeling applications. Bridges-2 has four types of GPU nodes: 10 HPE Cray 670 h100-80 nodes, with eight H100-SXM5-80GB GPUs each with 80GB of GPU memory and a total of 2TB RAM per node; 24 HPE v100-32 nodes with eight V100 GPUs with NVLink, each with 32GB of GPU memory and a total of 512GB RAM per node;9 v100-16 nodes containing eight V100 GPUs without NVLink, each with 16GB of GPU memory and a total of 192GB RAM per node; and 3 HPE l40s-48 nodes with 8 L40S GPUs without NVLink, each with 48GB of GPU Memory and a total of 1TB RAM per node.

The nodes are connected to Bridges-2's other compute nodes and its Ocean parallel filesystem and archive by two HDR-200 InfiniBand links, providing 400Gbps of bandwidth to enhance scalability of deep learning training.

Resource ID
646
Global Resource ID
bridges2-gpu.psc.access-ci.org
Resource Type
Compute
Latest Status
production
Latest Status Begin
Project Affiliation
ACCESS
Organization Name
Pittsburgh Supercomputing Center
RP Description

Bridges-2 GPU is the GPU-accelerated tier of Bridges-2, built from nodes that pair NVIDIA H100, L40S, or V100 GPUs (8 per node, 16 on the DGX-2) with two Intel Xeon CPUs and up to 2 TB of RAM. It is well suited to GPU-accelerated deep learning and molecular dynamics, and is often used for large-scale model training and inference and for biomolecular simulation. Its most heavily used software spans machine-learning frameworks and molecular-dynamics codes.

Top Software (cached)
[{"name":"python","job_count":158458,"description":"Python is a high-level, interpreted programming language known for its simplicity and readability. It supports multiple programming paradigms and has a vast ecosystem of libraries and frameworks.","research_field":"Computer & Information Sciences, Software Engineering, Systems & Development","web_page":"https:\/\/www.python.org\/","documentation":"https:\/\/docs.python.org"},{"name":"amber","job_count":8411,"description":"Amber is a suite of highly extensible molecular simulation programs. It is designed for simulations of biomolecules such as proteins, nucleic acids, and carbohydrates, and can also be used for small molecules.","research_field":"Biological Sciences","web_page":"https:\/\/ambermd.org\/","documentation":"https:\/\/ambermd.org\/doc12\/Amber23.pdf"},{"name":"gromacs","job_count":697,"description":"GROMACS (GROningen MAssive Parallel MD for Molecular Dynamics) is a versatile package for molecular dynamics simulations with a strong emphasis on high-performance computing capabilities.","research_field":"Biological Sciences","web_page":"https:\/\/www.gromacs.org\/","documentation":"https:\/\/manual.gromacs.org\/current\/index.html"},{"name":"lammps","job_count":154,"description":"LAMMPS (Large-scale Atomic\/Molecular Massively Parallel Simulator) is a classical molecular dynamics code designed for simulating large-scale atomistic systems. It is highly versatile and can be used to model a wide range of materials and complex molecular structures.","research_field":"Chemical Sciences","web_page":"https:\/\/www.lammps.org\/","documentation":"https:\/\/docs.lammps.org\/Manual.html"},{"name":"r","job_count":142,"description":"R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows, and MacOS.","research_field":"Computer Science","web_page":"https:\/\/www.r-project.org\/","documentation":"https:\/\/cran.r-project.org\/manuals.html"},{"name":"namd","job_count":54,"description":"NAMD (NAnoscale Molecular Dynamics) is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. It is optimized for the simulation of biomolecular systems containing millions of atoms.","research_field":"Biochemistry and Molecular Biology","web_page":"https:\/\/www.ks.uiuc.edu\/Research\/namd\/","documentation":"https:\/\/www.ks.uiuc.edu\/Research\/namd\/current\/ug\/"},{"name":"X11 applications","job_count":7},{"name":"castro","job_count":3},{"name":"charm++","job_count":1},{"name":"hh-suite","job_count":1,"description":"HH-suite is a software package for sensitive protein sequence searching based on profile hidden Markov models. It includes tools for the alignment of protein sequences, detecting remote homologs, and predicting protein structures.","research_field":"Biological Sciences","web_page":"https:\/\/github.com\/soedinglab\/hh-suite","documentation":"https:\/\/github.com\/soedinglab\/hh-suite\/wiki#summary-of-command-line-parameters"}]
MFA Required
Off
Jobs Information

You can run jobs at different sizes and durations on Bridges-2 GPU. The following lists the different queues that you can submit to, describing how many nodes you get, how long you can run, the type of resources you get, and the average wait time.

Jobs are submitted through Slurm. Unless you set otherwise, a job uses a one-hour walltime, up to a 48-hour maximum. Request GPUs by type and count with --gpus=type:n or --gres=gpu:type:n, where the type is one of h100-80, l40s-48, v100-32, or v100-16. In batch mode n is the total GPUs for the job (a multiple of 8); in an interactive session (interact) it is the GPUs per node. Pick a partition by how much of a node you need:

  • GPU: whole nodes, all eight GPUs on each (16 on the DGX-2), charged for the full node.
  • GPU-shared: 1 to 4 GPUs on a single node shared with other jobs, charged only for the GPUs you request.
  • GPU-small: interactive and small or quick jobs on a single V100-32 node.

For submission commands, sample scripts, and node-sharing details, see the Bridges-2 Batch Jobs guide. Partition-specific information is on the GPU and GPU-Shared partitions section of the user guide.

Queue Specifications
Queue Name
GPU-shared
Purpose
V100-32 Node. Jobs using 1 to 4 of a node's V100-32 GPUs, shared with other jobs; charged per GPU.
CPU Type
2x Intel Xeon Gold 6248 "Cascade Lake"
GPU Type
NVIDIA Tesla V100-32GB SXM2
GPU Count
8
GPU vRAM
32
CPU Count
40
Node RAM
512
Queue Name
GPU-shared
Purpose
H100-80 Node. Jobs using 1 to 4 of a node's H100-80 GPUs, shared with other jobs; charged per GPU.
CPU Type
2x Intel Xeon "Sapphire Rapids" 8470
GPU Type
NVIDIA H100-80GB SXM5
GPU Count
8
GPU vRAM
80
CPU Count
104
Node RAM
2000
Queue Name
GPU-shared
Purpose
V100-16 Node. Jobs using 1 to 4 of a node's V100-16 GPUs, shared with other jobs; charged per GPU.
CPU Type
2x Intel Xeon Gold 6148
GPU Type
NVIDIA V100-16GB
GPU Count
8
GPU vRAM
16
CPU Count
40
Node RAM
192
Queue Name
GPU-shared
Purpose
L40S-48 Node. Jobs using 1 to 4 of a node's L40S-48 GPUs, shared with other jobs; charged per GPU.
CPU Type
2x Intel Xeon 6740E
GPU Type
NVIDIA L40S-48GB
GPU Count
8
GPU vRAM
48
CPU Count
192
Node RAM
1000
Queue Name
GPU
Purpose
V100-32 Node. Jobs using one or more whole V100-32 nodes, all eight GPUs each; charged for the full node.
CPU Type
2x Intel Xeon Gold 6248 "Cascade Lake"
GPU Type
NVIDIA Tesla V100-32GB SXM2
GPU Count
8
GPU vRAM
32
CPU Count
40
Node RAM
512
Queue Name
GPU
Purpose
H100-80 Node. Jobs using one or more whole H100-80 nodes, all eight GPUs each; charged for the full node.
CPU Type
2x Intel Xeon "Sapphire Rapids" 8470
GPU Type
NVIDIA H100-80GB SXM5
GPU Count
8
GPU vRAM
80
CPU Count
104
Node RAM
2000
Queue Name
GPU
Purpose
V100-16 Node. Jobs using one or more whole V100-16 nodes, all eight GPUs each; charged for the full node.
CPU Type
2x Intel Xeon Gold 6148
GPU Type
NVIDIA V100-16GB
GPU Count
8
GPU vRAM
16
CPU Count
40
Node RAM
192
Queue Name
GPU
Purpose
L40S-48 Node. Jobs using one or more whole L40S-48 nodes, all eight GPUs each; charged for the full node.
CPU Type
2x Intel Xeon 6740E
GPU Type
NVIDIA L40S-48GB
GPU Count
8
GPU vRAM
48
CPU Count
192
Node RAM
1000
Queue Name
GPU
Purpose
DGX-2 special node. Jobs using the whole DGX-2 node and all 16 V100-32 GPUs; charged for the full node.
CPU Type
2x Intel Xeon Platinum 8168
GPU Type
NVIDIA Volta V100-32GB
GPU Count
16
GPU vRAM
32
CPU Count
48
Node RAM
1500
Queue Name
GPU-small
Purpose
Used for interactive and small or quick GPU jobs on a single V100-32 node.
CPU Type
2x Intel Xeon GoldĀ 6248
GPU Type
NVIDIA Tesla V100-32GB SXM2
GPU Count
8
GPU vRAM
32
CPU Count
40
Node RAM
512