Overview
CIERA has exclusive access to two high-performance computer clusters (CIERA and Grail). These clusters are used not only for running simulations and performing data analysis, but also serve as a training ground for students in high-performance computing and computational research. In total, the two clusters provide access to 4276 compute cores.
CIERA Cluster
The CIERA cluster is available for any CIERA researcher for astronomy-related research. The cluster consists of 12 CPU nodes with dual 64-core CPUs (Intel Xeon Platinum 8592+ at 1.9 GHz) for a total of 128 cores per node. All compute nodes contain 512 GB of error-correcting DDR5 RAM.
The CIERA Cluster also includes three specialty nodes:
- A GPU node containing two NVIDIA L40S GPUs. Each L40S GPU contains 48 GB of GDDR6 VRAM. This node contains two 32-core Intel Xeon Platinum 8592+ CPUs running at 1.9 GHz.
- A high-memory node with 1 TB of error correcting DDR4 RAM. This node contains two 26-core Intel Xeon Gold 6230 CPUs at 2.10 GHz.
- A high-memory node with 2 TB of error correcting DDR5 RAM. This node contains two 32-core Intel Xeon Platinum 8562Y+ CPUs running at 2.8 GHz.
The CIERA Cluster contains 200 TB of high-speed disk storage directly connected to the cluster in a RAID configuration.
All CIERA members with access to computing resources on Quest are automatically allocated 5 TB of individual scratch space.
Grail Cluster
The NSF-funded Grail cluster is available for use by CIERA researchers working on gravitational wave related research (NSF PHY-2406802). Grail provides 18 nodes for general-purpose high-performance computation. Each node contains dual 64-core CPUs (Intel Xeon Platinum 8592+ at 1.9 GHz), for a total of 128 cores per node. All compute nodes contain 512 GB of error-correcting DDR5 RAM.
Grail contains two types of specialty nodes:
- Three nodes containing four NVIDIA A30 GPUs, for a total of 12 NVIDIA A30 GPUs. Each A30 GPU contains 24 GB of HBM2 memory. Each of these nodes contains two 32-core Intel Xeon Gold 6338 CPUs running at 2.0 GHz.
- A GPU node containing four NVIDIA L40S GPUs. Each L40S GPU contains 48 GB of GDDR6 memory. This node contains two 32-core Intel Xeon Gold 6338 CPUs running at 2.0 GHz.
The nodes are interconnected with InfiniBand HDR. Grail includes 196 TB of high-speed disk storage directly connected to the cluster in a RAID configuration.
30Standard Compute Nodes
4276CPU Cores
132CPU Processing Power (TFLOPS)
5GPU Nodes
18GPUs
673GPU Processing Power (TFLOPS)
Quest
Quest currently comprises over 1,200 compute nodes with over 80,000 CPU cores, as well as over 90 GPU nodes and 22 high-memory nodes. All nodes run Red Hat Enterprise Linux 8.10, and Quest is supported by UPS for emergency power. Additional HPC resources, including high-memory nodes, are also available through Quest.
Several CIERA research groups maintain private allocations on Quest in addition to their use of the shared system.
Support
- A dedicated Computational Specialist is on staff to assist CIERA researchers. The Computational Specialist offers trainings and tutorials, walk-in consultations, and expert guidance for HPC code development and optimization, and more.
- The Computational Specialist holds workshops roughly twice a month during the academic year to support students and researchers. These workshops can be attended both in person and through Zoom, and are recorded.
- CIERA researchers looking for details on how to access and use the Grail and CIERA clusters should refer to the CIERA internal Guide page (Northwestern NetID required).
- Video tutorials from Northwestern IT Research Computing Services are available on YouTube (visit playlist).
Funding for the Grail computer cluster was provided by the National Science Foundation’s Major Research Instrumentation (MRI) program, grant PHY-1726951.
Additional Resources
Additional resources are available to certain research groups in CIERA. These include computing resources supported by the National Science Foundation and the U.S. National Laboratories.