Iris Overview¶
Iris is a Dell/Intel supercomputer which consists of 196 compute nodes, totaling 5824 compute cores and 52224 GB RAM, with a peak performance of about 1,072 PetaFLOP/s.
All nodes are interconnected through a Fast InfiniBand (IB) EDR network1, configured over a Fat-Tree Topology (blocking factor 1:1.5). Iris nodes are equipped with Intel Broadwell or Skylake processors. Several nodes are equipped with 4 Nvidia Tesla V100 SXM2 GPU accelerators. In total, Iris features 96 Nvidia V100 GPU-AI accelerators allowing for high speedup of GPU-enabled applications and AI/Deep Learning-oriented workflows. Finally, a few large-memory (fat) computing nodes offer multiple high-core density CPUs and a large live memory capacity of 3 TB RAM/node, meant for in-memory processing of huge data sets.
Two global high-performance clustered file systems are available on all ULHPC computational systems: one based on GPFS/SpectrumScale, one on Lustre.
Iris Compute Iris Interconnect Global Storage
The cluster runs a Red Hat Linux Family operating system. The ULHPC Team supplies on all clusters a large variety of HPC utilities, scientific applications and programming libraries to its user community. The user software environment is generated using Easybuild (EB) and is made available as environment modules from the compute nodes only.
Slurm is the Resource and Job Management Systems (RJMS) which provides computing resources allocations and job execution. For more information: see ULHPC slurm docs.
Cluster Organization¶
Login/Access servers¶
- Iris has 2 access servers (128 GB of memory each, general access)
access[1-2] - Each login node has two sockets, each socket is populated with an Intel Xeon E5-2697A v4 processor (2.6 GHz, 16 core)
Access servers are not meant for compute!
- The
modulecommand is not available on the access servers, only on the compute nodes - you MUST NOT run any computing process on the access servers.
Rack Cabinets¶
The Iris cluster (management, compute and interconnect) is installed across 7 racks within a row of cabinets in the premises of the Centre de Calcul (CDC), in the CDC-S02-005 server room.
| Server Room | Rack ID | Purpose | Type | Description |
|---|---|---|---|---|
| CDC-S02-005 | D02 | Network | Interconnect equipment | |
| CDC-S02-005 | D04 | Management | Management servers, Interconnect | |
| CDC-S02-005 | D05 | Compute | regular | iris-[001-056], interconnect |
| CDC-S02-005 | D07 | Compute | regular | iris-[057-112], interconnect |
| CDC-S02-005 | D09 | Compute | regular | iris-[113-168], interconnect |
| CDC-S02-005 | D11 | Compute | gpu, bigmem | iris-[169-177,191-193](gpu), iris-[187-188](bigmem) |
| CDC-S02-005 | D12 | Compute | gpu, bigmem | iris-[178-186,194-196](gpu), iris-[189-190](bigmem) |
In addition, the global storage equipment (GPFS/SpectrumScale and Lustre, common to both Iris and Aion clusters) is installed in another row of cabinets of the same server room.
-
Infiniband (IB) EDR networks offer a 100 Gb/s throughput with a very low latency (0,6\mus). ↩