ULHPC Data Center - Centre de Calcul (CDC)¶
The ULHPC facilities are hosted within the University's "Centre de Calcul" (CDC) data center located in the Belval Campus.
Power and Cooling Capacities¶
Established over two floors underground (CDC-S01 and CDC-S02) of ~1000~100m2 each, the CDC features five server rooms per level (each of them offering ~100m2 as IT rooms surface). When the first level CDC-S01 is hosting administrative IT and research equipment, the second floor (CDC-S02) is primarily targeting the hosting of HPC equipment (compute, storage and interconnect).
A power generation station supplies the HPC floor with up to 3 MW of electrical power, and 3 MW of cold water at a 12-18°C regime used for traditional Airflow with In-Row cooling. A separate hot water circuit (between 30 and 40°C) allow to implement Direct Liquid Cooling (DLC) solutions as for the Aion supercomputer in two dedicated server rooms.
| Location | Cooling | Usage | Max Capa. |
|---|---|---|---|
| CDC S-02-001 | Airflow | Future extension | 280 kW |
| CDC S-02-002 | Airflow | Future extension | 280 kW |
| CDC S-02-003 | DLC | Future extension - High Density/Energy efficient HPC | 1050 kW |
| CDC S-02-004 | DLC | High Density/Energy efficient HPC: aion |
1050 kW |
| CDC S-02-005 | Airflow | Storage / Traditional HPC: iris and common equipment |
300 kW |
Data-Center Cooling technologies¶
Airflow with In-Row cooling¶
Most server rooms are designed for traditional airflow-based cooling and implement hot or cold aisle containment, as well as In-row cooling systems work within a row of standard server rack engineered to take up the smallest footprint and offer high-density cooling. Ducting and baffles ensure that the cooling air gets where it needs to go.
Iris compute, storage and interconnect equipment are hosted in such a configuration
[Direct] Liquid Cooling¶
Traditional solutions implemented in most data centers use air as a medium to remove the heat from the servers and computing equipment and are not well suited to cutting-edge high-density HPC environments due to the limited thermal capacity of air. Liquids’ thermal conductivity is higher than the air, thus concludes the liquid can absorb (through conductivity) more heat than the air. The replacement of air with a liquid cooling medium allows to drastically improve the energy-efficiency as well as the density of the implemented solution, especially with Direct Liquid Cooling (DLC) where the heat from the IT components is directly transferred to a liquid cooling medium through liquid-cooled plates.
The Aion supercomputer based on the fan-less Atos BullSequana XH2000 DLC cell design relies on this water-cooled configuration.