Two Innovative Chip Technologies – One High-Performance System
HoreKa is an innovative hybrid system with almost 60.000 Intel Xeon "Ice Lake" Scalable Processor cores, 220 terabytes of main memory as well as 668 NVIDIA A100 Tensor Core GPUs. A non-blocking NVIDIA Mellanox InfiniBand HDR network with 200 GBit/s per port is used for communication between the nodes. Two Spectrum Scale parallel file systems offer a total storage capacity of more than 16 petabytes.
A central aspect in the design of the system have been the enormous amounts of data generated by scientific research projects. Depending on the application, several hundred terabytes of data can be generated by a single simulation. To keep up with the growing amounts of data, the computing nodes, the InfiniBand network, and the parallel file systems of HoreKa each provide up to four times the throughput of its predecessor system, ForHLR.
Computation and storage go hand in hand
A multi-level data storage concept also guarantees high-throughput processing of data on external storage systems. HoreKa is connected to the “Large Scale Data Facility” (LSDF) of the SCC, which has been providing a modern infrastructure for the storage, administration, archiving, and analysis of research data since 2010, with a data rate of up to 45 GByte/s.
HoreKa has been installed in a state-of-the-art data center constructed for its predecessor ForHLR on KIT‘s Campus North in 2015.