Karlsruhe - SCC has started pilot operation of the new supercomputer "Hochleistungsrechner Karlsruhe" (High Performance Computer Karlsruhe), or “HoreKa” for short. While HoreKa will be ramped up to its full capacity in the coming weeks, this also marks the beginning of the shutdown of its predecessor, ForHLR II, after five years of successful operation.
When KIT's “Forschungshochleistungsrechner II” (High Performance Research Computer II, short ForHLR II) was commissioned in March 2016, it was one of the few computers in the world that could reach a computing power of more than one PetaFLOPS - i.e. one quadrillion computing operations per second. More than 1150 compute nodes with almost 24,000 CPU cores and 74 terabytes of main memory in total were required to achieve this level of performance.
KIT was not only at the forefront in terms of computing power with ForHLR II, though: The system was not cooled with cold water, but used “hot” water at temperatures of up to 45 degrees Celsius. This was a novelty in the field of High-Performance Computing and also the reason for the construction of a new data center building for the supercomputer at KIT's Campus North. The SCC was awarded with the German Data Center Award in 2017 for the energy-efficient overall concept.
Three years of preparations for HoreKa
The planning for a successor system to ForHLR II already started in 2018. In 2019, the project was named "High Performance Computer Karlsruhe" - HoreKa for short - and the procurement process was started. But HoreKa will not just be a worthy successor to ForHLR II. With 769 compute nodes, nearly 60,000 CPU cores, more than 220 terabytes of main memory and 668 GPUs, the system will achieve a theoretical peak performance of more than 17 petaFLOPS, making it 17 times faster than its predecessor. The system is thus expected to be among the ten fastest computers in Europe in mid-2021.
Thanks to the new supercomputer, researchers will be able to gain a more detailed understanding of highly complex natural and technical processes, particularly in materials sciences, earth system sciences, energy and mobility research in engineering, and particle- and astroparticle physics. Of course HoreKa can also be used by scientists conducting research to understand the SARS-CoV-2 virus, helping to combat the COVID-19 disease.
Computing and storage go hand in hand
A key consideration in the design of the system were the enormous amounts of data created by scientific research projects. Depending on the application, several hundreds of terabytes can be generated from a single simulation. To keep up with the growing needs, HoreKa's compute nodes, InfiniBand network and parallel file systems each deliver up to four times the throughput of the previous ForHLR II system.
A Multi-level data storage concept also guarantees fast data processing on external storage systems. HoreKa is connected with speeds of up to 45 gigabytes per second to the "Large Scale Data Facility" (LSDF) at SCC, which has been providing a modern infrastructure for the storage, management, archiving and analysis of research data since 2010.
Full operation of HoreKa begins on June 1, 2021
HoreKa was built up alongside ForHLR II over the past months. The first user groups have already been granted access to be able to port and optimize their applications. Over the next few weeks, the system will be ramped up to its full capabilities and pilot operation is expected to seamlessly transition into full operation. Starting on June 1, 2021, HoreKa will then be available to scientists from all over Germany. Applications for computing time can already be submitted right now.
Since the data center building at Campus North cannot supply both systems at the same time, ForHLR II has to be gradually shut down in parallel. By mid-April, the system will no longer be available. The record after five years of operation is extremely positive: almost one billion CPU hours of computing time have been provided to more than 140 different research projects.
More Information about HoreKa and how to file applications for computing time on the new system: www.nhr.kit.edu/userdocs/horeka/
More information on COVID-19 research at KIT:
- www.kit.edu/kit/corona-pandemie-forschung-und-hilfsaktivitaeten-am-kit.php (in German)
- www.scc.kit.edu/en/aboutus/13531
With bwUniCluster 2.0, KIT operates a second supercomputer as a state service: www.scc.kit.edu/en/dienste/bwUniCluster_2.0
Jennifer Buchmüller
Simon Raffeiner