Large Scale Data Facility (LSDF)
Future scientific experiments and simulations are expected to produce more and more data. The utilization of this data and information always requires, in addition to modern computing infrastructures for analysis and interpretation, storage systems that are up to the growing demands.
Since 2010, researchers of the Karlsruhe Institute of Technology (KIT), their partners in the state of Baden-Württemberg and researchers of the Helmholtz Association have the Large Scale Data Facility (LSDF) as a multi petabyte storage system at their disposal. The LSDF provides a central and reliable storage resource for large research data from different data-intensive scientific disciplines.
The project is sponsored and funded by the Ministry of Science, Research and the Arts of Baden-Württemberg (MWK), the German Research Foundation (DFG) and the Helmholtz Association of German Research Centres. It is operated by the Scientific Computing Center (SCC) at KIT.
The LSDF enables scientists to store and access their research data via different services. These services, tailored to the requirements of the researchers, are offered on top of the LSDF storage. Important features are in particular high storage capacity, fast access to and transfer of the data, easy data access and management capabilities, as well as the connection to modern processing infrastructures such as the high-performance computing clusters based at KIT like the bwUniCluster or the ForHLR II.
Access to the data is possible via standard protocols. Permissions to access the data are controlled by the data owners using group memberships and access control lists (ACLs). The LSDF currently offers 10PB of usable storage capacity with a combined maximum read/write speed of the storage system of 25GByte/s. Today it is used by about 300 scientists.
The infrastructure of the LSDF constitutes the basis for different services and projects such as the Helmholtz Data Federation (HDF), the B2SAFE service in EUDAT and the service LSDF Online Storage.
With the HDF, the Helmholtz Association has begun with the development of a federated research data infrastructure. The project is coordinated by the KIT. In the first phase, 6 partner centres from the 5 research areas Earth and Environment, Matter, Health, Key Technologies and Energy will be federated. The storage system of the LSDF will also be part of the HDF.
The service B2SAFE allows a policy-based, multi-site use of large research data repositories. The LSDF uses iRODS for the data management and replication, providing an abstraction layer on top of the the actual LSDF storage.
The service LSDF Online Storage enables KIT users and their partners in the state of Baden-Württemberg to access the storage resources of the LSDF. The service is used by institutes with research funded by the Helmholtz Association as well as insitutes at the university level of the KIT from different scientific disciplines. Users of the service include scientists from the Institute for Applied Materials (IAM), the Institute for Hydromechanics (IFH), the Institute for Reciprocating Engines (IFKM), the Institute for Astroparticle Physics (IAP), the Institute of Meteorology and Climate Research (IMK), the Institute for Photon Science and Synchrotron Radiation (IPS), the Institute of Toxicology and Genetics (ITG) and the SFB/TRR 150 "Turbulente, chemisch reagierende Mehrphasenströmungen in Wandnähe". Parts of the service are offered in cooperation with the computing center of the University of Heidelberg. A usage information can be found in the description of the service.
Contact: Dr. Serge Sushkov
