During the Supercomputing 2024 conference in Atlanta (USA), the new annual IO500 list has been announced – ranking the current world’s fastest data storage systems. The PSNC data storage system for the Proxima HPC cluster is ranked at #10 on the 10 Node Production List.
During the Supercomputing 2024 conference in Atlanta (USA), the new annual IO500 list has been announced – ranking the current world’s fastest data storage systems. The PSNC data storage system for the Proxima HPC cluster charted at #10 on the 10 Node Production List. Previously, the PSNC storage system for the Altair HPC cluster charted in this category in June 2024. The new system is a custom-made configuration developed at PSNC by the Data Management Technology Department in the Data Processing Technology Division of the Poznań Supercomputing and Networking Center. It utilizes the infrastructure acquired and built as part of the National Data Storage (NDS) project.
As the demand for quick data access in AI, HPC and cloud systems, services, and applications grows rapidly, hardware manufacturers, storage system providers, network engineers and data center professionals are facing unprecedented challenges to achieve the highest ever I/O performance. Designing the hardware part of the storage system that is fast enough (currently it is typically based on flash memory) is not the only challenge that has to be faced, as ensuring a high-speed, low-latency network interconnect and applying scalable software to manage the data is equally important.
PSNC operates and manages cloud, HPC and AI infrastructure and large-scale data storage systems (20+ PFLOPS, 400+PB). The AI revolution, powered by computing infrastructure with GPU accelerators and large memory, is highly dependent on data. While the computing infrastructure is the engine – data, fed to processing units with relevant performance is the fuel needed for the revolution to go on.
Between 17 and 22 November 2024 Atlanta hosts Supercomputing – the biggest HPC conference in the world. I500 list with the fastest data storage systems is published annually during SC along with TOP500 and Green500. This year’s hot topic at the conference is building infrastructure for AI applications, services and systems. In this context PSNC’s Top 10 entry in IO500 confirms that the PSNC’s systems perfectly meet the needs of today’s AI, HPC and cloud applications and services.
The storage system for the Proxima HPC cluster is based on an open infrastructure of flash storage servers and highly efficient Infiniband network, as well as the open source Lustre filesystem.
The system was created at PSNC based on an original concept of specialists from the Data Management Technology Department in the Data Processing Technology Division of the Center. On the contrary, other Top 10 systems in the list including Intel DAOS and Weka are vendor-provided solutions. DDN Exascaler is a commercial version of Lustre, released as an integrated appliance.
PSNC’s IO500 entry this year confirms the Poznań Center’s strong position among data centers globally. It also shows PSNC’s competences in designing highly efficient data storage and computing systems. The infrastructure that enabled this success was acquired thanks to the National Data Storage and PRACE-LAB2 projects, co-financed by the European Regional Development Fund and the Polish government, including the Ministry of Development Funds and Regional Policy.
This year’s IO500 makes us at the PSNC really proud since our second system appeared at #12 of 10 Node Production List. Wrocław University of Science and Technology has two entries (#14 on 10 Node Production List and #24 on Production list) and ACK Cyfronet AGH has one (#12 on Production List). Both institutions were involved in PSNC-coordinated National Data Storage and PRACE-LAB2 projects; that created federated infrastructure located in ten computing and networking centers across Poland.
PSNC is clearly at the forefront of the AI revolution, which requires extremely fast data management. Highly efficient, large-scale and AI-oriented data infrastructure will be developed in the coming years in Poland parallelly to the development of the computing infrastructure including AI Factories.
Check the full IO 500 10 Node Production SC24 List:
