Countries across the globe are racing toward the creation of the next supercomputer, envisioned as exascale machines that are capable of a million trillion calculations per second. With implications for nearly every field of science and technology, exascale computing has been described as the key to creating and managing a better future.
One key component of exascale computing is fast reading and writing of data using storage systems. In recent years, growing demands for data storage have driven High-Performance Computing (HPC) platforms to adopt a variety of hardware devices as viable storage solutions. However, data processing requirements have also led to more frequent failures, prompting the needs for elastic system reconfigurations and failure recoveries.
Cao’s JDRD team seeks to address these challenges by creating a software-defined storage layer that can provide flexible, reliable, and elastic storage services and capabilities based on the underlying hardware. Cao proposes that by controlling the way that the storage is configured and data objects are moved across storage tiers, this layer can achieve great improvements on access speed, reliability, and operating cost.
“To our knowledge, this proposal initiates the first systematic research on workflow-aware software defined storage layers for HPC platforms,” wrote Cao. To administrators, the project aims to provide them with tools to specify work-flow oriented configurations, so that they can easily provision the system resources. Dealing with similar concerns, the LDRD program, carried out by Cao’s collaborators at ORNL, addresses broader concepts and approaches, while the JDRD project led by Cao focuses on software and implementation.
Their first joint paper based on this study between UT and ORNL has been accepted into the International Conference for High Performance Computing, Networking, Storage and Analysis (SC) 2015, which is the flagship conference in the area of high performance computing. Currently, Cao’s team continues their close collaboration with ORNL, by arranging periodic meetings, collaborating on challenging problems, and carrying out influential research work.