ABSTRACT
There are two conventional strategies for writing data from applications that are run on supercomputers: a file-per-process, or a single-shared-file for all processes. However, neither of these processes scale well for large-scale applications. At the application level, it is very hard to implement a flexible I/O strategy that works well on all supercomputers. Application developers cannot be tasked with waiting and maintaining custom I/O middleware. In designing ADIOS, the primary goal was to create an I/O framework that enables scientists to create their data-driven scientific processes, and write their
programming interface. Using ADIOS takes away the burden of implementing the I/O strategy at the application level, and chooses the best-performing I/O strategy depending on the parameters of an actual run and the target storage system. Each process in an application needs only to declare what data should be moved and when, but the “how,” i.e., the I/O strategy (e.g., aggregation and output file organization), is not implemented at this level. ADIOS takes care of the data movement using an I/O strategy chosen separately.
