ABSTRACT
Lawrence Livermore National Laboratory, Oak Ridge National Laboratory, and Swiss Center for Scientific Computing
Scott Klasky, Norbert Podhorszki
Oak Ridge National Laboratory
Karsten Schwan, Matthew Wolf
Georgia Institute of Technology, Atlanta
Manish Parashar
Rutgers University
Fan Zhang
Rutgers University
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 9.2 Tailored Co-Processing at High Concurrency . . . . . . . . . . . . . . . . . . . 174 9.3 Co-Processing With General Visualization Tools Via Adaptors 175
9.3.1 Adaptor Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 9.3.2 High Level Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . 178 9.3.3 In Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 9.3.4 Co-Processing Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
9.4 Concurrent Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 9.4.1 Service Oriented Architecture for Data Management in
HPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 9.4.2 The ADaptable I/O System, ADIOS . . . . . . . . . . . . . . . . . . . . 184 9.4.3 Data Staging for In Situ Processing . . . . . . . . . . . . . . . . . . . . 185 9.4.4 Exploratory Visualization with VisIt and Paraview
Using ADIOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
. . . . . . . . . . . . . . . . . . . . . 9.6 Data Exploration and In Situ Processing . . . . . . . . . . . . . . . . . . . . . . . 189
9.6.1 In Situ Visualization by Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . 190 9.6.2 In Situ Data Triage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Traditionally, visualization is done via post-processing: a simulation produces data, writes that data to disk, and then, later, a separate visualization program reads the data from disk and operates on it. In situ processing refers to a different approach: the data is processed while it is being produced by the simulation, allowing visualization to occur without involving disk storage. As recent supercomputing trends have simulations producing data at a much faster rate than I/O bandwidth, in situ processing will likely play a bigger and bigger role in visualizing data sets on the world’s largest machines.
