ABSTRACT
In response to the increasing volume and complexity of scientific data, many projects and activities have taken place over the last decade. In this book we have addressed these issues in depth, by describing existing technologies, and how they were used successfully in real example applications. Specifically, we addressed the state of the art of storage technologies, techniques for achieving efficient I/O, and standards for managing storage systems, and large-scale data movement. We also covered methods of efficient in-memory data movement, workflow technology to achieve real-time monitoring, and metadata and provenance management techniques to collect the history of runs. Other topics that were discussed in detail include optimization methods for managing streaming data from sensors and satellites, the integration of diverse geo-science datasets, efficient methods for data searching, analysis and visualization for data explorations, and new types of scientific data management systems.
