ABSTRACT
The broader technology industry has come to depend on the rapid, predictable, and cheap scaling of computing performance and storage density. For decades, exponentially increasing capability could be procured at roughly constant annual cost, and that expectation has permeated computing resource planning and decision making. For the past twenty years, we have become accustomed to a very steady technological progression where improvements in silicon lithography (Moore’s Law) have translated directly into improved computing and storage speed, density, and energy efficiency. All of our assumptions about how to program these systems implicitly assume the progression will continue unabated. In 2004, however, a confluence of events changed forever the architectural landscape that underpinned our current assumptions about what to optimize for when we design new algorithms and applications. Storage technologies must be refactored to handle massive parallelism, the severe constraints on the energy cost of data movement, and reduced reliability. Furthermore, this confluence of factors presents a more fundamental challenge to the traditional POSIX semantics that underpinned parallel file system design for the past few decades.
