ABSTRACT
With multi-core processors in virtually every segment of the computing market, mainstream programmers need to routinely write parallel software. While progress has been made with automatic parallelization (see Chapter 9), experience with parallel computing has shown that in most cases, programmers will need to create parallel software “by hand.” This transition from serial to parallel software is a huge challenge for the software industry [Hwu08]; perhaps the biggest it has ever faced. While parallel computing is new to mainstream programmers, it has a long his-
tory in scientific computing. In the 1980s, the scientific computing community was faced with a transition from vector supercomputers supported by vectorizing compilers to explicitly parallel massively parallel processors and clusters. Computer scientists responded by creating countless parallel programming languages, but these for the most part failed. Only a tiny fraction of programmers in the sciences took up
parallel programming; and those that did tended to use low-level message passing libraries [MPI] that emphasized portability and control over elegance. New high-level languageswere not enough to solve the parallel programming prob-
lem in the 1980s and 1990s, and there is no reason to believe it will be different this time as we help general purpose programmers adopt parallel programming. It is my hypothesis that the key to solving this problem is to focus on how experi-
enced programmers think about parallel programming. Experienced parallel programmers, once they understand how the concurrency in their problem supports parallel execution, can use any sufficiently complete parallel programming language to create parallel software. Therefore, to solve the parallel programming problem for mainstream programmers, we need to capture how expert parallel programmers think and provide that understanding to mainstream programmers. In other words, we do not need new languages or sophisticated tools (though they help); we need to help mainstream programmers learn to “think parallel.” Learning to reason about parallel algorithms is a daunting task. Fortunately, we
have a few decades of experience to draw on. All we need to do is distill this collective wisdom into a form that we can put down in writing and communicate to people new to parallel computing. We believe design patterns are an effective tool to help us accomplish this task.
