ABSTRACT
In this chapter, we look at how to improve the scalability of applications. In some sense, this is the most dicult challenge for the programmer. ere are many dierent reasons why an application might not scale and some of those may be impossible to improve without a total rewrite of the application. In order to hit as many of the scenarios as possible, we look at numerous applications and several dierent MPI implementations and discuss the reasons why they scale and why they eventually reach their limits of scalability. What are the principal hazards to scalability? e most prevalent is the lack of sucient work. When a xed problem is scaled to a large number of processors, the application will sooner or later quit scaling. e individual processors will run out of work to do and communication will dominate. e rst code we will examine, one of the SPEC® MPI benchmark suites, is a case of trying to run a small problem on too many processors. Other hazards are less obvious and sometimes no matter how big a problem is, the application does not scale. For example:
1. Having a poor decomposition that limits the number of processors that can be eectively used.
