Parallel Computer Architectures
A basic understanding of parallel processing hardware is required to design and implement parallel software that gets the best performance out of a computer and to choose a suitable hardware platform for running a particular application. In this chapter, we will first introduce various computer architectures, using Flynn’s taxonomy. We will then discuss the two main components defining a parallel computer, namely, the nodes and the network connecting the nodes, where a node is a set of processors grouped together and sharing memory and other resources. The discussion of parallel network architecture will compare the properties of various topologies for the networks connecting the nodes. Finally, we will provide a detailed discussion of the multipleinstruction, multiple-data (MIMD) architecture, which has emerged as the dominant architecture for large-scale computers, covering MIMD memory organization, reliability, and impact on parallel program design.