ABSTRACT
In Chapters 4 and 5 we discussed and developed parallel algorithms for various logical topologies, various data distributions, and various assumptions regarding the concurrency of communication and computation. But we made one assumption that was common to all our algorithms, namely, that the underlying computing platform is homogeneous. In this context homogeneity means that all processors have the same computing capabilities, and that all network links have the same physical characteristics. The homogeneous assumption is relevant as the most popular platforms for running distributed-memory applications today are commodity clusters, which are typically purchased and installed as homogeneous systems. Originally, the notion of heterogeneous parallel computing stemmed from the desire to build cheap parallel computers by using individual workstations already available in, say, an academic department. Such workstations are typically heterogeneous as they are purchased at different times, for different users, and possibly for different purposes. Such ad-hoc parallel computers were initially the main target of the PVM [58] message-passing library, which contributed to their popularity.
