chapter  4
12 Pages

Multi-Threading

In the previous chapter we discussed the use of message-passing for introducing parallelism across nodes in a distributed memory parallel computer. Each node in a distributed memory computer has its own separate memory that can be accessed by the other nodes by means of message-passing. Within a given node, however, the memory is typically directly available to all of the processors on the node, and there are two principal ways to introduce parallelism: the program can execute as separate processes that each have their own independent virtual memory and share data via message-passing; alternatively, multiple threads can be used within a process. In this chapter we will discuss how to parallelize execution of a program by using multiple threads. We will outline some of the advantages and potential pitfalls of using multi-threading, compare the multi-threading and message-passing approaches, and discuss hybrid programming, which uses both multi-threading and message-passing. In Appendices B and C, the use of multi-threading will be illustrated using two different approaches, namely Pthreads and OpenMPTM.