ABSTRACT
An important aspect to the performance of an application on an HPC system is how efficiently the work it performs can be done by independent tasks that may communicate and/or share data. The ability to apply multiple tasks to a computation is known as computational parallelism. Parallelism is implemented by the programmer at the software level but originates from parallelism in the hardware. This chapter will first cover where parallelism arises in the hardware of HPC systems, followed by how individual applications interact with the hardware-based parallelism through software and software libraries such as
CONTENTS
11.1 Hardware-Based Parallelism in HPC Systems .............................................................. 146 11.1.1 Instruction-Level Parallelism ............................................................................... 147 11.1.2 Core-Level Parallelism .......................................................................................... 147 11.1.3 Processor-Level Parallelism ................................................................................. 147 11.1.4 Node-Level Parallelism ......................................................................................... 149
11.2 Software-Based Parallelism in HPC Applications ........................................................ 150 11.2.1 Memory, Processes, and Threads ........................................................................ 150 11.2.2 Distributed Memory Programming and MPI ................................................... 150 11.2.3 Shared Memory Programming and OpenMP .................................................. 151
11.3 Batch System-Based Parallelism in HPC Systems ......................................................... 151 11.3.1 Job Schedulers ........................................................................................................ 152 11.3.2 Job Priority .............................................................................................................. 152 11.3.3 Additional Scheduling Methods ......................................................................... 153 11.3.4 Job Submission Scripts .......................................................................................... 153
Message Passing Interface (MPI) and OpenMP, and finally discuss how an HPC system’s users work in parallel through the use of a batch system.
