Table of Contents
Designing and Building Parallel Programs
Outline
Outline
Why Parallel Computing?
The Multicomputer: an Idealized Parallel Computer
Multicomputer Architecture
Multicomputer Cost Model
How do Real Parallel Computers Fit the Model?
Distributed Memory MIMD Multiprocessor
Shared Memory MIMD Multiprocessor
Distributed Shared Memory (DSM)
Workstation Clusters
A Simple Parallel Programming Model
Properties
Parallel Algorithm Design
A Design Methodology
Partitioning
Communication
Agglomeration
Mapping
Example: Atmosphere Model
Atmosphere Model: Numerical Methods
Atmosphere Model: Partition
Atmosphere Model: Communication
Atmosphere Model: Agglomeration
Atmosphere Model: Mapping
Modeling Performance
Bandwidth and Latency
Measured Costs
Typical Communication Costs
Example: Finite Difference
Time for Finite Difference
Using Performance Models
Design Alternatives: Finite Difference
Design Alternatives (2)
Finding Model Discrepancies
Impact of Network Topology
Competition for Bandwidth
Bandwidth-Constrained Model Versus. Observations
Tool Survey
High Performance Fortran (HPF)
HPF Example
HPF Analysis
Message Passing Interface (MPI)
MPI Example
MPI Analysis
PCF
PCF and OpenMP
PCF Example (SGI Variant)
OpenMP Example
PCF/OpenMP Analysis
Portable, Extensible Toolkit for Scientific Computations (PETSc)
PETSc Example
PETSc Analysis
|