|
See overview discussion in Parallel Computing Works
|
|
Synchronous: Data Parallel Tightly coupled and software needs to exploit features of problem structure to get good performance. Comparatively easy as different data elements are essentially identical.
|
Loosely Synchronous:
-
As above but data elements are not identical. Still parallelizes due to macroscopic time synchronization.
|
Asynchronous:
-
Functional (or data) parallelism that is irregular in space and time. Often loosely coupled and so need not worry about optimal decompositions to minimize communication. Hard to parallelize (massively) unless ....
|
Embarrassingly parallel:
-
Essentially independent execution of disconnected components. (can involve reductions)
|
Metaproblems
-
Asynchronous collection of (loosely) synchronous components where these programs themselves can be parallelized
|
