Parallel Computers -- Classic Overview
Parallel computers allow several CPUs to contribute to a computation simultaneously and different architectures take different steps to enable this coordinated action.
For our purposes, a parallel computer has three types of parts:
- Processors
- Memory modules
- Communication / synchronization network
Some Key points: Easier in Some Architectures/Applications than others!
- All processors must be busy for peak speed.
- Local memory is directly connected to each processor.
- Accessing local memory is much faster than other memory.
- Synchronization is expensive, but necessary for correctness.
Colors Used in Following pictures