-
Two theories of parallel execution:
-
łTwo heads are better than one.˛
-
łToo many cooks spoil the soup.˛
-
Both theories can happen in practice.
-
Keeping all the processors busy and accessing local memory means a good collaboration.
-
Synchronization and accessing nonlocal memory mean the chefs are fighting over the pots.
-
Generalities about modern parallel processors:
-
Individual processors are reasonably fast
-
Local memory access costs one to ten cycles
-
Nonlocal access costs tens to hundreds of cycles
-
Synchronization costs hundreds to thousands of cycles
|
Parallel computers allow several CPUs to contribute to a computation simultaneously.
|
For our purposes, a parallel computer has three types of parts:
-
Processors n
-
Memory modules n
-
Communication / synchronization network n
|
Key points:
-
All processors must be busy for peak speed.
-
Local memory is directly connected to each processor.
-
Accessing local memory is much faster than other memory.
-
Synchronization is expensive, but necessary for correctness.
|
