From Designing and Building Parallel Programs 3: MPI Message Passing System DoD Modernization Tutorial -- 1995-1998. byIan Foster, Gina Goff, Ehtesham Hayder, Chuck Koelbel
Provides standard interfaces to common global operations
Communications: broadcast
Computation: summation
A collective operation uses a process group
All processes in group call same operation at (roughly) the same time