1 | All to All Communication with i'th location in j'th processor being sent to j'th location in i'th processor |
2 | Processor 0 1 2 3 |
3 | Start (a0,a1,a2,a3) (b0,b1,b2,b3) (c0,c1,c2,c3) (d0,d1,d2,d3) |
4 | After (a0,b0,c0,d0) (a1,b1,c1,d1) (a2,b2,c2,d2) (a3,b3,c3,d3 |
5 | There are extensions MPI_ALLTOALLV to handle case where data stored in noncontiguous fashion in each processor and when each processor sends different amounts of data to other processors |
6 | Many MPI routines have such "vector" extensions |