I will draw some sketches as below. I hope you can understand them.


1. Model for computation and I/O



          parallel compute node (upper applications/client)

 _____		 ______			______		_______
|cn1 |           | cn2 |               | cn3 |          | cn4 |
|____|           |_____|               |_____|          |_____| 
      \                 \             /                 /
       \                 \           /                 /
    ________________________________________________________
    |          my  I/O base (consists of i/o nodes)         |
    |_______________________________________________________|
          /              /            \                \   
         /              /              \                \  
 ______		________ 		_______		_______
 |disk1|        | disk2 |               | disk3|        |disk4 |
 |_____|        |_______|               |______|        |______|


(Note: I just enumerate four compute nodes or disks arbitrarily )


2. Mapping global matrix data structures for scattered disks.

(1) Assume that I need to maintain a matrix A(4x4) as below. In fact, it 
can be matrix  A(mxn) 

A(4x4) =  [  a11   a12   a13   a14 ]
          [  a21   a22   a23   a24 ]
          [  a31   a32   a33   a34 ]
          [  a41   a42   a43   a44 ]


(2) Appearance of logic file (linear -- row-major for example)

 __________________________________________________________________
|  a11 a12 a13 a14 a21 a22 a23 a24 a31 a32 a33 a34 a41 a42 a43 a44 |
|__________________________________________________________________|


(3) Normal method of partitioning the above global data structures between 
local disks.

      disk1                disk2               disk3                disk4
 _________________   _________________   _________________    
_________________
| a11 a12 a13 a14 | | a21 a22 a23 a24 | | a31 a32 a33 a34 |  | a41 a42 a43 
a44 |
|_________________| |_________________| |_________________|  
|_________________|


(4) Potential disadvantages of above normal method (partition according to 
row-major/column-major)
  
  For instance, compute node (application) require the matrix factors in  
column, while the
  data are stored in row-major among scattered disks.

                            compute node

    ____		    _____		    _____	            ____
  _| a11|                  | a12 |                 | a13 |                 
|a14 |
 | |____|                  |_____|                 |_____|                 
|____|
 | | a21| -----            | a22 |                 | a23 |                 
|a24 |
 | |____|      \           |_____|                 |_____|                 
|____|
 | | a31| ------\----      | a32 |                 | a33 |                 
|a34 |
 | |____|        \   \     |_____|                 |_____|                 
|____|
 | | a41| --------\---\--- | a42 |                 | a43 |                 
|a44 |
 | |____|          \   \  \|_____|                 |_____|                 
|____|
 |                  \   \  \
 |                   \   \__\________________
 \                    \      \_______________\ _______________________
 _\________________   _\_________________    _\________________     
__\______________
| a11 a12 a13 a14  | | a21 a22 a23 a24   |  | a31 a32 a33 a34  |   | a41 
a42 a43 a44 |
|__________________| |___________________|  |__________________|   
|_________________|

                             disks


It if obvious that in order to meet clients requirements, we have to do 
4x4 = 16 (times)
transfering of data.

Of course, the above example is a extreme bad one.
However, I want to conceive a method which is eclective or proper in most 
circumstances.

I will state my method in the next mail soon.
 (5) Mapping global matrix data among disks in "BLOCKS".


Still, assume that I need to maintain a matrix A(4x4) as below. 

A(4x4) =  [  a11   a12   a13   a14 ]
          [  a21   a22   a23   a24 ]
          [  a31   a32   a33   a34 ]
          [  a41   a42   a43   a44 ]


I can divide it into blocks as follows:

                        
                        | 
          [  a11   a12  |   a13   a14 ]
          [  a21   a22  |   a23   a24 ]             block1  |  block2
                        |                                   | 
    ____________________|__________________      ___________|_____________
                        |                                   |  
          [  a31   a32  |   a33   a34 ]             block3  |  block4
          [  a41   a42  |   a43   a44 ]                     | 
                        | 



Appearance of logic file (linear -- according to "blocks")

         

   block1          block2          block3           block4 
 ___________________________________________________________________
|  a11 a12 a21 a22 a13 a14 a23 a24 a31 a32 a41 a42 a33 a34 a43 a44  |
|___________________________________________________________________|




"Blocks" method of partitioning the above global data structures between 
local disks.

      disk1                disk2               disk3                disk4
 _________________   _________________   _________________    
_________________
| a11 a12 a21 a22 | | a13 a14 a23 a24 | | a31 a32 a41 a42 |  | a33 a34 a43 
a44 |
|_________________| |_________________| |_________________|  
|_________________|


 
  For instance, compute node (application) require the matrix factors in  
column, while the
  data are stored in blocks among scattered disks.

                            compute node

    ____		    _____		    _____	            ____
   | a11|                  | a12 |                 | a13 |                 
|a14 |
  {|____|                 {|_____|                 |_____|                 
|____|
 /{| a21|                /{| a22 |                 | a23 |                 
|a24 |
 | |____|               |  |_____|                 |_____|                 
|____|
 | | a31|               |  | a32 |                 | a33 |                 
|a34 |
 | |____|}              |  |_____|                 |_____|                 
|____|
 | | a41|}              |  | a42 |                 | a43 |                 
|a44 |
 | |____|               |  |_____|                 |_____|                 
|____|
 |         _____________|
 |________/_    \                 
 ___/____/__\____\__  ___________________    __________________     
_________________
| a11 a12  a21  a22| | a13 a14 a23 a24   |  | a31 a32 a41 a42  |   | a33 
a34 a43 a44 |
|__________________| |___________________|  |__________________|   
|_________________|

                             disks


It is obvious that in order to meet clients requirements, we just have to 
do 4x2 = 8 (times)
transfering of data.

How can I give the formula prove of my thoughts to show that it is 
effective?


I have to leave now because the lab will be closed until 2:00 pm this 
afternoon.

Judy 
































 





(2) Appearance of logic file (linear -- row-major for example)

 __________________________________________________________________
|  a11 a12 a13 a14 a21 a22 a23 a24 a31 a32 a33 a34 a41 a42 a43 a44 |
|__________________________________________________________________|


(3) Normal method of partitioning the above global data structures between 
local disks.

      disk1                disk2               disk3                disk4
 _________________   _________________   _________________    
_________________
| a11 a12 a13 a14 | | a21 a22 a23 a24 | | a31 a32 a33 a34 |  | a41 a42 a43 
a44 |
|_________________| |_________________| |_________________|  
|_________________|


(4) Potential disadvantages of above normal method (partition according to 
row-major/column-major)
  
  For instance, compute node (application) require the matrix factors in  
column, while the
  data are stored in row-major among scattered disks.

                            compute node

    ____		    _____		    _____	            ____
  _| a11|                  | a12 |                 | a13 |                 
|a14 |
 | |____|                  |_____|                 |_____|                 
|____|
 | | a21| -----            | a22 |                 | a23 |                 
|a24 |
 | |____|      \           |_____|                 |_____|                 
|____|
 | | a31| ------\----      | a32 |                 | a33 |                 
|a34 |
 | |____|        \   \     |_____|                 |_____|                 
|____|
 | | a41| --------\---\--- | a42 |                 | a43 |                 
|a44 |
 | |____|          \   \  \|_____|                 |_____|                 
|____|
 |                  \   \  \
 |                   \   \__\________________
 \                    \      \_______________\ _______________________
 _\________________   _\_________________    _\________________     
__\______________
| a11 a12 a13 a14  | | a21 a22 a23 a24   |  | a31 a32 a33 a34  |   | a41 
a42 a43 a44 |
|__________________| |___________________|  |__________________|   
|_________________|

                             disks


It is obvious that in order to meet clients requirements, we have to do 
4x4 = 16 (times)
transfering of data.

Of course, the above example is an extremely bad one.
However, I want to conceive a method which is eclective or proper in most 
circumstances.































 








  





                            compute node

    ____		    _____		    _____	            ____
  _| a11|                  | a12 |                 | a13 |                 
|a14 |
 | |____|                  |_____|                 |_____|                 
|____|
 | | a21| -----            | a22 |                 | a23 |                 
|a24 |
 | |____|      \           |_____|                 |_____|                 
|____|
 | | a31| ------\----      | a32 |                 | a33 |                 
|a34 |
 | |____|        \   \     |_____|                 |_____|                 
|____|
 | | a41| --------\---\--- | a42 |                 | a43 |                 
|a44 |
 | |____|          \   \  \|_____|                 |_____|                 
|____|
 |                  \   \  \
 |                   \   \__\________________
 \                    \      \_______________\ _______________________
 _\________________   _\_________________    _\________________     
__\______________
| a11 a12 a13 a14  | | a21 a22 a23 a24   |  | a31 a32 a33 a34  |   | a41 
a42 a43 a44 |
|__________________| |___________________|  |__________________|   
|_________________|

                             disks


It is obvious that in order to meet clients requirements, we have to do 
4x4 = 16 (times)
transfering of data.

Of course, the above example is an extremely bad one.
However, I want to conceive a method which is eclective or proper in most 
circumstances.