Full HTML for

Scripted foilset CPS615-Align and Distribute in HPF

Given by Geoffrey C. Fox at Delivered Lectures of CPS615 Basic Simulation Track for Computational Science on 1 October 96. Foils prepared 27 December 1996
Outside Index Summary of Material Secs 64.8

This continues the discussion of HPF in the area of distribution and ALIGN statements.
The discussion of ALIGN should be improved as audio makes dubious statements about "broadcasting" information.
The distribution discussion includes a reasonable descriuption of block and cyclic and when you should use them.

Table of Contents for full HTML of CPS615-Align and Distribute in HPF

 Denote Foils where Image Critical
Denote Foils where HTML is sufficient
Indicates Available audio which is lightpurpleed out if missing
1 Delivered Lectures for CPS615 -- Base Course for the Simulation Track of Computational Science
Fall Semester 1996 --
Lecture of October 1 - 1996
2 Abstract of Oct 1 1996 CPS615 Lecture
3 Changing Rank in Align Directive
4 Replication in Align Directive
5 General Alignments in HPF
6 Formal Definition of Align Directive
7 More obscure Complicated Examples of Align Directive
8 Distribution Directive in HPF
9 Basic Examples of Distribute Directive
10 Two Dimensional Example of Distribute Directive
11 The Two Basic Distributions in HPF
12 The Example of Matrix Inversion
13 Example of Graphics Rendering
14 Example of Distribute Directive with Complex Alignment
15 Dynamic Data Mapping
16 Advanced Mapping Directives -- ReDistribution and ReAlign
17 Advanced Mapping Directives -- Allocatable arrays and pointers
18 Subprograms in HPF
19 Passing Distributed Arrays as Subprogram Arguments in HPF
20 Mapping Options for Dummy (Subroutine) Arguments
21 Inherit Distribution Directive in HPF
22 Summary of Mapping Directives in HPF
23 Fundamental Parallelism Assumption in HPF
24 Parallel statements and Constructs in HPF
25 Parallelism in Fortran 90 array assignments
26 WHERE (masked array assignment) in HPF

Outside Index Summary of Material

HTML version of Scripted Foils prepared 27 December 1996

Foil 1 Delivered Lectures for CPS615 -- Base Course for the Simulation Track of Computational Science
Fall Semester 1996 --
Lecture of October 1 - 1996

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index
Geoffrey Fox
NPAC
Room 3-131 CST
111 College Place
Syracuse NY 13244-4100
HTML version of Scripted Foils prepared 27 December 1996

Foil 2 Abstract of Oct 1 1996 CPS615 Lecture

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 64.8
This continues the discussion of HPF in the area of distribution and ALIGN statements.
The discussion of ALIGN should be improved as audio makes dubious statements about "broadcasting" information.
The distribution discussion includes a reasonable descriuption of block and cyclic and when you should use them.
HTML version of Scripted Foils prepared 27 December 1996

Foil 3 Changing Rank in Align Directive

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 129.6
Ranks of the alignee and the align-target may be different
Examples:
  • !HPF$ ALIGN A(:,j) WITH B(:)
  • CHPF$ ALIGN A(:,*) WITH B(:)
HTML version of Scripted Foils prepared 27 December 1996

Foil 4 Replication in Align Directive

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 308.1
... or other way round
  • !HPF$ ALIGN A(:) WITH TEMPL(:,*)
while this only puts A on some parts of template...
!HPF$ ALIGN A(:) WITH TEMPL(:,i)
HTML version of Scripted Foils prepared 27 December 1996

Foil 5 General Alignments in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 110.8
HPF allows for more general alignments such as:
  • REAL, ARRAY(5,8) :: A,B
!HPF$ TEMPLATE T(12,12)
!HPF$ ALIGN A(:,J) WITH T(:,J+1)
!HPF$ ALIGN B(I,J) WITH T(I+4,J+4)
But nobody is clear if they are useful!
HTML version of Scripted Foils prepared 27 December 1996

Foil 6 Formal Definition of Align Directive

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 53.2
Each align-dummy variable is considered to range over all valid index values for the corresponding dimension of the alignee. An align-subscript is evaluated for any specific combination of values for the align-dummy variables simply by evaluating each align-subscript as a expression. Their resulting subscript values must be legitimate subscripts for the align-target
HTML version of Scripted Foils prepared 27 December 1996

Foil 7 More obscure Complicated Examples of Align Directive

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 152.6
These examples have non-unit stride as perhaps in "red-black" Iterative Solver algorithms:
HTML version of Scripted Foils prepared 27 December 1996

Foil 8 Distribution Directive in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 185.7
Syntax:
!HPF$ DISTRIBUTE distributee (dist-format)
[ONTO dist-target]
Allowed forms of dist-format:
  • *
  • BLOCK
  • CYCLIC
  • BLOCK(int-expr)
  • CYCLIC(int-expr)
Examples:
  • CHPF$ DISTRIBUTE TEMP(BLOCK,CYCLIC)
  • !HPF$ DISTRIBUTE FRED(BLOCK(10)) ONTO P
  • *HPF$ DISTRIBUTE (BLOCK,*) :: MYTEMPLATE
HTML version of Scripted Foils prepared 27 December 1996

Foil 9 Basic Examples of Distribute Directive

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 362.8
!HPF$ PROCESSORS P(4)
  • REAL, ARRAY(16) :: A
!HPF$ TEMPLATE T(16)
!HPF$ ALIGN A(:) WITH T(:)
HTML version of Scripted Foils prepared 27 December 1996

Foil 10 Two Dimensional Example of Distribute Directive

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 218.8
*HPF PROCESSORS SQUARE(2,2)
*HPF TEMPLATE T(4,4)
*HPF ALIGN A(:,:) WITH T(:,:)
*HPF DISTRIBUTE T(BLOCK,CYCLIC)ONTO SQUARE
HTML version of Scripted Foils prepared 27 December 1996

Foil 11 The Two Basic Distributions in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 282.2
We used BLOCK in the Laplace equation example and so this is appropriate distribution for "local" or geometric type problems
CYCLIC is called scattered in our early work (or is a special case of scattered which is perhaps random distribution of objects on processors) is appropriate in cases where "load-balancing" is more important than locality
  • Simplest examples are matrix inversion and graphics rendering problems
  • In solving equations (we will do later) Ax=b , there is no "nearest neighbor" structure between rows and columns, but rather one eliminates rows and columns and cyclic distribution ensures work remains balanced
  • In calculating pixels, work depends on complexity of picture at that pixel and so best to distribute pixels cyclically (or randomly) to processors.
HTML version of Scripted Foils prepared 27 December 1996

Foil 12 The Example of Matrix Inversion

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 288
Matrix Inversion set up on two processors after
0 2 and 4 rows/columns eliminated
Note BLOCK decomposition leads to all work being on one processor at end even if starts off balanced
HTML version of Scripted Foils prepared 27 December 1996

Foil 13 Example of Graphics Rendering

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 646.5
Here we show a 16 by 16 array of pixels with either CYCLIC or 8 by 8 two dimensional BLOCK,BLOCK
HTML version of Scripted Foils prepared 27 December 1996

Foil 14 Example of Distribute Directive with Complex Alignment

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 28.8
CHPF$ PROCESSORS Q(4)
CHPF$ TEMPLATE FRED(16,16)
CHPF$ ALIGN A(:,:) WITH FRED(:,:)
CHPF$ ALIGN B(I,J) WITH FRED(I+2,J+2)
CHPF$ DISTRIBUTE FRED(BLOCK,*)
HTML version of Scripted Foils prepared 27 December 1996

Foil 15 Dynamic Data Mapping

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 211.6
One data mapping is often not appropriate for an entire program
  • Often one has phases in which different distributions are needed in different phases
  • e.g. in 2D FFT, one typically finds FFT of F(I,J) by first distributing so for each J all I (x values) are in same processor and then transform so that for each I all J are in same processor
  • This ensures no communication in FFT phases which is important as typically in distributed one dimensional FFT there is substantial overhead
ALLOCATABLE arrays can change size
REALIGN and REDISTRIBUTE are executable DISTRIBUTE and ALIGN commands but are only to be used if one declares arrays on which they act DYNAMIC
Naturally DYNAMIC arrays can be initialized by ALIGN or DISTRIBUTE statements
HTML version of Scripted Foils prepared 27 December 1996

Foil 16 Advanced Mapping Directives -- ReDistribution and ReAlign

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 76.3
This example illustrates remapping from one to two dimensional decomposition for A and changing B from alignment with columns to alignment with rows
  • REAL, ARRAY(64,64) :: A
  • REAL, ARRAY(64) :: B
!HPF$ PROCESSORS P(64)
!HPF$ PROCESSORS Q(8,8)
!HPF$ DYNAMIC :: A,B
!HPF$ ALIGN B(:) WITH A(:,*)
!HPF$ DISTRIBUTE A(*,BLOCK)ONTO P
  • ...
!HPF$ REALIGN B(:) WITH A(*,:)
  • ...
!HPF$ REDISTRIBUTE A(CYCLIC,CYCLIC) ONTO Q
  • ...
HTML version of Scripted Foils prepared 27 December 1996

Foil 17 Advanced Mapping Directives -- Allocatable arrays and pointers

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 152.6
!HPF$ PROCESSORS Q(64)
!HPF$ ALIGN B(I) WITH A(I+N)
!HPF$ DISTRIBUTE A(BLOCK(M))
!HPF$ DISTRIBUTE(BLOCK), DYNAMIC :: P
  • ...
  • ALLOCATE(A(128))
  • ALLOCATE(B(64))
  • ALLOCATE(P(1024))
  • ...
!HPF$ REDISTRIBUTE P(CYCLIC)
  • ...
  • RETURN
  • END
HTML version of Scripted Foils prepared 27 December 1996

Foil 18 Subprograms in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 126.7
Scope of any mapping directives is a single (sub)program unit
A template or distribution is not a first-class Fortran 90 object:
It cannot be passed as a subprogram argument and this creates significant complication!
HTML version of Scripted Foils prepared 27 December 1996

Foil 19 Passing Distributed Arrays as Subprogram Arguments in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 283.6
There are three typical cases:
Subroutine requires data to use a particular mapping determined by subroutine
  • Arguments must be remapped
Subroutine can use any mapping so actual argument should be passed and used with current mapping
  • Here we have two cases depending on whether programmer knows or not (and tells subroutine) what incoming distribution is
Sometimes we need to remap due to array sections being passed
Any remappings must be undone on return from subroutine
HTML version of Scripted Foils prepared 27 December 1996

Foil 20 Mapping Options for Dummy (Subroutine) Arguments

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 56.1
DISTRIBUTE
  • use * instead of dist-format or ONTO clause indicates that incoming distribution is acceptable i.e. leave data in place
  • * before dist-format or ONTO clause indicates that data should stay in place and asserts that distribution is what you claim
ALIGN
  • * instead of or before target has similar meanings to DISTRIBUTE
INHERIT
  • A new attribute allowing references back to the original full array and used when sections of array are passed
HTML version of Scripted Foils prepared 27 December 1996

Foil 21 Inherit Distribution Directive in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 181.4
(not a comprehensive discussion; just an example)
HTML version of Scripted Foils prepared 27 December 1996

Foil 22 Summary of Mapping Directives in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 36
PROCESSORS
TEMPLATE
ALIGN
DISTRIBUTE
INHERIT
DYNAMIC
REALIGN
REDISTRIBUTE
HTML version of Scripted Foils prepared 27 December 1996

Foil 23 Fundamental Parallelism Assumption in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 20.1
An operation on two or more data object is likely to be carried out much faster if they all reside in the same processor
  • i.e. minimize communication
it may be possible to carry out many such operations concurrently if they can be performed on different processors
  • data parallelism
HTML version of Scripted Foils prepared 27 December 1996

Foil 24 Parallel statements and Constructs in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 403.2
Parallel Statements
  • Fortran 90 array assignments
  • masked array assignments (WHERE)
  • FORALL statement
Parallel Constructs
  • WHERE and WHERE...ELSEWHERE construct
  • FORALL construct
  • INDEPENDENT DO
Intrinsic functions and the HPF library
Extrinsic functions
HTML version of Scripted Foils prepared 27 December 1996

Foil 25 Parallelism in Fortran 90 array assignments

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 53.2
This is as in CMFortran and Maspar MPFortran with example:
HTML version of Scripted Foils prepared 27 December 1996

Foil 26 WHERE (masked array assignment) in HPF

From CPS615-Align and Distribute in HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 1 October 96. *
Full HTML Index Secs 165.6
This is as in CMFortran and Maspar MPFortran with example:
  • WHERE (A .GT. 0) A = A - 100
Semantics of WHERE statement:
    • 1. evaluate mask (in parallel) and store as a temporary T1
    • 2. for each i that T1(i)=.TRUE. compute T2(i)=A(i) - 100
    • 3. for each i that T1(i)=.TRUE. assign A(i)=T2(1)
© Northeast Parallel Architectures Center, Syracuse University, npac@npac.syr.edu

If you have any comments about this server, send e-mail to webmaster@npac.syr.edu.

Page produced by wwwfoil on Fri Aug 15 1997