Full HTML for

Scripted foilset CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF

Given by Geoffrey C. Fox at Delivered Lectures of CPS615 Basic Simulation Track for Computational Science on 26 September 96. Foils prepared 29 December 1996
Outside Index Summary of Material Secs 48.9

This quickly completes the discussion of problem architecture but rather than continuing qualitative discussion of HPF applications in notes
Jumped to a discussion of HPF language describing
Basic Approach to Parallelism with "owner-computes" rule
Types of new constructs with
TEMPLATE ALIGN and PROCESSORS described
The lecture started with a description of the Web based Programming Laboratory developed by Kivanc Dincer

Table of Contents for full HTML of CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF

 Denote Foils where Image Critical
Denote Foils where HTML is sufficient
Indicates Available audio which is lightpurpleed out if missing
1 Delivered Lectures for CPS615 -- Base Course for the Simulation Track of Computational Science
Fall Semester 1996 --
Lecture of September 26 - 1996
2 Abstract of Sept 26 1996 CPS615 Lecture
3 The map of Problem ---> Computer is performed in two or more statges
4 The Mapping of Space of Problem Architectures onto Space of Machine Architectures
5 What determines when Parallelism is Clear ?
6 Evaluation of High Performance Fortran What applications need what features of HPF and its extensions ?
7 Parallelism in HPF
8 Fortran77 is part of Fortran90
9 HPF Features
10 What gives high performance in HPF
11 Compiler directives used in HPF
12 What does an HPF Compiler do?
13 Syntax of HPF Directives
14 Data Mapping in HPF
15 Staged Data Mapping in HPF
16 Template in HPF
17 Abstract Processors in HPF
18 Example of Template and Processors
19 Align Directive in HPF
20 Examples of Align Directive

Outside Index Summary of Material

HTML version of Scripted Foils prepared 29 December 1996

Foil 1 Delivered Lectures for CPS615 -- Base Course for the Simulation Track of Computational Science
Fall Semester 1996 --
Lecture of September 26 - 1996

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 1144.8
Geoffrey Fox
NPAC
Room 3-131 CST
111 College Place
Syracuse NY 13244-4100
HTML version of Scripted Foils prepared 29 December 1996

Foil 2 Abstract of Sept 26 1996 CPS615 Lecture

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 48.9
This quickly completes the discussion of problem architecture but rather than continuing qualitative discussion of HPF applications in notes
Jumped to a discussion of HPF language describing
Basic Approach to Parallelism with "owner-computes" rule
Types of new constructs with
TEMPLATE ALIGN and PROCESSORS described
The lecture started with a description of the Web based Programming Laboratory developed by Kivanc Dincer
HTML version of Scripted Foils prepared 29 December 1996

Foil 3 The map of Problem ---> Computer is performed in two or more statges

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 53.2
Architecture of "virtual problem" determines nature of language
HTML version of Scripted Foils prepared 29 December 1996

Foil 4 The Mapping of Space of Problem Architectures onto Space of Machine Architectures

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 93.6
HTML version of Scripted Foils prepared 29 December 1996

Foil 5 What determines when Parallelism is Clear ?

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 102.2
Key idea behind data-parallel languages - and perhaps all good languages
The language expresses problem and not the machine architecture
Need different approach to express functional parallelism
Use"object-oriented" feature when problems has natural objects
Do not use "object-oriented" features when objects are artifacts of target machine
Need data and functional (object) parallel paradigms in many problems - especially multidisciplinary
HTML version of Scripted Foils prepared 29 December 1996

Foil 6 Evaluation of High Performance Fortran What applications need what features of HPF and its extensions ?

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 28.8
See NPAC's High Performance Fortran Applications Resource
HTML version of Scripted Foils prepared 29 December 1996

Foil 7 Parallelism in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 110.8
Parallelism in HPF is expressed explicitly
  • Fortran 90 array expressions and assignments (including WHERE)
  • HPF Library and Array intrinsics
  • FORALL statement and construct
    • PURE labels procedures that can be used in FORALL as they have no "side-effects"
  • INDEPENDENT assertion on DO loops
Compiler may choose not to exploit information about parallelism
Compiler may detect parallelism in sequential code
HTML version of Scripted Foils prepared 29 December 1996

Foil 8 Fortran77 is part of Fortran90

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 144
A=B or more interestingly
WHERE( B > 0. ) A = B
ELSEWHERE A=0.
END WHERE
can be written
DO I = n1,n2
DO J = m1,m2
IF(B(I,J) >0.) THEN A(I,J) = B(I,J)
ELSE A(i,J) = 0.
END IF
END DO
END DO
Now a good HPF compiler will recognize the DO loops can be parallelized and give the same answer for Fortran90 and Fortran77 forms but often the detection of parallelism is not clear
Note FORALL is guaranteed to be parallelizeable as by definition no side effects.
HTML version of Scripted Foils prepared 29 December 1996

Foil 9 HPF Features

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 201.6
All of Fortran90
New instructions FORALL and INDEPENDENT enhancing DO loops
Data Alignment and Distribution Assertions
Miscellaneous Support Operations but
NO parallel Input/Output
Little Support for Irregular Computations
Little Support for any form of non mainstream data-parallelism
Extrinsics as supporting links with explicit message-passing
HTML version of Scripted Foils prepared 29 December 1996

Foil 10 What gives high performance in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 318.2
There is tradeoff between parallelism and communication
Programmer defines the data mapping and compiler uses this to assign processing
Underlying assumptions are that:
An operation on two or more data object is likely to be carried out much faster if they all reside in the same processor,
And that it may be possible to carry out many such operations concurrently if they can be performed on different processors
This is embodied in "owner computes" rule -- namely that in for instance
  • A(i,j)= .....
  • One brings everything on right hand side to process "owning" A(i,j) and performs computation in this processor
Owner computes algorithm is usually good and often best
HTML version of Scripted Foils prepared 29 December 1996

Foil 11 Compiler directives used in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 97.9
The directives are structured comments that suggest implementation strategies or assert facts about a program to the compiler
They may affect the efficiency of the computation performed, but do not change the value computed by the program
As in Fortran 90 statements, there are both:
  • declarative directives
  • executable directives
HTML version of Scripted Foils prepared 29 December 1996

Foil 12 What does an HPF Compiler do?

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 344.1
It must generate Fortran77(90) + Message Passing code or possibly in one pass map HPF code onto parallel machine code
Traditional dataflow and dependency analysis is especially critical in Fortran77 parts of code
It must use data mapping assertions to decide what is stored where and so organize computation
Code must be transformed to respect this owner-computes model
It must typically use "Loosely Synchronous" model with communicate-compute phases and then compiler generates all the communication needed
  • Due to latency issues compiler must minimize communication needed and maximize size of packets sent
We need an excellent run-time library which the compiler invokes with parallel Intrinsics etc.
HTML version of Scripted Foils prepared 29 December 1996

Foil 13 Syntax of HPF Directives

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 66.2
HPF directives are consistent with Fortran 90 syntax except for the special prefix for directive:
  • !HPF$
  • CHPF$
  • *HPF$
Two forms of the directives are allowed
  • Specification statements, such as
  • !HPF DISTRIBUTE MYTEMPLATE(BLOCK) ONTO P
  • Equivalent Attributed form, such as,
  • !HPF DISTRIBUTE (BLOCK) ONTO P :: MYTEMPLATE
HTML version of Scripted Foils prepared 29 December 1996

Foil 14 Data Mapping in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 223.2
Data Mapping in HPF is all you need to do to get parallelism as long as you use the explicit array type syntax such as A=B+C
The Owner Computes rule implies that specifying location of variables specifies (optimally or not) parallel execution!
The new HPF-2 ON HOME directive is exception to this rule as specifies where a particular statement is to be executed
(RE)DISTRIBUTE tells you where data is to be placed
(RE)ALIGN tells you how different data structures are to be placed relative to each other
HTML version of Scripted Foils prepared 29 December 1996

Foil 15 Staged Data Mapping in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 60.4
HTML version of Scripted Foils prepared 29 December 1996

Foil 16 Template in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 257.7
A template is an abstract space of indexed positions (an "array of nothings")
In CMFortran terminology, Template is set of Virtual Processors -- one per data point
  • Template typically specifies precisely the full natural data parallelism that is natural for problem and HPF maps this problem parallelism onto particular machine
A template is declared by the TEMPLATE directive that specifies:
  • name of the template
  • the rank (i.e., number of dimensions)
  • the extent in each dimension
Examples:
  • CHPF$ TEMPLATE T(1000)
  • !HPF$ TEMPLATE FRED(N, 2*N)
  • *HPF$ TEMPLATE, DIMENSION(5,100,50) :: MINE, YOURS
HTML version of Scripted Foils prepared 29 December 1996

Foil 17 Abstract Processors in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 174.2
Abstract processors always form a rectilinear grid in 1 or more dimensions
They are abstract coarse grain collections of data-points
  • Remember efficiency says we must "block" communication into large chunks -- processors give us a general target for this
The processor arrangement is defined by the PROCESSORS directive that specifies:
  • name of the processor arrangement
  • the rank (i.e., number of dimensions)
  • the extend in each dimension
Examples:
  • !HPF$ PROCESSORS P(N)
  • *HPF$ PROCESSORS BIZARRO(1972:1997,-20:17)
  • CHPF$ PROCESSORS SCALARPROC (by default sequential)
HTML version of Scripted Foils prepared 29 December 1996

Foil 18 Example of Template and Processors

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 511.2
!HPF$ PROCESSORS P(4)
!HPF$ TEMPLATE X(40)
!HPF$ ALIGN WITH X :: A, B, C
!HPF$ DISTRIBUTE X(BLOCK)
  • ...
  • C = A + B
  • ...
HTML version of Scripted Foils prepared 29 December 1996

Foil 19 Align Directive in HPF

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 144
Syntax of Align:
!HPF ALIGN alignee WITH align-target
  • where -- note [..] implies optional component
  • alignee: alignee [(align-source-list)]
  • align target: align-target[(align-subscript-list)]
Alternatively
*HPF ALIGN (align-source-list) WITH align-target :: alignee
HTML version of Scripted Foils prepared 29 December 1996

Foil 20 Examples of Align Directive

From CPS615-Introduction to Virtual Programming Lab -- Problem Architecture Continued and Start of Real HPF Delivered Lectures of CPS615 Basic Simulation Track for Computational Science -- 26 September 96. *
Full HTML Index Secs 256.3
Note a colon(:) in directive denotes all values of array index
Examples of array indices:
  • CHPF$ ALIGN A(i) WITH B(i)
  • *HPF$ ALIGN (i,j) WITH TEMPL(i,j) :: A, B
Use of : examples:
  • !HPF$ ALIGN A(:) WITH B(:)
  • CHPF$ align (:,:) WITH TEMPL(:,:) :: A, B
© Northeast Parallel Architectures Center, Syracuse University, npac@npac.syr.edu

If you have any comments about this server, send e-mail to webmaster@npac.syr.edu.

Page produced by wwwfoil on Fri Aug 15 1997