High Performance Fortran

A major hindrance to the development of parallel computing has been the lack of portable, industry standard parallel languages. Currently, almost all parallel computer vendors provide their own proprietary parallel languages which are not portable even to machines of the same architecture, let alone between SIMD and MIMD, distributed or shared memory, parallel or vector architectures.

This problem is now being addressed by the High Performance Fortran Forum (HPFF), a group of over 40 organizations including universities, national laboratories, computer and software vendors, and major industrial supercomputer users. HPFF was created in early 1992 to discuss and define a set of extensions to Fortran called High Performance Fortran. The goal was to address the problems of writing portable code which would run efficiently on any high performance computer, including parallel computers of any architecture (SIMD or MIMD, shared or distributed memory), vector computers, and RISC workstations. Here `efficiently' means `comparable to a program hand-coded by an expert in the native language of a particular machine'.

HPF is designed to support data parallel programming. It is an extension of Fortran 90, which provides for array calculations and is therefore a natural starting point for a data parallel language. HPF attempts to deviate minimally from the Fortran 90 standard, while providing extensions which will enable compilers to provide good performance on a variety of parallel and vector architectures. It is derived from a synthesis of ideas developed in many data parallel languages, such as DAP Fortran, Fortran 8X, Connection Machine Fortran, MasPar Fortran, Fortran 90, Fortran D (Rice University), Fortran 90D (Syracuse University), Vienna Fortran, Yale extensions, and Digital's HPF. While HPF is motivated by data parallel languages for SIMD machines, it should also be applicable to MIMD and vector machines. [7, 14]

HPF has a number of new language features:

1. New Directives
   These directives suggest implementation and data distribution strategies to the compiler. They are structured so that a standard Fortran compiler will see them as comments and thus ignore them. The directives are consistent with Fortran 90, so that if HPF were to be adopted as part of a future Fortran standard, only the comment character and the directive prefix would have to be removed.
2. New Language Syntax
   These are extensions to Fortran 90 to better express parallelism in a program. They include the FORALL statement and some extra intrinsic functions.
3. Library Routines
   HPF will provide a standard interface to a library of efficient parallel implementations of useful routines, such as sorting and matrix calculations.
4. Extrinsic Procedures
   Since HPF is a high level, machine independent language, it cannot express certain operations very well (or at all). It therefore allows the use of extrinsic procedures, which can be defined outside the language, for example by using Fortran with message passing.
5. Language Restrictions
   Some restrictions on the use of sequence and storage allocation in Fortran are defined in order to be compatible with the data distribution features of HPF.
6. Parallel I/O
   Language facilities for handling parallel I/O are being investigated, but have not been included in the initial HPF language standard.

The strategy behind HPF is that the user writes in an SPMD (Single Program Multiple Data) data parallel style, with conceptually a single thread of control and globally accessible data. The program is annotated with assertions giving information about desired data locality and/or distribution. The compiler then generates code implementing data and work distribution.

• For SIMD computers the implementation is parallel code with communication optimized by compiler placement of data.
• For MIMD computers the implementation is a multi-threaded message passing code with local data and optimized send/receive communications.
• For vector computers the implementation is vectorized code optimized for the vector units.
• For RISC computers the implementation is pipelined superscalar code with compiler generated cache management.

As its name suggests, a major goal of High Performance Fortran is to have efficient compilers for all these machines. Obtaining parallelism solely through dependency analysis has not proven to be effective in general, so for all commands in HPF the dependencies are implied directly, enabling the compiler to generate more efficient code.

• Data Distribution Directives
• Parallel Statements
• Intrinsic Functions and Standard Library
• Compilers and Fortran 90D