Digital Equipment Corportation External Research Proposal

Institution: Syracuse University, Syracuse, New York 13244-4100

Research Title: NPAC Proposal to Digital Equipment Corporation Applications and Portable Software for Massively Parallel Processing

Date: January 22, 1993

Center: Northeast Parallel Architectures Center

Contact: Geoffrey Fox, Kim Mills NPAC, 111 College Place Syracuse University, Syracuse NY 13244-4100 315-443-1723 (fax: 315-443-1973) gcf@npac.syr.edu, kim@npac.syr.edu

Principal Investigator: Geoffrey Fox, Director NPAC

Administration: Ann Hammersla, OSP

Digital Sponsor: Bob Nilsson (MPSG), Dave Loveman (HPF)

Digital Account Representative: Phil Gadsden

Total Funding Required: $1,375,000 Funding Requested from Digital: $1,375,000 Allowance towards the Purchase of Digital Products: $1,275,000 List Price of Digital Products: $1,700,000 Other Resources: $100,000

Abstract We propose to develop three new applications of Massively Parallel Processing, and to continue development of portable software approaches for High Performance Fortran. Applications that demonstrate the usefulness of MPP, with particular emphasis on the DECmpp 12000 system, will be jointly selected with Digital Equipment Corporation. Development of portable software approaches for High Performance Fortran will continue to concentrate on the relationship between applications and language constructs with the goal of understanding the specific language constructs required to solve application problems.

Term: April 1, 1993 - April 1, 1994, year 2 of three year project

Deliverables Applications of interest to Digital Equipment Corporation will be parallelized to run on the DECmpp 12000, with work on MIMD parallel systems to be included as needed. We will deliver parallel application codes, associated software demonstrations, technical reports, and presentation materials. Software tools and language constructs required for application development will be integrated into FortranD and proposed for High Performance Fortran.

NPAC Proposal to Digital Equipment Corporation Applications and Portable Software for Massively Parallel Processing January 22, 1993

Motivation

Digital’s support of this proposal with further NPAC’s mission of applying parallel computing technology to real world problems in science and industry. We ask for both applications support in the form of a cash grant, and an equipment grant that will primarily be used to upgrade our DECmpp to the largest, most recent model. While NPAC has a variety of state of the art MPP systems, all are modest in size. The DECmpp has proven to be well suited to a large number of application projects, and the FortranD/HPF development project. NPAC’s DECmpp will be made available to the Center for Research in Parallel Computation (CRPC) national consortium. NPAC is in a new phase of project development in our ACTION outreach program to apply MPP in industry. A variety of new application projects are underway, including contacts through Rita Foley of Digital. In Syracuse University’s computational science education program, we are using the DECmpp in the second major course in computational science (CSP 615). Students will implement a large variety of projects on the DECmpp, and NPAC will provide Digital with an evaluation of the use of the DECmpp in education. Linked to our research in FortranD/HPF development, we will apply HPF to Digital supported applications, and large scale grand challenge application problems on the DECmpp. Combining HPF and the DECmpp allows us the opportunity to evaluate this hardware/software for both performance and language capability.

Description of Research Projects

In year two, we will continue our work in applications research and development of Fortran90D/High Performance Fortran. We propose three new MPP applications. Digital researchers will continue to have access to NPAC computing facilities.

Applications

We will concentrate our efforts in applications research on the DECmpp 1200/Sx system, while using MIMD systems as necessary. As application problems, we propose the Center for Analysis and Prediction of Storms (CAPS) weather forecasting model from The University of Oklahoma, a proton radiation therapy model for cancer treatment from Loma Linda University Medical Center, and parallelization of a portion of MOPAC. The weather and radiation therapy applications represent collaborations with leading research centers that could benefit from MPP approaches. NPAC is actively working with both centers to develop a research plan and funding support. Digital’s support at this early stage would establish two promising applications of parallel computing. Parallelization of the computationally intensive matrix diagonalization module of MOPAC will be packaged as a template for use in related computational problems.

The CAPS is a National Science Foundation center for science and technology devoted to the development of a storm scale weather forecasting model. This effort is being carried out in conjunction with a national effort to place 175 scanning Doppler radars around the U.S. Along with research and educational goals, the CAPS model is seen as a prototype for regional weather prediction systems based on local radars, modeling, and dissemination to a regional constituency.

Developing a MPP strategy for the CAPS model is an essential step to developing improved models of the future.

The Loma Linda radiation therapy model is based on use of proton radiation for cancer treatment. This problem involves treatment planning, dose monitoring, and optimization. A related problem is identifying streak artifacts, due to radiation scattering, in scan images that serve as model input data. Planning models being used today were developed on a Vax, require one hour to run (the treatment period is 5-10 seconds), and are only 1/10 of the desired size. A long term goal in this application is development of biologically based treatment planning. This approach goes beyond calculation of radiation absorption, to include dose response modeling, estimating the likelihood of damage effects, and optimization of cancer treatment. NPAC is currently working with Dan Miller of the Loma Linda Radiation Therapy Department to understand the details of an improved computational model.

The amount of effort required to parallelize the entire MOPAC chemistry package is unclear due to software engineering issues. We will focus our efforts on the dense matrix diagonalization modules to produce parallel “template codes” or building blocks for this computational problem much as the LAPACK group at Oak Ridge National Laboratory is currently using to produce templates for iterative solutions of linear systems. This application will be implemented on a MIMD architecture, and will serve to build a collaboration between NPAC and users of this package within the Chemistry Department and CASE center at Syracuse University.

High Performance Fortran

NPAC is building the Fortran90D compiler with Rice, and a subset of Fortran90D will be incorporated into HPF due to be released in December, 1992. We view Fortran90D as a research prototype of HPF and expect FortranD to remain a generation ahead of industry standard HPF.

Fortran90D development is closely tied to application development. NPAC maintains an application benchmark suite for compiler evaluation. We propose to expand this applications oriented approach to software development by evaluating the specific details of applications to define the language constructs needed for implementation on parallel systems. For example, defining new data structures and algorithms for irregular problems will allow us to extend Fortran90D/HPF to a larger class of scientific and industrial applications.

Deliverables

Applications of interest to Digital Equipment Corporation will be parallelized to run on the DECmpp 12000, with work on MIMD parallel systems to be included as needed. We will deliver parallel application codes, associated software demonstrations, technical reports, and presentation materials. Software tools and language constructs required for application development will be integrated into FortranD and proposed for High Performance Fortran.

Interaction with Digital

NPAC will continue to interact with the MPSG, Rita Foley (ACTION contacts) and Bob Nilsson (member of ACTION advisory board) of Digital on ACTION contacts and, participate in related conferences and seminars such as the DPRI and Wall Street finance seminars. NPAC offers time on our machines for Digital’s purposes, and participates in the MasPar Challenge program.

Results from Previous Research

NPAC’s work in year one of our project with Digital Equipment Corporation emphasized a set of six application areas, and development of the Fortran90D compiler. This work is described in a set of technical papers and presentation slides submitted to Digital Equipment Corporation on September 15, 1992, and is summarized below.

NPAC’s Financial and Economic Modeling Project

A set of financial and economic modeling applications were surveyed and a stock option pricing model parallelized. Software and performance issues are evaluated, a pricing model and option market comparison is performed, and the potential for optimization to improve model accuracy is demonstrated.

Parallel Computing in Industry - An Initial Survey

Issues and examples of the use of high performance computing technology in industry are described. Information processing will likely emerge as an important HPC application. A classification of application problem classes is used to identify opportunities for HPC.

Industrial and Government Applications of Parallel Computing

This set of presentation slides summarizes issues of integrating HPC into industry, and NPAC’s ACTION program for industry.

Parallelization of CHARMM molecular dynamics code on multicomputers

This preliminary report describes work on algorithms for implementing molecular dynamics on parallel systems. Different strategies for parallelization are based on three main categories of methods for calculating nonbonded forces. Ease of parallelization, computational costs, load balancing, and architectural issues are summarized.

An Interactive Graphic Tool for Option Price Modeling on the DECmpp-12000 Using AVS

An AVS-framework for interactive model control and visualization is tested. Stock option pricing models are used as a test problem to demonstrate the potential for running multiple computational modules on a distributed heterogeneous network.

Draft Parallelization of MOPAC

The MOPAC software package is analyzed to define where parallelization has the greatest impact on performance. For typical geometry optimization calculations, diagonalization of the density matrix is the most time consuming component. Parallelization of the Householder transformation is described.

Scalable and Portable Fortran90D-High Performance Fortran for Parallel Computers

Current status and future plans for NPAC’s Fortran90D/HPF compiler and application benchmark suite are described. Project related activities include participation in the Fortran Forum, input into High Performance Fortran standards, and technology transfer to industry.

Qualifications of Researchers

The principal investigator Fox has been working in parallel computing for over ten years and pioneered an interdisciplinary approach involving applications and computer science. A description of his qualifications is listed immediately below. In addition, NPAC has a research staff of five Ph.D. level scientists who lead application and development projects, strong applications teams among faculty in computer science and physics, and strong ties with other parallel research projects worldwide. NPAC is a member of the NSF Center for Research in Parallel Computation CRPC, lead by Ken Kennedy of Rice. NPAC collaboration with Rice in Fortran D development is a central focus of CRPC parallel software research, and NPAC applications research.

Geoffrey Fox, NPAC Director, is an internationally recognized expert in the use of parallel architectures and the development of concurrent algorithms. He is also a leading proponent for the development of computational science as an academic discipline and a scientific method. His research on parallel computing has focused on development and use of this technology to solve large scale computational problems. Fox directs ACTION-NYS which is focused on accelerating the introduction of parallel computing into New York State industry. Fox earned his Ph.D. in theoretical physics from Cambridge University in 1967. His research experience includes work at the Institute for Advanced Study at Princeton; Lawrence Berkeley Laboratory; Cavendish Laboratory at Cambridge; Brookhaven National Laboratory; and Argonne National Laboratory. He has served as Dean for Educational Computing and Assistant Provost for Computing at Caltech. Fox is currently professor of Computer Science and Physics at Syracuse University and Director of the Northeast Parallel Architectures Center. He co-authored ”Solving Problems on Concurrent Processors” and edits ”Concurrency: Practice and Experience” and the ”International Journal of Modern Physics: C”.

Usage of Requested Hardware

In addition to the Loma Linda radiation therapy model, and development of a weather forecast model (CAPS) for MPP systems, the hardware request in this proposal will be used to support a wide range of application research projects. We need the largest DECmpp as we expand our set of ACTION program industry applications, and federally supported grand challenge applications.

The physics applications, which are centered in the area of very large scale condensed matter and statistical material science, include two disordered systems–random field Ising model, and spin glass on a large lattice. The random field Ising model runs small lattices in parallel with a communication efficiency of one. Our work on the spin glass on large lattice involves a new theory of disordered systems at finite dimensionality. Our code here updates 40 million spins per second (interactions are complex) The DECmpp is well suited for calculating diagonal interaction terms, and implementing a complex checkerboard parallelization scheme. We expect to show the existence of a phase transition in the finite dimensional model. The efficiency of this program is unique, and should show the DECmpp to be extremely powerful. Other applications in physics include calculation of superconductor flux lines, charge density waves and motions of electrons in random media.

Related to our partnership with Rice to develop FortranD is NPAC’s membership in the Center for Research on Parallel Computation (CRPC), centered at Rice University. NPAC will provide time on our upgraded DECmpp to Ken Kennedy’s (Director of the CRPC) group at Rice to support SIMD compiler work designed to allow irregular problems to run on MPP systems. Our work with Rice strengthens NPAC’s role as a research partner of Digital by broadening the research applications run on this system. We will put most of the equipment grant toward upgrading the DECmpp system.

NPAC is collaborating with NASA Goddard Space Flight Center to develop four-dimensional data assimilation models for massively parallel systems. This problem includes the following set of models- general circulation models, optimal interpolation models, transport models, chemistry models, and Kalman filtering. Our research approach will be to develop a set of coupled models in FortranD/High Performance Fortran to run on shared and distributed memory, and SIMD/MIMD architectures. NPAC’s DECmpp will be used to complement the NASA HPCC/Earth System Science testbed computer system, which includes the Connection Machine-5, MasPar MP-1, and Intel Touchstone Delta and Gamma machines. Parallel software issues to be examined include individual parallel models vs. integrated set loosely coupled models, performance and implementation effort required, portability, scalability, SIMD/MIMD compatibility, shared/distributed memory issues, and load balancing.

Future plans for the coming year include NPAC’s connection to the NYNet gigabit network between Syracuse, Cornell University, and Rome Laboratory. We plan research into heterogeneous and geographically distributed computing systems. For this work we are interested in obtaining a HIPPI interface to the DECmpp.

Budget

Hardware Budget

DECMPP Upgrade

DECMPP 100 to 200 upgrade  728,000
DECMPP 200 8K to 16K upgrade  800,432

maintenance (8 to 16K)

5,5616 × 8 units × 1 year  44,928
single unit of (2) 1.38GB disks  12,439
maintenance 1,488 × 1 yr  1,488
DECMPP upgrade total  1,587,287

DEC Workstations, Disks

ALPHA OSF/1 W/S with 1GB  24,013
maintenance 2,568 × 1 yr  2,568
Workstation, Disk Total  26,581

Equipment, Maintaince Total 1,613,868

$1,700,000 − $1,613,868 leaves $86,132. NPAC will become connected to NYNet, a regional gigabit network, and a HIPPI interface will be useful in this context. We wish to explore purchase of this interface, a price not to exceed $80,000 is currently estimated. We plan to use remaining funds toward the purchase of FUSE and ELIXIR software for the DECmpp. Phil Gadsden is getting price estimates.

Approximate total  1,700,000

× 75% discount

Net cost to NPAC  425,000

Research Budget

Our research budget totals $100,000 and is broken down into $25,000 level of support of NPAC’s ACTION program, and $75,000 for research support. NPAC researchers contributing to this project will include Geoffrey Fox, Director and Chief Scientist, Kim Mills, and Paul Coddington, Research Scientists, with additional contributions by senior NPAC staff as necessary.

Current Status

During our September 15, 1992, visit to Digital, we summarized research in year one of our project, and discussed this proposal at length. These discussions included Phil Gadsden, Digital’s Sales representative in upstate New York, and Steve Harrinton, Ajit Agrawal and Marco Annaratone of MPSG, Ken King of External Research, and others. We agreed on application development for MPP, and continued development of High Performance Fortran as the goals for year two of the project.