The fundamental mode of operation for PET at the CEWES MSRC is a direct and continual connection between the CEWES MSRC PET team universities and the CEWES MSRC users in support of the five Computational Technology Areas (CTAs) supported at the CEWES MSRC and three related technical infrastructure areas.
This is accomplished through on-site PET team members at the CEWES MSRC in close communication with PET team members at the supporting universities, who also make frequent visits to CEWES MSRC. The PET team members on-site at CEWES MSRC are full-time university personnel in most cases, supplemented by full-time NRC personnel. Some of the PET team members at the universities are full-time in the PET effort, spending regular time on-site at CEWES MSRC. The on-site PET team members at the CEWES MSRC are key to the CEWES MSRC PET operation, since these team members are the front line of contact with CEWES MSRC users. These six on-site team members are
Lead: Dr Wayne Mastin - NRC (Professor Emeritus of Mississippi State) CFD: Dr Steve Bova - Mississippi State CSM: Dr Rick Weed - Mississippi State EQM: Dr __ Fithen - Texas SPPT: Dr Clay Breshears - Rice SV: Dr Richard Strelitz - NRC (SAIC)
Also on-site at CEWES MSRC are the overall PET Project Leader, Dr Dick Pritchard, and the Training Coordinator, John Eberle, both of NRC. During most of Year 2, Dr Carey Cox of NRC was on-site in support of CWO, but has now left for an academic career. A complete listing of all the CEWES MSRC PET team personnel is given in Table 1.
During Year 2, the eight technical support teams in the CEWES MSRC PET effort operated as follows:
-----------------------------------------------------------------------------
CFD: Computational Fluid Dynamics CTA
CFD support in the PET effort at CEWES MSRC, as well as at the ASC and ARL MSRCs, is the responsibility of the NSF Engineering Research Center for Computational Field Simulation at Mississippi State University. At the beginning of Year 2, the CFD support team consisted of Dr David Huddleston (academic leader) and Dr Jianping Zhu at Mississippi State, and Dr Steve Bova of Mississippi State on-site at the CEWES MSRC. As the year progressed, the team was supplemented by the addition of Puri Banglore at MSU for MPI support and C Burg (graduate student). Although Mr Burg made valuable contributions his financial support was leveraged from another MSU project, and no CEWES MSRC PET resources were expended for his direct financial support.
The on-site leader (Bova) serves as an effective liaison between CEWES MSRC, NRC and MSU and as an effective technical liaison between CEWES MSRC users and the entire CFD support team. During Year 2, the CFD team duties evolved such that Bova now coordinates communication and facilitates interaction with other components of the CEWES MSRC PET team. This includes maintenance of the CFD webpages and biweekly activity reporting.
The CFD team in the CEWES MSRC PET effort serves the CEWES MSRC by providing
(a) Program-wide CFD support. (b) R&D expertise on selected technology enhancements. (c) HPC assistance for targeted codes.
The program-wide support pertains to direct CEWES MSRC user contact and cultivation, participation in workshops and technical meetings, user training in HPC, and other generic duties. HPC support for targeted codes and delivery of collaborative R&D expertise are more specific tasks selected to produce technology that has potential application and interest throughout the CEWES MSRC user community.
-------------------------------------------------------------------------------
CSM: Computational Structural Mechanics CTA
PET support for the Computational Structural Mechanics CTA at the CEWES MSRC was provided in Year 2 by NCSA (National Center for Supercomputing Applications) at the University of Illinois, under the leadership of John Towns (acting Academic Lead) and LeRay Dandy (CSM Project Team Leader), together with on-site support of structural codes of importance to CEWES MSRC users in CSM and visualization support for these users. The on-site support was provided by Dr Rick Weed of the ERC at Mississippi State, in communication with NCSA, and the visualization support was provided by NCSA.
The team also included project engineers at NCSA, including Dr Cristina Beldica (tracking and web support), Dr Bruce Loftis (CSM Visualization), Youngjin Woo (graduate student) and Gyuseok Kwak (graduate student). Dr M Fouad Ahmad served as CSM Academic Lead for part of Year 2.
The primary focus of the CEWES MSRC PET CSM team is to increase user productivity of widely deployed DoD codes. Productivity increases are realized by developing software tools and methods which automate time consuming processes often encountered within the CSM project cycle. In addition, the CSM team provides the best available solutions for specific problems encountered by end users.
-----------------------------------------------------------------------------
CWO: Climate/Weather/Ocean Modeling CTA
PET support for the Climate Weather and Oceanography CTA at CEWES MSRC is provided by The Ohio State University, Department of Civil and Environmental Engineering and Geodetic Science, under the leadership of Dr Keith Bedford. The co-project lead is Dr P Sadayappan of the Department of Computer and Information Science. The CWO team consists of Dr Shuxia Zhang, Dr David Welsh, and Sean O'Neil. Dr Carey Cox served as the on-site CWO leader until this past January. Welsh is regularly on-site at CEWES MSRC, and is responsible for working with CEWES MSRC personnel in the wave and circulation modeling activities. Zhang is a research scientist responsible for the focused effort work and continues to work with personnel in CFD on the CH3D-Sed and MM5 models. O'Neil is a graduate student working on circulation modeling. All staff share in programmatic responsibilities including ineraction with CEWES MSRC personnel, other CEWES MSRC PET team activities and cross-MSRC responsibilities, espesially with NAVO MSRC.
-----------------------------------------------------------------------------
EQM: Environmental Quality Modeling CTA
PET support for the Environmental Quality Modeling CTA at the CEWES MSRC is provided by TICAM (Texas Institute for Computational & Applied Mathematics) of the University of Texas at Austin, under the leadership of Dr Mary Wheeler. The support team for the EQM effort includes Drs Wheeler, Clint Dawson, Victor Parr, Srinivas Chippada, Carter Edwards, Monica Martinez, and Robert McLay; and graduate students Sujatha Sagiraju, Joe Eaton, and Jennifer Proft. Staff members include Connie Baxter, Sarah Woodruff and Stephanie Tomlinson. Wheeler is the technical area leader, and Dawson is co-leader.
Parr is a programming consultant and has also acted as a virtual "on-site" person. He is in frequent contact with the CEWES MSRC users. He has been primarily involved in the parallelization of CE-QUAL-ICM and more recently ADCIRC. Chippada and Proft have also been involved with ICM. Martinez has joined the team to help with the parallelization of ADCIRC. Edwards and Eaton have provided software support (i.e., a mesh partitioning algorithm) for both of these projects. Sagiraju, Bryant and McLay have been involved in the development of a web launching capability for the ParSSim code. Wheeler, Dawson, Edwards, Martinez and Parr also participated in the Parallelization Technology Workshop held at CEWES MSRC in January 1998. Baxter, Woodruff and Tomlinson provide staff support and webpage support.
------------------------------------------------------------------------------
FMS: Forces Modeling and Simulation/C4 CTA
The Northeast Parallel Architectures Center (NPAC) at Syracuse University has the lead role in FMS support at both the CEWES and ARL MSRCs, under the leadership of Dr Geoffrey Fox. NPAC's support team for the FMS area is composed primarily of NPAC's Interactive Web Technologies Group, lead by Prof Wojtek Furmanski. NPAC has extensive experience in both HPCC and commodity technologies, such as the world-wide web, Java and related tools, CORBA, and DCOM, as well as modeling and simulation. The Interactive Web Technologies group is comprised of more than a dozen researchers working on a variety of synergistic project. Group members focusing especially on activities related to the CEWES MSRC PET program include graduate research assistants Ozgur Balso, Hasan Ozdemir and Zeynep Ozdemir.
The Forces Modeling and Simulation CTA is in a fairly unusual position among the DoD CTAs, though it does share some characteristics with Integrated Modeling and Testing (IMT). Both FMS and IMT tools are widely used throughout the DoD, however only a very small portion of the user base is HPC users. Instead, for a variety of reasons, most FMS computing is done with local workstation or PC resources. To further complicate the situation, this field is in the midst of a transition from the well-established "distributed interactive simulation" standards for the development of applications to the new object-oriented "high-level architecture" (HLA) approach, following the mandate of the Defense Modeling and Simulation Office (DMSO).
Supporting a field in this situation is somewhat of a challenge. The approach adopted by the PET program (at both the CEWES and ARL MSRCs, where FMS is supported) involves tracking and monitoring this complex and changing field and its relationship to commodity and HPCC technologies. A small number of carefully chosen focused efforts are used to demonstrate the convergence of HLA with commodity distributed computing technologies, and to port applications with substantial HPC requirements to such systems. The overall goal of this approach is to develop exemplars which can then be used to highlight to the community the opportunities available through the use of HPC and commodity technologies.
-----------------------------------------------------------------------------
SPPT: Scalable Parallel Programming Tools
The SPP Tools support team is based at the Center for Research on Parallel Computation (CRPC), an NSF-funded Science and Technology Center with headquarters at Rice University. Rice plays the lead role in SPP Tools at CEWES MSRC, ARL MSRC, and ASC MSRC. The University of Tennessee at Knoxville is also a key CRPC site, and has a major role in SPP Tools at CEWES MSRC. The goal of the CRPC is "to make parallel computing truly usable by scientists and engineers". Toward this end, CRPC researchers have attacked the software and algorithmic problems posed by parallel and distributed machines. The solutions they have found are crucial elements in the DOD Modernization Programming Tools effort.
Individually, the SPP Tools support team includes internationally-recognized experts in the areas of distributed and parallel computing, dense and sparse linear algebra, parallel languages and compilers, parallel benchmarking and performance evaluation, and interprocessor and network communication. Team members have considerable experience in producing, deploying, and supporting software systems that turn their best research ideas into widely used tools, such as the Parallel Virtual Machine (PVM) system and the ScaLAPACK parallel linear algebra library. In addition, team members have led a number of significant standards efforts, including the High Performance Fortran (HPF) Forum, the Message Passing Interface (MPI) Forum, the BLAS Technical Forum, and ParkBench. They have also been instrumental in transferring technologies including compiler data dependence analysis and memory hierarchy optimization techniques to the commercial sector. Some of the key personnel involved in PET are Dr Ken Kennedy (Senior Technical Lead, Rice University), Dr Jack Dongarra (Senior Technical Lead, UTK), Dr Chuck Koelbel (SPP Tools Lead, Rice University), Dr Shirley Browne (SPP Tools Lead, UTK), and Dr Clay Breshears (On-Site SPP Tools Lead). Nearly a dozen additional PhD-level researchers also participate at Rice and UTK.
The primary goal of SPP Tools is to promote a uniform, high-level, easy-to-use environment for programming available to all CEWES MSRC users and, ultimately, to all of DoD. We view "programming" very broadly, encompassing any means of describing a sequence of executable actions. This includes writing traditional CFD simulations as well as developing quick-and-dirty filters to scan output files. Similarly, we view "tools" as including any software that eases the task of programming. This includes tools, such as compilers, that make programming possible at all; it includes other tools, such as libraries, that provide easier-to-use facilities for programming; and it includes some tools, such as performance profilers, that help users understand their programs. In order to move toward this goal, we emphasize several secondary objectives:
(a) Promote efficient, abstract programming. We work to supply languages and systems programmed in terms near the problem domain, and automatic tools that produce efficient machine-level programs. Since such systems are still some way in the future, we also work to improve lower-level languages and tools to provide short-term help to CEWES MSRC users.
(b) Enhance program understanding. We consider tools that collect static and/or dynamic data, relate the data to the original program, and assist the programmer in improving the program. Out of necessity, we also consider currently available tools that do not meet all of these criteria. (c) Establish standards for a uniform programming environment. We promote this by seeking out and promoting high-quality implementations of formal and informal standards, and by helping to develop new standards as technology matures.
SPP Tools is a cross-cutting area: if we are successful, all CTAs will benefit from the new tools. It has a long-term goal that will not be met by PET activities alone; in order to provide a complete set of tools, far more resources will be needed than DoD Modernization alone can provide. We therefore put great emphasis on collaboration with CEWES MSRC users (to understand the requirements for new tools), with other CEWES MSRC PET team members and other MSRC PET teams (to disseminate and test the tools), and with groups outside of DoD (to identify and develop the tools).
----------------------------------------------------------------------------
SV: Scientific Visualization
PET support for Scientific Visualization at the CEWES MSRC is the responsibility of NCSA (National Center for Supercomputing Applications) at the University of Illinois, under the leadership of Dr Polly Baker. The CEWES MSRC PET Visualization team includes the Senior Academic Lead (Dr Polly Baker from NCSA), an on-site NRC Lead (Dr Richard Strelitz from SAIC), and an NCSA Lead (Dr Alan Shih from NCSA). Dr Baker provides long-term direction and leadership for the effort. Dr Shih serves as the primary conduit between activities at NCSA and the CEWES MRSC. He is headquartered at NCSA and spends a significant amount of his time on-site at the CEWES MSRC. Dr Strelitz is located at the CEWES MSRC and works closely with Dr Shih.
The team also includes project developers at NCSA, such as Randy Heiland, Dave Bock, and Rob Stein. Ed Peters is a graduate student involved in project development. Alan Craig provides tracking and web support, along with two undergraduate students, Tom Devor and Nate Crane. In the summer of Year 2, the team also included Dr Jay Jackson, serving as the NCSA Lead.
The CEWES MSRC PET SV team interacts with end users to define user needs, provide information on available solutions, and prototype custom solutions where necessary. The team also coordinates with other CEWES MSRC personnel specializing in visualization. For example, we share information and tools with visualization developers at the CEWES MSRC. We also team with the local visualization specialists on selected development projects, such as the Chesapeake Bay or Damaged Structures visualizations.
Activity in support of Scientific Visualization in the CEWES MSRC PET effort is characterized by a combination of efforts intended to apply to a broad user community at the CEWES MSRC, as well as efforts that support a particular user group.
-----------------------------------------------------------------------------
C/C: Collaboration/Communication
Support for the Collaboration/Communication technical infrastructure on the CEWES MSRC PET team is provided by NCSA (National Center for Supercomputing Applications) at the University of Illinois and the Northeast Parallel Architectures Center (NPAC) at Syracuse University.
NCSA made transitions in this team in Year 2, from Dr John Ziebarth to Lex Lane as Senior Lead, and from Frank Baker to Sandie Kappes as Team Leader. Lisa Gatzke is the Web Developer. In addition to the NCSA staff, Herman Moore (E-Systems) - the CEWES MSRC PET webmaster - is the on-site C/C lead.
Prof. Geoffrey Fox leads the overall C/C effort at NPAC. Database-related activities were handled primarily by Dr David Bernholdt and Yuping Zhu, while work on the Tango collaboratory system and related tools was lead by Dr Marek Podgorny. The Tango group also included Research Scientists Roman Markowski, Tomasz Jurga, Tomasz Major, Konrad Olszewski, Piotr Sokolowski, Rawel Roman, Tomasz Stachowiak, Remek Trzaska, and Bart Winnowicz as well as Graduate Research Assistants Lukasz Beca and Grzegorz Lewandowski.