Programming Tools for High Performance Distributed Computing Geoffrey Fox, Wojtek Furmanski, Tom Haupt Collaboraters(unfunded): GEM Earthquake Modeling and Prediction Group (led by John Rundle, Colorado and involving Los Alamos and Livermore laboratories) Ian Foster (Argonne) John Reynders (Los Alamos) Executive Summary Programming tools that are simultaneously sustainable, highly functional, robust and easy to use have been hard to come by in the HPCC arena. This is partially due to the difficulty in developing sophisticated customized systems for what is a relatively small part of the worldwide computing enterprise. Thus we have developed a new strategy -- termed HPcc High Performance Commodity Computing -- which builds HPCC programming tools on top of the remarkable new software infrastructure being built for the commercial web and distributed object areas. This leverage of a huge industry investment naturally delivers tools with the desired properties with the one (albeit critical) exception that high performance is not guaranteed. Our approach naturally gives the user access to the full range of commercial capabilities (e.g. databases and compute servers), pervasive access from all platforms and natural incremental enhancement as the industry software juggernaut continues to deliver software systems of rapidly increasing power. We add high performance to commodity systems using a multi tier architecture with traditional HPCC technologies such as MPI and HPF supported as the backend of a middle tier of commodity web and object servers. Our approach can be applied to both parallel and distributed computing environments but here we focus on the latter. This exploits the full power of our approach; has very few tools to date (as it is not the preserve of a particular hardware vendor) and is of importance to ASCI linking multiple laboratory and user machines together. We build on existing runtime and propose to use MPI and Globus at the high performance-messaging layer with POOMA as the base parallel object technology. Our approach is general and this choice will evolve as the needs of ASCI and trends in the HPCC community dictate. We propose construction of a sequence of high level programming tools of increasing functionality and technical sophistication. We identify three milestones below corresponding roughly to each year of the proposal. NPAC and its collaborators will initially test each tool and then we will deploy in ASCI whose evaluation will guide further evolutionary and revolutionary changes in the tools. We can offer training courses (using if desired our novel distance training technology) in both the core tools and the underlying commodity technologies. The first ASCI deliverable will be a web based dataflow environment (similar in spirit to AVS and Khoros) WebFlow built using Java Applet front-ends, Java Web servers as middleware with a high performance backend built from Globus, MPI or POOMA. WebFlow can be considered as a high level visual interface to these key HPCC technologies. In the second stage, we will upgrade the Javasoft Web servers (and associated servlets) to JWORB servers, which will simultaneously support COM CORBA and HTTP (Web) protocols. JWORB embodies our vision of the "Pragmatic Object Web" with industry's impressive but still incomplete integration of web and distributed object technologies. NPAC has extensive experience in building powerful systems using this approach and based on this, we are writing a new book ("Building Distributed Systems on the Pragmatic Object Web") which we will finalize as we use it in a class next semester. The JWORB based WebFlow allows natural integration of specialized PC (COM) and Industry standard CORBA distributed objects with the powerful Web dissemination model with technologies such as Java and VRML. This will give users an ability to link high performance localized (MPI, POOMA) or distributed (Globus) compute services with commodity relational and object databases. The system will also support commodity visualization, collaboration and other important services that are typically lacking in the custom HPCC products. JWORB has been designed to support the military distributed simulation object standard HLA (High Level Architecture). On top of JWORB, we will add to the visual-computing interface, a user level scripting capability, which supports interpreted as well as compiled execution of modules, and enables interactive rapid prototyping of simulation software. In the final stage of the project we will add a hierarchical visual mode where one can expose the internals of a module and build visually in terms of finer grain POOMA objects. This would be the HPcc realization of a component-based software engineering approach to high performance computing environments. We do not of course have experience to validate the capabilities of such a programming environment but the success of componentware in the PC world suggests it will also lead to user friendly HPCC systems. We intend to test our developed environment using new application software, which we will be building to support the GEM earthquake prediction computational toolkit. This application is of importance to DoE and involves currently Los Alamos and Livermore. This focused test case of our new environments will be complemented by the broad tests in ASCI and our NCSA collaborators.