NAVOCEANO Project Proposal from NPAC/Syracuse U.
High Performance Commodity Computing (HPcc) Based HLA-Compliant Meta-Cluster Management and Meta-Federation Support for NAVO Applications – Phase 2
Project POC
David Bernholdt, NPAC/Syracuse University
Email: bernhold@npac.syr.eduTelephone: 315 443 3857Fax: 315 443 1973
Project Description
In our Year 3 proposal, we introduced the concept and we specified the initial core effort towards establishing an HLA-compliant metasystem support at NAVO. We proposed to build such environment by integrating/federating our WebHLA system in the Web/commodity domain with the Legion system in the HPC domain. WebHLA implements components of DMSO HLA such as RTI 1.3 as CORBA facilities within our JWORB (Java Web Object Request Broker) based multi-server multi-protocol (IIOP+HTTP+ORPC+…) middleware layer. HLA is a DoD-wide federation standard that originates from the modeling and simulation domain and is rapidly expanding to other distributed computing areas, both inside and outside DoD, and both in U.S. and overseas.
In this document, we propose the follow-on Year 4 effort that will customize, test and apply WebHLA environment for selected HPC applications of relevance for NAVO. In the current, early version of this proposal we focus on generic patterns common for several HPC applications and the corresponding general purpose federation techniques offered by our approach. Specific applications will be identified during the Phase 1 (Year 3) project and the current preliminary Phase 2 (Year 4) proposal will be then suitably focused, specified and quantified.
We have organized the whole project as a 17 month effort divided in to two phases:
Phase 1 Jan 99 – May 99, 5 months, NAVO PET Contract Year 3
Phase 2 Jun 99 – May 00, 12 months, NAVO PET Contract Year 4
The Phase 1 proposal has been submitted to NAVO earlier this year. The current Phase 2 proposal assumes that the Phase 1 project has been completed. Otherwise, we would suggest to modify the Phase 2 proposal by prepending the Phase 1 core effort, i.e. by merging both proposals and by combining the duration and the funding level accordingly.
Hence, we assume here that we have already installed WebHLA infrastructure at NAVO and that we have identified some target HPC application domains for which our HLA meta-federation techniques would add value and boost productivity by facilitating integrability and interoperability between indvidual components. Such univeral "glue" capability, offered by the HLA federation model is in fact useful for many typical HPC applications and hence we can present here an outline or skeleton of the Phase 2 effort without having yet selected any specific domain. Most typical HPC applications include one or more CPU-intense components that require MPP support and are coupled with a collection of utility components that usually map best for different, either dedicated or general purpose, nodes of a heterogeneous distributed metacomputing environment. Typical examples of such supporting components include:
The complexity and inherent heterogeneity of such environments was often neglected in previous generation HPC software systems, trying to address all services listed above in a bottom-up, MPI based heroic programming effort. Meanwhile, the Web technologies exploded, gained rapid acceptance and thereby accelerated the associated more advanced distributed object technologies of CORBA, Java, COM and WOM/XML. This new spectrum of Web/commodity technologies can be now employed for building much more efficiently several metasystem infrastructure components and individual utility services listed above. HLA, on the other hand, offers a complementary, solid and e-commerce unbiased top level federation model in terms of the RTI based powerful software bus that supports broad range of communication mechanisms (shared objects, event-driven, time-managed, publish/subscribe etc.) and facilitates plug-and-play support for metasystem components. WebHLA merges both capabilities and opens the road for Web, Commodity and DoD standards based metacomputing.
In our other ongoing experiments with WebHLA, so far conducted within the Forces Modeling and Simulation CTA (see for example http://iwt.npac.syr.edu/documents or http://bombay.npac.syr.edu/fms) we already developed a subset of utility services listed above (see http://iwt.npac.syr.edu/demos) and we will make them available for NAVO as part of this project. After exposing our current capabilities to NAVO, we will work with NAVO researchers to identify application domains where WebHLA would immediately add significant value and generated near term "success stories". For each selected application domain, we will interact with domain experts to identify the essential data structures to be shared / communicated between metasystem modules and we will formalize it in terms of an HLA-compliant Federation Object Model (FOM) specification for this computational domain or perhaps some broader subset of the CTA it belongs to. If a particular application domain or CTA decides to make their FOM publically available, there are already mechanisms and technologies in place to publish such more broadly reusable FOMs in the emergent HLA OMLs (Object Model Libraries). DMSO already develops and maintains public, Web accessible FOM repositories for modeling and simulation, and we believe HPCMO could start a related effort in several HPC specific CTAs.
Using WebHLA API and interface libraries, we will wrap existing HPC modules (e.g. Legion based) in the selected applications as HLA federates, and we will customize our existent general purpose and/or develop new dedicated utility components, supporting NAVO applications and conforming to the NAVO FOMs. We will also suitably customize our visual authoring tools by including NAVO FOM-compliant federates as module palettes and by assuring that all relevant data sharing or interaction patterns, routing spaces, and other more domain-specific communication or connectivity pathways can be easily assembled in a visual interactive mode in terms of FOM modules as nodes of suitable computational and communication graphs. Such high level domain specific frameworks will be delivered to application developers in NAVO.
Project Deliverables
(Timeline denoted as months relative to the start date "S". Schedule
assumes a single demonstration project is chosen. If two are chosen,
work may either proceed concurrently or sequentially, with appropriate
modifications to the schedule.)
Project Benefits
This project will result in specific DoD research teams/projects receiving direct assistance in the incorporation of HPcc ideas into their software development efforts, and at the same time provide experience and examples which will facilitate the wider dissemination of this approach through trainings and other means.
As a result of the Phase 1 project, JWORB technology will be deployed at NAVO and it will offer an integrated support for several leading commodity technologies for distributed objects such as Java, CORBA, COM and Web/XML. DMSO HLA compliance assured via the WebHLA technology will enable the DoD-mandated style of federating NAVO applications with other systems, labs and centers. Interoperability with Legion would enable federating with Legion applications and it would facilitate the addition of HLA-compliance to the Legion model.
The Phase 2 Project will boost productivity for selected application domains at NAVO by offering solid, consistent, standards-based plug-and-play framework for software engineering, integration and federation of various heterogeneous components of aselected NAVO metasystem(s).
We believe that the proposed process of WebHLA based meta-federation developmet, integration and customization, after being successfully tested for NAVO applications, will be likely assimilated by other MSRCs and will give NAVO opportunity to act as the leaderhip site for this new HPCMO technology.
Collaboration
While we are not yet in a position to identify specific research teams/projects which fit this proposal, the willingness and responsiveness of the team to work jointly on this project will be an important factor in the choice. The customers of this project will, of necessity, have a high level of involvement in this effort if it is to be successful.
Collaboration with the UVa Legion group and those involved in its deployment/operation at NAVO will also be required.
Budget Justification