Given by Geoffrey C. Fox, Wojtek Furmanski at Madison Wisconsin Internet server Performance Meeting and Sandia Review Albuquerque on June 23 and 24 1998. Foils prepared July 4 98
Outside Index
Summary of Material
We present here NPAC concepts for Web/Commodity based HPDC/Metacomputing including:
|
Next, we summarize the associated technologies such as: JWORB, WebFlow and Globus Wrappers. |
Finally, we describe prototype applications in selected domains such as Nanotechnology Monte Carlo (NSF Alliance) or Modeling and Simulation (DoD HPCMP) |
Outside Index Summary of Material
Madison Wisconsin June 23 1998 |
Geoffrey Fox and Wojtek Furmanski |
Northeast Parallel Architectures Center |
Syracuse University, Syracuse NY |
{gcf, furm}@npac.syr.edu |
We present here NPAC concepts for Web/Commodity based HPDC/Metacomputing including:
|
Next, we summarize the associated technologies such as: JWORB, WebFlow and Globus Wrappers. |
Finally, we describe prototype applications in selected domains such as Nanotechnology Monte Carlo (NSF Alliance) or Modeling and Simulation (DoD HPCMP) |
Pragmatic Object Web - integrate competing models for distributed objects: Java, CORBA, COM, WOM |
High Performance Commodity Computing - traditional HPC modules managed by POW on new commodity clusters (PC with NT, Linux or Solaris OS) using Distributed Computing Concepts (HLA,RTI) at coarse grain and classic HPCC for computational kernels |
Visual Authoring of HPDC Applications - towards a uniform high-level visual framework that integrates design / modeling, development / authoring and runtime / steering of distributed applications |
Distributed Computing becomes a commodity article (driven by Web Technologies) |
Market niches for orthodox MPP style HPC are shrinking |
NT clusters become a viable and more cost effective alternative to classic high performance systems |
HLA/RTI from distributed simulation community natural for coarse grain while MPI/HPF/.... Natural for fine grain -- must integrate which we claim can be done using a multi tier architecture |
Web/Commodity software - promising base to build new HPcc (commodity computing) |
3-(or more)-tier architecture - Web browser front-ends, legacy (e.g. databases, HPC modules) backends; fat (1+tier) middleware |
Alternative / competing Middleware models:
|
Each model does one or other service particularly well |
POW attempts at integrating various models and services in terms of multi-protocol middleware servers (JWORB) |
Server: e.g. |
Proprietary |
Database |
Lotus Notes |
Web or ORB |
Service e.g. Database Repository or file systems accessed by Web Servers |
Client |
Now in POW style, we add modular capabilities to get 3 4 or more tier |
Back End Server: e.g. |
Proprietary |
Database |
Service e.g. Database Repository |
ThickClient e.g. Java Applet GUI |
Middle Tier Server with "Business Logic" e.g. map user objects to relational tables as in Java Blend |
We get 4 tier by refining client .... |
But Middle Tier can be a plethora of servers linked in a dataflow model |
The backend servers would include CORBA objects from Educom's IMS projects; Video servers and Oracle database defined curricula pages from NPAC |
The front end servers would include distributed students with mirror sites to get performance |
In the middle tier, we have JDBC query processing and XLM servlet parsers mapping original data in optimal fashion to match needs of student -- choosing from pure HTML or Interactive Java Whiteboard views of a given object |
We have multiple supercomputers in the backend -- one doing CFD simulation of airflow; another structural analysis while in more detail you have linear algebra servers (Netsolve); Optimization servers (NEOS); image processing filters(Khoros);databases (NCSA Biology workbench); visualization systems(AVS, CAVEs)
|
All linked to collaborative information systems in a sea of middle tier servers(as on previous page) to support design, crisis management, multi-disciplinary research |
We can support any given paradigm at either high functionality (web server) or high performance (backend) level |
HPCC Messaging could be a Java/RMI middle tier MPI or Nexus/Optimized Machine specific MPI at backend |
JWORB supports CORBA based RTI already and we can bridge to high performance event driven simulation systems like SPEEDES at the high performance backend layer |
However most problems can be thought of a set of coarse grain entities which are internally data parallel but the coarse grain structure is "functional" parallelism |
So HLA/RTI is especially natural as tier 2 management level of these coarse entities |
Entities can be time synchronized simulations and use MPI(HPF?) at either middle or back end tier or in fact as in DMSO simulations a federate running a custom discrete event simulation |
Resource Management typically breaks down into either
|
So a) is all at middle tier and should use commodity solutions -- there are many queuing systems such as Condor, Codine, LSF which we can "wrap" and Microsoft does not yet have a fully scalable commodity solution
|
So it is still embryonic but we suggest adopting the HLA/RTI framework as this supports job placement, interdependencies (time management) and hierarchical systems of federations --> federates |
Optimized data placement has been largely solved as a mathematical problem by HPCC but not packaged broadly. Our suggestion suggests how to invoke as backend support for a commodity service |
So we have a hierarchy of entities Federation --> Federates --> Objects where can have many tiers in each category |
A Federation could be the set of all jobs to be run on a particular site |
A Federate could be a job consisting of multiple possibly shared objects |
Objects are just data structures in HLA -- you send interaction events instead of invoking methods |
These aspects are organized by Federation, Object and Ownership management services |
We can classify both jobs and computers as separate federations |
Declaration Management corresponds to publication and subscription model of matching services and needs
|
Time Management corresponds to scheduling of sequenced events in discrete event simulations -- it will allow support generally dependencies in jobs -- the CAVE visualization system must be used after simulation |
Data management is classic "load-balancing" problem of parallel computing where you map objects optimally to computers to minimize communication cost and load imbalance |
|
Facilities |
Object Bus over IIOP (Internet Inter-ORB Protocol) |
Services |
|
|
|
|
Pragmatic Object Web is multi-lingual and programmatically complex |
Web based Visual Programming offers a promising avenue towards uniform visual standards (e.g. UML accepted both by Microsoft and OMG) |
NPAC WebFlow starts from the visual composition support (a la AVS / Khoros) and extends it towards steering (NCSA apps) and design / development (planned integration with UML) |
Related NPAC technologies include: DARP, SciVis |
WebFlow - Web based visual dataflow programming |
JWORB - Java Web Object Request Broker - POW middleware server node |
Globus Wrappers - Globus services as WebFlow modules (to become CORBA objects and services) |
Object Web RTI - DMSO RTI of HLA as JWORB service |
Visual Authoring Tools - integrates POW with commodity front-ends (Java applets, ActiveX controls, Office+VBA apps etc.) |
Televirtual (TVR) Environments - Interactive Web/3D front-ends for multi-user POW applications (DirectX, VRML, Java3D) |
(FMS) Training Space - TVR front-ends to POW linked databases |
WebFlow - a visual programming environment for the Web with real-time dataflow graph authoring tools |
3-tier multi-server system with Web browser / Java applet front-end, Java Web Server middleware and (Java) computational modules in the backend |
Early prototype - pure Java system, controlled by servlet based Session, Module and Connection Managers |
POW middleware will support all languages |
Used to build both linked modules and as framework for "wrapping legacy codes" as "distributed scientific objects" |
|
|
|
|
|
Simulation |
Basic Display |
Image Filter |
Output Display after Filter |
Original Image |
Output Image |
Some of |
Available Image Filters |
Visual DataFlow |
Interface |
JWORB - Java Web Object Request Broker - multi-protocol middleware network server (HTTP + IIOP + DCE RPC + RTP + ..) |
Current prototype integrates HTTP+IIOP i.e. acts as Web Server and CORBA Broker |
Next step: add DCE RPC support to include Microsoft COM |
JWORB - our trial implementation of POW |
JWORB (Java Web Object Request Broker) is our multi-protocol (now HTTP + IOOP) middleware server |
Difficult to demo as middleware invisible... |
We illustrate HTTP and IIOP integration by timing a ping call over IIOP channel while invoking a Web page from JWORB - to notice a transient ping performance drop |
Just IIOP |
Slow down in Ping when JWORB answers HTTP request |
JacORB |
JWORB |
ORBIX |
RMI |
Variable Size Integer Arrays |
Java ORBs Transferring |
variable size Array of Structures |
(RMI slowed by serialization) |
RMI |
JacORB |
ORBIX, JWORB |
Arrays of Integers C++ about 20 times faster than Java |
RMI (Fastest Java) omniORB (C++) |
Implements DMSO RTI as JWORB service with 2 major CORBA objects: RTI Ambassador and Federate Ambassador |
Offers natural Web interfaces to HLA simulations via HTTP or IIOP channels |
Natural support for human-in-the-loop (Web surfers join WebHLA federation and can collabortate as WebHLA federates) |
Attractive model for High Level Metacomputing |
We pursue four visual authoring thrusts at NPAC:
|
Visual Debugging Using DARP + SciVis |
|
Microsoft is publishing now COM APIs to several major applications |
For example, Office `97 is now fully programmable via Visual Basic scripting |
This turns NT desktop into a powerful commodity front-end development toolkit - here illustrated for Excel, Outlook `98 and Project `98. |
Also can use to build database backend for PowerPoint used to make educational objects |
Rose by Rational is the leading implementation of the new standard UML (Uniform Modeling Language) |
UML supports OO design for various object-oriented languages in terms of a family of diagram topologies (class, sequence, interaction, activity, deployment..) |
Microsoft has a similar product called Visual Modeler as part of Visual Studio Enterprise Edition |
We are planning to build new WebFlow front-end that would conform to UML and extend it from the design stage towards composition and runtime steering stages |
Ultimate Internet medium is likely a multi-user multi-server multi-sensory 3D Televirtual (TVR) Environment |
We approach it via technology iterations: networked VR, VRML+Java, now POW |
Our TVR `98 will use Object Web RTI for collaboration servers and Microsoft DirectX for client sensory front-ends |
TVR is a joint project with IBM Watson on exploring scalable VRML/Java3D based multi-user collaboratory environments (worlds) |
Previous demo used JSDA collab server and represented users as simple VRML avatars (cones) |
New demo uses CORBA collaboratory server |
We are also including avatar/humanoid animation |
WebFlow over Globus - Steering a Complex Multi-stage Metacomputing Application: Nanotechnology Monte Carlo at NCSA |
WebHLA - JWORB/OW-RTI based Visual Authoring, Training and Virtual Prototyping Environment for DoD HPCMP Modeling and Simulation |
DMSO specifies FEDEP (Federation Development Process) that enforces tools reuse via common DIFs (Data Interchange Formats) |
This maps naturally on the dataflow model of WebFlow with modules linked by DIF channels |
Textual/spreadsheet editor modules can be built from commodity tools using VBA scripting |
This project uses WebFlow and commodity software to support visual HLA authoring |
|
SBA - Simulation based Acquisiton |
New concept in advanced DoD M&S - full product development cycle in Synthetic Environment (including CFD, CEA etc. HPC simulations) |
We claim that WebHLA is capable to deliver SBA in the near term due to the critical mass of accumulated open standards and freeware |