Given by Geoffrey C. Fox at Review Meeting of PSPTA Group Manchester UK, Garland Texas(E-systems), Winnipeg Canada(HPCS'97) on July 1,9,12 97. Foils prepared July 14 97
Outside Index
Summary of Material
We suggest that one can re-engineer existing HPCC concepts using emerging Web technology to produce a far more effective programming environment |
This presentation asserts that software architecture should be designed in three layers: client interface, high functionality middleware accessing when necessary specialized high performance third layer |
On client side, we discuss VPL as an example and suggest that a Java Computing Services framework can lead to universal interoperable interfaces |
In middleware, we discuss Collaboration, Integration software, Multidisciplinary Applications, databases and CORBA
|
Finally we discuss Java as the universal language to express all of this -- including scientific computing! |
Outside Index Summary of Material
http://www.npac.syr.edu/users/gcf/psptajuly97 |
1) Manchester July 1 1997 Second Technical Meeting of |
Portable Software Tools for Parallel Architectures (PSPTA) |
2) E-Systems Garland Texas Presentation July 9 1997 |
3)HPCS'97 Winnipeg Canada Keynote Talk July 12 1997 |
Geoffrey Fox |
Syracuse University NPAC |
111 College Place, Syracuse NY 13244 4100 |
Phone 3154432163 |
We suggest that one can re-engineer existing HPCC concepts using emerging Web technology to produce a far more effective programming environment |
This presentation asserts that software architecture should be designed in three layers: client interface, high functionality middleware accessing when necessary specialized high performance third layer |
On client side, we discuss VPL as an example and suggest that a Java Computing Services framework can lead to universal interoperable interfaces |
In middleware, we discuss Collaboration, Integration software, Multidisciplinary Applications, databases and CORBA
|
Finally we discuss Java as the universal language to express all of this -- including scientific computing! |
Ideas from HPCC research Good! |
Not enough people/funding in field to implement robust production systems |
Must re-use as much software (including infrastructure software) as possible |
Similarly must build HPCC software in a modular fashion with small enough modules that smallish groups can build effectively |
Different modules are likely to use different base technologies (Fortran v Java v C++ etc.) and so interoperability essential! |
No silver bullet on the horizon - maybe pessimistic but implies better HPCC environments implies better implementations of existing ideas. |
Need to support both production use of MPP's and "rapid prototyping" in development of new applications - latter is not well supported by current HPCC software systems even though need parallel support for prototyping of new 3D simulations |
PC and workstation clusters are of growing important and this typically distributed memory people's technology is contrasted with distributed shared memory tightly coupled MPP's. |
Computational science moving to multidisciplinary (multi-component) applications |
Corresponding growing use of databases (for data-intensive applications) |
Interoperability between disparate heterogeneous platforms, support of multidisciplinary applications, and metacomputing are three related important areas |
"full metacomputing" (decompose general problem on general networked resources) may not be relevant |
The Web is delivering a new operating environment (WebWindows) and a rich distributed computing software infrastructure with especially excellent support for software integration |
There is a need for a new scalable technical operating system (NT v UNIX v WebWindows) |
We can distinguish Decomposition and Integration |
Decomposition is performed by an HPF or other Parallelizing compiler; or by a user writing a Fortran + Message Passing code "by hand" |
MPI integrates decomposed parts together with high bandwidth latency constraints |
Systems such as AVS integrate larger modules together and much of "software engineering" (modular style of programming) involved with this |
Web is a powerful integration model suitable for large coarse modules with modest latency and sometimes modest bandwidth requirements
|
Collaboration, computational steering, multidisciplinary science are all integration and not decomposition problems! |
By definition, Web Software will be the "best" software ever built because it has the largest market (and so greatest leverage of investment dollars) and most creative business model (harness the world's best minds together with open interfaces)
|
One should build upwards from the "democractic Web"
|
This allows you to both deliver your application to the general public (not always required but often desireable) and use the best leveraged software |
Note Web Software tends to offer highest functionality as opposed to highest performance and HPCC often requires different trade-offs |
Web Technology is still uncertain and there may be major changes but "enough" capabilities are in place to build very general (~all) applications
|
Rapidly evolving Standards and a mechanism to get rapid consensus |
Fortran 77 -> Fortran90 --> HPF --> Fortran2000 (23 years) |
VRML Idea (1994) --> VRML1 deployed (95) --> VRML2 deployed (early 97) (2.3 years)
|
Classic Web: HTTP Mime HTML CGI Perl etc. |
Java and JavaScript Compiled to almost compiled (applet) to fully Interpreted Programming Language |
VRML2 as a dynamic 3D Datastructure for products and their simulation object |
Java Database Connectivity (JDBC) and general Web linked databases |
Dynamic Java Servers and Clients |
Rich Web Collaboration environment building electronic societies |
Security -- still needs maturing as very clumsy or non existent at present in many cases |
Compression/ Quality of Service for Web Multimedia
|
Emerging Web Object model including integration of Corba (see JavaBeans and Orblets) |
See Original Foil |
Java for the User Interface: This is roughly the "WebWindows Philosophy" of building applications to Web Server/Client Standards |
Java for Coarse Grain Software Integration: see collaboration and metacomputing |
Java as a high performance scientific language: for "inner" (and outer) loops Here parallelism is important but sequential issues also critical and first issues to examine! |
HPCC is a small field with limited resources for a very hard problem and must leverage as much software as possible |
Web Software provides an excellent pervasive user interface with Java Applets and WebWindows |
Web Software provides a potentially excellent high performance object oriented language (Java) for scientific and engineering computation |
Web Software provides a high functionality but modest performance distributed computing environment based on either Web Servers or Clients
|
All(!?) we need to do is to add high performance to the Web! |
Building from bottom of Computing pyramid starts with high functionality software which has an architecture that can be augmented with high performance |
3 Levels of Software
|
One universal language -- Java for all layers |
1:User View: Interoperable Web Interface accessing services through Java Compute Services Framework 2:Network of Java Servers provide distributed services to databases, compute engines, collaboratories, object brokers, instruments
|
High Performance layer accessed when distributed services have insufficient performance
|
Compute processes linked by MPI or distributed units of database, object broker, I/O system etc. |
Java Servers |
If performance adequate, use server level as universal and so certainly put all control at server level |
Coarse grain Integration at Server level |
Fine grain linkage at High Performance layer |
A |
B |
MPI |
Java Socket |
Compare two ways of linking two modules or two programs of a multidisciplinary application where each module has internal MPI implementation |
Server Based or High Performance MPI InterModule connectivity |
One example is VPL -- Virtual Programming Laboratory -- interface to HPF and MPI used by Syracuse and Cornell in courses/training |
If you agree that it can be web based then naturally Implemented as Java Framework for Computing Services |
Any Job |
Computer A |
Computer B |
Computer C |
Universal Interface |
Metaproblem Individual Computer or Metacomputer |
Software Bus |
Computer |
Sequential or parallel or |
Hardware Bus/Network of Metacomputer |
Mapping of general Problem or metaproblem onto general local or wide area network of computers |
or |
Any Job |
Computer A |
Computer B |
Computer C |
Computer D |
Includes cases like: with a --> A and b --> B which is classic distributed computing |
As well as classic parallel computing (e.g. HPF) with a heterogeneous target |
And arbitrary mixtures thereof |
HPF Job |
12 Node SP-2 |
8 node PC Cluster |
Multidisciplinary Applications (in their simplest implementation) are distributed computing which is integration at server level in my Java Framework for distributed computing |
General Metacomputing is the much harder combination of heterogeneous decomposition and integration |
Interoperable interfaces involve setting of standards for a client server integration (Java computing services framework) |
This is least controversial and is essentially WebWindows for User Interfaces |
Fortran was never good at user interfaces! |
Initially Aimed at education where usability higher priority than performance |
Teaching Java and JavaScript greatly aided by interpreted technology which allow integration of demonstrations into lectures |
VPL aimed at allowing embedding of F90, HPF and MPI (etc.) examples in lectures and convenient support of homeworks for transient inexperienced users. |
Features of VPL:
|
User registers data in Java Applet and running HPF/MPI program and transfers between client applet and running simulation in a fashion similar to AVS |
This interacts via wrappers to MPI/HPF/F90 running programs |
VRML naturally gives 3D visualization with usual Web advantage of running on PC's and Workstations |
Its universality implies can use in industry to specify products so can design, manufacture and market from the same(related) specification |
Should impact PDES/STEP and such industry product specification standards |
VRML will need extension to handle this but it is a good start and allows user defined types |
VRML and Parallel Computing?
|
NPAC Web Based Geographical Information System in Stand Alone Mode |
A GIS application is a specialized OpenInventor viewer, however it accepts any OpenInventor 2.1 scene model. That's why it's so easy to integrate it with third party applications, which produce IO/VRML output. The images show GIS integration with Weather Simulation application. A GIS viewer can also display animated objects controlled by Simulation Engine. |
This is "middleware" which is implemented in simplest form as a network of Java Servers
|
Access |
Resources |
Store |
Multimedia Information |
Collaboration Server |
File Systems |
and/or Database |
Object Broker |
Database |
Simulation (Network-enabled |
servers such as NEOS, Netsolve) |
Sequential |
or Parallel |
Computer |
Can use network of Web Clients and/or Web Servers |
Not clear if distinction (in capability) between web server and client will remain |
Web Client Models Include SuperWeb from UCSB and hotwired article "Suck your Mips". |
More powerful but less pervasive is a pure Web Server model as in NPAC WebFlow |
Can either use in controlled (IntraNets or run a server on every node of your MPP) or uncontrolled (the whole world wide web) fashion
|
Note total compute power in all Web "clients" is about 1000 times that in all Central Supercomputers |
http://www.packet.com/packet/ Hot Wired Tuesday January 7 Edition |
Applet calculates pi while you read article! |
High Level WebHPL (Interpreted Interface to parallel Java, Fortran, C++) |
or WebFlow (AVS on the Web) |
Low Level WebVM (MPI on the Web) is linked servers |
We suggest a hierarchical computing model with high functionality modest performance integration/coordination layer controlling classic low level HPCC high performance layer |
At Lowest layer hardest problem is decomposition |
At Top (computing) layer, flexible Integration is |
problem as decomposition natural |
User Interface gives third level |
WebFlow gives an AVS like Interface to integrate multilanguage modules |
A C++ PCRC SPMD Physics simulation is linked with disparate modules to filter output display |
This demo is available on a video tape and with RealAudio narration |
see Real Audio Scripted Discussion of PCRC and WebFlow |
This model can be combined with emerging Web Collaboration, Object broker and database capabilities to support multidisciplinary applications and computational steering |
Link CFD and CSM -- each of which codes has steps |
Input --> Mesh Generation --> Partitioning --> MPI based simulation --> Post Processing |
CFD |
COCOLIB communication library supplies Data and Control |
Structures |
MPI Processes of individual programs |
Partition Individual Grids |
COCOLIB (CISPAR Runtime Library)Interpolates between grids of 2 programs |
Natural Choice: |
Optimal High Performance Choice |
Data and Control |
Data Only |
CFD Server |
Structures Server |
Control |
Only |
Simplest Server level Implementation |
Structures Server |
CFD Server |
A large number of new Java APIs for advanced Web Services are emerging from JavaSoft and partners. Many current problems such as with the NPS WebDIS networking will be soon solved in a more robust, stable and elegant fashion by the new high-level APIs. |
Java APIs are organized in Java Frameworks. Current list of frameworks include:
|
Java Security Framework - support for authentication encryption, digital signatures. |
Java Commerce Framework - Java Wallet, Java Cassettes (digital credit cards). |
Java Beans Framework - componentware API with support for GUI negotiation and merging, persistence (JAR files), event filtering, introspection, visual application builders |
Java Media Framework - Java2D (with Adobe), animation (with Macromedia), audio/video (with Intel), Java3D (with SGI), JSDA (Java Shared Data API). |
All APIs listed are either already operational within JDK1.1, or still in works (spec only or alpha or beta release) but with the final release dates in 1997. |
Numerical Computing Framework
|
Distributed Simulation Framework
|
Computing Services Framework
|
High Performance Framework
|
Distributed Computing Framework
|
Enables development of Web Interfaces to run a given job on any computer compliant with this framework just as JDBC gives a universal interface to any relational database |
Compiling, Executing |
Scheduling jobs as in Codine or LSF |
Accessing and storing into File Systems |
Visualization Interface (how applets access server side information) |
The Computing Services Framework allows vendors to compete on User Front End (GUI) or back end services |
The framework is implemented as a set of drivers which map generic Java Interfaces to particular software (e.g. a compiler) on particular machines. |
We proposed an evolutionary process of developing user-friendly visual component based authoring tools for HPCC by integrating the current C++ and coming Java based parallel object technologies via CORBA with the JavaBeans based visual component technologies. |
Our proposed process starts from the existing parallel C++ object libraries and provides HP-ORBlet based interoperability with Java. |
Next, the CORBA/Enterprise Beans based server/backend side component technologies are incorporated in the Java server and adapted to the HPCC needs within the emergent Java Frameworks for Computing. |
Finally, the BeanBox based front-end visual tools provided by the industry are being linked via client-side ORBlets with the tier-2+3 components to offer visual authoring of HP-Components and HPCC applications. |
Examples are given from DoE HPCC Technologies |
Large gains in HPCC user productivity will be attained if we can integrate the ideas and technologies of modern (PC) visual programming with classical HPCC approaches |
Use of important emerging Web and CORBA technology allows HPCC object (C++.,Java) and visual (CODE, Hence, WebFlow, AVS, Khoros) systems to be enhanced to become parallel component-based visual programming systems. |
CORBA does not incorporate HPCC but as it specifies services and not implementation, we assert that one can build a high performance HP-CORBA environment where we separate a high functionality modest performance control layer from an optional high performance method invocation layer. |
HP-CORBA can be built on Nexus and Globus and it will allow HPCC users access to any CORBA service with an option for high performance when necessary. |
The NPAC WebFlow technology can be combined with emerging JavaBean technology to produce a prototype HPcomponent system which can be used with DoE HPCC object based systems such as POOMA and PETSc. |
We present the proposed process of integrating HPCC and Enterprise Computing technologies on 2-dimensional 3x3 chart, exposing the evolution of user and system perspectives on new computing technologies. |
On the (vertical) system/complexity axis, we start from PC desktop and we evolve towards distributed and finally HPCC computing. |
On the (horizontal) user/simplicity axis, we start from objects and we evolve towards reusable components and finally their visual development and runtime environments. |
The first row on our char represents the already established PC technologies exemplified by systems such as Visual C++/J++, VBScript, Borland Delphi, Visual Cafe etc. |
The second row corresponds to the emergent Object Web based Enterprise Computing that integrates Java/JavaBeans based component technologies with CORBA based distributed objects. |
Finally, the third row represents our proposed HPCC extensions of these technologies in terms of HP-CORBA based HP-Components. |
JavaBeans are Java objects with well defined methodology for providing hooks that enable design or even runtime composition. |
Beans connectivity is implemented in terms of events, posted by the source beans and selectively delivered to the destination beans that expressed interest in specific event types. |
Interests are expressed, either programmatically or dynamically in the design- or runtime, in terms of specialized objects - event listeners - that are created by (human or machine) bean connector and then specialized and registered with the destination beans. |
Bean design patterns for naming accessor methods and other control elements provide a uniform support for dynamic bean manipulation both by programmers and by authoring tools (beanboxes). |
At the moment, the most popular component technology at the PC desktop level comes from Microsoft in terms of OLE/ActiveX controls but the much more powerful and modern JavaBeans technology gains rapid acceptance by the software industry. |
Beans can be manipulated programmatically but their most efficient use is within visual development or runtime environments. |
A typical PC desktop example is a GUI builder in which both the control buttons and the application-specific computation and graphics modules are represented as beans and suitably positioned and connected in the visual interactive mode. |
At the moment, JavaSoft offers a simple BeanBox called BDK but a broad suite of more sophisticated and powerful IDEs is under development by the Web industry. |
Most notable BeanBox called InfoBus, developed by Lotus and used in their Kona product for Beans based electronic presentation tools will become part of future JDK releases. |
Another interesting BeanBox technology called BeanConnect will be offered by Netscape and will provide HTML/JavaScript support for bean manipulation and connectivity. |
At present, JavaBeans technology is focused on component programming within a single Java VM. |
Within the Sun philosophy of '100% Java', distributed Beans can be developed using RMI interconnect. However, the rest of the industry tries to protect their C++ investments while converting to Java. |
Hence, in parallel with JavaBeans development, the Web industry explores now the linkage of Java with CORBA based distributed object technologies which offer an full C++/Java interoperability. |
CORBA supports cross-language remote object invocation as well as IIOP (Internet Inter-ORB Protocol) based interoperability between object brokers from various vendors. |
Of particular interest are Java based ORBs or ORBlets which can be downloaded as applets to enable CORBA capabilities also at the client/browser side. |
An alternative, offered by Netscape, is a resident ORB support in all browser and server products. |
Java based ORBs will soon turn the Web, so far acting as a largely passive document publishing framework, into a powerful dynamic world-wide distributed object-based computing environment. |
We now start the discussion of the proposed HPCC extension and adaptation of the Object Web Architecture. In the object domain (chart column), we propose the extension of the CORBA model for the HPCC domain by developing a minimal high performance ORB on top of MPI/Nexus. |
Such HP-ORBlets, residing in the individual nodes of a parallel system (either as network daemons or as runtime libraries) would allow to hide explicit message passing calls in terms of higher level more user-friendly remote object invocations. |
HP-ORBlets would focus on high performance data/method communication support, whereas the lower bandwidth control communication would be passed to and handled by the full functionality lower performance commercial ORBs in the CORBA bus in the middleware layer. |
The split between data and control communication would be fully transparent at the programmatic level, i.e. both parallel object developers and integrators would be offered a uniform CORBA object based programming model. |
Two major advantages of the proposed approach are:
|
Although we have 384 foilsets imported to the Web -- this is not the correct way of doing it!
|
Originally we thought a "Web Foil" is an enhanced HTML and we built a prototype using HotJava |
However now believe this is not powerful and correct Web implementation of "foils" is a JavaBean stored as a serialized Java Object
|
Start with a full drawing program supporting "art" and importing of images where basic entities are Java objects |
Start with system integrated with Tango |
Basic Unit is a "foil" -- typically NOT scrolled (but supporting this) |
Foils are arranged into foilsets dynamically as in current WebWisdom and allow notes(addons) etc. |
Import existing PowerPoint/Persuasion from "outline" text with automatic font size scaling
|
Allow user to add highlighting and images "cut" from original screendump of PC foil |
Latest Tango Capabilities June 97 |
User Interpreted Commands |
invoking preloaded |
Java/Javascript objects |
Server invokes extrinsic |
processor |
(true interpreter |
or compiler invoked |
dynamically) |
Running Program |
interruptable at |
extrinsic interfaces |
Compiler offers:
|
Interpreter offers:
|
JAVA applet |
HTTP |
server |
HPF |
server |
Node 1 |
Node 2 |
Node 3 |
Node 4 |
Node 5 |
Node 6 |
Web Browser |
Server Host |
1. Contact HTTP server to download the Java applet |
2. Use the applet to start instrumented HPF application |
3. HPF server starts (implemented as an HPF extrinsic) |
4. Java applet establishes communication with the HPF server |
5. HPF server accepts user's requests: |
- suspend/resume execution of the HPF code |
- send data |
- interpret new HPF statements |
- dynamically link and execute shared objects |
HPF statements and Visualization requests in Java, HPF etc. |
HPF extrinsic procedure |
Shared Object |
Lexical Analysis: PCRC frontend. |
HPF compilation: pghpf |
Pool of shared objects |
HPF RTS |
HPF |
server |
Dynamical linking |
Instrumented HPF |
JAVA |
applet |
Applications requires a range of capabilities in any language |
High level ("Problem Solving Environment") manipulating"large" objects
|
Intermediate level Compiled Code targetted at "sequential" (multi-threaded) architecture
|
Lower level runtime exploiting parallelism and memory hierarchies
|
Numerical Objects in (C++/Fortran/C/Java) |
Expose the Coarse Grain Parallelism |
Expose All Levels of Memory Hierarchy |
a) Pure Script (Interpreted) |
c) High Level Language but Optimized Compilation |
d) Machine Optimized RunTime |
b) Semi- Interpreted |
a la Applets |
Memory Levels in High |
Performance CPU |
Nodes of Parallel/ Distributed System |
Java is basis of Web Collaboration Systems with Applets Coordinated by Java Server |
Habanero from NCSA was one of first |
TANGOsim uses more modern Web Technology and incorporates a Discrete Event Simulator |
TANGO is a software framework supporting computer-based communication and collaboration
|
TANGO supports client side electronic societies of people, instruments and their applications. |
TANGO links these to a network of Java Servers which manage TANGO sessions and interfaces them to the world wide Server Infrastructure that underlies our model of Web applications
|
Electronic societies or groups are managed by core database in TANGO |
Systems like Tango or Habanero built around Java Servers integrate a group of multiple clients as a "Service" at the middle Java Server level |
Group of collaborating clients |
and client applications |
Database |
Object Broker |
MPP |
Technically, TANGO is a distributed system based on event broadcasting
|
Core system is implemented in Java and as a browser plug-in |
Base programming module is a Java applet
|
From Tango - A Java/WWW-Based Internet Collaborative Software System part of NPAC Overview May 1997 |
From Tango - A Java/WWW-Based Internet Collaborative Software System part of NPAC Overview May 1997 |
Global architecture is a mesh of servers
|
Session metaphor is a meeting room with tools
|
Asynchronous collaboration is (will be) enabled via session recording to database backend |
TANGO modules can be downloaded from anywhere
|
Application select |
buttons |
Application |
Domains |
Open/close and |
floor control |
Session |
information |
User information |
TANGO session, sync, and |
floor control buttons built |
into the application control |
panel. |
Generic tools: chats (1D, 2D, 3D*), whiteboards (paint and draw) |
Multimedia tools: audio/video conferencing, collaborative indexed streaming video |
Shared browsers: slide shows, fully synchronized, shared JavaScript, customizable client-side apps |
Virtual University: WebWisdom - interactive contents delivery system, a set of applets for physics curriculum |
Domain specific applets: 2D and 3D (collaborative VRML) Visible Human, collaborative AutoCad (feasibility study only) |
Core system implemented by a team of 3 programmers within ~6 months, available.
|
APIs for Java, C/C++, and JavaScript available |
Two generations of GUI implemented |
"Lessons learned" are being incorporated into TANGO II, scheduled for release end of October
|
Basic architectures
|
Basic architectures
|
Sharing possible only if applications deterministic
|
Copies of applications must be present everywhere
|
Environments must be identical
|
Initial state of a newcomer impossible to define
|
Cannot maintain consistency among copies
|
The pros win! Web model enables event broadcasting! |
TANGOsim |
Basic |
Replicated Applications |
1)Virtual Users 2)Customized Views |
TANGO Java |
Collaboratory |
Server |
HTTP |
Server |
MultiMedia Mail |
C2 Commander |
Chat |
VTC |
Event Driven |
Simulation |
Engine |
C2 Radar Officer |
3D GIS |
Scripting |
Language |
C2 Weather Officer |
Message Routing |
SW/Data Distrib. |
Other |
Collaborators |
MultiMedia Mail |
Chat |
Simulation |
Engine Controller |
All Clients |
Typical Clients |
Feb 97 Demonstration of Tango |
From Tango Project for CEWES Collaborative Tool Meeting |
TANGO links people and shared applications such as chat board, audio video conferencing, visualizations, shared white board, common AUTOCAD design and related tools |
CFD |
TANGO Server |
Database |
Object Broker |
MPP |
Structures |
MPP |
Engineer |
+ core |
services |
Visualization e.g.CAVE |
Shared AutoCAD |
Engineer |
+ core |
services |
This combines TANGO for collaboration with WebFlow to link server side applications |
If necessary WebFlow would support high performance inter-module communication as in structures-CFD Linkage example but it would always implement control and this allows TANGO integration with server side computation
|
WebFlow communication model is a dynamic dataflow |
Of course other server side compute models are possible and in general need (web-linked) data bases, file systems, object brokers etc., |
WebFlow supports dataflow model where user must supply routines to process input of data that drives module and output of data for other modules |
TANGO supports shared state and user supplies routines that read or write either
|
Can be done for applications like AUTOCAD as vendor supplies necessary API |
CFD |
Structures |
We have a set of Services hosted by Web Servers and accessed by clients |
Groups of clients (electronic societies) are linked by collaboration systems such as TANGO |
Access |
Resources |
Store |
Multimedia Information |
TANGO Server |
File Systems |
and/or Database |
Object Broker |
Database |
Simulation |
Computer |
Person2 |
Shared |
WhiteBoard |
Shared Client Appl |
Person1 |
General User |
Clearly Java Collaboration Systems are natural implementations of general environments that mix computers and people |
Computational Steering -- a simulation is like a participant in a Tango session which has
|
Need to link to Tango, Java data analysis/visulaization front ends as well as distributed resource management systems such as ARMS from Cornell |
Note synergy with Java Server based distributed computing such as WebFlow which builds an AVS like environment with graphical interfaces to software Integration |
More ambitious to upgrade discrete event simulation component of TANGOsim to support full SIMNET/DSI (Distributed Simulation Internet) functionality. |
Note that Java is natural language for DSI/Forces Modelling because these typically use object parallelism which fits both language and applet/JavaBean capabilities. |
See discussion in http://www.npac.syr.edu/projects/javaforcse |
There is a web-based DIS project at the Naval Postgraduate School (NPS) at Monterey, CA, led by Don Brutzman who is NPS faculty, member of VRML Consortium and Moderator of newly formed dis-java-vrml mailing list. |
The goal of the project is to explore Java for DIS protocol implementation and networking, VRML2 for visualization of the simulated virtual DIS worlds, and Web browsers for the GUI packaging and integration. |
The current NPS experiment uses exclusively Java, VRML and HTML to implement a proof-of-the-concept demo. In a more complete Web DIS, the actual simulation engines will be likely provided by the optimized legacy codes, suitably interfaced to Java based communication and VRML based rendering. |
Front-end is provided by Netscape running one vehicle simulation applet on each node and by Cosmo Player VRML2 browser displaying the multi-vehicle simulated world. |
Java applet propagates local vehicle and communicates via the bridge daemon with other nodes, sending local vehicle PDUs and reading remote vehicles PDUs. |
Java applet and VRML browser interact via EAI (External API) as specified by SGI/Cosmo Player add-on to the VRML2 protocol. JavaScript is used to acquire handle to the VRML scene and LiveConnect is used to pass this handle to the Java applet. |
Applet performs dynamic real-time updates of the scene graph based on local vehicle simulation and remote vehicle locations received via PDUs. |
This prototype is successful and demonstrates that one can use Web technology for Distributed Event Driven Simulation -- the major use of simulation in DoD |
Web'97 is already quite elaborate at the client/browser side but still rather simplistic at the server side (base HTTP with CGI extensions). |
New Java Frameworks (Enterprise, Media) bring new family of Java servers that can implement an object based middleware and backend layers for the Web based services including computing. |
One critical development in the distributed objects domain is the IIOP (Internet Inter-ORB Protocol), mandatory in CORBA2 and offering a new server-to-server communication standard between commercial ORBs from various vendors. |
At the moment, Web Server market is dominated by customized or/and proprietary systems but there is a new strong tendency to standardize the Web middleware around the CORBA paradigm. |
'Java based ORB' is a CORBA2 compliant ORB written exclusively in Java, and hence fully portable and ready to run also on the Wintel platform, where distributed objects were so far under control of Microsoft DCOM/ActiveX model. |
The new emergent Object Web framework includes Web servers in the middleware, Java applets in the front-end and Java ORBs on both sides. |
Client-side ORB (or ORBlet) can be either downloaded as part of the applet, or resident on the client host, or embedded in the browser (as planned by Netscape). |
Applet downloads via HTTP and connects via ORBlet's IIOP to a middleware ORB. This middleware ORB can be either another server or part of the Web server. |
Middleware ORB connects via IIOP to other ORBs on the CORBA bus. This mechanism enables global connectivity via CORBA white/yellow page services such as Naming Service or Trader Service and their federations. |
Client-side ORBlet is also a lightweight server - hence clients can be called by servers via the CORBA callback mechanism. This enables interactive feedback, asynchronous notification and a variety of collaboratory frameworks over the CORBA bus. |
DoD modeling community is currently evolving towards the HLA(High level Architecture) framework with the RTI (Run Time Infrastructure) based communication bus. |
The goal of HLA/RTI is to enhance interoperability across more diverse simulators than in the DIS realm, ranging from real-time to time-stepped to event-driven paradigms. |
HLA defines a set of rules governing how simulators (federates) interact with each others. Federates describe their objects via Object Model Template (OMT) and agree on a common Federation Object Model (FOM). |
The overall HLA/RTI model is strongly influenced by the CORBA architecture and in fact the current prototype development is indeed CORBA based. |
We next step is to combine CORBA2 (Initial HLA/RTI is CORBA1) with NPS prototype ideas to give a fully object and Web integrated event driven simulation environment. |
Java3D is natural visualization environment in this scenario |
Our HPCC Server model is consistent with the Object Web trends |
The servers provide high functionality CORBA2 services with an event based control mechanisms |
A new HPCORBA data transfer layer implements high performance object method invocations |
Current WebFlow functionality, limited to classical dataflow, can be extended to support also visual authoring of CORBA business objects and in a component based software approach, WebFlow will be a CORBA vertical framework aimed at distributed computing. |
JavaBeans are the base software components which have a visual programming model which links ideas in systems such as Hence CODE AVS WebFlow with modern software engineering practice. |
In summary, we propose HPCORBA based WebFlow with JavaBeans front-ends as an effective delivery platform for integrating HPCC and DoD Modeling, Simulation and Testing environments with modern high productivity software development environments. |
The following figure illustrates a possible CORBA-HPCC integration framework ( HPCORBA ) |
Each node of a parallel machine runs a lightweight (e.g. Nexus based ) ORBlet which enables both local and remote object services. |
Coming CORBA3/MOM based messaging and object-by-value frameworks adapted to provide high-performance (e.g. MPI based) object-oriented (IIOP based) inter-node communication. |
Parallel Computation graphs are represented by Interface Definition Language (IDL) in a language independent way and optimized by parallel compiler technologies. |
Specific language bindings would result in HPF, HPC++ or HPJava implementations. |
This framework provides interoperability with the emergent Object Web software industry and offers an elegant solution for scalable parallel I/O ( linking HPCORBA with Oracle CORBA ) |
Java for User Interfaces and MetaComputing is natural from its design! |
Java for your favourite Conjugate Gradient routine (etc.) is less obvious ..... |
Java likely to be a dominant language as will be learnt and used by a broad group of users
|
Java may replace C++ as major system building language
|
Clearly Java can easily replace Fortran as a Scientific Computing Language as can be compiled as efficiently and has much better software engineering (object) and graphics (web) capabilities
|
Java can unify classic science and engineering computations with more qualitative macroscopic "distributed simulation and modelling" arena which is critical in military and to some extent industry |
Key question is performance of Java |
Note Web Software can be run on High Performance IntraNets such as Iway so hardware need NOT be a problem! |
Java is currently semi-interpreted and (as in Linpack online benchmark) is about 50 times slower than good C or Fortran
|
Java --> (javac)--> Downloadable Universal Bytecodes --> (Java Interpreter) |
--> Native Machine Code
|
However Language can be efficiently compiled with "native compilers" |
Java ----> (native compiler) |
---> Native (for Particular Machine) Code |
Lots of Interesting Compiler issues for both compiled and scripted Java |
My SGI INDY gets .54 Megaflops for Java 100 by 100 Linpack |
It has 200 Mhz R4400 and current Netlib benchmark for this chip is 32 mflops for optimized Fortran |
For better resolution see JPEG Version |
see http://www.netlib.org/benchmark/linpackjava/ |
Note Just in Time Compilers are giving a factor of 10 from June 96 Measurements! |
see http://www.netlib.org/benchmark/linpackjava/ |
Syracuse Workshop saw no serious problem to High Performance Java on sequential or Shared Memory Machines |
Some restrictions are needed in programming model |
For instance, Avoid Complicated Exception handlers in areas compilers need to optimize! |
Should be able to get comparable performance on compiled Java C and Fortran starting with either Java Language or JavaVM bytecodes |
The Interpreted (Applet) JavaVM mode would always be slower than compiled Java/C/Fortran -- perhaps by a factor of two with best technology |
One can use "native classes" which is just a predownloaded library of optimized runtime routines which can be high performance compiled Java, C, C++, Fortran, HPF etc. modules invoked by interpreted or compiled Java
|
Use Native Classes selectively for
|
1)Classic solution of large scale PDE or Particle dynamics problem
|
2)Modest Grain size Functional Parallelism as seen in overlap of communication and computation in a node process of a parallel implementation.
|
3)Object parallelism seen in Distributed Simulation where "world" modelled (typically by event driven simulation) as set of interacting macroscopic (larger than grid points) objects
|
4)MetaProblems consisting of several large grain functionally distinct components such as
|
Java: 1) Not Supported, 2) is Thread mechanism, 3) is Java Objects or Applets, 4) is JavaBeans or equivalent |
Fortran: 1)is supported in HPF, 2--4) are not supported |
As we saw large scale Applications need many forms of parallelism and it is not needed/appropriate to use the same mechanism for each form
|
Thus Java needs (runtime and perhaps language) extension to support HPF/HPC++ like (shared memory model for programmer) data parallelism but "Java plus message passing" is already here
|
Java Wrappers (native classes or Server socket connections) around existing data parallel Fortran or C++ |
Native Java and MPI
|
Data Parallel Extensions of Java
|
Java threads for data parallelism on SMP's |
Interpreted but limited (in functionality) Java client interface to Java wrapped HPF/C++ (not necessarily and perhaps best not parallel Java)
|
Note that we avoid many difficulties but lose elegance as we exchange information between the Host and running Parallel code using "text strings" |
Host and parallel node "synchronize" object reference by registering names with the communication broker |
This does not necessarily need one to use Java native class linkage to Fortran and C -- rather just to be able to send messages between running programs |
PreProcessors Can make this more "automatic"
|
More generally should study link between interpreted and compiled environments
|
Need an Interpreted Java -- JavaScript is interpreted but in limited domain |