Given by Geoffrey C. Fox at DSWA HPCC REquirements Workshop TASC Building Rosslyn VA on 20 February 98. Foils prepared 21 February 98
Outside Index
Summary of Material
We discuss role of commodity (Web) technologies in future high performance computing environments |
We describe how a network of Web/CORBA/COM servers architecture can naturally support both parallel and distributed computing while |
We describe applications to both metacomputing, and parallel computing and suggest this approach can provide attractive user environments leveraging the huge commodity investment |
We suggest critical importance of CORBA and component based software in HPCC -- Javabeans seem very important |
This approach unifies distributed event driven simulations with classic massively parallel time stepped computations and so combines DMSO HLA/RTI with classic HPCC |
We describe role of collaboration technology in linking computers with people |
We finish with remarks on University requirements and issues in HPCC connected to research in HPCC, use of HPCC systems and education |
Outside Index Summary of Material
Geoffrey Fox |
Syracuse University |
NPAC |
111 College Place Syracuse NY 13244 4100 |
3154432163 |
We discuss role of commodity (Web) technologies in future high performance computing environments |
We describe how a network of Web/CORBA/COM servers architecture can naturally support both parallel and distributed computing while |
We describe applications to both metacomputing, and parallel computing and suggest this approach can provide attractive user environments leveraging the huge commodity investment |
We suggest critical importance of CORBA and component based software in HPCC -- Javabeans seem very important |
This approach unifies distributed event driven simulations with classic massively parallel time stepped computations and so combines DMSO HLA/RTI with classic HPCC |
We describe role of collaboration technology in linking computers with people |
We finish with remarks on University requirements and issues in HPCC connected to research in HPCC, use of HPCC systems and education |
The two forms of Large Scale Computing Scale Computer for Scale Users in Proportion Power User to number of computers |
Parallel Commodity Distributed Computers Information Systems Technology <--------------- Internetics Technologies ---------------> |
Parallel Computer Distributed Computer |
HPF MPI HPJava HTML VRML |
HPCC has developed good research ideas but cannot implement them as solving computing's hardest problem with 1 percent of the funding
|
We have learnt to use commodity hardware either
|
Let us do the same with software and design systems with maximum possible commodity software basis |
The world is building a wonderful distributed computing (information processing) environment using Web (dissemination) and distributed object (CORBA COM) technologies |
This includes Java, Web-linked databases and the essential standards such as HTML(documents), VRML(3D objects), JDBC (Java database connectivity).
|
We will "just" add high performance to this commodity distributed infrastructure
|
The alternative strategy starts with HPCC technologies (such as MPI,HPF) and adds links to commodity world. This approach does not easily track evolution of commodity systems and so has large maintenance costs |
The current incoherent but highly creative Web will merge with distributed object technology in a multi-tier client-server-service architecture with Java based combined Web-ORB's |
COM(Microsoft) and CORBA(world) are competing cross platform and language object technologies
|
Need to abstract entities (Web Pages, simulations) and services as objects with methods(interfaces) |
How do we do this while infrastructure still being designed! |
One can anticipate this by building systems in terms of Java objects e.g. develop Web-based databases with Java objects using standard JDBC (Java Database Connectivity) interfaces |
Even better use Javabeans which are Java's componentware offering visual interfaces, containers (here they are consistent with CORBA standard) and standard software engineering interfacing rules |
By definition, Object Web software is and will even more so, be the "best" software ever built because it has the largest market and greatest leverage of investment dollars
|
As on next foil, we take technology produced for broad-based object web and customize in three ways
|
This allows you to both deliver your application to the general public (when required) and leverage best software |
Essential idea is consider a three tier model
|
Preserve the first two tiers as a high functionality commodity information processing system and confine HPCC to the third (lowest) tier.
|
Middle Server Tier |
Basic HTTP/CGI Web Server |
Java Web Server |
Transaction Processing Server |
Business Transaction Management |
Client and Middle Tier are Customizable |
You Write Web Technology Software |
at Client and Server |
Old and New Useful Backend Software |
Object Broker |
Back-end Tier |
The Services |
Client |
Front-end Tier |
W is Web Server |
PD Parallel Database |
DC Distributed Computer |
PC Parallel Computer |
O Object Broker |
N Network Server e.g. Netsolve |
T Collaboratory Server |
Clients |
Middle Layer (Server Tier) |
Third Backend Tier |
1)One can "just" use Object Web technologies as a software infrastructure for building parallel, distributed or sequential computing environments which can have a very different architecture from the Web
|
2)Harness the power of the Web as a computer -- use up the idle cycles on the WebTV's in every home -- typically a Web Client based system
|
3)One can view the Object Web as a distributed information system with modest performance and build a metacomputing system with the Web architecture
|
Larry Smarr and NCSA Collaboration have stressed analogy of deployment of computer/communication technology with impact that electrical and transportation grids had
|
The transportation system was built using lessons from and feed up/down from Sports cars, Cadillacs, Model T's, Ford Escorts etc. |
Computational Grid will be shaped by and shape all applications and technologies |
Internetics expresses synergy between high-end and commodity approaches |
A computational grid is a metacomputer or a "high performance distributed computer system" which must be influenced by and influence the "Object Web" which is here defined as "mass-market"/business IntraNet (low to low) use of Internet/distributed Information Systems |
Applications are metaproblems with a mix of module and data parallelism |
Modules are decomposed into parts (data parallelism) and composed hierarchically into full applications.They can be the
|
Modules are "natural" message-parallel components of problem and tend to have less stringent latency and bandwidth requirements than those needed to link data-parallel components
|
Assume that primary goal of metacomputing system is to add to existing parallel computing environments, a higher level supporting module parallelism
|
It is natural to base on either a network of Web Clients or Web Servers
|
Web Client Models Include SuperWeb (Javelin) from UCSB and are well illustrated by the January 1997 hotwired article "Suck your Mips". |
Greater functionality but less power and pervasiveness is a pure Web Server model as proposed by NPAC
|
Note total compute power in all Web "clients" is about 100 times that in all Central Supercomputers |
Object Web Software provides a high functionality but modest performance distributed computing (Metacomputing) environment based on either Web (soon to be CORBA IIOP and HTTP/Java Socket) Servers or Clients |
Here we will explore an architecture using servers for control as higher functionality than clients although currently less broadly deployed
|
Object Web Only addresses Integration of already decomposed parts!
|
1:User View: Interoperable Web Interface accessing services through Java Compute Services Framework |
2:Network of Java Servers provide distributed services with databases, compute engines, collaboratories, object brokers, instruments
|
Back end "Number Crunchers" linked either by communication at level 2 (slowish but easy) or at level 3 (high performance but more work) |
Compute processes linked either to servers or together by MPI if parallel |
Java Servers |
Here are some examples of using our approach where large scale industry investment in Web technology appears to add significant value to metacomputing systems built with Web architecture
|
Multidisciplinary and Computational Steering Applications
|
Visual and Interpreted Programming Environments
|
Technologies to get High Performance CORBA |
Integration with Forces Modeling (Distributed Event driven Simulation) |
Integration with Networked enabled servers such as NEOS and Netsolve
|
1)Simple Server Approach 2)Classic HPCC Approach |
3)Hybrid Approach with control at server and |
data transfer at |
HPCC level |
Client (Tier 1): Java Graph Editor for Webflow or interpreted debugger (DARP) linked to Java Visualizer SciViz |
Middle Tier 2: Network of Java Servers linking UNIX and Windows NT systems with "all" services |
Back-end Tier 3: Globus where available. In early 98, this is high performance UNIX system links with no databases and no NT |
Note this is a good high performance I/O architecture where file system, CORBA or database based |
After architecture foil, next 4 Foils show
|
Original Image |
Output Image |
Some of |
Available Image Filters |
Visual DataFlow |
Interface |
Simulation |
Basic Display |
Image Filter |
is another |
module |
Output Display after Filter |
Runs as a |
parallel |
module |
using |
Java Server |
host |
A screen dump from a Scivis Session |
HPcc can be applied to either parallel or high-performance distributed computing (aka metacomputing) |
In metacomputing, HPcc fills a void as few if any high level tools |
In parallel computing, HPcc provides uniform and perhaps more attractive sustainable user environment |
Can view a parallel computer either as a single tier 2 object
|
Both are interesting
|
This is classic host-node computing model |
Host is logically distinct but can be on same machine as a "node" |
The "Host" is logically a separate Corba object but could of course be instantiated on the same computer as one or more of the nodes. Using the protocol bridge of fig. 15, one could address objects using Corba with local parallel computing nodes invoking MPI and remote accesses using Corba where its functionality (access to very many services) is valuable. |
From HPcc as High Performance Commodity Components |
YES! If one uses the same separation between control and data transfer explained for metacomputing case |
Build a "bridge" that accepts MPI HTTP or CORBA invocation but invokes either the powerful slow CORBA mechanism or the nifty optimized MPI |
Why address nodes as CORBA? -- so you can build applications uniformly so they can access nodes and servers around the world in same message passing style |
Why address nodes with MPI? -- so you can get code that executes very fast! |
Why address nodes with HTTP? -- so you can get advantages of CORBA today as Web Servers dominate! |
This allows MPI (or equivalently Nexus, Globus or PVM) and commodity technologies to coexist with a seamless user interface. |
From HPcc as High Performance Commodity Components |
Need Flexibility of HTTP/RMI/COM/CORBA with performance of systems like MPI |
Can use compiler or runtime libraries to map in application (tier 3) which gives greatest performance |
More flexible is to map at server (tier 2) when you can access functionality of all mapped protocols |
NPAC is building JWORB which is a Java Web server which can bridge the Web and CORBA protocols
|
Commercial systems can bridge COM and CORBA |
DoD modeling and simulation (FMS,IMT) community is currently evolving towards the HLA(High level Architecture) framework with the RTI (Run Time Infrastructure) based communication bus. |
The goal of HLA/RTI is to enhance interoperability across more diverse simulators than in the DIS realm, ranging from real-time to time-stepped to event-driven paradigms. |
HLA defines a set of rules governing how simulators (federates) interact with each others. Federates describe their objects via Object Model Template (OMT) and agree on a common Federation Object Model (FOM). |
The overall HLA/RTI model is strongly influenced by the CORBA architecture and in fact the current prototype development is indeed CORBA based. |
Building HPCC on the Object Web implies that we can a common framework for both distributed (event driven) simulations and classic time stepped parallel computing |
Currently WebFlow uses a Java Server and manipulates Java applications which can be frontends with native methods to Fortran C or C++ routines |
Change Java Server to JWORB -- server integrating HTTP and IIOP (Web and CORBA) |
Change Java Applications to JavaBeans |
Change linkage in WebFlow to respect JavaBean conventions |
Then we get HPComponentware |
And using our multitier model high performance CORBA |
WebFlow is HPCC version of a |
Typical Visual Interface for JavaBeans |
They are Java's implementation of "component-based" visual programming |
This modern software engineering technique produces a new approach to libraries which become a "software component infrastructure(SCI)" |
There is a visual interface to discovery of and setting of values of and information about parameters used in a particular software component |
JavaBeans uses the event model of JDK1.1 to communicate between components
|
One expects Javabeans to become the CORBA component interface (defining containers in CORBA) |
The visual interface allows inspection of and implementation of both individual beans and their linkage . This visual construction of linkage allows one to form nontrivial programs with multiple communicating components
|
Apart from the event mechanism which is a communication/linkage mechanism, ComponentWare (and JavaBeans in particular) "just" give a set of universal rules (needed for interoperability) for rather uncontroversial (albeit good) object-oriented and visual programming practices
|
CORBA is natural distributed object formalism |
Java (with visual interfaces i.e. JavaBeans) is natural interface language
|
Linking this to tier 3 "classic applications" gives rise to JavaBean/CORBA wrappers for existing applications |
This turns legacy applications into CORBA distributed objects and so can be remotely executed and documented (via CORBA trader or yellow pages service) |
Further these applications now have a visual interface for linking them together in containers and inspecting their parameters |
A 2 Tier implementation is shown above |
The CORBA wrapper uses IDL for language of original (legacy) application (use CORBA C IDL for Fortran) |
One designs an IDL to reflect "application class" and re-uses it for several elated applications |
Javabean frontend can be same for each application class |
Encapsulate services (from databases to instruments to MPP's) as tier 2 distributed objects using an approach that will evolve to COM/CORBA in future but is deployable today
|
This already gives you an approach to seamless access and a framework for composing complex metaproblems by linking programs together or programs to databases |
Habanero or Tango allow you to share objects to give
|
We describe Tango which like Habanero is based on a tier-2 Java Server |
We first summarize overall design and capabilities |
Then one foil illustrating basic capabilities (chat, whiteboard etc.) |
Then two foils illustrating with 2D and 3D geographical information systems, the original command and control (crisis management) application |
Then a discussion of multidisciplinary applications using Tango and WebFlow |
An Applet based system using LiveConnect and plugin with Netscape3 and Signed Applets with Netscape4 |
Supports general shared event model of collaboration where it can share applications in Java, JavaScript, C, VRML, C++ (Open Inventor)
|
Has conventional general tools
|
Developed for command and control |
Most extensively used in education -- especially for course between Syracuse and Jackson State
|
From Tango - A Java/WWW-Based Internet Collaborative Software System part of NPAC Overview May 1997 |
Feb 97 Demonstration of Tango |
From Tango Project for CEWES Collaborative Tool Meeting |
TANGO links people and shared applications such as chat board, audio video conferencing, visualizations, shared white board, common AUTOCAD design and related tools |
CFD |
TANGO Server |
Database |
Object Broker |
MPP |
Structures |
MPP |
Engineer |
+ core |
services |
Visualization e.g.CAVE |
Shared AutoCAD |
Engineer |
+ core |
services |
This combines TANGO for collaboration with WebFlow to link server side applications |
If necessary WebFlowcould support high performance inter-module communication as in structures-CFD Linkage example but it would always implement control at middle tier and this allows TANGO integration with server side computation
|
WebFlow communication model is a dynamic dataflow |
Of course other server side compute models are possible and in general need (web-linked) data bases, file systems, object brokers etc., |
On client one can share tools such as CAD systems like CATIA or AUTOCAD so Tango interfaces with API to these system and drives "slaves" from state extracted from linkage to master. |
WebFlow supports dataflow model where user must supply routines to process input of data that drives module and output of data for other modules |
TANGO supports shared state and user supplies routines that read or write either
|
Can be done for applications like AUTOCAD as vendor supplies necessary API |
CFD |
Structures |
Wearable computers (with components such as battery, disks, CPU) wore on a belt with a handheld monitor (640 by 480 today) allow us to bring a computational grid to the warfighter |
There would be a wireless link using low speed digital cellular connection |
There is enough memory and disk to run a browser and Tango client on the remote unit |
This links warfighters together or to central intelligence source |
The 1997 technology is shown in next two foils |
Dave Warner demonstrates wearable computer 9pm Feb 16 98 -- key to new BotMasters Project |
Dave Warner demonstrates wearable computer 9pm Feb 16 98 -- key to new BotMasters Project |
Universities interact with HPCC in three distinct ways
|
The two forms of Large Scale Computing Scale Computer for Scale Users in Proportion Power User to number of computers |
Computational Commodity Technology Science Web, Networking, Databases ... |
<--------------- Internetics ---------------> |
Parallel Computer Distributed Computer |
Emerging field centered on technologies services and applications enabling and enabled by world wide communication and computing grids |
The contents come from Computer Communication and Information science fields but with an applied flavor so forms critical knowledge needed by many application fields such as scientific computing, telemedicine, electronic commerce, digital journalism and education |
Students with an interdisciplinary background will be encouraged |
The applied focus with many totally new and rapidly evolving technologies makes Internetics unique |
K-12 is Middle and High School Students |
These 2 courses must be passed to obtain Certificate
|
See NPAC's Java Academy at http://www.npac.syr.edu/projects/k12javaspring98/ |
Graduate and Continuing Education have same curriculum with 4 core and 2 electives needed for certificate |
Core Courses (total 4 courses) |
(There will also be a "booster course" offered to students who have taken the Undergraduate certificate so they can "place out" of graduate core course)
|
Need to take 2 electives |
Computer Science Electives
|
Application Electives:
|
Reduce cost and increase quality of curriculum and its delivery by
|
Most important in near term for remote students and new rapidly changing fields which are typically not available |
In long term can impact nature of educational enterprise |
Important Questions:
|
Answers to questions probably depends on field and student body |
Taught using current Tango/WebWisdom over Internet (now DREN via CEWES) every Tuesday and Thursday from Syracuse
|
Jackson State major HBC University with many computer science graduates |
Can now offer addon-on courses with "leading edge" material (Web Technology) which give JSU graduates skills that are important in their career
|
Needs guaranteed 30 (audio) to 100 (two way) kilobits per second bandwidth assuming course material mirrored at JSU
|
NPAC Web Server |
JSU Web Server |
JSU Tango Server |
... |
Audio Video Conferencing Chat Rooms etc. |
Teacher's View of Curriculum Page |
Student's View of Curriculum Page |
JavaScript |
JavaScript and Perl |
TANGO Server Client Chat etc. Java |
NPAC CGI Server |
Log of Access to Curriculum Pages |
Perl |
ICWU will join 6 or more universities together in fall 98 to use distance education across the world to teach Internetics at Graduate (6 courses) and High School ( 2 courses)
|
Internetics is the study of technologies, services and applications enabling and enabled by the world wide Internet
|
Will prototype and test education technologies, infrastructure and demonstrate the feasibility of new approaches to education using International Collaboration |
There are many universities/schools where such "augmentation would be valuable
|
Experience with teaching in China suggests huge demand in foreign universities where faculty less up to date in latest material |
Repeating course at JSU spring 98 semester with improved curricula and somewhat improved bandwidth |
Hope to produce a distance education version of grade 5-12 version of Web Technologies for kids offered February-April 98 in Syracuse -- http://www.npac.syr.edu/projects/k12javaspring98/ |
Jackson State will use Tango to teach CSC499 like course to other HBCU's next fall |
Note base material used in about 20 different courses/tutorials over last 2 years
|