Given by Geoffrey C. Fox at Computational Aerosciences in the 21st Century Hampton VA on April 23 98. Foils prepared 21 April 98
Outside Index
Summary of Material
We review emerging approaches to HPCC for both tightly coupled parallel hardware and computational grids -- geographically distributed metacomputers. |
We base the discussion on the growing power and capability of commodity computing and communication technologies largely driven by commercial distributed information systems. These systems are built from CORBA, Microsoft's COM, Javabeans, and less sophisticated web server and networked approaches. |
One can abstract these to a three-tier model with largely independent clients connected to a distributed network of servers. The latter host various services including object and relational databases and of course parallel and sequential computing. |
This network of servers encompassing disparate object and web standards is termed the "Pragmatic Object Web" and we show how to use these concepts even when there is no one universal standard. |
We look at the implications for educational curricula in "non computer science technical fields" |
Outside Index Summary of Material
Geoffrey Fox |
Syracuse University |
NPAC |
111 College Place Syracuse NY 13244 |
Phone:3154432163 |
ICASE/LaRC/NSF/ARO WORKSHOP ON |
COMPUTATIONAL AEROSCIENCES IN THE 21st CENTURY April 23 98 http://www.npac.syr.edu/users/gcf/cas21stapril98 http://www.icase.edu/workshops/CAS21st.html |
We review emerging approaches to HPCC for both tightly coupled parallel hardware and computational grids -- geographically distributed metacomputers. |
We base the discussion on the growing power and capability of commodity computing and communication technologies largely driven by commercial distributed information systems. These systems are built from CORBA, Microsoft's COM, Javabeans, and less sophisticated web server and networked approaches. |
One can abstract these to a three-tier model with largely independent clients connected to a distributed network of servers. The latter host various services including object and relational databases and of course parallel and sequential computing. |
This network of servers encompassing disparate object and web standards is termed the "Pragmatic Object Web" and we show how to use these concepts even when there is no one universal standard. |
We look at the implications for educational curricula in "non computer science technical fields" |
The two forms of Large Scale Computing Scale Computer for Scale Users in Proportion Power User to number of computers |
Parallel Commodity Distributed Computers Information Systems Technology <--------------- Internetics Technologies ---------------> |
Parallel Computer Distributed Computer |
HPF MPI HPJava HTML VRML |
HPCC has developed good research ideas but cannot implement them as solving computing's hardest problem with 1 percent of the funding
|
We have learnt to use commodity hardware either
|
Let us do the same with software and design systems with maximum possible commodity software basis |
The world is building a wonderful distributed computing (information processing) environment using Web (dissemination) and distributed object (CORBA COM) technologies |
This includes Java, Web-linked databases and the essential standards such as HTML(documents), VRML(3D objects), JDBC (Java database connectivity).
|
We will "just" add high performance to this commodity distributed infrastructure
|
The alternative strategy starts with HPCC technologies (such as MPI,HPF) and adds links to commodity world. This approach does not easily track evolution of commodity systems and so has large maintenance costs |
Basic Vision: The current incoherent but highly creative Web will merge with distributed object technology in a multi-tier client-server-service architecture with Java based combined Web-ORB's |
COM(Microsoft) and CORBA(world) are competing cross platform and language object technologies
|
Need to abstract entities (Web Pages, database entries, simulations) and services as objects with methods(interfaces)
|
How do we do this while infrastructure still being designed! |
Major Commercial Java Activity today is on Server NOT Client |
One can anticipate this by building systems in terms of Java objects e.g. develop Web-based databases with Java objects using standard JDBC (Java Database Connectivity) interfaces
|
Even better use (Enterprise) Javabeans which are Java's (middle tier) or client componentware offering visual interfaces, containers (here they are consistent with CORBA standard) and standard software engineering interfacing rules
|
Confused? Read "Building Distributed Systems on the Pragmatic Object Web" -- Book of class I teach to CS/CE students at Syracuse http://www.npac.syr.edu/users/shrideep/book |
Essential idea is consider a three tier model
|
Preserve the first two tiers as a high functionality commodity information processing system and confine HPCC to the third (lowest) tier.
|
Middle Server Tier |
Basic HTTP/CGI Web Server |
Java Web Server |
Transaction Processing Server |
Business Transaction Management |
Javabean |
Enterprise Javabean |
Old and New Useful Backend Software |
Object Broker |
Back-end Tier |
The Services |
Client |
Front-end Tier |
Client |
Middle Tiers |
Back End |
Thin Client |
W is Web Server |
PD Parallel Database |
DC Distributed Computer |
PC Parallel Computer |
O Object Broker |
N Network Server e.g. Netsolve |
T Collaboratory Server |
Clients |
Middle Layer (Server Tier) |
Third Backend Tier |
HPcc is High Performance commodity computing |
Encapsulate services (from databases to instruments to MPP's) as middle tier distributed objects using an approach that will evolve to COM/CORBA in future but is deployable today
|
Establish Java Frameworks and Equivalent CORBA Facilities
|
This already gives you an approach to seamless access and a framework for composing complex metaproblems by linking programs together or programs to databases |
Use of Java for: |
High Performance Network Computing |
Scientific and Engineering Computation |
(Distributed) Modeling and Simulation |
Parallel and Distributed Computing |
Data Intensive Computing |
Communication and Computing Intensive Commercial and Academic Applications |
HPCC Computational Grids ........ |
Very difficult to find a "conventional name" that doesn't get misunderstood by some community! |
Java has potential to be a better environment for "Grande application development" than any previous languages such as Fortran and C++ |
The Forum Goal is to develop community consensus and recommendations for either changes to Java or establishment of standards (frameworks) for "Grande" libraries and services |
These Language changes or frameworks are designed to realize "best ever Grande programming environment" |
First Meeting Mar 1 Palo Alto at Java 98 -- 200 Attendees set Agenda -- 30 permanent people with next meeting May 9-10 |
Around 3 more meetings this year -- Draft Report SC98 |
http://www.npac.syr.edu/projects/javaforcse |
1)One can "just" use Object Web technologies as a software infrastructure for building parallel, distributed or sequential computing environments which can have a very different architecture from the Web
|
2)Harness the power of the Web as a computer -- use up the idle cycles on the WebTV's in every home -- typically a Web Client based system
|
3)One can view the Object Web as a distributed information system with modest performance and build a metacomputing system with the Web architecture
|
Larry Smarr and NCSA Collaboration have stressed analogy of deployment of computer/communication technology with impact that electrical and transportation grids had
|
The transportation system was built using lessons from and feed up/down from Sports cars, Cadillacs, Model T's, Ford Escorts etc. |
Computational Grid will be shaped by and shape all applications and technologies |
Internetics expresses synergy between high-end and commodity approaches |
A computational grid is a metacomputer or a "high performance distributed computer system" which must be influenced by and influence the "Object Web" which is here defined as "mass-market"/business IntraNet (low to low) use of Internet/distributed Information Systems |
Applications are metaproblems with a mix of module and data parallelism |
Modules are decomposed into parts (data parallelism) and composed hierarchically into full applications.They can be the
|
Modules are "natural" message-parallel components of problem and tend to have less stringent latency and bandwidth requirements than those needed to link data-parallel components
|
Assume that primary goal of metacomputing system is to add to existing parallel computing environments, a higher level supporting module parallelism
|
It is natural to base on either a network of Web Clients or Web Servers
|
Web Client Models Include SuperWeb (Javelin) from UCSB and are well illustrated by the January 1997 hotwired article "Suck your Mips". |
Greater functionality but less power and pervasiveness is a pure Web Server model as proposed by NPAC
|
Note total compute power in all Web "clients" is about 100 times that in all Central Supercomputers |
Object Web Software provides a high functionality but modest performance distributed computing (Metacomputing) environment based on either Web (soon to be CORBA IIOP and HTTP/Java Socket) Servers or Clients |
Here we will explore an architecture using servers for control as higher functionality than clients although currently less broadly deployed
|
Object Web Only addresses Integration of already decomposed parts!
|
Here are some examples of using our approach where large scale industry investment in Web technology appears to add significant value to metacomputing systems built with Web architecture
|
Multidisciplinary and Computational Steering Applications
|
Visual and Interpreted Programming Environments
|
Technologies to get High Performance CORBA |
Integration with Forces Modeling (Distributed Event driven Simulation) |
Integration with Networked enabled servers such as NEOS and Netsolve
|
1)Simple Server Approach 2)Classic HPCC Approach |
3)Hybrid Approach with control at server and |
data transfer at |
HPCC level |
4)Invoke High Performance Message Transfer between Observers and Sources specified in Message Event |
Server Tier |
Data Source |
Data Sink (Observers) |
5)Actual Data Transfer |
High Performance Tier |
2)Prepare |
Message Event in Source Control |
1)Register Observers with Listener |
Client (Tier 1): Java Graph Editor for Webflow or interpreted debugger (DARP) linked to Java Visualizer SciVis
|
Middle Tier 2: Network of Java Servers linking UNIX and Windows NT systems with "all" services |
Back-end Tier 3: Globus where available. In early 98, this is high performance UNIX system links with no databases and no NT |
Note this is a good high performance I/O architecture whether file system, CORBA or database based |
After architecture foil, next Foils show
|
Client Tier |
IIOP High Functionality |
Middle Tier |
Future Globus |
Globus |
Future Parallel I/O |
Original Image |
Output Image |
Some of |
Available Image Filters |
Visual DataFlow |
Interface |
Simulation |
Basic Display |
Image Filter |
is another |
module |
Output Display after Filter |
Runs as a |
parallel |
module |
using |
Java Server |
host |
Client Tier 1 |
Middle Tier 2 IIOP |
Tier 3: VIA,MPI-IO,Globus |
High Performance Messaging |
High Functionality Messaging |
Use Event Mechanism of Javabeans to invoke High |
performance third tier data transfer from middle tier data control |
HPcc can be applied to either parallel or high-performance distributed computing (aka metacomputing) |
In metacomputing, HPcc fills a void as few if any high level tools |
In parallel computing, HPcc provides uniform and perhaps more attractive sustainable user environment |
Can view a parallel computer either as a single tier 2 object
|
Both are interesting
|
This is classic host-node computing model |
Host is logically distinct but can be on same machine as a "node" |
The "Host" is logically a separate Corba object but could of course be instantiated on the same computer as one or more of the nodes. Using the protocol bridge of fig. 15, one could address objects using Corba with local parallel computing nodes invoking MPI and remote accesses using Corba where its functionality (access to very many services) is valuable. |
From HPcc as High Performance Commodity Components |
YES! If one uses the same separation between control and data transfer explained for metacomputing case |
Build a "bridge" that accepts MPI HTTP or CORBA invocation but invokes either the powerful slow CORBA mechanism or the nifty optimized MPI |
Why address nodes as CORBA? -- so you can build applications uniformly so they can access nodes and servers around the world in same message passing style |
Why address nodes with MPI? -- so you can get code that executes very fast! |
Why address nodes with HTTP? -- so you can get advantages of CORBA today as Web Servers dominate! |
This allows MPI (or equivalently Nexus, Globus or PVM) and commodity technologies to coexist with a seamless user interface. |
From HPcc as High Performance Commodity Components |
Need Flexibility of HTTP/RMI/COM/CORBA with performance of systems like MPI |
Can use compiler or runtime libraries to map in application (tier 3) which gives greatest performance |
More flexible is to map at server (tier 2) when you can access functionality of all mapped protocols |
NPAC has built JWORB which is a Java server which supports both the Web and CORBA(distributed object) protocols
|
Commercial systems can bridge COM and CORBA |
Integrates protocols for inter-tier communication
|
CORBA - JWORB is an OMG compliant ORB written in Java and acting as middleware in the 3-(or more)-tier CORBA applications |
HTTP - JWORB is a Web Server using CORBA services where possible |
COM - we are planning to implement COM/CORBA bridge as specified by OMG to enable interoperability between CORBA and DCOM objects |
RMI - JWORB is written in Java so the interface to the 100% Pure Java world is naturally available via RMI -- new RMI uses IIOP |
W3C - we intent to implement new Web Object Model (XML+RDF+DOM) using CORBA services in JWORB |
Messaging Performance |
Ping measured in milliseconds Good MPP/SMP MPI gives few -- 30 microseconds |
DoD modeling and simulation (FMS,IMT) community is currently evolving towards the HLA(High level Architecture) framework with the RTI (Run Time Infrastructure) based communication bus. |
The goal of HLA/RTI is to enhance interoperability across more diverse simulators than in the DIS realm, ranging from real-time to time-stepped to event-driven paradigms. |
HLA defines a set of rules governing how simulators (federates) interact with each others. Federates describe their objects via Object Model Template (OMT) and agree on a common Federation Object Model (FOM). |
The overall HLA/RTI model is strongly influenced by the CORBA architecture and in fact the current prototype development is indeed CORBA based. |
Building HPCC on the Object Web implies that we can a common framework for both distributed (event driven) simulations and classic time stepped parallel computing |
Currently WebFlow uses a Java Server and manipulates Java applications which can be frontends with native methods to Fortran C or C++ routines |
Change Java Server to JWORB -- server integrating HTTP and IIOP (Web and CORBA) |
Change Java Applications to JavaBeans |
Change linkage in WebFlow to respect JavaBean conventions |
Then we get HPComponentware |
And using our multi-tier model high performance CORBA |
WebFlow is HPCC version of a |
Typical Visual Interface for JavaBeans |
They are Java's implementation of "component-based" visual programming |
This modern software engineering technique produces a new approach to libraries which become a "software component infrastructure(SCI)" |
There is a visual interface to discovery of and setting of values of and information about parameters used in a particular software component |
JavaBeans uses the event model of JDK1.1 to communicate between components
|
One expects Javabeans to become the CORBA component interface (defining containers in CORBA) |
The visual interface allows inspection of and implementation of both individual beans and their linkage . This visual construction of linkage allows one to form nontrivial programs with multiple communicating components
|
Apart from the event mechanism which is a communication/linkage mechanism, ComponentWare (and JavaBeans in particular) "just" give a set of universal rules (needed for interoperability) for rather uncontroversial (albeit good) object-oriented and visual programming practices
|
CORBA is natural distributed object formalism |
Java (with visual interfaces i.e. JavaBeans) is natural interface language
|
Linking this to tier 3 "classic applications" gives rise to JavaBean/CORBA wrappers for existing applications |
This turns legacy applications into CORBA distributed objects and so can be remotely executed and documented (via CORBA trader or yellow pages service) |
Further these applications now have a visual interface for linking them together in containers and inspecting their parameters |
A 2 Tier implementation is shown above |
The CORBA wrapper uses IDL for language of original (legacy) application (use CORBA C IDL for Fortran) |
One designs an IDL to reflect "application class" and re-uses it for several elated applications |
Javabean frontend can be same for each application class |
Encapsulate services (from databases to instruments to MPP's) as tier 2 distributed objects using an approach that will evolve to COM/CORBA in future but is deployable today
|
This already gives you an approach to seamless access and a framework for composing complex metaproblems by linking programs together or programs to databases |
Habanero or Tango allow you to share objects to give
|
We describe Tango which like Habanero is based on a tier-2 Java Server |
We first summarize overall design and capabilities |
Then one foil illustrating basic capabilities (chat, whiteboard etc.) |
Then two foils illustrating with 2D and 3D geographical information systems, the original command and control (crisis management) application |
Then a discussion of multidisciplinary applications using Tango and WebFlow |
An Applet based system using LiveConnect and plugin with Netscape3 and Signed Applets with Netscape4 |
Supports general shared event model of collaboration where it can share applications in Java, JavaScript, C, VRML, C++ (Open Inventor)
|
Has conventional general tools
|
Developed for command and control |
Most extensively used in education -- especially for course between Syracuse and Jackson State
|
From Tango - A Java/WWW-Based Internet Collaborative Software System part of NPAC Overview May 1997 |
Feb 97 Demonstration of Tango |
From Tango Project for CEWES Collaborative Tool Meeting |
TANGO links people and shared applications such as chat board, audio video conferencing, visualizations, shared white board, common AUTOCAD design and related tools |
CFD |
TANGO Server |
Database |
Object Broker |
MPP |
Structures |
MPP |
Engineer |
+ core |
services |
Visualization e.g.CAVE |
Shared AutoCAD |
Engineer |
+ core |
services |
This combines TANGO for collaboration with WebFlow to link server side applications |
If necessary WebFlowcould support high performance inter-module communication as in structures-CFD Linkage example but it would always implement control at middle tier and this allows TANGO integration with server side computation
|
WebFlow communication model is a dynamic dataflow |
Of course other server side compute models are possible and in general need (web-linked) data bases, file systems, object brokers etc., |
On client one can share tools such as CAD systems like CATIA or AUTOCAD so Tango interfaces with API to these system and drives "slaves" from state extracted from linkage to master. |
WebFlow supports dataflow model where user must supply routines to process input of data that drives module and output of data for other modules |
TANGO supports shared state and user supplies routines that read or write either
|
Can be done for applications like AUTOCAD as vendor supplies necessary API |
CFD |
Structures |
Taught using Tango/WebWisdom over Internet (now DREN via CEWES) every Tuesday and Thursday from Syracuse
|
Jackson State major HBC University with many computer science graduates |
Do not compete with base courses but offer addon-on courses with "leading edge" material (Web Technology) which give JSU graduates skills that are important in their career
|
Needs guaranteed 30 (audio) to 100 (two way) kilobits per second bandwidth assuming course material mirrored at JSU
|
Hope to offer our analogous K-12 Java Academy to middle and high school students |
JSU will use Tango/WebWisdom to teach to others -- train the trainers |
Universities "specialize" and deliver courses in areas of expertise
|
JSU will lead HBCU wide deployment |
NPAC will attempt to use internationally |
Pilot for distance training of DoD users. |
NPAC Web Server |
JSU Web Server |
JSU Tango Server |
... |
Audio Video Conferencing Chat Rooms etc. |
Address at JSU of Curriculum Page |
Teacher's View of Curriculum Page |
Student's View of Curriculum Page |
Participants at JSU |
Teacher/Lecturer at NPAC |
WebWisdom |
Chat |
Shared Browser |
Audio Control |
Tango Control App |
Reduce cost and increase quality of curriculum and its delivery by
|
Most important in near term for remote students and new rapidly changing fields which are typically not available |
In long term can impact nature of educational enterprise |
Important Questions:
|
Answers to questions probably depends on field and student body |
Tango supports a synchronous shared event model of collaboration |
Tango supports (more than) enough (over 40) applications and our current activity is to evaluate, improve and make robust release |
Core Collaboration Capabilities
|
Databases Linkage
|
Office and Authoring Tools
|
General Virtual University Applications
|
Special Virtual University Applications
|
CONTROL |
Shared Planetary Motion Applet |
Shared Cross Product Educational Applet |
Shared Neural Net |
Have some fun with Multi-player games
|
"Other" Applications
|
CONTROL |
Shared VRML 3D Visible Human |
Shared Browser Page |
Chat |
Wearable computers (with components such as battery, disks, CPU) wore on a belt with a handheld monitor (640 by 480 today) allow us to bring a computational grid to the warfighter |
There would be a wireless link using low speed digital cellular connection |
There is enough memory and disk to run a browser and Tango client on the remote unit |
This links warfighters together or to central intelligence source |
The 1997 technology is shown in next two foils |
Dave Warner demonstrates wearable computer 9pm Feb 16 98 -- key to new BotMasters Project |
Dave Warner demonstrates wearable computer 9pm Feb 16 98 -- key to new BotMasters Project |
What's inside the Java Ring?
|
Controlled at JavaOne Conference by JavaSpace (Sun's Java version of Linda) for 14,000 way parallelism |
Prototype of eventual Intermental Network which link people (mind's) together with computers as "just the interfaces" |
Universities interact with HPCC in three distinct ways
|
Students -- correctly -- perceive a growing opportunity in computer science related fields but outside biology, there is a decrease in interest in "technical sciences" such as physics, aerospace engineering etc. |
In particular physics departments may disappear in many Universities as the number of majors is dropping at both undergraduate and graduate level. |
Classical Computational Science is not the answer but we suggest that a generalization -- Internetics at the interface between applications and "web/commodity" technologies offers interesting attractive academic programs combining computing and the "technical sciences" |
It is not enough to justify physics (as studying Latin and Greek was motivated to me) as "training the mind" |
So depending on the source, one finds a shortage of 100,000 to 300,000 workers in Information Technology today -- this is forecast to grow with 1 million more jobs created by industry by year 2004 |
So physics and "physical technology" aspects of engineering (e.g. aerospace engineering) could compete with this trend and try to attract good students from this field |
My suggested alternative is to note that IT work typically requires the technical and problem solving skills abilities associated with physics or engineering and often NOT taught in Computer Science |
Thus set up new curricula opportunities within the general IT educational arena that we call Internetics
|
Note IT job opportunities are in applications -- perhaps more so than in "basic systems" |
There is the same opportunity available to any education area to use new delivery and preparation methods
|
Comparing "books" with the Web, we see that Web offers opportunities for "technical people" as well as those with good "communication skills" -- Java applets combined with numerical algorithms may be more effective than streams of beautiful English words
|
The new technologies should allow better integration of research into education -- this could help academic fields communicate their value more effectively |
The two forms of Large Scale Computing Scale Computer for Scale Users in Proportion Power User to number of computers |
Computational Commodity Technology Science Web, Networking, Databases ... |
<--------------- Internetics ---------------> |
Parallel Computer Distributed Computer |
Emerging field centered on technologies services and applications enabling and enabled by world wide communication and computing grids |
The contents come from Computer Communication and Information science fields but with an applied flavor so forms critical knowledge needed by many application fields such as scientific computing, telemedicine, electronic commerce, digital journalism and education |
Students with an interdisciplinary background will be encouraged |
The applied focus with many totally new and rapidly evolving technologies makes Internetics unique |
K-12 is Middle and High School Students |
These 2 courses must be passed to obtain Certificate
|
See NPAC's Java Academy at http://www.npac.syr.edu/projects/k12javaspring98/ |
Graduate and Continuing Education have same curriculum with 4 core and 2 electives needed for certificate |
Core Courses (total 4 courses) |
(There will also be a "booster course" offered to students who have taken the Undergraduate certificate so they can "place out" of graduate core course)
|
Need to take 2 electives chosen from: |
Computer Science Electives
|
Application Electives:
|