Full HTML for
Given by Geoffrey C. Fox, Bill asbury, Ken Flurchick, Tom Haupt at DoD MetaSystems Workshop Charlottesville Va. on August 11-13 99. Foils prepared August 16 99
Outside Index
Summary of Material
| We define Gateway project with its key features and initial implementation through SC99 |
| We make some general remarks on technology /worldview of Gateway as an object web distributed computing systems stressing the role of XML, multi-tier architecture and multiple object models |
| We describe relationship to Forces Modeling and the HLA/RTI Standards |
| We describe collaboration in Gateway-- both synchronous and asynchronous -- in terms of event federation |
| http://www.osc.edu/~kenf/theGateway/ |
Denote Foils where Image Critical
Denote Foils where Image has important information
Denote Foils where HTML is sufficient
Overview of Gateway Project http://www.npac.syr.edu/users/gcf/gatewayoverviewaug99 Bill Asbury, Ken Flurchick, Geoffrey Fox, Tom Haupt
Abstract of Gateway Overview
Gateway: Some People and Organizations
Gateway High Level Overview
Gateway Architecture
Gateway Software Hierarchy
Gateway Top Level Requirements
Initial Gateway Capabilities -- SC99
Gateway Project Timeline
Gateway Project in a Nutshell
LMS Goals and Front-End
WebFlow on Globus -- LMS at CEWES
What does Gateway do for you?
Another Gateway Prototype: Quantum Chemistry PSE for NCSA Alliance
Technical Assumptions
Gateway View of invoking a program
Basic Multi Tier architecture
Proxy -- Proxy -- Backend Capability
Implementation Architecture
Some Observations on Gateway
XML and the Front End
ScienceML
Scientific Notepad
Gateway Programming Model
Gateway Support of Simple Standalone Codes
Multi Tier Computing Model
Gateway Support of Simple Standalone Codes
Gateway Technologies
Gateway Architecture
Gateway System
Gateway Front End
Gateway PSE Example: CCM
Current Gateway Interface
Gateway Middle Tier Services
Middle Tier Objects
WebFlow in Gateway
File Operations from the Control Applet
Gateway Security Model
Gateway and IMT and FMS
File Formats I
File Formats II
Collaboration I
Collaboration II
Collaboration and Gateway I
Collaboration and Gateway II
Application Issues I
Application Issues II
Outside Index
Summary of Material
HTML version of Basic Foils prepared August 16 99 
Full HTML Index
| NPAC |
| 111 College Place |
| Syracuse NY 13244 4100 |
| Phone: 3154432163 |
| DoD HPCMP Metasystems Workshop |
| August 11-13 1999 |
| Charlottesville Va. |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| We define Gateway project with its key features and initial implementation through SC99 |
| We make some general remarks on technology /worldview of Gateway as an object web distributed computing systems stressing the role of XML, multi-tier architecture and multiple object models |
| We describe relationship to Forces Modeling and the HLA/RTI Standards |
| We describe collaboration in Gateway-- both synchronous and asynchronous -- in terms of event federation |
| http://www.osc.edu/~kenf/theGateway/ |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| OSC NPAC Nicholls Gateway project funded by ASC (and next year ARL) |
| Architecture, Vision etc.: Geoffrey Fox |
| Project Management at ASC: Bill Asbury (Nicholls) |
| CCM and General Front end Tools: Ken Flurchick, OSC |
| Middle Tier Implementation: Tom Haupt, NPAC |
| Original WebFlow technology and FMS applications: Wojtek Furmanski (NPAC) |
HTML version of Basic Foils prepared August 16 99 
Full HTML Index
| Goal: Provide DoD with a secure, problem oriented interface to more effectively utilize HPC resources from the desktop and Web browser. |
| Architecture for growth and changing needs: |
| commodity based |
| highly modular |
| extensible |
| Metasystems environment that supports: |
| multidisciplinary problem solutions |
| All CTA's including FMS and IMT |
| collaborative tools |
| scientific visualization |
| Heterogeneous HPC seamless access: |
| underlying complexities hidden |
| science focused |
| expert guidance |
| support database, compute etc resources |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Problem Description |
| Framework |
| Middleware |
| Core Grid Services |
| Resources |
| CAT |
| Gateway |
| c.f.: Juan Villacis, et.al., "CAT, High Performance, Distributed Component Architecture Toolkit..." |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Reliance on Standards is Key for Gateway Design and Implementation Efforts. |
| Driven by User Requirements |
| CTA Independent |
| Kerberos/SecurID Authentication |
| Interactive/Batch Jobs |
| Import/Export of Data Files |
| Information Facilities for Resources and Data Files |
| Multi-disciplinary Capability |
| Browser/Web Server Independent |
| Modular low maintenance Design |
| Batch System Independent |
| HPC Platform Independent |
| Mass Storage Independent |
| Uniform access to training and help material |
| Support User Service functions e.g. Job Accounting/Allocation |
| Compliant with Emerging National Standards |
| Consistent with Metaqueuing |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Prototype implementation on ASC prototyping system
|
| Utilize Kerberos and SecurID for security |
| Utilize minimal flat file implementation for information services (possibly include RIB) |
| Use current ASC allocation structure |
Demonstrate single CTA PSE for CCM
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Apr |
| May |
| Jun |
| Jul |
| Aug |
| Oct |
| Nov |
| Dec |
| Sep |
| Middleware Pre-alpha Release - OSC |
| CCM PSE Pre-alpha Release - OSC |
| Gateway Workshop |
| DoD Users Group |
| MAPINT99 |
| Middleware Alpha Release - All |
| Final API Spec |
| CCM PSE Alpha Release - ASC |
| Internal Demo - ASC |
| ITEA |
| Middleware Beta Release - All |
| CCM PSE Beta Release - ASC |
| SC99 Internal Demo -ASC |
| SC99 |
| Design & Interface Spec Documents |
| DoD Meta Computing Workshop |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Builds on Web-based problem solving environment built for ERDC PET: LMS Landscape Management System |
| Core technology synergistic with ARL and CEWES FMS PET WebHLA distributed object middleware |
| NCSA Alliance would call Gateway a Portal; others would call it a Problem Solving Environment (PSE) |
Initial project will produce
|
| Core system in alpha test today and SC99 will have fully functional demonstration |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Web based Interface Supporting
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| WebFlow |
| server |
| WebFlow |
| server |
| WebFlow |
| server |
| EDYS |
| CASC2D |
| Data Retrieval |
| High Performance SubSystem |
| CASC2D |
| proxy |
| IIOP |
| Web Browser |
| Data Wizard |
| WMS interface |
| Toolbar |
| HTTP |
| WMS |
| File Transfer |
| File Transfer |
| GLOBUS |
| Internet |
| WebFlow modules |
| (back-end) |
| WebFlow |
| middle-tier |
| WebFlow applet |
| (front-end) |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Toolkit to build high level application specific web based problem solving environments
|
| Simple strategy to convert existing codes into distributed objects that can be seamlessly launched |
| Simple default job status displays |
| Architecture naturally supports collaboration (using shared events and TangoInteractive) and visualization if structured as a multi tier service conforming to XML interfaces |
| Compositional Programming of modules |
MPI HPF etc programming tools are either outside Gateway but could be given integrated web interface as part of Gateway -- we demonstrated this (DARP) for HPF in early WebFlow work
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| User Interface will be through a Web Browser and be dynamic (generated server side) or static XML documents rendered as HTML |
| Commodity systems built from integration of Distributed Object and Web technologies |
| CORBA, Java, COM, W3C have approaches that differ in detail but we can use any of these if define object structures in XML and map to implementation (which was Java and is now CORBA but using component model based on Javabeans) |
One can avoid performance problems in CORBA by using proxies so that control and implementation separated
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Similar to invoking a web page |
| Java CORBA or WIDL (pure XML CGI specification) |
| Object Broker |
| Fortran Simulation Code on Sequential or |
| Parallel Machine |
| Convert Generic Run Request into Specific Request on Chosen Computer |
| Fortran Program |
| invoked from wrapper object |
| and Globus |
| W3C Web Page |
| (XML/XHTML) Rendering of Backend Object |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Objects (at "logical backend") can be on client of course |
| Front end can define a generic (proxy for a) object. The middle control tier brokers a particular instantiation |
| Broker or Server |
| XML |
| Result |
| XML Query |
| Rendering Engine |
| Browser |
| Rendering Engine |
| HTML |
| Universal Interfaces |
| IDL or Templates |
| XML Request for service |
| followed by return of XML result |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| The Proxies and actual instantiation are linked by messages whose semantic content is defined (best) in XML |
| The lower system level format can be HTTP RMI IIOP or ... |
| The client proxy is for rendering input and output including specification of object |
| The middle tier proxy allows choice of backend provider and functional integration (the user can specify integration at client proxy level) |
| Real Capability |
| XML |
| XML |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Between XML/HTML document and backend Fortran, MPP, Database, Scientific Instrument defined by XML Resource definition is a bunch of servers linked by XML Messages |
| Servers can either be commercial Web Servers (Apache with servlets), CORBA brokers or custom Servers and can if necessary support multiple object models simultaneously (JWORB) |
| Java/CORBA/WIDL Wrapper with IDL in XML |
| XSL Style Sheets and |
| XML/HTML Page Design |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Note Gateway supports application area (i.e. CTA's or other natural units) -- not just applications |
| Initial investigation from ASC suggests current CCM approach will generalize to other CTA's |
| Can in principle support Palmtop, PC, Workstation, CAVE front-ends |
Gateway does not define grid services but builds on them to provide Problem Solving Environments
|
| Linkage of Gateway to Legion needs more study |
| Applications should be defining XML data structures and "wrapping" existing applications as objects "anyway" |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
The front end is some document consisting of a mix of HTML or XML
|
| We supply suitable XML widgets which PSE designer puts together in different ways for different PSE's |
| XML can be rendered for PC's or Palmtops or .... |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| This we define as a group of defined formats that support scientific data, note taking and sketches |
XSIL (Scientific data Interchange) defines metadata needed to specify scientific data files including high level parameters and methods needed to read data
|
| VML is Vector Graphics Mark up Language |
| DrawML is designed to support simple technical drawings (easier than VML but VML should be able to do this?) |
| VRML (3D scenes) reimplemented in XML as X3D |
| MathML Mathematical Expressions |
| ChemML Support Chemistry |
| DoDHPCML Supports HPCMP Program and has extensions to support particular CTA's and service groups (programming tools, visualization etc.) |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Presumably this allows Scientists to make notes and record thoughts in a way that it supports important scientific constructs |
At its simplest this is an authoring tool like Microsoft Word, PowerPoint or Framemaker
|
| One useful utility would be a whiteboard that supported scientific notes using ScienceML |
Such a collaborative whiteboard (implemented in Tango for instance) would be useful in research and teaching
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Computing abstracted as a set of hierarchical Toolbars Toolbars are defined in XML and rendered in HTML for user interface. XML interpreted on middle tier as some suitable service. |
| Toolbars can access a direct middle-tier service or a "good-old HPCC tool" accessed via a middle-tier proxy (debugging, performance etc.) |
| Computing Toolbars include user profile, application area (CTA choice), code choice (application specific), results, visualization (where "command" could be AVS), collaboration, programming model, (HPF, Dataflow specialized tools), resource specification, resource status, |
HTML version of Basic Foils prepared August 16 99 
Full HTML Index
| Core capabilities allow simple wrapping of existing codes |
| Program objects is defined as an XML file e.g. my favorite physics simulation on my PC becomes: <program name="physicssimulation1"> <run domain="npac" machine="maryland" type="pc" os="nt" >c:\myprogs\a.out</run> <input button="prog1" toolbar="physics" type="htmlform" > <name>userinput</name> <field default="10" maximum="100000" >iterations</field> ............ </input> <output> ...</output> </program> |
Gateway generates a distributed object with
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Database |
| Matrix Solver |
| Optimization Service |
| MPP |
| MPP |
| Parallel DB Proxy |
| NEOS Control Optimization |
| Origin 2000 Proxy |
| NetSolve Linear Alg. Server |
| IBM SP2 Proxy |
| Grid Gateway |
| Supporting Seamless Interface |
| Agent-based Choice of Compute Engine |
| Multidisciplinary Control (WebFlow) |
| Gateway Programming Environment |
| Good Old Programming Tools |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Core capabilities allow simple wrapping of existing codes |
| Program objects is defined as an XML file e.g. my favorite physics simulation on my PC becomes: <program name="physicssimulation1"> <run domain="npac" machine="maryland" type="pc" os="nt" >c:\myprogs\a.out</run> <input button="prog1" toolbar="physics" type="htmlform" > <name>userinput</name> <field default="10" maximum="100000" >iterations</field> ............ </input> <output> ...</output> </program> |
Gateway generates a distributed object with
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Built with multi tier architecture using secure CORBA and sophisticated WebFlow Java front end allowing task composition
|
| Kerberos built on Secure CORBA today but PKI Security fine |
| All object interfaces defined in XML and so front end totally independent of middle tier implementation |
| Supports Dataflow and more general object based computing model (executing objects linked by events) |
| Middle Tier Components based on Javabean model |
| Custom Middle-tier event model consistent with security |
| Middle Tier is wrappers (proxies) and so insensitive to performance problems in CORBA or RMI |
| XML Interface between Middle Tier and backend -- object model for backend not required |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
HTML version of Basic Foils prepared August 16 99
Full HTML Index
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Invokes methods of the control applet |
| Middle Tier |
| SECIOP secure (kerberized) connection |
| Select tools |
| Select PSE |
| Navigate through PSE |
| Access to Middle Tier Services |
| Navigator |
| HTML pages |
| (forms and results) |
| Control Applet |
| submit |
| Generates result pages |
| Web Browser |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Ken Flurchick, http://www.osc.edu/~kenf/theGateway |
| Gateway Computing Process |
| 1. Enter the Gateway system |
| 2. Define your problem |
| 3. Identify resources (software and hardware) |
| 4. Create input file |
| 5. Run your application |
| 6. Analyze results |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
HTML version of Basic Foils prepared August 16 99
Full HTML Index
File service
|
Job services
|
Object Services
|
Database services
|
| Visualization (under design based on ARL DICE) |
| Collaboratory (under design based on Tango architecture) |
| Mass storage services (planned based on Globus/Grid Forum) |
| User Services (planned based on Globus/Grid Forum) |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| User |
| Profile |
| Result Descriptor |
| Problem |
| Descriptor |
| Task Descriptor |
| Abstract Task Descriptor |
| Job Descriptor |
| User/Group Profile |
| Project |
| Problem Descriptor |
| Abstract Task Descriptor |
| Archive |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| WebFlow server is given by a hierarchy of containers and components |
| These are CORBA objects written in Java acting if necessary as proxies to backend resources |
| WebFlow server hosts users and services |
| Each user maintains a number of applications composed of custom modules and common services |
| WebFlow supports both object based and dataflow computing model with visual interface at client and both tasks and their interrelationship defined in XML |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| SECIOP |
| Front End Applet |
| SECIOP |
| Kerberos authentication |
| and authorization |
| Gatekeeper |
| delegation |
| HPCC resources |
| GSSAPI |
| GSSAPI |
| Layer 1: secure Web |
| Layer 2: secure CORBA |
| Layer 3: Secure access to resources |
| Policies defined by resource owners |
| Can use Public Key Infrastructure but for HPC Modernization |
| use SecurID and Kerberized Workstation |
| Globus |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| The distributed object Middle Tier in Gateway supports the Java/CORBA version of HLA (DMSO's distributed object model) and RTI (DMSO's Run Time Infrastructure to supported distributed modeling and simulation) built by NPAC |
So one can naturally build applications that federate HPCC applications with FMS simulations
|
IMT requires real time data streams and databases
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Gateway should support input and output files with certain characteristics including |
| Native: internal format known to a particular application -- no checking of format matching if link output of one module to input of another |
parameterinput: special file constructed in XML defining basic parameters needed by a batch job
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Third file format is
|
Gateway modules need to specify nature of their input and output files and what type of linkage of modules is allowed or even required.
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Collaboration implies sharing of electronic objects and is needed in asynchronous and synchronous modes |
Asynchronous mode is
|
Notification and event update service
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Notification mechanism enhances collaboration as enables dynamic federation of relevant objects to be automatically maintained |
| Notification can be used for people so their presence on-line can be made known to those in particular collaborative users |
| Users decide if notification causes an active signal (send electronic mail, ring a buzzer) or passively alters a list on a web page. |
Event Model unifies synchronous and asynchronous models of collaboration
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Shared Objects need to accept data from Gateway compliant applications
|
| Gateway/Tango event service federates different "event domains" |
| Local Event/Message Bus |
| Collaboration == Sharing Event |
| in "Tango" Server |
| Local Event/Message Bus |
| Local Event/Message Bus |
| Gateway Events |
HTML version of Basic Foils prepared August 16 99
Full HTML Index
| Whiteboard and Shared Browser can be loaded with files and data from Gateway compliant systems |
More generally consider any client side rendering of a gateway system -- either data input or (visualization/data) output
|
Examples from Tango
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Classic Multidisciplinary Meta-Applications:
|
Power Users
|
Distributed Simulations
|
HTML version of Basic Foils prepared August 16 99
Full HTML Index
Distributed/Real Time Information and Visualization Systems
|
More Efficient Job Allocation Across Sites
|
Portals/Seamless Access
|
"Metasystem in a Box"
|