Full HTML for
Given by Geoffrey C. Fox at ASC Year 2 Review Columbus on July 2 98. Foils prepared July 4 98
Outside Index
Summary of Material
Many of developments in modern Large Scale Enterprise Systems can enhance HPCC effectiveness by linking in productive services built around web, database and distributed object technologies
|
HPCC (MPP) systems are not necessarily optimized for implementing such services
|
Denote Foils where Image Critical
Denote Foils where HTML is sufficient
Gateway Systems Commodity Interface to HPCC Systems
Overview
Evolving Distributed System Concepts
Distributed Object Based Computing
Today's Pragmatic Object Web: The Confusing Multi-Technology Real World Middleware Server Layer
NPAC Concepts : HPcc High Performance commodity computing
NPAC Concepts: Pragmatic Object Web
Traditional 2 Tier Client Server Architecture
Typical 3 Tier Architecture
4 Tier Architecture for database example
Two ways of Implementing Data Objects
Multi-Server Model
So in our WebWisdom Distance Education System
While for High Performance Computing
Multi-Server Scenario
Multi Tier Gateway Architecture
What are General Capabilities in Gateway Tier?
What Particular Programs could run in Gateway Tier?
Remaining Foils Discuss 1) NPAC Specific Related Technologies JWORB and WebFlow 2) A Novel Approach to Resource Management based on HLA/RTI Concepts
Summary of NPAC's WebFlow
WebFlow WaveFilter Module
WebFlow SC'97 Demo
WebFlow over Globus at NCSA Application View
WebFlow + Globus Functional Architecture
WebFlow 3 Tier Architecture with High Performance Globus and High Functionality IIOP Gateway Layer Services
Summary of NPAC's JWORB
JWORB - Java Web Object Request Broker
NPAC's Object Web RTI
JWORB based Object Web RTI
HLA/RTI versus HPF/MPI
HLA/RTI at Top Management Level
Architecture of HLA/RTI Services
HLA/RTI Management Structure
Resource Management and Scheduling
Outside Index
Summary of Material
HTML version of Basic Foils prepared July 4 98 
Full HTML Index
| ASC Year 2 Meeting Columbus July 2 1998 |
| Geoffrey Fox |
| Northeast Parallel Architectures Center |
| Syracuse University |
| 111 College Place |
| Syracuse NY |
| gcf@npac.syr.edu |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
Many of developments in modern Large Scale Enterprise Systems can enhance HPCC effectiveness by linking in productive services built around web, database and distributed object technologies
|
HPCC (MPP) systems are not necessarily optimized for implementing such services
|
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Pragmatic Object Web - Integrate with Web competing models for distributed objects: Java, CORBA, COM, WOM |
POW is middleware for multi-tier distributed enterprise applications
|
| High Performance commodity computing - traditional HPC modules managed by POW on new commodity clusters (PC with NT, Linux or Solaris OS) using Distributed Computing Concepts (HLA,RTI) at coarse grain and classic HPCC for computational kernels |
HTML version of Basic Foils prepared July 4 98 
Full HTML Index
|
|
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| W is Web Server |
| PD Parallel Database |
| DC Distributed Computer |
| PC Parallel Computer |
| O Object Broker |
| N Network Server e.g. Netsolve |
| T Collaboratory Server |
| Clients |
| Middle Layer (Server Tier) |
| Third Backend Tier |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Distributed Computing becomes a commodity article (driven by Web Technologies) |
| Market niches for orthodox MPP style HPC are shrinking |
| NT clusters become a viable and more cost effective alternative to classic high performance systems |
| HLA/RTI from distributed simulation community natural for coarse grain while MPI/HPF/.... Natural for fine grain -- must integrate which we claim can be done using a multi tier architecture |
| Web/Commodity software (Pragmatic Object Web) - promising base to build new HPcc (commodity computing) |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| 3-(or more)-tier architecture - Web browser front-ends, legacy (e.g. databases, HPC modules) backends; fat (1+tier) middleware |
Alternative / competing Middleware models:
|
| Each model has different tradeoffs |
| POW attempts at integrating various models and services in terms of multi-protocol middleware servers (JWORB) |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Server: e.g. |
| Proprietary |
| Database |
| Lotus Notes |
| Web or ORB |
| Service e.g. Database Repository or file systems accessed by Web Servers |
| Client |
| Now in POW style, we add modular capabilities to get 3 4 or more tier |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Back End Server: e.g. |
| Proprietary |
| Database |
| Service e.g. Database Repository |
| ThickClient e.g. Java Applet GUI |
| Middle Tier Server with "Business Logic" e.g. map user objects to relational tables as in Java Blend |
| We get 4 tier by refining client .... |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| But Middle Tier can be a plethora of servers linked in a dataflow model |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Old way: Use an Object Database |
| Current Approach: Use a Relational Database and business logic in EJB |
| Middle Tier |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Clients and their servers |
| Middle Tier Custom Servers |
| Back End Servers and |
| their services |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| The backend servers would include CORBA objects from Educom's IMS projects; Video servers and Oracle database defined curricula pages from NPAC |
| The front end servers would include distributed students with mirror sites to get performance |
| In the middle tier, we have JDBC query processing and XML servlet parsers mapping original data in optimal fashion to match needs of student -- choosing from pure HTML or Interactive Java Whiteboard views of a given object |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
We have multiple supercomputers in the backend -- one doing CFD simulation of airflow; another structural analysis while in more detail you have linear algebra servers (Netsolve); Optimization servers (NEOS); image processing filters(Khoros);databases (NCSA Biology workbench); visualization systems(AVS, CAVEs)
|
| All linked to collaborative information systems in a sea of middle tier servers(as on previous page) to support design, crisis management, multi-disciplinary research |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Database |
| Matrix Solver |
| Optimization Service |
| MPP |
| MPP |
| Parallel DB Proxy |
| NEOS Control Optimization |
| Origin 2000 Proxy |
| NetSolve Linear Alg. Server |
| IBM SP2 Proxy |
| Gateway Control |
| Agent-based Choice of Compute Engine |
| Multidisciplinary Control (WebFlow) |
| Data Analysis Server |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| High Performance Computing and Communication Tier |
| Clients |
| Gateway Systems |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
Seamless Interface -- an Enterprise Javabean which processes input from user's Java Applet interface and maps user generic commands to those on specific machine
|
| Resource management of heterogeneous MPP backend (linked to seamless interface) |
| Database and Object Brokers |
| Collaboration Servers including Tango, Lotus Notes and other commercial systems |
| Visualization Servers |
| "Business Logic" to map user data view (e.g. objects) to persistent store (e.g. Oracle database) and simulation engine (MPP) preferred format |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Most of a Command and Control Application |
| Several FMS and IMT Applications |
| Some I/O Intensive applications |
| High value services with modest computational needs e.g. grid generation and other pre-processing, data manipulation and other post-processing |
| Video Servers for Training |
| Design and Planning Tools |
| "Glue" for Multidisciplinary Interactions |
| Control of metacomputing applications |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| WebFlow - a visual programming environment for the Web with real-time dataflow graph authoring tools |
| 3-tier multi-server system with Web browser / Java applet front-end, Java Web Server middleware and (Java) computational modules in the backend |
| Early prototype - pure Java system, controlled by servlet based Session, Module and Connection Managers |
| POW middleware will support all languages |
| Used to build both linked modules and as framework for "wrapping legacy codes" as "distributed scientific objects" |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Original Image |
| Output Image |
| Some of |
| Available Image Filters |
| Visual DataFlow |
| Interface |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Client Tier |
| IIOP High Functionality |
| Middle Tier |
| Future Globus |
| Globus |
| Future Parallel I/O |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| JWORB - Java Web Object Request Broker - multi-protocol middleware network server (HTTP + IIOP + DCE RPC + RTP + ..) |
| Current prototype integrates HTTP+IIOP i.e. acts as Web Server and CORBA Broker |
| Next step: add DCE RPC support to include Microsoft COM |
| JWORB - our trial implementation of POW |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| Implements DMSO RTI as JWORB service with 2 major CORBA objects: RTI Ambassador and Federate Ambassador |
| Offers natural Web interfaces to HLA simulations via HTTP or IIOP channels |
| Natural support for human-in-the-loop (Web surfers join WebHLA federation and can collaborate as WebHLA federates) |
| Attractive model for High Level Metacomputing |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| We can support any given paradigm at either high functionality (web server) or high performance (backend) level |
| HPCC Messaging could be a Java/RMI middle tier MPI or Nexus/Optimized Machine specific MPI at backend |
| JWORB supports CORBA based RTI already and we can bridge to high performance event driven simulation systems like SPEEDES at the high performance backend layer |
| However most problems can be thought of a set of coarse grain entities which are internally data parallel but the coarse grain structure is "functional" parallelism |
| So HLA/RTI is especially natural as tier 2 management level of these coarse entities |
| Entities can be time synchronized simulations and use MPI(HPF?) at either middle or back end tier or in fact as in DMSO simulations a federate running a custom discrete event simulation |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
Resource Management typically breaks down into either
|
So a) is all at middle tier and should use commodity solutions -- there are many queuing systems such as Condor, Codine, LSF which we can "wrap" and Microsoft does not yet have a fully scalable commodity solution
|
| So it is still embryonic but we suggest adopting the HLA/RTI framework as this supports job placement, interdependencies (time management) and hierarchical systems of federations --> federates |
| Optimized data placement has been largely solved as a mathematical problem by HPCC but not packaged broadly. Our suggestion suggests how to invoke as backend support for a commodity service |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
HTML version of Basic Foils prepared July 4 98
Full HTML Index
| So we have a hierarchy of entities Federation --> Federates --> Objects where can have many tiers in each category |
| A Federation could be the set of all jobs to be run on a particular site |
| A Federate could be a job consisting of multiple possibly shared objects |
| Objects are just data structures in HLA -- you send interaction events instead of invoking methods |
| These aspects are organized by Federation, Object and Ownership management services |
| We can classify both jobs and computers as separate federations |
HTML version of Basic Foils prepared July 4 98
Full HTML Index
Declaration Management corresponds to publication and subscription model of matching services and needs
|
| Time Management corresponds to scheduling of sequenced events in discrete event simulations -- it will allow support generally dependencies in jobs -- the CAVE visualization system must be used after simulation |
| Data management is classic "load-balancing" problem of parallel computing where you map objects optimally to computers to minimize communication cost and load imbalance |