Full HTML for

Basic foilset Gateway: A High Level Interface to High performance Commodity Metacomputing

Given by Erol Akarsu at PhD Proposal NPAC on Sept 23 1999. Foils prepared October 25 1999
Outside Index Summary of Material


The research problem
How to approach to solution
Early Stage of Gateway Architecture : WebFlow
A sophisticated Distributed Object-based Gateway
Applications of Gateway
DARP - A Data analysis and Rapid Prototyping Environment
Conclusions

Table of Contents for full HTML of Gateway: A High Level Interface to High performance Commodity Metacomputing

Denote Foils where Image Critical
Denote Foils where Image has important information
Denote Foils where HTML is sufficient

1 Gateway : A High-Level Interface to High Performance Commodity Metacomputing
2 Outline
3 The research problem
4 How to approach to solution(1)
5 How to approach to solution(2)
6 Early Stage of Gateway : WebFlow: Mission (1)
7 WebFlow : Architecture(3)
8 WebFlow : Implementation(2)
9 WebFlow : Middle-Tier(4)
10 WebFlow: Backend Modules(5)
11 WebFlow: Run Method(6)
12 WebFlow: over Globus(7)
13 WebFlow : Pros & Cons(8)
14 WebFlow : Cons(9)
15 Distributed Object-Based Gateway- Implementation Strategy
16 Distributed Object-Based Gateway- Architecture
17 PPT Slide
18 Gateway Server
19 Gateway Server
20 PPT Slide
21 Some of Key properties of Gateway Server(1)
22 PPT Slide
23 Some of Key features- (2)
24 PPT Slide
25 PPT Slide
26 Some of Key features- (3)
27 PPT Slide
28 Some of Key features- (4)
29 PPT Slide
30 PPT Slide
31 PPT Slide
32 PPT Slide
33 Some of Key features- (5)
34 Some of Similar products
35 Effects of the previous enterprise models to Gateway(1)
36 Effects of the previous enterprise models to Gateway(2)
37 PPT Slide
38 PPT Slide
39 PPT Slide
40 PPT Slide
41 PPT Slide
42 PPT Slide
43 PPT Slide
44 PPT Slide
45 PPT Slide
46 PPT Slide
47 PPT Slide
48 Possible Usage of Gateway in complete collaborative environment
49 PPT Slide
50 Gateway Applications
51 LMS Objectives
52 LMS: Changes in Vegetation
53 PPT Slide
54 PPT Slide
55 Data Retrieval
56 Quantum Simulations
57 QS: Gateway implementation
58 QS Front End
59 PPT Slide
60 DARP - A Data analysis and Rapid Prototyping Environment
61 High Level Architecture of DARP
62 PPT Slide
63 PPT Slide
64 Instrumentation of the code
65 PPT Slide
66 PPT Slide
67 PPT Slide
68 PPT Slide
69 PPT Slide
70 PPT Slide
71 PPT Slide
72 PPT Slide
73 PPT Slide
74 Conclusions
75 Conclusions(Cont.)
76 Conclusions(Cont.)
77 Conclusions(Cont.)

Outside Index Summary of Material



HTML version of Basic Foils prepared October 25 1999

Foil 1 Gateway : A High-Level Interface to High Performance Commodity Metacomputing

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
September 23
presented by
Erol Akarsu
Northeast Parallel Architectures Center at Syracuse University

HTML version of Basic Foils prepared October 25 1999

Foil 2 Outline

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
The research problem
How to approach to solution
Early Stage of Gateway Architecture : WebFlow
A sophisticated Distributed Object-based Gateway
Applications of Gateway
DARP - A Data analysis and Rapid Prototyping Environment
Conclusions

HTML version of Basic Foils prepared October 25 1999

Foil 3 The research problem

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
We have applications which needs a lot of heterogeneous and remote computation resources: hardware or software
These resources can be : powerful supercomputers, visualization, database or instrumentation tools etc.
We have users who wants to use this heterogeneous environment as a virtual computer
We need to have a simple and uniform point of access to these remote resources
We need to have a user friendly Web-based or application-based interface to access these resources

HTML version of Basic Foils prepared October 25 1999

Foil 4 How to approach to solution(1)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
NPAC[G. Fox and W. Furmanski] developed a new strategy : HPcc-High Performance Commodity Computing
Main goal of HPcc is to develop High Performance environment on top of the commodity system that were created by commercial, academic and freeware programmers
We call this system as DcciS - Distributed Commodity Computing and Information System
This can benefit HPCC in several ways and allow the development of both more powerful parallel programming environments (DARP) and new distributed metacomputing systems (Gateway)

HTML version of Basic Foils prepared October 25 1999

Foil 5 How to approach to solution(2)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
We abstract these systems into three-tier architecture
The resultant system, called HPcc, combines the needed performance for large-scale HPCC applications with the rich functionality of commodity systems
High performance can be obtained by combining concurrency at the middle server tier with optimized parallel back end services
We successfully have built an HPcc environment that offers the evolving functionality of commodity systems without significant re-engineering as advances in hardware and software lead to new and better commodity products.

HTML version of Basic Foils prepared October 25 1999

Foil 6 Early Stage of Gateway : WebFlow: Mission (1)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
WebFlow was developed by other NPAC project members(Tom Haupt, Wojtek Furmanski, Timucin and Zeynep).
We extended and applied WebFlow to some applications such Quantum Simulations(QS)
The visual HPDC framework that offers an intuitive Web browser based interface and a uniform point of interactive control for a variety of computational modules and applications, running at various labs on different platforms.
Our technology goal is to build a high-level user friendly commodity software based visual programming and runtime environment for HPDC.

HTML version of Basic Foils prepared October 25 1999

Foil 7 WebFlow : Architecture(3)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
URL and
Socket
Distributed
WebFlow
Servers

HTML version of Basic Foils prepared October 25 1999

Foil 8 WebFlow : Implementation(2)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
WebFlow is implemented as a modern three-tier system
  • Tier 1 is a high-level front-end for visual programming built on top of the Web standards.
  • Distributed URL and socket-based broker Middleware forms Tier 2.
  • Back-end services comprise Tier 3. In particular, high performance services are implemented using the metacomputing toolkit of Globus.

HTML version of Basic Foils prepared October 25 1999

Foil 9 WebFlow : Middle-Tier(4)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Mesh of Web Servers running servlets that manage and coordinate distributed computation.
  • Session Manager,
  • Module Manager,
  • Connection Manager.
Front End

HTML version of Basic Foils prepared October 25 1999

Foil 10 WebFlow: Backend Modules(5)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
are Java objects that implement WebFlow specific module interface:
  • initialize
    • register itself to the module manager, and registers its input and output ports to the connection manager
  • run
    • implements the actual functionality of the module
  • destroy
    • release resources

HTML version of Basic Foils prepared October 25 1999

Foil 11 WebFlow: Run Method(6)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
The functionality of the module (run method) can be implemented entirely in Java
Existing applications written in languages different than Java can be easily encapsulated
Execution of the module can be delegated to an external system capable of resource allocation such as Globus, Condor, etc.

HTML version of Basic Foils prepared October 25 1999

Foil 12 WebFlow: over Globus(7)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
In order to run WebFlow over Globus there must be at least one WebFlow node capable of executing Globus commands, such as globusrun
Jobs that require computational power of massively parallel computers are directed to the Globus domain, while others can be launched on much more modest platforms, such as the user's desktop or even a laptop running Windows NT.
Bridge between WebFlow
and Globus

HTML version of Basic Foils prepared October 25 1999

Foil 13 WebFlow : Pros & Cons(8)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
PROS
It follows the industry-proved standard of three-tier model.
Developing back-end WebFlow modules and their front-end controls can be achieved independent of each other.
It supports session concept for each working user that doesn't interfere with each other.
Module developers don't need to deal low-level issues such as allocation computing resources and connection of modules.
It allows configuration and instantiation of multiple WebFlow servers
The architecture of WebFlow midller-tier was very well designed with three servlets : Session , Connection and Module managers.

HTML version of Basic Foils prepared October 25 1999

Foil 14 WebFlow : Cons(9)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Although the implementation of the WebFlow proved to be very successful, we are not satisfied with this to the large extend custom solution
Middle tier is based on a custom protocol implemented directly on top of URL and socket based interactions
Not scalable (Uses multi-threading to support concurrency of user modules in middle-tier)
Difficult to implement security features, fault tolerance, monitoring
Impossible to take advantage of new technologies
There is no flexibility to compose application
Supports only data-flow model
There is no persistent model( like with XML and java serialization)
There is no way to copy and paste any user module running in a application

HTML version of Basic Foils prepared October 25 1999

Foil 15 Distributed Object-Based Gateway- Implementation Strategy

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Follow industry standards
distributed objects: CORBA, Java, javaBeans, Enterprise JavaBeans...
security: https, PKI, GSSAPI, SSL, Keberos5
Participate in creation of standards
DATORR, Alliance PKI
Use commodity components
Commercial: Web servers and browsers, ORB
Academic: Globus, Akenti,..

HTML version of Basic Foils prepared October 25 1999

Foil 16 Distributed Object-Based Gateway- Architecture

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Problem
Solving
Environments
OO Visual
Authoring
Tools
Data-Flow
Visual
Authoring
Custom
Application
GUI
Other
Gateway
Back-End Resources

HTML version of Basic Foils prepared October 25 1999

Foil 17 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
PSE
Object
Oriented
Data Flow
Custom
GUI
WebFlow Middle Tier
DATORR/Computing Portal
Grid Forum
HPCC: Globus
Other as needed
DBMS: JDBC
Small tasks: Java
user codes
https, IIOP/SECIOP
Example: DoD HPCMO ASC/ARL
Prototype Version
NCSA Alliance
Example: LMS

HTML version of Basic Foils prepared October 25 1999

Foil 18 Gateway Server

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
The Gateway server is a container object, a.k.a. context - in fact it implements GatewayContext interface
The GatewayContext acts as a logical container for User modules ("Gateway modules and services") and other GatewayContexts.
Some of main responsibilities of the are :
Logical container for its children
Establishing associations among children
Maintain lifecycle service
Provides security and transaction services for its children(to be implemented)
Responsible for delivering events fired by its children.

HTML version of Basic Foils prepared October 25 1999

Foil 19 Gateway Server

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 20 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
How does it work? CORBA-based Middle Tier (Proxy modules - a simplified example)
Front End
Middle Tier
Back End
host 1
host 3
CORBA object implemented in Java
CORBA client

HTML version of Basic Foils prepared October 25 1999

Foil 21 Some of Key properties of Gateway Server(1)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Gateway is a middle-tier written in pure Java and CORBA and supports both data-flow and event-driven computation.
It supports the sessions of multiple users and even more each user can deal multiple applications at the same time
The Server and each context are just instantiation of our implementation class of GatewayContext interface
Servers can be put into a hierarchical tree so that user can view his distributed application as a unit.
It supports the server-side collaborative environment because of inherent collaborative feature of CORBA objects that anyone can access to if he knows its reference

HTML version of Basic Foils prepared October 25 1999

Foil 22 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
A
A/B
A/C
A/B/D
A/B/E
A/C/F
A/C/G
Proxy objects for slave servers
Master server
Distinguished name(DN)
of a server
Proxy for Master server
The hierarchical view of Gateway servers
running across distributed hosts
User
Modules

HTML version of Basic Foils prepared October 25 1999

Foil 23 Some of Key features- (2)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
After user set up his application, he can save its complete state in XML document which includes objects:context and user module objects and their attributes, and association among them.
User can recover his application from its save state in XML document.
Between run's of his application during testing his software, he can edit the saved XML document and update it for the next run
User has tremendous flexibility of interactively changing his application at run-time.

HTML version of Basic Foils prepared October 25 1999

Foil 24 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Transform
State
XML
Representation of
Application
User has control
Create
Application
with API
Create new XML document
OR Update attributes in
existing XML document
Database
Save&
Restore
Save&
Restore
Distributed
Application

HTML version of Basic Foils prepared October 25 1999

Foil 25 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Processing The Task Specification
XML
A visual representation is converted into a XML
document
XML
service
Web
Server
save
parse
Application Context
Front-End Applet
Middle-Tier
Database

HTML version of Basic Foils prepared October 25 1999

Foil 26 Some of Key features- (3)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
It creates a proxy object for each object created in the system
The original purpose of proxies is to forward requests from the Web client to remote objects (java sandbox problem)
Proxies simplify association of the distributed components
Proxies helps us for logging, tracking and filtering all messages between components in the system
We used these capabilities to implement a simple fault tolerance and security and transaction monitors are being implemented.

HTML version of Basic Foils prepared October 25 1999

Foil 27 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
M/S1
M/S2
M
M/S2/UC1
M/S2/UC2
M1
M2
PS2
PUC2
PM2
PM1
PUC1
PS1
Slave Server:S1
Slave Server:S2
Master Server:M
4
3
2
1
Execution path
followed during
firing event from M1 to M2

HTML version of Basic Foils prepared October 25 1999

Foil 28 Some of Key features- (4)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Complete state of a distributed application can be saved in a Database (Oracle) in addition to in XML
Therefore, the previous sessions can be recovered from database in addition to XML document
Gateway has a simple fault-tolerance model : when either proxy or the real object is down, the system will recover from the database
An application consisting of components running at different machines can be defined in XML and started automatically.
Instead of manually starting, Gateway servers also can be started through URL easily

HTML version of Basic Foils prepared October 25 1999

Foil 29 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
First slide
Monitor
Servlet
Database
Module M1
Module M2
Fire
Event
Call a module
method
Invoke
servlet
Recover
Proxy
A1
B3
B2
B1
A2
Simple recovery
When proxy is down
User context
Proxy
User

HTML version of Basic Foils prepared October 25 1999

Foil 30 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Database
Module M1
Module M2
Fire
Event
Call a module
method
Get old state &&
Recover remote
module
A1
B2
B1
Simple recovery
When Remote Object
is down
Delegate
Request
A2
A3
B3
User

HTML version of Basic Foils prepared October 25 1999

Foil 31 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Gateway middle-Tier
M1
M2
M3
Send Application Representation
In XML
WEB
Server
WEB
Server
Database
M4
M5
M6
M1
M4
M7
Module
Descriptions in XML
Start Gateway
Server
WEB Servers are used
to installation applications

HTML version of Basic Foils prepared October 25 1999

Foil 32 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
<?xml version="1.0"?>
<!DOCTYPE distributed_application SYSTEM "webflow.dtd" [
<!ENTITY timer-master-decls
SYSTEM "http://osprey2:1998/WebFlow/module-decls/master-module.xml">
<!ENTITY timer-slave-decls
SYSTEM "http://osprey2:1998/WebFlow/module-decls/slave-module.xml">
]>
<distributed_application>
<WebFlowContext
componentID="ntserver"
servlet_href="http://osprey2:1998/servlets/wfm?installer="http://osprey2:1998/WebFlow/module-decls/master-module.xml""
entityFlag="no"
>
&timer-master-decls;
<WebFlowContext componentID="uc_1" entityFlag="no">
<WebFlowContext componentID="uc_2" entityFlag="no">
<WebFlowContext componentID="uc_3" entityFlag="no">
<ModuleInstance>
<componentRef>helloModule_1</componentRef>
<moduleID>ntserver/uc_1/uc_2/uc_3/helloModule_1</moduleID>
<propertyInstances>
<propertyVal attrRef="structEventUse">
<binaryVal> ........ </binaryVal>
</propertyVal> ........
........................................................
A XML document portion to install
Gateway Application

HTML version of Basic Foils prepared October 25 1999

Foil 33 Some of Key features- (5)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
User modules can be defined as entity modules such that their attributes are saved across user requests for their methods.
Currently, we are incorporating Kerberos-based security protocol into proxy objects
We provide two main class: BeanContextChildSupport and GatewayContext. User define his module by extending from BeanContextChildSupport
User modules are put inside GatewayContext objects. User can define his own context by extending GatewayContext class. In this way, the system can evolve dynamically.

HTML version of Basic Foils prepared October 25 1999

Foil 34 Some of Similar products

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
EJB - Enterprise Javabeans: Server-side component model. We could not use it because it doesn't have event model yet and also it is expensive.
JINI, by SUN Microsystems. Our model of event binding is similar to it.
CORBA component model:not yet implemented
JavaBeans, by SUN Microsystems: especially JavaBeans Glasgow model.
WebSubmit and UniCore

HTML version of Basic Foils prepared October 25 1999

Foil 35 Effects of the previous enterprise models to Gateway(1)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
We adopted the JavaBeans event model and removed the requirement of addXlistener/removeXListener method pair to add/remove event "X". Event connection is handled by Gateway context
We took off the requirement of setX/getX method pairs for attribute "X". We get the attributes from XML document generated from IDL file of user module automatically.
As opposed to JavaBeans persistency of classic java serialization model, we serialize our objects in XML.

HTML version of Basic Foils prepared October 25 1999

Foil 36 Effects of the previous enterprise models to Gateway(2)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
We adopted the EJB container methodology- all of the transaction , security and persistency can be done by EJB container of one EJB object
Serialization of EJB objects carried out with XML document (EJB 1.1 model)
EJB objects can get the services attached to its container object.
As EJB can give opportunity of defining attributes for EJB objects and store/restore from Database through JDBC, Gateway implemented same functionality.

HTML version of Basic Foils prepared October 25 1999

Foil 37 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Services
User Modules
Data Flow
Front-End
Middle-Tier modules serve as proxies of Back-End Services
OO
Front-End
User Space Definition and Task Specification
Metacomputing Services

HTML version of Basic Foils prepared October 25 1999

Foil 38 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Context Objects
(Green circles)
Client send
addNewModule("M")
method request
1
8
3
4
6
7
5
2
New User module
Mobj with name "M"
Forward call
to context object
Object Factory
Instantiate module
with name `M"
Return module
Object: Mobj
Return
Mobj
Create Proxy object
for Mobj
Context:uc_1
Proxy for
context:uc_1
Add Mobj
into "uc_1" children
Context inserts new module
Mobj into its children list
Context:uc_0
Proxy for
context:uc_0
Adding new user module
to user Context of "UC_1"

HTML version of Basic Foils prepared October 25 1999

Foil 39 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Removing module "M2" from
user context of "UC_1"
Context Objects
(Green circles)
Client send
removeModule for
Module M2
1
Forward call
to context object
PUC_1
Context:UC0
PUC_0
Context:UC1
PM0
PM3
Module:M0
Module:M3
3
2
PM2
Module:M2
PM1
Module:M1
4
Fire property"beanContext"
change event:
PropertyChange method call
5
forward
6
Remove connection entry
from M1 to M2
Remove connection entry
in the local binding table
from M2 to M3 and
Module M2 and
its proxy PM2

HTML version of Basic Foils prepared October 25 1999

Foil 40 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Context Objects
(Green circles)
Client send
attachEvent call
with parameters:
PM0,PM1,
"propName",
"targetMethod"
1
Forward call
to context object
PUC_1
Context:UC0
PUC_0
Context:UC1
PM0
PM1
Module:M0
Module:M1
Add this connection
between modules
M1 and M2 into
local binding table
5
4
3
Add UC1 into Property
"Beancontext"
Listeners of module M0
Add UC1 into Property
"Beancontext"
Listeners of module M1
(addPropertyChangeListener)
2
Attach event from Module M1(in context
"UC1") to M0 (in context "UC1")

HTML version of Basic Foils prepared October 25 1999

Foil 41 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Context Objects
(Green circles)
Client send
detachEvent call
with parameters:
PM0,PM1,
"propName",
"targetMethod"
1
Forward call
to context object
PUC_1
Context:UC0
PUC_0
Context:UC1
PM0
PM1
Module:M0
Module:M1
Remove this connection
between modules
M1 and M2 from
the local binding table
5
4
3
Remove UC1 from
Property Listeners of M0
if there are no connections
after this detachment
Remove UC1 from
Property Listeners of M1
if there are no connections
after this detachment
2
Detaching the association between
module M1 and M0

HTML version of Basic Foils prepared October 25 1999

Foil 42 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
M/S1
M/S2
M
M/S2/UC1
M/S2/UC2
M1
M2
PS2
PUC2
PM2
PM1
PUC1
PS1
Slave Server:S1
Slave Server:S2
Master Server:M
4
3
2
1
Execution path
followed during
firing event from M1 to M2

HTML version of Basic Foils prepared October 25 1999

Foil 43 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
AcmeAccountHome
AcmeRemoteAcccout
JNDI
Client
AccountBean
6
5
4
3
2
1
AccountHome
Ask for
AcmeAccountHome
Return
AcmeAccountHome
Create EJB Object
Return
EJB Object
Business
methods
Delegate
requests
EJB Bean Provider
class and interfaces
Classes generated
by EJB container
Container
contract
EJB Container
A comparison with EJB:
a basic Execution flow
of EJB objects(Enterprise
JavaBeans)

HTML version of Basic Foils prepared October 25 1999

Foil 44 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
An example IDL Definition of user modules

HTML version of Basic Foils prepared October 25 1999

Foil 45 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
//store this XML document in file "module_properties.xml"
<UserModule idlFile="test1.idl" componentID="timerModule_1">
<repositoryID>IDL:Gateway/test/timer_1:1.0</repositoryID>
<properties>
<simpleDef attID="timerEvent" eventType="genericEvent">
<wrapper_interface>IDL:Gateway/test/timerEvent:1.0 </wrapper_interface>
<simple type="string" name="genericTimerEvent">
<description>this is optional desctiption</description>
<repositoryID>string</repositoryID>
</simple>
</simpleDef>
<structDef attID="timerProperty" eventType="propertyEvent">
<wrapper_interface>IDL:Gateway/test/timerProperty:1.0 </wrapper_interface>
<struct type="struct" name="timerPropertyEvent">
<description>this is optional desctiption</description>
<repositoryID>IDL:Gateway/test/timerstruct:1.0</repositoryID>
<sequence type="sequence<long>" name="pastValues">
<description>this is optional desctiption</description>
<repositoryID>IDL:Gateway/test/longArray:1.0</repositoryID>
</sequence>
<simple type="string" name="currenttime">
<description>this is optional desctiption</description>
</simple>
</struct>
...................//Other property definitions
</properties>
<methodList>
<method name="method_1" returnType="void">
<parameter name="p1" type="string"/>
<parameter name="p2" type="long"/>
</method>
<method name="method_2" returnType="void"> </method>
</methodList>
</UserModule>
Partial translation
of the idl file above
in XML document

HTML version of Basic Foils prepared October 25 1999

Foil 46 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
org.omg.CORBA.Object s1_obj, s2_obj,
uc1_obj, uc2_obj,m1_obj,m2_obj;
GatewayContext uc1, uc2,s1,s2;
//get IOR string from fixed url
String ref = getIORFromURL(masterURL);
//convert IOR to corba object
org.omg.CORBA.Object obj = orb.string_to_object(ref);
//narrow corba Object to specific corba server type
GatewayContext master =
GatewayContextHelper.narrow(obj);
//get the proxy reference of slave server S1
s1_obj = master.getContext("M/S1");
s1 = GatewayContextHelper.narrow(s1_obj);
//add new user context with name "uc1"
uc1_obj = s1.addNewContext("uc1");
//narrow it to WebFowContext
uc1 = GatewayContextHelper.narrow(uc1_obj);
m1_obj = uc1.addNewModule("timerModule_1");
//get the proxy reference of slave server S2
s2_obj = master.getContext("M/S2");
s2 = GatewayContextHelper.narrow(s2_obj);
//add new user context with name "uc2"
uc2_obj = s2.addNewContext("uc2");
//narrow it to WebFowContext
uc2 = GatewayContextHelper.narrow(uc2_obj);
m2_obj = uc1.addNewModule("timerModule_2");
//Finally make "push" type of connection between
//modules M1and M2. Make sure you issue
//the "attachPushEvent" //command to the parent
//of event source where M1 is the source of event.
//uc1.attachPushEvent(m1_obj," timerEventData",
//m2_obj,"targetMethod");
Continued
The sequence of Gateway API calls
to create the above distributed environment which
includes master and two slaves and two modules
and their association

HTML version of Basic Foils prepared October 25 1999

Foil 47 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
<GatewayContext componentID="M">
<GatewayContext componentID="M/S2">
<GatewayContext componentID="M/S2/uc2">
</GatewayContext>
<connections >
</connections >
</GatewayContext>
<GatewayContext componentID="M/S1">
<GatewayContext componentID="M/S1/uc1">
<ModuleInstance>
<componentRef>timerModule_2</componentRef>
<moduleID>M/S1/uc1/timerModule_2</moduleID>
</ModuleInstance>
<ModuleInstance>
<moduleID>M/S1/uc1/timerModule_1</moduleID>
<propertyInstances>
<propertyVal attrRef="timerPropertyData">
<structVal>
<sequenceVal>
<simpleValɭ</simpleVal>
<simpleValɮ</simpleVal>
<simpleValɯ</simpleVal>
<simpleValɰ</simpleVal>
</sequenceVal>
<simpleVal>startTime</simpleVal>
</structVal>
<propertyVal>
<propertyVal
attrRef = "timerVetoablePropertyData">
<structVal>
<sequenceVal>
<structVal>
<sequenceVal>
<simpleValɭ</simpleVal>
<simpleValɮ</simpleVal>
<simpleValɯ</simpleVal>
<simpleValɰ</simpleVal>
</sequenceVal>
<simpleVal>startTime
</simpleVal>
</structVal>
//...................... Continue
User can access middle tier
through abstract job
specification in XML document

HTML version of Basic Foils prepared October 25 1999

Foil 48 Possible Usage of Gateway in complete collaborative environment

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
As we mentioned , Gateway supports server-side collaboration.
New JavaScript (v 1.3) can provide the capturing of all the events of JavaScript objects. This implies that these events can be collected in Gateway middle-tier and broadcasted to other participants. In this way, JavaScript Web pages can be shared
Other scenario is about Java applets. If user write his applet following JavaBeans pattern, we can associate two applet running different hosts by interposing Gateway between them.

HTML version of Basic Foils prepared October 25 1999

Foil 49 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
CORBA Object
COM Object
Gateway context
Proxy objects acting
as CORBA and
COM object
Proxy objects helps us using the objects
based on different protocols (COM and
CORBA here) concurrently. It functions as
bridge between COM and CORBA objects

HTML version of Basic Foils prepared October 25 1999

Foil 50 Gateway Applications

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
LMS (Landscape management simulation)
QS-Quantum Simulations
Seamless Access to HPCC resources
We will discuss only first two: QS and LMS

HTML version of Basic Foils prepared October 25 1999

Foil 51 LMS Objectives

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
To develop a web based system that implements a "navigate-and-choose" paradigm and allows the end user to:
  • Select (a set of) computational modules that provide answers to the problem at hand
  • Retrieve input data sets from remote sources
  • Use adequate (remote) computational resources
  • Visualize and analyze output data on the local host
Anytime, anywhere, using any platform
(e.g., a connected to the Internet laptop PC)

HTML version of Basic Foils prepared October 25 1999

Foil 52 LMS: Changes in Vegetation

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
A decision maker (the end user of the system) wants to evaluate changes in vegetation in a geographical region over a long time period caused by short term disturbances such as a fire or human activity.
One of the critical parameters of the vegetation model (EDYS) is soil condition at the time of the disturbance.
This in turn is dominated by rainfall that possibly occurs at that time (CASC2D simulation)
Input data for the simulations are available from the Internet, such as Data Elevation Models (DEM) from USGS web site or from custom databases (spices characteristics)

HTML version of Basic Foils prepared October 25 1999

Foil 53 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
WMS
EDYS
CASC2D
DEM
Land Use
Soil
Texture
Vegetation
Data retrieval
Data preprocessing
Simulation: two interacting codes
  • EDYS
  • CASC2D
Visualization
LMS: Changes in Vegetation
WMS: Watershed Modeling System
EDYS: vegetation model
CASC2D: watershed model

HTML version of Basic Foils prepared October 25 1999

Foil 54 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Data pre- and post-processing
Simulations
Data retrieval
LMS Front End

HTML version of Basic Foils prepared October 25 1999

Foil 55 Data Retrieval

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
The data wizard allows the user to interactively select the data and
download them to the local machine. The raw data are then fed to the WMS system launched from the browser to generate input files for simulations.

HTML version of Basic Foils prepared October 25 1999

Foil 56 Quantum Simulations

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 57 QS: Gateway implementation

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 58 QS Front End

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 59 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 60 DARP - A Data analysis and Rapid Prototyping Environment

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Integrates compiled and interpreted HPF-High Performance Fortran which provides powerful program development environment
This DARP environment includes:
  • a source level debugger
  • data visualization and data analysis packages
  • HPF interpreter
  • Web-based Front-end

HTML version of Basic Foils prepared October 25 1999

Foil 61 High Level Architecture of DARP

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 62 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Current architecture of DARP.
The DARP server is a part of the instrumented HPF application and it is replicated
over the nodes participating in the computation. The client communicates with only
one server on a selected node. On this node the server acts as a manager.

HTML version of Basic Foils prepared October 25 1999

Foil 63 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Middle-tier : DARP manager controls HPF back-end and
communicates with other servers

HTML version of Basic Foils prepared October 25 1999

Foil 64 Instrumentation of the code

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
The instrumentation of the code involves three steps:
1.Adding server functions
2.Insertion calls to the HPF server before each HPF statement (SAGE++)
3.Identification of the types of all variables used in the application

HTML version of Basic Foils prepared October 25 1999

Foil 65 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
HPF interpreter

HTML version of Basic Foils prepared October 25 1999

Foil 66 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
HPF Front end

HTML version of Basic Foils prepared October 25 1999

Foil 67 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
A screen dump of a DARP session. The upper right panel shows the
front-end applet with a fragment of an HPF code. The action points at which SciVis
proxy library is called are highlighted, and a triangle on the right points to the
current line.

HTML version of Basic Foils prepared October 25 1999

Foil 68 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 69 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index

HTML version of Basic Foils prepared October 25 1999

Foil 70 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
Send_deferred
for DARP Command
Poll_response
for DARP command
Poll_response
for command result
Command result
DARP
request buffer
DARP
result buffer
GatewayContext
extended by DARP manager
User Modules
running on processors
Registered
clients
Put DARP command
with send_deferred
A complete Integration of DARP
with Gateway

HTML version of Basic Foils prepared October 25 1999

Foil 71 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
A simple representative DARP method

HTML version of Basic Foils prepared October 25 1999

Foil 72 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
This shows how DARP server gets
the next command from middle-tier

HTML version of Basic Foils prepared October 25 1999

Foil 73 PPT Slide

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
This shows how DARP server put the result of
previous client command into middle-tier

HTML version of Basic Foils prepared October 25 1999

Foil 74 Conclusions

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
We developed a platform independent, three-tier system
The system offers fine(DARP) and course(Gateway)-grain control on the used distributed application
It provides persistency model in XML document and Database

HTML version of Basic Foils prepared October 25 1999

Foil 75 Conclusions(Cont.)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
It allows message and source code-based tracking of all interactions among objects
Currently only supports simple transactions based on storing/recovering the state of any object from/to Database through JDBC
It permits to put security issues into proxy objects. Therefore it alleviates the security handling for module developer

HTML version of Basic Foils prepared October 25 1999

Foil 76 Conclusions(Cont.)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
It allows multiple sessions concurrently. Even more user can work multiple applications at the same time.
DARP system can be extended easily to other programming languages such Java, C/C++ by just writing its instrumentation code
Currently, it supports server-side collaboration but can be extended also client-side collaboration

HTML version of Basic Foils prepared October 25 1999

Foil 77 Conclusions(Cont.)

From Gateway: A High Level Interface to High performance Commodity Metacomputing PhD Proposal NPAC -- Sept 23 1999. *
Full HTML Index
We developed a highly-generic multi-purpose and extendible component model.
We will put the security and transaction handling into proxy functionality.
Because of using XML for a communication mechanism between front-end and Gateway middle-tier, we can benefit XSL to translate XML to HTML, any audio and visual effects if any.

© Northeast Parallel Architectures Center, Syracuse University, npac@npac.syr.edu

If you have any comments about this server, send e-mail to webmaster@npac.syr.edu.

Page produced by wwwfoil on Mon Oct 25 1999