Given by Erol Akarsu at PhD Proposal NPAC on Sept 23 1999. Foils prepared October 25 1999
Outside Index
Summary of Material
The research problem |
How to approach to solution |
Early Stage of Gateway Architecture : WebFlow |
A sophisticated Distributed Object-based Gateway |
Applications of Gateway |
DARP - A Data analysis and Rapid Prototyping Environment |
Conclusions |
Outside Index Summary of Material
September 23 |
presented by |
Erol Akarsu |
Northeast Parallel Architectures Center at Syracuse University |
The research problem |
How to approach to solution |
Early Stage of Gateway Architecture : WebFlow |
A sophisticated Distributed Object-based Gateway |
Applications of Gateway |
DARP - A Data analysis and Rapid Prototyping Environment |
Conclusions |
We have applications which needs a lot of heterogeneous and remote computation resources: hardware or software |
These resources can be : powerful supercomputers, visualization, database or instrumentation tools etc. |
We have users who wants to use this heterogeneous environment as a virtual computer |
We need to have a simple and uniform point of access to these remote resources |
We need to have a user friendly Web-based or application-based interface to access these resources |
NPAC[G. Fox and W. Furmanski] developed a new strategy : HPcc-High Performance Commodity Computing |
Main goal of HPcc is to develop High Performance environment on top of the commodity system that were created by commercial, academic and freeware programmers |
We call this system as DcciS - Distributed Commodity Computing and Information System |
This can benefit HPCC in several ways and allow the development of both more powerful parallel programming environments (DARP) and new distributed metacomputing systems (Gateway) |
We abstract these systems into three-tier architecture |
The resultant system, called HPcc, combines the needed performance for large-scale HPCC applications with the rich functionality of commodity systems |
High performance can be obtained by combining concurrency at the middle server tier with optimized parallel back end services |
We successfully have built an HPcc environment that offers the evolving functionality of commodity systems without significant re-engineering as advances in hardware and software lead to new and better commodity products. |
WebFlow was developed by other NPAC project members(Tom Haupt, Wojtek Furmanski, Timucin and Zeynep). |
We extended and applied WebFlow to some applications such Quantum Simulations(QS) |
The visual HPDC framework that offers an intuitive Web browser based interface and a uniform point of interactive control for a variety of computational modules and applications, running at various labs on different platforms. |
Our technology goal is to build a high-level user friendly commodity software based visual programming and runtime environment for HPDC. |
URL and |
Socket |
Distributed |
WebFlow |
Servers |
WebFlow is implemented as a modern three-tier system
|
Mesh of Web Servers running servlets that manage and coordinate distributed computation.
|
Front End |
are Java objects that implement WebFlow specific module interface:
|
The functionality of the module (run method) can be implemented entirely in Java |
Existing applications written in languages different than Java can be easily encapsulated |
Execution of the module can be delegated to an external system capable of resource allocation such as Globus, Condor, etc. |
In order to run WebFlow over Globus there must be at least one WebFlow node capable of executing Globus commands, such as globusrun |
Jobs that require computational power of massively parallel computers are directed to the Globus domain, while others can be launched on much more modest platforms, such as the user's desktop or even a laptop running Windows NT. |
Bridge between WebFlow |
and Globus |
PROS |
It follows the industry-proved standard of three-tier model. |
Developing back-end WebFlow modules and their front-end controls can be achieved independent of each other. |
It supports session concept for each working user that doesn't interfere with each other. |
Module developers don't need to deal low-level issues such as allocation computing resources and connection of modules. |
It allows configuration and instantiation of multiple WebFlow servers |
The architecture of WebFlow midller-tier was very well designed with three servlets : Session , Connection and Module managers. |
Although the implementation of the WebFlow proved to be very successful, we are not satisfied with this to the large extend custom solution |
Middle tier is based on a custom protocol implemented directly on top of URL and socket based interactions |
Not scalable (Uses multi-threading to support concurrency of user modules in middle-tier) |
Difficult to implement security features, fault tolerance, monitoring |
Impossible to take advantage of new technologies |
There is no flexibility to compose application |
Supports only data-flow model |
There is no persistent model( like with XML and java serialization) |
There is no way to copy and paste any user module running in a application |
Follow industry standards |
distributed objects: CORBA, Java, javaBeans, Enterprise JavaBeans... |
security: https, PKI, GSSAPI, SSL, Keberos5 |
Participate in creation of standards |
DATORR, Alliance PKI |
Use commodity components |
Commercial: Web servers and browsers, ORB |
Academic: Globus, Akenti,.. |
Problem |
Solving |
Environments |
OO Visual |
Authoring |
Tools |
Data-Flow |
Visual |
Authoring |
Custom |
Application |
GUI |
Other |
Gateway |
Back-End Resources |
PSE |
Object |
Oriented |
Data Flow |
Custom |
GUI |
WebFlow Middle Tier |
DATORR/Computing Portal |
Grid Forum |
HPCC: Globus |
Other as needed |
DBMS: JDBC |
Small tasks: Java |
user codes |
https, IIOP/SECIOP |
Example: DoD HPCMO ASC/ARL |
Prototype Version |
NCSA Alliance |
Example: LMS |
The Gateway server is a container object, a.k.a. context - in fact it implements GatewayContext interface |
The GatewayContext acts as a logical container for User modules ("Gateway modules and services") and other GatewayContexts. |
Some of main responsibilities of the are : |
Logical container for its children |
Establishing associations among children |
Maintain lifecycle service |
Provides security and transaction services for its children(to be implemented) |
Responsible for delivering events fired by its children. |
How does it work? CORBA-based Middle Tier (Proxy modules - a simplified example) |
Front End |
Middle Tier |
Back End |
host 1 |
host 3 |
CORBA object implemented in Java |
CORBA client |
Gateway is a middle-tier written in pure Java and CORBA and supports both data-flow and event-driven computation. |
It supports the sessions of multiple users and even more each user can deal multiple applications at the same time |
The Server and each context are just instantiation of our implementation class of GatewayContext interface |
Servers can be put into a hierarchical tree so that user can view his distributed application as a unit. |
It supports the server-side collaborative environment because of inherent collaborative feature of CORBA objects that anyone can access to if he knows its reference |
A |
A/B |
A/C |
A/B/D |
A/B/E |
A/C/F |
A/C/G |
Proxy objects for slave servers |
Master server |
Distinguished name(DN) |
of a server |
Proxy for Master server |
The hierarchical view of Gateway servers |
running across distributed hosts |
User |
Modules |
After user set up his application, he can save its complete state in XML document which includes objects:context and user module objects and their attributes, and association among them. |
User can recover his application from its save state in XML document. |
Between run's of his application during testing his software, he can edit the saved XML document and update it for the next run |
User has tremendous flexibility of interactively changing his application at run-time. |
Transform |
State |
XML |
Representation of |
Application |
User has control |
Create |
Application |
with API |
Create new XML document |
OR Update attributes in |
existing XML document |
Database |
Save& |
Restore |
Save& |
Restore |
Distributed |
Application |
Processing The Task Specification |
XML |
A visual representation is converted into a XML |
document |
XML |
service |
Web |
Server |
save |
parse |
Application Context |
Front-End Applet |
Middle-Tier |
Database |
It creates a proxy object for each object created in the system |
The original purpose of proxies is to forward requests from the Web client to remote objects (java sandbox problem) |
Proxies simplify association of the distributed components |
Proxies helps us for logging, tracking and filtering all messages between components in the system |
We used these capabilities to implement a simple fault tolerance and security and transaction monitors are being implemented. |
M/S1 |
M/S2 |
M |
M/S2/UC1 |
M/S2/UC2 |
M1 |
M2 |
PS2 |
PUC2 |
PM2 |
PM1 |
PUC1 |
PS1 |
Slave Server:S1 |
Slave Server:S2 |
Master Server:M |
4 |
3 |
2 |
1 |
Execution path |
followed during |
firing event from M1 to M2 |
Complete state of a distributed application can be saved in a Database (Oracle) in addition to in XML |
Therefore, the previous sessions can be recovered from database in addition to XML document |
Gateway has a simple fault-tolerance model : when either proxy or the real object is down, the system will recover from the database |
An application consisting of components running at different machines can be defined in XML and started automatically. |
Instead of manually starting, Gateway servers also can be started through URL easily |
First slide |
Monitor |
Servlet |
Database |
Module M1 |
Module M2 |
Fire |
Event |
Call a module |
method |
Invoke |
servlet |
Recover |
Proxy |
A1 |
B3 |
B2 |
B1 |
A2 |
Simple recovery |
When proxy is down |
User context |
Proxy |
User |
Database |
Module M1 |
Module M2 |
Fire |
Event |
Call a module |
method |
Get old state && |
Recover remote |
module |
A1 |
B2 |
B1 |
Simple recovery |
When Remote Object |
is down |
Delegate |
Request |
A2 |
A3 |
B3 |
User |
Gateway middle-Tier |
M1 |
M2 |
M3 |
Send Application Representation |
In XML |
WEB |
Server |
WEB |
Server |
Database |
M4 |
M5 |
M6 |
M1 |
M4 |
M7 |
Module |
Descriptions in XML |
Start Gateway |
Server |
WEB Servers are used |
to installation applications |
<?xml version="1.0"?> |
<!DOCTYPE distributed_application SYSTEM "webflow.dtd" [ |
<!ENTITY timer-master-decls |
SYSTEM "http://osprey2:1998/WebFlow/module-decls/master-module.xml"> |
<!ENTITY timer-slave-decls |
SYSTEM "http://osprey2:1998/WebFlow/module-decls/slave-module.xml"> |
]> |
<distributed_application> |
<WebFlowContext |
componentID="ntserver" |
servlet_href="http://osprey2:1998/servlets/wfm?installer="http://osprey2:1998/WebFlow/module-decls/master-module.xml"" |
entityFlag="no" |
> |
&timer-master-decls; |
<WebFlowContext componentID="uc_1" entityFlag="no"> |
<WebFlowContext componentID="uc_2" entityFlag="no"> |
<WebFlowContext componentID="uc_3" entityFlag="no"> |
<ModuleInstance> |
<componentRef>helloModule_1</componentRef> |
<moduleID>ntserver/uc_1/uc_2/uc_3/helloModule_1</moduleID> |
<propertyInstances> |
<propertyVal attrRef="structEventUse"> |
<binaryVal> ........ </binaryVal> |
</propertyVal> ........ |
........................................................ |
A XML document portion to install |
Gateway Application |
User modules can be defined as entity modules such that their attributes are saved across user requests for their methods. |
Currently, we are incorporating Kerberos-based security protocol into proxy objects |
We provide two main class: BeanContextChildSupport and GatewayContext. User define his module by extending from BeanContextChildSupport |
User modules are put inside GatewayContext objects. User can define his own context by extending GatewayContext class. In this way, the system can evolve dynamically. |
EJB - Enterprise Javabeans: Server-side component model. We could not use it because it doesn't have event model yet and also it is expensive. |
JINI, by SUN Microsystems. Our model of event binding is similar to it. |
CORBA component model:not yet implemented |
JavaBeans, by SUN Microsystems: especially JavaBeans Glasgow model. |
WebSubmit and UniCore |
We adopted the JavaBeans event model and removed the requirement of addXlistener/removeXListener method pair to add/remove event "X". Event connection is handled by Gateway context |
We took off the requirement of setX/getX method pairs for attribute "X". We get the attributes from XML document generated from IDL file of user module automatically. |
As opposed to JavaBeans persistency of classic java serialization model, we serialize our objects in XML. |
We adopted the EJB container methodology- all of the transaction , security and persistency can be done by EJB container of one EJB object |
Serialization of EJB objects carried out with XML document (EJB 1.1 model) |
EJB objects can get the services attached to its container object. |
As EJB can give opportunity of defining attributes for EJB objects and store/restore from Database through JDBC, Gateway implemented same functionality. |
Services |
User Modules |
Data Flow |
Front-End |
Middle-Tier modules serve as proxies of Back-End Services |
OO |
Front-End |
User Space Definition and Task Specification |
Metacomputing Services |
Context Objects |
(Green circles) |
Client send |
addNewModule("M") |
method request |
1 |
8 |
3 |
4 |
6 |
7 |
5 |
2 |
New User module |
Mobj with name "M" |
Forward call |
to context object |
Object Factory |
Instantiate module |
with name `M" |
Return module |
Object: Mobj |
Return |
Mobj |
Create Proxy object |
for Mobj |
Context:uc_1 |
Proxy for |
context:uc_1 |
Add Mobj |
into "uc_1" children |
Context inserts new module |
Mobj into its children list |
Context:uc_0 |
Proxy for |
context:uc_0 |
Adding new user module |
to user Context of "UC_1" |
Removing module "M2" from |
user context of "UC_1" |
Context Objects |
(Green circles) |
Client send |
removeModule for |
Module M2 |
1 |
Forward call |
to context object |
PUC_1 |
Context:UC0 |
PUC_0 |
Context:UC1 |
PM0 |
PM3 |
Module:M0 |
Module:M3 |
3 |
2 |
PM2 |
Module:M2 |
PM1 |
Module:M1 |
4 |
Fire property"beanContext" |
change event: |
PropertyChange method call |
5 |
forward |
6 |
Remove connection entry |
from M1 to M2 |
Remove connection entry |
in the local binding table |
from M2 to M3 and |
Module M2 and |
its proxy PM2 |
Context Objects |
(Green circles) |
Client send |
attachEvent call |
with parameters: |
PM0,PM1, |
"propName", |
"targetMethod" |
1 |
Forward call |
to context object |
PUC_1 |
Context:UC0 |
PUC_0 |
Context:UC1 |
PM0 |
PM1 |
Module:M0 |
Module:M1 |
Add this connection |
between modules |
M1 and M2 into |
local binding table |
5 |
4 |
3 |
Add UC1 into Property |
"Beancontext" |
Listeners of module M0 |
Add UC1 into Property |
"Beancontext" |
Listeners of module M1 |
(addPropertyChangeListener) |
2 |
Attach event from Module M1(in context |
"UC1") to M0 (in context "UC1") |
Context Objects |
(Green circles) |
Client send |
detachEvent call |
with parameters: |
PM0,PM1, |
"propName", |
"targetMethod" |
1 |
Forward call |
to context object |
PUC_1 |
Context:UC0 |
PUC_0 |
Context:UC1 |
PM0 |
PM1 |
Module:M0 |
Module:M1 |
Remove this connection |
between modules |
M1 and M2 from |
the local binding table |
5 |
4 |
3 |
Remove UC1 from |
Property Listeners of M0 |
if there are no connections |
after this detachment |
Remove UC1 from |
Property Listeners of M1 |
if there are no connections |
after this detachment |
2 |
Detaching the association between |
module M1 and M0 |
M/S1 |
M/S2 |
M |
M/S2/UC1 |
M/S2/UC2 |
M1 |
M2 |
PS2 |
PUC2 |
PM2 |
PM1 |
PUC1 |
PS1 |
Slave Server:S1 |
Slave Server:S2 |
Master Server:M |
4 |
3 |
2 |
1 |
Execution path |
followed during |
firing event from M1 to M2 |
AcmeAccountHome |
AcmeRemoteAcccout |
JNDI |
Client |
AccountBean |
6 |
5 |
4 |
3 |
2 |
1 |
AccountHome |
Ask for |
AcmeAccountHome |
Return |
AcmeAccountHome |
Create EJB Object |
Return |
EJB Object |
Business |
methods |
Delegate |
requests |
EJB Bean Provider |
class and interfaces |
Classes generated |
by EJB container |
Container |
contract |
EJB Container |
A comparison with EJB: |
a basic Execution flow |
of EJB objects(Enterprise |
JavaBeans) |
An example IDL Definition of user modules |
//store this XML document in file "module_properties.xml" |
<UserModule idlFile="test1.idl" componentID="timerModule_1"> |
<repositoryID>IDL:Gateway/test/timer_1:1.0</repositoryID> |
<properties> |
<simpleDef attID="timerEvent" eventType="genericEvent"> |
<wrapper_interface>IDL:Gateway/test/timerEvent:1.0 </wrapper_interface> |
<simple type="string" name="genericTimerEvent"> |
<description>this is optional desctiption</description> |
<repositoryID>string</repositoryID> |
</simple> |
</simpleDef> |
<structDef attID="timerProperty" eventType="propertyEvent"> |
<wrapper_interface>IDL:Gateway/test/timerProperty:1.0 </wrapper_interface> |
<struct type="struct" name="timerPropertyEvent"> |
<description>this is optional desctiption</description> |
<repositoryID>IDL:Gateway/test/timerstruct:1.0</repositoryID> |
<sequence type="sequence<long>" name="pastValues"> |
<description>this is optional desctiption</description> |
<repositoryID>IDL:Gateway/test/longArray:1.0</repositoryID> |
</sequence> |
<simple type="string" name="currenttime"> |
<description>this is optional desctiption</description> |
</simple> |
</struct> |
...................//Other property definitions |
</properties> |
<methodList> |
<method name="method_1" returnType="void"> |
<parameter name="p1" type="string"/> |
<parameter name="p2" type="long"/> |
</method> |
<method name="method_2" returnType="void"> </method> |
</methodList> |
</UserModule> |
Partial translation |
of the idl file above |
in XML document |
org.omg.CORBA.Object s1_obj, s2_obj, |
uc1_obj, uc2_obj,m1_obj,m2_obj; |
GatewayContext uc1, uc2,s1,s2; |
//get IOR string from fixed url |
String ref = getIORFromURL(masterURL); |
//convert IOR to corba object |
org.omg.CORBA.Object obj = orb.string_to_object(ref); |
//narrow corba Object to specific corba server type |
GatewayContext master = |
GatewayContextHelper.narrow(obj); |
//get the proxy reference of slave server S1 |
s1_obj = master.getContext("M/S1"); |
s1 = GatewayContextHelper.narrow(s1_obj); |
//add new user context with name "uc1" |
uc1_obj = s1.addNewContext("uc1"); |
//narrow it to WebFowContext |
uc1 = GatewayContextHelper.narrow(uc1_obj); |
m1_obj = uc1.addNewModule("timerModule_1"); |
//get the proxy reference of slave server S2 |
s2_obj = master.getContext("M/S2"); |
s2 = GatewayContextHelper.narrow(s2_obj); |
//add new user context with name "uc2" |
uc2_obj = s2.addNewContext("uc2"); |
//narrow it to WebFowContext |
uc2 = GatewayContextHelper.narrow(uc2_obj); |
m2_obj = uc1.addNewModule("timerModule_2"); |
//Finally make "push" type of connection between |
//modules M1and M2. Make sure you issue |
//the "attachPushEvent" //command to the parent |
//of event source where M1 is the source of event. |
//uc1.attachPushEvent(m1_obj," timerEventData", |
//m2_obj,"targetMethod"); |
Continued |
The sequence of Gateway API calls |
to create the above distributed environment which |
includes master and two slaves and two modules |
and their association |
<GatewayContext componentID="M"> |
<GatewayContext componentID="M/S2"> |
<GatewayContext componentID="M/S2/uc2"> |
</GatewayContext> |
<connections > |
</connections > |
</GatewayContext> |
<GatewayContext componentID="M/S1"> |
<GatewayContext componentID="M/S1/uc1"> |
<ModuleInstance> |
<componentRef>timerModule_2</componentRef> |
<moduleID>M/S1/uc1/timerModule_2</moduleID> |
</ModuleInstance> |
<ModuleInstance> |
<moduleID>M/S1/uc1/timerModule_1</moduleID> |
<propertyInstances> |
<propertyVal attrRef="timerPropertyData"> |
<structVal> |
<sequenceVal> |
<simpleValɭ</simpleVal> |
<simpleValɮ</simpleVal> |
<simpleValɯ</simpleVal> |
<simpleValɰ</simpleVal> |
</sequenceVal> |
<simpleVal>startTime</simpleVal> |
</structVal> |
<propertyVal> |
<propertyVal |
attrRef = "timerVetoablePropertyData"> |
<structVal> |
<sequenceVal> |
<structVal> |
<sequenceVal> |
<simpleValɭ</simpleVal> |
<simpleValɮ</simpleVal> |
<simpleValɯ</simpleVal> |
<simpleValɰ</simpleVal> |
</sequenceVal> |
<simpleVal>startTime |
</simpleVal> |
</structVal> |
//...................... Continue |
User can access middle tier |
through abstract job |
specification in XML document |
As we mentioned , Gateway supports server-side collaboration. |
New JavaScript (v 1.3) can provide the capturing of all the events of JavaScript objects. This implies that these events can be collected in Gateway middle-tier and broadcasted to other participants. In this way, JavaScript Web pages can be shared |
Other scenario is about Java applets. If user write his applet following JavaBeans pattern, we can associate two applet running different hosts by interposing Gateway between them. |
CORBA Object |
COM Object |
Gateway context |
Proxy objects acting |
as CORBA and |
COM object |
Proxy objects helps us using the objects |
based on different protocols (COM and |
CORBA here) concurrently. It functions as |
bridge between COM and CORBA objects |
LMS (Landscape management simulation) |
QS-Quantum Simulations |
Seamless Access to HPCC resources |
We will discuss only first two: QS and LMS |
To develop a web based system that implements a "navigate-and-choose" paradigm and allows the end user to:
|
Anytime, anywhere, using any platform |
(e.g., a connected to the Internet laptop PC) |
A decision maker (the end user of the system) wants to evaluate changes in vegetation in a geographical region over a long time period caused by short term disturbances such as a fire or human activity. |
One of the critical parameters of the vegetation model (EDYS) is soil condition at the time of the disturbance. |
This in turn is dominated by rainfall that possibly occurs at that time (CASC2D simulation) |
Input data for the simulations are available from the Internet, such as Data Elevation Models (DEM) from USGS web site or from custom databases (spices characteristics) |
WMS |
EDYS |
CASC2D |
DEM |
Land Use |
Soil |
Texture |
Vegetation |
Data retrieval |
Data preprocessing |
Simulation: two interacting codes
|
Visualization |
LMS: Changes in Vegetation |
WMS: Watershed Modeling System |
EDYS: vegetation model |
CASC2D: watershed model |
Data pre- and post-processing |
Simulations |
Data retrieval |
LMS Front End |
The data wizard allows the user to interactively select the data and |
download them to the local machine. The raw data are then fed to the WMS system launched from the browser to generate input files for simulations. |
Integrates compiled and interpreted HPF-High Performance Fortran which provides powerful program development environment |
This DARP environment includes:
|
Current architecture of DARP. |
The DARP server is a part of the instrumented HPF application and it is replicated |
over the nodes participating in the computation. The client communicates with only |
one server on a selected node. On this node the server acts as a manager. |
Middle-tier : DARP manager controls HPF back-end and |
communicates with other servers |
The instrumentation of the code involves three steps: |
1.Adding server functions |
2.Insertion calls to the HPF server before each HPF statement (SAGE++) |
3.Identification of the types of all variables used in the application |
HPF interpreter |
HPF Front end |
A screen dump of a DARP session. The upper right panel shows the |
front-end applet with a fragment of an HPF code. The action points at which SciVis |
proxy library is called are highlighted, and a triangle on the right points to the |
current line. |
Send_deferred |
for DARP Command |
Poll_response |
for DARP command |
Poll_response |
for command result |
Command result |
DARP |
request buffer |
DARP |
result buffer |
GatewayContext |
extended by DARP manager |
User Modules |
running on processors |
Registered |
clients |
Put DARP command |
with send_deferred |
A complete Integration of DARP |
with Gateway |
A simple representative DARP method |
This shows how DARP server gets |
the next command from middle-tier |
This shows how DARP server put the result of |
previous client command into middle-tier |
We developed a platform independent, three-tier system |
The system offers fine(DARP) and course(Gateway)-grain control on the used distributed application |
It provides persistency model in XML document and Database |
It allows message and source code-based tracking of all interactions among objects |
Currently only supports simple transactions based on storing/recovering the state of any object from/to Database through JDBC |
It permits to put security issues into proxy objects. Therefore it alleviates the security handling for module developer |
It allows multiple sessions concurrently. Even more user can work multiple applications at the same time. |
DARP system can be extended easily to other programming languages such Java, C/C++ by just writing its instrumentation code |
Currently, it supports server-side collaboration but can be extended also client-side collaboration |
We developed a highly-generic multi-purpose and extendible component model. |
We will put the security and transaction handling into proxy functionality. |
Because of using XML for a communication mechanism between front-end and Gateway middle-tier, we can benefit XSL to translate XML to HTML, any audio and visual effects if any. |