Given by Geoffrey C. Fox, Tom Haupt at NCSA Portal/Metadata Meeting on October 22-23 99. Foils prepared October 24 99
Outside Index
Summary of Material
3-Tier Architecture with two well defined API's in XML
|
Portals Built and in Progress
|
Features of WebFlow/Gateway
|
General Issues and Special Services
|
Outside Index Summary of Material
NPAC Syracuse University 111 College Place |
Syracuse NY 13244-4100 |
Geoffrey Fox |
Tom Haupt |
NCSA Alliance Portal Meeting UIUC |
Oct 22 1999 |
3-Tier Architecture with two well defined API's in XML
|
Portals Built and in Progress
|
Features of WebFlow/Gateway
|
General Issues and Special Services
|
Middle-Tier:Framework + CORBA |
Grid Services | JDBC | Local Resources |
Abstract Task Specification |
CTA specific knowledge databases |
Visual |
Authoring |
Tools |
User and |
Group |
Profiles |
Resource Identification and Access |
Visualizations |
Collaboration User Services |
Back-End Resources |
Problem Solving Environment |
"WebFlow" |
(Computing Portal) API |
Grid API |
WebFlow |
Portal Building Tools and Frameworks (XML, iPlanet, Portlets, www.desktop.com) |
Enterprise Portals |
Generic Portals |
Information Services |
Compute Services |
Education and Training Portals |
Science Portals |
K-12 |
University |
Biology |
Chem Egy |
Generic Services |
Collaboration |
Universal Access |
Security ....... |
Databases ....... |
MathML etc |
Quizzes Grading ... |
Education Services |
Grid Services |
Visualization ... |
User customization, component libraries, fixed channels |
One Possible Front End is Graphical Composition |
Use Web Browsers as new W3C standards with version 5 navigators have well defined standards and high functionality |
Middle Tier Supports Proxies as Components and Containers |
Layered |
PSE |
WebFlow |
Object |
Oriented |
WebFlow |
Data Flow |
Custom |
GUI |
WebFlow Middle Tier |
DATORR/Computing Portal |
Grid Forum |
HPCC: Globus |
Other as needed |
DBMS: JDBC |
Small tasks: Java |
user codes |
https, IIOP/SECIOP |
Example: DoD HPCMO ASC/ARL |
Prototype Version |
NCSA Alliance |
Example: LMS |
General or WebFlow Specific Authoring |
Control (Small Messages) |
Separated |
from Data |
Result Descriptor |
Problem |
Descriptor |
Task Descriptor |
Abstract Task Descriptor |
Job Descriptor |
User/Group Profile |
Client Side User View |
X M L |
Processing Task Specification |
XML |
WebFlow visual representation is converted into a XML |
document |
XML |
service |
Web |
Server |
save |
parse |
Application Context |
Front-End Applet |
Middle-Tier |
Database |
WebFlow system comprises a hierarchy of CORBA objects that implement interfaces defined in java.beans.beancontext (Java 1.2):
|
Although implemented differently, conceptually close to EJB and CORBA 3 Components models.
|
How does it work? CORBA-based Middle Tier (Proxy modules - a simplified example) |
Front End |
Middle Tier |
Back End |
host 1 |
host 3 |
CORBA object implemented in Java |
CORBA client |
WebFlow components (modules) are developed independently of each other. |
They can interact with each other through WebFlow events. (They can use remote invocation, but it requires the caller to have reference to the target module, which violates the WebFlow "module independence" rule). |
There is only one type of WebFlow events. An event (same as a message) is a (self-describing) object that encapsulates data to be transferred from one module to another. |
Each module can serve as a sink. |
Event sources and sinks are associated at run time by the client invoking CORBA dynamic interfaces. |
Method m |
Client |
B |
Register event to correspond to linkage of modules in visual WebFlow diagram: "Let firing event e by module A |
result in invocation of method m of module B" |
Context 2 |
Module A does not care who is expecting the event; method fireEvent invokes a method of its parent context |
Method m is a public method: anyone can invoke it, including the Event Adapter of Context 1. No protection against misuse! |
Dynamic |
Interfaces |
A |
B |
public class client |
extends Applet { |
... |
org.omg.CORBA.Object o =orb.string_to_object(IOR); |
WebFlowContext wfc1 = WebFlowContextHelper.narrow(o); |
moduleA moduleARef = moduleAHelper.narrow(wfc.addNewModule("moduleA")); |
... |
wfc1.attachPushEvent(moduleARef, "eventName", moduleBRef, "aMethod"); |
WebFlow supports both push and pull events
|
The WebFlow event model can be upgraded to the proposed CORBA 3 model:
|
Jini model with intermediate match maker should also be supported |
Objects register (using technology like Jini or that in WebFlow ) and then define their state or its change via a stream of events |
Need to share (federate) events between domains (your PC, Supercomputer, PDA and mine) whose event services are normally isolated |
"Tango or Habanero Server" |
Object or Visual |
Proxy for Object |
Event specifying Object |
or Change in Object |
Federate and |
Queue events |
Clients which receive events which are either queued (asynchronous) |
or processed synchronously |
Federated Message Processing Infrastructure seems to underlie PC Pager Palmtop Cellular Synchronous/Asynchronous Convergence and a Great Business |
Event Stream becomes set of time stamped XML messages
|
Content Server |
Shared by Collaboration Infrastructure |
Events |
Trapped by Browser |
Master |
Nonmaster |
Shared Multiple List |
Shared Buttons |
Shared Text field |
Shared Checkbox |
Tango Interactive Shared Web Browser (in Shared Form Mode) |
WebFlow server is given |
by a hierarchy of containers |
and components |
WebFlow server hosts users and services |
Each user maintains a number of applications composed of custom modules and common services |
User 1 |
User 2 |
Application 1 |
Application 2 |
App 2 |
App 1 |
WebFlow Services |
Clients |
Distributed Back-End Resources |
Download Applet |
WebFlowContext Proxies |
Task Descriptor |
Grid Interface ? JBDC ? Information Services |
"slave" WebFlow Server |
SECIOP |
Front End Applet |
SECIOP |
authentication |
& authorization |
Gatekeeper |
delegation |
HPCC resources |
GSSAPI |
GSSAPI |
Layer 1: secure Internet access |
Layer 2: secure |
middle tier |
Layer 3: Secure access to resources |
Policies defined by resource owners |
A principal is authenticated once by ORB and given a set of credentials, including one or more roles, privileges, and an authenticated ID. |
An authenticated ID is automatically propagated by a secure ORB; it is part of the caller context |
Principal |
Credentials |
Current |
Client |
Server |
set_credentials |
get_attributes |
authenticate |
kinit |
generate TGT |
locally |
ticket cache |
SECIOP |
Applet |
ORB.init() |
invoke server methods |
ORB |
WebFlow modules |
WebFlowServer |
ORBAsec SL2 2.0 |
Java 1.1 plugin |
Application Descriptor (XML) |
WebFlow Service: |
XML Parser |
Client (Java Applet) |
Select Application |
Select Host |
Select, Edit, Upload Input Files |
Submit |
View Results |
WebFlow Service: |
File Services |
WebFlow Service: |
Job Service |
Back-End Compute Server |
Middle Tier (User Context) |
1 |
2 |
3 |
6 |
5 |
4 |
1. Generate Application Front End from XML specification |
2. Use WebFlow to move files between hosts 3. Generate Task Descr. 4. Use Task Descriptor and XML Appl. Descr. to generate RSL 5. Submit job through GRAM (+GASS). Save job info in the |
User Context. Poll status of the job 6. Move output files back to the client and launch visualizer |
<?xml version="1.0"?> |
<!DOCTYPE application SYSTEM "ApplDescV2.dtd"> |
<application id="Casc2d" installable="No"> selected application |
<target id="aga.npac.syr.edu"> selected host |
<status installed="Yes"/> |
<installed> |
<CmdLine command="/npac/home/haupt/CASC2D/casc2d" /> |
<input> |
<inFile Path="/npac/home/haupt/CASC2D/lms/" Name="sand.map"/> |
<source Host="maine.npac.syr.edu" Path="C:\LMS\fromEdys\" Name="S.map" > |
</input> |
<output> |
<outFile Path="/npac/home/haupt/CASC2D/lms/" Name="sed.out"/> |
<dest Host="maine.npac.syr.edu" Path="C:\LMS\toEdys\" Name="sed.out" > |
</output> |
<stdout Host="aga.npac.syr.edu" Path="/npac/home/haupt/CASC2D/history/" Name="job2001.out" > |
<stderr Host="aga.npac.syr.edu" Path="/tmp/" Name="haupt_job2001.err" > |
</installed> |
</target> |
</application> |
how to run it |
it expects this input file |
Actual location of the file |
it generates this output file |
store it permanently here |
save stdout |
and stderr |
Currently each application is wrapped "by hand" |
Users would like to automate the process -- "especially for simple jobs" -- need a Object Wrapping Wizard |
Current XML Task Specification is at level of files |
Will extend XML to include parameters so can automate construction of
|
<input> |
<inFile Path="/npac/..../GEM/" |
Name="disloc.output"/> |
<source Host="osprey4.npac.syr.edu" |
Path="/npac.../tmp" Name="disloc.out"/ > |
</input> |
<output> <outFile Path="" Name="" /> |
<dest Host="" Path="" Name="" /> |
</output> |
Current Tool for creating Application Descriptor |
WebFlow |
server |
WebFlow |
server |
WebFlow |
server |
EDYS |
CASC2D |
Data Retrieval |
High Performance SubSystem |
CASC2D |
proxy |
IIOP |
Web Browser |
Data Wizard |
WMS interface |
Toolbar |
HTTP |
WMS |
File Transfer |
File Transfer |
GLOBUS |
Internet |
WebFlow modules |
(back-end) |
WebFlow |
middle-tier |
WebFlow applet |
(front-end) |
Navigate and choose an existing application to solve the problem at hand. Import all necessary data. |
Retrieve data |
Pre/post-processing |
Run simulations |
Select host |
Select model |
Set parameters |
Run |
<TaskDescriptor> |
<TaskName name=Gaussian> |
</TaskName> |
<application descriptor="gauss.xml"/> |
<application descriptor="analyze.xml"/> |
<application descriptor="analyze_failure.xml"/> |
<connection> |
<output application="gauss.xml" event="done" /> |
<input application="analyze.xml" method="run" /> |
<connection> |
<connection> |
<output application="gauss.xml" event="failure" /> |
<input application="analyze_failure.xml" method="run" /> |
<connection> |
</TaskDescriptor> |
Components are defined through their application descriptors |
Components interact through event notification |
module WebFlow { |
module FileBrowser { |
typedef sequence<string> FileList; |
interface WFFileDescriptor { |
Object getSource(); |
string getFileName(); |
string getFileDirectory(); |
string getFileHost(); |
string getFileAttribute(); |
void setFileName(in string Name); |
void setFileDirectory(in string Name); |
void setFileHost(in string Name); |
void setFileAttribute(in string Name); |
}; |
}; |
disloc |
ALARM |
Dial Stations(and database) |
GIPSY/auto_p |
simplex |
page |
web simplex |
Caltech |
JPL |
USGS |
JPL |
JPL |
Boulder |
(University of Colorado) |
JPL |
modem |
page |
quake location, size -- |
sorted station potential -- |
station raw files -- |
station motions -- |
WAKE UP! |
single-fault model |
--graphics |
--hazard model |
--graphics |
--refined fault model |
disloc |
--maps for |
civil authorities |
JPL |
Virtual_California |
multi-fault model |
page |
disp |
collaboration |
Prepare Data |
disloc proxy |
Post Process |
Generate disloc input |
disloc proxy |
disloc2simplex proxy |
simplex proxy |
compare |
disloc code written in C |
disloc2simplex script written in perl |
simplex code written in C |
Front End: |
Java Applet. |
CORBA client |
Middle Tier: |
distributed objects |
implemented in Java. |
CORBA servers |
Back End: |
computing |
resources |
Iterate |
Collaboration in GEM |
Earthquake Analysis System |
<?xml version="1.0"?> |
<!DOCTYPE application SYSTEM "ApplDescV2.dtd"> |
<application id="disloc"> |
<target id="osprey4.npac.syr.edu"> |
<status installed="Yes"/> |
<installed> |
<CmdLine command="/npac/home/webflow/GEM/JAY/dis2loc" /> |
<input> |
<inFile Path="/npac/home/webflow/GEM/JAY/" Name="disloc.output"/> |
<source Host="osprey4.npac.syr.edu" Path="/npac/home/Jigsaw/WWW/tmp" |
Name="disloc.out"/ > |
</input> |
<output> |
<outFile Path="/npac/home/webflow/GEM/JAY/" Name="simplex.input" /> |
<dest Host="osprey4.npac.syr.edu" Path="/npac/home/webflow/GEM/JAY/simplex/" Name="s.in" /> |
</output> |
<stdout Host="aga.npac.syr.edu" Path="/npac/home/haupt/webflow/history/" Name="job2001.out" > |
<stderr Host="aga.npac.syr.edu" Path="/tmp/" Name="haupt_job2001.err" > |
</installed> |
</target> |
</application> |
Application area specific information system allows the user to define his/her problem and express it in terms of computational methods, algorithms, ... |
Once the problem is defined the user is assisted in selecting codes that provide solution tothe problem, generate input files, submit the job and analyze results. |
Project context represents a problem. A particular selection of application(s), input data, and results is encapsulated as a session context. All attributes of a context are persistent. (Saved as XML) |
Select & Submit Job |
File Service |
Job Service |
Application |
Task Descriptor |
Globus RSL |
Filter |
AVS Express |
Select Data |
Middle Tier |
Front End |
Back End |