Given by Geoffrey Fox at Supercomputing 96 on 18 November 96. Foils prepared Sept 30 1996
Outside Index
Summary of Material
Secs 76.3
We describe key Web Technologies with illustrations of their use
|
We describe a Web Software Architecture with NII (National Information Infrastructure) applications built on top of generic services which are themselves built with Web Technologies and implemented on top of the "WebWindows" operating environment |
We review NII services which include Commerce, Desk(Web)Top productivity, InfoVision (multimedia information dissemination), Collaboration and MetaComputing |
We describe NII applications such as manufacturing briefly and focus on three areas:
|
Outside Index
Summary of Material
Geoffrey Fox |
David Warner |
NPAC |
Room 3-131 CST |
111 College Place |
Syracuse NY 13244-4100 |
We describe key Web Technologies with illustrations of their use
|
We describe a Web Software Architecture with NII (National Information Infrastructure) applications built on top of generic services which are themselves built with Web Technologies and implemented on top of the "WebWindows" operating environment |
We review NII services which include Commerce, Desk(Web)Top productivity, InfoVision (multimedia information dissemination), Collaboration and MetaComputing |
We describe NII applications such as manufacturing briefly and focus on three areas:
|
Firstly we can use this technology to implement HPCC on a broad technology base
|
Secondly we can use technology to implement Virtual University to teach internally and across the Globe |
Thirdly we can teach our students about these concepts
|
Parallel Computing Works! |
Technology well understood for Science and Engineering
|
Supercomputing market small (few percent at best) and probably decreasing in size
|
No silver programming bullet -- I doubt if new language will revolutionize parallel programmimng and make much easier
|
Social forces are tending to hinder adoption of parallel computing as most applications are areas where large scale computing already common
|
ATM ISDN Wireless Satellite advancing rapidly in commercial arena which is adopting research rapidly |
Social forces (deregulation in the U.S.A.) are tending to accelerate adoption of digital communication technologies
|
Not clear how to make money on Web(Internet) but growing interest/acceptance by general public
|
Integration of Communities and Opportunities
|
Technology Opportunities in Integration of High Performance Computing and Communication Systems
|
New Business opportunities linking Enterprise Information Systems to Community networks to current cable/network TV journalism |
New educational needs at interface of computer science and communications/information applications |
Major implications for education -- the Virtual University |
This is both Grand Challenges augmented by National Challenges but also |
Build HPCC technologies on a broad not niche base starting at bottom not top of computing pyramid |
It provides a pervasive and highly innovative technology base
|
Note Intel Teraflop computer will have 9000 P6's but WWW will have at least one thousand times as many possible nodes varying (eventually) from supercomputers to settop boxes |
WWW is important as it allows rich (world wide linkage of simulation and information capabilities) open and portable environment |
Each of three components (network connections, clients, servers) has capital value of order $10 to $100 Billion |
InfoVision is ultimate "client-server" application
|
Democracy on the NII (Gore)
|
Application Specific NII Specific Services for
|
Java -- a partially interpreted C++ like language (script) allowing fully interactive clients which execute applets.
|
JavaScript -- A fully interpreted Web Systems integration Language |
VRML -- a 3 dimensional HTML allowing universal description of physical objects and allowing interchange of virtual worlds, commercial product designs etc. Latest version 2.0 suuports interactivity |
Web-linked Databases such as Oracle connected to emerging JDBC (Java Database Connection) |
See http://www.genmagic.com/internet/trends/ |
See http://www.genmagic.com/internet/trends/ |
See http://www.genmagic.com/internet/trends/ |
See http://www.genmagic.com/internet/trends/ |
In future one will NOT write software for either
|
Rather one will write software for WebWindows defined as the operating environment for World Wide Web |
WebWindows builds on top of Web Servers and Web Client open interfaces as in
|
Applications written for WebWindows will be portable to all computers running Web Servers or Clients which hide hardware and native O/S specifics |
WebWindows Interface |
Further WebWindows Software will be modular and allow plug and play insertion of capabilities developed around the Web World -- not a bunch of isolated stovepipe solutions
|
As an example some of Current Netscape and last year(!) NPAC's WebTools implements UNIX shell/PC file manager capabilities in terms CGI scripts -- allows universal access to these capabilities including powerful Web based (mh) mail |
NPAC's WebFoil is HotJava/Netscape 1,2,3 Open replacement for Powerpoint/Persuasion |
Particular Application areas (Business, Healthcare, Education) will be built on top of generic NII services so that for instance
|
Persuasion and Powerpoint are rather similar monolithic packages which can for instance only be clumsily ported to UNIX as cannot access internal data-structures defining foils |
WebFoil (NPAC prototype WebWindows presentation package) has |
Extended open HTML source manipulated by powerful PERL5 scripts allowing global changes and linkages of foils from many sources
|
Backend Oracle database illustrating modular WebWindows approach |
Using Appropriate templates WebFoil Uses Hotjava or Netscape 1,2 or 3 to display HTML with full Web Power including applets to enable Multimedia and dynamic presentations |
Initial webfoil 0.1 release Halloween 1995 |
Rome Laboratory Collaborative and Interactive Visualization Jan 31,96 |
The WebTop Productivity environment will be built in a more modular fashion than current PC Windows or Macintosh arena
|
Java or equivalent future technology is key to understanding how WebWindows application/service software will look as it allows balanced client server applications to be built |
Note require an open display software so can produce appropriate customized interfaces for browsing, presenting, word processing etc. |
The World Wide Web will develop a marvellous WebTop Environment allowing users to access Web versions of Word Excel(Lotus123) Persuasion(Powerpoint) Databases Lotus Notes etc.
|
This (as developed) enables a wonderful new software industry as Microsoft no longer has key advantages |
This (as used) enables very effective (business) enterprise systems where "Web" used internally to an enterprise
|
Web Servers use "Web Technology" to service World Wide Web and other forms of networked multimedia information |
InterNet versus IntraNet Web Business |
From Information Week Jan 29, 96 issue |
Application Specific NII Specific Services for
|
Browsers have SAME interface on ALL Computers |
CGI Programs are typically written in PERL but can be essentially ANY UNIX Process and so do simulation, database access, advanced document processing etc. |
There are evolving/confusing/overlapping capabilities ... |
PERL is a relatively old technology which is being overtaken by Java tidal wave. Still PERL has much better Systems and Document handling capability than Java
|
MIME stands for Multipart Internet Mail Extensions and is the developing standard for the contents of all messages passed over the Internet. |
HTTP is Hypertext Transport Protocol and is the protocol that provides the basis of the World Wide Web: transmitting multimedia documents across the Internet. HTTPD is the daemon running the HTTP Web server. |
URL stands for Uniform Resource Locator and is the universal addressing scheme for all documents (multimedia) on the WWW. |
CGI is the Common Gateway Interface and is the scheme to interface other programs and systems to the HTTP Web protocol, using the same data protocols as the HTTP clients and servers. |
References:
|
The client sends a request, conforming to the URL standard and formatted with a MIME header, to the server. |
The server parses the request and decides what to do:
|
The CGI program parses the input from the server and MUST generate a response - even if there is no data to send back, the CGI program must send an error or empty message since the http connection is still open and must be closed by the server. The CGI program will send a header to the server:
|
When the CGI program terminates, the server closes the connection. |
This example consists of a simple form with just a submit button to activate the CGI program. Note that no data is being sent from the form to the CGI program in this simple example. |
The Perl program returns output which is properly formatted HTML. The server returns it to the browser, which displays it as a page. |
Returning the html output is pretty simple as the server and browser handle the encoding and decoding of the MIME formatted message. The complications arise from sending text from the form to the CGI program; there are several ways to do it and the CGI program must decode the message. |
This subroutine works with either the GET or POST method, obtaining the user input string from the form into a scalar variable "$in". It then splits this string into fields into the array "@in", where each element contains the encoded string for one field. |
For each field string, the subroutine converts all the encoding symbols. It then creates an associative array "%in" with a keyword,value pair from each field of the web form. |
This subroutine can be used without change in any Perl CGI program, unless you wish to have checkboxes on the form which may return the same name with more than one value. |
Java -- Objected Oriented version of C/C++ supporting Interactive Distributed Computing. Previous Web computing (eg CGI) was server-side. Java allows design and Implementation of balanced Client Server Applications
|
Java likely to be a dominant language as will be learnt and used by a broad group of users
|
Clearly Java can easily replace Fortran as a Scientific Computing Language as can be compiled as efficiently and has much better software engineering (object) and graphics (web) capabilities |
Java may replace C++ as major system building language
|
An example of HotJava applet that makes essential use of Java multithreading. |
Three different sorting algorithms are visualized on a single HotJava page. |
Each algorithm can be started independently or they can all run concurrently. |
Concurrent mode allows for real-time visual comparison of various algorithms and their performance. |
HotJava Demonstration |
HotJava Demonstration |
From Wojtek Furmanski and Zeynep Odcikin Ozdemir |
From Wojtek Furmanski and Zeynep Odcikin Ozdemir |
Since Java is object-oriented, programs are organized into modules called classes, which may have data in variables and functions called methods. |
Java applets are part of the class hierarchy that can call methods to display on a screen (within the browser window). This example defines the public method paint in this class and calls a method drawString defined in the class Graphics. |
You should name the file with your applet name, HelloWorldApplet.java, run the compiler (javac), getting a bytecode file HelloWorldApplet.class, which you put in a web directory. |
An object of the graphics class represent a rectangular drawing grid, with a coordinate system in pixels. |
When you draw objects, there is a current "state" consisting of a font and a color. |
Fonts and colors are objects (sometimes called instances) of the font and color class in the awt package. |
l Good l Fair l Poor |
My SGI INDY gets .54 Megaflops for Java 100 by 100 Linpack |
It has 200 Mhz R4400 and current Netlib benchmark for this chip is 32 mflops for optimized Fortran |
For better resolution see JPEG Version |
See Original Foil |
Java is currently semi-interpreted and (as in Linpack online benchmark) is about 50 times slower than good C or Fortran |
Java --> (javac)--> Downloadable Universal Bytecodes --> (Java Interpreter) |
--> Native Machine Code
|
However Language can be efficiently compiled with "native compilers" |
Java ----> (native compiler) |
---> Native (for Particular Machine) Code |
Lots of Interesting Compiler issues for both compiled and scripted Java |
see http://www.netlib.org/benchmark/linpackjava/ |
Note Just in Time Compilers are giving a factor of 10 from June 96 Measurements! |
One can use "native classes" which is just a predownloaded library of optimized runtime routines which can be high performance compiled Java, C, C++, Fortran, HPF etc. modules invoked by interpreted or compiled Java
|
Use Native Classes selectively for
|
Web Servers and HTTP are not as efficient as PVM/MPI daemons and their messaging but
|
Deploy Web technology first in education and in program development where high functionality of "Web Productivity Environment" is more important than performance |
Then run production in classic "bare-bones" HPCC environment |
Internet is quite slow and getting slower but in fact many Web activities focus on IntraNets -- domain and perhaps geographically specialized hardware running pervasive Web Softwate
|
Superficially one can state goal as adding to the distributed computing model of the Web, the HPCC lessons and algorithms needed for high performance and tight synchronization of multiple servers and clients (Web is typically loose coarse grained coupling).
|
JavaScript -- only superficially related to Java and was called LiveScript -- is Netscape's fully interpreted Client side extension of HTML. This is a good integration/customization technology where flexibility more important than performance |
i.e. use JavaScript for Rapid Prototyping
|
There are some cases where JavaScript is a rapid prototyping alternative to Java (Java can in principle do anything!) |
Current use (demonstrated in WebWisdom) is for customized display's of information where base information, simulations, audio(video), and indices can be linked in a friendly fashion. |
Syracuse is extending to a set of WebTools (originally developed as Server CGI scripts) with which you can build customized front-ends with user defined configurations choosing between type of access (administrator and naive user would be different) and display capabilities (resolution, color) of terminal |
Note client side and so fast even though interpreted |
However does not yet have (but needs) text processing capabilities of Perl |
Top document has JavaScript in <head> </head> and defines <frameset></frameset> split in two by columns |
Rome Laboratory Collaborative and Interactive Visualization Jan 31,96 |
Rome Laboratory Collaborative and Interactive Visualization Jan 31,96 |
Rome Laboratory Collaborative and Interactive Visualization Jan 31,96 |
Historically JavaScript was called LiveScript and developed by Netscape with some of the same goals as Java but focussed on a "smaller world" -- manipulation of text and options connected with Netscape Clients |
Now we can use it as an alternative to Java where the "rapid prototyping" of a fully scriped language is helpful |
JavaScript can be thought of as Java with the AWT (Abstract Windowing Toolkit) replaced by Netscape Client |
JavaScript particularly useful for multi-frame windows and for manipulating forms without complex CGI (Server Side) scripts |
<HTML> |
<HEAD> |
<Title>A Test of JavaScript</Title> |
</HEAD> |
<BODY> |
<SCRIPT LANGUAGE="JavaScript" > |
<!-- A Conventional comment to hide JavaScript from old browsers |
document.writeln("<h1>Hello World!</h1>"); |
//scriptend--> |
</SCRIPT> |
<b>Continue with conventional HTML</b> |
</BODY></HTML> |
There is only one real JavaScript statement here -- namely |
document.writeln("textstring"); |
This outputs into current page the text in quotes followed by a newline |
And note the rather peculiar way we "hide" JavaScript from browsers that can't understand it by enclosing in convential HTML Comment syntax |
<SCRIPT LANGUAGE="JavaScript" > |
<!-- A Conventional comment to hide JavaScript from old browsers |
...... Bunch of JavaScript Statements ....... |
//scriptend--> |
</SCRIPT> |
Note depending on your needs, JavaScript can be in Header or Body section of document |
<HTML><HEAD><TITLE>Javascript with Forms</TITLE> |
<SCRIPT LANGUAGE="JavaScript" > |
<!-- A Conventional comment to hide JavaScript from old browsers |
function compute(form) { |
if( confirm("Is this what you want?")) |
form.result.value = eval(form.expr.value); |
else alert("Enter a new expression then!"); } |
//scriptend--> |
</SCRIPT></HEAD> |
<BODY><FORM> |
Enter An Expression: |
<INPUT TYPE="text" NAME="expr" SIZE=15> |
<INPUT TYPE="button" VALUE="DoIt!" ONCLICK="compute(this.form)"> |
<BR>Result: |
<INPUT TYPE="text" NAME="result" SIZE=15> |
<BR> |
</FORM></BODY></HTML> |
Enter An Expression: 9+5 |
Result: 14 |
confirm is a native Javascript method popping up a window, requesting confirmation of requested action |
alert is a native Javascript method popping up a window with a message requiring user to place OK to get rid of. |
onclick="Javascript Statement Block" naturally executes statement(s) when button clicked |
<HTML><HEAD><TITLE>Javascript for Parameterizing HTML</TITLE> |
<SCRIPT LANGUAGE="JavaScript" > |
<!-- A Conventional comment to hide JavaScript from old browsers |
var imagewidth=600; // These could be changed by form input or some |
var imagefile="npac.gif"; // computation based on size of window etc. |
//scriptend--> |
</SCRIPT></HEAD> |
<BODY> ......Bunch of Normal Stuff |
<SCRIPT LANGUAGE="JavaScript" > |
<!-- A Conventional comment to hide JavaScript from old browsers |
document.writeln('<img align=top width=' + imagewidth + ' src="' + imagefile + '" >'); |
//scriptend--> |
</SCRIPT> |
.... Yet More Normal Stuff |
</BODY></HTML> |
Note single quotes used for JavaScript, Double quotes for HTML -- can use \' if necessary to hide special meaning from JavaScript |
Netscape renames Livescript as Javascript and this is an interesting variant of Java which is fully interpreted -- use for overall customization of client |
Use Java for detailed programming and JavaScript for overall integration of client interface and system |
JavaScript: Interpreted by client and NOT compiled |
Java: Compiled on Server before execution on client
|
JavaScript: Object based -- no classes or inheritance -- built in extensible objects |
Java: Object-oriented. Programs consist of object classes with inheritance |
JavaScript: Integrated with HTML as embedded ascii but of course HTML looks rather irrelevant at times! |
Java: Applets distinct from HTML but invoked from HTML Pages |
JavaScript: do not declare variables' datatypes -- Loose typing |
Java: MUST declare variables' datatypes -- Strong typing |
JavaScript -- Dynamic Binding -- object references computed at runtime |
Java -- Static Binding -- object references must exist at compile time |
Java and JavaScript are secure and cannot write to disk |
VRML 1.0 widely available and specifies static 3D scenes through which you can navigate. Already provides universal visualization environment and we have examples of use In Geographical Information Systems
|
VRML 2.0 is just released with prototype browsers but still aspects of technology are under intense research/debate.
|
Data is stored in Illustra Database System
|
VRML representation is created in real time when requested |
The same data may be visualized in various ways (terrain, objects) |
Parameters like resolution, size, altitude magnification, etc. are set by the user |
From Chris Walczak |
Living SchoolBook Material for SC95 San Diego Dec 95 |
NPAC Web Based Geographical Information System in Stand Alone Mode |
A GIS application is a specialized OpenInventor viewer, however it accepts any OpenInventor 2.1 scene model. That's why it's so easy to integrate it with third party applications, which produce IO/VRML output. The images show GIS integration with Weather Simulation application. A GIS viewer can also display animated objects controlled by Simulation Engine. |
NPAC Web Based Geographical Information System in Stand Alone Mode |
A GIS application is a specialized OpenInventor viewer, however it accepts any OpenInventor 2.1 scene model. That's why it's so easy to integrate it with third party applications, which produce IO/VRML output. The images show GIS integration with Weather Simulation application. A GIS viewer can also display animated objects controlled by Simulation Engine. |
Mission Statement: |
VRML (Virtual Reality Modeling Language) is an interpreted language for describing multi-participant interactive simulations - virtual worlds networked via global Internet and hyperlinked within the World Wide Web. |
History: |
VRML concept was proposed by Tim Berners-Lee and David Ragget in spring '94 during the first WWW conference in Geneva. After the conference the VRML forum was created by Mark Pesce to discuss the development of a specification. In May '95 the first version of specification was available. |
Current status:
|
The current version of VRML specification - 1.0 is a minimal starting point for a much larger concept. |
The specification is based on Open Inventor ASCII file format. |
VRML 1.0 is a subset of Open Inventor with some additions that allow linking the objects with another VRML or HTML sites on the Web. The linking concept is similar to HREF in HTML. |
VRML 1.0 provides only mechanisms for synthetic 3D "clickable worlds" and does not specify yet any constructs for object animation, behavior and interaction. These issues are currently under intense discussion by the VRML forum (and spinoffs such as VAG) and is included in version 2.0 of the language. |
Basics:
|
Nodes are characterized by the following information:
|
Shape nodes (Cube, Sphere) define the geometry of the objects in the scene. |
The Material node defines the current surface material properties for all subsequent shapes. |
Separator node isolates its children from the rest of the scene graph. |
Texture node defines the texture map used to subsequent shapes. |
VRML nodes can be classified into three categories:
|
Replaced by Transform in VRML2.0 |
This group node performs a push (save) of the traversal state before traversing its children and a pop (restore) after traversing them. |
This isolates the separator's children from the rest of the scene graph. |
A separator can include lights, cameras, coordinates, normals, bindings, and all other properties. |
Separators can also perform render culling. |
Render culling skips over traversal of the separator's children if they are not going to be rendered, based on the comparison of the separator's bounding box with the current view volume. |
Culling is controlled by the renderCulling field.
|
CULLING ENUMS
|
FILE FORMAT/DEFAULTS
|
Coordinate3 {
|
} |
The point field defines the vertices of the object. |
There are five points defined. The points are labeled form 0 to 4. |
Coordinate3 just defines points to be used later in IndexedFaceSet |
Replaced by Coordinate in VRML2.0 |
IndexedFaceSet {
|
} |
Builds Pyramid as 6 triangles formed from original points defined in Coordinate3 |
This part coordIndex unchanged in VRML2 |
This node represents a 3D shape formed by constructing faces (polygons) from vertices located at the current coordinates. |
IndexedFaceSet uses the indices in its coordIndex field to specify the polygonal faces. |
An index of -1 indicates that the current face has ended and the next one begins. |
The vertices of the faces are transformed by the current transformation matrix. |
Treatment of the current material and normal binding is as follows:
|
The DEFAULT material binding is equal to OVERALL. |
The DEFAULT normal binding is equal to PER_VERTEX_INDEXED;
|
MaterialBinding Specification totally changed in VRML2 -- its easier! |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
Screen of User1. Talking Heads and Whiteboard applets started. User2 collaborates with User 1 in 2 sessions (Talking Heads and Whiteboard). |
Screen of User1. Various applets started. Three users logged in. Collaboration in different sessions presented. |
The Web provides a convenient integration environment for "mature" technologies migrating from existing computer environments. |
Relational databases are a good example where it is now straightforward in Oracle, DB2, Sybase etc. to provide a Web Interface which can be used for data (mail, curricula material etc.) with Java/JavaScript/Forms based Interfaces |
Object databases such as Illustra also interfaced to Web |
Several excellent Java to Database packages becoming available |
CORBA will have good Web and Java Interfaces |
This uses NPAC Web linked Database Technology |
NewsGroup and "Chat" Messages stored in Database giving searchable record of collaboration or discussion. |
Enterprise IntraNets -- Carrier Corporation (started) and Other Fortune 500 companies (under negotiation) for external (catalog) and internal databases |
Support of Option Pricing on Demand for financial industry |
Images from New York State for Education and Tourism |
Text from CD-ROM's and other digital Information sources |
Close Caption and Programming Text to Index Video for Digital Wire Service and other media applications |
Electronic Mail -- how do I keep track of 50 messages a day? |
Digital Books to support Computer Science Education |
Search all URL's inside a particular information domain (from NPAC Web Site to particular Course in our virtual University |
All Map data (for New York State) and objects therein -- from 3D weather simulations to converted AutoCad specification of your home |
Note: the gateway wowstub program simply passes PL/SQL program name and input parameters gathered from forms to DB server. |
The DB server does both SQL query and HTML processing/formatting |
Original Form of Data is mapped in batch or real time |
Natural Storage Format for particular type of Information |
Optimal Format for network transmission incorporating synchronization as in audio and video streams as well as compression |
Local Client formatting to (HTML,VRML) needed for standard browser display standards |
HyperText Procedures (HTP)
|
HTF (HyperText Functions) is the corresponding package which contains PL/SQL functions for generating HTML tags |
i.e. it returns HTML string whereas HTP writes HTML
|
Details about syntax and use of each procedure and function in HTP and HTF can be found from the handout |
CREATE PACKAGE EXAMPLE is procedure get_phoneno_by_name(name IN VARCHAR2); -- only one procedure |
END EXAMPLE; |
CREATE PACKAGE BODY EXAMPLE is
|
BEGIN
|
/* print out the query result */ |
htp.p(htp.item||'The ' || ptype ||' phone no. of ' || fname || ' ' |
|| lname || ': ' || phone); |
END LOOP; |
CLOSE person_cur; -- close the cursor after it is done |
htp.ulistClose; |
END get_person_by_name; |
END EXAMPLE; |
A HTML form page to accept user search input and invoke the CGI script on the web server to access the database
|
Query results are displayed in another Web page as (if user typed in 'geoffrey') |
<html><header> |
<title>Query Results</title></header> |
<body><h1>Query Results</h1> |
<ul> |
<li>The office phone no. of Geoffrey Fox: 3154434889 |
<li>The home phone no. of Geoffrey Fox: 3154238422 |
</ul> |
</body> |
</html> |
LOWER is a built-in function converting a string into its lower case |
|| concatenates two strings into one. E.g., 'CPS'||'616' equals 'CPS616' (same as + in Java, . in Perl) |
pname dbname%TYPE implies PL/SQL variable pname inherits data type of dbname |
LIKE is a comparison operator such that x LIKE y is TRUE if x does match the pattern y. |
The character '%' matches any string of zero or more characters |
The SQL statement looks at rows whose last_name or first_name field has the input string cname as a substring in lower case |
Details of the phone-list example can be found from the handout or at http://pacman.npac.syr.edu:1996 |
Further application examples can be found at
|
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
Digital Video and Audio. This will enable collaboration and dissemination of fixed assets (audio/video clips) stored in multi-media databases or of information encoded in real-time.
|
Note these are typically streaming and not "batch" approachs. Current default Web downloads video to client before playing and this approach cannot scale! |
Can use traditional (relational) databases to store metadata and text with which to index video. |
Digital video works well over ISDN (128 kbits) but for full screen needs about .5 megabits per second even with wavelets |
This will be CNN/Network digital delivery technology for future infinite cable channel world |
Clearly digital audio and video supports the type of collaborative environments needed for good teaching
|
Digital video (in fashion similar to VRML 2.0) can be annotated to provide much more interactive results of scientific simulations
|
Digital Audio and Video is currently less than 5% of web data but eventually it will be dominant (95%) form of digital information and serving will be major Web activity. |
Broad distribution requires ISDN-ATM speeds but can deliver from local WebServers as only needs Ethernet or less for each client
|
Living SchoolBook Material for SC95 San Diego Dec 95 |
Video on Demand Project |
Links on the HTML page initiate seeking to a particular position in the movie. |
he Video Client displays ActiveMovie OLE control with a video window (right lower corner). |
ActiveMovie Control Properties window provides an extended interface to the Video Client (right upper corner). |
one of screendumps presenting the user interaction with Video on Demand Client for Windows 95/NT |
Video on Demand Project |
VSAM is the system to manipulate video archive metadata. |
Application Specific NII Specific Services for
|
Desktop Publishing and Productivity Tools in WebWindows (WebTop Productivity/Publishing)
|
InfoVision: Delivery on Demand of Information from:
|
Commerce -- digital Cash and signatures with appropriate authentication and security. Enables both Web Commerce (shopping) and use of Web for proprietary information |
Interactive and "batch" Collaboration
|
Metacomputing -- the collection of world wide computers as a coordinated (in subgroups of computers) computational engine (for simulation or information processing)
|
This popular word is meant to summarize fact that most problems these todays require not just a fast computer but rather a heterogeneous mix of resources including people, computers, information, visualization etc. |
Collaboration technologies link people to people and computers and this linkage is heart of PSE's -- Problem Solving Environments |
The Bridge Concept of Warner and Balch is heart of PSE for TeleMedicine |
We expect this type of system to grow in importance in all applications! |
Such as CareWeb |
CareWeb for Telemedicine -- local community network to support electronic student health record database and collaborative diagnosis by nurses, nurse practitioners and pediatricians. |
Command and Control -- innovative use of Web technologies for integrating a suite of large scale applications (weather, electromagnetic scattering, telemedicine, GIS) contributing to a military Command and Control. |
Distance Education and Science Collaboratory -- content (Virtual University, Living Schoolbook) and technology (WebFoil, WebSpace/LabSpace) development for delivering education over the Internet and providing collaboratory links between students and mentors. |
Large Scale Numerical Computing -- A set of pilot projects that explore Web based HPCC starting from simple computational topologies. Current prototypes include: RSA Factoring-by-Web, Adaptive Mesh Refinement for PDEs, 3D Visible Human. |
Most of the real world WebTop Systems will involve multi-user collaboratory modules. Even for scientific computing, complex toolkits such as WebAMR will be most conveniently supported by interactive consultation between developers and users. |
Collaboratory multi-user components will be further enhanced in enterprise, commerce and community systems. |
This is illustrated in our recent telemedicine prototype for nursing triage. Here we start from the collaboratory component involving nurses, nurse practioners and pediatricians and add HPCC components such as medical imaging and agent based diagnosis. |
We view the Bridge topology (Warner & Balch '95), underlying such telemedicine systems, as a promising generic framework, applicable also for other problem domains. |
A generic bridge includes "points of need", "points of expertise" and intelligent middleware that manages information resources and provides connectivity between customers and optimal services.
|
We present here examples of the bridge topology, instantiated in various application domains: |
Domain Points of Need Points of Expertise Typical Services |
TeleMedicine Nurses, Nurse Practioners Diagnosis
|
Command Troops Commanders Decision |
and Control Making |
Distance Learners Teachers Mentoring |
Education Students Consultants |
Commerce Consumers Vendors Product Support |
Science Schools Scientists Popular Science |
Collaboratory Small Businesses Technology Transfer |
This is distributed databases for supporting cost effective healthcare with less fraud, better use of existing information for establishing care-plans etc |
This is collaboration, remote instrument control for telemedicine |
Could be full televirtuality for virtual-reality controlled remote surgery in battlefield or accident scenarios |
This is basis of Virtual University and NII can be expected to be much more succesful than videosystems because interactive, collaborative, explorative and full multimedia information at student and teachers fingertips |
Base of Distance Education is InfoVision for full interactive curricula
|
Rich Collaboration environments including televirtual MOO's for student-student and student-teacher interactions |
InfoVision is "successor" to TV with any multimedia information delivered on demand( choice from infinite number of channels) |
Community Networks will define local societies and will have significant impact on local government |
Shopping on the Web will include multimedia digital yellow pages , Commerce, online catalogs (which is InfoVision again) as well as perhaps powerful VR interfaces where you can try out new clothes and cars |
Collaboration (gossip as in computer forums) could be an important and dominant use but perhaps not initial major one |
OLTP (Online Transaction Processing) such as approving and recording credit card transactions. This is a type of InfoVision and Commerce which is operational today
|
Decision Support varies from fraud detection, inventory planning and market segmentation for optimizing shopping by mail |
Decision support can vary from search of a single giant relational database to galaxy-wide search of heterogeneous distributed database |
Workflow Support as in LOTUS Notes integrates Collaboration, Project Management and Distributed Databases. This supports ongoing distributed enterprise -- this is also essential in collaboratory |
Critical feature is realtime, "come as you are" adaptive linked computers and people |
Judgement support extends decision support into an "exploratorium" scenario as crises tend to be unexpected |
Major need to search community and government databases for immediate information on resource location and availability etc. Metadata(summaries) essential |
Collaboration between commander in field , specialized anchor desks (domain experts) and those in field. |
User Interface essential as user will be tired and searching unfamiliar data |
Geographical Information Systems and browsing multiple video sources (InfoVision again) will be critical as judgements will depend critically on spatial data. |
This is future model for (inter)national distributed research |
Rich Collaboration environment essential as collaborative research requires nontrivial person to person interactions |
Searching distributed databases basis of much scholarly work (infoVision and Digital Library) |
Metacomputing for multidisciplinary, multi-institutional simulations and for control and data storage and analysis from remote instruments whether satellites, telescopes, accelerators etc. |
Distributed (people) Software Engineering support for production of distributed computing software |
Needs workflow support as distributed enterprise |
Critical feature is closely integrated collaboration of very many people and computers requiring |
Metacomputing Simulation and distributed database support as in Collaboratory |
Workflow support including configuration management and central CAD databases |
Rich collaboration environment to support distributed design decisions |
Standards and Security to allow interlinking of people and software from different organizations |
Key special requirement is distributed coordination |
WorldWide LifeLong Learning |
New Curricula |
This is basis of Virtual University and NII can be expected to be much more succesful than videosystems because interactive, collaborative, explorative and full multimedia information at student and teachers fingertips |
Base of Distance Education is InfoVision for full interactive curricula
|
Rich Collaboration environments including televirtual MOO's for student-student and student-teacher interactions |
We can use Web technologies to develop new very important tools for education enabling
|
We can teach these new technologies as world hungry for people understanding these technologies
|
Living Schoolbook -- Prototype of K-12 Educational Environment of year 2000
|
HPDC95 Tutorial and Other Computational Science Courses CPS600, CPS615
|
Physics 105/106 -- Science for the 21st Century (for non-Scientists) -- Some course modules built around Multimedia Information Systems
|
This hyperlinked material accounts for about 10% Phy 105, 20% Phy 106 and expect to increase if success continues
|
The Living Schoolbook is a New York State funded Initiative to create K-12 educational applications that exploit leading information technologies |
Living Schoolbook Base Educational Services are:
|
Living Schoolbook Leverages Information Technologies
|
The Project Team
|
Upstate Project Schools
|
Downstate (New York City) Project Schools
|
askERIC Educational Database for teachers with consultation |
The Discovery Channel -- Video Content Provider |
NewsBank -- Text Content Provider |
Reuters News Service |
Syracuse Language Systems -- Learn Foreign Languages on Demand |
TravelVenture -- Interactive travel information on Demand |
US Air Force Rome Laboratory -- InfoVision technologies |
WorldView Corp. : Interactive Client-Server Geographic Information System |
FLAG is an enterprise in which NYNEX Network Systems Company is a major (roughly 40%) investor. |
Current fiber link from U.K. to Japan (via Africa) is being constructed at an approximate $1.4B cost |
Natural Strategy is for Web Servers in U.S.A.. to deliver educational material via FLAG to Africa and Far East |
This would set up hierarchically as:
|
Real-Time Compression critical for interactive use of FLAG as lines are costly -- note Internet is free but needs compression as overloaded .... |
Many students from these countries come to the U.S.A.. now but ...
|
In Virtual University, one sets up a partnership between U.S.A.. and overseas University where key curricula and its delivery are provided over GII (Global Information Infrastructure of which FLAG is one example) |
High-end PC's should be included as possible Web server for Africa even we use SP-2, multi-processor SGI etc. as server in U.S.A.. University |
A NYNEX Joint Venture |
This shows fiber draping Africa with coast off ramps |
See electronically for actual home page |
Computational Science is an interdisciplinary field that integrates computer science and applied mathematics with a wide variety of application areas that use significant computation to solve their problems |
Includes the study of computational techniques
|
Includes the study of new algorithms, languages and models in computer science and applied mathematics required by the use of high performance computing and communications in any (?) important application
|
Includes computation of complex systems using physical analogies such as neural networks and genetic optimization. |
SImulation Track -- Graduate
|
Masters in Computational Science built around these core copurses CPS615,713 |
Simulation Track -- Undergraduate
|
CPS 401/601 IMM 601 Multimedia Systems and Tools -- Core course for Track 1 of IMM degree |
CPS 403/703 Computational Science Elective: 1 Lecture a week -- rest Laboratory -- elective for IMM |
CPS 406/606 Introduction to Information technologies and Systems -- designed as elementary elective for IMM |
CPS 616 Overview of Information Track Computational Science -- advanced elective for IMM |
CPS 714 Case Studies in Information Applications of Computational Science -- advanced elective for IMM |
"Web Technology"/Information Track of Computational Science Masters built around CPS 616,703,714 |
Certificate in Computational Science offered at Harbin Institute of Technology (China) Spring 1996 -- started March 1,1996 -- is a prototype of Syracuse University extension school |
Although Internet Connections to China are not guaranteed to be great(!) we can still use Web Technology but assume that Electronic Course material will be packaged at NPAC(SU) in a Pentium PC running WindowsNT and Java
|
This will be updated in batch mode by Syracuse -- China Connection |
Students in China will Interact with mentors in China and with NPAC via Internet (web and email) |
Project designed by Fox,Leskiw(NPAC), Xiaoming Li(Harbin and Peking visiting NPAC) |
Graduate Students -- initially 5 guinea pigs |
4 Courses -- each 10 1 hour lectures -- offered Spring 1996 over 20 week period |
ICPS 700: Introduction to Simulation Track of Computational Science
|
ICPS 701: Parallel Programming Laboratory
|
ICPS710: World Wide Web Technology
|
ICPS711: Web Programming
|
We suggest that Universities historically are "approved" to give "licenses" i.e. certification that students have a certain level of knowledge |
In the emerging Web based Virtual University, the traditional type and length of learening cycles is not obviously appropriate |
Further we need to bootstrap WebWisdom and "suck customers in" a little bit at a time! |
Thus suggest Scalable Certificates
|
Note these certificates are also portable as can take anywhere at any time! |
Traditionally, University courses are 1/3 interaction and 2/3 study/homework
|
I expect Java to be the the dominant implementation language for everything on the Web from Servers, Clients, MOO's, filters, Animations, Simulations etc. |
Can build neat clickable images including Visible Human (biology education!) and our 2D "Interactive Journey" clickable GIS for New York State |
This has several obvious near term applications including some examples of education in HPCC arena:
|
The above examples are quite interesting reasons for introduction of electronic journals as you should be able to explain complex ideas better this way |
Note Web was initially book/libraries done digitally. |
Now we are the second wave of opportunities exploiting the specific advantages of distributed computing backbone |
VRML allows user annotated 3D visualizations so that you can explain features of results of simulation in a universal fashion which can be viewed on any machine
|
VRML is Very good GIS (Geographical Information System) technology and can be used by NASA/EPA etc for presentation of spatial data with again clickable annotations |
VRML is currently and perhaps intrinsically (numbers ascii) inefficient and Java is alternative / augmentation technology
|
VRML 2.0 will allow projects like Argonne Labspace to produce collaborative televirtual environments |
You can use structured databases such as Oracle to store resource data such as Glossaries in an organized fashion with (well-known technology) to generate HTML on the fly
|
Can also store unstructured information as illustrated by Usenet computer and other relevant newsgroups |
Use the powerful Search engines with domain (i.e. particular courses) specific keywords . These can be attached to commercial (e.,g. altavista) or inhouse Web Search sites so search confined to subset of curricula material |
Object databases can be very useful for VRML and other data which is naturally of this form such as images and output of simulations |
Clearly digital audio and video supports the type of collaborative environments needed for good teaching
|
Digital video (in fashion similar to VRML 2.0) can be annotated to provide much more interactive results of scientific simulations
|
Digital Audio and Video is currently less than 5% of web data but eventually it will be dominant (95%) form of digital information and serving will be major Web activity. |
Broad distribution requires ISDN-ATM speeds but can deliver from local WebServers as only needs Ethernet or less for each client
|
There are some cases where JavaScript is a rapid prototyping alternative to Java (Java can in principle do anything!) |
Current use (demonstrated in WebWisdom) is for customized display's of information where base information, simulations, audio(video), and indices can be linked in a friendly fashion. |
Syracuse is extending to a set of WebTools (originally developed as Server CGI scripts) with which you can build customized front-ends with user defined configurations choosing between type of access (administrator and naive user would be different) and display capabilities (resolution, color) of terminal |
Note client side and so fast even though interpreted |
However does not yet have (but needs) text processing capabilities of Perl |
Tightly Coordinated Distributed People and Computers |
This is future model for (inter)national distributed research |
Rich Collaboration environment essential as collaborative research requires nontrivial person to person interactions |
Searching distributed databases basis of much scholarly work (infoVision and Digital Library) |
Metacomputing for multidisciplinary, multi-institutional simulations and for control and data storage and analysis from remote instruments whether satellites, telescopes, accelerators etc. |
Distributed (people) Software Engineering support for production of distributed computing software |
Needs workflow support as distributed enterprise |
Critical feature is closely integrated collaboration of very many people and computers requiring |
Metacomputing Simulation and distributed database support as in Collaboratory |
Workflow support including configuration management and central CAD databases |
Rich collaboration environment to support distributed design decisions |
Standards and Security to allow interlinking of people and software from different organizations |
Key special requirement is distributed coordination |
A set of manufacturing companies -- Rockwell International, Northrop Grumman, McDonnell Douglas, General Electric and General Motors is studying the NII implications for a particular MAD system "Affordable Systems Optimization Process" (ASOP) |
Interesting parameters are that next major aircraft to be built could involve:
|
Customizes general NII technologies to ASOP specific requirements |
Provides the object, wrapper and agent services to share data in the ASOP design environment |
Conforms to applicable protocol, ontological and data standards |
Defines management, retrieval, and control functions for heterogeneous data and methods distribution |
Interfaces to Team Infrastructures |
Employs emerging "open" NII services to support aeronautics teams and collaboration
|
Expanded for large concurrent user base
|
Access additional NII bandwidth as needed |
Focused on configuration control for aeronautics products and associated data
|
Provides services for managing common product model across distributed team
|
Supports rollback of configurations and long-term archive of product and data configurations |
Facilitates distributed "computing" services available across an ASOP project team |
Supports access to shared "supercomputer" resources on as needed basis for aeronautics ASOP projects |
Interfaces to "open" NII services for ASOP users
|
Provides range of data/system security for ASOP
|
Support access control to ASOP capabilities
|
Insure only authorized users access, modify, use ASOP services and information |
Specializes generic "object/wrapper/agent" technologies to ASOP requirements |
Provides object management; distribution; control and retrieval services |
Stores and archives ASOP related data objects for re- use |
Implements specific data exchange standards for data objects; i.e. STEP Application Protocols |
Ranges from desktop video conferencing to 3-D graphics to full televirtual interactions |
Includes CAD conferencing / whiteboard capabilities |
Enables collaboratories - geographically distributed laboratories to support product development |
Requires interactive and "batch" data exchange services |
Supports collaboration among dispersed development community |
Open NII services driven by ASOP requirements for rigorous configuration control / management |
Broadly applicable to wide range of product development efforts beyond aeronautics |
Collection of world wide computers as coordinated computational engine |
Supports distribution of high compute problems across widely distributed systems |
Enables world wide searches for information to resolve user queries |
Includes control of remote scientific instruments |
Multiple levels and types of security requirements supplied to meet military and commercial needs |
Extended to support large dataset security as opposed to small transaction protection |
Includes access (login) permissions; user roles; to reach available services |
Address administration of services to user community - billing/authorization/pricing/approvals |
For instance as part of my New York State Industrial HPCC outreach program InfoMall, I found that a major manufacturer of large consumer machines had no interest in MPP's for simulating them to improve design |
However very interested in
|
Here Perl manipulates text from database to HTML |
Java enables down-loaded simulations |
VRML enables universal definition of 3D objects -- products and buildings |
The Web Originated as an Information system but we can clearly use it as a base for distributed computing and as parallel computing is just a special case of this with low latency and tight synchronization, for parallel Computing as Well! |
WebWork was our first Concept (with Boston University and Cooperating Systems) which concentrated on software engineering gotten by using information capabilities of Web linked to computing environments |
DataFlow (for coarse grain software integration) and Embarassingly Parallel applications are natural first Web thrusts as not so sensitive to performance issues |
Overview --- http://www.npac.syr.edu/projects/webbasedhpcc |
WebTools --- http://king.syr.edu:2006/WebTools.html |
RSA Factoring-by-Web -- http://www.npac.syr.edu/factoring |
Distance Education / Virtual University -- -- http://www.npac.syr.edu/users/gcf/foilsbyarea.html |
WebSpace/Labspace -- http://www.npac.syr.edu/projects/webspace |
Web based Telemedicine -- http://www.npac.syr.edu/projects/careweb |
Embarassingly Parallel: Succesful RSA130 Factorization |
Linkage of Databases with MPP's: Financial Modelling on Demand |
Java GUI for Distributed Computing and Performance Visualization |
Java as WebFlow -- AVS done with the Web and applied to Image Processing |
HPF on the Web -- General Principles and use in Education as a web programming laboratory |
Putting it together -- Java frontend to a domain specific problem solving environment -- WebAMR -- Adaptive Mesh Refinement
|
This implies that we look at both Grand Challenges and National Challenges but we suggest this is not enough: |
WebWork Builds HPCC technologies on a broad not niche base starting at bottom (Web,PC's) |
not top (MPP's, Supercomputers) of computing pyramid |
WebWork is an open, world-wide distributed computing environment based on computationally extended Web Technologies |
The backend computation and information infrastructure is provided by the World-Wide Virtual Machine -- a mesh of computationally extended Web Servers (called Compute Servers) |
These servers manage (via CGI mechanisms) a collection of standardized computational units called WebWork Modules. |
Geographically distributed and Web-published WebWork modules interact by HTTP/MIME based message/object passing and form distributed computing surfaces called Compute-Webs |
The front-end user/client interfaces are provided by evolving Web browsers with increasing support for two-way interactivity (e.g. Java, VRML) that facilitates client side control and authoring. |
A natural user-level metaphor -- WebFlow -- is supported in terms of visual interactive compute-web authoring tools. |
Collaboration with Arjen Lenstra and Boston/CSC. New NFS factoring algorithm successfully applied to RSA130 factoring on a tree of Web+CGI servers (FAFNER by Jim Cowie/CSC). |
SC'95 Teraflop Challenge Award. |
Next Challenge -- RSA155. |
RSA security systems based on numbers
|
Bank of England and English Savings and Loan based on m=155 (512 binary digits) |
RSA129 cracked by factoring with email team using sophisticated version of Quadratic Sieve. RSA155 will use better Number Field Sieve |
Need x2 = y2 mod(RSAm) as then gcd(x+y,RSAm) likely to be interesting factor |
Find x and y by finding lots of interesting a's
|
Given these a's factored into primes, multiply together so powers of primes are even. This gves desired x |
This last step requires graph theory and solution (for Bank of England) of 5 million linear equations |
RSA155 requires about 300 teraops hours to solve with NFS |
RSA129 needed about an order of magnitude less time. Can be done today faster if use Number Field Sieve |
We have roughly one to five million independent calculations which form the rows of matrix (after clever graph theory manipulates and combines) |
Set of master servers publish problem to solved with suitable demos, description of algorithm and full marketing attention.
|
Clients return results -- not so easy except by email and cut and paste |
Best done as a set of cooperating servers where server performing factorization publishs it solution as a file on the WWW. |
Cooperating servers also better for computer administrators as can control set of clients at a given site |
Initially use humans but replace by agents when software ready |
Proposed Architecture of WWVM |
Illustrates implementation of WebWork message passing in terms of
|
This diagram illustrates point-to-point communication between Web servers, used to implement a webflow channel between compute-web modules. Two extreme implementation modes are described: a) based on today's Web server technology, and b) based on thread memory mapped high performance implementation, expected in future Web compute-servers. Subsequent steps, represented by a sequence of labelled lines in the figure, are described below in both implementation modes. |
a) Today's Web server mode: (1) -- M1 locks O1 on S1 disk. (2) -- M1 sends POST HTTP message to S2 with M2 URL in the header sector and with O1 URL in the body sector. (3) -- S2 activates M2 via CGI and passes O1 URL as a command-line argument. (4) -- M2 sends GET method to S1 with O1 URL in the header. (5) -- S1 fetches O1 from its document tree. (6) -- S1 sends the content of O1 to M2 which completes the GET exchange. (7) -- M2 saves O1 by overwriting current I2 on the S2 disk. If I2 is locked, M2 waits (blocks). (8) -- After O1 is saved on the S2 disk, M2 returns 'end-of-transfer' acknowledgment to M1 which completes the POST exchange. (9) -- M1 unlocks O1 and exists. |
b) Compute-server (future Web server) mode: (1) - M1 locks its memory object O1. (2) - M1 checks if socket connection to M2 is in M1 connection table. If yes, go to (5) below. Otherwise, M1 connects to S2 and sends M2 creation script. (3) - S2 spawns M2 and acknowledges. (4) - M1 receives acknowledge message and saves new socket in connection table. (5) - M1 gets O1 handle. (6) - M1 writes O1 to M2 using socket lib calls. (7) - M2 reads O1 using socket lib calls. If I2 is free, O1 buffer is copied directly to I2 buffer. If I2 is locked, M2 creates O1 clone and blocks. (8) - M2 sends acknowledge to M1. (9) - M1 unlocks O1 and blocks. |
http://cooperate.com/cgi-bin/FAFNER/factor.pl |
Features
|
months of runtime, dozens of collaborators, |
eight nations, four continents |
hardware platforms from an i386 laptop to an IBM SP/2 (including HPs, Alphas, MIPS, Suns, SGI machines, RS6000s) |
Most Heterogeneous and Geographically Dispersed Award, 3rd Annual HPC Challenge, Supercomputing '95. |
Implemented as Perl scripts, invoked via CGI |
Hierarchy of cooperating World-Wide Web servers used for many functions in the collaboration:
|
General Number Field Sieve (GNFS)
|
GNFSD Wrapper Code
|
http://www.npac.syr.edu/factoring/status.html |
Web Sieving started in September 1995. |
On April 10, 1996, we found that |
RSA-130 = 1807082088687404805951656164405905566278102516769401349170127021450056662540244048387341127590812303371781887966563182013214880557 has the following factorization: RSA-130 = 39685999459597454290161126162883786067576449112810064832555157243 * 45534498646735972188403686897274408864356301263205069600999044599 |
Factoring on the Web Project |
Can grab stock data from real-time services |
Combine with historical data stored in databases (such as Oracle) |
Use in Monte Carlo simulations of sophisticated financial instruments |
Cooperative distributed (and parallel) computing will become mainstream in financial engineering due to a convergence of the following factors: |
Increased volatility due to globalization of financial markets |
Global distribution of data sources |
Increase in complexity of derivatives and risk management vehicles |
Increased demand for real-time asset allocation decision support |
Increased volume of raw data and need to process large databases |
Increased volume on the retail side of the spectrum in part due to on-line technologies (Internet and WWW) |
HPCC is becoming indispensable in the application domains such as:
|
NPAC is engaged in development of new tools for quantitative financial modeling which take advantage of scalable computer architectures |
The ultimate goal is to integrate various quantitative analysis transparently using Web technologies into a seamless cooperative computing environment, capable of supporting all aspects of enterprise-wide risk management. |
We developed new algorithms for risk neutral valuation of derivative financial instruments |
Theoretical prices of derivative instruments are obtained by discounting their expected payoffs under the equivalent martingale measure using money market interest rate. |
The core algorithm is Path Integral Monte Carlo which used to generate arbitrary distributions of underlying risk factors (stocks, bonds, short interest rates, commodities, indices etc.) |
The advantage of the new algorithm is that sensitivities of derivative prices with respect to changes in all model parameters are computed in a single simulation.
|
Parallel version of the algorithm is written in C and MPI and relies on task parallelism and functional decomposition (could also use HPF) |
Monte Carlo samples are generated on multiple processors in embarrassingly parallel fashion |
Pricing modules can either run in lock-step with the Monte Carlo module which generates histories of risk factors or asynchronously perform valuation functions on the histories which are broadcast as they are generated by the Monte Carlo module |
We are linking this flexible algorithm with a novel scheme based on Maximum Entropy method which generates implied probability distributions from reported option prices. |
The implied distributions can be used within the Path Integral Monte Carlo module to price exotic contracts consistently with exchange-traded contracts and they can also be used to search for arbitrage opportunities |
Estimation of implied distributions requires large scale global optimizers. |
We are developing two parallel stochastic optimizers based on mean field approximation (Laplace formula) and Langevin equation |
Derivative valuation functions are integrated using Web technologies into a service which can be accessed from any platform which supports a graphical browser |
Using a combination of HTML forms or Java front-end, CGI mechanism, Perl scripts and modules written in C and MPI, which are executed on multiple NPAC RS 6000 and Sun workstations and the SP-2, the user can:
|
In the next stage, flat files will be replaced with a parallel Oracle server |
Ultimately, the graphical user interface will be supplemented with an agent-based middleware layer, implemented in Java, where derivative pricing and risk management services will be requested and dispatched to the parallel Monte Carlo engine and returned to the client using an EDI-like protocol encapsulated within the KQML envelope. |
This will be a prototype of the new service economy that will flourish on the Web. |
We can exploit the convenience of Java to build sophisticated user interfaces |
Further if computing (such as HPF programs) is linked in real-time to the web, we can get a rich window into execution using Web to process and display information produced by the programs
|
New powerful Web'96 technologies from Netscape, JavaSoft, Oracle, NeXT etc. will result in a new generation of interactive services |
A natural next step is to start Chaining (Integrating) such services to a distributed PSE by providing a server to server communication and dataflow support |
However Web'96 becomes also increasingly complex with its competing and overlapping multi-lingual standards
|
Visual Programming for a multi-server Web (We call it WebVM) based dataflow (we call it WebFlow) is a natural next generation user-friendly programming environment |
We view the area of distributed Web based computing for PSE as a promising niche for NPAC and academic R and D where we expect industry to continue their focus on client-server aspects of the Web where near term profits can be made |
See Original Foil |
See Original Foil |
See Original Foil |
See Original Foil |
1)Classic solution of large scale PDE or Particle dynamics problem
|
2)Modest Grain size Functional Parallelism as seen in overlap of communication and computation in a node process of a parallel implementation.
|
3)Object parallelism seen in Distributed Simulation where "world" modelled (typically by event driven simulation) as set of interacting macroscopic (larger than grid points) objects
|
4)MetaProblems consisting of several large grain functionally distinct components such as
|
Java: 1) Not Supported, 2) is Thread mechanism, 3) is Java Objects or Applets, 4) is JavaBeans or equivalent |
Fortran: 1)is supported in HPF, 2--4) are not supported |
As we saw large scale Applications need many forms of parallelism and it is not needed/appropriate to use the same mechanism for each form
|
Thus Java needs (runtime and perhaps language) extension to support HPF/HPC++ like (shared memory model for programmer) data parallelism but "Java plus message passing" is already here
|
NPAC HPJava Activity -- Michael Chang and Bryan Carpenter |
NPAC HPJava Activity -- Michael Chang and Bryan Carpenter |
NPAC HPJava Activity -- Michael Chang and Bryan Carpenter |
As Java lies "inbetween" Fortran and C++, one can expect that data parallel Java can learn from corresponding HPF and HPC++ studies |
"Parallel Compiler Runtime Consortium" produced a very rough draft |
Java does not support templates and STL approach of C++ not so natural |
Need to recognize that performance of Objects in Java poorer than that of "simple types" |
Java spans high level interpreted objects to low level optimally compiled "simple types" |
We have proposed an approach which uses native classes for "compiler runtime" and follows an HPF style with an interpreted front-end like Matlab or APL or "host" programming model as in *LISP on CM-2 |
e.g. A = HParray.matmul(B,C)
|
This leads again to Java wrappers invoked by HPF-style Java(Script) interpreter which interfaces to native HPF or other implementations.
|
Establish bottom-up constituency by teaching Java in Middle and High Schools |
Start working groups/meetings to study requirements and issues
|
Build Prototype Web Coarse Grain Computing Environments
|
Design and build "Java Wrappers" to both sequential and parallel Fortran77/90
|
Link above technologies in the WebWindows Programming Laboratory
|
Build Expertise/Infrastructure on High Performance Optimizing Java Compilers aiming at data structures of importance to Science! |
Collaborate with Web/Compiler groups in China .... |
From NIST Sp2 Web Interface by Robert Lipman http://www.nist.gov/itl/div887/sasg/websubmit/ |
From NIST Sp2 Web Interface by Robert Lipman htt/www.nist.gov/itl/div887/sasg/websubmit/ |
From NIST Sp2 Web Interface by Robert Lipman htt/www.nist.gov/itl/div887/sasg/websubmit/ |
Web Interface was developed by NPAC through NSF REU Program |
Web HPF Interface was based on thesis work of Kivanc Dincer at NPAC |
An example of Web-based Computing |
It lets researchers author tools and leave them on the machine of choice on the web |
It allows multiple data bases to intercommunicate with each other and the functional operators that the software tools represent and to make a web browser the window into this system. |
From Gregor von Laszewski |
From Gregor von Laszewski |
From Gregor von Laszewski |
From Gregor von Laszewski |
From Gregor von Laszewski |
http://www.npac.syr.edu/users/gcf/npacaddons -- see foil 105 |
Java provides a convenient way to build flexible graphics interfaces |
The screendump shows the message passing traffic and status of (36) parallel nodes used in a sorting algorithm |
In the example, the 36 threads are running the explicit algorithm on the client |
Alternatively and more generally, the threads are replaying a trace of the program which is or was running on a separate set of nodes |
If one uses Web Servers to control master parallel computation or more generally integrates Web into computing, these Web servers can naturally feed event traces into Java based display
|
From Kemal Ispirli with 36 threads running client-side |
Colors represent node status and links message-passing |
Web front-end to HPF compiler and PVM-based distributed runtime. |
Supports CASE tools for program development, process management and performance monitoring. |
We have illustrated Java Pablo and Distributed Computing Interfaces already |
This will be initially deployed as a Programming Lab Interface for Web Courses |
PCRC embodies the Parallel Computing Synchronization and collective parallel algorithms and runtime that will enable efficient Web-based computing |
Replace user interface of HPF or HPC++ with the Web(work) and use pervasive Web Technologies in infrastructure (World Wide Virtual Machine -- WWVM) |
The compilation system is accessed through a Web Browser. The HPF program and requests are POSTed to the HTTP Server using HTTP Protocol. |
HTTP Server analyzes the incoming request and activates the HPF Service Master which is a CGI module. |
HPF Service Master starts the Compile module which translates a given HPF program into Fortran 77 with message passing calls (i.e.MPI), and produces an object file using a node compiler. |
Link module links the generated object code with Common Runtime Support (CRS) and MPI libraries. |
Run module forks copies of the executable code on the nodes of a workstation cluster. |
After the request has been satisfied, HPF Service Master puts the results into HTML format and sends back to the HTTP Server. |
HTTP Server sends the results to the Web Browser via the HTTP Protocol. |
The WWVM is accessed through a Web Browser and requests are sent to the server through HTTP Protocol. |
HTTP Server starts a service master which translates given HPF program to F90 with message passing calls and starts-up other client servers. |
HTTP Servers at the client side get the requests through the HTTP protocol connection, and activates a Interpreter CGI module. |
Interpreter makes calls to the Runtime Support and Communication Server that sends and receives messages using PVM daemons. |
From Kivanc Dincer |
From Kivanc Dincer |
We illustrate here how the individual component technologies cooperate in a complete application, WebAMR (Adaptive Mesh Refinement) |
A mesh of computationally extended Web servers, connected via HTTP based message passing, acts as WebVM that runs PDE solver modules for individual grids |
In a simple static AMR topology (WebWork model), a tree of refined meshes is constructed by the user via the AVS like visual programming tools (WebFlow) |
Dynamic AMR trees require interpreted programming support -- a pilot "little language" design towards WebHPL |
WebAMR applications can be configured and run on heterogeneous clusters, including any WebWindows compliant platform |
Example of WebTop System in this domain in a set of WebVM/WebFlow modules, packaged and customized as a PDE Toolkit for a given Grand Challenge community. |
We need to use compilers on tightly coupled systems such as MPP's (shared and distributed memory) |
But for metacomputing, the hardware intrinsically has latencies that suggests increased flexibility of interpreters is more appropriate
|
Maybe frontends should be built with interpreters such as object-oriented PERL5 so easier to link with Web. |
Note that interpreted environment will have best software engineering support and so suggestion is -- taking SP2 as example:
|
This implies that we should allow hybrid model not just for task (interpreted) versus data parallelism(compiled)
|
Current Web Interpreters include Java TCL and PERL(5) which are optimized for different application domains
|
This leads to WebScript Concept of interoperable interpreters optimized for different domains
|
Java is a C++ subset which interestingly does not have pointers as these are unsafe in necessary secure metacomputing environment. |
Thus Java has removed the part of C++ which is hardest to parallelize |
Java may not "survive" but if it doesnt something better will! Thus it makes sense to study and experiment with it |
Natural first step is to use Java to build the interpreted "shell" which we called HPFCL for HPF coordination Language.
|
Java is partially compiled as you take basic Java high-level code and compile down to a universal Java machine language. This is very similar to concepts in ANDF (Architecture Neutral Distribution Format) but with a different goal
|
VRML -- Virtual Reality Model Language -- is an object oriented database built as a subset of the SGI Inventor System |
VRML can be considered as another script optimized for graphics but not many interesting processing (compute) capabilities are in current standard |
VRML can be considered as an example of a universal data structure allowing exchange of 3D objects over the Web.
|
Thus useful to consider data parallel VRML and building CC++ or HPF(Fortran90) modules to support VRML |
HPCC community should join with the Web to ensure that standards such as VRML can be implemented efficiently either in parallel (maybe a niche) but also in a distributed network (similar issues where HPCC can contribute and clearly very important) |
Features of the Java language and runtime
|
Can build on existing work on HPF HPC++ -- especially latter |
Two basic types of opportunities:
|
Applet mechanism naturally incorporates task parallelism -- need to add "channel" class (as in Fortran-M, CC++) to augment "thread" and "socket" classes
|
Java can be used both on Server and Client side as expect soon good Java based clients and servers |
See the E language http://www.communities.com/e.html |
or my class notes http://www.npac.syr.edu/users/gcf/cps616java96 or |
http://www.npac.syr.edu/users/gcf/cps616tech96 |