Abstract * Foil Index for this file
See also color IMAGE
Given by Geoffrey C. Fox at Icase/HPDC95/HPCS95 on 15 September 95. Foils prepared 15 September 1995
Abstract * Foil Index for this file
See also color IMAGE
| This describes the forces motivating use of Web in MetaComputing |
| First the need for a pervasive technology base for HPCC which is otherwise a nonviable niche market. |
| The growing number of Web-enabled machines and the development of WebWindows giving the productivity tools needed for a true distributed HPCC software engineering environment |
| We describe the 3 layer model of WebWork |
| At the high end we describe our early example -- WebFlow and how this can be extended to a full WebHPL with mixed interpreted and compiled systems. |
| At the base level, we have the World-Wide Virtual Machine as a mesh of computationally extended web servers |
| At the Intermediate level, we propose WebScript to integrate VRML, PERL5 Java and similar domain specific scripting systems |
This table of Contents
Abstract
Full HTML Index
| Geoffrey Fox |
| NPAC |
| Syracuse University |
| 111 College Place |
| Syracuse NY 13244-4100 |
Full HTML Index
| This describes the forces motivating use of Web in MetaComputing |
| First the need for a pervasive technology base for HPCC which is otherwise a nonviable niche market. |
| The growing number of Web-enabled machines and the development of WebWindows giving the productivity tools needed for a true distributed HPCC software engineering environment |
| We describe the 3 layer model of WebWork |
| At the high end we describe our early example -- WebFlow and how this can be extended to a full WebHPL with mixed interpreted and compiled systems. |
| At the base level, we have the World-Wide Virtual Machine as a mesh of computationally extended web servers |
| At the Intermediate level, we propose WebScript to integrate VRML, PERL5 Java and similar domain specific scripting systems |
Secs 87
Full HTML Index
| Parallel Computing Works! |
Technology well understood for Science and Engineering
|
Supercomputing market small (few percent at best) and probably decreasing in size
|
Secs 141
Full HTML Index
No silver programming bullet -- I doubt if new language will revolutionize parallel programmimng and make much easier
|
Social forces are tending to hinder adoption of parallel computing as most applications are areas where large scale computing already common
|
Secs 86
Full HTML Index
| ATM ISDN Wireless Satellite advancing rapidly in commercial arena which is adopting research rapidly |
Social forces (deregulation in the U.S.A.) are tending to accelerate adoption of digital communication technologies
|
Not clear how to make money on Web(Internet) but growing interest/acceptance by general public
|
Integration of Communities and Opportunities
|
Secs 63
Full HTML Index
Technology Opportunities in Integration of High Performance Computing and Communication Systems
|
| New Business opportunities linking Enterprise Information Systems to Community networks to current cable/network TV journalism |
| New educational needs at interface of computer science and communications/information applications |
| Major implications for education -- the Virtual University |
Full HTML Index
| 1) need for better debuggers, profilers, performance monitoring tools |
| 2) need for more stable operating systems |
| 3) need for tools to aid in code migration to parallel systems, whether it be in the form of libraries, or other software engineering tools. |
| 4) need to reduce the latencies due to system software |
| 5) need for looking at exciting and innovative applications areas, (to help the HPCC industry by stimulating new demands). This might involve very data intensive applications (in contradistinction to compute intensive ones) but also harder and more complex problems, irregular data structures and less obviously load balanceable problems. |
Full HTML Index
| Need to involve a larger group of non HPCC communities |
For instance, most of the messages on networks are
|
| But MPI standrards set internally to HPCC and did not explicitly involve ATM/Internet community/standard processes |
HPF focusses on regular multidimensional arrays in an excellent standards forum that ignores
|
| Need HPVRML and a broader community |
Full HTML Index
| NASTRAN |
| Real-time embedded systems |
| Aerospace manufacturing |
| Crisis management |
| Nuclear Weapons |
| Environmental Modeling |
| Mission to Planet Earth |
| Data Intensive Applications |
| High end CFD |
| Centric |
| Computational Chemistry |
| QCD |
Full HTML Index
| This implies that we look at both Grand Challenges and National Challenges but we suggest this is not enough: |
| WebWork Builds HPCC technologies on a broad not niche base starting at bottom (Web,PC's) |
| not top (MPP's, Supercomputers) of computing pyramid |
Full HTML Index
| WebWork is an open, world-wide distributed computing environment based on computationally extended Web Technologies |
| The backend computation and information infrastructure is provided by the World-Wide Virtual Machine -- a mesh of computationally extended Web Servers (called Compute Servers) |
| These servers manage (via CGI mechanisms) a collection of standardized computational units called WebWork Modules. |
Full HTML Index
| Geographically distributed and Web-published WebWork modules interact by HTTP/MIME based message/object passing and form distributed computing surfaces called Compute-Webs |
| The front-end user/client interfaces are provided by evolving Web browsers with increasing support for two-way interactivity (e.g. Java, VRML) that facilitates client side control and authoring. |
| A natural user-level metaphor -- WebFlow -- is supported in terms of visual interactive compute-web authoring tools. |
Full HTML Index
| Implements the "Viable Base" Enterprise Model of HPCC Software identified in Pasadena2 workshop |
| This will allow good programming tools to be developed and mnaintained as larger enough base to support software industry |
| Implements a powerful software engineering framework for parallel computing by integrating parallel programming with the World Wide Web Productivity Tools |
Full HTML Index
| They have communication difficulties hampering best use of their team of over 200 software engineers in Madras India |
| As with other large companies, problem reports are rotated around world on 24 hour basis. If East Coast hasn't solved by COB, customer problem sent to open office the the West so that by 8 am EST, solution is waiting for customer with perhaps many offices having worked on it as it moved (virtually) around the globe. |
| We have proposed to them that local database solution is Oracle but World Wide Web linkage be in terms of WebWindows (WebSpace/WebWork). |
Full HTML Index
RSA security systems based on numbers
|
| Bank of England and English Savings and Loan based on m=155 (512 binary digits) |
| RSA129 cracked by factoring with email team using sophisticated version of Quadratic Sieve. RSA155 will use better Number Field Sieve |
| Need x2 = y2 mod(RSAm) as then gcd(x+y,RSAm) likely to be interesting factor |
Find x and y by finding lots of interesting a's
|
| Given these a's factored into primes, multiply together so powers of primes are even. This gves desired x |
| This last step requires graph theory and solution (for Bank of England) of 5 million linear equations |
Full HTML Index
| RSA155 requires about 300 teraops hours to solve with NFS |
| RSA129 needed about an order of magnitude less time. Can be done today faster if use Number Field Sieve |
| We have roughly one to five million independent calculations which form the rows of matrix (after clever graph theory manipulates and combines) |
Set of master servers publish problem to solved with suitable demos, description of algorithm and full marketing attention.
|
| Clients return results -- not so easy except by email and cut and paste |
| Best done as a set of cooperating servers where server performing factorization publishs it solution as a file on the WWW. |
| Cooperating servers also better for computer administrators as can control set of clients at a given site |
| Initially use humans but replace by agents when software ready |
Full HTML Index
| 1:Server-to-Server Communication Diagram |
| 2:WebWork System OverView |
| 3:WebTools CASE tools sample page |
| 4:Java documenation sample page |
| 5:Java class database manager |
| 6:Java screendump -- sorting algorithms |
| 7:Java screendump -- WebFlow Editor prototype |
| 8:Java screendump -- WebFlow application prototype --- Project Manager |
| 9:VRML screendump |
| 10:VRML source code example |
| 11:Java source code example |
Full HTML Index
| WebWork is based on a three-layer architecture shown in figure 2, including: World_Wide Virtual Machine (WWVM) in the (bottom) layer 1, Middleware layer 2 of agents, wrappers, mediators etc., and high level programming environments (e.g. HPFCL) and user interfaces (e.g. WebFlow) in the (top) layer 3. |
| All base WebWork concepts can be implemented in terms of today's Web technologies (HTTP, MIME, CGI) and a prototype is under development at NPAC. |
| The overall design is open and ready to upgrade the existent (e.g. browsers or servers) and include new (e.g. agents or distributed object brokers) Internet/Web technologies |
| One starting point for the WebWork construction is provided by NPAC WebTools -- a CGI-extended Web server with enhanced content authoring and database navigation functionalities. WebTools Server is used as a prototype WebWork node server. |
Full HTML Index
| Illustrates 3 base layers of WebWork architecture and all main system components. |
A 4--node compute-web is represented
|
| Java/HotJava model is used for WebFlow front-end implementation |
|
The paper describing this project is available at SCCS715 in NPAC technical report series |
Full HTML Index
| NPAC WebTools is a CGI-extended Web server that offers a HyperWorld based metaphor for organized content authoring and navigation, currently implemented in terms of the following tools: HyperWorld Manager, HyperWorld Navigator, On-Line HTML Editor, WebMail and CASE tools for HySource Worlds authoring. |
HyperWorld Manager offers database management support for the server document tree, integrated with browser GUI tools for remote file/document and directory/folder handling (create, destroy, copy etc.). The model assures concurrency control, atomicity and integrity of the document datatbase.
|
Full HTML Index
HyperWorld Navigator offers a consistent navigation metaphor.
|
| On-Line HTML Editor offers remote authoring support for documents, created by the HyperWorld Manager. |
WebMail offers the Web interface to the MH mailing system and initial support for collaborative forums.
|
CASE tools offer disciplined WebTools software development environment, integrated with the HyperWorld database.
|
Full HTML Index
| NPAC WebTools can be viewed as an instance of Web Productivity Tools (navigators, editors, databases), developed collectively by the Internet/Web community. |
| We view these emergent open tools as central to develop and maintain Web based World-Wide Metacomputing. |
| Software exchange and integration tools are urgently needed. Without it, 'pervasive Web' will become soon too complex to maintain and will be dominated by closed corporate products. |
| One such attempt is made by the HySource CASE package in NPAC WebTools. So far, we developed HyPerl World (Screen 3) of the WebTools source code and we now integrate it with Java (Screen 4) in the form of HyJava World (Screen 5) |
| These tools will evolve towardsVirtual Software Laboratory -- a collective distributed CASE framework for virtual corporation of WebWork developers. |
Full HTML Index
| HyPerl World page, generated automatically by the WebTools CASE package, and integrating documentation with the source. |
| More generally, we call by HySource the hypertext documentation with navigable source code included. |
| Function calls and external variable references are 'blue' and point to the corresponding HySource pages. |
Full HTML Index
Java documentation shares some common aspects with HySource,
|
| Java and VRML CASE support will be included as next steps in WebTools CASE package. |
Full HTML Index
| WebWork pilot project is a collaboration between NPAC, Boston University and Cooperative Systems Corporation, MA. It will prototype a candidate VSL, WWVM, Java based user interfaces, and port selected Grand/National Challenge applications to this platform. |
| The project will use NPAC WebTools to bootstrap the software process and will prototype WWVM in terms of current Web technologies (Screen 1) |
| Technically, early WWVM will include existent Web Servers with add-on CGI (Perl) scripts that build server-to-server communication and offer document database management, and module publication and linkage/instantiation support. |
| This base model will be further extended and refined by using and driving evolving Web technologies. For example, the disk-based model in Screen1a will likely evolve towards memory-mapped model based on multi-threaded interpreted compute-servers (Screen 1b) |
Full HTML Index
Illustrates implementation of WebWork message passing in terms of
|
| This diagram illustrates point-to-point communication between Web servers, used to implement a webflow channel between compute-web modules. Two extreme implementation modes are described: a) based on today's Web server technology, and b) based on thread memory mapped high performance implementation, expected in future Web compute-servers. Subsequent steps, represented by a sequence of labelled lines in the figure, are described below in both implementation modes. |
| a) Today's Web server mode: (1) -- M1 locks O1 on S1 disk. (2) -- M1 sends POST HTTP message to S2 with M2 URL in the header sector and with O1 URL in the body sector. (3) -- S2 activates M2 via CGI and passes O1 URL as a command-line argument. (4) -- M2 sends GET method to S1 with O1 URL in the header. (5) -- S1 fetches O1 from its document tree. (6) -- S1 sends the content of O1 to M2 which completes the GET exchange. (7) -- M2 saves O1 by overwriting current I2 on the S2 disk. If I2 is locked, M2 waits (blocks). (8) -- After O1 is saved on the S2 disk, M2 returns 'end-of-transfer' acknowledgment to M1 which completes the POST exchange. (9) -- M1 unlocks O1 and exists. |
| b) Compute-server (future Web server) mode: (1) - M1 locks its memory object O1. (2) - M1 checks if socket connection to M2 is in M1 connection table. If yes, go to (5) below. Otherwise, M1 connects to S2 and sends M2 creation script. (3) - S2 spawns M2 and acknowledges. (4) - M1 receives acknowledge message and saves new socket in connection table. (5) - M1 gets O1 handle. (6) - M1 writes O1 to M2 using socket lib calls. (7) - M2 reads O1 using socket lib calls. If I2 is free, O1 buffer is copied directly to I2 buffer. If I2 is locked, M2 creates O1 clone and blocks. (8) - M2 sends acknowledge to M1. (9) - M1 unlocks O1 and blocks. |
Full HTML Index
User-level WebWork metaphor is given by WebFlow -- a distributed dataflow model built in terms of WebWork modules and MIME object/document communication channels.
|
| WebWork users will build and control distributed computing applications (compute-webs) using Web browsers based visual interactive editors and monitors. |
| We are currently prototyping such WebFlow front-ends at NPAC using Java/HotJava model. WebWork modules are represented by Java threads (Screen 6) and visualized as interactive interconnected icons (Screen 7) |
Full HTML Index
| An example of HotJava applet that makes essential use of Java multithreading. |
| Three different sorting algorithms are visualized on a single HotJava page. |
| Each algorithm can be started independently or they can all run concurrently. |
| Concurrent mode allows for real-time visual comparison of various algorithms and their performance. |
Full HTML Index
| Early prototype of AVS or Khoros like visual compute-web editor. |
Two interactive modes are supported:
|
| In mode a), each click in the active editor window places a new module box there. |
| In mode b), each click on module port generates links with all other modules. |
Full HTML Index
| Latest results prepared for HPDC95 Tutorial August 1,1995 |
Full HTML Index
| In WebWork, we also propose an interpreter of HPFCL -- High Performance Fortran Coordination Language, which will support coarse grain distributed HPF computation. Compiled HPF modules, published on individual nodes of the WWVM, will be easily invoked by HPFCL scripts, integrated with GUI front-ends (such as Khoros etc.) and employed in collective computation on the WWVM. |
| In WebHPL we further explore the concept of interpreted HPCC language environments and we propose an object-oriented Web based parallel programming environment supporting HPF and C++ for distributed metacomputing. |
| WebHPL, or Web based High Performance Languages, is our most ambitious project in the area of Web and HPCC integration. It addresses both base software engineering and applications, and it refers both to backend and frontend layers of language compilers and interpreters, seeking a uniform programming model for interactive HPCC. |
Full HTML Index
| WebHPL is a hybrid Compiled/Interpreted environment with a more or less seamless use interface. |
| We use compilers or optimized message passing in the local environments where low latency hardware benefits from careful optimization. |
| We use interpreters where flexibility and power (e.g. fault tolerance) of agent approach beneficial and performance overhead in software matches intrinsic lower performance of hardware. |
| We expect that our two-prong approach in WebHPL, attacking the problem both from the full compiler and the full interpreter perspective, will result in an interesting software engineering framework, 'scalable' along the compiler<--->interpreter axis. |
| The interpolation mechanism along this axis is provided in terms of domain specific "little languages" that are interpreted but operate on compiled modules. UNIX shell, Perl or Tcl are examples of sequential little languages. WebHPL will develop parallel analogs. |
Full HTML Index
| One current WebWork/WebFlow application, prototyped at NPAC, is Software Project Manager (Screen 8). Each software developer runs his/her WebTools server and uses HySource CASE tools. These servers are WWVM-connected to agent and manager servers. Agent server receives automatic notifications from developers servers on each software volume update, and uses customizable thresholds to decide when to fire a report to the manager or a deadline reminder to a developer. |
| Software Project Manager tools contains a simple agent server that mediates between client/consumer ( here manager) and servers/producers (here developers). |
Full HTML Index
| A front-end for the software project manager tool. |
Three types of modules are supported:
|
| Developer modules are linked to the agent module and report automatically all changes in the software volume (handled by WebTools CASE toolkit integrated with WebTools editor). |
| The agent module integrates the results and uses customizable threshold to decide when to fire a report to the manager or a deadline reminder to a developer. |
Full HTML Index
| More generally, this Middleware Layer 2 will be rather complex and populated by a spectrum of proprietary (e.g. Telescript, ScriptX, CORBA) and public (e.g. Perl, Tcl, Harvest, Java, VRML) scripted languages, brokers, agents, wrappers, mediators etc. see Screens |
| In WebWork, we refer collectively by WebScript to the whole ensable of these models. |
| At the current stage, it isn't clear if WebScript as a common intermediate language is a practical concept. An alternative is to live in the multi-language Web medium and emply interoperability agents to translate between various protocols. |
| Practical initial implementation platfrom for this dual approch is provided in WebWork by an integrated collection of WebTools CASE tools based HySource Worlds for various languages. |
Full HTML Index
| A sample VRML page produced by Black Hole Simulation group at NCSA and displayed by SGI WebSpace, cooperating with Netscape Navigator. |
| A set of 3D spaces related to gravity research is represented as Netscape icons and linked to the corresponding VRML worlds. |
| WebSpace window displays one of these spaces -- a space-time diagram for two black hole collision ('Pair of Pants' diagram) |
Full HTML Index
| Example of VRML source code (for the black-hole space-time diagram of previous figure). |
| The full file is 0.5Mb and has been reduced here by removing many lines of numerical data for 'Coordinate3' and 'IndexFaceSet' vectors. |
|
Get source here |
Full HTML Index
| Example of Java code, forming an applet for interactive performance measurement of Java thread scheduler functions. |
| The example illustrates typical objects and methods used by Java multithreading. |
|
Get source here |
Full HTML Index
| WebWork Interpolates and Integrates pervasive Web HPCC and (nonHPCC) commercial software as in following table comparing computing concepts in three "worlds"; HPCC -- Commercial mainstream -- Web |
| Current Web model needs computational extensions for banking/financial applications, manufacturing, interactice shopping/videogames etc |
| HPCC can provide Web both parallel computing programming models, libraries and language/runtime concepts which coordinate components of distributed or parallel system |
| HPCC needs the Web (or equivalent) to give it viable distributed computing and software engineering base |
| The Web interpolates between "flaky" research software and solid but closed corporate solution. Clear trend away from proprietary towards open software models. |
Full HTML Index
| Current HPCC, Current Commercial Mainstream, Current and conjectured future Web |
|
Print this from Postscript Version |
|
The paper describing this project is available at SCCS715 in NPAC technical report series |
Full HTML Index
Agent
|
Application
|
Bottom-Up Process
|
Channel
|
Client
|
Compute-Server
|
Compute-Web
|
Database
|
Full HTML Index
Document
|
Editor
|
HPFCL -- HP-Fickle for High Performance Fortran Coordination Language
|
Middleware
|
Module
|
Object
|
Object Type
|
Port
|
Full HTML Index
Problem
|
Problem Solving Environment
|
Publication
|
Server
|
Software Process
|
Solution
|
Top-down Process
|
Full HTML Index
VSL or Virtual Software Laboratory
|
WebFlow
|
Web Productivity Tools
|
WebScript
|
WebTools
|
WebWork
|
WWVM or World Wide Virtual Machine (Layer 1 of WebWork)
|