\setcounter{chapter}{5}
\setcounter{equation}{0}
\setcounter{figure}{0}
\chapter{A Meta-Computing Software Development Environment}
\markright{Chapter 6. A Meta-computing Software Development Environment}
\section {Software Integration Approach in Meta-Computing}

The complex problem structures and domain found in a meta-application
require that the programming paradigms/models used to represent
problem structures also be complex and heterogeneous. 
Mixed programming languages and paradigms are expected to be integrated
in order to collectively solve a complex meta-problem.
Problem abstraction and task decomposition requirements
in a meta-problem imposes the following 
requirements on software engineering:
\begin{enumerate}
\item
Abstraction of task/module and data objects in an integration framework
\item
Templates, modules, and objects concepts at language level and framework level
\item
Wrappers for existing programs probably written in different languages
\item
Problem decomposition essential in abstraction to partition
meta-problem into logical, relatively independent smaller pieces
\item
Exploit parallelisms at all levels of abstractions
\end{enumerate}

To be able to efficiently and effectively 
map a complex meta-application onto a meta-computer,
a general software integration environment will have the following common
features:
\begin{enumerate}
\item
A single unified integration framework:
Components/tools in the environment should be tightly coupled and readily accessible to
developers and end-users. This unified framework
 should be able to coordinate interactions among groups of tools and program components
in a flexible way.

\item
An abstraction hierarchy of program components and data objects, 
and their interfaces, should be preserved to allow the refinement of
data and task decomposition and implementation.

\item
Multiple, yet different, programming paradigms/models should be able to coexist 
in the framework so that parallelisms at different levels can be exploited.

\item
Simplicity and effectiveness: New and existing program components
can be easily ``glued'' together.

\item
Extensibility, flexibility, and portability:
The framework should allow seamless extension of a meta-problem and a meta-computer;
for example, new components can be added into an 
existing one.

\end {enumerate}

A software integration framework will build a data and execution model
that is based on a dataflow message-passing paradigm to permit 
fine-grained parallelisms within
modules, while at the topmost level dataflow execution is
used to integrate all functional and software components (program
modules) in a HPDC application. The framework will be using
restricted data-driven message passing while providing
some of the benefits of data-parallel languages such
as automatic data decomposition and collective communication and will
still allow users to write in a familiar language,
using standard software packages.

This process-based distributed integration framework, 
although originally designed
for workstation operation (AVS, SGI
Explorer, IBM Data Explore, Khorus, etc.), 
offers a means by which distributed systems may be connected
by workstations or high-performance computing systems.
They are characterized by a user-built framework of interconnected
modules representing the dataflow for a given application. Such systems
support rapid prototyping, plug-in-like modular composition,
interactive application steering, and location 
independence. From the application writer's perspective, this approach 
relieves the programmer of needing a detailed understanding of architectural details 
and offers significant software savings through the reuse of modules. Because
the interfaces between modules are well defined, modules developed
at different computer sites/platforms may be freely shared.

The dataflow network constructed at runtime supports distributed computing in a
heterogeneous LAN or WAN network environment of high-performance systems
and graphics workstations. Modules themselves perform the underlying work of data
input, simulation computation, fine-grained parallel programming in data-parallel or
message-passing computation, data mapping, data transformation, rendering, and output. 
It hides the
inherent complexity of different internal numerical representations, byte ordering,
vector or scalar visualization algorithm dependencies, or
serial or parallel I/O in a modular HPDC
heterogeneous framework. Overall, it gives the user control over the
optimization of heterogeneous resources and network capabilities.

Users may interactively steer simulations through the adjustment of an application's 
critical parameters. In a similar manner, modification and adjustment of the
process itself may be performed in an interactive manner. The same
framework that facilitates the easy creation of complex application tasks
also provides for its quick modification to include new and different data-mapping
and parallel-processing functions.

The modular approach of the framework lends itself  well to software sharing, module
reuse, extensibility and flexibility. Software sharing means that modules
created for one high-performance architecture can readily be ported
to another similar architecture. The reusability aspect of the system means the HPDC 
parallel code and algorithm are modularly developed and compiled once, then reused
indefinitely in a plug-and-play mode in a variety of diverse applications. The
extensibility of the framework is such that users may write their
own modules in their favorite HPDC language/program binding. System adaptability is a 
key goal in software engineering and remains 
a major feature in  integration architectures. 
Custom integration approaches usually yield point-to-point 
integration solutions, and 
the system becomes difficult to extend and modify due to the complexity
and hardwired solution for coupling components. This dataflow-based approach
provides an open interface and flexible architecture to realize
adaptability on a meta-computer for HPDC applications.

\section{A Layered Meta-Computing Software Engineering Environment}
A meta-computing programming system should have a layered software
engineering environment to hide system complexity from end-users and
to represent problem and program abstraction hierarchy. As proposed
in \cite{vdce} and shown in Figure \ref{layers}, 
a meta-computing software system has in general three
major layers:
\begin{enumerate}
\item
{\bf Application specification layer}.
End-users and application developers use this layer to develop and configure
a meta-application in a conceptual and modular fashion. Using a coarse-grained,
task-level, functional decomposition mechanism, a meta-application is represented
as a task graph in which each task is independent of the programming model, 
programming language, and computer architecture. A visual programming
interface is essential to allow developers to describe task decomposition and
their relationships graphically. Another important component in this layer is
a high-level description language to represent task descriptions
of a meta-application abstraction, independent of its meta-computer
implementation. In the JavaFlow framework, to be described in the next section, 
we develop a Network Editor to create, connect, and formulate a dataflow-based
task graph from a ``drag-plug'' GUI interface. A FlowScript language is used
to record the application specification and task graph configuration.
\item
{\bf Application configuration layer}.
Mapping from a task graph to a virtual meta-computer is conducted at this layer.
Given a library of task implementation and a description of a physical meta-computer,
a task configuration manager allocates resources to map/bind task programs onto a
physical meta-computer configuration.  An optimized scheduling mechanism can be 
considered to achieve
high-performance program execution and load balance. 
The result of this layer is a task-node
configuration to be scheduled to run on a meta-computer infrastructure.
In JavaFlow, we propose a virtual machine concept of JavaFlow VM that is used
to separate the construction of task and meta-computer graphs from physcial
resource allocation and task-node mapping. 
\item 
{\bf Application execution layer}.
This layer operates as a runtime system of this meta-computing software
environment. It schedules the execution of the allocated tasks on a physical meta-computer
configuration. In JavaFlow we use a generic Web/Java server technology
``Servlet'' to deploy dynamically the dataflow execution runtime system
on a configuration of WWW servers that consists of the meta-computer software/hardware
infrastructure.

\end{enumerate}

\begin{figure}
\centerline{\epsfxsize=420pt \epsfbox[0 0 612 750]{chapter6-javaflow/layers.eps}}
%BoundingBox: 0 420 612 750
\caption{A Three-Layered Meta-Computing Software Development Environment}
\label{layers}
\end{figure}

The next section elaborates on key components of a specific layered meta-computing software
engineering environment.

\section {JavaFlow --- A Java Implementation Framework of the
Dataflow Model on WWW}

In this section we outline JavaFlow, an implementation plan of
a software integration system to support 
publishing, composing, and executing distributed Java programs/modules for
distributed computation. 
Javaflow is a design realization of the proposed general dataflow framework in Chapter 2
on a Java/WWW platform.
Major architectural and design issues in building the proposed
framework are touched upon. The implementation is based on an object-oriented system that supports
messages and threads. It uses a data-driven execution model to 
coordinate parallel and distributed coarse-grained Java and other software
component programs in a Web/Java programming environment.

\begin{figure}
\centerline{\epsfxsize=420pt \epsfbox[70 70 550 710]{chapter6-javaflow/javaflow.eps}}
\caption{The JavaFlow Architecture and Major Components}
\label{javaflow}
\end{figure}

As shown in Figure \ref{javaflow}, major components and key system issues
in Javaflow include:
\begin{enumerate}
\item
A JavaFlow Virtual Machine (JavaFlow VM)
\item
JavaFlow servers
\item
JavaFlow modules
\item
A JavaFlow protocol for dataflow message-passing
\item
JavaFlow API
\item
A JavaFlow runtime support facility
\item
A JavaFlow module and VM node registration subsystem

\end{enumerate}


\subsection{A JavaFlow Virtual Machine (JavaFlow VM)} 

A JavaFlow VM is a meta-computer for
coarse-grained task parallel computing and software integration
in a dataflow message-passing paradigm.
It consists of a directed graph of JavaFlow nodes
where nodes represent virtual processors, each of which runs a JavaFlow
server, and edges represent dataflow directions and dependencies
between connected nodes. 

Execution of a JavaFlow application involves three steps:
\begin{enumerate}
\item
{\bf Construct a JavaFlow module network that represents a task-parallel
application}. A JavaFlow module repository and a JavaFlow editor
are designed to allow module composition from a library of
system and user-defined, reusable JavaFlow modules through
a ``drag-plug'' visual programming interface. A JavaFlow module registration
subsystem is designed to provide a standard mechanism with which
distributed developers can author, publish, and contribute user-defined
JavaFlow modules to the module repository.

\item
{\bf Construct a JavaFlow VM so that the module network in (1) can be
effectively mapped to the JavaFlow VM}. Because a JavaFlow VM is
a virtual machine and a JavaFlow network uses 
a directed acyclic graph as network topology, theoretical
or heuristic algorithms can be developed so that an optimal
mapping can be achieved by using resource and
performance meta-data gathered from
a JavaFlow VM node registration subsystem that is designed to 
register, store, and share JavaFlow servers among willing participants.
 
\item
{\bf Allocate a JavaFlow VM runtime configuration}. Mapping from
the JavaFlow VM to physical processors includes the allocation of
processing resources, setup of execution facility, and
deployment of the module network. A JavaFlow Loader is designed
for this purpose.
\end{enumerate}

The JavaFlow VM runtime configuration can be a loosely distributed system
consisting of machine hosts over a WAN (such as the WWWVM) or
a network of workstations over a LAN (such as a cluster), or
a tightly coupled MIMD or SMP parallel system.
The only system constraint is that each JavaFlow VM node
must support JavaVM \cite{java} runtime and that
the network supports the TCP/IP layer.

Note that when a JavaFlow VM is mapped to a WWWVM (i.e., Web servers as VM nodes), 
a one-to-one mapping
is expected, as a JavaFlow server is essentially an HTTP/Java server
with extended facility, although one or more Web servers can
be mapped (hosted) on a single physical machine.

The concept of a JavaFlow network, a JavaFlow VM, and a VM runtime
configuration and their mappings gives three system design
advantages:
\begin{enumerate}
\item
Construction of a JavaFlow network can be completely application-oriented,
independent of choices of physical processors and network topologies.
\item
Allocation of physical resources can be delayed until execution, independent of
module composition and the mapping from a module network to a JavaFlow VM.
An optimized allocation scheme can be sought at runtime
and a fault-tolerant configuration
can be supported. For example, Figure \ref{javaflow} shows a
5-node JavaFlow VM configuration. The only allocation dependency is
the local client machine V and the remote host D running
native Java class interfaces to non-JavaVM systems. If nodes I, 
II, and IV all fail before or after the initial allocation, the architecture
is able to support a dynamic remapping and reallocation scheme
using a 2-node VM configuration consisting of nodes V and III.
Modules $A_{1}$, $A_{2}$, $B_{1}$, $D_{1}$ and $D_{2}$ are all reallocated to node III.
\end{enumerate}


\subsection{JavaFlow Servers}

A JavaFlow server consists of an HTTP server, a JavaFlow Daemon, and
one or more JavaFlow application modules. A special JavaFlow server
is on a local client machine where a Java-enabled
Web browser replaces the HTTP server to load a JavaFlow Daemon
and modules as applets. A JavaFlow VM runtime system consists 
of a set of JavaFlow servers with loaded
JavaFLow Daemons and modules.

The HTTP server is written in Java in order to be able to support
the standard Java servlet \cite{servlet} mechanism for spawning
external persistent processes --- for example, Jeeves \cite{jeeves} and
JigSaw \cite{jigsaw}. The main functions of
the HTTP server in a JavaFlow system include:
\begin{enumerate}
\item
Provides a JavaVM runtime and programming platform to execute
JavaFlow modules.
\item
Using HTTP, it loads the JavaFlow Daemon and module servlets over the network or
from local disk at runtime.
\item
If required, it uses HTTP to access conventional WWW information and resources on the remote
host.
\end{enumerate}

Once the daemon and modules are loaded on a host,
the HTTPD server will no longer participate in further
JavaFlow execution and is thus free to serve other standard WWW services.
The JavaFlow control thread is then transferred to the daemon for
JavaFlow execution.

In the data-driven computation model 
a JavaFlow module can bind one of its output ports to an arbitrary
number of input ports. Methods invoked on an output port are
broadcast to each input port to which it is bound.
Messages from an output port to an input port are delivered in the order
sent. The message stream delivered to an input port is a merge of
message streams sent to the
input port to which it is bound. The only means of interaction
between JavaFlow modules is message passing sent through
dynamically connected JavaFlow channels between output ports 
of a module and input ports
of another. A module doesn't share an address space with other modules.
It has a persistent state and
must be executed whenever its input ports change. The underlying
data transfer and synchronization is actually carried out by participating
JavaFlow daemons.
A JavaFlow daemon is a distributed flow execution manager that
implements the daemon-to-daemon and daemon-to-modules message passing
in JavaFlow protocol and ensures dataflow execution and data transfer.
Major tasks of a JavaFlow daemon include:
\begin{enumerate}
\item
Accepts incoming messages for input ports of all modules. For example,
at startup a separate thread should be created to listen in a loop to 
a particular port for incoming message streams and to dispatch them to
corresponding handlers for message deserialization and buffering/queuing.
\item
Delivers outgoing messages for output ports of all modules. For example,
using a JavaFlow protocol,
at the request of a running module a thread will be created to get a
serialized Java object stream and then broadcast it to all the daemons whose
JavaFlow modules have an input port bound to the output port.
\item
Synchronizes the asynchronous incoming/outgoing data stream requests and
module executions, including channel/module registration, 
message queuing and buffering, module activation/suspension, 
state monitoring/persistence, resource sharing, error handling, etc.
\end{enumerate}
For data transfer between two JavaFlow daemons on different
hosts, a JavaFlow data transport protocol is used. For data and
control flows between a daemon and a JavaFlow module or
between two JavaFlow modules on the same host, the industry standard 
protocol JavaBean \cite{javabean} can be used. Note that data transfer
between two JavaFlow modules on
the same host must be supervised by the daemon.
Data transfer between two JavaFlow modules on different
hosts is coordinated through the two on-behalf daemons.

JavaFlow modules in a JavaFlow server are Java servlets or applets
loaded for individual application tasks.

\subsection{JavaFlow Modules}

The fundamental unit of computation in JavaFlow is the
module. Modules which process inputs to generate outputs
are intended to be fairly coarse-grained units of computation.
A JavaFlow module is a stand-alone Java applet or servlet that consists of two
distinct parts, the kernel and the interface. The kernel is a typical Java program body in JDK or
various Java APIs for task and object-oriented computations. The JavaFlow
interface defines dataflow input and output ports associated with
each JavaFlow module, using a JavaFlow API.  
A JavaFlow module is actually a wrapped Java applet/servlet in JavaFlow API.
Conversion from a standard Java program to a JavaFlow module
should be straightforward and simple so that automatic generators
may be developed to simplify JavaFlow module authoring and wrap-up.

There are generally two categories of JavaFlow module: system and user-defined.
System modules are JavaFlow system building blocks such as the 
JavaFlow daemon, JavaFlow Editor, JavaFlow Loader,
JavaFlow Registrar, or general-purpose modules such as RDBMS readers,
common GUI instruments, data filters or compressors, 2D image viewers, 
3D VRML viewers, audio/video players, Web agents, etc. 
 
A JavaFlow module can be either downloadable or native.
A downloadable JavaFlow module is completely written in JDK and Java APIs
with imported JavaFlow classes. It
is compiled by a standard Java compiler into a stand-alone Java byte-code,
to be able to ``write once; run anywhere; reuse everywhere,'' i.e,
downloadable to any remote JavaFlow server or Web browser via the servlet/applet
mechanism. A native module is a wrapped JavaFlow module consisting
of a JavaFlow part in JavaFlow API and a native part. The native
part is usually written in other 
parallel systems such as MPI, PVM, or HPF 
to interface with non-Java, fine-grained node programs 
that are attached to a particular host. For example, in Figure
\ref{javaflow} the module $C_{1}$, which provides a Java interface to
non-Java systems such as CORBA or PVM, is a native JavaFlow module
that must run only on host III. Note that many Java APIs
and standards are currently being developed in industry for replacing or 
interfacing legacy systems. We expect that the majority of JavaFlow modules 
are downloadable programs. For example, in Figure \ref{javaflow}
a downloadable module $A_{1}$ may use a portable Java interface 
JDBC \cite{jdbc} to access
multi-vendor legacy databases.

A JavaFlow module can be either a servlet to be loaded to an HTTPD/Java
server or an applet to be downloaded by a Web browser. Servlets are similar to
applets in that they are Java-object bytecode that can be dynamically loaded
off the net. Servlets differ from applets in that they are
faceless objects.
Most applet JavaFlow
modules are source or terminal dataflow modules. For example, in
Figure \ref{javaflow} applet $E_{1}$ can be a GUI to provide input graphical
widgets to get input data on the client machine; applet $E_{2}$ may be
an image viewer to display application results.

Note that a JavaFlow module is different from a Java object. A module is an active
agent whose state can  be changed only by the module itself or by a JavaFlow daemon
at the request of another JavaFlow module/daemon.


\subsection{A JavaFlow Protocol for Dataflow Message Passing}
As defined in the dataflow execution model, a JavaFlow module 
interacts with other modules by 
receiving and sending object data for
message passing in a data-driven fashion. 
The JavaFlow protocol defines when and how object data is transferred
between JavaFlow daemons and modules in a reliable manner, and
how data transfer and program execution are synchronized to guarantee a deterministic 
dataflow computation semantics.

While TCP/IP is the base data transport protocol on which JavaFlow
protocol is built, there can be many approaches for
defining and implementing a data-driven protocol. Instead of giving
a detailed protocol specification here, we merely identify several
important issues.

Each JavaFlow module has
a set of predefined input ports through which it receives
input data and a set of output ports through which it sends output data.
Each of them has a unique identification that can be
communicated between modules. Javaflow modules and their associated ports
are the only objects with a global identification scope that is visible
to the rest of a JavaFlow application.

A JavaFlow daemon handles messages that arrive and messages to be sent on
behalf of running JavaFlow modules. 
For example, it can queue and buffer incoming/outgoing message
streams for each JavaFlow module. 

Message passing between a JavaFlow daemon and modules
or among JavaFlow modules on the same host should use the
most efficient mechanism possible, such as shared memory or message queues.
The emerging industry standard component model JavaBean API 
can serve this purpose. In JavaBean
API, a JavaFlow daemon can be implemented as a parent container
and JavaFlow modules as components. This provides the context for module components
to be arranged and to interact with one another.
More specifically, 
two of the proposed JavaBean services can be 
used---namely, the persistent service that allows a JavaFlow daemon
to buffer incoming and outgoing message streams
and to store the state information for
active modules, and the event-handling service for events raised
in a JavaFlow applet module to be delivered to the JavaFlow daemon
and appropriate modules on the same host. 

Objects sent by a JavaFlow daemon
are usually converted into a string, sent across the network, and then
reconstructed into their original types by the receiving server. Java
methods are used to convert an object to a string and to create
an instance of the sending object at the receiver. A more advanced
mechanism, such as Java serialization \cite{java-serial}, can
be used to create a serialized representation of objects or a graph
of objects with sufficient information to insure that
the equivalent typed object can be recreated.

The actual implementation of the daemon-daemon protocol
has to consider a trade-off between reliability and performance.
It can be sessionless, similar to HTTP, to achieve  reliable
message passing over wide area networks.
A sessionless protocol usually has a poorer performance
and higher latency than a session-oriented protocol. Due to the
coarse-grained feature in both dataflow requests and server/module
granularity, an HTTP-like protocol with extended support for
Java serialization may be a good solution. The base protocol should 
be message-based so that the most recent input messages received for a module
can be buffered. For stream-based 
data types such as audio/video feeds, a connection-oriented 
protocol extension has to 
be developed and a direct port-to-port transfer channel 
should be used to feed stream data between two modules, bypassing
the buffering mechanism.

In the current architecture we rely on standard HTTP protocol
to load JavaFlow modules to distributed JavaFlow servers over the network.

\subsection{JavaFlow API}

JavaFlow API defines a Java programming interface to allow
a JavaFlow developer to write, author or wrap a JavaFlow module. While
the computation part of a JavaFlow program is usually written in
Java Core API or other Java standard APIs such as JDBC and Java Media,
JavaFlow API specifies Java data types, object classes, and interfaces that
are specific to the JavaFlow system. In the form of the
importable Java class library, JavaFlow API should include
two major functional components:

\begin{enumerate}
\item
{\bf Primitive and composite data types}. 
Each input and output port has an interface type 
associated with it; this type defines the types
of object data that can be sent through the output 
port or received from
the input port. 

All classes in Java Core API should be allowed as primitive JavaFlow data types.
For composite data types, selected classes defined in major
Java APIs can be considered; for example, file and file directory
such as HTML page, relational table in JDBC, audio/video stream
in Java Media. User-defined data types should also be supported.

One of the major advantages
in adopting Java programming as a uniform and object-oriented
software development platform in that JavaFlow data are object-oriented.
This means that when a JavaFlow object is transferred, all operations (methods)
defined on the object type are implicitly included. This is significantly
better than stream-type systems such as PVM and AVS in which
explicit conversion procedures must be used in a module to
pack/unpack a data stream. In the JavaFlow system this can be
transparently handled by
the Java Serialization mechanism.

\item
{\bf Interfaces and classes to define a JavaFlow module
and associated ports}. These include initializing and describing
Java programs to JavaFlow, filling objects in output ports
or fetching objects from input ports, error handling, etc.
\end{enumerate}

\subsection{A JavaFlow Runtime Support Facility}

This facility has three components:

\begin{enumerate}
\item
{\bf A JavaFlow Editor}. This provides a visual programming interface in a Java applet
for creating, revising and starting a JavaFlow network consisting of 
JavaFlow modules from a repository of reusable JavaFlow modules.
A JavaFlow network represents a
distributed application described as a coarse-grained task graph
in data-driven execution. A JavaFlow Editor should consider
these major issues:
\begin{enumerate}
\item
From the client machine this applet accesses to a module repository on one
or more Web servers and makes graphical presentations for each module
and its ports.
\item
Allows a user to interactively
compose a JavaFlow network by drag-plug using a mouse
to connect network channels. A channel must agree on the same
data type on both ends.
\item
If a user chooses to explicitly bind/map the constructed
JavaFlow network to a JavaFlow VM runtime configuration, this interface
should also provide visual means to interactively facilitate the mapping 
by accessing to a JavaFlow server repository
on a repository server. Note that certain modules have 
particular node dependency.

\item
A constructed JavaFlow network, mapped or unmapped to a JavaFlow VM runtime,
should be represented in a JavaFlow script language (FlowScript),
to be sent to a JavaFlow Loader on the JavaFlow
Repository server to start the deployment and execution of the JavaFlow
application on a JavaFlow VM runtime system.
A user can also save it as 
a JavaFlow application for future reference.

\end{enumerate}

\item
{\bf FlowScript, the JavaFlow script language}. 
FlowScript is a readable text language designed to record basic user interaction
and a JavaFlow network configuration in the form of script files or streams.
The major purpose of the scripting language is to separate and bridge
the process of a JavaFlow network construction in a network Editor from the 
process of 
mapping the network to a JavaFlow VM runtime system in a Loader. 
Module networks in FlowScript
are interpreted by a Java Loader on a Repository server.

We expect the script language to be HTML-like, in particular,
the URL form of servlet syntax should be preserved, i.e.,
a servlet with a source host and a destination host is denoted as
{\sl http://$<$server host$>$:$<$port$>$/servlet/$<$servlet URL$>$}, where
the {\sl $<$servlet URL$>$} is a regular URL that points to the location
of the servlet. For example, to represent a servlet $A_{1}$ stored
on a Web server {\tt http://npac-repository/servlet/A1} that
is to be loaded
on another Web server {\tt http://greatwall:1963}, the 
URL is \\
{\tt http://greatwall:1963/servlet/http://npac-repository/servlet/A1}.

FlowScript is also used to save networks and parameters settings
that can later be read for modification or execution with or without requiring the
user to use the JavaFlow Editor directly.

\item
{\bf A JavaFlow Loader}.
Using a JavaFlow network configuration in FlowScript and a
JavaFlow VM node repository, the Loader servlet running on the repository
server performs dynamic mapping/allocation from the module network
to a JavaFlow VM runtime system. It needs to check at runtime
the availability of nodes and networks. Certain optimization
techniques may be used to find an efficient mapping. After the runtime
configuration is chosen, the Loader drives remote HTTPD servers to load
JavaFlow daemons and modules servlets over the network or from local disks via HTTP. 
Note that the daemon and module applets to be run on the local client machine must be
loaded by the Editor.
 
\end{enumerate}

\subsection{A JavaFlow Module and VM Node Registration Subsystem}
This subsystem is 
used to build a JavaFlow module repository and a JavaFlow VM node repository.
Together with the
JavaFlow API, it provides a standard
mechanism to:

\begin{enumerate} 
\item
Allow module developers to publish, contribute and share
standard user-defined JavaFlow modules to a central or distributed
module repository. Benefited from a URL representation of servlets/applets,
only a symbolic identification is needed to store a JavaFlow module information
in the repository. The actual Java bytecode can be kept on any
contributing Web server over the network. 

\item
Allow individual Web servers to be added into a server repository
to form a meta JavaFlow VM. A JavaFlow VM node presence in the repository
is also a text form such as in URL.

\end{enumerate}

Note that for a native-class module, both the module and
the remote host must participate in the JavaFlow runtime system.
Thus the two identification data are stored together.

\subsection{Summary} 

A typical procedure for a developer
to create and run a JavaFlow application can be:
\begin{enumerate}
\item
Write JavaFlow modules in Java and JavaFlow API and
place them on a Web/Java server that supports servlet extension.
\item
Publish the modules by using a JavaFlow Registrar
applet on a known JavaFlow repository server.
If you are willing to participate in the JavaFlow VM configuration,
you may also publish the Web server using the same interface.
\item
Use the JavaFlow Editor on the repository server
through a Web browser to construct a JavaFlow network
consisting of your own or other modules available on
the repository server. You may also want to construct
a JavaFlow VM and map the network onto it, using the same
Editor. Or you can leave the system to find an optimized mapping for you.
\item
From the same Editor interface, start to execute the JavaFlow network.
\end{enumerate}

Actually, step 1 and step 2 can be optional. A user can start to build
a JavaFlow module network by directly re-using existing modules from one or
more module repositories. By providing the
JavaFlow Registration mechanism, we expect that many JavaFlow applications
can be created in this fashion, free of programming by a JavaFlow user.

In summary, this JavaFlow implementation has the following key features:
\begin{enumerate}
\item
It is Web-based. Web servers are extended virtual processors for 
general-purpose distributed computing.
\item
As both a uniform programming environment and a software engineering
platform, Java becomes the single vehicle to carry out the implementation of the
proposed software integration framework. It allows the combination of
object architecture, Web architecture, and a component architecture for
an integrated programming platform, and a new software engineering environment for a wide range of
HPDC applications.

\item
This open architecture and objected-oriented approach allows us to adopt the open system in Web and Internet 
to be able to grow with the emerging evolution of WWW technologies, Java, in particular.
For example,
JavaBean is a major effort in the Java community in this direction.
It is an industry component technology standard to facilitate
composition/interaction of reusable Java programs in a seamless, independent, transparent and
portable fashion, focusing on the software engineering aspect of Java programming and engineering.
\item
This plan is truly portable and independent of platforms and operating systems.
It frees developers from having to know
details of platforms, machine architectures, and network topologies.

\item
This is a fully distributed implementation in that there is no centralized
facility to coordinate control and data flow among distributed programs.
The protocol implementation can
consider the performance and reliability trade-offs of a network
that may have medium-bandwidth and high-latency and yet consist of different
data-transfer protocols.

\item

Utilizing existing and emerging WWW/Java technologies, the overall implementation
is built on a tiny kernel with a modular architecture. Most JavaFlow
components are stand-alone Java programs or even JavaFlow modules, including
the JavaFlow daemons, the JavaFlow Editor, and the JavaFlow Loader.

\end{enumerate}

