\section{Introduction\label{intro}}


The Internet is characterized by large fluctuations in resource
availability and quality.  These variations are primarily caused by
congestion and as such their occurrence cannot be predicted
reliably. The magnitude of the problem is illustrated by
Figure~\ref{fig:bw-var} which shows both short-term and long-term
variation in end-to-end network bandwidth between two machines, one at
UCLA and the other at the University of Wisconsin. This is apparent to
anyone who uses a web browser over the period of a single day.

\begin{figure}[tp]
\begin{tabular}{cc}
\psfig{file=short-term-bandwidth.ps,width=3in}&
\psfig{file=long-term-bandwidth.ps,width=3in}\\
(a) Short-term variation &
(b) Long-term variation 
\end{tabular}
\caption{Variation in application-level network bandwidth.}
\label{fig:bw-var}
\end{figure}

One of the biggest challenges for efficient data access over the
Internet is effective adaptation to resource fluctuations.  There are
a variety of emerging high end applications that are particularly
likely to benefit from an ability to adapt to changing patterns of
network and computational resources. The applications fall into two
roughly defined categories, metacomputing applications and customized
multicast (customcast) applications. Metacomputing applications
require synthesis of information obtained from multiple sources.

Metacomputing applications may generate data products that combine
results obtained from a variety of data sources. Data sources can
consist of archived information, information generated by physical
simulations, information from scientific instruments such as light or
electron microscopes or data from sensors based on land, aircraft, or
satellites.

In metacomputing scenarios, we generate a sequence of data products
by using a number of programs that are configured as a pipeline. For
instance in defense and disaster response scenarios, there is a need
to generate data products that combine information from combinations
of satellite sensors, aircraft sensors and ground based sensors. These
synthesized data products need to be generated on an ongoing basis.

We anticipate that the process of data product generation will require
moving large amounts of data through variable quality network
links. There are often significant computational costs associated with
data combining.  Resource aware scheduling may have to occur due to
changes in network characteristics, changes in the computational
demands associated with data combination, or changes in availability
of computational resources.

As an example, suppose a major earthquake strikes and the decision is
made to provide massive emergency relief.  Crisis managers will need
to quickly combine archival, reconnaissance and seismic data to
delineate areas of maximal structural damage and merge this
information with census and business data to determine local resources
that could be mobilized. For the prototype that is modeling these
situations, dynamic hierarchies of mobile programs could be configured
into a pipeline to process and combine data from multiple sensors and
databases.

Significant characteristics of metacomputing applications are that (1)
the applications are data-intensive and coordinate data from
geographically distributed data sources, (2) while components must
interact with each other, they may use different types of parallelism
and different data formats, or execute only on specific platforms.

Customized multicast (customcast) applications involve customized
propagation of data obtained from a single source. A simple example of
customcast from the non-high performance computing domain involves
periodic distribution of news clippings selected from a database on
the basis of user profiles. High performance computing issues come
into play when information to be disseminated involves large
quantities of filtered sensor data. Customcast can play an important
role in complex metacomputing scenarios. Metacomputing applications
typically involve synthesizing data obtained from a variety of
sources. For instance, customcast would be used when several
metacomputing applications need to use different subsets of a given
data source.

An example of customcast arises in an application under development by
our group in the Department of Pathology at Johns Hopkins Medical
School.  The application (the Virtual Microscope) under development at
Johns Hopkins involves development of software that makes it possible
to carry out a realistic digital emulation of a high power light
microscope. Raw data is captured by scanning collections of full
microscope slides under high power. For each microscope slide, high
power images are captured at multiple focal planes. Once the image is
captured, this application will make it possible to emulate the
behavior of a microscope by allowing users to continuously move the
stage and to simulate changing magnification and focus. The Virtual
Microscope is being designed so that it will be possible to achieve an
interactive level of response as the same dataset is simultaneously
explored by multiple users.  Customcast can be employed due to spatial
and temporal locality. While each datasets is quite large (10-100
Gbytes per specimen), most users will access only the portions of the
data set that contain features that are of medical interest.
Moreover, due to psychophysical limitations of human data processing,
users will frequently dwell on portions of the dataset that are of
particular interest.


Mobile programs can move an active thread of control from one site to
another during execution. This flexibility has many potential
advantages.  For example, a program that searches distributed data
repositories can improve its performance by migrating to the
repositories and performing the search on-site instead of fetching all
the data to its current location. Similarly, an Internet
video-conferencing application can minimize overall response time by
positioning its server based on the location of its
users. Applications running on mobile platforms can react to a drop in
network bandwidth by moving network-intensive computations to a proxy
host on the static network. The primary advantage of mobility in these
scenarios is that it can be used as a tool to adapt to variations in
the operating environment.  Applications can use online information
about their operating environment and knowledge of their own resource
requirements to make judicious decisions about placement of
computation and data.

Many systems provide some form of support for program mobility.  The
simplest form of support is the ability to download code and execute
it to completion at a single site -- as provided by systems like
Omniware~\cite{AdlTabatabaiLLW96}, Safe-TCL~\cite{Borenstein94},
Java~\cite{GoslingM95}. Other systems like Avalon~\cite{ClamenLNW90},
NCL~\cite{Falcone87} REV~\cite{StamosG90} and Obliq~\cite{Cardelli95}
allow programs in execution to initiate computation on remote nodes
and wait for their completion. The most sophisticated support is
provided by systems like Agent TCL~\cite{Gray96},
Emerald~\cite{JulLHB88}, Mole~\cite{StraberBH96},
Aglets~\cite{LangeO96}, TACOMA~\cite{JohansenRS95} and
Telescript~\cite{White96} which permit an executing program to move
while it is in execution.

Programs that use mobility as a mechanism to adapt to resource changes
have three requirements that are not shared with other mobile
programs. First, they need to be aware of their execution environment.
In particular, they need to monitor the level and quality of resources
in their operating environment. We refer to this as the {\em
awareness} requirement. Second, they need to be able to react to
changes in resource availability. We refer to this as the {\em
agility} requirement. Third, they need to be able to control the way
in which resources are used on their behalf (by libraries and other
support code). We refer to this as the {\em authority} requirement.

In this paper, we describe the design and implementation of Sumatra,
an extension of Java that supports resource-aware mobile programs. We
also describe the design and implementation of a distributed resource
monitor that provides the information required by Sumatra programs.

We first discuss the constraints that the requirements of awareness,
agility and authority place on languages for mobile programs. We then
describe Sumatra and discuss how its design decisions have been guided
by these constraints. Next, we discuss the considerations that guide
the design of resource monitors, in particular, fault-tolerance and
the need to allow applications to control the use of resources on
their behalf.  We describe Komodo, a distributed resource monitor and
discuss how well its design meets the requirements. Both Sumatra and
Komodo have been implemented and are available to individuals and
organizations with a Sun JDK license. We provide implementation
details wherever appropriate. Finally, we describe our initial
experience with a prototypical application that combines and composes
weather images from multiple geographically-distributed sources.

In this paper, we assume that the reader is familiar with mobile code
languages. For introduction to mobile code languages and mobility
mechanisms, see
\cite{AdlTabatabaiLLW96,Borenstein94,Cardelli95,ClamenLNW90,Falcone87,GoslingM95,Gray96,JohansenRS95,JulLHB88,LangeO96,StamosG90,StraberBH96,White96}.


