\section{Design and implementation of Sumatra}
\label{sec:sumatra}

Sumatra is an extension of Java that supports resource-aware mobile
programs.  Platform-independence was the primary rationale for
choosing Java as the base for our effort. In the design of Sumatra, we
have not altered the Java language. Sumatra can run all legal Java
programs without modification.  All added functionality was provided
by extending the Java class library and by modifying the Java
interpreter, without affecting the virtual machine interface.

Sumatra adds four programming abstractions to Java: {\em
object-groups}, {\em execution-engines}, {\em resource-monitoring}
and {\em asynchronous events}. An object-group is a dynamically
created group of objects. Objects can be dynamically added to or
removed from object-groups.  All objects within an object-group are
treated as a unit for mobility-related operations. This allows the
programmer to customize the granularity of movement and to amortize
the cost of moving and tracking individual objects.  Object-groups
also allow the programmer to control the life-time of objects. Objects
that are included in an object-group continue to live on a host even
after the thread that created them completes execution or migrates to
some other host. Objects that do not belong to an object-group are
subject to garbage-collection as usual.

An execution-engine corresponds to the notion of a ``location'' in a
distributed environment.  In concrete terms, it corresponds to an
interpreter executing on a host.  Multiple engines can exist on a
single host.  Sumatra allows object-groups to be moved between
execution-engines. An execution-engine may also host active threads of
control. Currently, multiple threads on the same engine are scheduled
in a {\em run-to-completion} manner. We plan to implement other
scheduling strategies in future. Threads can move between engines.

The resource-monitoring support in Sumatra allows programs to either
query the level of resource availability or to control the extent to
which various resources are used by the program itself as well as
library code it is linked to. Both on-demand as well as continuous
monitoring is supported. For continuous monitoring, the
resource-monitoring interface allows programs to register jitter
thresholds which determine which resource changes should be reported.

In Sumatra, asynchronous events are used to notify executing programs
about urgent changes in their execution environment. These
notifications can come either from the interpreter or from the
external environment (the operating system or some other
administrative process). We expect that the interpreter would use
asynchronous events to notify the program about violations of
resource-restrictions requested by the program itself; we expect that
the external environment would use asynchronous events to inform the
program about changes in the environment of the interpreter including
resource revocation.  Sumatra allows programs to register handlers for
asynchronous events. The handlers for asynchronous events are able to
inspect the current state of execution and can take appropriate action
including moving away from the current execution site or changing the
resource-restrictions in force.

In Sumatra, computation begins at a single site and spreads to other
sites in three ways: (1) remote method instantiation, (2) remote
thread creation, and (3) thread migration. Remote method instantiation
corresponds to the familiar notion of RPC (remote-procedure-call)
whereby the calling thread is suspended while an operation is
performed, on its request, at a remote site. Remote thread creation
differs from remote method instantiation in that the new thread is
independent of the creating thread; the creating thread continues
execution once the creation is complete. Finally, thread migration
involves stopping the execution of the calling thread at the current
site, transferring its state to another site and resuming execution
at that site.

In the following subsections, we describe the design and
implementation of Sumatra. The first three subsections describe the
programming abstractions mentioned above.  The final subsection
discusses how the design decisions have been guided by the constraints
described in the previous section.

\subsection{Execution-engines}
\label{sec:engines}

Execution-engines correspond to interpreters and are identified by a
hostname and a port number that they listen on. They are created by
specifying these parameters to the constructor. Several error
conditions are possible -- the remote host could be unreachable (due
to a network partition), the remote host could be down, the remote
host may not allow creation of interpreters, the desired port number
could be in use. An exception is returned for each of these error
conditions along with an error message that provides additional
information. Other possible error conditions include authorization
errors. Execution-engines support three operations: thread migration,
remote thread creation and downloading code. Figure~\ref{fig:engine}
presents the execution-engine interface.

Sumatra allows explicit thread migration using a {\tt go()} method
that bundles up the stack and the program counter and moves the thread
to the specified execution-engine. Execution is resumed at the first
instruction after the call to {\tt go}. To automatically marshal the
stack, the Sumatra interpreter maintains a type stack parallel to the
value stack, which keeps track of the types of all values on the
stack.  When a thread migrates, Sumatra transports with it all local
objects that are referenced by the stack but do not belong to any
object-group. Objects that belong to an object-group move only when
that object-group is moved. Stack references to the objects that are
left behind (i.e were part of some object-group) are converted to
proxy references. After the thread is moved to the target site, it is
possible that its stack contains proxy references that point to
objects that used to be remote but are now local. These references are
converted back to local references before the call to {\tt go}
returns. Several error conditions can occur during the execution of
{\tt go} -- the remote host could be unreachable, the remote host may
be down, the interpreter implementing the execution-engine might have
died, the remote site may not have all the local classes that this
program might need while executing on that site. An exception is
returned for each of these error conditions along with an error
message that provides additional information.

A new thread can be created by {\em rexec}'ing the {\tt main} method
of a class existing on a remote engine. The arguments for the new
thread are copied and moved to the remote site. Remote thread creation
is non-blocking and the calling thread resumes immediately after the
{\tt main} method call is sent to the remote engine. Note that remote
calls to the {\tt main} method are blocking -- the calling thread is
suspended till the execution of {\tt main} completes. Remote thread
creation is different from thread migration as it creates a new thread
at the remote site that runs concurrently with the original thread;
thread migration moves the current thread to the remote site without
creating a new thread.  Concurrent threads communicate using calls to
shared objects. The thread creating a new thread can share objects
with the child by passing it references to these objects as arguments
to {\tt main}.

Sumatra does not automatically move code for either the {\tt go}
operation or the {\tt rexec} operation.  The {\tt downloadClass}
method can be used to download the class template for an object (and
the associated bytecode) to an execution-engine. This allows programs
to control their environment to some extent -- for each class, a
program can decide whether to use its own implementation or an
implementation provided by the host on which it is executing.
Downloaded class-templates are cached; the {\tt ClassLoader} checks
this cache before checking the local file system.  The host, however,
retains complete control over which classes can be downloaded and can
reject {\tt downloadClass} operations that attempt to replace critical
classes. 


\subsubsection{Implementation}
\label{sec:engine-impl}

We use a mechanism similar to the familiar {\tt inetd} daemon to
manage the creation of execution-engines. A master daemon runs on all
machines that allow creation of execution-engines and listens on a
well-known socket. When a execution-engine creation request is
received, it creates a new interpreter process which attempts to bind
to specified socket. If either of these operations fail, an exception
value is returned. The master daemon is also responsible for checking
authorization. Currently, no authorization checks have been
implemented. Execution-engine operations are currently implemented in
C (as native methods). Figure~\ref{fig:comm} shows the interface for
the Sumatra communication package which is used to implement these
operations. Note that except {\tt saveState()}, none of these {\em
need} to be in C -- {\tt invokeRPC()} can be implemented using Java
RMI~\cite{JavaRMI} and the others can be layered on other Java
classes. Java RMI was not used as it was not available when we were
implementing Sumatra. Since Java does not provide a user-level
primitive to serialize the stack, {\tt saveState()} has to be
implemented in C.

\begin{figure}
\tightboxit{
\begin{tabbing}
tttttttt\=tttttt\=ttt\=thdt\= \kill
\> {\tt public final class Engine \{ }\\
\>\>{\tt   public String hostname;}\\
\>\>{\tt        public int    port;}\\
\\
\>\>{\tt        public Engine(String hname,int portno); }\\
\>\>{\tt        public void downloadClass(String Classname);}\\
\>\>{\tt        public void go();}\\
\>\>{\tt        public native void rexec(String classname, String[] args);}\\
\> {\tt \}}\\
\end{tabbing}
}
\caption{The Sumatra engine interface}
\label{fig:engine}
\end{figure}


\begin{figure}
\tightboxit{
\begin{tabbing}
tttttttttttt\=tttt\=tdksksjtt\=tdjwfggfghghhhghhdt\= \kill
\> {\tt public final class Comm \{ }\\
\>\>   /* save state of current thread and transmit */\\
\>\>{\tt   public static native void saveState(String hostname, }\\
\>\>\>{\tt int portno);}\\
\>\>    /* invoke an rpc call, walks stack to get arguments  */\\
\>\>{\tt    public static native Object invokeRPC()}\\
\>\>\>{\tt                throws ObjectMovedException;}\\
\>\>   /* Download a class to an execution engine */\\
\>\>{\tt   public static native void downloadClass(String classname,}\\
\>\>\>{\tt  Engine engine);}\\
\>\>   /* Start an execution engine at a given machine */\\
\>\>{\tt   public static native void startEngine(String hostname, }\\
\>\>\>{\tt                                     Engine engine);}\\
\>\>   /* Find my current engine */\\
\>\>{\tt   public static native Engine myEngine();}\\
\> {\tt \}}\\
\end{tabbing}
}
\caption{The Sumatra communication package}
\label{fig:comm}
\end{figure}


\subsection{Object-groups}
\label{sec:objgroups}

There are three primary properties of object-groups. First, they are
aggregates. That is, they move only as a group. Second, they are
sticky. That is, they can only be moved by an explicit
move-object-group operation; they do not move with migrating
threads. Third, they are persistent. That is, they are not
garbage-collected. As long as an object is a member of some
object-group, it is spared by the garbage-collector.

Figure~\ref{fig:objgroup} presents the Sumatra object-group interface.
A string name can be associated, at creation time, with each
object-group. The name can be used to identify different
object-groups. Sumatra does not check for system-wide uniqueness of
this name. It does, however, check for local uniqueness -- attempts to
create an object-group with a name that is in-use raises an exception.

Objects can be dynamically added to or removed from an object-group
using the {\tt checkIn()} and {\tt checkOut()} methods respectively.
The {\tt moveTo()} method is used to move the object-group to a
different execution-engine. Membership of object-groups is explicit,
that is, every member of an object-group must be checked in
explicitly.  Also, {\tt moveTo()} does a shallow move -- only the
objects that have explicitly been checked in are moved. This is in
accordance with the {\em authority} requirement -- no communication
takes place without an explicit request.

Thread objects cannot be checked into an object-group. This
restriction is imposed as including a thread object in object-group
would require the thread to move along with the object-group.  This
causes two problems. First, this could cause migration of executing
threads at arbitrary points. Restricting migration to syntactically
marked program points has advantages (see~\cite{AcharyaS97Dynamic} for
one such advantage).  Second, since moving a thread could cause other
objects to be moved, it would violate the {\em authority} requirement.

During a {\tt moveTo()} operation, objects in an object-group are
automatically marshaled using type-information stored in their class
templates.  When an object-group is moved, all local references to
objects in the group (stack references and references from other
objects) are converted into {\em proxy references} which record the
new location of the object. Some objects, such as I/O objects, are
tightly bound to local resources and cannot be moved.  References to
such objects are reset and must be reinitialized at the new site.
Several error conditions are possible -- the remote host could be
unreachable (due to a network partition), the remote host could be
down, the remote execution-engine may not contain the classes
corresponding to the objects being moved and the remote
execution-engine may already contain an object-group with the same
name. An exception is returned for each of these error condition along
with an error message that provides additional information.

Method invocations on proxy objects are translated into calls at the
remote site.  Type information stored in class-templates is used to
achieve remote-procedure-call functionality without a stub
compiler. Exceptions generated at the called site are forwarded to the
caller. In accordance with the {\em authority} requirement, Sumatra
does not automatically track mobile objects. Requesting a remote
method invocation on an object that is no longer at the called site
results in an {\em object-moved} exception at the calling site. To
facilitate application-level tracking, the exception carries with it a
forwarding address. The caller can handle the exception as it deems
fit (e.g., re-issue the request to the new location, migrate to the
new location, raise a further exception and so on).  This mechanism
allows applications to locate mobile objects lazily, paying the cost
of tracking only if they need to. It also allows applications to abort
tracking if need be and pursue an alternative course of action.  

It is also possible for an application to specify that an exception
should be delivered on all methods invoked on proxy objects.  This
allows the application to avoid communication if it so desires.
Furthermore, Sumatra does not support direct access to instance
variables of remote objects.  Such variables should be accessed
through remote invocation of access methods that return the values of
instance variables.  This is in keeping with the {\em authority}
requirement.

\subsubsection{Implementation}
\label{sec:objgroup-impl}

Object-groups are implemented as lists of objects. Each\\
execution-engine has a list of such lists.  The object-group
operations have been implemented in C (as native methods). The main
reason for this is the interaction between object-groups and the
garbage-collector. Objects that belong to an object-group are not
garbage-collected. It is, however, possible to re-implement this
feature portably using a keep-alive thread that lives forever and
exists for the sole purpose of keeping these objects from being
garbage-collected.


\begin{figure}
\tightboxit{
\begin{tabbing}
tttttttttttttttttttt\=tttt\=tttt\=thdt\= \kill
\>{\tt public final class ObjGroup \{}\\
\>\>{\tt        public String groupname;}\\
\\
\>\>{\tt        public ObjGroup (String name);}\\
\>\>{\tt        public native void  checkIn(Object object);}\\
\>\>{\tt        public native void  checkOut(Object object);}\\
\>\>{\tt        public native void  moveTo(Engine engine);}\\
\>\>{\tt        public native Engine location();}\\
\>\>{\tt        private native void  internGroup(); }\\
\> {\tt \}}\\
\end{tabbing}
}
\caption{The Sumatra object-group interface}
\label{fig:objgroup}
\end{figure}


\subsection{Resources and asynchronous events}
\label{sec:resource}

Sumatra provides two resource-monitoring interfaces, one that allows
the application to poll the state of a particular resource and the
other that allows it to request asynchronous notification when the
availability of a resource goes outside a specified threshold.
Figure~\ref{fig:monitoring-1} presents the interface for on-demand
monitoring.  A program indicates its interest in a resource by
creating a {\tt Resource} object with appropriate arguments. If the
resource to be monitored is associated with a single host (e.g. server
load), the last argument is ignored.  A request for continuous
monitoring consists of a pair of values defining a range of resource
availability (e.g. upper and lower bounds on bandwidth), an upper
bound on frequency of polling and a function to be called when the
resource availability is no longer within the range (this uses the
interface for asynchronous events which will be presented
shortly). 

Sumatra provides a single interface (shown in
Figure~\ref{fig:monitoring-2}) for handling all kinds of asynchronous
events.  To create an event handler, the user creates a new subclass
of the {\tt Callback} class, creates an object of that class and
registers it using {\tt System.registerCallback()}. The event handler
function is specified by overriding the {\tt callback()} method. Note
that this method has been declared to be {\tt abstract} and every
subclass has to override it.  After the {\tt callback()} method
completes, control returns to the point where the interruption
happened. If a call to {\tt go()} is embedded in a {\tt callback()}
method, execution resumes on the target host.

Three kind of asynchronous events are currently supported --
asynchronous notification for continuous resource monitoring,
violation of resource-restrictions and external events in the form of
Unix signals. The first class of events require upper and lower bounds
and a frequency bound which are specified using the {\tt setLow()},
{\tt setHigh()} and {\tt setFreq()} methods. The second class of
events requires only an upper bound.  The third class of events, Unix
signals, require only the signal number.  Signals can be used by the
external environment (the operating system or some other
administrative process) to inform the application about urgent
asynchronous events, in particular resource revocation.  Using a
handler, the application can take appropriate action including moving
away from the current execution site.

\subsubsection{Implementation}
\label{sec:res-impl}

Sumatra assumes that a local resource monitor is available which can
be queried for information about the environment.  When an application
makes a monitoring request, Sumatra forwards the request to the local
resource monitor.  If the monitor does not support the requested
operation, or if no monitor is available, an exception is raised.  The
communication between Sumatra and the monitor is via a well-known
shared memory segment. This allows Sumatra to cheaply acquire rapidly
changing resource information. On-demand monitoring requests are
implemented by directly reading this segment. For on-demand
monitoring, default polling frequency of the resource monitor is used.

Event handlers need to be registered explicitly. Depending on the type
of the event being registered, different structures are set up inside
the interpreter. Asynchronous notification for continuous monitoring
is implemented using Unix signals. The interpreter uses the same
handler for all Unix signals - the identity of the signal is saved and
an event is queued. The event queue is checked between every Java
virtual machine instruction.

Resource-restrictions are implemented within the interpreter by
activating counters that keep track of resource usage. These counters
have been collected in a single module which is currently implemented
in C\@. It is easy to re-implement this entirely within Java/Sumatra,
much along the same lines as the Security Manager. Currently, only the
``memory-use'' restriction is implemented.

\begin{figure}
\tightboxit{
\begin{tabbing}
tttttttttttttt\=tttt\=tttt\=tt\= \kill
\> {\tt public final  class Resource \{}\\
\>\> {\tt public Resource(String type, String from, String to);}\\
\>\> {\tt public int read\_value();}\\
\> {\tt \}}\\
\end{tabbing}
}
\caption{The Sumatra on-demand monitoring interface.}
\label{fig:monitoring-1}
\end{figure}

\begin{figure}
\tightboxit{
\begin{tabbing}
ttttttttttttt\=tttt\=tttt\=tt\= \kill
\> {\tt public abstract class Callback  \{}\\
\>\> /* these callbacks are used for asynchronous notification */\\
\>\> {\tt public Callback(Resource rsid);}\\
\>\>     /* these callbacks are used for external event handlers */ \\
\>\> {\tt  public Callback(int type);}\\
\>\> {\tt        public void SetLow(int low);}\\
\>\> {\tt        public void SetHigh(int high);}\\
\>\> {\tt        public void SetFreq(int freq);}\\
\>\> {\tt        public Resource get\_resource();}\\
\>\> {\tt        abstract public void callback(); }\\
\> {\tt \}}\\
\end{tabbing}
}
\caption{The Sumatra event handling interface.}
\label{fig:monitoring-2}
\end{figure}


\subsection{Discussion}
\label{sec:sumatra-discussion}

In this section, we discuss the ways in which the design of Sumatra is
influenced by the {\em agility} and {\em authority} requirements.

\begin{itemize}
\item {\bf Calls to {\tt go()} can occur anywhere:} In 
      particular, they can be embedded inside callback methods.  This
      allows Sumatra programs to react quickly to asynchronous events
      like revocation of allocated bandwidth/memory or qualitative
      changes in network connectivity (e.g. on a mobile host). The
      design alternative to using a {\tt go()}-like interface was to
      allow migration only at function-call boundaries. To ensure a
      prompt response to asynchronous events, this would require one
      of two things: (1) either the language automatically captures
      the continuation at the point at which an event occurs and makes
      it available for use within the corresponding callback method ;
      or (2) the programmer emulates this functionality by writing a
      large number of functions that represent continuations at
      different points in the programs. In both cases, the language
      has to allow the use of a continuation-passing style. We
      believe, the {\tt go()}-based interface provided by Sumatra is
      both simpler to use and easier to implement.
\item {\bf All remote accesses can be trapped:} there are two 
      parts to this. First, Sumatra does not allow programs to access
      instance variables of remote objects. Attempts to do result in
      an exception being raised. Second, programs can request that an
      exception be raised for all methods invoked on proxy
      objects. Using both these features, a program can turn off all
      communication if it so desires.
\item {\bf Objects moved and tracked in groups:} this allows application
      to control the granularity of both operations.
\item {\bf Object-groups are tracked lazily:} and under application-control.
       Requesting a remote method invocation on an object that is no
       longer at the called site results in an {\em object-moved}
       exception at the calling site. The exception carries with it a
       forwarding address which allows the application to continue
       tracking by re-issuing the request to the new location or to
       abort tracking if it so desires. 
\item {\bf Membership of object-groups is explicit:} only those 
      objects that have been explicitly added to an object-group
      belong to it.  If an object with one or more component objects
      is added to an object-group, only the top-level object becomes a
      member of the object-group. To include the component objects in
      the object-group, each of them has to be explicitly added. This
      allows the application to precisely control which objects are
      moved and when. 
\item {\bf No distributed garbage-collection:} Sumatra provides no 
      distributed \\ garbage-collection. It is the responsibility of the
      application to ensure that objects that are no longer needed are
      removed from object-groups. Note that Sumatra does provide local
      garbage-collection. 
\item {\bf Object-groups are sticky:} Objects that belong to an object-group 
      move only when that object-group is moved. When a thread
      migrates, Sumatra transports with it all local objects that are
      referenced by the stack but do not belong to any object-group. 
\item {\bf Life-time of an object can be controlled:} Object-groups and their 
      member objects are not subject to garbage-collection. This is
      implemented by adding the list of object groups to the set of
      roots used by the garbage-collector. This ensures that all
      objects that belong to object-groups (and their transitive
      closure) are spared by the garbage-collector. This allows
      applications to temporarily deposit data at intermediate
      execution sites as well as to extend existing servers by
      downloading objects that extend current functionality.  It is
      legal for an object to belong to multiple object-groups.
      Membership of multiple object-groups can be useful in situations
      where there are multiple reasons to keep an object alive. This
      can, however, lead to unexpected communication if one of the
      object-groups moves and takes the object with it. It is,
      however, the programmer's responsibility to ensure that multiple
      membership does not lead to unwanted communication.
\item {\bf Location of an object can be queried:} Sumatra allows programs
      to query for the location of an object. This allows programs to
      selectively control communication -- if desired, a program can
      allow critical remote operations while restricting all other
      remote operations.
\item {\bf Memory use can be bounded:} Sumatra programs can specify 
      an upper bound on memory allocation. Attempts to allocate memory
      beyond this bound results in an asynchronous event which be
      handled if so desired.
\end{itemize}

\subsection{Example} 
\label{example}

In this section, we describe the use of the Sumatra programming model
using an example scenario involving an extension of the Virtual
Microscope.  In addition, we describe our initial experience with a
multi-site image composition application in Section~\ref{sec:app}.  As
described in Section~\ref{intro}, the Virtual Microscope project
involves the development of software that makes it possible to carry
out a realistic digital emulation of a high power light microscope.
The Virtual Microscope will use large datasets obtained by scanning
sets of full microscope slides at high power.  We estimate that 10-100
Gbytes will be required to store data that pertains to a single
biopsy.

The Virtual Microscope will be used both for directly emulating a
microscope and as part of various data processing chains used to
analyze biopsy data. Analysis of a large biopsy dataset will typically
be carried out in at least two phases. During the first phase,
portions of the dataset with potentially significant pathology will be
isolated. During the next phase, analysis will be carried out on the
database subset. The second phase of analysis will further
characterize the pathology with the goal of providing information that
can guide treatment or contribute prognostic information.  Such
characterization can include quantification of cancer related changes
to cell morphology or micro-anatomy, computations can also be carried
out to characterize the three dimensional topology of the
malignancy. In cases that involve characterization of three
dimensional topologies, we will have a third phase of processing
associated with image rendering.

Processing can be carried out on the server, on the client or the
processing can be split between the client and server. In most cases,
we would expect to carry out the first phase of processing on the
server.  However if the network is very fast, the server very loaded,
and if the client has the necessary storage, it could be advantageous
to carry out the first phase of processing on the client.  The
placement of the second phase of the analysis would involve
considerations that would depend on: (1) the ratio of computation to
communication associated with the second phase, (2) the computational
rates of the server and client (taking into account resource
contention due to other demands placed on the server or client), (3)
network performance, and (4) storage availability on the client and
server. Any necessary rendering could take place at the server, client
or on a third specialized graphics platform. Considerations would be
analogous to those described above.

In Figure ~\ref{fig:sumatraexample}, we show code for a simplified
scenario in which we have made an a-priori decision to carry out the
first phase of processing on the server. In the simplified example
below, we base our placement decision only on the amount of data that
would have to be moved from server to client and on the current
estimate of bandwidth between server and client.

\begin{figure}
\tightboxit{
\begin{tabbing}
tttttttttttt\=tt\=tt\=tt\= \kill
\>.....  \\
\>{\tt   filter\_object =  new find\_pathology\_filter();                    }\\
\>{\tt   characterize\_object =  new characterize\_anatomy(filter\_object);     }\\
\>{\tt   myengine    =   System.comm.myEngine();                   }\\
\>{\tt                                                            }\\
\>       // Create a engine at the microscope sensor database site.             \\ 
\>{\tt   remote\_engine = new Engine("microscope.gov");                }\\
\>       // Indicate interest in monitoring bw to xrays.gov       \\
\>{\tt   bw = new Resource("bandwidth",myengine.hostname,"microscope.gov");}\\
\>       // Send the find\_pathology\_filter class to the remote engine      \\
\>{\tt     remote\_engine.downloadClass("find\_pathology\_filter");           }\\
\>       // Create a new object group.                             \\
\>{\tt     objgroup = new ObjGroup("find\_pathology\_filter\_group");        }\\
\>       // Add the find\_pathology\_filter\_object to the object group       \\
\>{\tt     objgroup.checkIn(filter\_object);                      }\\
\>       // Move the object group to the database site             \\
\>{\tt     objgroup.moveTo(remote\_engine);                       }\\
\>                                                                 \\
\>         // a remote method call finds portion of a biopsy dataset with interesting pathology\\ 
\>{\tt     size = filter\_object.phase1(db,"John Smith Biopsy 13");           }\\
\>                                                                 \\
\>         // compute estimated time to transfer data           \\
\>{\tt     transfer\_time = size * bw.read\_value();              }\\
\>         // Does it take too long?                              \\
\>{\tt     if (transfer\_time > threshold)  \{                   }\\ 
\>\>            // Migrate thread, process sensor data and return.      \\ 
\>\>{\tt        remote\_engine.go();                              }\\
\>\>{\tt        result = characterize\_object.define\_anatomy();         }\\
\>\>{\tt        myengine.go();                                    }\\
\>{\tt     \}                                                     }\\ 
\>{\tt    else  \{                                                }\\
\>\>        // the estimated transfer time is small enough         \\
\>\>        // Fetch them and process locally.                     \\
\>\>{\tt        objgroup.moveTo(myengine);                        }\\
\>\>{\tt        result = characterize\_object.define\_anatomy();         }\\
\>{\tt     \}                                                     }\\ 
\>                                                                 \\
\>       // display result locally                                 \\
\>{\tt   System.display(result);                                  }\\
\end{tabbing}
}
\caption{Excerpt of a Sumatra program that migrates depending on 
the time required to transfer data.}
\label{fig:sumatraexample}
\end{figure}

Sumatra assumes that a local resource monitor is available which can
be queried for information about the environment. In the next section,
we describe Komodo, a distributed resource monitor which can provide
information for Sumatra applications.

