\documentclass[twoside,leqno,11pt]{article}
\usepackage{proc2e}
\input{psfig}

\begin{document}
\cleardoublepage
\pagestyle{headings}

\title{Web Technologies for Collaborative Visualization and
Simulation}

\author{
Lukasz Beca
\and
Gang Cheng
\and
Geoffrey C. Fox
\and 
Tomasz Jurga
\and
Konrad Oszewsk
\and
Marek Podgorny
\and
Krzysztof Walczak\thanks{Syracuse University, Northeast Parallel
Architectures Center, 111 College Place, Syracuse, New York  13244,
gcf@npac.syr.edu, http://www.npac.syr.edu}
}
\date{}
\maketitle

\pagenumbering{arabic}

\begin{abstract} 
Web technologies---in particular linked Java servers and
clients---allow new dynamic collaborative environments linking people
and computers (collaboratories). We describe the architecture of a system, TANGOsim,
that combines a Java collaborative environment with an executive
providing general message filters, and an event driven simulator.  The
initial application is to command and control, but we describe how
this approach can also be used in other areas, such as health care,
scientific visualization, and (distance) education.
\end{abstract}

\section{Introduction}

We believe that current Web technology offers remarkable opportunities
for sophisticated new environments linking people and computers.  This
includes traditional video conferencing, telemedicine, command and
control, crisis management, distance education, computational steering
and visualization, and ``human in the loop'' distributed discrete event
simulations.  The basic technology is a distributed set of Java
servers and clients that communicate.  The Java clients are applets
launched from traditional browsers.  The cited environments can be
implemented as a set of communicating processes with the servers
coordinating traffic between the clients.  In a traditional
collaborative environment, the servers run session managers that log
new users, and ensure a common world view so that applications spawned
on one client (white boards, lessons for distance education, etc.) are
shared on all others.  This simple sharing of information is already a
powerful environment, but we can generalize this by allowing client
messages to be processed by arbitrary filters before they are passed on
to other clients.  This allows different users to receive different
views of the same data---in a complex command and control environment,
it is clearly necessary to present the different decision makers with
the different detail needed for their responsibility.  Further, we can
drive these filters by dynamic scripts that allow one to include
simulated users for training and modelling applications.  Note that the
``users'' (more accurately, clients) generating and receiving messages
can be either people or computers.  For instance computational steering
in its simplest case corresponds to a single person interacting with a
single computer.

% \begin{figure}
% \vspace{1.0in}
% \centerline{\psfig{figure=fig1.tango.ps,clip=true,width=4.5in}}
% \caption{}
% \label{fig1.tango}
% \end{figure}
 
The overall system is now a set of Java servers implementing a
(distributed) event driven simulator that accepts events from either
scripts or clients corresponding to computers or people.

In this paper, we describe in Section~\ref{sec2} the design and
implementation of our experimental system TANGOsim, shown in
Figure~\ref{fig2.tango}, which has successfully implemented the above
concept.  We have already used it in one significant command and
control application, described in Section~\ref{sec3}, and illustrated
in Figure~\ref{fig2.tango}.

\begin{figure}
\centerline{\psfig{figure=fig2.tango.ps,clip=true,width=4.5in}}
\caption{The Command and Control TANGO Application}
\label{fig2.tango}
\end{figure}

In Section~\ref{sec4}, we speculated on its use in other areas
including computational steering distance education and health care.

\section{TANGOsim: Design Goals, Architecture, and Implementation}
\label{sec2}

Computer mediated collaboratory systems have a relatively long history
(for an extensive review, see \cite{Schooler:96}). Despite the extensive
body of research, collaboratory systems are hardly out of the
laboratory. Academic implementations have been evolving together with
the enabling multimedia and networking technologies:  only recently the
sophistication of the affordable desktop technology reached the level
making multimedia collaboratory a viable prospect.  The commercial
collaboratory implementations are exceedingly primitive:  video
teleconferencing, shared whiteboard with limited graphical capability,
and a textual chat tool represent industrial state of the art, as
exemplified by Insoft's Communique!, SGI's InPerson, or Intel's
ProShare products. It is not very surprising that this type of
collaboratory is perceived by some as an activity in search of a need
\cite{Grudin:90}.

We have designed and implemented a web-based collaboratory system,
code-named TANGO, which is optimized for system flexibility and
capability of integrating existing applications.  TANGO is not the only
web-based collaboratory system.  There are multiple ongoing projects in
the area of Web based distributed systems.  Some of them aim at
development of a platform for implementation of distributed software in
Java \cite{NCSA:96}. Some of the others build software simplifying
development of such distributed applications \cite{CIP:96}. The
majority of other projects concentrate on development of single-purpose
distributed or collaborative systems themselves. Designing TANGO we
tried to build a generalized, extensible, Web based collaboratory
system capable of creation of diverse shareable information spaces. The
system provides support for coordinated but independent views of
related information streams, with capability for high-end visualization
and interactive manipulation of multimedia information. TANGO extends
Web paradigm to the domain of collaborative computing and well beyond
the concept of the chat, shared whiteboard, and replicated, identical
instances of simple generic applications.  

\subsection{System Requirements}

We have based design and implementation of the TANGO system on the
following requirements:  integrate both standalone and Web-based
applications by providing a uniform interface to communicate with their
instances on remote machines; allow execution and control
of collaborative applications from the Internet browser environment;
provide means for session control (user authentication, starting and
ending sessions, tracing participants activities, changing user
privileges); enable integration of existing applications written in any
programming language, assuming socket communication as the only
necessary communication mechanism; provide ability to download certain
applications across the network through the use of Java applet
technology, allowing main parts of the system to be automatically
distributed across the Internet; provide logging mechanism, so all user
activities may be stored in the persistent form in a database and
retraced if necessary; support definition of compatible message and
application classes to enable multiple, task oriented views of the
information streams.

\subsection{System Overview}

The main functionality provided by the system consists of the
following elements:
\begin{itemize}
\item Session management

Each shared application defines a session that has a single {\em
master\/} user---which status is transferable.

\item Communication between collaborating applications (data and event
distribution)

There are {\em control\/} messages that are invisible to user and {\em
application\/} messages, which are means by which user applications
exchange state, and that are not interpreted by system (unless they
interact via TANGOsim filters).

\item User authentication and authorization

\item Event logging in back-end database

Each time a control or application specific message passes the server,
this fact is recorded in the database together with the date, exact
time, and sender information.  These data can then be accessed, and
the whole system activity can be replayed.

Event logging is a basic feature supporting {\em asynchronous
collaboration}.  Note that the term asynchronous collaboration is
frequently used to describe e-mail, and other messaging systems.  In
the collaboratory context, asynchronous collaboration denotes ability
of the collaboratory system to replay, and even summarize at a desired
level the entire collaboratory sessions.
\end{itemize}

\subsection{System Architecture}

\subsubsection{System Elements}

Figure~\ref{fig3.tango} presents the global architecture of the
system. The system consists of the following components:
\begin{figure}
\centerline{\psfig{figure=fig3.tango.ps,clip=true,width=3.5in}}
\caption{Global Architecture of the TANGO System}
\label{fig3.tango}
\end{figure}

\begin{itemize}
\item {\em Local daemon's\/} main task
is to maintain two way communication between user applications,
applets and the central server and to lunch local applications.
Daemon is implemented as a plug-in in Netscape browser. 

\item {\em Central server\/} is the main communication element. All
local daemons communicate with the central server. The server maintains
the system state data.  Its main tasks are routing of messages among
applications participating in each session and recording all events in
a database.  
\end{itemize}

All user applications which run as standalone programs are called {\em
local applications}. Local application may be written in any
programming language. All local applications communicate with the local
daemon by the use of sockets. The daemon is responsible for starting
these applications and routing messages to and from applications.

{\em Java applets\/} are also user applications but written in Java
language, downloaded through the network from an HTTP server, and
executed in Netscape environment.  Communication between Java applets
and central server is also maintained by the local daemons. Java applets
communicate with local daemon by calling its method functions.

{\em Control application\/} is a specialized application which acts as
an interface to the user.  Control application is used to launch
applications locally or remotely, create and join existing sessions,
exit applications, etc. This application allows also logging into the
system. User interface to the control application depends on his/her
privileges, giving a user access to only these features that he/she is
allowed to use. The Control application is customizable.

\subsubsection{TANGOsim Extension}

An example of TANGO architecture flexibility and extensibility is
TANGOsim---an discrete event simulator. A multithreaded simulation
engine implementing virtual time can be driven by either a scripting
language or interactively controlled by a user via Simulation
Controller. The engine and the controller are implemented as a Java
application and a Java applet, respectively. The simulation engine can
create messages for any application compatible with TANGO system, to
create and control sessions, and to realize scenarios in which the
course of action depends on user input. TANGOsim is a prime example of
the system capability to go beyond the collaboratory model of
``cooperating twins.''  TANGOsim also supports filtering messages, and
linkage of messages between different applications sessions.

\subsection{Implementation Issues}

Ideally, for portability reasons, one we would like to implement all
system components using only Java. This approach turns out, however, to
be currently infeasible for several reasons.  The most important are
security constraints imposed on Java..  Using pure Java model we would
have to create a client-server model in which all communication and
data distribution are located in one host. This idea had to be rejected
because of its inflexibility.

Performance and advanced functionality is another constraint. At least
for now, high-end 3D visualization or video encoding is impractical in
Java. This may be a transient situation.  Digital signatures, Java
performance improvements, and a growing body of advanced libraries
written in Java may allow us to in the future to implement the entire
system in this language. For the time being, we felt that the benefit
of building a flexible, powerful system with rich functionality
outweighs aesthetically architectural concerns.

{\bf Daemon Implementation}

We use the LiveConnect technology introduced by Netscape Inc. in its
Navigator version 3.0. This product provides means for Netscape
components such as Java applets, JavaScript scripts and plug-ins to
talk to each other. The {\em daemon\/} has been implemented as a
plug-in. Using LiveConnect mechanisms, each applet residing in the same
page with the plug-in has access to the plug-in handle. Message passing
between plug-in and the applet is achieved by calling appropriate
methods of each other. In case of standalone applications, the
connection to the daemon is implemented with use of standard socket
communication.

Plug-ins are usually written in C (the API provided by Netscape uses
this language). To improve the porting procedure of the only platform
dependent component of TANGO we associate Java code with the native C
code and implement large parts of the plug-in code in Java. The only
the part of the plug-in implemented is C is the set of routines used to
start standalone applications and to provide intranode
applet-application message passing. In this way we have minimized
porting effort to an absolute minimum.  TANGO is currently available on
both Unix and Wintel platforms.

{\bf Application API}

Porting application to TANGO is facilitated by a simple API, and
libraries currently available for C, C++, and Java.

At present, messages are represented by byte arrays.  It is
programmer's responsibility to convert these data properly.  Object
serialization and RMI will be used as soon as these technology is
available from JavaSoft.

{\bf Server Implementation}

The {\em central server\/} is a multithreaded standalone Java
application. The server keeps all the information about connections and
users in dynamic data structures and also stores them in the database.
In our implementation we use JDBC package and Oracle RDBMS.
\newpage

{\bf Videoteleconferencing}

For scalability reasons, the real time multimedia streams are not sent
via central server. Instead, a distributed architecture akin to the
Insoft's (currently Netscape) OpenDVE has been implemented. This
architecture supports multicast. Session control remains with the TANGO
session manager. Audio/video decode/playback capability is implemented
in Java for certain supported codecs.  Backend database repository and
playback capability for the audio/video streams is provided by a random
access video server and is a part of the global system session
archiving/retrieval facility.

\section{Application of TANGOsim to Command and Control}
\label{sec3}

Command and Control is the military description of a general real time
decision (or judgment) support environment involving a complex set of
people, datasets, and computational resources.  A critical
characteristic is the need to make the ``best possible,'' as opposed to
``optimal,'' choice.  In a  civilian context, crisis management
\cite{NRC:96a} has essentially similar needs.  However, most
application areas have a component that links computers, information,
and one or more people to make decisions.  Correspondingly, command
and control has system requirements that are generally applicable.

We are implementing a simple scenario corresponding to an unmanned
aircraft flying into U.S. airspace carrying a bacterial agent.  We
implement planning, execute (does one intercept or not?), and finally
response phases to health and life threatening consequences of the
downed aircraft.  This scenario involves military and civilian radar
sensors and personnel tracking aircraft; NORAD as military command in
execution stage; federal and state leaders at highest level (President
and Governor); weather simulation to assess intercept possibilities,
and in response stage, the dispersal of bacterial agents; FEMA as
civilian command in response stage; and finally, medical authorities
for expertise and treatment.

Let us see how some of the key capabilities of TANGOsim are used in
those various parts of this scenario.  Generally, digital video
conferencing, text-based chatboards, and shared white boards are used
by the participants to interact in a typical unstructured
collaborative fashion.

One of the simplest shared applications is the common information
source that, in our case, is a shared Web page, but in general, can be
information obtained from any database.  This is, for instance, used
by the tracking officer to search the Web for information on the
identified aircraft, and display characteristics and photos with other
participants.  Both the Web search process, and resultant pages are
shared.  Note this information is generated by the {\em master},
and broadcast to other participants.  We would use the filter
capability of TANGOsim to ensure that each participant only received
appropriate information.  For instance, the President and Governor
would be spared a lot of the detailed shared displays used by those
lower down the command chain.

More generally, one wishes to present a given object in different ways
to different participants.  Thus, the radar and weather officers could
use complex three-dimensional geographical information systems to
study the event in detail.  The expected weather and tracking data
would be presented to others less involved in those areas, as simpler
two-dimensional summaries.  This flexibility is easily implemented by
passing all status messages from the master for each application
through a filter that is controlled by a dynamic script.  Note that
one will need to select different masters for different applications,
and allow the ``master role'' be passed from one participant to
another as the collaboration evolves.  This capability is supported by
TANGOsim.

Training is an important component of many decision making processes,
including command and control, crisis management, and health care.
Here, the discrete event simulation capability of TANGOsim is critical
as it allows one to ``script'' (simulate) any of the participants, and
enable general training sessions focusing on any role.  Note that in
the simple TANGO collaborative mode, ``time'' is the real wall clock
time.  This mode will support scripting, which can be used to present
information at particular times to given clients.  For instance, this
is a valuable capability in the training mode for carousel-like
presentations.  In the more general simulation mode, one differentiates
real and virtual time as is traditional in discrete event simulators.

Command and control, and crisis management both make decisions about
events in the real world, and so particularly critical applications
are two- and three-dimensional geographical information (GIS) systems.
We believe that Web technology is excellent for GIS, and had started
development of these for education---the virtual field trip.  We
originally implemented our three-dimensional GIS in VRML (Virtual
Reality Modelling Language), but found this unacceptedly slow.  We
re-implemented the VRML version in terms of its Open Inventor basis,
which led to a much faster C++ application.  This we can still
integrate into TANGOsim which supports any client application
language.

As usual in GIS, these applications support overlays---in the case of
the three-dimensional GIS, these include weather simulations with
volume rendering of the clouds, and other weather phenomena.  We are
currently extending this to other three-dimensional fields, including
electromagnetic simulations.

\section{Other Applications of TANGOsim}
\label{sec4}

In the previous section, we highlighted some capabilities of TANGOsim
and described their use in our current command and control project.
Here, we will more speculatively discuss the role of this technology
in health care, computational steering, and distance education.

Health care has, like command and control, interesting training
applications.  Today, the latter could involve a mix of say residents
in training, experts, observers, and a set of test cases that form the
shared information space.  Clearly, TANGOsim would allow a distributed
implementation of this with the experts playing the role of ``anchor
desks'' in the crisis management notation.  An important
characteristic of this, and other applications is the number of
participants as the current TANGOsim involves a single centralized Java
server.  This is not a scalable model, and future systems will surely
involve a Web of cooperating servers.  Turning to team health care, we
see the need for both spontaneous collaboration between members of the
initial care team, and later asynchronous collaboration as specialists
are called in for later consultations.  A key feature of TANGOsim is
the ability to log and replay collaborative sessions.  This is
obviously of great potential value (and some legal concern) in health
care and other applications.  A key capability that must be added is a
notation so that initial sessions are abstracted by the initial
participants for the later experts.  We have some experience with this
in education where lengthy audio clips are available for elaboration
of summary presentations.  Of course, TANGOsim supports traditional
asynchronous collaboration, including multimedia mail.

In computational steering, TANGOsim is particularly suitable for team
activities, such as grand challenge simulations or access to a remote
instrument for a group of investigators.  The scripted feature of
TANGOsim could link to a resource allocation system, such as CODINE,
LSF, NQS, DQS or Condor
(http://nhse.cs.rice.edu/NHSEreview/96-1.html), which would activate
client applications when the resource was available.  In industrial
applications, such as vehicle design, many engineers may be involved
in a multidisciplinary optimization with different roles.  This
illustrates how one can use server filters to present each engineer
only those components of the linked computation that are of relevance
to them.  Inherent in TANGOsim is the use of Web technologies for
visualization.  As we have discussed
(http://www.npac.syr.edu/projects/javaforcse), we believe Java
visualization and data analysis ``wrappers'' will prove to be an
attractive way for users to interact with conventional or high
performance simulation servers.  We have also shown how one can emulate
and generalize popular visualization systems, such as AVS or Khoros
with the WebFlow concept, which can be naturally integrated with
TANGOsim, as both are based on Java servers.  Finally, we note that
the current NSF PACI program involves collaborative support for
distributed computation.  This is, again, an area where we expect Web
technologies to be very important.

Finally, we mention educational applications.  Many sites have
demonstrated the great value of the Web in disseminating
information---Cornell's Virtual Workshop is a good example.  However,
education involves both curricula material and a rich complex
interaction between teachers and students.  This involves both
synchronous (deliver lectures) and asynchronous (grade homework,
answer questions) collaboration.  We see that systems like TANGOsim
provide a Web framework for addressing the collaborative component of
education (the missing link?).  It is clear that we can provide far
more attractive features than traditional video based distance
education, but it will require many experiments to identify which
capabilities of Web collaborative systems are needed with what
functionalities.  These developments could be very important for the
education community because it is possible that successful deployment
of Web collaboration systems could alter the basic (business) model
for education.  One critical point is scaling---how many simulataneous
students can one and does one want to support in a single session.
Reliability and performance will also be critical---it is essential
that server interruptions will be very rare and that one minimizes
transmission delays.  Good caching of the basic information so that
the network is only used in real time for dynamic interaction seems
essential here.  We are currently investigating some of these issues
with some rather modest experiments.  It is important that many
different such trials be undertaken for although the general concept
is clear, it is not either clear how to implement the details, and how
important are either the general or particular capabilities.

\begin{thebibliography}{99}

\bibitem[CIP:96]{CIP:96} Caltech Infospheres Project (1996)
http://www.infospheres.caltech.edu/

\bibitem[Grudin:90]{Grudin:90} Grudin, J. (1990) Computer-supported
cooperative work: history and focus. IEEE Computer 27: 19 - 26

\bibitem[NCSA:96]{NCSA:96} NCSA Habanero Project (1996)
http://www.ncsa.uiuc.edu/SDG/Software/Habanero/

\bibitem[NRC:96a]{NRC:96a}
{Steering Committee}.
\newblock ``Computing and communications in the extreme: Research for crisis
  management and other applications,'' in {\em Workshop Series on High
  Performance Computing and Communications}. National Academy Press,
  Washington, D.C., 1996.
\newblock Computer Science and Telecommunications Board, Commission on Physical
  Sciences, Mathematics, and Applications.

\bibitem[Schooler:96]{Schooler:96} Schooler E.M. (1996) Conferencing and
Collaborative Computing.  Multimedia Systems 4: 210-225

\end{thebibliography}
\end{document}

