\chapter{Introduction}

The initial growth of interest in Java was driven
largely by its role in bringing a new generation of interactive
pages to the World Wide Web. Undoubtedly various features of the
language---compactness, byte-code portability, security, and so
on---make it particularly attractive as an implementation language for
applets embedded in Web pages and servlets embedded in servers. 
But it was always clear that the ambitions of the Java development
team go well beyond enhancing the functionality of HTML documents. 
According to \cite{JavaWhitePaper}
\begin{quote}
``Java is designed to meet the challenges of application development in
the context of heterogeneous, network-wide distributed environments.
Paramount among these challenges is secure delivery of applications
that consume the minimum of system resources, can run on any hardware
and software platform, and can be extended dynamically.''
\end{quote}
and in fact recent development have emphasized server-side applications 
over applets.

Several of the concerns of the Java development are mirrored in
developments in the High Performance Computing world over a number of
years.  A decade ago the focus of interest in the parallel computing
community was on parallel hardware.  A parallel computer was typically
built from specialized processors integrated through a proprietary
high-performance communication switch.  If the machine also had to be
programmed in a proprietary language, that was an acceptable price for
the benefits of using a supercomputer.  This attitude was not
sustainable as one parallel architecture gave way to another, and the
cost of porting software became exorbitant.  For several years now,
portability across platforms had been a central concern in parallel
computing \cite{p4Book,p4Article,MPIFORUM,HPFFORUM,HPFBOOK}.

More fundamentally, the assumption that high performance computing will
be done primarily on specialized supercomputers is questioned increasingly.
Rapid strides in performance and connectivity of ordinary workstations
and PCs make it look equally possible that the future of parallel
computing will be on local area networks, or even the Internet
\cite{PVMBook,WebComputing}. 


With Java positioned to become a standard programming language on the
Internet, and scientific parallel processing edging towards
network-based computation, it is natural to ask how these two
technologies will interact. How suitable is Java for scientific
computing, and do lessons from research in parallel computing have
implications for the development of Java?

Popular acclaim aside, there are some reasons to think that Java may
be a good language for scientific and parallel programming.
\begin{itemize}
	\item {Java is a descendant of C++. C and C++ are used
	increasingly in scientific programming: they are already used
	almost universally by {\it implementers} of parallel libraries and
	compilers. In recent years numerous variations on the theme of C++
	for parallel computing have appeared. See, for example
	\cite{ATLAS,CHANDY,DONGARRA,FINK,GRIMSHAW,WILSON} }	

	\item {Java omits various features of C and C++ that are
	considered ``difficult''---notably, pointers. Poor compiler analysis
	has often been blamed on these features. The inference is that
	Java, like Fortran, may be a suitable source language for highly
	optimizing compilers.
        An interesting series of papers from IBM
        \cite{Moreira98A,Moreira98B,Wu99}, confirmed that the
        current generation of Java virtual machines have rather poor
        performance on Fortran-like, array-intensive computations, but went on
        to demonstrate how to apply aggressive optimizations in Java compilers
        to obtain performance competitive with Fortran.
        In a recent paper \cite{Moreira99} they described a case study
        involving a data mining application.  Using the
        experimental IBM HPCJ Java compiler they reported obtaining over 90\%
        of the performance of Fortran.} 

	\item {Java comes with builtin multi-threading. On a shared memory
	platform independent threads may be scheduled on different
	processors by a suitable runtime. Even without physical shared
	memory, muliti-threading can be very convenient in explicit
	message-passing styles of parallel programming \cite{OCCAM}.}

\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Issues}

%See more in HPspmd:Data Parallel SPMD Programming Models from Fortran to Java

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Approaches to Parallelism in Java}
\label{PJAVA}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Java already supports concurrency through the thread mechanism and
monitor synchronization built into the language. In this dissertation
we are more interested in truly {\it distributed parallel}
computation, involving multiple CPUs. Such parallelism could be
introduced into Java in a number of ways.

It could be achieved through automatic parallelization of sequential
code, but it is unclear why this easier for Java than for other
languages. Alternatively, the Java virtual machine for a shared memory
multiprocessor can schedule the threads of a multi-threaded Java
program on different processors. For computation on a network (or
distributed memory computer) realistic options include language
extensions or directives akin to HPF, or provision of
libraries---class libraries---to support task parallelism or data
parallelism.

A popular approach in C++ has been to defer language extensions and
concentrate on class library support for parallel programming. The
similarities between the two languages suggest this may be a fruitful
avenue in Java too. The success of this analogy is by no means
automatic, however. Features such as {\it templates} and user-defined
{\it operator overloading} make the C++ language more customizable
than Java. In C++ library-defined types can be used on an identical
footing to primitive types---{\it inline} methods mean they can be
almost as efficient as primitive types. Less importantly, but
conveniently, new control constructs can often be simulated in C or
C++ through use of macros. On grounds, presumably, of simplicity and
transparency many of these features have been omitted from Java.

Such caveats notwithstanding, this dissertation will concentrate on class
libraries rather than language extensions.  We have been working with
class libraries implemented in the standard Java development
environment.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\subsection{Running a Parallel Java Program}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Another open question is how multiple interacting Java tasks are to
be initiated.  Conventionally in network computing, instructions to execute
a particular user task are sent to a specialized daemon or secure
server running on the target host.  Alternatively standard
operating system daemons ({\em rshd}, {\em rexecd}, \ldots) can be used
to the same effect.  Java brings some options of its own.  One
is to use standard or enhanced Web servers to execute Java byte-codes.
This approach is predicated on the existence of cooperative servers,
probably running suitable CGI scripts.  Another option is to exploit
Java-enabled browsers.  Applets downloaded from a particular server can
perform computations and return the results to that server.  In any
case the substantive improvement over conventional network computing is
that the byte-codes can be downloaded on the fly to target hosts, even
in heterogeneous networks.  Conventionally, it was often necessary to
install and compile an application on each target host before starting
the task remotely.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Communication in Java}
\label{COMMJAVA}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The standard Java API provides a simplified interface to Internet
sockets and RMI, an object-oriented Java version of Remote Procedure
Call (RPC).

The Java socket interface hides much of the ugly detail involved in
socket-programming at the traditional C/Unix level. The {\it java.net}
interfaces provides less flexibility than using the system calls
directly. On the other hand, Java's built-in support for threads adds
some flexibility in scheduling communications that is missing from
raw C. 

We will give an example of socket programming in Section
\ref{sec:sockets}, but traditionally this has not been a popular paradigm
in the parallel processing world.  Scientific programmers have
expected to program inter-process communication at a higher level, if
at all.  More successful schemes include
%Traditionally socket programming has not been a popular paradigm in
%the parallel processing world, where more successful schemes include
 
\begin{itemize}
	\item {Message-passing through language-level support \cite{OCCAM,
	FortranM} or higher-level library interfaces \cite{MPIFORUM}.}

	\item {Data parallelism, which we take to mean the style of
	programming in which parallelism is achieved through operations on
	distributed arrays, with synchronization typically limited to bulk
	synchronization occurring naturally through collective array
	operations.} 

	\item {Communication through shared memory or shared objects,
	involving some more or less intricate mechanism for inter-process
	synchronization.} 
\end{itemize}

As observed in the previous section, communication through true shared
memory is already implicit in the Java thread model. 
Communication through remote objects (i.e RMI) is undoubtedly a natural and
important paradigm in Java, especially for access to remote services.
%However, RMI is too restrictive and its overhead for dealing with network
%problems is too verbose for the cluster setting. 
However, it suffers from a significant startup time and marshaling/unmarshaling
of data is more costly than in the socket method.

An orthogonal issue is whether the high-level libraries used to
implement these paradigms (presuming class-library implementations)
should be written in Java on top of the standard API, or whether
they should be implemented as direct interfaces to native code.  


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Research Objectives}

The main goal of the research presented in this dissertation is to
investigate issues pertinent to high performance programming in Java, 
design Java interfaces to High Performance Computing softwares, 
and research implementation issues associated with their development. 
%During this resarch, primary objectives include
The dissertation has the following objectives:
\begin{itemize}

\item
Investigate various issues and options for parallel programming in Java---data
parallelism and message passing libraries.
These include discussing motivations for introducing HPJava, an HPspmd 
programming model.
It allows programs to combine data parallel code and SPMD library
calls directly. We present sample code that directly calls to MPI from
within the data parallel program.

\item
Develop mpiJava, an object-oriented Java interface to MPI. 
This could be implemented by ``native methods'' wrappers around some 
pre-existing MPI implementation.
However, interfacing Java to MPI is not always trivial. 
The new {\em native} thread feature in JDK 1.2 might be a solution to
eliminate conflicts between the Java runtime and the interrupt
mechanisms used in the MPI implementations.
Also, the syntax of mpiJava should be easy to understand and use,
thus making it relatively simple for programmers with either a Java or
Scientific background to take up. 

\item
Incorporate automatic object serialization in mpiJava. 
In order to support derived datatypes or handle a multidimensional
array, the Java object serialization model for marshalling general
communication data in MPI-like APIs should be introduced, because the
standard MPI approach describing user-defined types does not map very
naturally into Java.
In addition, we should evaluate overheads introduced by object
serialization in mpiJava, then consider optimized methods that reduce
serialization overheads.

\item
Develop real applications written in mpiJava and evaluate their performance.
It is also interesting to compare the results of benchmarks written in Java
with equivalent C or Fortran code.
Applications include graphical visualization parallel Potts model simulation
with GUI using Java AWT.  

\item
Publically release mpiJava on a Web site. This includes complete source,
makefiles, configuration scripts, compiled libraries for WMPI, test
codes (based on the IBM MPI test suite), example applications, javadoc
documentation, and installation and usage notes.

\end{itemize}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Organization of the Thesis}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The outline of this thesis is as follows.
Chapter \ref{chap:grande} provides a general overview of the Java
Grande Forum, its activities, and processes.
The Java Grande Forum is a group looking at
general issues for large-scale computing in Java.
We also discuss Java for scientific parallel programming issues and
look at work related to Java Grande in Chapter \ref{chap:grande}. 
Chapter \ref{chap:hpjava} introduce the design of our HPJava framework
and its core features. HPJava is a programming language extended from
Java to support SPMD parallel programming style.
In Chapter \ref{chp:mpiJava}, we discuss the design of the
mpiJava API, an object-oriented Java interface to MPI, and
the issues associated with its development. 
Chapter \ref{chap:objseri} describes use of the Java object
serialization model for marshalling communication data in Java
bindings of MPI. 
Chapter \ref{chap:applications} presents applications and analyzes
the performance of Ising model simulation implemented using mpiJava, 
and compared with Fortran and C versions. 
Finally, Chapter \ref{chap:conclusion} contains conclusion and the
directions for future work. 
An earlier version of some of the material in this dissertation has
been presented previously in the publications listed in Publications
section on page \pageref{sec:publications}.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Acknowledgements}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

This work was supported in part by the National Science Foundation
Division of Advanced Computational Infrastructure and Research, contract
number 9872125.


