
\documentclass{article}
\usepackage{psfig}

%% Margins etc
%\setlength{\oddsidemargin}{0.50truein}
%\setlength{\evensidemargin}{0.50truein}
%\setlength{\topmargin}{-0.5truein}
%\setlength{\textwidth}{5.5truein}
%\setlength{\textheight}{9.0truein}

\title{{\bf MPI for Java} \\
       Java Grande Technical Report TR-03-98}

\author{Bryan Carpenter, Vladimir Getov, Glenn Judd, Tony Skjellum,
        Geoffrey Fox}

\date{1 November, 1998}

\begin{document}

\maketitle

\section{Introduction and background}

A basic prerequisite for parallel programming is a good message passing
API. Java comes with various ready-made packages for communication, 
notably an easy-to-use interface to BSD sockets, and the Remote Method 
Invocation (RMI) mechanism. Interesting as these interfaces are, it is 
questionable whether parallel programmers will find them especially 
convenient. Sockets and remote procedure calls have been around for about 
as long as parallel computing has been fashionable, and neither of them 
has been popular in that field. Both communication models are optimized 
for client-server programming, whereas the parallel computing world
is mainly concerned with ``symmetric'' communication, occurring in groups 
of interacting peers.

This symmetric model of communication is captured in the successful 
Message Passing Interface (MPI) standard, established a few years 
ago~\cite{MPI-1}. MPI directly supports the Single Program Multiple Data 
(SPMD) model of parallel computing, wherein a group of processes cooperate
by executing identical program images on local data values. Reliable 
point-to-point communication is provided through a shared, group-wide 
communicator, instead of socket pairs. MPI allows numerous blocking, 
non-blocking, buffered or synchronous communication modes. It also 
provides a library of true collective operations (broadcast is the most 
trivial example). An extended standard, MPI-2~\cite{MPI-2}, allows for 
dynamic process creation and access to memory in remote processes. 

The MPI standard document thus far has provided language-independent 
specification as well as language-specific (C and Fortran) 
bindings~\cite{MPI-1}. While the MPI-2 release of the standard added a 
C++ binding~\cite{MPI-2}, no Java binding has been offered or is planned
by the MPI Forum. With the evident success of Java as a programming 
language, and its inevitable use in connection with parallel as well as 
distributed computing, the absence of a well-designed language-specific 
binding for message-passing with Java will lead to divergent, 
non-portable practices. Therefore, a standard specification is urgently 
needed to enable the development of portable Java Grande applications 
using MPI. 


\section{Current status}

There are several known efforts towards the design of early MPI
interfaces for Java with three fully functional but different
Java-MPI implementations -- mpiJava, JavaMPI, and MPIJ. The design of
mpiJava is based on the use of native methods to build a wrapper to
existing MPI library (MPICH). A comparable approach has been followed in
the development of JavaMPI, but the JavaMPI wrappers were automatically
generated by a special-purpose code generator. A large subset of MPI 
is implemented in pure Java within the DOGMA system for Java-based 
parallel programming. MPI Software Technology, Inc. have also announced
that there is a commercial effort under way to develop a message-passing
framework and parallel support environment for Java called JMPI
\cite{CrawfordIII97}. Some of these ``proof-of-concept''
implementations have been available for more than a year with successful
ports on clusters of workstations running Solaris or Windows NT, as well
as IBM SP2, SGI Origin-2000, Fujitsu AP3000, and Hitachi SR2201 parallel
platforms.

\subsection{The mpiJava wrapper}

The mpiJava API is modelled as closely as practical on the C++ binding
defined in the MPI 2.0 standard but currently supports only the MPI 1.1 
subset~\cite{NPACJavaMPI97a}. A number of changes to argument lists are
forced by the restriction that arguments cannot be passed by reference 
in Java. In general outputs of mpiJava methods come through the result 
value of the function. In many cases MPI functions return more than one
value. This is dealt with in mpiJava in various ways. Sometimes an MPI 
function initializes some elements in an array and also returns a count 
of the number of elements modified. In Java we typically return an array
result, omitting the count. The count can be obtained subsequently from 
the length member of the array. Sometimes an MPI function initializes an
object conditionally and returns a separate flag to say if the operation
succeeded. In Java we return an object handle which is null if the 
operation fails.  Occasionally an extra field is added to an existing 
MPI class to hold extra results -- for example the Status class has an 
extra field, index, initialized by functions like Waitany. Rarely none 
of these methods work and we resort to defining auxilliary classes to 
hold multiple results from a particular function. In another change to 
C++, we often omit array size arguments, because they can be picked up 
within the wrapper by reading the length member of the array argument. 

As a result of these changes mpiJava argument lists are often more
concise than the corresponding C or C++ argument lists. 

Normally in mpiJava, MPI destructors are called by the Java finalize
method for the class. This is invoked automatically by the Java garbage
collector. For most classes, therefore, no binding of the 
MPI\_class\_FREE function appears in the Java API. Exceptions are Comm 
and Request, which do have explicit Free members. In those cases the MPI
operation could have observable side-effects (beyond simply freeing 
resources), so their execution is left under direct control of the 
programmer. 

\subsection{Automatic generation of MPI wrappers}

In principle,
the binding of existing MPI library to Java using JNI amounts to either
dynamically linking the library to the Java virtual machine, or linking
the library to the object code produced by a stand-alone Java compiler. 
Complications stem from the fact that Java data formats are in general 
different from those of C. Java implementations will have to use JNI 
which allows C functions to access Java data and perform format 
conversion if necessary. Such an interface is fairly convenient for 
writing {\em new} C code to be called from Java, but is not adequate for
linking {\em existing} native code.

Clearly an additional interface layer must be written in order to bind a
legacy library to Java. A large library like MPI has over a hundred
exported functions, therefore it is preferable to automate the creation
of the additional interface layer. The {\em Java-to-C interface
generator} (JCI) \cite{MintchevGetov97c} takes as input a header file 
containing the C function prototypes of the native library. It outputs 
a number of files comprising the additional interface:
a file of C stub-functions;
files of Java class and native method declarations;
shell scripts for doing the compilation and linking.
The JCI tool generates a C stub-function and a Java native method
declaration for each exported function of the MPI library.
Every C stub-function takes arguments whose types correspond directly
to those of the Java native method, and converts the arguments into the
form expected by the C library function.

As the JavaMPI bindings have been generated automatically from
the C prototypes of MPI functions, they are very close to the C binding.
However, there is nothing to prevent 
from parting with the C--style binding and adopting a Java-style
object--oriented approach by grouping MPI functions into a hierarchy of
classes.

\subsection{Pure Java implementation of MPI}
MPIJ is a completely Java-based implementation of MPI which runs as part
of the Distributed Object Group Metacomputing Architecture (DOGMA) 
system. Being closely based on the C++ binding, MPIJ implements a large 
subset of MPI functionality including point-to-piont communication (all
modes), intracommunicator operations, groups, user-defined reduction 
operations. Noteable capabilities that are not yet implemented include 
process topologies, caching, intercommunicators, and user-defined 
datatypes (these are arguably needed for legacy code only).

MPIJ communication uses native marshaling of primitive Java types. 
This technique allows MPIJ to achieve communication speeds comparable
to, and frequently exceeding that, of native MPI implementations. (Java
communication speed would be greatly increased if native marshaling were
a core Java function.)

Current MPIJ work involves porting native marshalling to platforms 
other than Win32, investigation of standard libraries (e.g. BLAS) 
for improved performance, and porting to an improved version of DOGMA.

\section{Proposed joint efforts}

\subsection{Java-MPI -- API Specification}
The MPI standard is explicitly object-based. The C and Fortran bindings
rely on ``opaque objects'' that can be manipulated only by acquiring 
object handles from constructor functions, and passing the handles to 
suitable functions in the library. The C++ binding specified in the 
MPI-2 standard collects these objects into suitable class hierarchies 
and defines most of the library functions as class member functions. 
The Java-MPI API specification follows this model, lifting the structure
of its class hierarchy directly from the C++ binding.  

The immediate infrastructure to be provided builds literally on the MPI-1
infrastructure offered by the MPI Forum, together with language bindings
motivated by the MPI-2 forum's C++ bindings.  The purpose of that effort
is to provide an immediate, ad hoc standardization for common message
passing programs in Java, as well as to provide a basis for conversion
between C, C++, Fortran77, and Java.  Eventually, support for aspects of
MPI-2 belong under this category as well, particularly dynamic process
management, but not necessarily all of MPI-2, given its spartan
implementation in the non-Java space.

\begin{figure}[hbt]
\centerline{\psfig
{figure=/home/vsg/latex/work/javagrande/mpiJava.eps,width=4.5in}}
\caption{Principal classes of Java-MPI\label{hierarchy}}
\end{figure}

The major classes of Java-MPI are illustrated in Figure~\ref{hierarchy}.
The class {\tt MPI} only has static members. It acts as a module
containing global services, such as initialization of MPI, and many
global constants including the default communicator {\tt COMM\_WORLD}.
The most important class in the package is the communicator class {\tt
Comm}.  All communication functions in Java-MPI are members of {\tt
Comm} or its subclasses.  As usual in MPI, a communicator stands for a
``collective object'' logically shared by a group of processors. The 
processes communicate, typically by addressing messages to their peers 
through the common communicator. Another class that is important for the
Java-MPI specification is the {\tt Datatype} class. This describes the 
type of the elements in the message buffers passed to send, receive, and
all other communication functions.

The complete draft Java-MPI API specification is included as an appendix
to this document.

\subsection{Advanced Message Passing Infrastructure for Java}
The present effort's purpose is to offer a first principles study of how
to present MPI-like services to Java programs, in an upward compatible
fashion.  The purposes are twofold: performance and portability.  For
performance, we seek to take advantage of what has been learned since
MPI-1 and MPI-2 were finalized, or which were ignored in MPI
standardization for various reasons.  The study will, for instance, draw
on the body of knowledge just recently completed within the MPI/RT
Forum,
which strives to enhance both real-time and performance of message
passing programs.  From MPI/RT, we will at least glean design hints 
concerning channel abstractions, and the more direct use of 
object-oriented design for message passing than was done in MPI-1 or 
MPI-2 (despite existence of C++ bindings). Additionally, a fundamental
look at data marshalling and unmarshalling in the Java context will be 
undertaken, and preference for Java-natural mechanisms and policies will
be attempted. Along the lines of portability, a detachment from legacy 
implementations of Java over existing native methods will be emphasized,
while also considering the possibility of layering the messaging 
middleware over standard transports and other Java-compliant middleware
(such as CORBA). In a sense, the middleware developed at this level 
should offer a choice of a performance or generality emphasis, while 
always supporting portability. A policy/opportunity to support aspects 
of real-time and fault detection/fault-aware programs will be studied 
and standardized insofar as possible, again drawing on the concepts 
learned in the MPI/RT activity, and also drawing on experience from 
distributed computing real-time activities. The validity of this type of
messaging middleware in the embedded and real-time Java application 
spaces will also be considered.

\subsection{Further Actions}
The ACG currently consideres to release a Request for Proposals for the
Java-MPI efforts to cover:
\begin{itemize}
\item Standard Java-MPI API Specification
\item Java-MPI wrapper publicly available on the Web
\item Intelligent generator of wrappers to legacy MPI libraries
\item Pure Java MPI implementation 
\item Test suite
\item Java-MPI Benchmarks
\end{itemize}

\begin{thebibliography}{99}

\bibitem{NPACJavaMPI97a}
B.~Carpenter, G.~Fox, G.~Zhang, and X.~Li.
A draft {J}ava binding for {MPI}.
http://www.npac.syr.edu/projects/pcrc/HPJava/mpiJava.html, November, 1997.

\bibitem{Java-MPI}
B.~Carpenter, G.~Fox, V.~Getov, G.~Judd, and A.~Skjellum.
Java-MPI -- API Specification (Draft Position Document). Java Grande
TR-03-98, November 1998.

\bibitem{CrawfordIII97}
G.~{Crawford III}, Y.~Dandass, and A.~Skjellum.
The {JMPI} commercial message passing environment and specification:
Requirements, design, motivations, strategies, and target users.
http://www.mpi-softtech.com/publications/JMPI\_121797.html, MSTI,
December 1997.

\bibitem{MintchevGetov97c}
S.~Mintchev and V.~Getov.
Towards portable message passing in Java: Binding MPI. In: M. Bubak, J.
Dongarra, J. Wa\'sniewski (Eds.), ``Recent Advances in PVM and MPI'',
{\em Lecture Notes in Computer Science}, {\bf 1332}, 135--142, Springer
Verlag, 1997.

\bibitem{MPI-1}
Message Passing Interface Forum. {MPI}: A message-passing interface 
standard. {\em International Journal of Supercomputer Applications}, 
{\bf 8}(3/4), 1994.

\bibitem{MPI-2}
Message Passing Interface Forum. {MPI-2}: Extensions to the
Message Passing Interface,
http://www.mpi-forum.org/docs/mpi-20-html/mpi2-report.html, July 1997.

\end{thebibliography}

\end{document}

