\documentstyle[psfig]{article}

\newenvironment{rationale}{\begin{list}{}{}
\item{\em Rationale.\hspace{0.1in}}}
{{\em (End of rationale.)}
\end{list}}

\newenvironment{implementors}{\begin{list}{}{}
\item{\em Advice to implementors.\hspace{0.1in}}}
{{\em (End of advice to implementors.)}
\end{list}}

\newenvironment{arguments}{\begin{tabular}{ll}}
{\end{tabular}\vspace{0.15in}\newline}

\newenvironment{argsNoSpace}{\begin{tabular}{ll}}
{\end{tabular}}

\begin{document}

\bibliographystyle{plain}
\pagestyle{plain}

\title{\vspace{1in}{\bf MPI for Java:} \\
Position Document and Draft API Specification}

\author{
Bryan Carpenter$^1$, Vladimir Getov$^2$, Glenn Judd$^3$, \\
Tony Skjellum$^4$ and Geoffrey Fox$^1$ \\ \\
$^1$NPAC, Syracuse University, Syracuse, USA \\
$^2$School of Computer Science, University of Westminster, London, UK \\
$^3$Computer Science Department, Brigham Young University, Provo, USA \\
$^4$MPI Software Technology, Inc., Starkville, USA \\ \\ \\ \\ \\ 
Technical Report JGF-TR-03 \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ 
Java Grande Forum \\ \\
November, 1998}

\date{}

\thispagestyle{empty}

\maketitle

\newpage

\tableofcontents

\newpage

\section{Introduction and background}

A basic prerequisite for parallel programming is a good message passing
API. Java comes with various ready-made packages for communication, 
notably an easy-to-use interface to BSD sockets, and the Remote Method 
Invocation (RMI) mechanism. Interesting as these interfaces are, it is 
questionable whether parallel programmers will find them especially 
convenient. Sockets and remote procedure calls have been around for about 
as long as parallel computing has been fashionable, and neither of them 
has been popular in that field. Both communication models are optimized 
for client-server programming, whereas the parallel computing world
is mainly concerned with ``symmetric'' communication, occurring in groups 
of interacting peers.

This symmetric model of communication is captured in the successful 
Message Passing Interface (MPI) standard, established a few years 
ago~\cite{MPI-1}. MPI directly supports the Single Program Multiple Data 
(SPMD) model of parallel computing, wherein a group of processes cooperate
by executing identical program images on local data values. Reliable 
point-to-point communication is provided through a shared, group-wide 
communicator, instead of socket pairs. MPI allows numerous blocking, 
non-blocking, buffered or synchronous communication modes. It also 
provides a library of true collective operations (broadcast is the most 
trivial example). An extended standard, MPI-2~\cite{MPI-2}, allows for 
dynamic process creation and access to memory in remote processes. 

The MPI standard document thus far has provided language-independent 
specification as well as language-specific (C and Fortran) 
bindings~\cite{MPI-1}. While the MPI-2 release of the standard added a 
C++ binding~\cite{MPI-2}, no Java binding has been offered or is planned
by the MPI Forum. With the evident success of Java as a programming 
language, and its inevitable use in connection with parallel as well as 
distributed computing, the absence of a well-designed language-specific 
binding for message-passing with Java will lead to divergent, 
non-portable practices. Therefore, a standard specification is urgently 
needed to enable the development of portable Java Grande applications 
using MPI. 


\section{Current status}

There are several known efforts towards the design of early MPI
interfaces for Java with three fully functional but different
Java-MPI implementations -- mpiJava, JavaMPI, and MPIJ. The design of
mpiJava is based on the use of native methods to build a wrapper to
existing MPI library (MPICH). A comparable approach has been followed in
the development of JavaMPI, but the JavaMPI wrappers were automatically
generated by a special-purpose code generator. A large subset of MPI 
is implemented in pure Java within the DOGMA system for Java-based 
parallel programming. MPI Software Technology, Inc. have also announced
that there is a commercial effort under way to develop a message-passing
framework and parallel support environment for Java called JMPI
\cite{CrawfordIII97}. Some of these ``proof-of-concept''
implementations have been available for more than a year with successful
ports on clusters of workstations running Solaris or Windows NT, as well
as IBM SP2, SGI Origin-2000, Fujitsu AP3000, and Hitachi SR2201 parallel
platforms.

\subsection{The mpiJava wrapper}

The mpiJava software implements a Java binding for MPI proposed
late in 1997 \cite{NPACJavaMPI97a}.  That proposal built on work on
Java wrappers for MPI started at NPAC about a year earlier.

The mpiJava API is modelled as closely as practical on the C++ binding
defined in the MPI 2.0 standard, specifically supporting the MPI 1.1
subset of that standard.  In some cases the extra runtime information
available in Java objects allows argument lists to be simplified
relative to the C++ binding.  In other cases restrictions of Java,
especially the fact that all arguments are passed by value in Java,
forces some changes to argument lists.  But in general mpiJava adheres
closely to earlier standards.

The implementation of mpiJava is through JNI wrappers to native MPI
software.  Interfacing Java to MPI is not always trivial.  We often see
low-level conflicts between the Java runtime and the interrupt
mechanisms used in MPI implementations.  The situation is improving as
JDK matures, and the mpiJava software now works reliably on top of
Solaris MPI implementations and various shared memory platforms.  A
port to Windows NT (based on WMPI) is available, and other ports are in
progress.

Other work in progess includes development of demonstrator
applications, and Java-specific extensions such as support for direct
communication of serializable objects.

\subsection{Automatic generation of MPI wrappers}

In principle,
the binding of existing MPI library to Java using JNI amounts to either
dynamically linking the library to the Java virtual machine, or linking
the library to the object code produced by a stand-alone Java compiler. 
Complications stem from the fact that Java data formats are in general 
different from those of C. Java implementations will have to use JNI 
which allows C functions to access Java data and perform format 
conversion if necessary. Such an interface is fairly convenient for 
writing {\em new} C code to be called from Java, but is not adequate for
linking {\em existing} native code.

Clearly an additional interface layer must be written in order to bind a
legacy library to Java. A large library like MPI has over a hundred
exported functions, therefore it is preferable to automate the creation
of the additional interface layer. The {\em Java-to-C interface
generator} (JCI) \cite{MintchevGetov97c} takes as input a header file 
containing the C function prototypes of the native library. It outputs 
a number of files comprising the additional interface:
a file of C stub-functions;
files of Java class and native method declarations;
shell scripts for doing the compilation and linking.
The JCI tool generates a C stub-function and a Java native method
declaration for each exported function of the MPI library.
Every C stub-function takes arguments whose types correspond directly
to those of the Java native method, and converts the arguments into the
form expected by the C library function.

As the JavaMPI bindings have been generated automatically from
the C prototypes of MPI functions, they are very close to the C binding.
However, there is nothing to prevent 
from parting with the C--style binding and adopting a Java-style
object--oriented approach by grouping MPI functions into a hierarchy of
classes.

\subsection{Pure Java implementation of MPI}
MPIJ is a completely Java-based implementation of MPI which runs as part
of the Distributed Object Group Metacomputing Architecture (DOGMA) 
system. Being closely based on the C++ binding, MPIJ implements a large 
subset of MPI functionality including point-to-piont communication (all
modes), intracommunicator operations, groups, user-defined reduction 
operations. Noteable capabilities that are not yet implemented include 
process topologies, caching, intercommunicators, and user-defined 
datatypes (these are arguably needed for legacy code only).

MPIJ communication uses native marshaling of primitive Java types. 
This technique allows MPIJ to achieve communication speeds comparable
to, and frequently exceeding that, of native MPI implementations. (Java
communication speed would be greatly increased if native marshaling were
a core Java function.)

Current MPIJ work involves porting native marshalling to platforms 
other than Win32, investigation of standard libraries (e.g. BLAS) 
for improved performance, and porting to an improved version of DOGMA.

\section{Proposed joint efforts}

\subsection{Java-MPI -- API Specification}
The MPI standard is explicitly object-based. The C and Fortran bindings
rely on ``opaque objects'' that can be manipulated only by acquiring 
object handles from constructor functions, and passing the handles to 
suitable functions in the library. The C++ binding specified in the 
MPI-2 standard collects these objects into suitable class hierarchies 
and defines most of the library functions as class member functions. 
The Java-MPI API specification follows this model, lifting the structure
of its class hierarchy directly from the C++ binding.  

The immediate infrastructure to be provided builds literally on the MPI-1
infrastructure offered by the MPI Forum, together with language bindings
motivated by the MPI-2 forum's C++ bindings.  The purpose of that effort
is to provide an immediate, ad hoc standardization for common message
passing programs in Java, as well as to provide a basis for conversion
between C, C++, Fortran77, and Java.  Eventually, support for aspects of
MPI-2 belong under this category as well, particularly dynamic process
management, but not necessarily all of MPI-2, given its spartan
implementation in the non-Java space.

\begin{figure}[hbt]
\centerline{\psfig
{figure=mpiJava.eps,width=4.5in}}
\caption{Principal classes of Java-MPI\label{hierarchy}}
\end{figure}

The major classes of Java-MPI are illustrated in Figure~\ref{hierarchy}.
The class {\tt MPI} only has static members. It acts as a module
containing global services, such as initialization of MPI, and many
global constants including the default communicator {\tt COMM\_WORLD}.
The most important class in the package is the communicator class {\tt
Comm}.  All communication functions in Java-MPI are members of {\tt
Comm} or its subclasses.  As usual in MPI, a communicator stands for a
``collective object'' logically shared by a group of processors. The 
processes communicate, typically by addressing messages to their peers 
through the common communicator. Another class that is important for the
Java-MPI specification is the {\tt Datatype} class. This describes the 
type of the elements in the message buffers passed to send, receive, and
all other communication functions.

The complete draft Java-MPI API specification is included as an appendix
to this document.

\subsection{Advanced Message Passing Infrastructure for Java}
The present effort's purpose is to offer a first principles study of how
to present MPI-like services to Java programs, in an upward compatible
fashion.  The purposes are twofold: performance and portability.  For
performance, we seek to take advantage of what has been learned since
MPI-1 and MPI-2 were finalized, or which were ignored in MPI
standardization for various reasons.  The study will, for instance, draw
on the body of knowledge just recently completed within the MPI/RT
Forum,
which strives to enhance both real-time and performance of message
passing programs.  From MPI/RT, we will at least glean design hints 
concerning channel abstractions, and the more direct use of 
object-oriented design for message passing than was done in MPI-1 or 
MPI-2 (despite existence of C++ bindings). Additionally, a fundamental
look at data marshalling and unmarshalling in the Java context will be 
undertaken, and preference for Java-natural mechanisms and policies will
be attempted. Along the lines of portability, a detachment from legacy 
implementations of Java over existing native methods will be emphasized,
while also considering the possibility of layering the messaging 
middleware over standard transports and other Java-compliant middleware
(such as CORBA). In a sense, the middleware developed at this level 
should offer a choice of a performance or generality emphasis, while 
always supporting portability. A policy/opportunity to support aspects 
of real-time and fault detection/fault-aware programs will be studied 
and standardized insofar as possible, again drawing on the concepts 
learned in the MPI/RT activity, and also drawing on experience from 
distributed computing real-time activities. The validity of this type of
messaging middleware in the embedded and real-time Java application 
spaces will also be considered.

\subsection{Further Actions}
The ACG currently consideres to release a Request for Proposals for the
Java-MPI efforts to cover:
\begin{itemize}
\item Standard Java-MPI API Specification
\item Java-MPI wrapper publicly available on the Web
\item Intelligent generator of wrappers to legacy MPI libraries
\item Pure Java MPI implementation 
\item Test suite
\item Java-MPI Benchmarks
\end{itemize}

\newpage

\appendix

\section{A Draft Java binding for MPI}

This appendix outlines a Java language binding for MPI 1.1. It has been
adapted from the proposal in reference \cite{NPACJavaMPI97a},
incorporating especially suggestions and comments from Vladimir Getov
and Glen Judd.  It is presented here as a basis for further discussion
and comment.

This is not a standalone specification of the behaviour of MPI---it is
meant to be read in conjunction with the MPI standard document.
Subsections are laid out in the same way as in the standard document,
to allow cross-referencing.  Where the Java binding makes no
significant change to a particular section of the standard document, we
will just note here that there are no special issues for the Java
binding.  This does not mean that the corresponding section
of the standard is irrelevant to the Java binding---it
may mean it is 100\% relevant!
Where practical the current proposals are modelled on the MPI C++
interface defined in the MPI standard version 2.0.

Changes to the mpiJava interface:
\begin{itemize}
\item
Name conventions have been shifted to bring them into line with
Sun's Java code conventions \cite{codeconv}.
\item
The {\tt MPI.OBJECT} basic type has been added.
\item
The public fields of {\tt Status} have been replaced by inquiry
methods.
\item
The MPI exception classes are no longer specified to be
subclasses of {\tt IOException}.
\item
The {\tt Request} method {\tt is\_null} has been replaced by
{\tt isVoid}, and the constant {\tt MPI.REQUEST\_NULL} is removed
in favour of providing a default constructor for {\tt Request}.
\item
Derived type constructors {\tt contiguous}, {\tt vector}, {\tt
hvector}, {\tt indexed}, {\tt hind\-exed} become non-static methods on
{\em oldtype}.
\item
A new variant of {\tt Comm.pack} has been added.
\item
The bindings of {\em MPI\_ATTR\_PUT} and {\em MPI\_ATTR\_DELETE}
have been removed.
\item
The {\tt translateRanks} member of {\tt group} becomes a non-static
method on {\em group2}.
\item
The interface to {\tt Cartcomm.dimsCreate} has been corrected.
\end{itemize}
Controversies and open questions:
\begin{itemize}
\item
It is unclear whether the Java interface should support
MPI derived data types.  A proposal for a Java-compatible
subset of derived types is included in this document, but 
deleting it could simplify the API significantly.
\item
It has been suggested that many of the communication operations
should be overloaded to provide simplified variants that omit arguments
such as {\tt offset}, {\tt count} (possibly {\tt datatype}).
This suggestion is not included in the current proposal, but it could
be added later if there is general support.  The obvious
counter-argument is that it significantly increases the total number of
methods in the API.
\item
It has been suggested that some specific support for multidimensional
arrays may be desirable.  In the current proposal, communicating
multidimensional arrays depends on Java object serialization, which
might be a performance bottleneck.  A plausible position is to
wait and see what happens in Java regarding multidimensional
arrays and efficient object serialization before complicating this API.
\item
It has been suggested that the specification of {\tt UserFunction} could
be altered to improve type security.  This suggestion isn't incorporated
in the current draft, but perhaps it should be.
\end{itemize}
\newpage

\subsection{Introduction to MPI}

Evidently, this document adds Java to the C and Fortran bindings
defined in the MPI standard.  Otherwise no special issues for the Java
binding.


\newpage

\subsection{MPI Terms and Conventions}

\subsubsection{Document Notation}

No special issues for Java binding.

\subsubsection{Procedure Specification}

In general we use {\em italicized} names to refer to entities in the MPI
language independent procedure definitions, and {\tt typewriter} font for
concrete Java entities.

As a rule Java argument names will be the
same as the corresponding language independent names.  In non-static
methods of {\tt Comm}, {\tt Status}, {\tt Request}, {\tt
Datatype}, {\tt Op} or {\tt Group} (and subclasses), the class instance
generally stands for the argument called {\em comm}, {\em status}, {\em
request}, {\em datatype}, {\em op} or {\em group}, respectively in the
language independent procedure definition\footnote{Exceptions to this
rule include the derived datatype construction methods on {\tt Datatype}
and the {\tt translateRank} method on {\tt Group}.}.  Names may be altered to
bring them into line with Sun's Java code conventions, as discussed in
section \ref{naming}.  For example, underscores in variable names are
typically removed in favour of capitalizing internal words.

\subsubsection{Semantic terms}

No special issues for Java binding.

\subsubsection{Data types}

Opaque objects will be presented as Java objects.
This introduces the
option of simplifying the user's task in managing these objects.  MPI
destructors can be absorbed into Java object destructors, which are
called automatically by the Java garbage collector.  We adopt this
strategy as the general rule.  Explicit calls to MPI destructor
functions are typically omitted from the Java user interface.  An
exception is made for the {\tt Comm} classes.  {\em MPI\_COMM\_\-FREE}
is a collective operation, so the user must ensure that calls are made
at consistent times by all processors involved---the call can't be left
to the vagaries of the garbage collector.

\begin{implementors}
For JNI-based ``wrapper'' implementations based on the C binding of MPI, one
should ensure that all MPI functions, including the {\tt
MPI\_\ldots\_free} object destructors, are called {\em before} {\tt
MPI\_Finalize} is called.  Because invocation of Java destructors
is under the control of the garbage collector there is no guarantee
that all Java-level {\tt finalize} methods will be called before the
explicit call to the Java-level {\tt MPI.finish}.  A possible solution is
to maintain a global count of the number of outstanding {\tt
MPI\_\ldots\_free} calls.  This is initialized to 1 by {\tt MPI.init}
and incremented by relevant constructors.  All relevant Java {\tt
finalize} methods and {\tt MPI.finish} should decrement this
counter, and be prepared to call the physical {\tt MPI\_Finalize}
function when and only when the count falls to zero.
\end{implementors}

\subsubsection{Language Binding\label{binding}}

\paragraph{Naming Conventions\label{naming}}
All MPI classes belong to the package {\tt mpi}.
Conventions for capitalization, etc, in class and member names
generally follow the recommendations of Sun's Java code conventions
\cite{codeconv}.  
Class names are in mixed case with the first letter of each internal
word capitalized.  Method and ordinary variable names are in mixed
case, with the first letter lowercase.  Constant variables are all
uppercase with words separated by underscores (``\_'').

\begin{rationale}
In general these conventions are consistent with the naming conventions
of the MPI 2.0 C++ standard.  Notable exceptions include the
use of lower case for the first letters of method names,
and avoidance of underscore in variable names.
\end{rationale}

\paragraph{Restrictions on {\em struct} derived type.}
Some options allowed for {\em derived data types} in the C and Fortran
binding are deleted in the proposed Java binding.  The Java VM does not
incorporate a concept of a global linear address space.  Passing physical
addresses to data type definitions is not allowed.  The use of the {\em
MPI\_TYPE\_STRUCT} datatype constructor is also restricted in a way that
makes it impossible to send mixed {\em basic datatypes} in a single
message.  Since, however, the set of basic datatypes recognised by MPI
is extended to include serializable Java {\em objects}, this
should not be a serious restriction in practice.

\paragraph{Multidimensional arrays and offsets.}
The C and Fortran languages define a straightforward mapping
(or ``sequence association'') between their multidimensional
arrays and equivalent one-dimensional arrays.  So in C or Fortran a
multidimensional array passed as a message buffer argument is first
interpreted as a one-dimensional array with the same element type as
the original multidimensional array.  Offsets in the buffer (such as
offsets occuring in derived data types) are then interpretted in terms of
the effective one-dimensional array (or---equivalent up to a constant
factor---in terms of physical memory).  In Java the relationship between
multidimensional arrays and one dimensional arrays is different.
An ``$n$-dimensional array'' is equivalent to a one-dimensional array
of $(n - 1)$-dimensional arrays.  In the proposed Java
binding, message buffers are always one-dimensional arrays.
The element type {\em may} be an object, which {\em may} have array type.
Hence multidimensional arrays can appear as message buffers, but the
interpretation is subtly different.  In distinction to the C or Fortran
case {\em offsets in multidimensional message buffers are always
interpretted as offsets in the outermost one-dimensional array}.

\paragraph{Start of message buffer.}
C and Fortran both have devices for treating a section of an array,
offset from the beginning of the array, as if it was an array in its
own right.  Java doesn't have any such mechanism.  To provide the same
flexibility, an {\tt offset} parameter is associated with any buffer
argument.  This defines the position of the first actual buffer element
in the Java array.

\paragraph{Error codes.}
Unlike the C and Fortran interfaces, the Java interfaces to MPI
calls will not return explicit error codes.  The Java exception
mechanism will be used to report errors.
\begin{rationale}
The exception mechanism is very widely
used by Java libraries.  It is inconvenient to use up the single
return value of a Java function with an error code.  (Java doesn't allow
function arguments to be passed by reference, so returning multiple
values tends to be more clumsy than in other languages.)
\end{rationale}

\paragraph{Multiple return values.}
A few functions in the MPI interface return multiple values, even after
the error code is eliminated.  This is dealt with in the proposed
binding in various ways.  Sometimes an MPI function initializes some
elements in an array and also returns a count of the number of elements
modified.  In Java we typically return an array result, omitting the
count.  The count can be obtained subsequently from the {\tt length}
member of the array.  Sometimes an MPI function initializes an object
conditionally and returns a separate flag to say if the operation
succeeded.  In Java we typically return an object reference which is
{\tt null} if the operation fails.  Occasionally extra internal state
is added to an existing MPI class to hold extra results---for example
the {\tt Status} class has extra state initialized by functions like
{\tt Waitany} to hold the {\em index} value.  Rarely none of these
methods work and we resort to defining auxilliary classes to hold
multiple results from a particular function.

\paragraph{Array count arguments.}
The proposed binding often omits array size arguments, because they can
be picked up within the function by reading the {\tt length} member of
the array argument.  A major exception is for message buffers, where
an explicit count is always given.
\begin{rationale}
In the proposed binding, message buffers have explicit {\tt offset} and
{\tt count} arguments whereas other kinds of array argument typically
do not.  Message buffers aside, typical array arguments to MPI
functions (eg, vectors of request structures) are small arrays.  If
subsections of these must be passed to an MPI function, the sections
can be copied to smaller arrays at little cost.  In contrast message
buffers are typically large and copying them is expensive, so it is
worthwhile to pass the extra arguments.   Also, if derived data types
are being used, the required value of the {\tt count} argument is
always different to the buffer length.
\end{rationale}

\paragraph{Concurrent access to arrays.}
\begin{implementors}
In JNI-based wrapper implementations it may be
necessary to impose some non-interference rules for concurrent read
and write operations on arrays.  When an array is passed to an
MPI method such as a send or receive operation, the wrapper code will
probably extract a pointer to the contents of the array using a JNI
{\tt Get{\ldots}ArrayElements} routine.  If the garbage collector {\em does
not} support ``pinning'' (temporarily disabling run-time relocation of
data for specific arrays---see \cite{JNIBook} for more
discussion), the pointer returned by this {\tt Get} function may be to
a temporary copy of the elements.  The copy will be written back to the
true Java array when a subsequent call to {\tt
Release{\ldots}ArrayElements} is made.  If two operations involving the
same array are active concurrently, this copy-back may result in
failure to save modifications made by one or more of the concurrent
calls.

%Operations may occur concurrently through explicit use of threads, {\em
%or through use of non-blocking communication operations}.  So far as
%use of non-blocking communication operations is concerned, the
%restriction could be relaxed at the price of having the wrapper always
%explicitly copy buffers using {\em MPI\_PACK} and {\em MPI\_UNPACK}.
%Of course this is an undesirable overhead.  If concurrency arises
%through use of threads, there is no obvious solution to the problem.
Such an implementation may have to enforce a safety rule such as:  {\em
when several MPI send or receive (etc) operations are active
concurrently, if any one of those operations writes to a particular
array, none of the other operations must read or write {\em any
portion} of that array.}

If the garbage collector supports pinning, this problem does not arise.
\end{implementors}

\subsubsection{Processes}

No special issues for Java binding.

\subsubsection{Error Handling}

As explained in section \ref{binding}, the Java methods do not return
error codes.  The Java exceptions thrown instead are defined in section
\ref{errorCodes}.

\newpage

\subsection{Point-to-Point Communication}

\subsubsection{Introduction}

In general the Java binding of point-to-point communication operations will
realize the MPI functions as methods of the {\tt Comm} class.
The basic point-to-point communication operations are {\em send} and
{\em receive}.  Their use is illustrated in the example below.

\small
\begin{verbatim}
  import mpi.* ;

  class Hello {
    static public void main(String[] args) {
      MPI.init(args) ;

      int myrank = MPI.COMM_WORLD.rank() ;
      if(myrank == 0) {
        char [] message = "Hello, there".toCharArray() ;
        MPI.COMM_WORLD.send(message, 0, message.length, MPI.CHAR, 1, 99) ;
      }
      else {
        char [] message = new char [20] ;
        MPI.COMM_WORLD.recv(message, 0, 20, MPI.CHAR, 0, 99) ;
        System.out.println("received:" + new String(message) + ":") ;
      }

      MPI.finish();
    }
  }
\end{verbatim}
\normalsize

\subsubsection{Blocking Send and Receive operations\label{sec:blocking}}

\begin{verbatim}

void Comm.send(Object buf, int offset, int count,
               Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag
\end{arguments}
Blocking send operation.
Java binding of the MPI operation {\em MPI\_SEND}. 
The data part of the message consists of a sequence of
{\tt count} values, each of the type indicated by {\tt datatype}.
The actual argument associated with {\tt buf} must be an
array.  The value {\tt offset} is a subscript in
this array, defining the position of the first item of the message.

The elements of {\tt buf} may have primitive type or class type.
If the elements are objects, they must be serializable objects.
If the {\tt datatype} argument represents an MPI basic type, its
value must agree with the element type of {\tt buf}:
the basic MPI datatypes supported, and their correspondence to Java types,
are as follows\vspace{0.2in} \\
\centerline{
\begin{tabular}{| l | l |} \hline
MPI datatype & Java datatype \\
\hline
MPI.BYTE     & {\tt byte} \\
MPI.CHAR     & {\tt char} \\
MPI.SHORT    & {\tt short} \\
MPI.BOOLEAN  & {\tt boolean} \\
MPI.INT      & {\tt int} \\
MPI.LONG     & {\tt long} \\
MPI.FLOAT    & {\tt float} \\
MPI.DOUBLE   & {\tt double} \\
MPI.OBJECT   & {\tt Object} \\
\hline
\end{tabular}
} \vspace{0.2in} \newline
If the {\tt datatype} argument represents an MPI derived type, its
{\em base type} must agree with the element type of {\tt buf}
(see section \ref{derived}).
\begin{rationale}
The {\tt datatype} argument is not redundant in Java, because
this proposal includes support for MPI derived types.  If it was decided
to remove derived types from the API, {\tt datatype} arguments
could be removed from various functions, and Java runtime inquiries
could be used internally to extract the element type of the buffer,
or methods like {\tt send} could be overloaded to accept
buffers with elements of the 9 basic types.  (The disadvantage of the
latter approach is that it leads to a large proliferation in the
number of methods.  Historically MPI has eschewed this kind of
overloading.  Java at least provides runtime mechanisms for checking
type correctness, so the extra security provided by overloading is
probably not an essential requirement.)
\end{rationale}
If a data type has {\tt MPI.OBJECT} as its base type, the objects in
the buffer will be transparently serialized and unserialized inside the
communication operations.
\begin{implementors}
While straightforward in principle, this extension involves several
complications for JNI-based wrapper implementations.  The size of
the physical receive buffer cannot be computed in advance, so
it may be necessary to split the physical message into a header 
message containing buffer size, followed by the data.  This
notably complicates wrappers for communication methods based on
{\tt Request} classes.  Also, the wrapper has to be prepared to
interpret derived types with {\tt MPI.OBJECT} as the base type,
because of course the underlying C MPI cannot do this.
\end{implementors}

\begin{verbatim}

Status Comm.recv(Object buf, int offset, int count,
                 Datatype datatype, int source, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & receive buffer array \\
{\tt offset}   & initial offset in receive buffer \\
{\tt count}    & number of items in receive buffer \\
{\tt datatype} & datatype of each item in receive buffer \\
{\tt source}   & rank of source \\
{\tt tag}      & message tag \\
\\
{\em returns:} & status object
\end{arguments}
Blocking receive operation.  Java binding of the MPI operation {\em
MPI\_RECV}.  The actual argument associated with {\tt buf} must be an
array.  The value {\tt offset} is a subscript in this array, defining
the position into which the first item of the incoming message will be
copied.

The elements of {\tt buf} may have primitive type or class type.
If the {\tt datatype} argument represents an MPI basic type, its
value must agree with the element type of {\tt buf};
if {\tt datatype} represents an MPI derived type, its
{\em base type} must agree with the element type of {\tt buf}
(see section \ref{derived}).

The MPI constants {\em MPI\_ANY\_SOURCE} and {\em MPI\_ANY\_TAG} are
available as {\tt MPI.ANY\_SOURCE} and {\tt MPI.ANY\_TAG}.

The following methods can be used to interrogate the return status
of a receive operation.
\begin{verbatim}

int Status.getSource()
\end{verbatim}
\begin{argsNoSpace}
{\em returns:} & source of the received message.
\end{argsNoSpace}

\begin{verbatim}

int Status.getTag()
\end{verbatim}
\begin{argsNoSpace}
{\em returns:} & tag of the received message.
\end{argsNoSpace}

\begin{rationale}
Following common object-oriented practices, source and
tag values are returned by inquiry functions {\tt getSource}
and {\tt getTag}, rather than through publicly accessible fields
in the {\tt Status} class.
\end{rationale}

\begin{verbatim}

int Status.getCount(Datatype datatype)
\end{verbatim}
\begin{arguments}
{\tt datatype} & datatype of each item in receive buffer \\
\\
{\em returns:} & number of received entries
\end{arguments}
Java binding of the MPI operation {\em MPI\_GET\_COUNT}. 

\subsubsection{Data type matching and data conversion}

The Java language definition places quite detailed constraints on the
representation of its primitive types---for example it requires
conformance with IEEE 754 for {\tt float} and {\tt double}.  There may
still be a requirement for representation conversion in heterogenous
systems.  For example, source and destination computers (or virtual
machines) may have different endianess.

\subsubsection{Communication Modes}

\begin{verbatim}

void Comm.bsend(Object buf, int offset, int count,
                Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag
\end{arguments}
Send in buffered mode.
Java binding of the MPI operation {\em MPI\_BSEND}. 
Further comments as for {\tt send}.

\begin{verbatim}

void Comm.ssend(Object buf, int offset, int count,
                Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag
\end{arguments}
Send in synchronous mode.
Java binding of the MPI operation {\em MPI\_SSEND}. 
Further comments as for {\tt send}.

\begin{verbatim}

void Comm.rsend(Object buf, int offset, int count,
                Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag
\end{arguments}
Send in ready mode.
Java binding of the MPI operation {\em MPI\_RSEND}. 
Further comments as for {\tt send}.

\subsubsection{Semantics of point-to-point communication}

No special issues for Java binding.

\subsubsection{Buffer allocation and usage}

\begin{verbatim}

void MPI.bufferAttach(byte [] buffer)
\end{verbatim}
\begin{arguments}
{\tt buffer}   & buffer array
\end{arguments}
Provides to MPI a buffer in user's memory to be used for buffering
outgoing messages.
Java binding of the MPI operation {\em MPI\_BUFFER\_ATTACH}. 

\begin{verbatim}

byte [] MPI.bufferDetach()
\end{verbatim}
\begin{arguments}
{\em returns:} & buffer array
\end{arguments}
Detach the buffer currently associated with MPI.
Java binding of the MPI operation {\em MPI\_BUFFER\_DETACH}. 
The MPI constant {\em MPI\_BSEND\_OVERHEAD} is available as {\tt
MPI.BSEND\_\-OVERHEAD}.

{\em [It is probably impossible to implement this function
properly in a JNI interface to native Java, because there is no
way to return the system-defined buffer as a Java array.]}

\subsubsection{Nonblocking communication}

Nonblocking communications use methods of the {\tt Request}
class to identify communication operations and match the operation
that initiates the communication with the operation that terminates
it.

\begin{verbatim}

Request Comm.isend(Object buf, int offset, int count,
                   Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & communication request
\end{arguments}
Start a standard mode, nonblocking send.
Java binding of the MPI operation {\em MPI\_ISEND}. 
Further comments as for {\tt send}.

\begin{verbatim}

Request Comm.ibsend(Object buf, int offset, int count,
                    Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & communication request
\end{arguments}
Start a buffered mode, nonblocking send.
Java binding of the MPI operation {\em MPI\_IBSEND}. 
Further comments as for {\tt send}.

\begin{verbatim}

Request Comm.issend(Object buf, int offset, int count,
                    Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & communication request
\end{arguments}
Start a synchronous mode, nonblocking send.
Java binding of the MPI operation {\em MPI\_ISSEND}. 
Further comments as for {\tt send}.

\begin{verbatim}

Request Comm.irsend(Object buf, int offset, int count,
                    Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & communication request
\end{arguments}
Start a ready mode, nonblocking send.
Java binding of the MPI operation {\em MPI\_IRSEND}. 
Further comments as for {\tt send}.

\begin{verbatim}

Request Comm.irecv(Object buf, int offset, int count,
                   Datatype datatype, int source, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & receive buffer array \\
{\tt offset}   & initial offset in receive buffer \\
{\tt count}    & number of items in receive buffer \\
{\tt datatype} & datatype of each item in receive buffer \\
{\tt source}   & rank of source \\
{\tt tag}      & message tag \\
\\
{\em returns:} & communication request
\end{arguments}
Start a nonblocking receive.
Java binding of the MPI operation {\em MPI\_IRECV}. 
Further comments as for {\tt recv}.

The following functions are used to complete nonblocking communication
operations (and also communications started using the persistent
communication requests---subclass {\tt Prequest}---introduced later).
We use the following terminology.  A request is ``active'' if it is
associated with an ongoing communication.  Otherwise it is inactive.
An inactive instance of the base class {\tt Request} is called a ``void
request''.  Note, however, that an inactive instance of the {\tt Prequest}
subclass is not said to be ``void'', because it retains detailed
information about a communication pattern even when no corresponding
communication is ongoing.
\begin{rationale}
A ``void request'' corresponds to what is called a ``null handle'' in
the C and Fortran MPI bindings.  It seems impractical to have
completion operations like {\tt wait} set request object references to
null references in the Java sense (because Java methods cannot directly
modify references passed to them as arguments).  To avoid a confusing
semantic distinction between null MPI handles and null Java references
we introduce the terminology of a ``void request object''.  The {\tt
Request} default constructor can be used to explicitly create a void
request, replacing assignment of the value {\em MPI\_REQUEST\_NULL}.
The inquiry {\tt Request.isVoid} can be used to determine if a
particular request is void, replacing a test for equality to {\em
MPI\_REQUEST\_NULL}.
\end{rationale}

\begin{verbatim}

Status Request.wait()
\end{verbatim}
\begin{arguments}
{\em returns:} & status object
\end{arguments}
Blocks until the operation identified by
the request is complete.
Java binding of the MPI operation {\em MPI\_WAIT}. 
After the call returns, the request object becomes inactive.

\begin{verbatim}

Status Request.test()
\end{verbatim}
\begin{arguments}
{\em returns:} & status object or null reference
\end{arguments}
Returns a status object if the operation
identified by the request is complete, or a null reference otherwise.
Java binding of the MPI operation {\em MPI\_TEST}.
After the call, if the operation is complete (ie, if the return value
of {\tt test} is non-null), the request object becomes an inactive request.

\begin{verbatim}

Request.Request()
\end{verbatim}
Default constructor for {\tt Request}.  Constructs a void
request.

\begin{verbatim}

boolean Request.isVoid()
\end{verbatim}
\begin{arguments}
{\em returns:} & true if the request object is void, false otherwise
\end{arguments}
Note that {\tt isVoid} is always false on instances of the
subclass {\tt Prequest}.

\begin{verbatim}

void Request.finalize()
\end{verbatim}
\begin{implementors}
In a JNI-based wrapper implementation, the finalize method for 
{\tt Request} should call {\tt MPI\_Request\_free} only if
{\tt isVoid()} is false.  If the request is void, for
example because a {\tt wait} operation on the object recently succeeded,
the {\tt MPI\_Request\_free} call is redundant.
\end{implementors}

\begin{verbatim}

static Status Request.waitAny(Request [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of requests \\
\\
{\em returns:}        & status object
\end{arguments}
Blocks until one of the operations associated with the active
requests in the array has completed.  Java binding of the MPI operation
{\em MPI\_WAITANY}.  The index in {\tt arrayOfRequests} for the
request that completed can be obtained from the status object
through the {\tt Status.getIndex} inquiry.  The corresponding element
of {\tt arrayOfRequests} becomes inactive.

The {\tt arrayOfRequests} may contain inactive requests.  If
the list containts no active requests, the method immediately returns a
status for which the {\tt getIndex} inquiry will return {\tt
MPI.UNDEFINED}.

\begin{verbatim}

static Status Request.testAny(Request [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of requests \\
\\
{\em returns:}        & status object or null reference
\end{arguments}
Tests for completion of either one or none of the operations associated
with active requests.
Java binding of the MPI operation {\em MPI\_TESTANY}. 
If some request completed, the index in {\tt arrayOfRequests} of
that request can be obtained from the status object
through the {\tt Status.getIndex()} inquiry.  The corresponding element
of {\tt arrayOfRequests} becomes inactive.
If no request completed, {\tt testAny} returns a null reference. 

The {\tt arrayOfRequests} may contain inactive requests.  If
the list containts no active requests, the method immediately returns a
status for which the {\tt getIndex} inquiry will return {\tt
MPI.UNDEFINED}.

\begin{verbatim}

int Status.getIndex()
\end{verbatim}
\begin{arguments}
{\em returns:} & index of the the operation that completed.
\end{arguments}
This method can be applied to any status object returned by
{\tt Request.waitAny}, {\tt Request.testAny},
{\tt Request.waitSome}, or {\tt Request.testSome}.

\begin{verbatim}

static Status [] Request.waitAll(Request [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of requests \\
\\
{\em returns:}        & array of status objects
\end{arguments}
Blocks until all of the operations associated with the active requests
in the array have completed.  Java binding of the MPI operation {\em
MPI\_WAITALL}.  The result array will be the same size as {\tt
arrayOfRequests}.  On exit, requests become inactive.  If the {\em
input} value of {\tt arrayOfRequests} contains any inactive
requests, corresponding elements of the result array will contain null
status references.

\begin{verbatim}

static Status [] Request.testAll(Request [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of requests \\
\\
{\em returns:}        & array of status objects, or a null reference
\end{arguments}
Tests for completion of {\em all} of the operations associated
with active requests.
Java binding of the MPI operation {\em MPI\_TESTALL}. 
If all operations have completed, the exit values of the argument array
and the result array are as for {\tt waitAll}.  If any
operation has not completed, the result value is null and no
element of the argument array is modified.

\begin{verbatim}

static Status [] Request.waitSome(Request [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of requests \\
\\
{\em returns:}        & array of status objects
\end{arguments}
Blocks until at least one of the operations associated with the active
requests in the array has completed.
Java binding of the MPI operation {\em MPI\_WAITSOME}. 
The size of the result array will be the number of operations that
completed.  The index in {\tt arrayOfRequests} for each request that
completed can be obtained from the returned status objects using the
{\tt Status.getIndex} inquiry.  The corresponding elements in {\tt
arrayOfRequests} become inactive.

If {\tt arrayOfRequests} list contains no active requests, {\tt
testAll} immediately returns a null reference.

\begin{verbatim}

static Status [] Request.testSome(Request [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of requests \\
\\
{\em returns:}        & array of status objects
\end{arguments}
Behaves like {\tt waitSome}, except that it returns immediately.
Java binding of the MPI operation {\em MPI\_TESTSOME}. 
If no operation has completed, {\tt testSome} returns an array of length zero
and elements of {\tt arrayOfRequests} are unchanged.
Otherwise, arguments and return value are as for {\tt waitSome}.

\subsubsection{Probe and Cancel}

\begin{verbatim}

Status Comm.iprobe(int source, int tag)
\end{verbatim}
\begin{arguments}
{\tt source}   & source rank \\
{\tt tag}      & tag value \\
\\
{\em returns:} & status object or null reference
\end{arguments}
Check if there is an incoming message matching the pattern specified.
Java binding of the MPI operation {\em MPI\_IPROBE}. 
If such a message is currently available, a status object
similar to the return value of a matching {\tt recv} operation is returned.
Otherwise a null reference is returned.

\begin{verbatim}

Status Comm.probe(int source, int tag)
\end{verbatim}
\begin{arguments}
{\tt source}   & source rank \\
{\tt tag}      & tag value \\
\\
{\em returns:} & status object or null reference
\end{arguments}
Wait until there is an incoming message matching the pattern specified.
Java binding of the MPI operation {\em MPI\_PROBE}. 
Returns a status object
similar to the return value of a matching {\tt recv} operation.

\begin{verbatim}

void Request.cancel()
\end{verbatim}
Mark a pending nonblocking communication for cancellation.
Java binding of the MPI operation {\em MPI\_CANCEL}. 

\begin{verbatim}

boolean Status.testCancelled()
\end{verbatim}
\begin{arguments}
{\em returns:} & true if the operation was succesfully cancelled,
false otherwise
\end{arguments}
Test if communication was cancelled.
Java binding of the MPI operation {\em MPI\_\-TEST\_CANCELLED}. 

\subsubsection{Persistent communication requests}

\begin{verbatim}

Prequest Comm.sendInit(Object buf, int offset, int count,
                       Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & persistent communication request
\end{arguments}
Creates a persistent communication request for a standard mode send.
Java binding of the MPI operation {\em MPI\_SEND\_INIT}. 
Further comments as for {\tt send}.

\begin{verbatim}

Prequest Comm.bsendInit(Object buf, int offset, int count,
                        Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & persistent communication request
\end{arguments}
Creates a persistent communication request for a buffered mode send.
Java binding of the MPI operation {\em MPI\_BSEND\_INIT}. 
Further comments as for {\tt send}.

\begin{verbatim}

Prequest Comm.ssendInit(Object buf, int offset, int count,
                        Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & persistent communication request
\end{arguments}
Creates a persistent communication request for a synchronous mode send.
Java binding of the MPI operation {\em MPI\_SSEND\_INIT}. 
Further comments as for {\tt send}.

\begin{verbatim}

Prequest Comm.rsendInit(Object buf, int offset, int count,
                        Datatype datatype, int dest, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & send buffer array \\
{\tt offset}   & initial offset in send buffer \\
{\tt count}    & number of items to send \\
{\tt datatype} & datatype of each item in send buffer \\
{\tt dest}     & rank of destination \\
{\tt tag}      & message tag \\
\\
{\em returns:} & persistent communication request
\end{arguments}
Creates a persistent communication request for a ready mode send.
Java binding of the MPI operation {\em MPI\_RSEND\_INIT}. 
Further comments as for {\tt send}.

\begin{verbatim}

Prequest Comm.recvInit(Object buf, int offset, int count,
                       Datatype datatype, int source, int tag)
\end{verbatim}
\begin{arguments}
{\tt buf}      & receive buffer array \\
{\tt offset}   & initial offset in receive buffer \\
{\tt count}    & number of items in receive buffer \\
{\tt datatype} & datatype of each item in receive buffer \\
{\tt source}   & rank of source \\
{\tt tag}      & message tag \\
\\
{\em returns:} & persistent communication request
\end{arguments}
Creates a persistent communication request for a receive operation.
Java binding of the MPI operation {\em MPI\_RECV\_INIT}. 
Further comments as for {\tt recv}.

\begin{verbatim}

void Prequest.start()
\end{verbatim}
Activate a persistent communication request.
Java binding of the MPI operation {\em MPI\_START}. 
The communication is completed by using the request in
one of the operations {\tt Request.wait}, {\tt Request.test}, {\tt
Request.waitAny}, {\tt Request.test\-Any}, {\tt Request.waitAll}, {\tt
Request.testAll}, {\tt Request.waitSome}, or {\tt Request.\-testSome}.
On successful completion the request becomes inactive again.
It can be reactivated by a further call to {\tt start}.

\begin{verbatim}

static void Prequest.startAll(Prequest [] arrayOfRequests)
\end{verbatim}
\begin{arguments}
{\tt arrayOfRequests} & array of persistent communication requests \\
\end{arguments}
Activate a list of communication requests.
Java binding of the MPI operation {\em MPI\_STARTALL}. 


\subsubsection{Send-receive}

\begin{verbatim}

Status Comm.sendrecv(Object sendbuf, int sendoffset,
                     int sendcount, Datatype sendtype,
                     int dest, int sendtag,
                     Object recvbuf, int recvoffset,
                     int recvcount, Datatype recvtype,
                     int source, int recvtag)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items to send \\
{\tt sendtype}   & datatype of each item in send buffer \\
{\tt dest}       & rank of destination \\
{\tt sendtag}    & send tag \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcount}  & number of items in receive buffer \\
{\tt recvtype}   & datatype of each item in receive buffer \\
{\tt source}     & rank of source \\
{\tt recvtag}    & receive tag \\
\\
{\em returns:}   & status object
\end{arguments}
Execute a blocking send and receive operation.
Java binding of the MPI operation {\em MPI\_SENDRECV}. 
Further comments as for {\tt send} and {\tt recv}.

\begin{verbatim}

Status Comm.sendrecvReplace(Object buf, int offset,
                            int count, Datatype datatype,
                            int dest, int sendtag,
                            int source, int recvtag)
\end{verbatim}
\begin{arguments}
{\tt buf}        & buffer array \\
{\tt offset}     & initial offset in buffer \\
{\tt count}      & number of items to send \\
{\tt datatype}   & datatype of each item in buffer \\
{\tt dest}       & rank of destination \\
{\tt sendtag}    & send tag \\
{\tt source}     & rank of source \\
{\tt recvtag}    & receive tag \\
\\
{\em returns:}   & status object
\end{arguments}
Execute a blocking send and receive operation, receiving message
into send buffer.
Java binding of the MPI operation {\em MPI\_SENDRECV\_REPLACE}. 
Further comments as for {\tt send} and {\tt recv}.

\subsubsection{Null processes}

The constant {\em MPI\_PROC\_NULL} is available as {\tt MPI.PROC\_NULL}.

\subsubsection{Derived datatypes\label{derived}}

In C or Fortran bindings of MPI, derived datatypes have two roles.
One is to allow messages to contain mixed types (for example they
allow an integer count followed by a sequence of real numbers to be
passed in a single message).  The other is to allow noncontiguous data
to be transmitted.
In the Java binding presented here the first role is abandoned.
Any derived type can only include elements of a single basic type.
\begin{rationale}
In the C binding of MPI, for example, the {\em MPI\_TYPE\_\-STRUCT}
constructor for derived types might be used to describe the physical
layout of a {\em struct} containing mixed types.  This will not
work in Java, because Java does not expose the low-level layout of its
objects.  In C and Fortran another use of {\em MPI\_TYPE\_\-STRUCT}
involves incorporating offsets computed as differences between absolute
addresses, so that parts of a message can come from separately declared
entities.  It might be possible to contrive something analogous in a Java
binding, somehow encoding object references instead of physical
addresses.  Such a device is likely to be
cumbersome---even in the C and Fortran the mechanism is not
particularly elegant.  
Meanwhile, the effect of either of these applications of
{\em MPI\_TYPE\_\-STR\-UCT} can be achieved by using {\tt MPI.OBJECT}
as the buffer type, and relying on Java object serialization.
\end{rationale}
This leaves description of noncontiguous buffers as the
essential role for derived data types in the Java binding.

Every derived data type constructable in the Java binding has
a uniquely defined {\em base type}.  This is one of the 9 basic
types enumerated in section \ref{sec:blocking}.
Derived types inherit their base types from their precursors in
a straightforward way.

In the proposed binding a {\bf general datatype} is an object that
specifies two things
\begin{itemize}
\item
A base type
\item
A sequence of integer displacements
\end{itemize}
In contrast to the C and Fortran bindings the displacements are
in terms of subscripts in the buffer array argument, {\em not}
byte displacements.

The base types for the predefined MPI datatypes are 
\vspace{0.2in} \\
\centerline{
\begin{tabular}{| l | l |} \hline
MPI datatype & base type \\
\hline
MPI.BYTE     & {\tt byte} \\
MPI.CHAR     & {\tt char} \\
MPI.SHORT    & {\tt short} \\
MPI.BOOLEAN  & {\tt boolean} \\
MPI.INT      & {\tt int} \\
MPI.LONG     & {\tt long} \\
MPI.FLOAT    & {\tt float} \\
MPI.DOUBLE   & {\tt double} \\
MPI.OBJECT   & {\tt Object} \\
MPI.LB       & $\perp$  \\
MPI.UB       & $\perp$  \\
MPI.PACKED   & {\tt byte} \\
\hline
\end{tabular}
} \vspace{0.2in} \newline
The symbol $\perp$ is a special undefined value.
The displacement sequence for the predefined types consists of a single zero.

If the displacement sequence of a datatype is
\begin{displaymath}
\mbox{\em DispSeq} = \{ \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1} \}
\end{displaymath}
then
\begin{eqnarray*}
\mbox{\em lb}(\mbox{\em DispSeq}) & = & \min_j \mbox{\em disp}_j, \\
\mbox{\em ub}(\mbox{\em DispSeq}) & = & \max_j (\mbox{\em disp}_j + 1),
                                                        \quad \mbox{and} \\
\mbox{\em extent}(\mbox{\em DispSeq}) & = &
  \mbox{\em ub}(\mbox{\em DispSeq}) - \mbox{\em lb}(\mbox{\em DispSeq})
\end{eqnarray*}
\begin{rationale}
This definition of the extent differs from the definition in the C or
Fortran.  It is in units of the buffer array index, {\em not} in units
of bytes.
\end{rationale}
\begin{implementors}
In a JNI-based wrapper implementation, the return values of the
{\tt extent}, {\tt size}, {\tt lb} and {\tt ub} inquiries will
have to be suitably scaled from the result obtained from the
corresponding C inquiries.
\end{implementors}
These definitions have to be modified if the type construction involves
{\tt MPI.LB}, {\tt MPI.UB}.

\begin{verbatim}

Datatype Datatype.contiguous(int count)
\end{verbatim}
\begin{arguments}
{\tt count}      & replication count \\
\\
{\em returns:}   & new datatype
\end{arguments}
Construct new datatype representing replication of this datatype into
contiguous locations.
Java binding of the MPI operation {\em MPI\_TYPE\_CONTIGUOUS}.
The base type of the new datatype is the same as the base type of
this type.  Assume the displacement sequence of this type is
\begin{displaymath}
\{ \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1} \}
\end{displaymath}
with extent {\em ex}.  Then the new datatype has a displacement sequence
with {\tt count} $\cdot$ {\tt n} entries defined by:
\begin{eqnarray*}
&\{ & \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1}, \\
&   & \mbox{\em disp}_0 + ex, \ldots, \mbox{\em disp}_{n - 1} + ex, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 + ex \cdot (\mbox{\tt count} - 1),
      \ldots, \mbox{\em disp}_{n - 1} + ex \cdot (\mbox{\tt count} - 1)
\quad \}
\end{eqnarray*}

\begin{verbatim}

Datatype Datatype.vector(int count, int blocklength, int stride)
\end{verbatim}
\begin{arguments}
{\tt count}       & number of blocks \\
{\tt blocklength} & number of elements in each block \\
{\tt stride}      & number of elements between start of each block \\
\\
{\em returns:}    & new datatype
\end{arguments}
Construct new datatype representing replication of this datatype into
locations that consist of equally spaced blocks.
Java binding of the MPI operation {\em MPI\_TYPE\_VECTOR}.
The base type of the new datatype is the same as the base type of
this type.  Assume the displacement sequence of this type is
\begin{displaymath}
\{ \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1} \}
\end{displaymath}
with extent {\em ex}.  Let {\tt bl} be {\tt blocklength}.
Then the new datatype has a displacement sequence
with {\tt count} $\cdot$ {\tt bl} $\cdot$ {\tt n} entries defined by:
\begin{eqnarray*}
&\{ & \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1}, \\
&   & \mbox{\em disp}_0 + \mbox{\em ex}, \ldots,
       \mbox{\em disp}_{n - 1} + \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 + \mbox{\em ex} \cdot (\mbox{\tt bl} - 1),
      \ldots, \mbox{\em disp}_{n - 1} +
          \mbox{\em ex} \cdot (\mbox{\tt bl} - 1), \\
\\
&   & \mbox{\em disp}_0 + \mbox{\em ex} \cdot \mbox{\tt stride},
      \ldots, \mbox{\em disp}_{n - 1} +
          \mbox{\em ex} \cdot \mbox{\tt stride}, \\
&   & \mbox{\em disp}_0 + \mbox{\em ex} \cdot (\mbox{\tt stride} + 1),
      \ldots,
      \mbox{\em disp}_{n - 1} + \mbox{\em ex} \cdot (\mbox{\tt stride} + 1), \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
          \mbox{\em ex} \cdot (\mbox{\tt stride} + \mbox{\tt bl} - 1),
      \ldots,
      \mbox{\em disp}_{n - 1} +
          \mbox{\em ex} \cdot (\mbox{\tt stride} + \mbox{\tt bl} - 1), \\
\\
&   & \ldots , \\
\\
&   & \mbox{\em disp}_0 +
          \mbox{\em ex} \cdot \mbox{\tt stride} \cdot (\mbox{\tt count} - 1),
      \ldots,
      \mbox{\em disp}_{n - 1} +
          \mbox{\em ex} \cdot \mbox{\tt stride} \cdot (\mbox{\tt count} - 1), \\
&   & \mbox{\em disp}_0 +
        \mbox{\em ex} \cdot (\mbox{\tt stride}
           \cdot (\mbox{\tt count} - 1) + 1),
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}_{n - 1} +
        \mbox{\em ex} \cdot (\mbox{\tt stride}
           \cdot (\mbox{\tt count} - 1) + 1), \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
        \mbox{\em ex} \cdot (\mbox{\tt stride}
           \cdot (\mbox{\tt count} - 1) + \mbox{\tt bl} - 1),
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}_{n - 1} +
        \mbox{\em ex} \cdot (\mbox{\tt stride}
           \cdot (\mbox{\tt count} - 1) + \mbox{\tt bl} - 1)
\quad \}
\end{eqnarray*}


\begin{verbatim}

Datatype Datatype.hvector(int count, int blocklength, int stride)
\end{verbatim}
\begin{arguments}
{\tt count}       & number of blocks \\
{\tt blocklength} & number of elements in each block \\
{\tt stride}      & number of elements between start of each block \\
\\
{\em returns:}    & new datatype
\end{arguments}
Identical to {\tt vector} except that the stride is expressed
directly in terms of the buffer index, rather than the units of this type.
Java binding of the MPI operation {\em MPI\_TYPE\_HVECTOR}.
{\em Unlike other language bindings}, the value
of {\tt stride} is {\em not} measured in bytes.
The displacement sequence of the new type is:
\begin{eqnarray*}
&\{ & \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1}, \\
&   & \mbox{\em disp}_0 + \mbox{\em ex}, \ldots,
       \mbox{\em disp}_{n - 1} + \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 + \mbox{\em ex} \cdot (\mbox{\tt bl} - 1),
      \ldots, \mbox{\em disp}_{n - 1} +
          \mbox{\em ex} \cdot (\mbox{\tt bl} - 1), \\
\\
&   & \mbox{\em disp}_0 + \mbox{\em ex} \cdot \mbox{\tt stride},
      \ldots, \mbox{\em disp}_{n - 1} + \mbox{\tt stride}, \\
&   & \mbox{\em disp}_0 + \mbox{\tt stride} + \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} + \mbox{\tt stride} + \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
          \mbox{\tt stride} + \mbox{\em ex} \cdot (\mbox{\tt bl} - 1),
      \ldots,
      \mbox{\em disp}_{n - 1} +
          \mbox{\tt stride} + \mbox{\em ex} \cdot (\mbox{\tt bl} - 1), \\
\\
&   & \ldots , \\
\\
&   & \mbox{\em disp}_0 + \mbox{\tt stride} \cdot (\mbox{\tt count} - 1),
      \ldots,
      \mbox{\em disp}_{n - 1} +
          \mbox{\tt stride} \cdot (\mbox{\tt count} - 1), \\
&   & \mbox{\em disp}_0 +
           \mbox{\tt stride} \cdot (\mbox{\tt count} - 1) + \mbox{\em ex},
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}_{n - 1} +
        \mbox{\tt stride} \cdot (\mbox{\tt count} - 1) + \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
        \mbox{\tt stride} \cdot (\mbox{\tt count} - 1) +
           \mbox{\em ex} \cdot (\mbox{\tt bl} - 1),
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}_{n - 1} +
        \mbox{\tt stride} \cdot (\mbox{\tt count} - 1) +
           \mbox{\em ex} \cdot (\mbox{\tt bl} - 1)
\quad \}
\end{eqnarray*}

\begin{verbatim}

Datatype Datatype.indexed(int [] arrayOfBlocklengths,
                          int [] arrayOfDisplacements)
\end{verbatim}
\begin{arguments}
{\tt arrayOfBlocklengths}  & number of elements per block \\
{\tt arrayOfDisplacements} & displacement of each block in units of old type \\
\\
{\em returns:}             & new datatype
\end{arguments}
Construct new datatype representing replication of this type into
a sequence of blocks where each block can contain a different number
of copies and have a different displacement.
Java binding of the MPI operation {\em MPI\_TYPE\_\-INDEXED}.  The number
of blocks is taken to be size of the {\tt arrayOf\-Block\-lengths}
argument.  The second argument, {\tt arrayOfDisplacements}, should
be the same size.
The base type of the new datatype is the same as the base type of
this type.  Assume the displacement sequence of this type is
\begin{displaymath}
\{ \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1} \}
\end{displaymath}
with extent {\em ex}.  Let {\tt B} be the {\tt arrayOfBlocklengths}
argument and {\tt D} be the {\tt arrayOfDisplacements} argument.
Then the new datatype has a displacement sequence with
$n \cdot \sum_{i=0}^{\mbox{\em count} - 1} \mbox{\tt B}[i]$ entries:
\begin{eqnarray*}
&\{ & \mbox{\em disp}_0 + \mbox{\tt D}[0] \cdot \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} + \mbox{\tt D}[0] \cdot \mbox{\em ex}, \\
&   & \mbox{\em disp}_0 + (\mbox{\tt D}[0] + 1) \cdot \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} + (\mbox{\tt D}[0] + 1) \cdot \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
         (\mbox{\tt D}[0] + \mbox{\tt B} [0] - 1) \cdot \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} +
         (\mbox{\tt D}[0] + \mbox{\tt B} [0] - 1) \cdot \mbox{\em ex}, \\
\\
&   & \ldots , \\
\\
&   & \mbox{\em disp}_0 + \mbox{\tt D}[\mbox{\em count} - 1]
         \cdot \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} +
         \mbox{\tt D}[\mbox{\em count} - 1] \cdot \mbox{\em ex}, \\
&   & \mbox{\em disp}_0 +
         (\mbox{\tt D}[\mbox{\em count} - 1] + 1) \cdot \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} +
         (\mbox{\tt D}[\mbox{\em count} - 1] + 1) \cdot \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
         (\mbox{\tt D}[\mbox{\em count} - 1] +
            \mbox{\tt B} [\mbox{\em count} - 1] - 1) \cdot \mbox{\em ex},
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}_{n - 1} +
         (\mbox{\tt D}[\mbox{\em count} - 1] +
            \mbox{\tt B} [\mbox{\em count} - 1] - 1) \cdot \mbox{\em ex}
\quad \}
\end{eqnarray*}
Here, {\em count} is the number of blocks.


\begin{verbatim}

Datatype Datatype.hindexed(int [] array_of_blocklengths,
                           int [] array_of_displacements)
\end{verbatim}
\begin{arguments}
{\tt arrayOfBlocklengths}  & number of elements per block \\
{\tt arrayOfDisplacements} & displacement in buffer for each block \\
\\
{\em returns:}             & new datatype
\end{arguments}
Identical to {\tt indexed} except that the displacements are expressed
directly in terms of the buffer index, rather than the units of this type.
Java binding of the MPI operation {\em MPI\_TYPE\_HINDEXED}.
{\em Unlike other language bindings}, the values
in {\tt arrayOfDisplacements} are {\em not} measured in bytes.
The displacement sequence of the new type is:
\begin{eqnarray*}
&\{ & \mbox{\em disp}_0 + \mbox{\tt D}[0],
      \ldots,
      \mbox{\em disp}_{n - 1} + \mbox{\tt D}[0], \\
&   & \mbox{\em disp}_0 + \mbox{\tt D}[0] + \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} + \mbox{\tt D}[0] + \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
         \mbox{\tt D}[0] + (\mbox{\tt B} [0] - 1) \cdot \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} +
         \mbox{\tt D}[0] + (\mbox{\tt B} [0] - 1) \cdot \mbox{\em ex}, \\
\\
&   & \ldots , \\
\\
&   & \mbox{\em disp}_0 + \mbox{\tt D}[\mbox{\em count} - 1],
      \ldots,
      \mbox{\em disp}_{n - 1} + \mbox{\tt D}[\mbox{\em count} - 1], \\
&   & \mbox{\em disp}_0 +
         \mbox{\tt D}[\mbox{\em count} - 1] + \mbox{\em ex},
      \ldots,
      \mbox{\em disp}_{n - 1} +
         \mbox{\tt D}[\mbox{\em count} - 1] + \mbox{\em ex}, \\
&   & \ldots , \\
&   & \mbox{\em disp}_0 +
         \mbox{\tt D}[\mbox{\em count} - 1] +
            (\mbox{\tt B} [\mbox{\em count} - 1] - 1) \cdot \mbox{\em ex},
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}_{n - 1} +
         \mbox{\tt D}[\mbox{\em count} - 1] +
            (\mbox{\tt B} [\mbox{\em count} - 1] - 1) \cdot \mbox{\em ex}
\quad \}
\end{eqnarray*}


\begin{verbatim}

static Datatype Datatype.struct(int [] arrayOfBlocklengths,
                                int [] arrayOfDisplacements,
                                Datatype [] arrayOfTypes)
\end{verbatim}
\begin{arguments}
{\tt arrayOfBlocklengths}  & number of elements per block \\
{\tt arrayOfDisplacements} & displacement in buffer for each block \\
{\tt arrayOfTypes}         & type of elements in each block \\
\\
{\em returns:}             & new datatype
\end{arguments}
The most general type constructor.
Java binding of the MPI operation {\em MPI\_\-TYPE\_STRUCT}.  The number
of blocks is taken to be size of the {\tt arrayOf\-Block\-lengths}
argument.  The second and third arguments, {\tt arrayOfDisplace\-ments}
and {\tt arrayOfTypes}, should be the same size.
{\em Unlike other language bindings}, the values
in {\tt arrayOf\-Displacements} are {\em not} measured in bytes.
All elements of {\tt arrayOfTypes} with definite base types {\em must have the
{\em same} base type}: this will be the base type of new datatype.
Let {\tt T} be the {\tt arrayOfTypes} argument.
Assume the displacement sequence of the old type $\mbox{\tt T}[i]$ is
\begin{displaymath}
\{ \mbox{\em disp}^i_0, \ldots, \mbox{\em disp}^i_{n_i - 1} \}
\end{displaymath}
with extent $\mbox{\em ex}_i$.  Let {\tt B} be the {\tt arrayOfBlocklengths}
argument and {\tt D} be the {\tt arrayOfDisplacements} argument.
Then the new datatype has a displacement sequence with
$\sum_{i=0}^{c - 1} \mbox{\tt B}[i] \cdot n_i$ entries:
\begin{eqnarray*}
&\{ & \mbox{\em disp}^0_0 + \mbox{\tt D}[0],
      \ldots,
      \mbox{\em disp}^0_{n_0 - 1} + \mbox{\tt D}[0], \\
&   & \mbox{\em disp}^0_0 + \mbox{\tt D}[0] + \mbox{\em ex}_0,
      \ldots,
      \mbox{\em disp}^0_{n_0 - 1} + \mbox{\tt D}[0] + \mbox{\em ex}_0, \\
&   & \ldots , \\
&   & \mbox{\em disp}^0_0 +
         \mbox{\tt D}[0] + (\mbox{\tt B} [0] - 1) \cdot \mbox{\em ex}_0,
      \ldots,
      \mbox{\em disp}^0_{n_0 - 1} +
         \mbox{\tt D}[0] + (\mbox{\tt B} [0] - 1) \cdot \mbox{\em ex}_0, \\
\\
&   & \ldots , \\
\\
&   & \mbox{\em disp}^{c - 1}_0 + \mbox{\tt D}[c - 1],
      \ldots,
      \mbox{\em disp}^{c - 1}_{n_{c - 1} - 1} + \mbox{\tt D}[c - 1], \\
&   & \mbox{\em disp}^{c - 1}_0 +
         \mbox{\tt D}[c - 1] + \mbox{\em ex}_{c - 1},
      \ldots,
      \mbox{\em disp}^{c - 1}_{n_{c - 1} - 1} +
         \mbox{\tt D}[c - 1] + \mbox{\em ex}_{c - 1}, \\
&   & \ldots , \\
&   & \mbox{\em disp}^{c - 1}_0 +
         \mbox{\tt D}[c - 1] +
            (\mbox{\tt B} [c - 1] - 1) \cdot \mbox{\em ex}_{c - 1},
      \ldots, \\
&   & \quad \quad \quad \quad \quad \mbox{\em disp}^{c - 1}_{n_{c - 1} - 1} +
         \mbox{\tt D}[c - 1] +
            (\mbox{\tt B} [c - 1] - 1) \cdot \mbox{\em ex}_{c - 1}
\quad \}
\end{eqnarray*}
Here, $c$ is the number of blocks.

If any elements of {\tt arrayOfTypes} are {\tt MPI.LB} or {\tt MPI.UB},
the corresponding displacements are omitted in the displacement
sequence.  These displacements only affect the computation of
{\tt Datatype.lb}, {\tt Datatype.ub} and {\tt Datatype.ext\-ent},
as explained below.


\begin{verbatim}

int Datatype.extent()
\end{verbatim}
\begin{arguments}
{\em returns:}             & datatype extent
\end{arguments}
Returns the extent of a datatype.
Java binding of the MPI operation {\em MPI\_\-TYPE\_\-EXTENT}.
Return value is equal to
\begin{displaymath}
  \mbox{\tt ub()} - \mbox{\tt lb()}
\end{displaymath}

\begin{verbatim}

int Datatype.size()
\end{verbatim}
\begin{arguments}
{\em returns:}             & datatype size
\end{arguments}
Returns the total size of the type.
Java binding of the MPI operation {\em MPI\_\-TYPE\_SIZE}.
Size is defined as the total number of buffer elements incorporated by
the data type, or equivalently as the length of the displacement
sequence.  {\em Unlike other language bindings}, the size is {\em not}
measured in bytes.

\begin{verbatim}

int Datatype.lb()
\end{verbatim}
\begin{arguments}
{\em returns:}             & displacement of lower bound from origin
\end{arguments}
Find the lower bound of a datatype.
Java binding of the MPI operation {\em MPI\_\-TYPE\_LB}.
Assume the displacement sequence of the datatype is
\begin{displaymath}
\{ \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1} \}
\end{displaymath}
In general the return value of {\tt lb} is
\begin{displaymath}
\min_j \mbox{\em disp}_j
\end{displaymath}
However, if construction of the derived type involved any basic
components of type {\tt MPI.LB}, the return value is the least
displacement specified for any of those {\tt MPI.LB} fields, regardless
of the values appearing in the displacement list proper.

\begin{verbatim}

int Datatype.ub()
\end{verbatim}
\begin{arguments}
{\em returns:}             & displacement of upper bound from origin
\end{arguments}
Find the upper bound of a datatype.
Java binding of the MPI operation {\em MPI\_\-TYPE\_UB}.
Assume the displacement sequence of the datatype is
\begin{displaymath}
\{ \mbox{\em disp}_0, \ldots, \mbox{\em disp}_{n - 1} \}
\end{displaymath}
In general the return value of {\tt lb} is
\begin{displaymath}
\max_j (\mbox{\em disp}_j + 1)
\end{displaymath}
However, if construction of the derived type involved any basic
components of type {\tt MPI.UB}, the return value is the largest
displacement specified for any of those {\tt MPI.UB} fields, regardless
of the values appearing in the displacement list proper.
\begin{implementors}
Meeting the detailed requirements of the MPI standard regarding
handling of {\tt MPI.UB} and {\tt MPI.LB} needs some extra care.
A possible implementation is to maintain two separate
``pseudo-displacements'' whose values are undefined unless a {\tt
MPI.UB} (respectively {\tt MPI.LB}) has appeared somewhere.  If
defined, these extra displacements are propagated to derived types
using the same formulae as for true displacements, except that only the
greatest (respectively least) term contributed by any precursor
pseudo-displacement need be retained as the pseudo-displacement
for the new type.
\end{implementors}

\begin{verbatim}

void Datatype.commit()
\end{verbatim}
Commit a derived datatype.
Java binding of the MPI operation {\em MPI\_TYPE\_\-COMMIT}.

\begin{verbatim}

void Datatype.finalize()
\end{verbatim}
Destructor.  Java binding of the MPI operation {\em MPI\_TYPE\_FREE}.

\begin{verbatim}

int Status.getElements(Datatype datatype)
\end{verbatim}
\begin{arguments}
{\tt datatype}         & datatype used by receive operation \\
\\
{\em returns:}         & number of received basic elements
\end{arguments}
Retrieve number of basic elements from status.
Java binding of the MPI operation {\em MPI\_GET\_ELEMENTS}.

\subsubsection{Pack and unpack}

\begin{verbatim}

int Comm.pack(Object inbuf, int offset, int incount,
              Datatype datatype,
              byte [] outbuf, int position)
\end{verbatim}
\begin{arguments}
{\tt inbuf}    & input buffer array \\
{\tt offset}   & initial offset in input buffer \\
{\tt incount}  & number of items in input buffer \\
{\tt datatype} & datatype of each item in input buffer \\
{\tt outbuf}   & output buffer \\
{\tt position} & initial position in ouput buffer \\
\\
{\em returns:} & final position in output buffer
\end{arguments}
Packs message in send buffer {\tt inbuf} into space specified in
{\tt outbuf}.
Java binding of the MPI operation {\em MPI\_PACK}.
The return value is the output value of {\tt position}---the
inital value incremented by the number of bytes written.

\begin{verbatim}

byte[] Comm.pack(Object inbuf, int offset, int incount,
                 Datatype datatype)
\end{verbatim}
\begin{arguments}
{\tt inbuf}    & input buffer array \\
{\tt offset}   & initial offset in input buffer \\
{\tt incount}  & number of items in input buffer \\
{\tt datatype} & datatype of each item in input buffer \\
\\
{\em returns:} & output buffer
\end{arguments}
Packs message in send buffer {\tt inbuf} into buffer space allocated
internally.
\begin{rationale}
This variant of {\tt pack} is essentially equivalent to calling {\em
MPI\_\-PACK\_SIZE} to find the amount of space needed for a packed
message, allocating a suitable byte vector, then calling the previous
variant of {\tt pack}.  The object serialization
mechanisms of Java do not provide a convenient way of precomputing the
amount of space required to pack objects (there is no analogue to
{\em MPI\_PACK\_SIZE}).  So if the base type of the message is {\tt
MPI.OBJECT} the second form of {\tt pack} is more convenient.
\end{rationale}

\begin{verbatim}

int Comm.unpack(byte [] inbuf, int position,
                Object outbuf, int offset, int outcount,
                Datatype datatype)
\end{verbatim}
\begin{arguments}
{\tt inbuf}    & input buffer \\
{\tt position} & initial position in input buffer \\
{\tt outbuf}   & output buffer array \\
{\tt offset}   & initial offset in output buffer \\
{\tt outcount} & number of items in output buffer \\
{\tt datatype} & datatype of each item in output buffer \\
\\
{\em returns:} & final position in input buffer
\end{arguments}
Unpacks message in receive buffer {\tt outbuf} into space specified in
{\tt inbuf}.
Java binding of the MPI operation {\em MPI\_UNPACK}.
The return value is the output value of {\em position}---the
inital value incremented by the number of bytes read.

\begin{verbatim}

int Comm.packSize(int incount, Datatype datatype)
\end{verbatim}
\begin{arguments}
{\tt incount}  & number of items in input buffer \\
{\tt datatype} & datatype of each item in input buffer \\
\\
{\em returns:} & upper bound on size of packed message
\end{arguments}
Returns an upper bound on the increment of {\tt position} effected
by {\tt pack}.
Java binding of the MPI operation {\em MPI\_PACK\_SIZE}.
{\em It is an error to call this function if the base type of {\tt datatype}
is {\tt MPI.OBJECT}}.

\newpage

\subsection{Collective Communication}

\subsubsection{Introduction and Overview}

In general the Java binding of collective communication operations will
realize the MPI functions as members of the {\tt IntraComm} class.

\subsubsection{Communicator argument}

No special issues for Java binding.

\subsubsection{Barrier synchronization}

\begin{verbatim}

void Intracomm.barrier()
\end{verbatim}
A call to {\tt barrier} blocks the caller until all processes
in the group have called it.
Java binding of the MPI operation {\em MPI\_BARRIER}.

\subsubsection{Broadcast}

\begin{verbatim}
void Intracomm.bcast(Object buffer, int offset, int count,
                     Datatype datatype, int root)
\end{verbatim}
\begin{arguments}
{\tt buf}      & buffer array \\
{\tt offset}   & initial offset in buffer \\
{\tt count}    & number of items in buffer \\
{\tt datatype} & datatype of each item in buffer \\
{\tt dest}     & rank of broadcast root
\end{arguments}
Broadcast a message from the process with rank {\tt root}
to all processes of the group.
Java binding of the MPI operation {\em MPI\_BCAST}.

\subsubsection{Gather}

\begin{verbatim}

void Intracomm.gather(Object sendbuf, int sendoffset,
                      int sendcount, Datatype sendtype,
                      Object recvbuf, int recvoffset,
                      int recvcount, Datatype recvtype, int root)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items to send \\
{\tt sendtype}   & datatype of each item in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcount}  & number of items in receive buffer \\
{\tt recvtype}   & datatype of each item in receive buffer \\
{\tt root}       & rank of receiving process
\end{arguments}
Each process sends the contents of its send buffer to the
root process.
Java binding of the MPI operation {\em MPI\_GATHER}.

\begin{verbatim}

void Intracomm.gatherv(Object sendbuf, int sendoffset,
                       int sendcount, Datatype sendtype,
                       Object recvbuf, int recvoffset,
                       int [] recvcounts, int [] displs,
                       Datatype recvtype, int root)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items to send \\
{\tt sendtype}   & datatype of each item in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcounts} & number of elements received from each process \\
{\tt displs}     & displacements at which to place incoming data \\
{\tt recvtype}   & datatype of each item in receive buffer \\
{\tt root}       & rank of receiving process
\end{arguments}
Extends functionality of {\tt gather} by allowing varying
counts of data from each process.
Java binding of the MPI operation {\em MPI\_GATHERV}.
The sizes of arrays {\tt recvcounts} and {\tt displs} should be the
size of the group.  Entry $i$ of {\tt displs} specifies the
displacement relative to element {\tt recvoffset} of {\tt recvbuf}
at which to place incoming data.  Note that if {\tt recvtype} is a
derived data type, elements of {\tt displs} are in units of the derived
type extent, (unlike {\tt recvoffset}, which is a direct index into the
buffer array).

\subsubsection{Scatter}

\begin{verbatim}

void Intracomm.scatter(Object sendbuf, int sendoffset,
                       int sendcount, Datatype sendtype,
                       Object recvbuf, int recvoffset,
                       int recvcount, Datatype recvtype,
                       int root)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items sent to each process \\
{\tt sendtype}   & datatype of send buffer items \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcount}  & number of items in receive buffer \\
{\tt recvtype}   & datatype of receive buffer items \\
{\tt root}       & rank of sending process
\end{arguments}
Inverse of the operation {\tt gather}.
Java binding of the MPI operation {\em MPI\_SCAT\-TER}.

\begin{verbatim}

void Intracomm.scatterv(Object sendbuf, int sendoffset,
                        int [] sendcounts, int [] displs,
                        Datatype sendtype,
                        Object recvbuf, int recvoffset,
                        int recvcount, Datatype recvtype,
                        int root)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcounts} & number of items sent to each process \\
{\tt displs}     & displacements from which to take outgoing data \\
{\tt sendtype}   & datatype of each item in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcount}  & number of elements in receive buffer \\
{\tt recvtype}   & datatype of receive buffer items \\
{\tt root}       & rank of sending process
\end{arguments}
Inverse of the operation {\tt gatherv}.
Java binding of the MPI operation {\em MPI\_\-SCATTERV}.

\subsubsection{Gather-to-all}

\begin{verbatim}

void Intracomm.allgather(Object sendbuf, int sendoffset,
                         int sendcount, Datatype sendtype,
                         Object recvbuf, int recvoffset,
                         int recvcount, Datatype recvtype)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items sent to each process \\
{\tt sendtype}   & datatype of send buffer items \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcount}  & number of items in receive buffer \\
{\tt recvtype}   & datatype of receive buffer items
\end{arguments}
Similar to {\tt gather}, but all processes receive the result.
Java binding of the MPI operation {\em MPI\_ALLGATHER}.

\begin{verbatim}

void Intracomm.allgatherv(Object sendbuf, int sendoffset,
                          int sendcount, Datatype sendtype,
                          Object recvbuf, int recvoffset,
                          int [] recvcounts, int [] displs,
                          Datatype recvtype)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items to send \\
{\tt sendtype}   & datatype of each item in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcounts} & number of elements received from each process \\
{\tt displs}     & displacements at which to place incoming data \\
{\tt recvtype}   & datatype of each item in receive buffer \\
\end{arguments}
Similar to {\tt gatherv}, but all processes receive the result.
Java binding of the MPI operation {\em MPI\_GATHERV}.

\subsubsection{All-to-All Scatter/Gather}

\begin{verbatim}

void Intracomm.alltoall(Object sendbuf, int sendoffset,
                        int sendcount, Datatype sendtype,
                        Object recvbuf, int recvoffset,
                        recvcount, Datatype recvtype)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcount}  & number of items sent to each process \\
{\tt sendtype}   & datatype of send buffer items \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcount}  & number of items received from any process \\
{\tt recvtype}   & datatype of receive buffer items
\end{arguments}
Extension of {\tt allgather} to the case where each process sends
distinct data to each of the receivers.
Java binding of the MPI operation {\em MPI\_ALLTOALL}.

\begin{verbatim}

void Intracomm.alltoallv(Object sendbuf, int sendoffset,
                         int [] sendcount, int [] sdispls,
                         Datatype sendtype,
                         Object recvbuf, int recvoffset,
                         int [] recvcount, int [] rdispls,
                         Datatype recvtype)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt sendcounts} & number of items sent to each process \\
{\tt sdispls}    & displacements from which to take outgoing data \\
{\tt sendtype}   & datatype of each item in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcounts} & number of elements received from each process \\
{\tt rdispls}    & displacements at which to place incoming data \\
{\tt recvtype}   & datatype of each item in receive buffer \\
\end{arguments}
Adds flexibility to {\tt alltoall}: location of data for send is
specified by {\tt sdispls} and location to place data on receive
side is specified by {\tt rdispls}.
Java binding of the MPI operation {\em MPI\_ALLTOALLV}.

\subsubsection{Global Reduction Operations}

\begin{verbatim}

void Intracomm.reduce(Object sendbuf, int sendoffset,
                      Object recvbuf, int recvoffset,
                      int count, Datatype datatype,
                      Op op, int root)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt count}      & number of items in send buffer \\
{\tt datatype}   & data type of each item in send buffer \\
{\tt op}         & reduce operation \\
{\tt dest}       & rank of root process
\end{arguments}
Combine elements in input buffer of each process using the reduce operation,
and return the combined value in the output buffer of the root process.
Java binding of the MPI operation {\em MPI\_REDUCE}.

The predefined operations are available in Java as {\tt MPI.MAX}, {\tt
MPI.MIN}, {\tt MPI.\-SUM}, {\tt MPI.PROD}, {\tt MPI.LAND}, {\tt
MPI.BAND}, {\tt MPI.LOR}, {\tt MPI.BOR}, {\tt MPI.LXOR}, {\tt
MPI.BXOR}, {\tt MPI.\-MINLOC} and {\tt MPI.MAXLOC}.

The handling of {\em MINLOC} and {\em MAXLOC} is modelled on the
Fortran binding.  The extra predefined types {\tt MPI.SHORT2}, {\tt
MPI.INT2}, {\tt MPI.LONG2}, {\tt MPI.FLOAT2}, {\tt MPI.DOUBLE2}
describe pairs of Java numeric primitive types.

\begin{verbatim}

Op.Op(UserFunction function, boolean commute)
\end{verbatim}
\begin{arguments}
{\tt function}   & user defined function \\
{\tt commute}    & {\tt true} if commutative, {\tt false} otherwise
\end{arguments}
Bind a user-defined global reduction operation to an {\tt Op}
object.
Java binding of the MPI operation {\em MPI\_OP\_CREATE}.
The interface {\tt UserFunction} is defined by
\small
\begin{verbatim}
  interface UserFunction {
    public abstract void call(Object invec, int inoffset,
                              Object inoutvec, int inoutoffset,
                              int count, Datatype datatype) ;
  }
\end{verbatim}
\normalsize
To define a new operation, the programmer should define a class
that implements the {\tt call} method of {\tt UserFunction},
then pass an object from this class to the {\tt Op} constructor.  The
{\tt UserFunction.call} method plays exactly the same role as the
{\tt function} argument in the standard bindings of MPI.  The actual
arguments {\tt invec} and {\tt outvec} passed to {\tt call} will be
arrays with {\tt count} elements of the type specified in the {\tt
datatype} argument.   The user-defined {\tt call} method combines these
in the way specified in the MPI standard.
%\begin{implementors}
%There is a problem for JNI-based wrapper implementations here: how does
%the C code inside the wrapper persuade the MPI reduction operation to
%invoke the Java method, {\tt call}?  Some C {\tt MPI\_User\_function}
%must dispatch the {\tt call} method of a Java object of type {\tt
%UserFunction}.  The difficulty is in creating a C function at runtime
%that will do this.  A generic C function can reference a Java object
%through a reference held in some static variable.  But if more than one
%user defined operation is needed, how does a particular invocation find
%the proper Java function object?  One possibility is to assume a
%constant bound, $N$, on the number of permitted user-defined
%operations.  The interface predefines $N$ different {\tt
%MPI\_User\_function} functions, invoking {\tt call} methods through $N$
%different static variables.  These {\tt
%MPI\_User\_function}/static-variable pairs are managed as a pool.
%Calling the {\tt Op} constructor involves copying a reference to its {\tt
%function} object argument into one of the static variables in the pool
%and passing the corresponding {\tt MPI\_User\_function} to the physical
%{\tt MPI\_Op\_create}.
%\end{implementors}

\begin{verbatim}

void Op.finalize()
\end{verbatim}
Destructor.  Java binding of the MPI operation {\em MPI\_OP\_FREE}.

\begin{verbatim}

void Intracomm.allreduce(Object sendbuf, int sendoffset,
                         Object recvbuf, int recvoffset,
                         int count, Datatype datatype,
                         Op op)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt count}      & number of items in send buffer \\
{\tt datatype}   & data type of each item in send buffer \\
{\tt op}         & reduce operation \\
\end{arguments}
Same as {\tt reduce} except that the result appears in receive
buffer of all process in the group.
Java binding of the MPI operation {\em MPI\_ALLREDUCE}.

\subsubsection{Reduce-Scatter}

\begin{verbatim}

void Intracomm.reduceScatter(Object sendbuf, int sendoffset,
                             Object recvbuf, int recvoffset,
                             int [] recvcounts,
                             Datatype datatype, Op op)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt recvcounts} & numbers of result elements distributed to each process \\
{\tt datatype}   & data type of each item in send buffer \\
{\tt op}         & reduce operation \\
\end{arguments}
Combine elements in input buffer of each process using the reduce operation,
and scatter the combined values over the output buffers of the processes.
Java binding of the MPI operation {\em MPI\_REDUCE\_SCATTER}.

\subsubsection{Scan}

\begin{verbatim}

void Intracomm.scan(Object sendbuf, int sendoffset,
                    Object recvbuf, int recvoffset,
                    int count, Datatype datatype,
                    Op op)
\end{verbatim}
\begin{arguments}
{\tt sendbuf}    & send buffer array \\
{\tt sendoffset} & initial offset in send buffer \\
{\tt recvbuf}    & receive buffer array \\
{\tt recvoffset} & initial offset in receive buffer \\
{\tt count}      & number of items in input buffer \\
{\tt datatype}   & data type of each item in input buffer \\
{\tt op}         & reduce operation \\
\end{arguments}
Perform a prefix reduction on data distributed across the group.
Java binding of the MPI operation {\em MPI\_SCAN}.

\subsubsection{Correctness}

No special issues for Java binding.

\newpage

\subsection{Groups, Contexts and Communicators}

\subsubsection{Introduction}

No special issues for Java binding.

\subsubsection{Basic Concepts}

The constant {\em MPI\_GROUP\_EMPTY} is available as {\tt MPI.GROUP\_EMPTY}.
The constants {\em MPI\_COMM\_WORLD}, {\em MPI\_COMM\_SELF} are available as
{\tt MPI.COMM\_\-WORLD}, {\tt MPI.COMM\_SELF}.


\subsubsection{Group Management}

\begin{verbatim}

int Group.size()
\end{verbatim}
\begin{arguments}
{\em returns:} & number of processors in the group
\end{arguments}
Size of group.
Java binding of the MPI operation {\em MPI\_GROUP\_SIZE}.

\begin{verbatim}

int Group.rank()
\end{verbatim}
\begin{arguments}
{\em returns:} & rank of the calling process in the group
\end{arguments}
Rank of this process in group.
Java binding of the MPI operation {\em MPI\_\-GROUP\_RANK}.
Result value is {\tt MPI.\-UNDEFINED} if this process is not
a member of the group.

\begin{verbatim}

int [] Group.translateRanks(Group group1, int [] ranks1)
\end{verbatim}
\begin{arguments}
{\tt group1}   & another group \\
{\tt ranks1}   & array of valid ranks in {\tt group1} \\
\\
{\em returns:} & array of corresponding ranks in this group
\end{arguments}
Translate ranks within another group to ranks within this group.
Java binding of the MPI operation {\em MPI\_GROUP\_TRANSLATE\_RANKS}.
Result elements are {\tt MPI.UNDEFINED} where no correspondence exists.

\begin{verbatim}

static int Group.compare(Group group1, Group group2)
\end{verbatim}
\begin{arguments}
{\tt group1}   & first group \\
{\tt group2}   & second group \\
\\
{\em returns:} & result
\end{arguments}
Compare two groups.
Java binding of the MPI operation {\em MPI\_GROUP\_COMP\-ARE}.
{\tt MPI.IDENT} results if the group members and group order are
exactly the same in both groups.  {\tt MPI.SIMILAR} results if
the group members are the same but the order is different.
{\tt MPI.UNEQUAL} results otherwise.

\begin{verbatim}

Group Comm.group()
\end{verbatim}
\begin{arguments}
{\em returns:} & group corresponding to this communicator
\end{arguments}
Return group associated with a communicator.
Java binding of the MPI operation {\em MPI\_COMM\_GROUP}.

\begin{verbatim}

static Group Group.union(Group group1, Group group2)
\end{verbatim}
\begin{arguments}
{\tt group1}   & first group \\
{\tt group2}   & second group \\
\\
{\em returns:} & union group
\end{arguments}
Set union of two groups.
Java binding of the MPI operation {\em MPI\_GROUP\_UN\-ION}.

\begin{verbatim}

static Group Group.intersection(Group group1, Group group2)
\end{verbatim}
\begin{arguments}
{\tt group1}   & first group \\
{\tt group2}   & second group \\
\\
{\em returns:} & intersection group
\end{arguments}
Set intersection of two groups.
Java binding of the MPI operation {\em MPI\_GRO\-UP\_INTERSECTION}.

\begin{verbatim}

static Group Group.difference(Group group1, Group group2)
\end{verbatim}
\begin{arguments}
{\tt group1}   & first group \\
{\tt group2}   & second group \\
\\
{\em returns:} & difference group
\end{arguments}
Result contains all elements of the first group that are not in the
second group.
Java binding of the MPI operation {\em MPI\_GROUP\_DIFFERENCE}.

\begin{verbatim}

Group Group.incl(int [] ranks)
\end{verbatim}
\begin{arguments}
{\tt ranks}    & ranks from this group to appear in new group \\
\\
{\em returns:} & new group
\end{arguments}
Create a subset group including specified processes.
Java binding of the MPI operation {\em MPI\_GROUP\_INCL}.

\begin{verbatim}

Group Group.excl(int [] ranks)
\end{verbatim}
\begin{arguments}
{\tt ranks}    & ranks from this group {\em not} to appear in new group \\
\\
{\em returns:} & new group
\end{arguments}
Create a subset group excluding specified processes.
Java binding of the MPI operation {\em MPI\_GROUP\_EXCL}.

\begin{verbatim}

Group Group.rangeIncl(int [] [] ranges)
\end{verbatim}
\begin{arguments}
{\tt ranges}   & array of integer triplets \\
\\
{\em returns:} & new group
\end{arguments}
Create a subset group including processes specified
by strided intervals of ranks.
Java binding of the MPI operation {\em MPI\_GROUP\_RANGE\_INCL}.
The triplets are of the form $(\mbox{first rank}, \mbox{last rank},
\mbox{stride})$ indicating ranks in this group to be included in
the new group.  The size of the first dimension of {\tt ranges}
is the number of triplets.  The size of the second dimension is 3.

\begin{verbatim}

Group Group.rangeExcl(int [] [] ranges)
\end{verbatim}
\begin{arguments}
{\tt ranges}   & array of integer triplets \\
\\
{\em returns:} & new group
\end{arguments}
Create a subset group excluding processes specified
by strided intervals of ranks.
Java binding of the MPI operation {\em MPI\_GROUP\_RANGE\_EXCL}.
Triplet array is defined as for {\tt rangeIncl}, the ranges
indicating ranks in this group to be excluded from
the new group.

\begin{verbatim}

void Group.finalize()
\end{verbatim}
Destructor.  Java binding of the MPI operation {\em MPI\_GROUP\_FREE}.

\subsubsection{Communicator Management}

\begin{verbatim}

int Comm.size()
\end{verbatim}
\begin{arguments}
{\em returns:} & number of processors in the group of this communicator
\end{arguments}
Size of group of this communicator.
Java binding of the MPI operation {\em MPI\_\-COMM\_SIZE}.

\begin{verbatim}

int Comm.rank()
\end{verbatim}
\begin{arguments}
{\em returns:} & rank of the calling process in the group of this communicator
\end{arguments}
Rank of this process in group of this communicator.
Java binding of the MPI operation {\em MPI\_COMM\_RANK}.

\begin{verbatim}

static int Comm.compare(Comm comm1, Comm comm2)
\end{verbatim}
\begin{arguments}
{\tt comm1}   & first communicator \\
{\tt comm2}   & second communicator \\
\\
{\em returns:} & result
\end{arguments}
Compare two communicators.
Java binding of the MPI operation {\em MPI\_COMM\_\-COMPARE}.
{\tt MPI.IDENT} results if the {\tt comm1} and {\tt comm2}
are references to the same object (ie, if {\tt comm1 == comm2}).
{\tt MPI.CONGRUENT} results if
the underlying groups are identical but the communicators
differ by context.
{\tt MPI.SIMILAR} results if
the underlying groups are similar but the communicators
differ by context.
{\tt MPI.UNEQUAL} results otherwise.

\begin{verbatim}

Object Comm.clone()
\end{verbatim}
\begin{arguments}
{\em returns:} & copy of this communicator
\end{arguments}
Duplicate this communicator.
Java binding of the MPI operation {\em MPI\_COMM\_\-DUP}.
The new communicator is ``congruent'' to the old one,
but has a different context.
\begin{rationale}
The decision to use the standard Java {\tt clone} method means the
static result type must be {\tt Object}.  The dynamic type will be that
of the {\tt Comm} subclass of the parent.  MPI-defined and user-defined
subclasses of {\tt Comm} will generally override {\tt clone} to ensure
all relevant attributes are copied.
\end{rationale}

\begin{verbatim}

Intracomm Intracomm.create(Group group)
\end{verbatim}
\begin{arguments}
{\tt group}    & group which is a subset of the group of this communicator \\
\\
{\em returns:} & new communicator
\end{arguments}
Create a new communicator.
Java binding of the MPI operation {\em MPI\_COMM\_\-CREATE}.

\begin{verbatim}

Intracomm Intracomm.split(int color, int key)
\end{verbatim}
\begin{arguments}
{\tt color}    & control of subset assignment \\
{\tt key}      & control of rank assignment \\
\\
{\em returns:} & new communicator
\end{arguments}
Partition the group associated with this communicator and create
a new communicator within each subgroup.
Java binding of the MPI operation {\em MPI\_COMM\_\-SPLIT}.

\begin{verbatim}

void Comm.free()
\end{verbatim}
Destroy this communicator.
Java binding of the MPI operation {\em MPI\_COMM\_\-FREE}.
\begin{rationale}
An explicitly called {\tt free} method is required rather than an
implicitly called {\tt finalize} method, because {\em MPI\_COMM\_FREE}
is a collective operations.  We cannot assume that
the Java garbage collector will call a {\tt finalize} method
synchronously on all processors.
\end{rationale}

\subsubsection{Motivating Examples}

No special issues for Java binding.

\subsubsection{Inter-Communication}

\begin{verbatim}

boolean Comm.testInter()
\end{verbatim}
\begin{arguments}
{\em returns:} & {\tt true} if this is an inter-communicator,
                 {\tt false} otherwise
\end{arguments}
Test if this communicator is an inter-communicator.
Java binding of the MPI operation {\em MPI\_COMM\_TEST\_INTER}.

\begin{verbatim}

int Intercomm.remoteSize()
\end{verbatim}
\begin{arguments}
{\em returns:} & number of process in remote group of this communicator
\end{arguments}
Size of remote group.
Java binding of the MPI operation {\em MPI\_COMM\_REM\-OTE\_SIZE}.

\begin{verbatim}

Group Intercomm.remoteGroup()
\end{verbatim}
\begin{arguments}
{\em returns:} & remote group of this communicator
\end{arguments}
Return the remote group.
Java binding of the MPI operation {\em MPI\_COMM\_\-REMOTE\_GROUP}.

\begin{verbatim}

Intercomm Comm.createIntercomm(Comm localComm, int localLeader,
                               int remoteLeader, int tag)
\end{verbatim}
\begin{arguments}
{\tt localComm}    & local intra-communicator \\
{\tt localLeader}  & rank of local group leader in {\tt localComm} \\
{\tt remoteLeader} & rank of remote group leader in this communictor \\
{\tt tag}          & ``safe'' tag \\
\\
{\em returns:}     & new inter-communicator
\end{arguments}
Create an inter-communicator.
Java binding of the MPI operation {\em MPI\_INTER\-COMM\_CREATE}.
\begin{rationale}
This operation is defined as a method on the ``peer communicator'',
making it analogous to a {\tt send} or {\tt recv} communication with the
remote group leader.
\end{rationale}

\begin{verbatim}

Intracomm Intercomm.merge(boolean high)
\end{verbatim}
\begin{arguments}
{\tt high}     & true if the local group has higher ranks in combined group \\
\\
{\em returns:} & new intra-communicator
\end{arguments}
Create an intra-communicator from the union of the two groups in
the inter-communicator.
Java binding of the MPI operation {\em MPI\_INTERCOMM\_MER\-GE}.

\subsubsection{Caching}

It is assumed that, to achieve the effect of caching attributes in
user-customized communicators, programmers will create subclasses of
the library-defined communicator classes with suitable additional
fields.  These fields may be copied or deleted by suitably overridden
{\tt clone} and {\tt finalize} methods.

Hence the only ``caching'' operation surviving here is the binding of
{\em MPI\_\-ATTR\_GET}, which is needed to access values of attributes
predefined by the implementation.  According the standard, the key
values for such attributes include {\tt MPI.TAG\_UB}, {\tt MPI.HOST},
{\tt MPI.IO} and {\tt MPI.WTIME\_IS\_GLOBAL}.

\begin{verbatim}

Object Comm.attrGet(int keyval)
\end{verbatim}
\begin{arguments}
{\tt keyval}   & one of the key values predefined by the implementation \\
\\
{\em returns:} & attribute value
\end{arguments}
Retrieves attribute value by key.
Java binding of the MPI operation {\em MPI\_AT\-TR\_GET}.
\begin{implementors}
If the attribute value is naturally represented by primitive type such
as an {\tt int}, this function may wrap the return value as an object such
as an {\tt Integer}.
\end{implementors}

\newpage

\subsection{Process Topologies}

\subsubsection{Introduction}

Communicators with Cartesian or graph topologies will be realized as
instances of the subclasses {\tt Cartcomm} or {\tt Graphcomm}, respectively
of {\tt Intracomm}.

\subsubsection{Virtual Topologies}

No special issues for Java binding.

\subsubsection{Embedding in MPI}

No special issues for Java binding.

\subsubsection{Overview of the Functions}

No special issues for Java binding.

\subsubsection{Topology Constructors}

\begin{verbatim}

Cartcomm Intracomm.createCart(int [] dims, boolean [] periods,
                              boolean reorder)
\end{verbatim}
\begin{arguments}
{\tt dims}     & the number of processes in each dimension \\
{\tt periods}  & {\tt true} if grid is periodic, {\tt false} if not,
                 in each dimension \\
{\tt reorder}  & {\tt true} if ranking may be reordered, {\tt false} if not \\
\\
{\em returns:} & new Cartesian topology communicator
\end{arguments}
Create a Cartesian topology communicator whose group is a subset
of the group of this communicator.  Java binding of the MPI operation
{\em MPI\_CART\_CREATE}.  The number of dimensions of the Cartesian
grid is taken to be the size of the {\tt dims} argument.  The array {\tt
periods} must be the same size.

\begin{verbatim}

static Cartcomm.dimsCreate(int nnodes, int [] dims)
\end{verbatim}
\begin{arguments}
{\tt nnodes}   & number of nodes in a grid \\
{\tt dims}     & array specifying the number of nodes in each dimension
\end{arguments}
Select a balanced distribution of processes per coordinate direction.
Java binding of the MPI operation {\em MPI\_DIMS\_CREATE}.
Number of dimensions is the size of is {\tt dims}.  Note that
{\tt dims} is an {\em inout} parameter.

\begin{verbatim}

Graphcomm Intracomm.createGraph(int [] index, int [] edges,
                                boolean reorder)
\end{verbatim}
\begin{arguments}
{\tt index}    & node degrees \\
{\tt edges}    & graph edges \\
{\tt reorder}  & {\tt true} if ranking may be reordered, {\tt false} if not \\
\\
{\em returns:} & new graph topology communicator
\end{arguments}
Create a graph topology communicator whose group is a subset
of the group of this communicator.
Java binding of the MPI operation {\em MPI\_GRAPH\_CREATE}.
The number of nodes in the graph, {\em nnodes}, is taken
to be size of the {\tt index} argument.  The size of array {\tt
edges} must be $\mbox{\tt index}[\mbox{\em nnodes} - 1]$.

\begin{verbatim}

int Comm.topoTest()
\end{verbatim}
\begin{arguments}
{\em returns:} & topology type of communicator
\end{arguments}
Returns the type of topology associated with the communicator.
Java binding of the MPI operation {\em MPI\_TOPO\_TEST}.
The return value will be one of
{\tt MPI.GRAPH}, {\tt MPI.CART} or {\tt MPI.UNDEFINED}.

\begin{verbatim}

GraphParms Graphcomm.get()
\end{verbatim}
\begin{arguments}
{\em returns:} & object defining node degress and edges of graph
\end{arguments}
Returns graph topology information.
Java binding of the MPI operations {\em MPI\_\-GRAPHDIMS\_GET} and
{\em MPI\_GRAPH\_GET}.
The class of the returned object is
\small
\begin{verbatim}
  public class GraphParms {
    public int [] index ;
    public int [] edges ; 
  }
\end{verbatim}
\normalsize
The number of nodes and number of edges can be extracted
from the sizes of the {\tt index} and {\tt edges} arrays.

\begin{verbatim}

CartParms Cartcomm.get()
\end{verbatim}
\begin{arguments}
{\em returns:} & object containing dimensions, periods and local coordinates
\end{arguments}
Returns Cartesian topology information.
Java binding of the MPI operations {\em MPI\_CARTDIM\_GET} and
{\em MPI\_CART\_GET}.
The class of the returned object is
\small
\begin{verbatim}
  public class CartParms {
    public int [] dims ;
    public booleans [] periods ;
    public int [] coords ; 
  }
\end{verbatim}
\normalsize
The number of dimensions can be obtained from the size of (eg)
the {\tt dims} array.
\begin{rationale}
The inquiries {\em MPI\_GRAPHDIMS\_GET}, {\em MPI\_GRAPH\_GET}, {\em
MPI\_CARTDIM\_GET}, and {\em MPI\_CART\_GET} are unusual in returning
multiple independent values from single calls.  This is a
problem in Java.  The Java binding could split these inquiries into
several independent ones, but that would complicate JNI-based wrapper
implementations.  Hence we introduced the auxilliary classes {\tt
GraphParms} and {\tt CartParms} to hold multiple results.
\end{rationale}

\begin{verbatim}

int Cartcomm.rank(int [] coords)
\end{verbatim}
\begin{arguments}
{\tt coords}   & Cartesian coordinates of a process \\
\\
{\em returns:} & rank of the specified process
\end{arguments}
Translate logical process coordinates to process rank.
Java binding of the MPI operation {\em MPI\_CART\_RANK}.

\begin{verbatim}

int [] Cartcomm.coords(int rank)
\end{verbatim}
\begin{arguments}
{\tt coords}   & rank of a process \\
\\
{\em returns:} & Cartesian coordinates of the specified process
\end{arguments}
Translate process rank to logical process coordinates.
Java binding of the MPI operation {\em MPI\_CART\_COORDS}.

\begin{verbatim}

int [] Graphcomm.neighbours(int rank)
\end{verbatim}
\begin{arguments}
{\tt coords}   & rank of a process in the group of this communicator \\
\\
{\em returns:} & array of ranks of neighbouring processes to one specified
\end{arguments}
Provides adjacency information for general graph topology.
Java binding of the MPI operations {\em MPI\_GRAPH\_NEIGHBOURS\_COUNT}
and {\em MPI\_GRAPH\_\-NE\-IGHBOURS}.
The number of neighbours can be extracted from the size of the result.

\begin{verbatim}

ShiftParms Cartcomm.shift(int direction, int disp)
\end{verbatim}
\begin{arguments}
{\tt direction} & coordinate dimension of shift \\
{\tt disp}      & displacement \\
\\
{\em returns:}  & object containing ranks of source and destination processes
\end{arguments}
Compute source and destination ranks for ``shift'' communication.
Java binding of the MPI operation {\em MPI\_CART\_SHIFT}.
The class of the returned object is
\small
\begin{verbatim}
  public class ShiftParms {
    public int rankSource ;
    public int rankDest ; 
  }
\end{verbatim}
\normalsize

\begin{verbatim}

Cartcomm Cartcomm.sub(boolean [] remainDims)
\end{verbatim}
\begin{arguments}
{\tt remainDims} & by dimension, {\tt true} if dimension is to be kept,
                   {\tt false} otherwise \\
\\
{\em returns:}   & communicator containing subgrid including this process
\end{arguments}
Partition Cartesian communicator into subgroups of lower dimension.
Java binding of the MPI operation {\em MPI\_CART\_SUB}.

\begin{verbatim}

int Cartcomm.map(int [] dims, boolean [] periods)
\end{verbatim}
\begin{arguments}
{\tt dims}     & the number of processes in each dimension \\
{\tt periods}  & {\tt true} if grid is periodic, {\tt false} if not, in each
                 dimension \\
\\
{\em returns:} & reordered rank of calling process
\end{arguments}
Compute an optimal placement.
Java binding of the MPI operation {\em MPI\_\-CART\_MAP}.
The number of dimensions is taken to be size of the {\tt dims} argument.

\begin{verbatim}

int Graphcomm.map(int [] index, int [] edges)
\end{verbatim}
\begin{arguments}
{\tt index}    & node degrees \\
{\tt edges}    & graph edges \\
\\
{\em returns:} & reordered rank of calling process
\end{arguments}
Compute an optimal placement.
Java binding of the MPI operation {\em MPI\_\-GRAPH\_MAP}.
The number of nodes is taken to be size of the {\tt index} argument.

\newpage

\subsection{MPI Environmental Management}

\subsubsection{Implementation information}

The constants {\em MPI\_TAG\_UB}, {\em MPI\_HOST} and {\em MPI\_IO}
are available as
{\tt MPI.\-TAG\_UB}, {\tt MPI.HOST}, {\tt MPI.IO}.

\begin{verbatim}

static String MPI.getProcessorName()
\end{verbatim}
\begin{arguments}
{\em returns:} & A unique specifier for the actual node.
\end{arguments}
Returns the name of the processor on which it is called.
Java binding of the MPI operation {\em MPI\_GET\_PROCESSOR\_NAME}.

\subsubsection{Error handling}

The constants {\em MPI\_ERRORS\_ARE\_FATAL}, {\em MPI\_ERRORS\_RETURN}
are available as
{\tt MPI.ERRORS\_ARE\_FATAL}, {\tt MPI.ERRORS\_RETURN}.

If the effective error handler is {\em MPI\_ERRORS\_RETURN}, the wrapper
codes will throw appropriate Java exceptions (see section \ref{errorCodes}).

For now, we omit a specification of Java interface for creating new MPI error
handlers, because the detailed interface of the handler function
depends on unstandardized features of the MPI implementation.

\begin{verbatim}

static void Comm.errorhandlerSet(Errhandler errhandler)
\end{verbatim}
\begin{arguments}
{\tt errhandler} & new MPI error handler for communicator
\end{arguments}
Associates a new error handler with communicator at the calling process.
Java binding of the MPI operation {\em MPI\_ERRORHANDLER\_SET}.

\begin{verbatim}

static Errhandler Comm.errorhandlerGet()
\end{verbatim}
\begin{arguments}
{\em returns:} & MPI error handler currently associated with communicator
\end{arguments}
Returns the error handler currently associated with the communicator.
Java binding of the MPI operation {\em MPI\_ERRORHANDLER\_GET}.


\subsubsection{Error codes and classes\label{errorCodes}}

The {\tt Exception} subclasses
\begin{verbatim}
  MPIErrBuffer
  MPIErrCount
  MPIErrType
  MPIErrTag
  MPIErrComm
  MPIErrRank
  MPIErrRequest
  MPIErrRoot
  MPIErrGroup
  MPIErrOp
  MPIErrTopology
  MPIErrDims
  MPIErrArg
  MPIErrUnknown
  MPIErrTruncate
  MPIErrOther
  MPIErrIntern
\end{verbatim}
correspond to the standard MPI error classes.

\subsubsection{Timers}

\begin{verbatim}

static double MPI.wtime()
\end{verbatim}
\begin{arguments}
{\em returns:} & elapsed wallclock time in seconds since some time in the past
\end{arguments}
Returns wallclock time.
Java binding of the MPI operation {\em MPI\_WTIME}.

\begin{verbatim}

static double MPI.wtick()
\end{verbatim}
\begin{arguments}
{\em returns:} & resolution of {\tt wtime} in seconds.
\end{arguments}
Returns resolution of timer.
Java binding of the MPI operation {\em MPI\_WTICK}.

\subsubsection{Startup}

\begin{verbatim}

static void MPI.init(String[] argv)
\end{verbatim}
\begin{arguments}
{\tt argv} & arguments to {\tt main} method.
\end{arguments}
Initialize MPI.
Java binding of the MPI operation {\em MPI\_INIT}.

\begin{verbatim}

static void MPI.finish()
\end{verbatim}
Finalize MPI.
Java binding of the MPI operation {\em MPI\_FINALIZE}.

\begin{verbatim}

static boolean MPI.initialized()
\end{verbatim}
\begin{arguments}
{\em returns:} & {\tt true} if {\tt init} has been called, {\tt false}
                 otherwise.
\end{arguments}
Test if MPI has been initialized.
Java binding of the MPI operation {\em MPI\_INIT\-IALIZED}.

\begin{verbatim}

void Comm.abort(int errorcode)
\end{verbatim}
\begin{arguments}
{\tt errorcode} & error code for Unix or POSIX environments
\end{arguments}
Abort MPI.
Java binding of the MPI operation {\em MPI\_ABORT}.

\newpage

\section{Full public interface of classes}

Section names appearing in comments refer to the preceding appendix.
Specification of the methods immediately following those comments
should be found in the referenced section.

(Notably missing from this draft are {\tt throws} clauses for exceptions.)

\newpage

\subsection{\tt MPI}

\small
\begin{verbatim}
public class MPI {
  public static Intracomm COMM_WORLD;

  public static Datatype BYTE, CHAR, SHORT, BOOLEAN, INT, LONG,
                         FLOAT, DOUBLE, PACKED, LB, UB ;

  public static int ANY_SOURCE, ANY_TAG ;

  public static int PROC_NULL ;

  public static int BSEND_OVERHEAD ;

  public static int UNDEFINED ;

  public static Op MAX, MIN, SUM, PROD, LAND, BAND,
                   LOR, BOR, LXOR, BXOR, MINLOC, MAXLOC ;

  public static Datatype SHORT2, INT2, LONG2, FLOAT2, DOUBLE2 ;

  public static Group GROUP_EMPTY ;

  public static Comm COMM_SELF ;

  public static int IDENT, CONGRUENT, SIMILAR, UNEQUAL ;

  public static int GRAPH, CART ;

  public static ErrHandler ERRORS_ARE_FATAL, ERRORS_RETURN ;

  public static int TAG_UB, HOST, IO ;


  // Buffer allocation and usage

  public static void bufferAttach(byte [] buffer) {...}

  public static byte [] bufferDetach() {...}


  // Environmental Management

  public static void init(String[] argv) {...}

  public static void finish() {...}

  public static String getProcessorName() {...}

  public static double wtime() {...}

  public static double wtick() {...}

  public static boolean initialized() {...}

  ...
}
\end{verbatim}
\normalsize

\small
\begin{verbatim}
public class Errhandler {
  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{\tt Comm}

\small
\begin{verbatim}
public class Comm {

  // Communicator Management

  public int size() {...}

  public int rank() {...}

  public Group group() {...}  // (see ``Group management'')

  public static int compare(Comm comm1, Comm comm2) {...}

  public Object clone() {...}

  public void free() {...}


  // Inter-communication

  public boolean testInter() {...}

  public Intercomm createIntercomm(Comm localComm, int localLeader,
                                   int remoteLeader, int tag) {...}


  // Caching

  public Object attrGet(int keyval) {...}


  // Blocking Send and Receive operations

  public void send(Object buf, int offset, int count, 
                   Datatype datatype, int dest, int tag) {...}

  public Status recv(Object buf, int offset, int count, 
                     Datatype datatype, int source, int tag) {...}


  // Communication Modes

  public void bsend(Object buf, int offset, int count, 
                    Datatype datatype, int dest, int tag) {...}

  public void ssend(Object buf, int offset, int count, 
                    Datatype datatype, int dest, int tag) {...}

  public void rsend(Object buf, int offset, int count, 
                    Datatype datatype, int dest, int tag) {...}


  // Nonblocking communication

  public Request isend(Object buf, int offset, int count, 
                       Datatype datatype, int dest, int tag) {...}

  public Request ibsend(Object buf, int offset, int count, 
                        Datatype datatype, int dest, int tag) {...}

  public Request issend(Object buf, int offset, int count, 
                        Datatype datatype, int dest, int tag) {...}

  public Request irsend(Object buf, int offset, int count, 
                        Datatype datatype, int dest, int tag) {...}

  public Request irecv(Object buf, int offset, int count, 
                       Datatype datatype, int source, int tag) {...}


  // Probe and cancel

  public Status iprobe(int source, int tag) {...}

  public Status probe(int source, int tag) {...}


  // Persistent communication requests

  public Prequest sendInit(Object buf, int offset, int count, 
                           Datatype datatype, int dest, int tag) {...}

  public Prequest bsendInit(Object buf, int offset, int count, 
                            Datatype datatype, int dest, int tag) {...}

  public Prequest ssendInit(Object buf, int offset, int count, 
                            Datatype datatype, int dest, int tag) {...}

  public Prequest rsendInit(Object buf, int offset, int count, 
                            Datatype datatype, int dest, int tag) {...}

  public Prequest recvInit(Object buf, int offset, int count, 
                           Datatype datatype, int source, int tag) {...}


  // Send-receive

  public Status sendrecv(Object sendbuf, int sendoffset,
                         int sendcount, Datatype sendtype,
                         int dest, int sendtag,
                         Object recvbuf, int recvoffset,
                         int recvcount, Datatype recvtype,
                         int source, int recvtag) {...}

  public Status sendrecvReplace(Object buf, int offset,
                                int count, Datatype datatype,
                                int dest, int sendtag,
                                int source, int recvtag) {...}

  // Pack and unpack

  public int pack(Object inbuf, int offset, int incount, 
                  Datatype datatype,
                  byte [] outbuf, int position) {...}

  byte[] pack(Object inbuf, int offset, int incount,
              Datatype datatype) {...}

  public int unpack(byte [] inbuf, int position,
                    Object outbuf, int offset, int outcount, 
                    Datatype datatype) {...}

  public int packSize(int incount, Datatype datatype) {...}


  // Process Topologies

  int topoTest()


  // Environmental Management

  public static void errorhandlerSet(Errhandler errhandler) {...}

  public static Errhandler errorhandlerGet() {...}

  void abort(int errorcode) {...}

  ...
}
\end{verbatim}
\normalsize


\newpage

\subsection{{\tt Intracomm} and {\tt Intercomm}}

\small
\begin{verbatim}
public class Intracomm extends Comm {

  public Object clone() { ... }

  public Intracomm create(Group group) {...}

  public Intracomm split(int colour, int key) {...}

  // Collective communication

  public void barrier() {...}

  public void bcast(Object buffer, int offset, int count,
                    Datatype datatype, int root) {...}

  public void gather(Object sendbuf, int sendoffset,
                     int sendcount, Datatype sendtype,
                     Object recvbuf, int recvoffset,
                     int recvcount, Datatype recvtype, int root) {...}

  public void gatherv(Object sendbuf, int sendoffset,
                      int sendcount, Datatype sendtype,
                      Object recvbuf, int recvoffset,
                      int [] recvcount, int [] displs,
                      Datatype recvtype, int root) {...}

  public void scatter(Object sendbuf, int sendoffset,
                      int sendcount, Datatype sendtype,
                      Object recvbuf, int recvoffset,
                      int recvcount, Datatype recvtype, int root) {...}

  public void scatterv(Object sendbuf, int sendoffset,
                       int [] sendcount, int [] displs,
                       Datatype sendtype,
                       Object recvbuf, int recvoffset,
                       int recvcount, Datatype recvtype, int root) {...}

  public void allgather(Object sendbuf, int sendoffset,
                        int sendcount, Datatype sendtype,
                        Object recvbuf, int recvoffset,
                        int recvcount, Datatype recvtype) {...}

  public void allgatherv(Object sendbuf, int sendoffset,
                         int sendcount, Datatype sendtype,
                         Object recvbuf, int recvoffset,
                         int [] recvcounts, int [] displs,
                         Datatype recvtype) {...}

  public void alltoall(Object sendbuf, int sendoffset,
                       int sendcount, Datatype sendtype,
                       Object recvbuf, int recvoffset,
                       int recvcount, Datatype recvtype) {...}

  public void alltoallv(Object sendbuf, int sendoffset,
                        int [] sendcount, int [] sdispls,
                        Datatype sendtype,
                        Object recvbuf, int recvoffset,
                        int [] recvcount, int [] rdispls,
                        Datatype recvtype) {...}

  public void reduce(Object sendbuf, int sendoffset,
                     Object recvbuf, int recvoffset,
                     int count, Datatype datatype,
                     Op op, int root) {...}

  public void allreduce(Object sendbuf, int sendoffset,
                        Object recvbuf, int recvoffset,
                        int count, Datatype datatype,
                        Op op) {...}

  public void reduceScatter(Object sendbuf, int sendoffset,
                            Object recvbuf, int recvoffset,
                            int [] recvcounts, Datatype datatype,
                            Op op) {...}

  public void scan(Object sendbuf, int sendoffset,
                   Object recvbuf, int recvoffset,
                   int count, Datatype datatype,
                   Op op) {...}


  // Topology Constructors

  public Graphcomm createGraph(int [] index, int [] edges,
                               boolean reorder) {...}

  public Cartcomm createCart(int [] dims, boolean [] periods,
                             boolean reorder) {...}

  ...
}
\end{verbatim}
\normalsize

\small
\begin{verbatim}
public class Intercomm extends Comm {

  public Object clone() { ... }

  // Inter-communication

  public int remoteSize() {...}

  public Group remoteGroup() {...}

  public Intracomm merge(boolean high) {...}

  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{\tt Op}

\small
\begin{verbatim}
public class Op {
  Op(UserFunction function, boolean commute) {...}

  void finalize() {...}

  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{\tt Group}

\small
\begin{verbatim}
public class Group {

  // Group Management

  public int size() {...}

  public int rank() {...}

  public int [] translateRanks(Group group1, int [] ranks1) {...}

  public static int compare(Group group1, Group group2) {...}

  public static Group union(Group group1, Group group2) {...}

  public static Group intersection(Group group1, Group group2) {...}

  public static Group difference(Group group1, Group group2) {...}

  public Group incl(int [] ranks) {...}

  public Group excl(int [] ranks) {...}

  public Group rangeIncl(int [] [] ranges) {...}

  public Group rangeExcl(int [] [] ranges) {...}

  public void finalize() {...}

  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{\tt Status}

\small
\begin{verbatim}
public class Status {

  public int index ;


  // Blocking Send and Receive operations

  public int getCount(Datatype datatype) {...}

  public int getSource() {...}
  public int getTag() {...}

  // Nonblocking communication

  public int getIndex() {...}

  // Probe and Cancel

  public boolean testCancelled() {...}


  // Derived datatypes

  public int getElements(Datatype datatype) {...}

  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{{\tt Request} and {\tt Prequest}}

\small
\begin{verbatim}
public class Request {

  // Nonblocking communication

  public Status wait() {...}

  public Status test() {...}

  public Request() {...}

  public void finalize() {...}

  public boolean isVoid() {...}

  public static Status waitAny(Request [] arrayOfRequests) {...}

  public static Status testAny(Request [] arrayOfRequests) {...}

  public static Status [] waitAll(Request [] arrayOfRequests) {...}

  public static Status [] testAll(Request [] arrayOfRequests) {...}

  public static Status [] waitSome(Request [] arrayOfRequests) {...}

  public static Status [] testSome(Request [] arrayOfRequests) {...}


  // Probe and cancel

  public void cancel() {...}

  ...
}
\end{verbatim}
\normalsize

\small
\begin{verbatim}
public class Prequest extends Request {

  // Persistent communication requests

  public void start() {...}

  public static void startAll(Request [] arrayOfRequests) {...}

  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{\tt Datatype}

\small
\begin{verbatim}
public class Datatype {

  // Derived datatypes

  public Datatype contiguous(int count) {...}

  public Datatype vector(int count, int blocklength, int stride) {...}

  public Datatype hvector(int count, int blocklength, int stride) {...}

  public Datatype indexed(int [] arrayOfBlocklengths,
                          int [] arrayOfDisplacements) {...}

  public Datatype hindexed(int [] arrayOfBlocklengths,
                           int [] arrayOfDisplacements) {...}

  public static Datatype struct(int [] arrayOfBlocklengths,
                                int [] arrayOfDisplacements,
                                Datatype [] arrayOfTypes) {...}

  public int extent() {...}

  public int size() {...}

  public int lb() {...}

  public int ub() {...}

  public void commit() {...}

  public void finalize() {...}

  ...
}
\end{verbatim}
\normalsize

\newpage

\subsection{Classes for virtual topologies}

\small
\begin{verbatim}
public class Cartcomm extends Intracomm {

  public Object clone() { ... }

  // Topology Constructors

  static public dimsCreate(int nnodes, int [] dims) {...}

  public CartParms get() {...}

  public int rank(int [] coords) {...}

  public int [] coords(int rank) {...}

  public ShiftParms shift(int direction, int disp) {...}

  public Cartcomm sub(boolean [] remainDims) {...}

  public int map(int [] dims, boolean [] periods) {...}
}
\end{verbatim}
\normalsize

\small
\begin{verbatim}
public class CartParms {

  // Return type for Cartcomm.get()

  public int [] dims ;
  public booleans [] periods ;
  public int [] coords ; 
}
\end{verbatim}
\normalsize

\small
\begin{verbatim}
public class ShiftParms {

  // Return type for Cartcomm.shift()

  public int rankSource ;
  public int rankDest ; 
}
\end{verbatim}
\normalsize


\small
\begin{verbatim}
public class Graphcomm extends Intracomm {

  public Object clone() { ... }

  // Topology Constructors

  public GraphParms get() {...}

  public int [] neighbours(int rank) {...}

  public int map(int [] index, int [] edges) {...}
}
\end{verbatim}
\normalsize

\small
\begin{verbatim}
public class GraphParms {

  // Return type for Graphcomm.get()

  public int [] index ;
  public int [] edges ; 
}
\end{verbatim}
\normalsize

\newpage

\begin{flushleft}
\bibliography{position}
\end{flushleft}

\end{document}


