Full HTML for

Basic foilset Object serialization for Marshalling Data in a Java Interface to MPI

Given by Bryan Carpenter,Geoffrey C. Fox, Sung-Hoon Ko, Sang Lim at ACM Java Grande Meeting on June 12-13 99. Foils prepared July 6 99
Outside Index Summary of Material

Review the mpiJava wrapper interface.
Discuss incorporation of derived datatypes in the Java API, and limitations.
Adding object serialization at the API level.
Describe implementation using JDK serialization.
Benchmarks for naïve implementation.
Ongoing work: optimizing serialization.

Table of Contents for full HTML of Object serialization for Marshalling Data in a Java Interface to MPI

 Denote Foils where Image Critical
Denote Foils where Image has important information
Denote Foils where HTML is sufficient
1 Object Serialization for Marshalling Data in a Java Interface to MPI
2 Some issues in design of a Java API for MPI
3 Representing Message Buffers
4 Overview of this talk
5 Related projects
6 The mpiJava wrapper
7 mpiJava features.
8 Class hierarchy
9 Minimal mpiJava program
10 mpiJava implementation issues
11 mpiJava performance
12 MPI datatypes
13 Basic Datatypes
14 Derived datatypes
15 Restricted model
16 Limitations
17 Object datatypes
18 Automatic serialization
19 Implementation issues for Object datatypes
20 Modifications to mpiJava
21 Benchmarking mpiJava with serialization
22 Timing model
23 Platform
24 Non-shared memory: byte
25 Non-shared memory: float
26 Shared memory: byte
27 Shared memory: float
28 Parameters in timing model (microseconds)
29 Benchmark lessons
30 Improving serialization
31 Eliminating overheads of element serialization
32 Improved protocol
33 Customized output stream class
34 Customized input stream class
35 Non-shared memory: float (optimized in red)
36 Non-shared memory: byte (optimized in red)
37 Shared memory: float (optimized in red)
38 Shared memory: byte (optimized in red)
39 Comments
40 Conclusions

Outside Index Summary of Material

HTML version of Basic Foils prepared July 6 99

Foil 1 Object Serialization for Marshalling Data in a Java Interface to MPI

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Bryan Carpenter, Geoffrey Fox, Sung-Hoon Ko, and Sang Lim
www.npac.syr.edu/projects/pcrc/HPJava/mpiJava.html
HTML version of Basic Foils prepared July 6 99

Foil 2 Some issues in design of a Java API for MPI

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Class hierarchy. MPI is already object-based. "Standard" class hierarchy exists for C++.
Detailed argument lists for methods. Properties of Java language imply various superficial changes from C/C++.
Mechanisms for representing message buffers.
HTML version of Basic Foils prepared July 6 99

Foil 3 Representing Message Buffers

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Two natural options:
Follow the MPI standard route: derived datatypes describe buffers consisting of mixed primitive fields scattered in local memory.
Follow the Java standard route: automatic marshalling of complex structures through object serialization.
HTML version of Basic Foils prepared July 6 99

Foil 4 Overview of this talk

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Review the mpiJava wrapper interface.
Discuss incorporation of derived datatypes in the Java API, and limitations.
Adding object serialization at the API level.
Describe implementation using JDK serialization.
Benchmarks for naïve implementation.
Ongoing work: optimizing serialization.
HTML version of Basic Foils prepared July 6 99

Foil 5 Related projects

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
mpiJava (Syracuse)
JavaMPI (Getov et al, Westminster)
JMPI (MPI Software Technology)
MPIJ (Judd et al, Brigham Young)
jmpi (Dincer et al)
HTML version of Basic Foils prepared July 6 99

Foil 6 The mpiJava wrapper

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Implements a Java API for MPI suggested in late `97.
Builds on work on Java wrappers for MPI started at NPAC about a year earlier.
People: Bryan Carpenter, Yuh-Jye Chang, Xinying Li, Sung Hoon Ko, Guansong Zhang, Mark Baker, Sang Lim.
HTML version of Basic Foils prepared July 6 99

Foil 7 mpiJava features.

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Fully featured Java interface to MPI 1.1
Object-oriented API based on MPI 2 standard C++ interface
Initial implementation through JNI to native MPI
Comprehensive test suite translated from IBM MPI suite
Available for Solaris, Windows NT and other platforms
HTML version of Basic Foils prepared July 6 99

Foil 8 Class hierarchy

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 9 Minimal mpiJava program

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
import mpi.*
class Hello {
static public void main(String[] args) {
MPI.Init(args) ;
int myrank = MPI.COMM_WORLD.Rank() ;
if(myrank == 0) {
char[] message = "Hello, there".toCharArray() ;
MPI.COMM_WORLD.Send(message, 0, message.length, MPI.CHAR, 1, 99) ;
}
else {
char[] message = new char [20] ;
MPI.COMM_WORLD.Recv(message, 0, 20, MPI.CHAR, 0, 99) ;
System.out.println("received:" + new String(message) + ":") ;
}
MPI.Finalize() ;
}
}
HTML version of Basic Foils prepared July 6 99

Foil 10 mpiJava implementation issues

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Interfacing Java to MPI not always trivial, eg, see low-level conflicts between the Java runtime and interrupts in MPI.
Situation improving as JDK matures.
Now reliable on Solaris MPI (SunHPC, MPICH), shared memory, NT (WMPI).
Other ports in progress.
HTML version of Basic Foils prepared July 6 99

Foil 11 mpiJava performance

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 12 MPI datatypes

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Send and receive members of Comm:
void send(Object buf, int offset, int count,
Datatype type, int dst, int tag) ;
Status recv(Object buf, int offset, int count,
Datatype type, int src, int tag) ;
buf must be an array. offset is the element where message starts. Datatype class describes type of elements.
HTML version of Basic Foils prepared July 6 99

Foil 13 Basic Datatypes

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 14 Derived datatypes

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
MPI derived datatypes have two roles:
Non-contiguous data can be transmitted in one message.
MPI_TYPE_STRUCT allows mixed primitive types in one message.
Java binding doesn't support second role. All data come from a homogeneous array of elements (no MPI_Address).
HTML version of Basic Foils prepared July 6 99

Foil 15 Restricted model

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
A derived datatype consists of
A base type. One of the 9 basic types.
A displacement sequence. A relocatable pattern of integer displacements in the buffer array:
{disp , disp , . . . , disp }
0 1 n-1
HTML version of Basic Foils prepared July 6 99

Foil 16 Limitations

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Can't mix primitive types or fields from different objects.
Displacements only operate within 1d arrays. Can't use MPI_TYPE_VECTOR to describe sections of Java multidimensional arrays.
HTML version of Basic Foils prepared July 6 99

Foil 17 Object datatypes

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
If type argument is MPI.OBJECT, buf should be an array of objects.
Allows to send fields of mixed primitive types, and fields from different objects, in one message.
Allows to send multidimensional arrays, because they are arrays of arrays (and arrays are effectively objects).
HTML version of Basic Foils prepared July 6 99

Foil 18 Automatic serialization

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Send buf should be an array of objects implementing Serializable.
Receive buf should be an array of compatible reference types (may be null).
Java serialization paradigm applied. Output objects (and objects referenced through them) converted to a byte stream. Object graph reconstructed at the receiving end.
HTML version of Basic Foils prepared July 6 99

Foil 19 Implementation issues for Object datatypes

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Initial implementation in mpiJava used ObjectOutputStream and ObjectInputStream classes from JDK.
Data serialized and sent as a byte vector, using MPI.
Length of byte data not known in advance. Encoded in a separate header so space can be allocated dynamically in receiver.
HTML version of Basic Foils prepared July 6 99

Foil 20 Modifications to mpiJava

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
All mpiJava communications, including non-blocking modes and collective operations, now allow objects as base types.
Header + data decomposition complicates, eg, wait and test family.
Derived datatypes complicated.
Collective comms involve two phases if base type is OBJECT.
HTML version of Basic Foils prepared July 6 99

Foil 21 Benchmarking mpiJava with serialization

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Assume in "Grande" applications, critical case is arrays of primitive element.
Consider N x N arrays:
float [] [] buf = new float [N] [N] ;
MPI.COMM_WORLD.send(buf, 0, N, MPI.OBJECT,
dst, tag) ;
float [] [] buf = new float [N] [] ;
MPI.COMM_WORLD.recv(buf, 0, N, MPI.OBJECT,
src, tag) ;
HTML version of Basic Foils prepared July 6 99

Foil 22 Timing model

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
For comparision time float [NxN] (no serialization),
float [1] [NxN] (1d serialization), and
byte and int versions.
HTML version of Basic Foils prepared July 6 99

Foil 23 Platform

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Cluster of 2-processor, 200 Mhz Ultrasparc nodes
SunATM-155/MMF network
Sun MPI 3.0
"non-shared memory" = inter-node comms
"shared memory" = intra-node comms
HTML version of Basic Foils prepared July 6 99

Foil 24 Non-shared memory: byte

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 25 Non-shared memory: float

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 26 Shared memory: byte

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 27 Shared memory: float

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 28 Parameters in timing model (microseconds)

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
byte float
t = 0.043 t = 2.1
ser ser
byte float
t = 0.027 t = 1.4
unser unser
byte float
t = 0.062 t = 0.25 (non-shared)
com com
byte float
t = 0.008 t = 0.038 (shared)
com com
HTML version of Basic Foils prepared July 6 99

Foil 29 Benchmark lessons

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Cost of serializing and unserializing an individual float one to two orders of magnitude greater than communication!
Serializing subarrays also expensive:
vec vec
t = 100 t = 53
ser unser
HTML version of Basic Foils prepared July 6 99

Foil 30 Improving serialization

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Sources of ObjectOutputStream, ObjectInputStream are available, and format of serialized stream is documented.
By overriding performance-critical methods in classes, and modifying critical aspects of the stream format, can hope to solve immediate problems.
HTML version of Basic Foils prepared July 6 99

Foil 31 Eliminating overheads of element serialization

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Customized ObjectOutputStream replaces primitive arrays with short ArrayProxy object. Separate Vector holding the Java arrays is produced.
"Data-less" byte stream sent as header.
New ObjectInputStream yields Vector of allocated arrays, not writing elements.
Elements then sent in one comm using MPI_TYPE_STRUCT from vector info.
HTML version of Basic Foils prepared July 6 99

Foil 32 Improved protocol

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 33 Customized output stream class

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Class ArrayOutputStream extends ObjectOutputStream {
Vector dataVector ;
public Object replaceObject(Object obj) {
if obj is a primitive array, then
if obj is (eg) an int[], then
int len = ((int []) obj).length ;
dataVector.addElement(new ArrayInfo(INT, len, obj)) ;
return new ArrayProxy(INT, len) ;
etc (deal with other primitive types).
else, if not a primitive array, then
return obj ;
}
}
HTML version of Basic Foils prepared July 6 99

Foil 34 Customized input stream class

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Class ArrayInputStream extends ObjectInputStream {
Vector dataVector ;
public Object resolveObject(Object obj) {
if (obj instanceof ArrayProxy) then
ArrayProxy proxy = (ArrayProxy) obj ;
switch (proxy.type) {
case INT :
int [] dat = new int [proxy.length] ;
dataVector.addElement(new ArrayInfo(INT, dat.length, dat)) ;
return dat ;
etc (deal with other primitive types).
else return obj ;
}
}
HTML version of Basic Foils prepared July 6 99

Foil 35 Non-shared memory: float (optimized in red)

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 36 Non-shared memory: byte (optimized in red)

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 37 Shared memory: float (optimized in red)

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 38 Shared memory: byte (optimized in red)

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
HTML version of Basic Foils prepared July 6 99

Foil 39 Comments

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Relatively easy to get dramatic improvements.
Have only truly optimized one dimensional arrays embedded in stream.
Next look at direct optimizations for rectangular multi-dimensional arrays---replace wholesale in stream?
HTML version of Basic Foils prepared July 6 99

Foil 40 Conclusions

From Object serialization for Marshalling Data in a Java Interface to MPI ACM Java Grande Meeting -- June 12-13 99. *
Full HTML Index
Derived datatypes workable for Java, but slightly limited.
Object basic types attractive on grounds of simplicity and generality.
Naïve implementation too slow for bulk data transfer.
Optimizations should bring asymptotic performance in line with C/Fortran MPI.
www.npac.syr.edu/projects/pcrc/HPJava/mpiJava.html
© Northeast Parallel Architectures Center, Syracuse University, npac@npac.syr.edu

If you have any comments about this server, send e-mail to webmaster@npac.syr.edu.

Page produced by wwwfoil on Tue Jul 6 1999