%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% This document is best edited with MSWord
% since it has a good spellchecker. Thus all sentences are 
% very long!!!
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass{article}
\usepackage{times}
\usepackage{epsfig}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% PsFIGURE command
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\PsFIGURE}[5]{%
\typeout{PsFIGURE: #2}
\begin{figure}[#1]
\centerline{\resizebox{#3\linewidth}{!}{\includegraphics{#2}}}

\caption{{\em #4}}
      \label{#5}
     \bigskip

\end{figure}
}

\newcommand{\pptFIGURE}[5]{%
\typeout{PsFIGURE: #2}
\begin{figure}[#1]

\centerline{\vspace{-1in}\resizebox{#3\linewidth}{!}{\includegraphics{#2}}}

\caption{{\em #4}}
      \label{#5}
     \bigskip

\end{figure}
}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% TITLE
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\title{Using Java Communication Patterns for High-End 
	and High-Performance Applications}

\author{Bryan Carpenter(?), 
	Ian Foster(?), \\
	Vladimir Getov, 
	Gregor von Laszewski, 
	Michael Philippsen}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\maketitle                 % Produces the title.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% DOCUMENT
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Note to the Editor}

We have discussed various outlines for an article and are very flexible towards suggestions from the editorial board. Footnotes are used in order to give the editor some hint about our intentions for a final version of the paper. At present we are discussing the following outline:

\tableofcontents

\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction} 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

This paper focuses on how Java can be used for the development of High-End applications. It covers the full range necessary to develop such applications in today and tomorrow's infrastructures. Based on the diverse requirements of High-End Applications, we discuss 

\begin{itemize}
\item the use of communicating components as part of complex infrastructures that are needed to support the next generation of scientific applications on computational Grids, 
\item (potentially also: the use of CORBA to support this industry standard, and the use of the DOE CCAT for those applications that need higher performance,) 
\item the benefits of using Java remote objects and describe the necessary  performance improvements to support High-End applications, and
\item the provision of an MPI based message passing library to simplify existing and future scientific program development.
\end{itemize}

Our claim is that a scientific programmer must have the possibility to chose from a variety of supporting frameworks since the requirements for today's application do not spawn only a single programming framework or paradigm but contain the composition of complex programs written in potentially different frameworks. We show that Java is not only a programming language but through the availability of standard and custom extensions a programming framework is created that can be used in many high-end applications. 

In particular, we will show that the Java framework 
\begin{itemize}
\item provides sufficient support to develop interfaces for various communication patterns demanded by high-end applications as listed above,
\item provides a bridge between the access to diversified computational, storage, and networking infrastructure, and commodity technologies.
\end{itemize}

Besides the technical advantages we will point out that the Java framework 

\begin{itemize}
\item reduces the development overhead and thus the overall development costs
\item provides a uniform integration framework that makes maintenance of large-scale applications easier, and
\item allows rapid prototyping.
\end{itemize}

Due to the characteristics of high-end applications the later points are critical during the software development phase.\footnote{It is difficult to quantify these points as they are a research topic in itself. Nevertheless, we can deduct these results from the experience of the authors.}

We are proving our claims while analyzing the development of a computing portal for high-end applications. One such application that benefits from the development of a computing portal is the determination of the structure of macromolecules.\footnote{Another application is the aerodynamic design of an aircraft. Due to didactic reasons we will only use one application within this paper. For our initial proposal we used the structural biology application but it would also be possible to use the aircraft design, as it may be more familiar to the readers.} This complex application is used to determine the structure of molecules while using unique instruments such as an advanced photon source and a combination of supercomputing resources that are made available through a computational grid.\footnote{We will explain the application and its setup in more detail in a sidebar.}

The rest of the paper will gradually take each component of the portal and we will illustrate in concrete examples which Java technology can be used to support the development of this component. We will provide a table that lists the components and their requirements in order to derive the correct framework to be chosen for the implementation.

This table is used to motivate the outline of the rest of the paper and allow the seamless description of the various frameworks. Each section will provide an analysis of advantages and disadvantages as well as a scalability or performance requirement analysis\footnote{no formulas though, as this was a desire by the journal and the editors ;-)}. 

The paper will continue to discuss

\begin{itemize}
\item the use of the Grid via Java to fulfill the requirements of a grand challenge application that utilizes diverse resources on the grid including large scale compute and storage resources, as well as quality of service requirements posed to the network infrastructure, 
\item the use of remote object invocation, to access state variables of the physical instruments in order to build appropriate acquisition and computational steering abilities of the physical hardware,   
\item the use of MPI as part of the computational most demanding tasks that require tightly coupled communication between the distributed tasks,
\item the use of component wrappers to interface with proprietary software installed within the existing infrastructure,\footnote{there are probably more issues which we may address in the final version, we would like to make a cross-reference to a Java numerical paper in case such a paper is accepted within this issue.}
\end{itemize}


The paper will conclude with a small summary about the issues we learned during our joint experience. We are convinced that Java is a viable technology for the next generation of high-end applications.\footnote{We will also include some negative points and maybe including a Figure that lists advantages and disadvantages of using Java for high-end applications}. 
 
The following sections are intended to give the editor more detailed information about these issues. Several sections are not yet augmented with the motivating application requirements. Thus, they may appear at the moment to be a bit unconnected. We intend to provide a seamless integration of these section excerpts in the next version of the paper.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section {High End and High Performance Applications}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

{\em We will follow with a more detailed discussion of the High-End applications (components, mixed-language, distributed, etc.)  for  scientific computing. We will discuss the various issues based on the example we describe next.}

In the science domain of structural biology (SB) the use of x-rays as a nondestructive tool for investigating the \emph{internal} structure of materials at the micron length scale has grown rapidly over the past decade as a result of the advent of synchrotron radiation sources\cite{Las2000sbc}. Obtaining the molecular structure of macromolecules is an important scientific achievement as it is used to enhance, for example, the capabilities of modern drug design in the fields of basic and applied research.  In a typical SB experiment, a beam of x-rays illuminates a sample, and data is collected for multiple sample orientations by using a charge-coupled device (CCD). These images are then used to derive a three-dimensional representation of the analyzed objects. However, the effective use of such an expensive instrument requires the ability to archive, analyze, and visualize the collected data at orders of magnitude more than is currently possible. The data rates and compute power required to address this Grand Challenge problem are prodigious, easily reaching one gigabit per second and a teraflop per second.

For small structures, the solution usually does not pose a problem. For macromolecules, the computational demand on even the fastest processors can be months. This is due to the enormous and irregular solution space for which it is currently difficult to write fast deterministic algorithms to obtain an expectable solution. Fortunately, it is possible to utilize Grid infrastructure and use specialized parallelized versions of structural determination programs.

A typical experiment follows the following steps:

    \begin{enumerate}
    \item Experiment planning and preparation   
    \item Data collection  
    \item Data reduction   
    \item Data analysis and model construction 
    \end{enumerate}

Each of these steps has its own challenges and due to the diverse solution methods needed in the different step suggest to look for possibilities of an integrated programming approach.

{\em In the final paper we will explain the requirements and design choices for each step and align them with our proposed communication patterns as indicated in the introduction. }

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section {Programming Paradigms and Frameworks}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Based on our application requirement analysis we determine that we need to support various patterns of communication as used in our parallel applications and a variety of existing infrastructures. Though these issues are usually viewed as orthogonal one has to consider that they can be addressed both while using Java as a programming language and compute framework with the appropriate extensions.

It is obvious that we have to support SPMD and MPMD style programs that are using coarse and fine grain parallelism since these programs are run on a very diversified infrastructure including Von-Neumann style computers, single supercomputers, distributed workstation clusters, and distributed networked supercomputer and data storage environment (also known as computational grid).

It is clear from this diversified infrastructure that a significant software engineering task must be conducted in order to successfully develop and deploy such an ambitious application.

>From our motivating application we have identified that the following component and comunication patterns must be supported:

	\begin{itemize}
	\item Component usage in an adaptive Grid environment
	\item Component usage in a local environment
	\item Message passing 
	\item Client Server programming 
	\item Fast remote objects
	\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Component Usage in an adaptive Grid environment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

First we will describe what the Grid\cite{GridBook} is, what challenges we face and what solutions may exist for some of the issues. We will point to the JavaCoG Kit that allows utilizing an adaptive Grid environment based on the Globus programming toolkit. This includes access to the necessary grid services such as authentication, information services, remote resource management, as well as the access to quality of service within networks. 

Besides these elementary grid services the Java framework allows to significantly enhance and prototype new grid services in rapid manner that are currently not yet part of the Globus toolkit. Such services are essential for the development of a computing portable as described in an earlier section. Examples for such services are the definition of a task flow component that can visually animate the execution of tasks formulated in a direct acyclic graph, the definition of jobs on a desktop similar to the Microsoft desktop to provide the science users (this community uses frequently MS products) a familiar interface, the development of intelligent job brokers, etc..

Besides these very technical enhanced services Java provides also the advantage for the developer to utilize IDE's and while casting these services as JavaBeans allowing to have a sophisticated GridCase tool available with little overhead during the development of the components and services. 

Some Figures will be used to explain the concepts in graphical form. 
    

    \PsFIGURE{htb}{images/nasa-ipg.eps}{.9}{The NASA Information Power Grid is an example for the development of an infrastructure that allows many scientists to share resources and use the infrastructure for high-end applications such as the aerostatic design of an airplain or a space craft.}{F:ipg}

    \PsFIGURE{htb}{images/lysozyme.eps}{.9}{This lysozyme structure has been determined during the use of a parallel program designed by ANL and the Structural Molecular Cooperation. The original infrastructure used (a single computer) returned the result in 3 days. A Grid enabled version of the same program with performance improvements returned the result in 10 minutes. {\em this has to be explained better in the text.} }{F:molecule}

    \PsFIGURE{htb}{images/portal.prn}{.9}{A computing portal contains various components that are accessed through a portal interface custom designed for a particular application domain. The components within such a portal are hidden from the end user and allow the utilization of the Gridinfrastructure without additional knowledge on the application users side.}{F:portal}

    \PsFIGURE{htb}{images/huffman-structure.eps}{.9}{Using a Grid environment has the advantage of accessing high end visualization devices that can stream their result in real time to devices with less capabilities. Thus the task of collaborative discussions between researchers at geographically distributed locations is possible. This image shows a structure .. provided by John Huffman, Indiana University, that is rendered on a special purpose graphics pipe but could be distributed quickly to the remote sites in a computational Grid.}{F:structure}

    %\PsFIGURE{htb}{images/overflow.prn}{.9}{Portal}{F:overflow}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Message Passing in Java}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The Java language has several built-in mechanisms that allow the parallelism inherent in scientific programs to be exploited. Threads and concurrency constructs are well suited to shared memory computers, but not large-scale distributed memory machines. Although sockets and the RMI interface allow the development of big network applications, they have been designed and optimized for client-server programming, whereas the parallel computing world is mainly concerned with a more symmetric model, where communications occur in groups of interacting peers. Therefore, codes based on sockets and RMI would naturally underperform platform-specific implementations of standard communication libraries based on the successful Message Passing Interface (MPI) standard \cite{MPI-1}. By contrast with sockets and RMI, MPI directly supports the Single Program Multiple Data (SPMD) model of parallel computing, wherein a group of processes cooperate by executing identical program images on local data values.

With the evident success of Java as a programming language, and its inevitable use in connection with parallel as well as distributed computing, the absence of a well-designed language-specific binding for message-passing with Java would lead to divergent, non-portable practices. The Message-Passing Working Group of the Java Grande Forum was formed in the Fall of 1998 as a response to the appearance of the various APIs for message-passing.  Some of these early ``proof-of-concept'' implementations \cite{HPjava,Getov97,Judd99} have been available since 1997 with successful ports on clusters of workstations running Linux, Solaris, Windows NT, Irix, AIX, HP-UX, and MacOS, as well as on parallel platforms such as the IBM SP-2 and SP-3, Sun E4000, SGI Origin-2000, Fujitsu AP3000, Hitachi SR2201 and others.
An immediate goal was to discuss and agree on a common API for MPI-like libraries for Message Passing in Java (MPJ) \cite{MPJ-Spec}.

The MPI standard is explicitly object-based. The C and Fortran bindings rely on ``opaque objects'' that can be manipulated only by acquiring object handles from constructor functions, and passing the handles to suitable functions in the library. The C++ binding specified in the MPI-2 standard collects these objects into suitable class hierarchies and defines most of the library functions as class member functions. The MPJ API specification follows this model, lifting the structure of its class hierarchy directly from the C++ binding. The purpose of this phase of the effort is to provide an immediate, ad hoc standardization for common message passing programs in Java, as well as to provide a basis for conversion between C, C++, Fortran, and Java.

In this paper we present performance analysis and comparisons of evaluation results for both Java and C/Fortran on three different message-passing parallel platforms -- a shared memory multi-processor (Sun E4000), a Linux cluster, and a distributed memory computer (IBM SP-2). The NAS parallel Embarrassingly Parallel (EP) and the Integer Sort (IS) benchmarks were used in our performance evaluation. The IS routine evaluates integer operations and bi-directional communications (the sorted keys are exchanged between nodes), while the EP kernel tests floating point operations performance but requires minimal communications.

The JVM and the Java compiler used on the IBM SP-2 machine were part of the JDK for AIX.  The execution environment consisted of IBM's Parallel Operating Environment (POE), which supports the loading and execution of parallel processes across the nodes of the IBM SP-2. The machine is built of thin nodes with POWER2 Super Chip (P2SC) processors and 256 Mbytes of memory on each processor. The communication subsystem of the SP-2 features a high-performance switch which was used throughout the experiments.  The NAS EP and IS benchmarks were also run on a 200 MHz dual Pentium Pro processor cluster running Linux Red Hat 6.0 on a 10baseT Ethernet.  The same experiments were performed on a 14x336 MHz Ultra Sparc II processor Sun E4000 running Solaris 2.6. The LAM MPI library was used on the Linux cluster, whilst both the SP-2 and the E4000 provided native MPI libraries for message passing.

\PsFIGURE{htb}{images/ep-b.eps}{.9}%
{Execution times for the NPB EP kernel (class B) on the IBM SP-2}%
{F:ep-b}

A native code compiler for Java can be used instead of the JVM in order to overcome the above problem. Fortunately, rapid progress is being made in this area by developing optimizing Java compilers, such as the IBM High-Performance Compiler for Java (HPCJ), which generates native codes for the RS6000 architecture~\cite{HPCJ}. It works in the same manner as compilers for C, C++, Fortran, etc. and unlike JIT compilers, the static compilation occurs only once, before execution time. Thus, traditional resource-intensive optimisations can be applied in order to improve the performance of the generated native executable code. In our experiments, we have used a version of HPCJ, which generates native code for the RS/6000 architecture. The input of HPCJ is usually a bytecode file, but the compiler will also accept Java source as input. In the latter case it invokes the JDK source-to-bytecode compiler to produce the bytecode file first. This file is then processed by a translator which passes an intermediate language representation to the common back-end from the family of compilers for the RS/6000 architecture. The back-end outputs standard object code which is then linked with other object modules and the previously bound legacy libraries to produce native executable code. Further experiments to evaluate the performance of the environment based on HPCJ have been carried out with the IS kernel on an IBM SP-2 machine.  The results obtained are shown in Figure~\ref{F:is-a}.

\PsFIGURE{htb}{images/is-a.eps}{.9}
{Execution time for the NPB IS kernel (class A) on the IBM SP-2}
{F:is-a}

Back in 1994, MPI-1 was originally designed with relatively static platforms in mind.  To better support computing in volatile Internet environments, modern message passing designs for Java will have to support (at least) features such as dynamic spawning of process groups and parallel client/server interfaces as introduced in the MPI-2 specification.  In addition, a natural framework for dynamically discovering new compute resources and establishing connections between running programs already exists in Sun's Jini project \cite{Arnold99}, and one line of investigation is into MPJ implementations operating in the Jini framework.

Closely modeled as it is on the MPI standards, the existing MPJ specification should be regarded as a first phase in a broader program to define a more Java-centric high performance message-passing environment.  In future a detachment from legacy implementations involving Java on top of native methods will be emphasized.  We should consider the possibility of layering the messaging middleware over standard transports and other Java-compliant middleware (like CORBA). Of course, a primary goal in the above mentioned, both current and future work, should be the aim to offer MPI-like services to Java programs in an upward compatible fashion.  The purposes are twofold: performance and portability.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Fast Remote Method Invocation}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Good latency times and high bandwidths are essential for distributed and parallel programs. However, the remote method invocation (RMI) of common Java distributions is in many cases too slow for high performance applications, since RMI was developed for wide area networks, builds upon the slow object serialization, and does not support any high speed networks. With regular Java, a remote method invocation takes milliseconds -- concrete times depend on the number and the types of arguments.  A third of that time is needed for the RMI itself, a third for the serialization of the arguments (their transformation into a machine-independent byte representation), and a third for the data transfer (TCP/IP-Ethernet).

In order to achieve a fast remote method invocation, work must be done at all levels. This means that one needs a fast RMI implementation, a fast serialization, and the possibility of using communication hardware that does not employ TCP/IP protocols.

Within the framework of the JavaParty Project at the University of Karlsruhe \cite{JavaParty}, all three of these requirements were attacked to create the fastest (pure) Java implementation of a remote method invocation.  On a cluster of DEC-Alpha computers connected by Myrinet, called ParaStation, currently a remote method invocation takes about 80 $\mu$s although it is completely implemented in Java.\footnote{Of course, the connection of the card driver was not realized in Java.}  Figure~\ref{rmibench} shows, that for benchmark programs 96\% of the time can be saved, if the UKA serialization, the high-speed RMI (KaRMI), and the faster communication hardware is used. The central ideas of the optimization will be highlighted in the next sections.


\PsFIGURE{htb}{images/all.ps}{.9}
{ The bottom three box plots each show 2$\cdot$64 measured results for diverse benchmarks (64 points represent Ethernet on PC, 64 stand for FastEthernet on Alpha). The first (bottom-most) box plot shows the run-time improvement that was achieved with regular RMI and the UKA serialization. The second box plot shows the improvement that KaRMI achieves when used with Java's regular serialization.  The third box plot shows the combined effect. The top line demonstrates what happens, if Myrinet cards are used in addition to the UKA serialization and KaRMI (64 measured results)}
{rmibench}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{UKA Serialization.}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The UKA serialization \cite{Phlipp99a} can be used instead of the official serialization (and as a supplement to it) and saves 76\%--96\% of the serialization time. It is based on the following main points:

\begin{itemize}

\item Explicit serialization routines (``marshalling routines'') are faster than those used by classical RMI that automatically derive a byte representation with the help of type introspection.

\item A good deal of the costs of the serialization are needed for the time-consuming encoding of the type information that is necessary for persistent object storage. For the purposes of communication, especially in work station clusters with common file systems, a reduced form of the type encoding is sufficient and faster. The Java Grande Forum has convinced Sun Microsystems to make the method of type encoding plugable in one of the next versions.

\item Copied objects need to be transferred again for each call in RMI. RMI does not differentiate between type encoding and useful data, meaning that the type information is transferred redundantly.

\item Sun has announced (without concretely naming a version) it will pick up on the idea of a separate reset of type information and user data.

\item The official serialization uses several layers of streams that all possess their own buffers. This causes frequent copying operations and results in unacceptable performance. The UKA serialization only needs one buffer, which the byte representation can be directly written in.

\item Although Sun remains steadfast about layering for reasons of clearer object-oriented design, they are at least improving the implementation of the layers.

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{KaRMI.}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

A substitute implementation of RMI, called KaRMI, was also created at the University of Karlsruhe. KaRMI \cite{PhilippsenJ010} can be used instead of the official RMI and gets rid of the following deficiencies, as well as some others found in official RMI:

\begin{itemize}

\item KaRMI supports non-TCP/IP networks.  Suns plans to add support in the official RMI-Version as well.

\item KaRMI possesses clearer layering, which will make it easier to employ other protocol semantics (i.e. Multicast) and other network hardware (i.e. Myrinet-Cards).

\item In RMI, objects can be connected to fixed port numbers. Therefore, a certain detail of the network layer is passed to the application. Since this is in conflict with the guidelines of modular design, KaRMI only supports use of explicit port numbers when the underlying network offers them.

\item The distributed garbage collection of the official RMI was created for wide area networks. Although there are optimized garbage collectors for tightly coupled clusters and for other platforms \cite{PhilippsenJ012}, the official RMI sees no alternative garbage collector as being necessary, in contrast to KaRMI.

	\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conclusions}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

{\em Cherri suggested we also discuss the trade-off between s/w development efforts and the efficiency. I guess this will require an overview of the corresponding programming models (SPMD, client/server, etc.) (Cherry suggested an example and follow the ideas through based on this example) }

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% END OF THE MAIN PART 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bibliographystyle{plain}
\bibliography{references}


\appendix


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Sidebar on Java Portability}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{enumerate}
\item Source Code
\item Byte Code
\item Data Format
\item Output Results (Reproducibility)
\item Performance
\item Executable Code
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

