Using Java Communication Patterns for High-End and High-Performance Applications

Bryan Carpenter(?), Ian Foster(?), Vladimir Getov, Gregor von Laszewski, Michael Philippsen

September 16, 2000

1 Note to the Editor

We have discussed various outlines for an article and are very flexible towards suggestions from the editorial board. Footnotes are used in order to give the editor some hint about our intentions for a final version of the paper. At present we are discussing the following outline:

Contents 2 Introduction

This paper focuses on how Java can be used for the development of High-End applications. It covers the full range necessary to develop such applications in today and tomorrow’s infrastructures. Based on the diverse requirements of High-End Applications, we discuss

• the use of communicating components as part of complex infrastructures that are needed to support the next generation of scientific applications on computational Grids,

• (potentially also: the use of CORBA to support this industry standard, and the use of the DOE CCAT for those applications that need higher performance,)

• the benefits of using Java remote objects and describe the necessary performance improvements to support High-End applications, and

• the provision of an MPI based message passing library to simplify existing and future scientific program development.

Our claim is that a scientific programmer must have the possibility to chose from a variety of supporting frameworks since the requirements for today’s application do not spawn only a single programming framework or paradigm but contain the composition of complex programs written in potentially different frameworks. We show that Java is not only a programming language but through the availability of standard and custom extensions a programming framework is created that can be used in many high-end applications.

In particular, we will show that the Java framework

• provides sufficient support to develop interfaces for various communication patterns demanded by high-end applications as listed above,

• provides a bridge between the access to diversified computational, storage, and networking infrastructure, and commodity technologies.

Besides the technical advantages we will point out that the Java framework

• reduces the development overhead and thus the overall development costs

• provides a uniform integration framework that makes maintenance of large-scale applications easier, and

• allows rapid prototyping.

Due to the characteristics of high-end applications the later points are critical during the software development phase.^{1}

We are proving our claims while analyzing the development of a computing portal for high-end applications. One such application that benefits from the development of a computing portal is the determination of the structure of macromolecules.^{2} This complex application is used to determine the structure of molecules while using unique instruments such as an advanced photon source and a combination of supercomputing resources that are made available through a computational grid.^{3}

1It is difficult to quantify these points as they are a research topic in itself. Nevertheless, we can deduct these results from the experience of the authors.

The rest of the paper will gradually take each component of the portal and we will illustrate in concrete examples which Java technology can be used to support the development of this component. We will provide a table that lists the components and their requirements in order to derive the correct framework to be chosen for the implementation.

This table is used to motivate the outline of the rest of the paper and allow the seamless description of the various frameworks. Each section will provide an analysis of advantages and disadvantages as well as a scalability or performance requirement analysis^{4}.

The paper will continue to discuss

• the use of the Grid via Java to fulfill the requirements of a grand challenge application that utilizes diverse resources on the grid including large scale compute and storage resources, as well as quality of service requirements posed to the network infrastructure,

• the use of remote object invocation, to access state variables of the physical instruments in order to build appropriate acquisition and computational steering abilities of the physical hardware,

• the use of MPI as part of the computational most demanding tasks that require tightly coupled communication between the distributed tasks,

• the use of component wrappers to interface with proprietary software installed within the existing infrastructure,^{5}

The paper will conclude with a small summary about the issues we learned during our joint experience. We are convinced that Java is a viable technology for the next generation of high-end applications.^{6}.

The following sections are intended to give the editor more detailed information about these issues. Several sections are not yet augmented with the motivating application requirements. Thus, they may appear at the moment to be a bit unconnected. We intend to provide a seamless integration of these section excerpts in the next version of the paper.

2Another application is the aerodynamic design of an aircraft. Due to didactic reasons we will only use one application within this paper. For our initial proposal we used the structural biology application but it would also be possible to use the aircraft design, as it may be more familiar to the readers.

3We will explain the application and its setup in more detail in a sidebar.

4no formulas though, as this was a desire by the journal and the editors ;-)

5there are probably more issues which we may address in the final version, we would like to make a cross-reference to a Java numerical paper in case such a paper is accepted within this issue.

6We will also include some negative points and maybe including a Figure that lists advantages and disadvantages of using Java for high-end applications

3 High End and High Performance Applications

We will follow with a more detailed discussion of the High-End applications (components, mixed-language, distributed, etc.) for scientific computing. We will discuss the various issues based on the example we describe next.

In the science domain of structural biology (SB) the use of x-rays as a nondestructive tool for investigating the internal structure of materials at the micron length scale has grown rapidly over the past decade as a result of the advent of synchrotron radiation sources[?]. Obtaining the molecular structure of macromolecules is an important scientific achievement as it is used to enhance, for example, the capabilities of modern drug design in the fields of basic and applied research. In a typical SB experiment, a beam of x-rays illuminates a sample, and data is collected for multiple sample orientations by using a charge-coupled device (CCD). These images are then used to derive a three-dimensional representation of the analyzed objects. However, the effective use of such an expensive instrument requires the ability to archive, analyze, and visualize the collected data at orders of magnitude more than is currently possible. The data rates and compute power required to address this Grand Challenge problem are prodigious, easily reaching one gigabit per second and a teraflop per second.

For small structures, the solution usually does not pose a problem. For macromolecules, the computational demand on even the fastest processors can be months. This is due to the enormous and irregular solution space for which it is currently difficult to write fast deterministic algorithms to obtain an expectable solution. Fortunately, it is possible to utilize Grid infrastructure and use specialized parallelized versions of structural determination programs.

A typical experiment follows the following steps:

1. Experiment planning and preparation

2. Data collection

3. Data reduction

4. Data analysis and model construction

Each of these steps has its own challenges and due to the diverse solution methods needed in the different step suggest to look for possibilities of an integrated programming approach.

In the final paper we will explain the requirements and design choices for each step and align them with our proposed communication patterns as indicated in the introduction.

4 Programming Paradigms and Frameworks

Based on our application requirement analysis we determine that we need to support various patterns of communication as used in our parallel applications and a variety of existing infrastructures. Though these issues are usually viewed as orthogonal one