REFEREE'S REPORT
Concurrency and Computation:Practice and Experience
---------------------------------------------------------------------------
A: General Information
Please return to:
Geoffrey C. Fox
Electronically Preferred fox@csit.fsu.edu
Concurrency and Computation: Practice and Experience
Computational Science and Information Technology
Florida State University
400 Dirac Science Library
Tallahassee Florida 32306-4130
Office FAX 850-644-0098
Office Phone 850-644-4587 but best is cell phone 3152546387
Please fill in Summary Conclusions (Sec. C) and details as appropriate in
Secs. D, E and F.
B: Refereeing Philosophy
We encourage a broad range of readers and contributors. Please judge papers
on their technical merit and separate comments on this from those on style
and approach. Keep in mind the strong practical orientation that we are
trying to give the journal. Note that the forms attached provide separate
paper for comments that you wish only the editor to see and those that both
the editor and author receive. Your identity will of course not be revealed
to the author.
C: Paper and Referee Metadata
* Paper Number Cnnn: C497
* Date: 4/20/01
* Paper Title: Java for High Performance Computing
* Author(s): Lobosco,Amorim,Loques
* Referee: Hans Zima
* Address: University of Vienna zima@par.univie.ac.at
Referee Recommendations. Please indicate overall recommendations here, and
details in following sections.
3. reject
D: Referee Comments (For Editor Only)
This paper makes the claim to "... describe and classify several important
proposals and environments that tackle Java's performance bottlenecks
in order to make the language an effective option for high performance
computing...". Further, the point is made that "most significant performance
issues are surveyed" based on a classification taking into account "some
combination of three basic parameters: the model adopted for inter-process
communication, language extensions, and implementation strategy".
A paper actually fulfilling this promise would be of very high value.
Unfortunately, this paper does not come even close to achieving this
goal. It lacks a coherent critique of Java, the precise characterization
of evaluation criteria, and a systematic and concise evaluation of approaches.
Furthermore, the authors in many places do not clearly distinguish between
language (semantics) and libraries; they also often confuse the role
of compiler and runtime system.
The fact that style and language display considerable weaknesses, that
section numbers etc. are missing is almost negligible in view of
the technical shortcomings. I do not see an obvious way how to improve this
paper in order to produce an acceptable publication.
Below are detailed notes, many of which will underline the points made
above.
(1) Performance issues for sequential Java
This important topic, which has been extensively discussed by Moreira et al.
in the IBM System Journal as well as in other publications is just mentioned
in passing. A paper with such a general goal as this one must at least provide
an overview of issues such as the array concept, exception handling, and
the complex data type.
(2) Memory model
Bill Pugh calls the Java memory model "totally flawed" (CPE 12(6):445-455):
this should be discussed.
-p14: the proposal is made to include "collective communication in the language,
such as scatter and gather,...". Such an inclusion of MPI-like constructs would
have wide-ranging consequences.
-p15ff Classification parameters
The paper mentions "three approaches for interprocess communication:
distributed shared memory, message passing, or a combination of both."
What about shared-memory communication and high-level approaches such as those
in HPF or OpenMP?
The discussion of the remaining parameters is likewise fuzzy: for example,
the discussion of the implementation strategy does not deal with optimizations
(and the specific role of the Java semantics specification in this context).
-p18 MultiJav
For someone not already knowing this system this description is unreadable.
Similar comments apply to other systems discussed in the paper.
-p26 Using Java Library
A library cannot modify symtax or semantics of a language -- in contrast to
the statement made here.
-p27 Charlotte
Charlotte is classified as a library but offers compiler annotations.
-pp30-31
The conclusions drawn here with respect to performance are highly
questionable -- not supported by any experimental data.
-p34 JCI
JCI is not a tool for HPC and does not belong here.
-pp36ff Manta
The description of Manta is very long, and much room is devoted
to present very old runtime results (JDK 1.1.3 vs. actual 1.3).
-pp40ff HPJava
HPJava is called (for unknown reasons) JDPE in this paper. The
bibliographic reference is wrong and should read
"HPJava: a Data-Parallel Extension to Java", by Carpenter, Zhang,
Fox, Li, and Wen
It should also be noted that HPJava is based upon adlib.
-p43, second paragraph: "the permission for methods ... to be
remotely accessed, which is not allowed in RMI": this is
nonsense.
-p44, second paragraph: "... Java supports passive objects but only
Java// supports active objects (threads and actors)." What about Java
thread objects, remote objects?
-p45: the characterization of a future object is wrong.
-p46ff: UKA-Serialization and KaRMI cannot be regarded as "model",
but implementation improvement.
-p49ff: Javia is not a tool for HPC but a basic network interface.
-p50ff: the conclusions drawn are again quite questionable ("JCI
outperforms mpiJava" 1st line of pg. 51; "Manta should also
outperform all the other systems ..." 1st line of pg. 52.
-p55: last three lines:
"...passing all the overhead of *dynamic* inspection to the compiler..."
How can the compiler perform dynamic inspection?
"new features in the language ... following MPI
or PVM". The authors should distinguish between language and libraries.
-p61: Reference 31: "Lea D." instead of "Doug L."
E: Referee Comments (For Author and Editor) [identical to D]
This paper makes the claim to "... describe and classify several important
proposals and environments that tackle Java's performance bottlenecks
in order to make the language an effective option for high performance
computing...". Further, the point is made that "most significant performance
issues are surveyed" based on a classification taking into account "some
combination of three basic parameters: the model adopted for inter-process
communication, language extensions, and implementation strategy".
A paper actually fulfilling this promise would be of very high value.
Unfortunately, this paper does not come even close to achieving this
goal. It lacks a coherent critique of Java, the precise characterization
of evaluation criteria, and a systematic and concise evaluation of approaches.
Furthermore, the authors in many places do not clearly distinguish between
language (semantics) and libraries; they also often confuse the role
of compiler and runtime system.
The fact that style and language display considerable weaknesses, that
section numbers etc. are missing is almost negligible in view of
the technical shortcomings. I do not see an obvious way how to improve this
paper in order to produce an acceptable publication.
Below are detailed notes, many of which will underline the points made
above.
(1) Performance issues for sequential Java
This important topic, which has been extensively discussed by Moreira et al.
in the IBM System Journal as well as in other publications is just mentioned
in passing. A paper with such a general goal as this one must at least provide
an overview of issues such as the array concept, exception handling, and
the complex data type.
(2) Memory model
Bill Pugh calls the Java memory model "totally flawed" (CPE 12(6):445-455):
this should be discussed.
-p14: the proposal is made to include "collective communication in the language,
such as scatter and gather,...". Such an inclusion of MPI-like constructs would
have wide-ranging consequences.
-p15ff Classification parameters
The paper mentions "three approaches for interprocess communication:
distributed shared memory, message passing, or a combination of both."
What about shared-memory communication and high-level approaches such as those
in HPF or OpenMP?
The discussion of the remaining parameters is likewise fuzzy: for example,
the discussion of the implementation strategy does not deal with optimizations
(and the specific role of the Java semantics specification in this context).
-p18 MultiJav
For someone not already knowing this system this description is unreadable.
Similar comments apply to other systems discussed in the paper.
-p26 Using Java Library
A library cannot modify symtax or semantics of a language -- in contrast to
the statement made there.
-p27 Charlotte
Charlotte is classified as a library but offers compiler annotations.
-pp30-31
The conclusions drawn here with respect to performance are highly
questionable -- not supported by any experimental data.
-p34 JCI
JCI is not a tool for HPC and does not belong here.
-pp36ff Manta
The description of Manta is very long, and much room is devoted
to present very old runtime results (JDK 1.1.3 vs. actual 1.3).
-pp40ff HPJava
HPJava is called (for unknown reasons) JDPE in this paper. The
bibliographic reference is wrong and should read
"HPJava: a Data-Parallel Extension to Java", by Carpenter, Zhang,
Fox, Li, and Wen
It should also be noted that HPJava is based upon adlib.
-p43, second paragraph: "the permission for methods ... to be
remotely accessed, which is not allowed in RMI": this is
nonsense.
-p44, second paragraph: "... Java supports passive objects but only
Java// supports active objects (threads and actors)." What about Java
thread objects, remote objects?
-p45: the characterization of a future object is wrong.
-p46ff: UKA-Serialization and KaRMI cannot be regarded as "model",
but implementation improvement.
-p49ff: Javia is not a tool for HPC but a basic network interface.
-p50ff: the conclusions drawn are again quite questionable ("JCI
outperforms mpiJava" 1st line of pg. 51; "Manta should also
outperform all the other systems ..." 1st line of pg. 52.
-p55: last three lines:
"...passing all the overhead of *dynamic* inspection to the compiler..."
How can the compiler perform dynamic inspection?
"new features in the language ... following MPI
or PVM". The authors should distinguish between language and libraries.
-p61: Reference 31: "Lea D." instead of "Doug L."
F: Presentation Changes