Subject: Review C497 From: Hans Zima Date: Fri, 20 Apr 2001 15:46:17 +0200 (MEST) To: fox@csit.fsu.edu CC: zima@par.univie.ac.at Geoffrey, Please find attached my review for the paper. I do not find it acceptable for publication. Best regards, --Hans ======================================================= Professor Hans P. Zima University of Vienna, Institute for Software Science Liechtensteinstrasse 22, A-1090 Vienna, Austria Tel.: +43-1-4277 38801; Fax: +43-1-4277 9388 E-Mail: zima@par.univie.ac.at REFEREE'S REPORT Concurrency and Computation:Practice and Experience --------------------------------------------------------------------------- A: General Information Please return to: Geoffrey C. Fox Electronically Preferred fox@csit.fsu.edu Concurrency and Computation: Practice and Experience Computational Science and Information Technology Florida State University 400 Dirac Science Library Tallahassee Florida 32306-4130 Office FAX 850-644-0098 Office Phone 850-644-4587 but best is cell phone 3152546387 Please fill in Summary Conclusions (Sec. C) and details as appropriate in Secs. D, E and F. B: Refereeing Philosophy We encourage a broad range of readers and contributors. Please judge papers on their technical merit and separate comments on this from those on style and approach. Keep in mind the strong practical orientation that we are trying to give the journal. Note that the forms attached provide separate paper for comments that you wish only the editor to see and those that both the editor and author receive. Your identity will of course not be revealed to the author. C: Paper and Referee Metadata * Paper Number Cnnn: C497 * Date: 4/20/01 * Paper Title: Java for High Performance Computing * Author(s): Lobosco,Amorim,Loques * Referee: Hans Zima * Address: University of Vienna zima@par.univie.ac.at Referee Recommendations. Please indicate overall recommendations here, and details in following sections. 3. reject D: Referee Comments (For Editor Only) This paper makes the claim to "... describe and classify several important proposals and environments that tackle Java's performance bottlenecks in order to make the language an effective option for high performance computing...". Further, the point is made that "most significant performance issues are surveyed" based on a classification taking into account "some combination of three basic parameters: the model adopted for inter-process communication, language extensions, and implementation strategy". A paper actually fulfilling this promise would be of very high value. Unfortunately, this paper does not come even close to achieving this goal. It lacks a coherent critique of Java, the precise characterization of evaluation criteria, and a systematic and concise evaluation of approaches. Furthermore, the authors in many places do not clearly distinguish between language (semantics) and libraries; they also often confuse the role of compiler and runtime system. The fact that style and language display considerable weaknesses, that section numbers etc. are missing is almost negligible in view of the technical shortcomings. I do not see an obvious way how to improve this paper in order to produce an acceptable publication. Below are detailed notes, many of which will underline the points made above. (1) Performance issues for sequential Java This important topic, which has been extensively discussed by Moreira et al. in the IBM System Journal as well as in other publications is just mentioned in passing. A paper with such a general goal as this one must at least provide an overview of issues such as the array concept, exception handling, and the complex data type. (2) Memory model Bill Pugh calls the Java memory model "totally flawed" (CPE 12(6):445-455): this should be discussed. -p14: the proposal is made to include "collective communication in the language, such as scatter and gather,...". Such an inclusion of MPI-like constructs would have wide-ranging consequences. -p15ff Classification parameters The paper mentions "three approaches for interprocess communication: distributed shared memory, message passing, or a combination of both." What about shared-memory communication and high-level approaches such as those in HPF or OpenMP? The discussion of the remaining parameters is likewise fuzzy: for example, the discussion of the implementation strategy does not deal with optimizations (and the specific role of the Java semantics specification in this context). -p18 MultiJav For someone not already knowing this system this description is unreadable. Similar comments apply to other systems discussed in the paper. -p26 Using Java Library A library cannot modify symtax or semantics of a language -- in contrast to the statement made here. -p27 Charlotte Charlotte is classified as a library but offers compiler annotations. -pp30-31 The conclusions drawn here with respect to performance are highly questionable -- not supported by any experimental data. -p34 JCI JCI is not a tool for HPC and does not belong here. -pp36ff Manta The description of Manta is very long, and much room is devoted to present very old runtime results (JDK 1.1.3 vs. actual 1.3). -pp40ff HPJava HPJava is called (for unknown reasons) JDPE in this paper. The bibliographic reference is wrong and should read "HPJava: a Data-Parallel Extension to Java", by Carpenter, Zhang, Fox, Li, and Wen It should also be noted that HPJava is based upon adlib. -p43, second paragraph: "the permission for methods ... to be remotely accessed, which is not allowed in RMI": this is nonsense. -p44, second paragraph: "... Java supports passive objects but only Java// supports active objects (threads and actors)." What about Java thread objects, remote objects? -p45: the characterization of a future object is wrong. -p46ff: UKA-Serialization and KaRMI cannot be regarded as "model", but implementation improvement. -p49ff: Javia is not a tool for HPC but a basic network interface. -p50ff: the conclusions drawn are again quite questionable ("JCI outperforms mpiJava" 1st line of pg. 51; "Manta should also outperform all the other systems ..." 1st line of pg. 52. -p55: last three lines: "...passing all the overhead of *dynamic* inspection to the compiler..." How can the compiler perform dynamic inspection? "new features in the language ... following MPI or PVM". The authors should distinguish between language and libraries. -p61: Reference 31: "Lea D." instead of "Doug L." E: Referee Comments (For Author and Editor) [identical to D] This paper makes the claim to "... describe and classify several important proposals and environments that tackle Java's performance bottlenecks in order to make the language an effective option for high performance computing...". Further, the point is made that "most significant performance issues are surveyed" based on a classification taking into account "some combination of three basic parameters: the model adopted for inter-process communication, language extensions, and implementation strategy". A paper actually fulfilling this promise would be of very high value. Unfortunately, this paper does not come even close to achieving this goal. It lacks a coherent critique of Java, the precise characterization of evaluation criteria, and a systematic and concise evaluation of approaches. Furthermore, the authors in many places do not clearly distinguish between language (semantics) and libraries; they also often confuse the role of compiler and runtime system. The fact that style and language display considerable weaknesses, that section numbers etc. are missing is almost negligible in view of the technical shortcomings. I do not see an obvious way how to improve this paper in order to produce an acceptable publication. Below are detailed notes, many of which will underline the points made above. (1) Performance issues for sequential Java This important topic, which has been extensively discussed by Moreira et al. in the IBM System Journal as well as in other publications is just mentioned in passing. A paper with such a general goal as this one must at least provide an overview of issues such as the array concept, exception handling, and the complex data type. (2) Memory model Bill Pugh calls the Java memory model "totally flawed" (CPE 12(6):445-455): this should be discussed. -p14: the proposal is made to include "collective communication in the language, such as scatter and gather,...". Such an inclusion of MPI-like constructs would have wide-ranging consequences. -p15ff Classification parameters The paper mentions "three approaches for interprocess communication: distributed shared memory, message passing, or a combination of both." What about shared-memory communication and high-level approaches such as those in HPF or OpenMP? The discussion of the remaining parameters is likewise fuzzy: for example, the discussion of the implementation strategy does not deal with optimizations (and the specific role of the Java semantics specification in this context). -p18 MultiJav For someone not already knowing this system this description is unreadable. Similar comments apply to other systems discussed in the paper. -p26 Using Java Library A library cannot modify symtax or semantics of a language -- in contrast to the statement made there. -p27 Charlotte Charlotte is classified as a library but offers compiler annotations. -pp30-31 The conclusions drawn here with respect to performance are highly questionable -- not supported by any experimental data. -p34 JCI JCI is not a tool for HPC and does not belong here. -pp36ff Manta The description of Manta is very long, and much room is devoted to present very old runtime results (JDK 1.1.3 vs. actual 1.3). -pp40ff HPJava HPJava is called (for unknown reasons) JDPE in this paper. The bibliographic reference is wrong and should read "HPJava: a Data-Parallel Extension to Java", by Carpenter, Zhang, Fox, Li, and Wen It should also be noted that HPJava is based upon adlib. -p43, second paragraph: "the permission for methods ... to be remotely accessed, which is not allowed in RMI": this is nonsense. -p44, second paragraph: "... Java supports passive objects but only Java// supports active objects (threads and actors)." What about Java thread objects, remote objects? -p45: the characterization of a future object is wrong. -p46ff: UKA-Serialization and KaRMI cannot be regarded as "model", but implementation improvement. -p49ff: Javia is not a tool for HPC but a basic network interface. -p50ff: the conclusions drawn are again quite questionable ("JCI outperforms mpiJava" 1st line of pg. 51; "Manta should also outperform all the other systems ..." 1st line of pg. 52. -p55: last three lines: "...passing all the overhead of *dynamic* inspection to the compiler..." How can the compiler perform dynamic inspection? "new features in the language ... following MPI or PVM". The authors should distinguish between language and libraries. -p61: Reference 31: "Lea D." instead of "Doug L." F: Presentation Changes