I enclose 3 Referee reports on your paper. We would be pleased to accept it and could you please send me a new version before November 5 99 Please send a memo describing any suggestions of the referees that you did not address Ignore any aggressive remarks you don't think appropriate but please tell me. I trust you! Thank you for your help in writing and refereeing papers! Referee 1 **************************************************************** Subject: C444 JGSI Review Paper no.: C444 Title: Design, Implementation, and Evaluation of Optimizations in a Java Just-In-Time Compiler Overall recommendation: Publish with minor revisions Comments to authors: An interesting and clearly presented paper. Some of the English, although correct, is a little awkward in places. section 3.3, para 1: the last sentence (Because CSE ...) does not make sense. section 5.1, line 2: small method -> small methods section 5.2, last para: Most people would argue that Java interfaces are not true multiple inheritance. section 5.2, last para, line 2: only a class -> only one class section 6, line 7: "Swing 1.0.3 was executed with clicks to all tabs" - please explain what this means. section 6.2 and following. In the performance bar charts it is hard to distinguish between the different keys - maybe a problem in converting ti PDF? also the key panel overlaps the main panel in some cases. section 6.7: It would be good to see the performance of the applications with *none* of the optimisations applied, in addition to what is presented. It is not obvious that the optimisations are completely independent. section 7, para 3, lines 5-6: introduce a runtime test newly -> introduce a new runtime test section 7, para 5, line 2: replacement -> replacements section 8, line 1: delete full stop after "compiler" Comments to editor: No significant problems. Referee 2 ************************************************************************ Subject: C444 JGSI Review ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Overall Recommendation: Accept ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This paper describes a number of optimizations performed by IBM's Java JIT compiler, and evaluates their effectiveness. The paper is technically correct and presents results relevant to the Java community. The wording could be improved in places; below are some suggestions for such improvements. I appreciate that the authors solicited the help of a native English speaker to polish the prose; I would recommend a second pass. Detailed comments ~~~~~~~~~~~~~~~~~ Page 2: "Section 0" should read "Section 7" Page 2: "Inlining of Dynamic method call is applied ..." should read "Inlining of Dynamic method calls is applied ..." Page 2: The authors mention that they introduced a new byte code instruction to represent an object's interior pointer. It is not clear to me why they needed to introduce a new byte code, as opposed to just generating the appropriate PowerPC instruction. Page 3: "... was reduced to 6 by the existing method, ..." would be clearer as "... was reduced to 6 by Gupta's method, ..." Example 2: (Keep in mind that I don't know PowerPC assembly.) I did not understand the instruction "twi LLT, r7, 1". I would have expected to see something like "twi EQ, r7, 0" (i.e. trap if r7 is 0), since the next instruction divides r3 by r7. Page 4: "... the top pointer to the object must be kept in the memory or register. Because CSE generates an interior pointer of an object and the garbage collector does not scan an interior pointer of an object because of its performance." is rather confusing. Here's a stab at rephrasing it: "... the top pointer to the object must be kept in memory or in a register, so that the garbage collector classifies the object as reachable. For performance reasons, the collector does not scan the interior pointers generated by CSE." Page 4: "First, ... Finally, ..." don't go together well. Replace by: "First, ... Second, ...." Page 5: "If all of them tests fail" should read "If all of these tests fail" Page 5: "then the C runtime library is executed" should read "then a C function in the Java runtime is executed" Page 5: "In this section, we describe two optimizations of method calls: first, inlining of static method calls. Secondly, and then devirtualization of dynamic method calls." could be phrased more succinctly as: "In this section, we describe two optimizations of method calls: inlining of static method calls, and devirtualization of dynamic method calls." Page 6: "the constructor is invoked when a new class is created" should read "the constructor is invoked when a new object is created". Page 6: "the JIT compiler inlines small method" should read "our JIT compiler inlines small methods". Page 6: "Dynamic method call is an important feature of object-oriented language because of the flexibility, and it is therefore used frequently." I would prefer: "Dynamic method call is the defining feature of object- oriented languages, and is used frequently." Page 6: "... to improve the performance of dynamic method call." --> "... to improve the performance of dynamic method calls." Page 6: "... in which the class hierarchy may change in the future." --> "... in which the class hierarchy may change at runtime." Page 7: "If CHA finds that only a class implements an interface class, ..." --> "If CHA finds that only one class implements an interface, ..." Page 7: "inlining of static method call" --> "inlining of static method calls" Page 7: "raytrace (a body of mtrt)" What is mtrt? Page 8/Figure 2: The authors indicate that Figure 2 shows some dark bars some white bars. However, in my copy of the paper all the bars are white The same is true for Figures 3 through 7; the text suggests that some bars should be non-white, but in my copy, all bars are white. (This problem might have been introduced during the conversion of the paper to PDF ...) Page 9: "Figure 3 shows the distribution of object types in type inclusion test at runtime." --> "Figure 3 shows the distribution of object types in type inclusion tests at runtime." Page 9: "In each case, 14%, 28%, and 45% of the original array accesses are used." I'm not sure what the authors are trying to say here. One interpretation would be: "For db, 14% or the original array accesses are used; for tsgp, 28%; and for tmix, 45%." Page 9: "In all cases, it optimizes ..." --> "In all cases, our method optimizes ..." Page 10: "The performance in devirtualizing dynamic method call ..." --> "The performance in devirtualizing dynamic method calls ..." Page 11: "Therefore, the is still scope to improve ..." --> "Therefore, there is still room to improve ..." Page 12/Section 6.7: I would like to see one extra bar labeled NONE in Figure 8, showing the baseline performance of the benchmark programs, without any of the optimization applied. Also, the authors should clarify if the reported times include the time needed to perform the optimizations, or if this overhead was factored out. Page 12: "Figure 8 thus shows that all optimizations contribute to an improvement in the performance." I would qualify this statement as follows: "Figure 8 thus shows that all optimizations contribute to an improvement in the performance of some of the programs." Page 14: "For javac, the inlined codes of both approaches account for 92%." I'm not sure what the authors are trying to say. Maybe: "For javac, the inlined codes of both approaches are sufficient to decide 92% of all type inclusion tests." Page 14: "... because direct-binded code is undone ..." --> "... because direct binding code is undone ..." Page 14: "... we showed that there is still scope for further improvements of the Java performance." --> "... we showed that there is still room for further performance improvements." ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Referee 3 ******************************************************************* Subject: C444 JGSI Review C444:Design, Implementation, and Evaluation of Optimizations in a Java Just-In-Time Compiler a)Overall Recommendation paper needs changes and should be re-submitted before being accepted b)Words suitable for authors This paper discusses and evaluates optimizations that can be performed by a JIT compiler for Java. Several of the author's optimization techniques improve on previous work, or propose new solutions to optimizing OO code. The problem with this paper is that the authors fail to show timing results that measure overall performance improvements that result from their optimizations. The timing measures they present, compare only times of various combinations of their optimizations. They need to add a base case with which to compare their optimizations...for each benchmark, they should measure the running time on a JIT with none of their optimizations, and then show % improvements in execution time by applying different optimizations. Some of their optimization techniques are interesting, and I suspect would result in improvements in total execution time for a least some of these benchmarks, but they never show these numbers and these measures need to be in this paper...adding compiler optimizations to a JIT will result in longer running times, they need to show that the increase in compiling time due to their optimizations is recovered in producing faster code. A secondary problem is the authors claim in the Related Work section that "Finally, nothing can outperform devirtualization with inlining". This is a strong claim with no reference supporting it. I would guess that polymorphic in-line caching will actually do better in some cases because it really can handle overridden method calls. Devirtualization only can be applied to virtual method calls that are not polymorphic at run-time. So, if their claim is true (and I think it is too strong), they need to cite a result that shows this, or demonstrate this result in this paper. minor changes: The bar graphs in figures 4-7 are unreadable. In the pdf version, each bar is colored with the same stripe pattern. As a result, there is no way to tell which bar is which in graphs that show multiple values for each benchmark.