Subject: C444 JGSI Review
Resent-Date: Thu, 30 Sep 1999 23:18:09 -0400
Resent-From: Geoffrey Fox <gcf@npac.syr.edu>
Resent-To: p_gcf@npac.syr.edu
Date: Sat, 18 Sep 1999 11:29:36 -0400 (EDT)
From: newhall@cs.swarthmore.edu (Tia Newhall)
To: gcf@npac.syr.edu

C444:Design, Implementation, and Evaluation of Optimizations in a Java
     Just-In-Time Compiler

a)Overall Recommendation

   paper needs changes and should be re-submitted before being accepted

b)Words suitable for authors

  This paper discusses and evaluates optimizations that can be performed
  by a JIT compiler for Java.  Several of the author's optimization
  techniques improve on previous work, or propose new solutions to
  optimizing OO code.  The problem with this paper is that the authors
  fail to show timing results that measure overall performance improvements
  that result from their optimizations.  The timing measures they present,
  compare only times of various combinations of their optimizations.  They
  need to add a base case with which to compare their optimizations...for
  each benchmark, they should measure the running time on a JIT with none
  of their optimizations, and then show % improvements in execution time by
  applying different optimizations.  Some of their optimization techniques
  are interesting, and I suspect would result in improvements in total
  execution time for a least some of these benchmarks, but they never show
  these numbers and these measures need to be in this paper...adding
compiler
  optimizations to a JIT will result in longer running times, they need to
  show that the increase in compiling time due to their optimizations is
  recovered in producing faster code.

  A secondary problem is the authors claim in the Related Work section
  that "Finally, nothing can outperform devirtualization with inlining".
  This is a strong claim with no reference supporting it.  I would guess
that
  polymorphic in-line caching will actually do better in some cases because
  it really can handle overridden method calls.  Devirtualization only can
  be applied to virtual method calls that are not polymorphic at run-time.
  So, if their claim is true (and I think it is too strong), they need to
  cite a result that shows this, or demonstrate this result in this paper.

  minor changes:  The bar graphs in figures 4-7 are unreadable. In the pdf
        version, each bar is colored with the same stripe pattern.  As a
        result, there is no way to tell which bar is which in graphs that
show
        multiple values for each benchmark.

c)Words for me if necessary