Subject: Re: C427 JGFSI Reviews
Resent-Date: Fri, 12 Nov 1999 08:38:43 -0500
Resent-From: Geoffrey Fox <gcf@npac.syr.edu>
Resent-To: p_gcf@npac.syr.edu
Date: Fri, 5 Nov 1999 11:44:27 -0500
From: Bryan Carpenter <dbc@npac.syr.edu>
To: Geoffrey Fox <gcf@npac.syr.edu>

Geoffrey,

Postscript for corrected version is attached.

Latex, PDF, etc is in
/usr/local/archives/public/projects/pcrc/papers/Java99:CPE/

On Mon, 11 Oct 1999, Geoffrey Fox wrote:
> [...]
> Please send a memo describing any suggestions of the
> referees that you did not address

See below

> Referee 1 *************************************************************

>
> - From the text I cannot see which JIT has been used on the Solaris
> machines? Was it HotSpot? HotSpot-people mentioned that they have
> built in better support for arrays of primitive types. I'm curious.
>

It wasn't HotSpot.  As stated in the paper we used JDK 1.2beta4.
Unfortunately this version of JDK is no longer installed here or
readily available, so it is now difficult to obtain more detailed
information about the JIT in that system.

> - Since I've seen nice performance numbers for Symmantec's JIT on
> Wintel machines, I would like to see the results of your benchmarks
> benchmarks between PCs.

I agree this would be interesting, but it takes a long time to rerun
all the benchmarks.  Given more time we could do it, but I think the
paper stands as it is.

> - Please try to give an average improvement factor for both versions
> of you streams in the text.

I've added some qualitative words in the discussion section, but I'm
not sure how to give a meaningful single numeric value to summarize
all cases.

>
> Referee 2 *************************************************************

>
> Good science and good statistics: I assume the authors did some
> regressions on the formulae (1) through (3) to get to Table 1.  Missing
> are statements regarding sample size, residuals, general statistics and
> general error analysis (for example, there appear to be consistent
> systematic errors in the plots).
>

Yes we used linear regression in some places to extract parameters.
I'm not sure detailed error analysis is particularly useful for this
kind of benchmark data.  Changes from one version of the software to the
next probably overwhelm/invalidate statistical error estimates made at
any one time...  probably nobody is ever going to try to repeat exactly
this set of measurements...

For the editor:

> References:
> -----------
> - [5] an updated version will appear in the same issue of CPE.
>   same for [17] and [18]

I don't know the correct form to deal with this.  After changes,
self-references are [5], [18] and [19]

> * Incorrect hyphenation - First sentence, para 2, page 11,
> "straightforward" is incorrectly hyphenated.

I couldn't persuade latex to break it any better way...

Bryan

    ---------------------------------------------------------------------
                   Name: serialize.ps
   serialize.ps    Type: Postscript Document (APPLICATION/PostScript)
               Encoding: BASE64