Subject: Re: C429 JGFSI Reviews Resent-Date: Fri, 12 Nov 1999 08:45:29 -0500 Resent-From: Geoffrey Fox Resent-To: p_gcf@npac.syr.edu Date: Mon, 8 Nov 1999 17:28:18 -0800 (PST) From: heydon@pa.dec.com (Allan Heydon) To: Geoffrey Fox CC: najork@pa.dec.com Dear Geoff, We apologize for not replying to you by last Friday. Attached is a revised version of our article, in LaTeX, PDF, and PostScript formats. We have FedEx'ed you a paper copy together with a copyright transfer agreement; you should receive it by Wednesday. Appended are the referee comments you sent us, along with our replies. We have acted on almost all of the suggestions. Best, - Allan and Marc ----------------------------------------------------------------------------- > Referee 1 ************************************************** > > 2. Comments for Author(s) > ********************* > A great article. Exclamation use excessive. Attitude toward Sun is a > bit contentious. Liked the reference to THX1138. We have attempted to tone down the paper by removing some of the exclamation points. We have searched the paper for all references to Sun, and we believe the only statement that might be thought of as being contentious is our plea in the conclusions that Sun address some of the problems described in the paper. We feel this is a reasonable statement to make. > Referee 2 ******************************************************* > > Overall recommendation: interesting and worth reading, but frustrating > as well. Weak recommendation for publication. > > I found this article interesting but frustrating. While the author's > have made some interesting observations, they fail to provide important > details. > > For example, in Section 2.1, the authors note that they have a tool > that can give detailed information about the number of synchronization > operations performed by a Java application. Then they talk about all of > the transformations they performed to reduce the number of synchronization > operations. But they don't actually tell us the before and after number of > synchronization operations. Good point! Fortunately, we have saved versions of Mercator throughout its development, so we were able to measure the before and after synchronization and allocation costs. We have added these figures to the paper. > Similarly, it is unclear which of the issues raised in the paper are > significant, and which, if any, are insignificant. > > I suspect that the reason for this is that they performed the > optimizations as Mercator evolved, and they don't have two versions of > Mercator that they can compare (e.g., versions with the same > functionality, but with and without the changes described in this > paper). This would be most unfortunate; the paper would be much stronger > with this information. Unfortunately, the referee's hypothesis is correct. Although we have reported before-and-after numbers, those numbers come with the caveat that functional changes were made to Mercator along with the performance improvements. We have added this caveat to the paper. However, as we state there, we believe that the performance gains we report are conservative. We agree that the paper would be stronger if we could separate the performance changes from the functional ones; unfortunately, that's impractical at this point. > Also, JVM's vary greatly in how efficiently they implement synchronization. > In many of the early JVM's, synchronization was very expensive. In more > modern JVM's, it is relatively cheap, particularly for the uncontested > cases. How efficient is synchronization on the JVM the authors used? The srcjava runtime we use is extremely efficient; it currently holds the SPECjvm98 record. The combined operation of acquiring and releasing a lock requires only 23 cycles. We've added these points to the paper. > It isn't clear which of the performance issues were fixed in 1.2.2, besides > the InetAddress.getByName cache lock. It isn't clear to me that performance > bugs which have been fixed are interesting at all. I would be happy to > see discussion of all bugs fixed in 1.2.2 removed from the paper. The implementations of Integer.toString and Long.toString were also fixed. We believe it is worth acknowledging Sun's efforts at improving the core libraries, so we've decided to retain that material. > It really is unclear to me if the authors are performing premature/ > unnecessary optimizations in some places. For example, in Section 5, > the authors complain that the design of the java.net library causes > hundreds of allocations per second. On a 533Mhz Alpha, this is a problem? > Maybe so, but my gut impression is that this would be in the noise. If > the authors provided information on the performance impact of these > allocations, I might be convinced otherwise. In our initial version of Mercator, we allocated an average of 19.4KB worth of objects per downloaded document. The optimizations described in section 5 reduced this number to 3.6KB per document. Most of the optimizations were achieved through object reuse. We believe these savings are significant enough to be reported on. The point of the paragraph on Sockets mentioned by the referee is that the design of the class libraries prevents complete reuse; in other words, it is impossible to completely eliminate per-document allocations. Again, we feel it is worth reporting that fact. > Some of the "inefficiencies" the author's describe are there for a reason. > The authors complain that some of the String constructors are inefficient: > that is just wrong. Some of them perform more computationally demanding > tasks (e.g., converting from a UTF8 encoding to Unicode) than others. > Calling this an "inefficiency" is just plain wrong. Now, if the problem > is that when going through the constructor that takes an encoder and > providing a constructor that simply uses a zero for the high byte, the > result is much slower than using the constructor that doesn't take an > encoder, then that is an efficiency problem. We agree completely with the referee's point. We have changed the text to refer to this problem as a "pitfall" rather than an inefficiency; we have also pointed out that our main complaint is that we wished the performance difference between the two kinds of constructors had been better advertised. > Similarly, if StringBuffer were to be made unsynchronized, then when a > StringBuffer is converted into a String, you would have to copy the > character array (otherwise, unsynchronized access might allow you to > mutate a String, which would cause all kinds of security problems). We believe the referee did not understand our argument, since our suggestion was that an unsynchronized variant of StringBuffer should be used only in places where it is completely safe to do so, namely, where it can be statically proven that the StringBuffer is accessible by only one thread. We have modified that part of the paper to avoid confusion. > And again, if FileInputStream, FileOutputStream and RandomAccessFile > publicly supported reopen methods, it would have serious security > implications (passing a FileOutputStream to a method would allow that > stream to be reassociated with a different file). We do not completely understand the referee's objection. One way to interpret it is that the reopen method could be used to bypass Java's security manager. However, the reopen method could obviously be coded to use the same access control check as the constructor to prevent that problem. The parenthetical example indicates that the referee seems to have a different objection in mind, however. After considering a number of possible attacks, we have been unable to invent a scenerio in which the addition of a reopen method would allow a security breach. We have therefore not changed our paper in response to this comment. > Referee 3 ************************************************* > > 7) COMMENTS: > In section 2 "Our Development Tools", I think that it is not a > contribution of the paper. I suggest the section is shrunk. Then, > section 7 that is interested can be extended. We agree, section 2 was too long relative to its contributions. We have shortened it substantially. > If possible, in section 4.3, please show the improvement on each step. Good idea. We've added these results. ----------------------------------------------------------------------------- --------------------------------------------------------------------- Name: final.tex final.tex Type: TeX Document (application/x-tex) Encoding: base64 Name: final.pdf final.pdf Type: Acrobat (application/pdf) Encoding: base64 Name: final.ps final.ps Type: Postscript Document (application/postscript) Encoding: base64