I enclose 3 Referee reports on your paper. We would be pleased to accept it and could you please send me a new version before November 5 99 Please send a memo describing any suggestions of the referees that you did not address Ignore any aggressive remarks you don't think appropriate but please tell me. I trust you! Thank you for your help in writing and refereeing papers! Referee 1 *************************************************************** Subject: C433 JGSI Review Review of: A tale of two directories: Implementing Distributed Shared Objects in Java by Maurice Herlihy and Michael P. Warres a)Overall Recommendation Originality: Low to medium Technical quality: Medium Interest to audience: Medium to High b)Words suitable for authors A lot of work has been done on directory schemes; it is hard to come up with a new and exciting scheme. The novelty of the proposed scheme lies more in it being implemented in Java than in being new in its own right. The experiments are interesting, yet do not cover all the questions that are implied in the paper. The paper is quite readable and reasonably well organized. If only the directory scheme were more novel. c)Words for me if necessary This is not really all that much exciting. There has already been a lot of work on directory scheme *very* much like the one described here. I guess since it was accepted for the conference you have to accept it for CP&E, however, if this were a new submission to CP&E I would give it a clear leaning to reject, since it is just not new and exciting enough for a journal, and the experiments are of a rather limited nature. Just because you apply some old thing in the context of Java doesn't necessarily make it novel and exciting. Other than that, for a conference paper it's OK, I guess, and everything is quite readable and well presented. Referee 2 ************************************************************** Subject: C433 JGSI Review Title: A Tale of Two Directories: Implementing Distributed Shared Objects in Java Authors:Maurice Herlihy and Michael P. Warres a)Overall Recommendation ------------------------ CONDITIONAL ACCEPT b)Words suitable for authors ---------------------------- Sections 1-3 are well written and nice. The experiment section however is not very sound. The problem is that the authors discuss *one* small benchmark application (a shared counter). This particular benchmark has certain features (contention, message paths, etc.) that obviously make it unique. However, from the benchmarks results gained from this unique example, the authors derive general claims on the performance of the directory protocols. They even derive a hybrid protocol that better suits this benchmark but is inferior for other benchmarks (see section 6). I strongly recommend that the authors should try to present their analysis on more benchmarks instead of focusing on the shared counter alone. Some remarks to section 4.1: - Both RMI and UDP create threads. This is not properly analyzed. - Adding measurement code to threads will change their temporal behavior. What has been done to avoid heisenberg-effects? - Please repeat the Java benchmarks on a Wintel PC with Sun's JDK (including Symmantec's JIT). This JIT is known to be very efficient. Some remarks to section 4.2: - The authors mention that the message distance is longer (7) for the arrow protocol. They then use Fig 10 and Fig 11 to show that this is not true. This however is a feature of the shared counter benchmark and cannot be generalized. - I agree that prefetch latency can be hidden if enough work can be done concurrently. - Unfortunately, prefetching of objects is not discussed in the related work. - Again, it is a feature of the benchmark that there is enough contention to make this effect significant. Some remarks to section 6: >From the lines in Fig 13 it can be seen that the hybrid protocol performs better than the arrow protocol (for the shared counter benchmark). However, for 4, 7, 8, and 9 PEs there is either no clear advantage or even a disadvantage. Therefore, it is not reasonable to present performance results for other benchmarks for a single fixed number of PEs. This number might be carefully selected to show a particular result. I strongly recommend to have a combined analysis of all benchmarks in section 4. There should be results for all numbers of PEs. There should be results for different platforms (both hardware and JVM). If non of the protocols is superior for every case, the authors should try to derive rules of thumb for a programmer to select a protocol that is likely to show best performance for a given program. Some minor bugs: ---------------- - misunderstanding: At the end of section 2 the authors explain that in general a new thread is started that serves an incoming message. The first sentence of section 4.1 mentions that run() blocks at the home PE. I understand that there is a difference between the blocking of communication subsystem and a run() method that blocks at an object. However it took me longer than necessary to figure it out Please try to help the reader. - figures: what are the dashed lines in figure 3? Isn't there are cyclic arc missing in figure 3 (from the node that has the object to itself)? What is the reason for having solid lines in figures 4-6? Are these any different from the dashed lines in Figures 3, 7, and 8? Please say "node P" in Figure 4, "node Q" in Figure 5, and "node R" in Figure 6. - near the end of section 2: insert space in run()method - please use a consistent font/spelling for home/"arrow"/hybrid throughout the paper, i.e., italics or not, "" or not. - first line on p5: "multiprocessors([..." should be "multiprocessors [..." - bottom of p5: "...message to the the PE indicated..." - 2nd paragraph of section 4: "...time needed to for a collection..." - line 9 on p9: "In this abstract..." --> "In this paper..." - line 3 on p15: "When PE opens..." --> "When a PE opens ..." References: ----------- - be consistent: Proc. or Proceedings, Intl. or International - [9] insert spaces: J.*B. Carter, J.*K. Benner similar [22], [26], [27], and [28] - [7] and [8] are the same. - [11] "A.dam" --> "Adam", "java" --> "Java", "internet" --> "Internet" - [15] better: @article{ JavaParty ,author="Michael Philippsen and Matthias Zenger" ,title="Java{P}arty: Transparent Remote Objects in {J}ava" ,journal="Concurrency: Practice and Experience" ,volume="9" ,number="11" ,pages="1225--1242" ,month=NOV ,year="1997" } - [16] "kan" --> "Kan" - [20] did it appear? - [23] number and month missing - [28] pages missing - [29] don't insert - in URLs - [30] better: @article{ YuCox97 ,author="Weimin Yu and Alan Cox" ,title="{Java/DSM}: A Platform for Heterogeneous Computing" ,journal="Concurrency: Practice and Experience" ,volume="9" ,number="11" ,month=NOV ,year="1997" ,pages="1213--1224" } Referee 3 **************************************************************** Subject: C433 JGSI Review C433:A Tale of Two Directories: Implementing Distributed Shared Objects in Java a)Overall Recommendation accept provided changes suggested are made b)Words suitable for authors This paper presents a study comparing two directory implementations for supporting distributed shared objects in Java. The results of some of the authors initial performance measurements led them to develop a new directory scheme based on the best qualities of the initial two schemes. This is a very well written paper. The initial discussion of tracking down the performance critical issues and testing certain Java language features to explain some of the observed behavior is complete and very well described. Also, the description of how they developed their Hybrid protocol based on the results and analysis of their measurements of these other two protocols is thorough and well presented. The problem with this paper is that suddenly in section 6, the authors fail to explain or provide any analysis of their results. Everything leading up to section 6 would suggest that their Hybrid scheme should perform better than the Arrow and Home schemes. However, when they run larger benchmarks, the Arrow directory scheme outperforms Hybrid in all cases. This result is not obvious and must be explained. Other than this one problem, I think this is a very good paper with interesting results. I'd like to see this paper accepted. However, it really needs some analysis/explanation of the results presented in section 6 before it should be accepted. minor changes: In figure 6, adding labels to the nodes and/or edges indicating node Q and which edge something from node R comes from, would make the prose associated with this figure easier to follow. In figure 15, why does Arrow do better than Hybrid for the 9 PE case? c)Words for me if necessary Of the three papers I reviewed, this is the best one.