Subject: CCPE Portal C549 From: "Didier, Brett T" Date: Fri, 14 Sep 2001 15:19:20 -0700 To: "'gcf@indiana.edu'" CC: "Schuchardt, Karen L" X-UIDL: 9ca6ef8dbc1e0000 X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000 Received: by mailer.csit.fsu.edu (mbox gcfpc) (with Cubic Circle's cucipop (v1.31 1998/05/13) Sun Oct 14 17:36:03 2001) X-From_: fox@mailer.csit.fsu.edu Sun Oct 14 17:31:42 2001 Return-Path: Delivered-To: gcfpc@csit.fsu.edu Received: from dirac.csit.fsu.edu (dirac.csit.fsu.edu [144.174.128.44]) by mailer.csit.fsu.edu (Postfix) with ESMTP id BB8C323A07 for ; Sun, 14 Oct 2001 17:31:42 -0400 (EDT) Received: from localhost by dirac.csit.fsu.edu (AIX4.2/UCB 8.7) id RAA17674; Sun, 14 Oct 2001 17:31:37 -0400 (EDT) Resent-Message-Id: <200110142131.RAA17674@dirac.csit.fsu.edu> Delivered-To: fox@csit.fsu.edu Received: from mask.uits.indiana.edu (mask.uits.indiana.edu [129.79.6.184]) by mailer.csit.fsu.edu (Postfix) with ESMTP id 7310223A19 for ; Fri, 14 Sep 2001 18:19:30 -0400 (EDT) Received: from gate.pnl.gov (gate.pnl.gov [130.20.64.137]) by mask.uits.indiana.edu (8.10.1/8.10.1/IUPO) with ESMTP id f8EMHYp02394 for ; Fri, 14 Sep 2001 17:17:35 -0500 (EST) Received: from vscan2.pnl.gov ([130.20.64.141]) by pnl.gov (PMDF V5.2-33 #42505) with SMTP id <01K8BY9TXQH08WZ6QA@pnl.gov> for gcf@indiana.edu; Fri, 14 Sep 2001 15:19:27 PDT Received: from 130.20.128.21 by vscan2.pnl.gov (InterScan E-Mail VirusWall NT) ; Fri, 14 Sep 2001 15:19:28 -0700 Received: by PNLMSE1.pnl.gov with Internet Mail Service (5.5.2650.21) id ; Fri, 14 Sep 2001 15:19:27 -0700 Message-id: <29E6E93D92576F4DB85FE89FF2C23F44C17648@pnlmse03.pnl.gov> MIME-version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-type: text/plain; charset="ISO-8859-1" Resent-To: Geoffrey Fox Resent-Date: Sun, 14 Oct 2001 17:31:36 -0400 Resent-From: Geoffrey Fox C: Paper and Referee Metadata Paper Number: C549 Date: September 7, 2001 Paper Title: Engineering Interoperable Computational Collaboratories on the Grid - Advance in the Discover Project Authors: Vijay Mann and Manish Parashar Referee: Karen Schuchardt/Brett Didier Address: Pacific Northwest National Laboratory, PO Box 999, Richland WA 99352 Recommendation: Accept provided changes suggested are made D: Referee Comments to Editor Our recommendation is this paper should undergo a substantial revision prior to journal publication. Several team members at PNNL reviewed this paper and we all agreed that the abstract really had us interested in reading this paper, but our interest had disappeared by the time we reached page 3. The paper is very redundant, several sentences, phrases and pieces of information are repeated throughout the paper. We also had a hard time understanding exactly what was trying to be conveyed; we didn't think the different elements of the paper (interoperability between collaboratories, middleware substrate, DISCOVER computational collaboratory) were tied together very well and that led to our confusion. Technically, we think there is some interesting work to report here, but the paper has a feel of a couple of different papers tossed together instead of a single well-thought out paper being presented. A technical editor should definitely be hired by the authors to review the paper. E: Referee Comments to Author and Editor Technically, this paper poses an interesting problem in how to combine focused collaboratories and allow them to inter-operate. It provides some approaches on how this could be performed and also postulates that a middleware substrate has been built that supports this interoperation. As written, the first part of the paper (through Section 4) is not supported by the remaining sections of the paper. The paper does not describe the process of combining different collaboratories, but only how the distributed operation of the DISCOVER collaboratory is being supported. The paper also does not motivate the concept of tying collaboratories together from a domain scientists perspective, it really don't establish the need for this work. The implied message is "If we build it, they will come". What would make this an interesting paper is an example of two of the different collaboratories mentioned, being modified to use this middleware substrate and reporting what was required to make this happen, what problems were encountered and overcome, and what was learned from the process. The paper doesn't outline or analyze what would be required for an existing collaboratory to utilize this middleware substrate. Does it require modifying client-side software, server-side software or both to use the CORBA IDL specification? Furthermore, while this is technically possible, experience shows isn't a practical or feasible solution because of organizational considerations, unless there is almost universal acceptance of the solution and a real impetus to change existing software. CORBA has not been able to attain this status. The paper also seems to equate focused Collaboratories and PSEs. In our opinion this isn't accurate, and not all of the collaboratories mentioned in the introduction are focused, therefore they definitely do not qualify as a PSE. Several of the collaboratories are also not grid-enabled, unless a very broad definition of the grid is being used that essentially equates to distributed computing. Although, there are technical topics of interest in this paper, the different concepts brought forth (interoperability between collaboratories, middleware substrate, DISCOVER computational collaboratory) were not tied together well and caused confusion about what was trying to be conveyed. Section 3.1 states that true interoperability can be achieved with Shared Protocols, but the implementation that has been followed by the project is Shared Interfaces and APIs. A path towards the Shared Protocol approach is mentioned by integrating the services into the CORBA ORB as standard CORBA services. Is this feasible? Does this require involvement in the standards process? Section 4 states that a server that provides a single instance of an application or a service is only required to provide the second level interface. A justification for why this was done was not provided. It seems, like this service would still need to advertise its capabilities. Also, in this section the paper states CORBA IDL was chosen instead of XML to describe interfaces, but no justification was given. The experimental evaluation is also not very extensive and should undergo further evaluation. More than ten simulations should be used for the values used in the graphs. Not all of the diagrams said they were mean averages, were they? This is important for Figure 9, since the explanation is not sufficient if it is a mean response time. Regarding server memory consumption in Figure 10, given the total memory use of the server is less than 10MB for 25 clients then this doesn't seem like a very important evaluation criteria. F: Presentation Changes The test describing the hourglass model in Section 3.2 should have a diagram, the text alone was confusing. The conclusion mainly summarizes the paper contents, what are the real conclusions? The information regarding future work is interesting and could be further expanded upon. The appendices did not add to the paper and should be removed. I recommend the use of a technical editor, there were several instances throughout the document where sentences were missing words, were poorly structured or were otherwise incomplete. .