Subject: Your NSF ITR Preproposal 0077161 "ITR/ACS:Information Infrastruct ure for Distributed Collaborative Science ..." Date: Mon, 21 Feb 2000 13:03:13 -0500 From: "Koelbel, Charles H." To: gcf@nu.cs.fsu.edu CC: "Koelbel, Charles H." , "Jacobs, Clifford A" Geoffrey Fox Department of Computer Science Florida State University Ref: ITR-0077161 During the week of February 7, 2000 ITR Preproposal Review Panels met at the National Science Foundation in Arlington, and at other locations in Chicago and San Francisco to consider the 930 preproposals that were submitted in response to the Fiscal Year 2000 ITR Competition for proposals with budgets exceeding $500K. The preproposals were evaluated by the panels using the standard NSF merit review criteria: criterion (1) reflecting scientific importance and quality, and criterion (2) reflecting broader impact of the proposed research. In addition panelists were asked to identify preproposals that approach research activities in innovative ways rather than suggesting routine applications of existing technology. Proposals were selected for scientific excitement and promise, and for scope of vision, rather than for certainty of achieving their goals. Proposals were expected to represent new departures in research. Large projects, with budget well in excess of $500K, required sufficient justification for why they are large. The number and quality of the preproposals was high, and consequently not all preproposals with promising ideas could be invited to submit full proposals. The panels were asked to identify approximately the top 12% of the preproposals for submission as full proposals on April 17. NSF regrets to inform you that based on this review of your preproposal you are not encouraged to submit a full ITR proposal. Copies of the individual reviewers' comments and the panel summary for your preproposal are attached. If you have any questions regarding the review of your preproposal or about the ITR program, please contact us by e-mail at one of the addresses shown below: Charles Koelbel, ckoelbel@nsf.gov Clifford Jacobs, cjacobs@nsf.gov ITR Working Group The National Science Foundation 4201 Wilson Blvd. Arlington, VA 22230 ITR Preproposal Panel Summary Preproposal Number: 0077161 PI: Geoffrey Fox Institution: Florida State University Title: ITR/ACS:Information Infrastructure for Distributed Collaborative Science applied to Earthquake Analysis Relevance to Information Technology: Infrastructure for earthquake analysis and threat response management. Intellectual Merit: Earthquake modeling to drive IT (as weather modeling has in the past). Good team, good vision, but... It is mostly components based development. It is supposed to outlive web technology, but how? Iterative design and test-shown to work poorly in practice. Access control and accounting for remote computing not there. How do you prevent overload of the system? Broader Impacts: Well connected to earth science. ITR Evaluation Criteria (innovation in approach, scientific excitement and promise, justification for large size, community extending activities): Must be creative to succeed. Wide distribution. Other Strengths or Weaknesses: The panel was concerned about the lack of strong support from USGS and DOE to this project in the absence of funding. This is something FEMA should fund, not NSF ITR. Working with SCEC a plus. Panel Recommendation: _____ NSF should encourage further submission __X__ Not recommended for further submission PROPOSAL NO.: 0077161 INSTITUTION: Florida State University NSF PROGRAM: INFORMATION TECHNOLOGY RESEARC PRINCIPAL INVESTIGATOR: Fox, Geoffrey C. TITLE: ITR/ACS:Information Infrastructure for Distributed Collaborative Science applied to Earthquake Analysis RATING: Excellent REVIEW: Criterion 1. What is the intellectual merit of the proposed activity? How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? We are just putting into place a comprehensive sensor system for seismic activity. This project would establish the corresponding comprehensive simulation system. Very significant. A lot of these ITR proposals are for a non-ITR area with ITR tacked on. This particular proposal convinces me that it will advance ITR at the same time it advanced earth science. For instance, I thought to myself, if this project is supposed to advance IT so much, how come climate science has not already done so? Then I got to section 4.5 where it points out that it builds on IT developed for climate science. The proposal makes the case that we have to turn earthquake prediction into more of an observational science. We need to see the patterns as well as do the simulations (as we now do for weather). Seeing underground patterns will require new IT for visualization and integration of widely distributed and heterogeneous sensor data. I agree. I think the final proposal should give more details about how many bytes per year we are gathering and how many bytes per second we will need to transmit from place to place. The proposed simulation methods build on astrophysics. I see this proposal as well connected with the rest of earth science and technology as well as very different technologies. It is likely to be equally beneficial within its discipline and broadly to other disciplines. How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, please comment on the quality of prior work.) Very well qualified. The proposal is very clear on both the simulation activity and the IT activity and how they would relate to each other. It is clear that the authors have a deep knowledge (no pun intended). To what extent is the proposed activity creative and original? It is very comprehensive, and it will have to be creative to succeed, which I think it will. How well conceived and organized is the proposed activity? Very. Is there sufficient access to the necessary resources? Yes, 11 universities and 3 government agencies would participate. It builds on a very large existing infrastructure.n Criterion 2. What are the broader impacts of the proposed activity? How well does the activity advance discovery and understanding while promoting teaching, training, and learning? It will work with existing activities in education and crisis management. How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, geographic, etc.)? To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? It will establish a massive infrastructure for cooperative distributed development of science. Will the results be disseminated broadly to enhance scientific and technological understanding? Yes. What may be the benefits of the proposed activity to society? Understanding/predicting earthquakes has obvious huge benefits. This project is a necessary part of that. PROPOSAL NO.: 0077161 INSTITUTION: Florida State University NSF PROGRAM: INFORMATION TECHNOLOGY RESEARC PRINCIPAL INVESTIGATOR: Fox, Geoffrey C. TITLE: ITR/ACS:Information Infrastructure for Distributed Collaborative Science applied to Earthquake Analysis RATING: Good REVIEW: Proposal has two goals - building an information infrastructure for large-scale collaborations and earthquake analysis involving significant upgrade of the support for the General Earthquake Models. Project is closely tied to GEM but no explanation is given as to why this project is only funded at the level of 100K/yr. Some of the team members are well known, but it is not clear the most senior members will work together toward the same goals. Success of this project will have broad impact. [E] PROPOSAL NO.: 0077161 INSTITUTION: Florida State University NSF PROGRAM: INFORMATION TECHNOLOGY RESEARC PRINCIPAL INVESTIGATOR: Fox, Geoffrey C. TITLE: ITR/ACS:Information Infrastructure for Distributed Collaborative Science applied to Earthquake Analysis RATING: Poor REVIEW: ITR Proposal Requirements: Proposal for fundamental research in information technology No. Infrastructure for Earthquake analysis and threat response management. Component-based development. Science component is interesting and suitable for Geosciences directorate. Produces major innovations No. A few commercial systems available for business and industry in other application areas (e.g., utilities). Utilizes innovative approaches, not application of existing technology No. Application of existing technologies by Fox et al. High risk with high payoff upon success No. Not high risk. Justification for large project No. Science and proposed collaborative technology are almost entirely separable. Includes community extending concepts Yes. Merit Review: Advances knowledge and understanding No. Provides advanced capabilities. More suitable as USGS RFP or commercial effort. Qualifications of individual or team Very Good to Excellent. The unfunded groups at DoE and USGS currently have a full workload. Without funding, these groups are unlikely to produce significant results. Creative and original concepts No. Excellent vision that application area will outlive present technology (web), but no apparent plan to design a system that can evolve with technology. Well conceived and organized No. Iterative "prototype and test" paradigm has been shown many times by the Software Process Improvement community to be a poor development strategy. Sufficient access to resources Yes. Promotes teaching, training, and learning Yes. Inclusion of SCEC a plus. Broadens participation of underrepresented groups Unclear. Enhances infrastructure for research and education Unclear. Obviously there has to be access control and accounting so that the number of computational requests and collaborative sessions does not exceed available resources. Unclear who would be allowed to use the system and thus benefit from the effort. Results disseminated broadly Likely. Benefits society Yes.