Subject: Revised Draft BRP Quick Look Letter Report Date: Fri, 12 May 2000 12:06:24 -0600 From: "John H. Birely" To: venky@deas.harvard.edu, jbirely@ix.netcom.com, Bryson@nas.nasa.gov, gcf@nu.cs.fsu.edu, Happer@pupgg.princeton.edu, leith1@llnl.gov, Rflucas@lbl.gov, wrm@umich.edu, Steven.orszag@yale.edu, sugar@sakar.physics.ucsb.edu, teter@ccmr.cornell.edu, mhw@research.bell-labs.com CC: Paul.Messina@ns.doe.gov, messina@cacr.caltech.edu, william.reed@ns.doe.gov, david.luginbuhl@ns.doe.gov, nowak1@llnl.gov, christensen5@llnl.gov, obrien12@llnl.gov, dmccoy@lanl.gov, srlee@lanl.gov, dona@sandia.gov, jekelly@sandia.gov, macaluso@apo.saic.com, deb.rubin-bice@ns.doe.gov ASCI Blue Ribbon Panel Colleagues, Thanks for your prompt and useful comments on the first draft of our quick look letter report. Further review of the first draft by NNSA headquarters has determined that there is no need to treat this as Official Use Only, which means that we can now use open E-mail to transmit the document. I have attached a revised draft of the quick look letter report that I believe accommodates all of your comments and suggestions for improvement. It is in Word 2000, and employs change tracking so that it's easier to see what revisions have been made. I am also including the revised draft as an included text message below, but my E-mail program loses the change tracking. If you have trouble opening or reading the attachment, I'll be glad to fax you a copy of the version with change tracking (I'll do this immediately for Margaret and Michael). Please give me your comments on the revised draft by Midnight Pacific Daylight Time this Sunday so I can transmit the next draft on Monday, May 15. Your silence will be construed as your approval of the draft. Also, I would like your comments on whether we are close enough to a final draft that, after we accommodate your comments, Venky can transmit our draft report to General Gioconda without another round of revisions. Thanks very much, John DRAFT FOR COMMENTS Division of Engineering and Applied Sciences Harvard University Pierce Hall 217A Cambridge, MA 02138 May 15, 2000 Brig Gen Thomas F. Gioconda Acting Deputy Administrator for Defense Programs United States Department of Energy 1000 Independence Avenue, SW Washington, DC 20585 Dear General Gioconda: I write to give you a quick look report of the ASCI Blue Ribbon Panel. Our review examined the ASCI program in response to your letter of April 17, 2000 (See Appendix A) and Dr. David Crandall’s charge to the Panel. The Blue Ribbon Panel comprised a distinguished group of experts from academe, the United States Government, the nuclear weapons laboratories and the private sector (see Appendix B). We met in Washington, DC on May 4-5, 2000. The panel appreciated the opportunity to discuss the ASCI program with you, and to review our preliminary observations, findings and recommendations in depth with Dr. Crandall and Dr. Paul Messina on May 5, 2000. We find that the ASCI Program remains generally on track, but must address significant risks, some of which are found in the entire Stockpile Stewardship Program (SSP). The Panel finds that the ASCI Program has made substantial progress since our last review in January 1999. ASCI is clearly one of the crown jewels of the SSP, and the program pace appears about right. Many of the issues we raised in our February 1999 report have been addressed, although significant concerns linger and new issues have arisen. Our preliminary observations, findings and recommendations on these points are elaborated in the balance of this letter, which is organized around the questions that you and David Crandall asked us to consider. We intend to give you a final report by June 1, 2000. Are technical program objectives properly aligned to support our Nation’s shift from test-based to science-based stockpile stewardship? Is ASCI making a contribution to the Stockpile Stewardship Program? We have refined our earlier statement of the most important ASCI program objectives: 1) Attract, train and retain the next generation of technical personnel to maintain and improve the stockpile. 2) Develop the computational capabilities necessary to solve present and future stockpile stewardship issues 3) Perform predictive simulations that integrate experimental data, better models, and past nuclear test data to help designers address relevant stockpile issues in the absence of nuclear testing. We still believe that these objectives are very ambitious, and that the time line is aggressive. This implies that there is risk in meeting these deadlines This risk has three technical dimensions. First, aggressive goals are proper in this type of program to achieve the maximum progress in the shortest possible time. Second, there is technical risk involved in many of the sub-areas since so many of the program objectives have never been attempted before. Third, there is risk that an unanticipated problem may occur for which no resources have been allocated. We believe risks such as these are acceptable, and even necessary, for such an ambitious project. The Panel believes that the level of risk of ASCI meeting its objectives has increased since January 1999, largely owing to budget issues. The training of the next generation of technical professionals remains the single largest problem facing not only ASCI, but also the core competencies throughout stockpile stewardship. ASCI is the magnet that will attract the computational talent that will lead the DOE labs through 2030 and beyond. ASCI is also a magnet that will attract talented students to the computational sciences, and we commend the program for its encouragement of university alliances to attract high quality, technically trained young people. ASCI should and will enable integration of new and improved physics modules to solve problems not heretofore addressable. These physics modules require a minimum critical level of computer capability to be useful and this capacity to a large extent governs the aggressive upgrade plans for ASCI hardware. New physics will reveal new phenomena, and new physics is enabled by new computer capability. Algorithmic and physics modeling advances are driven by smart scientists attacking new problems that were previously intractable , and ASCI is the enabler. Hardware and software plans for ASCI are aggressive, but the program should give increasing attention to areas in the commercial world such as data bases and communications where development is also rapid. The Panel finds that ASCI is generally on-track The achievements of the CY99 mileposts and milestones are impressive. ASCI is making strong contributions to SSP, and could make even stronger contributions if some of the major issues discussed below are addressed successfully. Overall, we continue to believe that the ASCI program objectives are reasonably aligned to assure program success, and only their relative emphases need to be addressed. Is the ASCI program being properly executed to achieve its technical objectives? Is the program meeting its milestones? Two days of meetings did not allow us to complete a detailed analysis of a program as large and complex as ASCI. In addition, the program is still in a relatively early development stage. Nonetheless, ASCI has already achieved spectacular computational science. ASCI is the world leader in developing and linking large applications and has demonstrated remarkable simulations of complex, multidisciplinary science on large-scale, parallel computers both in the core program and the Alliances. This computational science tour-de-force is critical for stockpile stewardship and is also a beacon for simulation-based science in industry and academia. The software (problem solving) environments developed in ASCI are the most sophisticated in the world. In some areas we do not have sufficient information to draw definitive conclusions, and in others it would be premature to do so. Therefore, in addressing the question of whether the ASCI program is being properly executed to achieve its technical objectives, we have focused primarily on identifying those areas that we believe to be crucial to the success of the program, and that require the ongoing attention of its management. Science The Panel still believes that a major question regarding the program is whether the level of scientific input is adequate to meet the objectives. Close ties to the SSP scientific work external to ASCI are essential. ASCI efforts must be tightly coupled to an active research program in the underlying scientific areas being modeled. The current state of the art in materials theory is insufficient to meet SSP requirements. DOE should support development of fundamental underpinnings of quantum simulation of materials. Hardware The hardware program is very impressive, and continues to be a world-class effort. The current plan is appropriate and has been successful. We endorse continuation of the current strategy, noting that dependence on one large system at a time entails significant but acceptable risk. It is important to invest in the future. In the near term, the Program should continue and enhance the Path Forward element. It is also imperative to engage in 5-10 year horizon research. We are pleased to hear that, since our earlier review, the necessity for the labs to take on the task of system integration and/or system software development in order to meet the schedules for the 30- to 100- TeraOps machines has been reduced. Budget pressures that have led to the cancellation of plans to acquire a 50-TeraOps machine as well as a delay in construction of the facility to house it have substantially increased the risk that ASCI will not meet its SSP objectives. We say more about this in the major section below on Resources. Algorithms Improvements in algorithms have traditionally played an equal role with increases in computer speed in advancing large-scale simulations. Attention to algorithms remains essential, with a need to balance between generality and strongly focused efforts. As in our earlier report, we stress the importance of work on nonlinear algorithms. Software ASCI software strategies are excellent. We applaud the progress in integration, attention to software engineering, and code verification. The Panel believes that further software enhancements are required, including development of a software life cycle/architecture plan, and development of a strategy for new software such as Problem Solving Environments similar to the hardware implementation strategy. Although ASCI’s current problem solving environment activities are excellent, there is remarkable progress in industry and academia in core software technologies driven by the Internet. These emerging ideas need to be evaluated and inserted into the ASCI environment as appropriate. State of the art software is no less important to ASCI than having the best available hardware, and we suggest a strategy of “software technology insertions” roughly every three years as is done with the hardware. ASCI can expect industry to supply base software technologies but currently it does not seem likely that commercial organizations will supply the needed high performance computing environments. ASCI needs an overall software plan addressing life cycle costs and enhancements within the context of an architecture that addresses the needs of ASCI and the leverage of commercial developments. We believe that the program can further increase code efficiency using general purpose software tools. For instance, ASCI should make operating systems investments for utilization and invest in compiler optimization (C++). The Panel was gratified to see the beginning of formulation of metrics for code development. The panel is pleased with the current balance among hardware resources, software development and algorithm research Visualization and Data Management In our previous report we observed that the Visualization Interactive Environment for Weapons Simulation (VIEWS) effort should be focused on increasing the understanding of the designers. Our impression at that time was that many designers were not convinced of the usefulness of visualization. We are pleased to report that this situation has been turned around, and that there are now good connections between visualization experts and the designers. This interaction is leading all three laboratories to develop enhanced visualization capabilities with appropriate balance between theaters and desktop visualization systems. We see that the data management effort has a good scope, and applaud the labs for engaging the very best experts in the visualization research community. We see a problem, however, in the transmission of very large amounts of data over networks for remote and collaborative analysis. To address this problem, the Panel recommends more focus on information extraction, that more in-house resources be applied to adapt state-of-the-art technologies, and that the labs continue their plans to pursue independent, large-scale visualization computational capability. Personnel The major problem in human resources noted in our February 1999 report has clearly become a crisis. The difficulty in attracting and retaining the best people for ASCI and the rest of the SSP threatens the nation’s ability to continue to certify the safety, security and reliability of the U.S. nuclear stockpile. A substantial contributor to this problem is actions taken in the name of improved security that have had a serious impact on morale, productivity, and retention of the best people. Every possible effort needs to be made to attract new talent. This should start with emphasizing the exciting science in the ASCI Program. The Computational Science Graduate Fellowship program has proved successful in bringing the best students to the labs, and doubling the program would be an inexpensive, highly leveraged investment to attract and retain outstanding people. We also support the emerging fellowships in computer science. The Academic Strategic Alliances Program (ASAP) is an excellent vehicle for attracting students to computational and computer science as well as for engaging the broader community. We applaud your resistance to cutting this program, and recommend that you consider expanding it. Support of the Blue Horizon system at the San Diego Supercomputing Center is a fine example of how ASCI's computers could be an important magnet to attracting students into computational science. Access to the world’s fastest computers, even on a limited basis, would provide unique opportunities to do the type of exciting research that stirs the imagination of excellent students. Initiatives such as retraining on-board employees are a good move. The labs and DOE have taken creative steps toward competitive compensation through salary and benefit management, but more needs to be done to improve the working environment, especially to address what appears to the Panel to be the heavy-handed implementation of security procedures DOE. Encouraging the professional development of ASCI personnel should continue to be another major human resource goal. This means fostering a research environment with freedom and flexibility to pursue new ideas that are not necessarily part of the ASCI implementation plan. An important step would be for the Department to make every effort possible to restore cuts in the Laboratory Directed Research and Development program. It is important for personnel to continually update their technical skills. Interactions with scientists in universities and industry should be strongly encouraged. This should include extended visits to universities and industrial laboratories by ASCI personnel. Cuts in laboratory travel budgets have made this sort of initiative much more difficult to pursue. Verification and Validation The Panel finds that the ASCI Program has made very good progress in code verification. We applaud the increased effort in code validation since our February 1999 report, and encourage further attention owing to the importance of code validation to the success of the SSP. We also encourage the Program to continue to strive to improve the agreement between simulations and experimental data. Cross-validation among diverse codes is one desirable way to achieve this. The Panel has concerns whether the Program is using state-of-the-art data management techniques, and about the path from, as well as the integration with, VIEWS in support of code verification and validation. Integration Within ASCI Integration of ASCI with laboratory experiments and the nuclear test data base remains a difficult and critical problem that is even more challenging in the absence of new nuclear tests. We did not see explicit paths and linkages among components except for occasional hints in the presentations we received. The Panel continues to believe that this is one of the areas of greatest exposure for the program, and that ASCI still needs to create integration experiences, both real and simulated, for new designers at the laboratories as well as in the university efforts. Are current and projected resources (budget, staff, facilities) sufficient and properly balanced to enable ASCI to meet its current objectives and thereby support national objectives? What would be the impact of a 10% reduction in the ASCI budget? We did not undertake an in-depth analysis of the ASCI budget, but it appears to us that since the Panel’s last review, actual and projected budget cuts have substantially increased program risk. It is not an overstatement to note that the future quality of the labs could depend upon their success in recruiting and retaining people within the ASCI program. In contrast to our position in February 1999, we find that currently projected ASCI facilities may be insufficient to meet SSP requirements. Budget Top-level ASCI resources appeared about right to us at the time of our February 1999 report. Subsequent cuts in projected growth of the program mean that it shares a worrisome attribute that the DOE 30-day internal review of November 1999 found with the overall SSP, namely that the ASCI program is wound too tight. This problem will only be exacerbated if the ASCI budget for the next few years is cut further to accommodate budget issues in the National Ignition Facility. A 10% cut in the ASCI budget would have serious impact on the ability of the SSP to continue to underwrite the safety, security and reliability of the U. S. nuclear stockpile. The Panel believes that it would be counterproductive to cut a successful and essential program that has demonstrated that it is one of the crown jewels of SSP. Not only would this cut substantially increase the risk that SSP will fail to meet its objectives, it would also send the negative signal that the importance of ASCI is reduced, making it even harder to recruit and retain the very best people. Facilities The Panel is pleased with the substantial progress toward realization of 10- and 30-TeraOps ASCI capability. We are pleased that our past concern about the success-oriented schedule for the Los Alamos 30-TeraOps facility employing concurrent design and construction has proved to be unwarranted. Cancellation of the acquisition of the 50-TeraOps machine and delay of the facility to house it will have a negative impact on ASCI code development mileposts and directed stockpile work. At the time of our February 1999 report, the program was already projecting a computing capacity shortfall by about a factor of two. This shortfall was validated by the DOE internal 30-Day Review. The additional computing capacity shortfall brought about by canceling the 50-TeraOps machine will substantially increase the risk toward meeting SSP objectives. SUMMARY In summary, the Panel finds that the ASCI remains generally on track. We believe that ASCI technical program objectives are properly aligned to support our Nation’s shift from test-based to science-based stockpile stewardship and only their relative emphases need to be addressed. ASCI is making strong contributions to the Stockpile Stewardship Program, and could make even stronger contributions if some of the major issues discussed herein are addressed. The ASCI program being properly executed to achieve its technical objectives and is meeting its milestones. We did not undertake an in-depth analysis of the ASCI budget, but it appears to us that since the Panel’s last review, actual and projected budget cuts have substantially increased program risk. It is not an overstatement to note that the future quality of the labs could depend upon their success in recruiting and retaining people within the ASCI program. In contrast to our position in February 1999, we find that currently projected ASCI facilities may be insufficient to meet SSP requirements. We offer the following for your continuing consideration: · ASCI has made substantial progress since our last review in January 1999. · ASCI is clearly a crown jewel of the SSP, and the program pace appears about right. · Training the next generation of technical professionals remains the single largest problem facing ASCI, and the core competencies throughout stockpile stewardship. · There is still a major question whether the level of scientific input to ASCI is adequate to meet the objectives. · The state of the art in materials theory is insufficient to meet SSP requirements. DOE should support development of fundamental underpinnings of quantum simulation of materials. · ASCI is the world leader in developing and linking large applications and has demonstrated remarkable simulations of complex, multidisciplinary science on large-scale, parallel computers both in the core program and the Alliances. · The hardware program is very impressive, and continues to be a world-class effort. The current hardware plan is appropriate and has been successful. · ASCI software strategies are excellent. ASCI has made progress in integration, attention to software engineering, and code verification · The major problem in human resources noted in our February 1999 report has clearly become a crisis. The difficulty in attracting and retaining the best people for ASCI and the rest of the SSP threatens the nation’s ability to continue to certify the safety, security and reliability of the U.S. nuclear stockpile. · The Computational Science Graduate Fellowship program has proved successful in bringing the best students to the labs, and doubling the program would be an inexpensive, highly leveraged investment to attract and retain outstanding people. · The Academic Strategic Alliances Program (ASAP) is an excellent vehicle for attracting students to computational and computer science as well as for engaging the broader community. We applaud your resistance to cutting this program, and recommend that you consider expanding it. · The labs and DOE have taken creative steps toward competitive compensation through salary and benefit management, but more needs to be done to improve the working environment, especially to address what appears to the Panel to be the heavy-handed implementation of security procedures DOE. · ASCI has made very good progress in code verification and has increased the effort in code validation. · Cuts in projected ASCI growth mean that the Program shares the worrisome attribute that the DOE 30-day internal review of November 1999 found with the overall SSP, namely that the ASCI program is wound too tight. This problem will only be exacerbated if the ASCI budget for the next few years is cut further to accommodate budget issues in the National Ignition Facility. · A 10% cut in the ASCI budget would have serious impact on the ability of the SSP to continue to underwrite the safety, security and reliability of the U. S. nuclear stockpile. · ASCI has made substantial progress toward realization of 10- and 30-TeraOps capability. · The additional computing capacity shortfall brought about by canceling the 50-TeraOps machine will substantially increase the risk toward meeting SSP objectives. I hope that you find these preliminary results of the ASCI Blue Ribbon Panel review to be useful. In closing, I would like to acknowledge the professional and cooperative nature of all of the participants in the review. Special thanks are due to Paul Messina and his staff, as well as to the laboratories’ ASCI program managers. The ability of Tina Macaluso of SAIC to accurately capture the contents of our briefings and discussions, and make them available to us on the web, made our job go much more smoothly. Once again, CAPT Jay McDonald, USN (Ret), ably documented our classified sessions. Mary Goroff of the Caltech Center for Advanced Computing Research provided outstanding technical coordination and administrative support. The hospitality and technical support of your DP staff contributed greatly to the effectiveness of our meetings. We plan to present you a final report by June 1, 2000. Until then, please contact me if you need amplification or clarification of my remarks. Sincerely yours, Venkatesh Narayanamurti Chair, ASCI Blue Ribbon Panel For the ASCI Blue Ribbon Panel: John H. Birely Stephen Bryson Geoffrey C. Fox William Happer Chuck Leith Robert Lucas William Martin Steven A. Orszag Robert Sugar Michael P. Teter Margaret H. Wright APPENDIX A Text from the Letter Chartering the Formation of the ASCI Blue Ribbon Panel APPENDIX B ASCI Blue Ribbon Panel Members APPENDIX C Blue Ribbon Panel Meeting Agendas --------------------------------------------------------------------- Name: quick4.doc quick4.doc Type: Microsoft Word Document (application/msword) Encoding: base64 --------------------------------------------------------------------- John H. Birely 1341 Monterrey Rd NE Rio Rancho, NM 87124, USA Voice: 505-896-5525 Mobile: 505-379-1069 Fax: 505-896-5517 jbirely@ix.netcom.com