Subject: Revised Draft BRP Quick Look Letter Report
Date: Fri, 12 May 2000 12:06:24 -0600
From: "John H. Birely" <jbirely@ix.netcom.com>
To: venky@deas.harvard.edu, jbirely@ix.netcom.com, Bryson@nas.nasa.gov,
     gcf@nu.cs.fsu.edu, Happer@pupgg.princeton.edu, leith1@llnl.gov,
     Rflucas@lbl.gov, wrm@umich.edu, Steven.orszag@yale.edu,
     sugar@sakar.physics.ucsb.edu, teter@ccmr.cornell.edu,
     mhw@research.bell-labs.com
CC: Paul.Messina@ns.doe.gov, messina@cacr.caltech.edu, william.reed@ns.doe.gov,
     david.luginbuhl@ns.doe.gov, nowak1@llnl.gov, christensen5@llnl.gov,
     obrien12@llnl.gov, dmccoy@lanl.gov, srlee@lanl.gov, dona@sandia.gov,
     jekelly@sandia.gov, macaluso@apo.saic.com, deb.rubin-bice@ns.doe.gov

ASCI  Blue Ribbon Panel Colleagues,

Thanks for your prompt and useful comments on the first draft of our quick
look letter report.  Further review of the first draft by NNSA headquarters
has determined that there is no need to treat this as Official Use Only,
which means that we can now use open E-mail to transmit the document.

I have attached a revised draft of the quick look letter report that I
believe accommodates all of your comments and suggestions for
improvement.  It is in Word 2000, and employs change tracking so that it's
easier to see what revisions have been made.  I am also including the
revised draft as an included text message below, but my E-mail program
loses the change tracking.  If you have trouble opening or reading the
attachment, I'll be glad to fax you a copy of the version with change
tracking (I'll do this immediately for Margaret and Michael).

Please give me your comments on the revised draft by Midnight Pacific
Daylight Time this Sunday so I can transmit the next draft on Monday, May
15. Your silence will be construed as your approval of the draft.  Also, I
would like your comments on whether we are close enough to a final draft
that, after we accommodate your comments, Venky can transmit our draft
report to General Gioconda without another round of revisions.

Thanks very much, John

DRAFT FOR COMMENTS

                                         Division of Engineering and
Applied Sciences
                                         Harvard University
                                         Pierce Hall 217A
                                         Cambridge, MA 02138
                                         May 15, 2000

Brig Gen Thomas F. Gioconda
Acting Deputy Administrator for Defense Programs
United States Department of Energy
1000 Independence Avenue, SW
Washington, DC  20585

Dear General Gioconda:

I write to give you a quick look report of the ASCI Blue Ribbon Panel.  Our
review examined the ASCI program in response to your letter of April 17,
2000 (See Appendix A) and Dr. David Crandall’s charge to the Panel.  The
Blue Ribbon Panel comprised a distinguished group of experts from academe,
the United States Government, the nuclear weapons laboratories and the
private sector (see Appendix B). We met in Washington, DC on May 4-5,
2000.  The panel appreciated the opportunity to discuss the ASCI program
with you, and to review our preliminary observations, findings and
recommendations in depth with Dr. Crandall and Dr. Paul Messina on May 5,
2000.

We find that the ASCI Program remains generally on track, but must address
significant risks, some of which are found in the entire Stockpile
Stewardship Program (SSP).  The Panel finds that the ASCI Program has made
substantial progress since our last review in January 1999.  ASCI is
clearly one of the crown jewels of the SSP, and the program pace appears
about right.  Many of the issues we raised in our February 1999 report have
been addressed, although significant concerns linger and new issues have
arisen.  Our preliminary observations, findings and recommendations on
these points are elaborated in the balance of this letter, which is
organized around the questions that you and David Crandall asked us to
consider.  We intend to give you a final report by June 1, 2000.

Are technical program objectives properly aligned to support our Nation’s
shift from test-based to science-based stockpile stewardship?  Is ASCI
making a contribution to the Stockpile Stewardship Program?

We have refined our earlier statement of the most important ASCI program
objectives:

1)      Attract, train and retain the next generation of technical
personnel to maintain and improve the stockpile.

2)      Develop the computational capabilities necessary to solve present
and future stockpile stewardship issues

3)      Perform predictive simulations that integrate experimental data,
better models, and past nuclear test data to help designers address
relevant stockpile issues in the absence of nuclear testing.

We still believe that these objectives are very ambitious, and that the
time line is aggressive.  This implies that there is risk in meeting these
deadlines This risk has three technical dimensions.  First, aggressive
goals are proper in this type of program to achieve the maximum progress in
the shortest possible time.  Second, there is technical risk involved in
many of the sub-areas since so many of the program objectives have never
been attempted before.  Third, there is risk that an unanticipated problem
may occur for which no resources have been allocated.  We believe risks
such as these are acceptable, and even necessary, for such an ambitious
project.  The Panel believes that the level of risk of ASCI meeting its
objectives has increased since January 1999, largely owing to budget issues.

The training of the next generation of technical professionals remains the
single largest problem facing not only ASCI, but also the core competencies
throughout stockpile stewardship. ASCI is the magnet that will attract the
computational talent that will lead the DOE labs through 2030 and
beyond.  ASCI is also a magnet that will attract talented students to the
computational sciences, and we commend the program for its encouragement of
university alliances to attract high quality, technically trained young people.

ASCI should and will enable integration of new and improved physics modules
to solve problems not heretofore addressable.  These physics modules
require a minimum critical level of computer capability to be useful and
this capacity to a large extent governs the aggressive upgrade plans for
ASCI hardware.  New physics will reveal new phenomena, and new physics is
enabled by new computer capability.  Algorithmic and physics modeling
advances are driven by smart scientists attacking new problems that were
previously intractable , and ASCI is the enabler.  Hardware and software
plans for ASCI are aggressive, but the program should give increasing
attention to areas in the commercial world such as data bases and
communications where development is also rapid.

The Panel finds that ASCI is generally on-track  The achievements of the
CY99 mileposts and milestones are impressive. ASCI is making strong
contributions to SSP, and could make even stronger contributions if some of
the major issues discussed below are addressed successfully.
Overall, we continue to believe that the ASCI program objectives are
reasonably aligned to assure program success, and only their relative
emphases need to be addressed.

Is the ASCI program being properly executed to achieve its technical
objectives?  Is the program meeting its milestones?

Two days of meetings did not allow us to complete a detailed analysis of a
program as large and complex as ASCI. In addition, the program is still in
a relatively early development stage. Nonetheless, ASCI has already
achieved spectacular computational science.  ASCI is the world leader in
developing and linking large applications and has demonstrated remarkable
simulations of complex, multidisciplinary science on large-scale, parallel
computers both in the core program and the Alliances.  This computational
science tour-de-force is critical for stockpile stewardship and is also a
beacon for simulation-based science in industry and academia.  The software
(problem solving) environments developed in ASCI are the most sophisticated
in the world.

In some areas we do not have sufficient information to draw definitive
conclusions, and in others it would be premature to do so. Therefore, in
addressing the question of whether the ASCI program is being properly
executed to achieve its technical objectives, we have focused primarily on
identifying those areas that we believe to be crucial to the success of the
program, and that require the ongoing attention of its management.

Science
The Panel still believes that a major question regarding the program is
whether the level of scientific input is adequate to meet the objectives.
Close ties to the SSP scientific work external to ASCI are essential.  ASCI
efforts must be tightly coupled to an active research program in the
underlying scientific areas being modeled. The current state of the art in
materials theory is insufficient to meet SSP requirements.  DOE should
support development of fundamental underpinnings of quantum simulation of
materials.

Hardware
The hardware program is very impressive, and continues to be a world-class
effort. The current plan is appropriate and has been successful. We endorse
continuation of the current strategy, noting that dependence on one large
system at a time entails significant but acceptable risk.  It is important
to invest in the future.  In the near term, the Program should continue and
enhance the Path Forward element.  It is also imperative to engage in 5-10
year horizon research.  We are pleased to hear that, since our earlier
review, the necessity for the labs to take on the task of system
integration and/or system software development in order to meet the
schedules for the 30- to 100- TeraOps machines has been reduced.  Budget
pressures that have led to the cancellation of plans to acquire a
50-TeraOps machine as well as a delay in construction of the facility to
house it have substantially increased the risk that ASCI will not meet its
SSP objectives.  We say more about this in the major section below on
Resources.

Algorithms
Improvements in algorithms have traditionally played an equal role with
increases in computer speed in advancing large-scale
simulations.  Attention to algorithms remains essential, with a need to
balance between generality and strongly focused efforts.  As in our earlier
report, we stress the importance of work on nonlinear algorithms.

Software
ASCI software strategies are excellent.  We applaud the progress in
integration, attention to software engineering, and code verification.  The
Panel believes that further software enhancements are required, including
development of a software life cycle/architecture plan, and development of
a strategy for new software such as Problem Solving Environments similar to
the hardware implementation strategy.  Although ASCI’s current problem
solving environment activities are excellent, there is remarkable progress
in industry and academia in core software technologies driven by the
Internet.  These emerging ideas need to be evaluated and inserted into the
ASCI environment as appropriate.  State of the art software is no less
important to ASCI than having the best available hardware, and we suggest a
strategy of “software technology insertions” roughly every three years as
is done with the hardware.  ASCI can expect industry to supply base
software technologies but currently it does not seem likely that commercial
organizations will supply the needed high performance computing
environments.  ASCI needs an overall software plan addressing life cycle
costs and enhancements within the context of an architecture that addresses
the needs of ASCI and the leverage of commercial developments.  We believe
that the program can further increase code efficiency using general purpose
software tools.  For instance, ASCI should make operating systems
investments for utilization and invest in compiler optimization (C++).  The
Panel was gratified to see the beginning of formulation of metrics for code
development.

The panel is pleased with the current balance among hardware resources,
software development and algorithm research

Visualization and Data Management
In our previous report we observed that the Visualization Interactive
Environment for Weapons Simulation (VIEWS) effort should be focused on
increasing the understanding of the designers. Our impression at that time
was that many designers were not convinced of the usefulness of
visualization. We are pleased to report that this situation has been turned
around, and that there are now good connections between visualization
experts and the designers. This interaction is leading all three
laboratories to develop enhanced visualization capabilities with
appropriate balance between theaters and desktop visualization systems.  We
see that the data management effort has a good scope, and applaud the labs
for engaging the very best experts in the visualization research
community.  We see a problem, however, in the transmission of very large
amounts of data over networks for remote and collaborative analysis.  To
address this problem, the Panel recommends more focus on information
extraction, that more in-house resources be applied to adapt
state-of-the-art technologies, and that the labs continue their plans to
pursue independent, large-scale visualization computational capability.

Personnel
The major problem in human resources noted in our February 1999 report has
clearly become a crisis. The difficulty in attracting and retaining the
best people for ASCI and the rest of the SSP threatens the nation’s ability
to continue to certify the safety, security and reliability of the U.S.
nuclear stockpile.  A substantial contributor to this problem is actions
taken in the name of improved security that have had a serious impact on
morale, productivity, and retention of the best people.

Every possible effort needs to be made to attract new talent.  This should
start with emphasizing the exciting science in the ASCI Program. The
Computational Science Graduate Fellowship program has proved successful in
bringing the best students to the labs, and doubling the program would be
an inexpensive, highly leveraged investment to attract and retain
outstanding people.  We also support the emerging fellowships in computer
science.  The Academic Strategic Alliances Program (ASAP) is an excellent
vehicle for attracting students to computational and computer science as
well as for engaging the broader community. We applaud your resistance to
cutting this program, and recommend that you consider expanding it. Support
of the Blue Horizon system at the San Diego Supercomputing Center is a fine
example of how ASCI's computers could be an important magnet to attracting
students into computational science. Access to the world’s fastest
computers, even on a limited basis, would provide unique opportunities to
do the type of exciting research that stirs the imagination of excellent
students.

Initiatives such as retraining on-board employees are a good move.  The
labs and DOE have taken creative steps toward competitive compensation
through salary and benefit management, but more needs to be done to improve
the working environment, especially to address what appears to the Panel to
be the heavy-handed implementation of security procedures DOE.

Encouraging the professional development of ASCI personnel should continue
to be another major human resource goal. This means fostering a research
environment with freedom and flexibility to pursue new ideas that are not
necessarily part of the ASCI implementation plan.  An important step would
be for the Department to make every effort possible to restore cuts in the
Laboratory Directed Research and Development program. It is important for
personnel to continually update their technical skills. Interactions with
scientists in universities and industry should be strongly
encouraged.  This should include extended visits to universities and
industrial laboratories by ASCI personnel.  Cuts in laboratory travel
budgets have made this sort of initiative much more difficult to pursue.

Verification and Validation
The Panel finds that the ASCI Program has made very good progress in code
verification.  We applaud the increased effort in code validation since our
February 1999 report, and encourage further attention owing to the
importance of code validation to the success of the SSP.  We also encourage
the Program to continue to strive to improve the agreement between
simulations and experimental data.  Cross-validation among diverse codes is
one desirable way to achieve this.  The Panel has concerns whether the
Program is using state-of-the-art data management techniques, and about the
path from, as well as the integration with, VIEWS in support of code
verification and validation.

Integration Within ASCI
Integration of ASCI with laboratory experiments and the nuclear test data
base remains a difficult and critical problem that is even more challenging
in the absence of new nuclear tests. We did not see explicit paths and
linkages among components except for occasional hints in the presentations
we received.  The Panel continues to believe that this is one of the areas
of greatest exposure for the program, and that ASCI still needs to create
integration experiences, both real and simulated, for new designers at the
laboratories as well as in the university efforts.

Are current and projected resources (budget, staff, facilities) sufficient
and properly balanced to enable ASCI to meet its current objectives and
thereby support national objectives?  What would be the impact of a 10%
reduction in the ASCI budget?

We did not undertake an in-depth analysis of the ASCI budget, but it
appears to us that since the Panel’s last review, actual and projected
budget cuts have substantially increased program risk.  It is not an
overstatement to note that the future quality of the labs could depend upon
their success in recruiting and retaining people within the ASCI
program.  In contrast to our position in February 1999, we find that
currently projected ASCI facilities may be insufficient to meet SSP
requirements.

Budget
Top-level ASCI resources appeared about right to us at the time of our
February 1999 report.  Subsequent cuts in projected growth of the program
mean that it shares a worrisome attribute that the DOE 30-day internal
review of November 1999 found with the overall SSP, namely that the ASCI
program is wound too tight.  This problem will only be exacerbated if the
ASCI budget for the next few years is cut further to accommodate budget
issues in the National Ignition Facility.

A 10% cut in the ASCI budget would have serious impact on the ability of
the SSP to continue to underwrite the safety, security and reliability of
the U. S. nuclear stockpile.  The Panel believes that it would be
counterproductive to cut a successful and essential program that has
demonstrated that it is one of the crown jewels of SSP.  Not only would
this cut substantially increase the risk that SSP will fail to meet its
objectives,  it would also send the negative signal that the importance of
ASCI is reduced, making it even harder to recruit and retain the very best
people.

Facilities
The Panel is pleased with the substantial progress toward realization of
10- and 30-TeraOps ASCI capability.  We are pleased that our past concern
about the success-oriented schedule for the Los Alamos 30-TeraOps facility
employing concurrent design and construction has proved to be unwarranted.

Cancellation of the acquisition of the 50-TeraOps machine and delay of the
facility to house it will have a negative impact on ASCI code development
mileposts and directed stockpile work.  At the time of our February 1999
report, the program was already projecting a computing capacity shortfall
by about a factor of two.  This shortfall was validated by the DOE internal
30-Day Review.  The additional computing capacity shortfall brought about
by canceling the 50-TeraOps machine will substantially increase the risk
toward meeting SSP objectives.

SUMMARY

In summary, the Panel finds that the ASCI remains generally on track. We
believe that ASCI technical program objectives are properly aligned to
support our Nation’s shift from test-based to science-based stockpile
stewardship and only their relative emphases need to be addressed.  ASCI is
making strong contributions to the Stockpile Stewardship Program, and could
make even stronger contributions if some of the major issues discussed
herein are addressed. The ASCI program being properly executed to achieve
its technical objectives and is meeting its milestones. We did not
undertake an in-depth analysis of the ASCI budget, but it appears to us
that since the Panel’s last review, actual and projected budget cuts have
substantially increased program risk.  It is not an overstatement to note
that the future quality of the labs could depend upon their success in
recruiting and retaining people within the ASCI program.  In contrast to
our position in February 1999, we find that currently projected ASCI
facilities may be insufficient to meet SSP requirements.

We offer the following for your continuing consideration:

·       ASCI has made substantial progress since our last review in January
1999.

·       ASCI is clearly a crown jewel of the SSP, and the program pace
appears about right.

·       Training the next generation of technical professionals remains the
single largest problem facing ASCI, and the core competencies throughout
stockpile stewardship.

·       There is still a major question whether the level of scientific
input to ASCI is adequate to meet the objectives.

·       The state of the art in materials theory is insufficient to meet
SSP requirements.  DOE should support development of fundamental
underpinnings of quantum simulation of materials.

·       ASCI is the world leader in developing and linking large
applications and has demonstrated remarkable simulations of complex,
multidisciplinary science on large-scale, parallel computers both in the
core program and the Alliances.

·       The hardware program is very impressive, and continues to be a
world-class effort. The current hardware plan is appropriate and has been
successful.

·       ASCI software strategies are excellent.  ASCI has made progress in
integration, attention to software engineering, and code verification

·       The major problem in human resources noted in our February 1999
report has clearly become a crisis. The difficulty in attracting and
retaining the best people for ASCI and the rest of the SSP threatens the
nation’s ability to continue to certify the safety, security and
reliability of the U.S. nuclear stockpile.

·       The Computational Science Graduate Fellowship program has proved
successful in bringing the best students to the labs, and doubling the
program would be an inexpensive, highly leveraged investment to attract and
retain outstanding people.

·       The Academic Strategic Alliances Program (ASAP) is an excellent
vehicle for attracting students to computational and computer science as
well as for engaging the broader community. We applaud your resistance to
cutting this program, and recommend that you consider expanding it.

·       The labs and DOE have taken creative steps toward competitive
compensation through salary and benefit management, but more needs to be
done to improve the working environment, especially to address what appears
to the Panel to be the heavy-handed implementation of security procedures
DOE.

·       ASCI has made very good progress in code verification and has
increased the effort in code validation.

·       Cuts in projected ASCI growth mean that the Program shares the
worrisome attribute that the DOE 30-day internal review of November 1999
found with the overall SSP, namely that the ASCI program is wound too
tight.  This problem will only be exacerbated if the ASCI budget for the
next few years is cut further to accommodate budget issues in the National
Ignition Facility.

·       A 10% cut in the ASCI budget would have serious impact on the
ability of the SSP to continue to underwrite the safety, security and
reliability of the U. S. nuclear stockpile.

·       ASCI has made substantial progress toward realization of 10- and
30-TeraOps capability.

·       The additional computing capacity shortfall brought about by
canceling the 50-TeraOps machine will substantially increase the risk
toward meeting SSP objectives.

I hope that you find these preliminary results of the ASCI Blue Ribbon
Panel review to be useful.  In closing, I would like to acknowledge the
professional and cooperative nature of all of the participants in the
review.  Special thanks are due to Paul Messina and his staff, as well as
to the laboratories’ ASCI program managers.  The ability of Tina Macaluso
of SAIC to accurately capture the contents of our briefings and
discussions, and make them available to us on the web, made our job go much
more smoothly.  Once again, CAPT Jay McDonald, USN (Ret), ably documented
our classified sessions. Mary Goroff of the Caltech Center for Advanced
Computing Research provided outstanding technical coordination and
administrative support.  The hospitality and technical support of your DP
staff contributed greatly to the effectiveness of our meetings.

We plan to present you a final report by June 1, 2000.  Until then, please
contact me if you need amplification or clarification of my remarks.

                                         Sincerely yours,

                                         Venkatesh Narayanamurti
                                         Chair, ASCI Blue Ribbon Panel

For the ASCI Blue Ribbon Panel:

John H. Birely          Stephen Bryson  Geoffrey C.  Fox        William Happer
Chuck Leith             Robert Lucas            William Martin  Steven A.
Orszag Robert Sugar           Michael P. Teter        Margaret H. Wright

APPENDIX A
Text from the Letter Chartering the Formation of the ASCI Blue Ribbon Panel

APPENDIX B
ASCI Blue Ribbon Panel Members

APPENDIX C
Blue Ribbon Panel Meeting Agendas

    ---------------------------------------------------------------------
                 Name: quick4.doc
   quick4.doc    Type: Microsoft Word Document (application/msword)
             Encoding: base64

    ---------------------------------------------------------------------

John H. Birely
1341 Monterrey Rd NE
Rio Rancho, NM 87124, USA
Voice: 505-896-5525
Mobile: 505-379-1069
Fax: 505-896-5517
jbirely@ix.netcom.com