High Level Descriptor-driven Integration of Prototyping, Communication, and Large

Parallel Linear Solvers for Learning and for Simulation of Physical Systems M. Choptuik R. van de Geijn Richard A. Matzner The University of Texas at Austin Scott Klasky NorthEast Parallel Architecture Center Syracuse University (subcontracted participation) Project Description

Introduction:

The development of scientific and engineering computational models involves discretization of the differential equtions, followed by evolution of the code and evaluation of results.

Each of these steps can involve very complicated procedures at the lowest, technical level. However, what the investigator wants to control is typically at a much higher, abstract level. For instance, repeated experiments may be run using different discretization prescriptions. The task of modifying such prescriptions at the level of manipulating matrices or loops is very tedious and error prone, and can be very profitably taken over by an automatic system driven by a high level language, enabling the scientist to concentrate on the fundamental features of the algorithm, rather than the details of the algorithm.. Similarly, study of graphical output is essentially the only way to understand even one-dimensional situations and becomes essential in multi-dimensional simulations. Automatic, interactive and collaborative graphical presentations thus can dramatically improve the ease (and speed) with which computational methods are tried, analysed, understood and, if necessary, modified.

In the current computational context, parallel approaches are required to achieve competitive performance. Thus, we require support for parallelization, for parallel I/O and for visualization of large parallel data sets, and we require large, parallel, linear solvers.

Precisely these same features are desirable in the teaching of computational techniques to a new generation of scientists. It certainly is important that students understand discretization schemes at the lowest level, and they can be taught. But it is important to provide to the student rapid, reliable access to prototyping tools that can describe the fundamental features of the desired code, without explicit low level writing, can handle parallellization and adaption, can efficiently run on todays parallel machines, and can be viewed responsively, and typically in a collaborative setting. The integrated environment here will be demonstrated in computational science classes taught at The University of Texas, and at Syracuse University, and will include collaborative instruction, initially in a research environment, eventually in a classroom setting, between the Center for Relativity, The Department odf Computer Science at The Universityof Texas, and at Syracuse University. .

The approach we propose here is to extend existing functional examples of these tools, to parallel systems and to multi-dimensional problems.

The existing systems include:

RNPL (Rapid Prototyping Numerical Language), which supports high level description of discretization methods, and handles many “housekeeping” features of the coding, and can genetate full update code if desired. [156]

DAGH (Distributed Adaptive Grid Heirarchy), which supports automatic parallelization, load balancing, and adaptivity. [8]

PLAPACK, a library supporting parallel solution of large linear systems and algebraic eigenvalue problems.[161]

SCIVIS, which is a collaborative visulization and data manipulation tool. [2] [4]

These tools will be developed and extended to handle development and analysis of large, parallel systems, and to provide responsive tools for direct and collaborative teaching of computational science concepts and techniques..

The acronym RNPL stands for Rapid Numerical Prototyping Language[156]. It is a language for expressing time-dependent systems of partial differential equations and the information necessary for solving them using finite-difference techniques. It has advantages over traditional programming languages such as C and FORTRAN because it only requires the user to enter the essential structure of the program while it fills in the details [156].

RNPL can produce complete working programs, or a “skeleton” that the user can complete. The programs include facilities for reading parameters from a file, interactive control over output times and functions, memory management, and state dumping for calculation interruption and restart. The compilable code generated by RNPL is (at the user’s descretion) in C, C++, or FORTRAN.

Thus, RNPL provides the capability rapidly to generate C, C++, or FORTRAN code from high level specifications. The current implementation of RNPL introduces a number N of grid functions, ϕ^{A}[x^{μ}] A = 1 ⋯ N; μ = 1, μ = 1, 2, or μ = 1, 2, 3. on a Grid, G, labelled by i, j, k (or a superindex I, I = 1...(number of grid points)). The state of the system is then described by u^{ι}ι = 1 ⋯ N×(number of grid points).

Notice that we consider the temporal integration separately. We discuss a finite-difference evolution of the temporal dependences. The finite representation of the system (finite difference, finite element ...) discretizes the system into a matrix acting on the state vector.

Existing RNPL implimentation solves this system in the following way: The basic set of equations:

f_{ι}(u^{κ}) = 0

is iterated:

u_{i+1}^{κ} = u_{i}^{κ} − f_{i}(u_{i}^{κ})/ (\frac{∂f_{i}}{∂u_{i}^{κ}}

This iterative update succeeds in many contexts, but in some contexts (particularly with strong inhomogenieties in the domain) may not converge. The solution is to supply a general linear solver for the problem.

Our approach is to use PLAPACK, which we will extend here to large sparse parallel systems. PLAPACK Coding parallel algorithms is generally regarded as a formidable task. To make this task man-

ageable in the arena of linear algebra algorithms, the Parallel Linear Algebra Package (PLAPACK)

[161] is an infrastructure for coding such algorithms at a high level of abstraction. It is often believed that by raising the level of abstraction in this fashion, performance is sacrificed. Indeed there is a performance penalty, but by coding at a higher level of abstraction, more sophisticated algorithms can be implemented, which allows high levels of performance to be regained. This has been shown to be the case for the parallel solver package implemented using PLAPACK, which includes Cholesky, LU, and QR factorization based solvers for symmetric positive definite, general, and overdetermined systems of equations, respectively. Performance comparisons with other such packages, in particular ScaLAPACK, shows there is no real performance penalty when coding at this high level of abstraction.

In a dissertation by Tim Collins (Ph.D. in CS, UT-Austin, 1995, co-supervised by J.C. Browne and R. van de Geijn), hierarchical specification of matrices was shown to be a highly effective way of storing and manipulating large sparse linear systems arising from regular and irregular meshes [160]. In that research, type information for submatrices in the hierarchy was also encoded and exploited, allowing for highly elegant and compact implementation of sparse iterative and direct linear solvers. In particular, type information allows a solver to take advantage of structure in subproblems. The hierarchical specification allows for pruning of the matrix where it has zero entries. In addition, it allows for natural detection of ”supernodes” in sparse direct methods.

While PLAPACK was designed for dense linear systems, there are a number of reasons why PLAPACK is a natural candidate for integration with RNPL and DAGH, and systematic implementation of a hierarchical matrix infrastructure:

- The matrix distribution used by PLAPACK is physically based by which we mean that the distribution of the matrix is induced by the distribution of the physical domain for which the linear system is to be solved.

- Such domains are frequently partitioned hierarchically, e.g. in nested disection methods.

- It is uniquely appropriate for implementation of direct methods for solution of sparse linear systems since physically based matrix distribution implicitly specifies the distribution of dense interface matrices to distributed memory architectures.

- PLAPACK uses an object based programming paradigm, allowing hierarchical specification of matrices to be encoded using an elegant and simple interface.

- PLAPACK shares a common communication layer with DAGH, the Managed Message-Passing Infrastructured developed at UT-Austin by Carter Edwards [162].

- PLAPACK allows elegant and efficient implementation of the orthogonalization of vectors that define Krylov subspaces as occur in iterative solvers like conjugate gradient and GMRES.

DAGH

DAGH (which stands for Distributed Adaptive Grid Hierarchy) [?] was developed as a computational toolkit for the Binary Black Hole NSF Grand Challenge Project. It provides the framework to solve systems of partial differential equations using adaptive finite difference methods. The computations can be executed sequentially or in parallel according to the specification of the user. DAGH also provides a programming interface so that these computations can be performed by traditional Fortran 77 and Fortran 90 or C and C++ kernels.

Scivis

Scivis is a client-server data visualization and analysis system, which enables users to manipulate data from their scientific codes.[4] One of the major advantages of Scivis is that it is written in Java, so that it works on all computer platforms. Scivis uses a client-server model for data visualization. Users can send the data to the server anywhere sockets are supported so that the Scivis system can be used as a front end to other applications that originally produce the data sets. The application producing a data set can write the output data to a socket, which is connected to the Scivis server. Scivis then allows the user to analyze and visualize the data. The system is flexible and very responsive. The system allows vsualization rendering on remote fast servers, and only polygon lists are transported for the interactive visualization of 3D objects.

One of the key Scivis features is called User-Definable-Filters (UDF). Filters are user definable actions, performed on data set(s) to create new data based on this action. (For instance a filter may take a derivative in space and/or time, of the original data, to create a new dataset.) The Scivis system contains Wizards to help enbale users design their own filters. Filters in Scivis can be created on-the- fly so that users can immediately apply these new filters to her/his dataset. Since the Scivis system is collaborative, it is possible to create a powerful platform where many different users can can evaluate the performance of collaboratively developed filters.

Another major feature of Scivis is that it is collaborative. Scivis currently uses it’s own collboration system, but will eventually switch from using its own collaboration system, to one which takes advantage of the TANGO system.[139] Using TANGO, Scivis will become part of a very powerfull collaboration system which will allow it’s users to take advantage of many of the tools already introduced into TANGO. For example, TANGO has many features such as shared web-browsers, whiteboards, audio-visual communications, etc. TANGO has been incorporated with Web Wisdom to allow for a new virtual University.[140] This has allowed NPAC to offer courses at university while teaching them from Syracuse University.

Since filters in Scivis are user-definable, Scivis users Serilization in Java to tranfer the algorithms to other collaborators, in case the collaborator does not have the filter being used during the collaboration. Using this collaborative framework, users can exchange their data and their own personalized filters. Users of the system collaboratively are identified as Masters, or Slaves. A Master can modify data, a Slave can only view data. Multiple Masters are allowed in a collaborative session.

Proposal:

Working collaboratively, and using the tools we are developing, Choptuik, van de Geijn, Matzner and Klasky will develop and extend the four components of this effort into a unified coherent development and pedagogical tool for large scale scientific computational development and for collaborative education and collaborative computational simulation.

Extensions include:

RNPL extension to include automatic solution of the generated equations by linearization and full solution of the equivalent linear-system (matrix) problem.

DAGH extension to inclusion of large linear system solvers such as PLAPACK, and PLAPACK extensions.

PLAPACK extensions to parallel algorithms for the solution of sparse systems of linear and nonlinear equations arising on structured and unstructured meshes, as well as for gridless methods, through extensions to PLAPACK that express sparse matrices as hierarchical matrices. This will include routines which recognize the sparsity structure of the matrices, and adapt the algorithm accordingly.

SCIVIS extension to large parallel dataset manipulation, and to user manipulation and visualization of extremely large parallel datasets.

INTEGRATION:

The components described above will be integrated into a coherent environment. RNPL will be extended to directly produce DAGH code, and will be modified to interface the sparse-solver development of PLAPACK. DAGH will be extended to incorporate large parallel solvers through PLAPACK. RNPL and SCIVIS will provide theuser interface element to the developer.

APPLICATIONS:

The environment as conceived will have broad applicability in the development of simulation codes. The 1D description currently in RNPL will be extended to full 2- and 3- dimensional support. Applications include: One- and two- dimensional simulations of the behavior of matter in strong gravitational fields, and in particular the appearance of critical phenomena in gravitational collapse, and simulations of 3-dimensional strong-field gravitating systems.

The topics described in this proposal address the KDI topics listed in the announcement:

Knowledge Networking: -focus on attaining new levels of knowledge integration, information flow, and interactivity among people, organizations, and communities. -enhance communication across disciplines, languages, and cultures -improve the processing and integration of knowledge from different sources, domains, and non-text media types -increase the effectiveness of teams, organizations, classrooms, or communities that work together across distances or over time -deepen understanding of the ethical, legal, and social implications of new developments in connectivity

In particular the research would focus on interactive and collaborative tools for rapid development and analysis of multi-dimensional structured adaptive PDE codes. This constitutes Knowledge Networking Foundational Research in: -Computational aspects of distributed intelligence: dynamic task allocation, interaction, coordination, process and organization representation, collective learning, consistency management, protocol, negotiation -Managing heterogeneity and interoperability in dimensions such as syntax, semantics, scale, and structure -Large-scale remote data acquisition, distributed data analysis, experiment and sensor control, and simulation; especially interactive and real-time aspects -Distributed knowledge: sharable ontologies, processes for distributed classification and taxonomy, collaborative knowledge construction, representation and filtering tools, digital libraries and repositories across disciplines and application domains, and translation of representations

In The New Computational Challenges, these topics contribute to: -Interplay Between Computations and Data Matthew Choptuik and Richard A. Matzner Binary Black Holes: Coalescence and Gravitational Radiation, ASC/PHY 9318152 (ARPA Supplemented)

1. Summary of Previous Results

The ASC/PHY 9318152 Grand Challenge Project was initiated in September 1993. Richard Matzner is the Principal Investigator and M. Choptuik is a coInvestigator on this grant. More than 100 publications have so far resulted from the Binary Black Hole Grand Challenge Alliance. (see: http://www.npac.syr.edu/projects/bh/) Three of the technical components to be extended here (RNPL, SCIVIS, and DAGH) were developed in the context of the Binary Black Hole Grand Challenge Alliance. Firther the research of that Grand Challenge presented and presents a suite of pedagogical and research applications for the integrated environment proposed here.

The purpose of the Alliance is to compute the interaction of black holes and the gravitational radiation produced. The Alliance developed a parallel adaptive structure to take advantage of the fast, large computers being promised. Computer science structures, such as data structures definitions, and interface definitions, to direct the construction of the “toolbox” (which has become the “problem solving environment”) were immediately addressed. One feature was adaptive mesh refinement, which was immediately demonstrated in 1-d by Choptuik^{[22]}, and developed within the Alliance^{[23, 24]}.

Initial effort concentrated on High Performance Fortran, which is an extension of FORTRAN90^{[11]}. The Alliance began with a collection of legacy codes, and these codes were successfully expressed in HPF (High Performance Computing)^{[11]}. This work was carried out by computational scientists (especially Masso, Seidel , Cook, …) and computer scientists (especially Haupt, …). Further, the “toolkit” effort produced early results^{[25]}. However, we were aware of the need to accommodate parallel computing, and immediately undertook efforts to do so.^{[26]}. It became clear that the limitations of HPF inhibited easy implementation of adaptive parallel structures such as would become necessary in the Alliance codes. Thus, an effort was begun to develop DAGH (Distributed Adaptive Grid Hierarchy) which was written in C and built on an underlying structure called SDDA (Symmetric Distributed Data Array).

In early 1996 the HPF efforts were concluded, and concentration centered exclusively on DAGH development. DAGH consists of a C++ driver which handles parallelization (it uses space-filling curves to convert from the conceptually 3-d computational space to a linear layout of processors). It also handles adaptive refinement, given routines to define an error measure and threshold^{[27, 28, 29, 30, 31, 32]}. It contains standard clustering routines, as well as injection and restriction routines, to communicate at the boundaries between different levels of refinement. In most applications as intricate as computational relativity, however, specially-tuned programs have to be supplied by the computational scientist. DAGH is written so that this C++ structure calls application science modules which can be written in C or in FORTRAN^{[8]}. In the Alliance, the practice has been to write these codes in FORTRAN90. Codes are typically written in a unigrid context by the developer, following certain rules: no global variables, array bounds passed in at call, etc., that guarantee parallelizability, and with a FORTRAN90 driver which calls the various science modules. We have written a script which reads the F90 driver and constructs a DAGH driver to run the program in parallel (adaptive refinement is being debugged). This means that we have a fast and efficient development process which can be translated immediately into production-style DAGH code for large-scale development or production runs. DAGH has been adopted into other research applications^{[33, 34]} and has been extended in other research directions^{[35]}. Relativity, i.e. applications-science use of the computational resources, began with the set of legacy codes referred to above. A period of testing and evaluation ensued. A number of systems were studied: spherical systems, typically with scalar radiation to provide a dynamical field;^{[37, 38, 39]} initial studies of axisymmetric vacuum collapse;^{[40]} and collapse to black holes with matter sources^{[41]}. Fully 3-dimensional data setting and initial computation was carried out by Matzner and Laguna in cosmology^{[42]} using an ADM (g-dot, K-dot) code developed by Kurki -Suonio, and a similar code was used at NCSA^{[137]}.Additionally, we had expertise on 2+1-d characteristic codes, which allow integration all the way to radiative infinity[43]. Work soon began on a 3-dimensional characteristic code. A number of technical approaches were studied to expand the computational science resources available to the Alliance^{[44]}. Some of these took the form of internal reports^{[46, 47, 48]}.

Validation of codes proceeded in the 1-d case in work by Evans and Coleman^{[49]}, who studied gravitational collapse of a radiation fluid, and found critical phenomena related to the Choptuik^{[50]} effect. Development proceeded on broad paths with a number of test and example cases used to develop technique for the Einstein equations and for the important problem of radiation extraction^{[121, 51]}. Critical Phenomena remain a topic of current interest, and code development, especially in the spherical case, has continued to touch on this problem^{[52, 53, 54, 55]}.

There are a number of ways that data can be set for multiple black hole configurations. Cook et al^{[56]} had, prior to the initiation of the Alliance, shown computationally convergent methods of preparing data, based on the “classic” approach of using the York technique on conformally flat backgrounds. Cook extended this to a study of quasi-circular black hole orbits^{[57]}.

Early 2-dimensional black hole collision simulations were done using maximal lapse (a singularityavoiding coordinate system)^{[58]}. A number of these studies were carried out, and laid the groundwork for the considerably more demanding 3-d simulations: These included studies of distorted and rotating black holes,^{[59, 60, 61, 155]} both vacuum and from collapse ^{[121]}, and a thorough development of the dynamics of the collision^{[62, 63, 64]}. These studies (based also on previous work by Shapiro and Teukolsky^{[41]}) culminated in a thorough description of the surface of merging black holes, explicating many previously unclear aspects of the process^{[65]}. A fuller exposition subsequently was given in^{[66, 67, 68]}.

An important aspect of computational physics is to compare with other (perhaps approximate) methods^{[130, 101, 100, 103, 102]}. Perturbation theory of distorted black holes surely describes the late evolution of a black hole merger. A number of studies have been carried out predicting radiation in situations that can be treated by close (standard perturbative) or other approximate methods. For axisymmetric problems, these results are in often surprising agreement with computation^{[69, 70, 71, 72, 73]}. See also work on perturbative fields on fixed backgrounds^{[102]}. Three-dimensional black hole work has progressed with maximal slicing in 3-d^{[45, 114]}. However, it became clear that an Eddington-Finkelstein adapted coordinate system is preferable because the representation of a single black hole is static in that case.

At present we have succeeded in long-term single black hole evolution for both nonrotating (Schwarschild) an drotating (Kerr) black holes. By imposing blending boundary conditions to analytically known solutions, we can in some situations run the code forever. Norms of changes of field variables across the grid decrease to, and remain at, machine round-off levels. [36] We are investigating the range of parameters that allows this result. We have also demonstrated a black hole moving across the grid for periods up to 60M^{[74]}, and shorter times for moving spinning black holes [36]. Further, we have been able to demonstrate brief evolutiuons of interacting spinning black holes moving through the computational grid [36]. Apparent horizon “locking” and is necessary for better performance. This means that we will activelytrack the apparent horizon and fix coordinates to it. We have developed three separate approaches: expansion in spherical harmonics;^{[75, 76, 77, 78, 79]} direct solution of the apparent horizon elliptic equation,^{[80]} and curvature flow methods^{[81]} to the tracking problem, and we are enabling horizon tracking for new runs.

Essential to simulations of gravitational waveforms is the ability to extract waves at infinity. We have developed a method based on matching to a perturbative outer module,^{[73, 82, 83]} and a matching to characteristic evolution^{[84, 85, 86, 87, 89, 90, 91]}. Various technical aspects of the matching approaches require computation or integration on the matching sphere, and these have been developed^{[92, 93, 94]}.

A substantial effort was made to rewrite the Einstein system in a way that more closely presented its hyperbolic nature. This work proceeded on two fronts. A scheme which is based on the Arnowitt-Deser-Misner formalism, but which introduces additional spatial variables, was introduced by Bona and Masso,^{[95]} and developed substantially^{[96],[97, 98]}. A new formulation, which deals with one higher time derivative of the metric, and demonstrated explicit hyperbolicity, was also developed^{[12, 99, 82, 104, 105, 106]}. This latter formalism has had significant impact outside relativity, and has been studied in the context of more general field theory.

A number of more or less general reports have been given describing the Binary Black Hole Grand Challenge Alliance^{[107, 108, 109, 110, 111, 112, 113, 45]} and other work supported/stimulated by the Alliance^{[44]}.

Cray Research used figures from Grand Challenge Alliance research on the cover of its 1997 University Research and Development Grant Report.

Human Resource Development

Our group has a large and growing number of graduate students who share our enthusiasm regarding the future of computational science. These students receive an outstanding grounding in computational physics, numerical analysis and systems-development skills, and a continued high level of graduate training is a primary goal of our overall research program. They contribute substantially to the research program.

A very large number of students and postdocs were involved in the Binary Black Hole Grand Challenge. Those from Texas supported by the Alliance or by other NSF funding to Matzner and Choptuik were: Matthew Choptuik (postdoc, now Associate professor of Physics), Robert Marsa (PhD 1995, now a post-doctoral fellow in the Center), Dale Choi (PhD expected 1997), Steven Liebling (PhD expected 1997), Randall Correll (PhD expected in 1999), Mark Dubal (postdoc), Tim Galavan (graduate student), Reid Guenther (PhD 1995), Eric Hirschman (postdoc), Paulo Holvorcem (postdoc), Mijan Huq (PhD 1996, now a postdoc in the Center), Scott Klasky (PhD 1994, now a postdoc at Syracuse), John LaChapelle (postdoc), Juan Lara (graduate student), Manish Parashar (postdoc), Premana Premadi (PhD 1996), Jonathan Thornburg (postdoc), Robin Tuluie (PhD 1993) David Neilsen (graduate student), Scott Hawley (graduate student) Chun Yen Wang (graduate student), Carmen Molina-Paris(graduate student), Jodie Martin (undergraduate student), Mike Hicks (undergraduate student), Johnathan Thornburg (postdoc).

Results of Prior Research Sponsored by NSF

Robert van de Geijn

Grant: CMS-9522964 Principle Investigators: Greg Rodin (PI), J.C. Browne (Co-PI), J. Tinsley Oden (Co-PI), Robert van de Geijn (Co-PI) Period: April 1, 1995 - March 31, 1998 Amount: $1,200,000 Title: Application of High Performance Computing to Micromechanical Characteristics of Composite Materials

This grant has supported 4 Ph.D. students (1 has completed his Ph.D., 1 will defend in Aug. 1998, and 2 will finish within the next two years), 1 MS student, and 3 post-doctoral students. In addition, the project supported a visiting scientist from the People’s Republic of China.

The principal results obtained during the project are as follows:

Fast Boundary Element Methods: We have developed FLEMS, a 3-D boundary element code for rapid solution of many-particle problems in linear elasticity [163, 164] This code allows the integration of any boundary element method with any iterative strategy that includes a fast summation method. The present version includes the standard collocation boundary elemetn mehthod and a GMRES solver in which matrix-vector multiplication is performed with the fast multipole method. FLEMS is the first code of its kind for solving multiscale materials problems.

Fast Dislocation Dynamics Methods: Following theoretical ideas developed in connection with FLEMS, we have developed a similar approach and code for dislocation dynamics simulations [165, 164]. This code is being adapted by researchers at LLNL, ALCOA and in France.

Modeling Error Estimates: We have introduced and analyzed the concept of modeling errors. These errors are associated with replacing a fine-scale model with a coarse-scale model. Clearly this displacement procedure is central to multiscale problems. Main results have been published by Zohdi et al. [166] and Oden and Zohdi [167].

Domain Decomposition Methods: We have developed two complementatry domain decomposition methods for multiscale problems. The first method [166, 167]. is based on hierarchical partitioning of the domain. The second [168] allows one to represent complex microstructures as a combination of problems that involve individual miscrostructural features whose interactions are computed using a structured mesh.

References

[1] http://www.npac.syr.edu/projects/bh/ Click on “New Developments”. Multiple interacting black holes are an active research project, and funding for the continuation of this work is requested under this proposal. Results will be posted to this website as they are developed.

[2] http://kopernik.npac.syr.edu:8888/scivis/

[3] http://godel.ph.utexas.edu/)(link GC Toolkit).

[4] B. Ki, S. Klasky, “Collaborative Scientific Visualization” Concurrency: Practice and Experience, (Nov, 1997).

[5] R. Matzner, H.E. Seidel, S. Shapiro, L. Smarr, W-M. Suen, S. Teukolsky, J. Winicour, “Geometry of a Black Hole Collision,” Science, (Oct, 1995).

[6] S.L. Shapiro, S.A. Teukolsky and J. Winicour, “Toroidal Black Holes and Topological Censorship,” Phys. Rev. D 52, 6982 (1995).

[7] http://kopernik.npac.syr.edu:8888/scivis/index.html

[8] http://www.ticam.utexas.edu/parashar/public html /DAGH/ http://www.npac.syr.edu/projects/bbh/DAGH/ M. Parashar and J.C. Browne, “System Engineering for High Performance Computing Software: The HDDA/DAGH Infrastructure for Implementation of Parallel Structured Adaptive Mesh Refinement,” to be published in the Structured Adaptive Mesh Refinement Grid Methods, IMA Volumes in Mathematics and its Applications, Springer-Verlag. M. Parashar and J.C. Browne,“Systems Engineering Issues in the Implementation of an Infrastructure for Parallel Structure Adaptive Meshes,” Invited paper, Workshop of Structured Adaptive Mesh Refinement Grid Methods, SIAM Conference on Parallel Processing for Scientific Computing, March, 1997. M. Parashar and J.C. Browne,“Object-Oriented Programming Abstractions for Parallel Adaptive Mesh-Refinement,” Parallel Object-Oriented Methods and Applications (POOMA), February, 1996. M. Parashar and J.C. Browne, “On Partitioning Dynamic Adaptive Grid Hierarchies,” Proceedings of the 29^{th} Annual Hawaii International Conference on System Sciences, January,1996. M. Parashar and J.C. Browne,“Distributed Dynamic Data-Structures for Parallel Adaptive Mesh-Refinement,” Proceedings of the International Conference for High Performance Computing, December,1995.

[9] “Numerical Evolution of Black Holes with a Hyperbolic Formulation of General Relativity”, M. A. Scheel, T. W. Baumgarte, G. B. Cook, S. L. Shapiro, and S. A. Teukolsky, submitted to Phys. Rev D

[10] “Well-behaved harmonic tome slices of charged, rotating, boosted black hole”, G. Cook, M. Scheel, submitted to Phys. Rev D (1997)

[11] http://www.npac.syr.edu/ T. Haupt, ”Fortran 90 Implementation of the Berger-Oliger Algorithm for Adaptive Mesh Refinement,” Status Report-02. N. Chrisochoides, G. Fox, T. Haupt, ”A Computational Tookit for Colliding Black Holes and CFD,” SCCS-626, in proceedings of AAIA Conference, (June, 1994). T. Haupt, M. Choptuik, “Definition of Fortran90 data structures and an example program that uses them.” T. Haupt, “Fortran 90 Implementation of the Adaptive Mesh Refinement Algorithm,” http://www.npac.syr.edu/project/bbh/AMR/AMR-02.ps M. Miller, T. Haupt,“Gravitational Wave Extraction - A Benchmark?” SCCS-681.

[12] Y.Choquet-Bruhat and J.W. York, “Geometrical Well Posed Systems for the Einstein Equations,” C.R. Acad. Sci. Paris, 321 1089 (1995).

[13] A. Abrahams, A. Anderson, Y. Choquet-Bruhat, J. W. York Jr., “Geometrical Hyperbolic Systems for General Relativity and Gauge Theories,”Phys Rev Lett. 75 3377 (1995). “A non-strictly hyperbolic system for the Einstein equations with arbitrary lapse and shift,” gr-qc/9607006.

[14] Nigel T. Bishop, Roberto Gomez, Paulo R. Holvorcem, Richard A. Matzner, Philippos Papadopoulos, Jeffrey Winicour. “Cauchy-Characteristic Evolution and Waveforms,” Journal of Computational Physics, Volume 136, Number 1, pg.140-167, (September 1, 1997).

[15] http://wwwrel.ph.utexas.edu/Members/mijan/GC/June97/june.html

[16] http://wwwrel.ph.utexas.edu/ mijan/blackhole/

[17] P. Anninos, G. Daues, J. Massó, E. Seidel, and W-M. Suen, “Horizon boundary conditions for black hole spacetimes,” Phys. Rev. D 51, 5562 (1995).

[18] M.A. Scheel, T.W. Baumgarte, G.B. Cook, S.L. Shapiro S.A. Teukolsky, “Numerical Evolution of Black Holes with a Hyperbolic Formulation of General Relativity,” gr-qc/9708067 [19] The Binary Black Hole Grand Challenge Alliance, “Gravitational wave extraction and outer

boundary conditions by perturbative matching,” submitted to Phys. Rev. Let. (1997).

[20] R.A. Matzner, “Null Adapted Data and Coordinate Conditions for Multiple Black Holes,” Texas/IGPP Workshop Presentation, Feb. 3,4, 1997 (http://www.npac.syr.edu/projects/bh/).

[21] R.P. Kerr and A. Schild, “Some Algebraically Degenerate Solutions of Einstein’s Gravitational Field Equations,” Applications of Nonlinear Partial Differential Equations in Mathematical Physics, Proc. of Symposia b Applied Math., Vol. XV11, (1965). “A New Class of Vacuum Solutions of the Einstein Field Equations,” Atti del Convegno Sulla Relativita Generale: Problemi Dell’Energia E Onde Gravitazionale, G. Barbera,ed.,(1965).

[22] “An Implementation of the Berger-Oliger Mesh Refinement Algorithm for the Wave Equation in Spherical Symmetry,” Matthew W. Choptuik, technical report (March 1994).

[23] “Adaptive Mesh Refinement in Numerical Relativity,” J. Masso, E. Seidel, and P. Walker, in General Relativity (MG7 proceedings), R. Ruffini and M. Keiser (eds.), World Scientific (1995).

[24] “Fortran 90 Implementation of the Berger-Oliger Algorithm for Adaptive Mesh Refinement,” Tomasz Haupt (1995).

[25] “A Computational Toolkit for Colliding Black Holes and CFD,” N.P. Chrisochoides, G.C. Fox, and T. Haupt (1994).

[26] “Multithread model for dynamic load balancing parallel adaptive PDE computations,” Nikos Chrisochoides, to appear in Applied Numerical Mathematics Journal (1995).

[27] “Distributed Dynamic Data-Structures for Parallel Adaptive Mesh-Refinement,” M. Parashar and J.C. Browne, in Proceedings of the International Conference for High Performance Computing (December 1995).

[28] “Object-Oriented Programming Abstractions for Parallel Adaptive Mesh-Refinement,” M. Parashar and J.C. Browne, in Parallel Object-Oriented Methods and Applications (POOMA) (February 1996).

[29] “On Partitioning Dynamic Adaptive Grid Hierarchies,” M. Parashar and J.C. Browne, in Proceedings of the 29th Annual Hawaii International Conference on System Sciences (January 1996) (IEEE).

[30] “System Engineering for High Performance Computing Software: The HDDA/DAGH Infrastructure for Implementation of Parallel Structured Adaptive Mesh Refinement,” M. Parashar and J.C. Browne, to be published in the Structured Adaptive Mesh Refinement Grid Methods, IMA Volumes in Mathematics and its Applications, Springer-Verlag.

[31] “Systems Engineering Issues in the Implementation of an Infrastructure for Parallel Structured Adaptive Meshes,” M. Parashar and J.C. Browne, invited paper, Workshop in Structured Adaptive Mesh Refinement Grid Methods, SIAM Conference on Parallel Processing for Scientific Computing (March 1997).

[32] “A Common Data Management Infrastructure for Parallel Adaptive Algorithms for PDE Solutions,” M. Parashar, J.C. Browne, C. Edwards, and K. Klimkowski, accepted for presentation at Supercomputing 97 (November 1997).

[33] “A New Generation EOS Compositional Reservoir Simulator: Formulation and Discretization,” P. Wang, I. Yotov, M. Wheeler, T. Arbogast, C. Dawson, M. Parashar, and K. Sepehrnoori, in Proceedings of the Society of Petroleum Engineers, Reservoir Simulation Symposium (June 1997).

[34] “A New Generation EOS Compositional Reservoir Simulator: Framework and Multiprocessing,” M. Parashar, J.A. Wheeler, G. Pope, K. Wang, and P. Wang, in Proceedings of the Society of Petroleum Engineers, Reservoir Simulation Symposium (June 1997).

[35] “Integrated Data-Management for Computational Steering,” M. Parashar and J.C. Browne, submitted for publication to the 31st Annual Hawaii International Conference on System Sciences (January 1998).

[36] “Computational Evolution of Ninary Black Hole Spacetimes” R. Correll, PhD dissertation, The university of Texas at Austin (1998)

[37] R.L. Marsa and M.W. Choptuik, “Black Hole-Scalar Field Interactions in Spherical Symmetry,” Physical Review D, 54 4929-4943 (1996)

[38] “Collapse to Black Holes in Brans-Dicke Theory: I. Horizon Boundary Conditions for Dynamical Spacetimes,” Mark A. Scheel, Stuart L. Shapiro, and Saul A. Teukolsky, Physical Review D51 4208 (April 1995).

[39] “Collapse to Black Holes in Brans-Dicke Theory: II. Comparison with General Relativity,” Mark A. Scheel, Stuart L. Shapiro, and Saul A. Teukolsky, Physical Review D51Physical Review D51 4236 (April 1995).

[40] A.M. Abrahams and C.R. Evans,“Universality in Axisymmetric Vacuum Collapse”, Physical Review D51 3998-4003 (1994)

[41] S.L. Shapiro and S. Teukolsky, Physical Review 45 2739 (1992).

[42] H. Kurki-Suonio, P. Laguna and Richard A. Matzner, “Inhomogeneous Inflation: Numerical Evolution”, Physical Review D47 3611 (1993).

[43] “Null Cone Evolution of Axisymmetric Vacuum Spacetimes,” R. Gomez, P. Papadopoulos, and J. Winicour, J. Math. Phys. 35 4184 (1994).

[44] P. Laguna,“Alternatives to Finite Difference Methods in Numerical Relativity,” in Relativity and Scientific Computing, eds. F.W. Hehl, R.A. Puntigam, and H. Ruder (Springer/Verlag Berlin, 1996).

[45] “3-D Numerical Relativity at NCSA,” P. Anninos, K. Camarda, J. Masso, E. Seidel, W-M Suen, M. Tobias, and J. Towns, to appear in General Relativity (MG7 proceedings), eds. R. Ruffini and M. Keiser, World Scientific (1995).

[46] “Steps in the construction of an extraction module,” Nigel T. Bishop, Roberto Gomez, Philippos Papadopoulos, and Jeffrey Winicour, Pitt-Penn State-UniSA BBH Report (1996).

[47] “Stereographic Coordinate Patches,” R. Gomez, P. Papadopoulos, and J. Winicour, BBH working paper – program listing also available (1997).

[48] “Computational application of the eth formalism,” R. Gomez, P. Papadopoulos, and J. Winicour, in preparation (1997).

[49] “Observation of Critical Phenomena and Self-Similarity in the Gravitational Collapse of Radiation Fluid,” C.R. Evans and J.S. Coleman, in Phys. Rev. Lett. 72, 1782 (1994).

[50] Matthew W. Choptuik, “Universality and Scaling in Gravitational Collapse of a Massless Scalar Field,” Physical Review Letters 70 9-12 (1993).

[51] “Calculation of Gravitational Wave Forms From Black Hole Collisions and Disk Collapse: Applying Perturbation Theory to Numerical Spacetimes,” Andrew M. Abrahams, Stuart L. Shapiro, and Saul Teukolsky, Physical Review D51 4295 (April 15, 1995).

[52] “Critical Behaviour in Scalar Field Collapse,” Matthew W. Choptuik, in Deterministic Chaos in General Relativity, eds. D. Hobill et al, Plenum Press, New York (January 1994).

[53] “Critical Phenomena in Gravitational Collapse,” Matthew W. Choptuik, to appear in Proc. XI Congress Mathematical Physics, D. Iagolnitzer (ed.) (1995).

[54] “Black Hole-Scalar Field Interactions in Spherical Symmetry,” Robert L. Marsa and Matthew W. Choptuik, gr-qc/9607034 (1996).

[55] “Critical Behaviour in Gravitational Collapse of a Yang-Mills Field,” Matthew W. Choptuik, Tadeusz Chmaj, and Piotr Bizon, gr-qc/9603051 (1996).

[56] G.B. Cook, M. W. Choptuik, M. R. Dubal, S. Klasky, Richard A. Matzner and S.R. Oiveira, “Three-Dimensional Initial Data for the Collision of Two Black Holes,” Phys.Rev. D47 1471 (Feb.1993).

[57] “Three-dimensional initial data for the collision of two black holes II: Quasi-circular orbits for equal-mass black holes,” Gregory B. Cook (submitted for publication).

[58] “Singularity Avoidance in Numerical Black Hole Spacetimes,” Peter Anninos, Greg Daues, Joan Masso, Edward Seidel, Wai-Mo Suen, General Relativity (MG7 Proceedings) Ruffini and Keiser (eds.), World Scientific (1995).

[59] “The Evolution of Distorted Rotating Black Holes I: Methods and Tests,” Steven R. Brandt and Edward Seidel, Physical Review D52 856 (July 15, 1995).

[60] “The Evolution of Distorted Rotating Black Holes II: Dynamics and Analysis,” Steven R. Brandt and Edward Seidel, Physical Review D52 856 (July 15, 1995).

[61] “Rotating Black Hole Spacetimes,” S. Brandt and E. Seidel, to appear in General Relativity (MG7 Proceedings), R. Ruffini and M. Keiser (eds.), World Scientific (1995).

[62] “The collision of Two Black Holes,” Peter Anninos, David Hobill, Edward Seidel, Larry Smarr, and Wai-Mo Suen, Physical Review Letters 71 2851 (1993).

[63] “The Head-On Collision of Two Equal Mass Black Holes,” Peter Anninos, David Hobill, Edward Seidel, Larry Smarr, and Wai-Mo Suen, D52 2044 (1995).

[64] “The Head-On Collision of Two Equal Mass Black Holes: Numerical Methods,” Peter Anninos, David Hobill, Edward Seidel, Larry Smarr, and Wai-Mo Suen, NCSA Technical Report 24 (1994).

[65] “Geometry of a Black Hole Collision,” Richard A. Matzner, H.E. Seidel, S.L. Shapiro, L. Smarr, W-M Suen, Saul A. Teukolsky, and J. Winicour, Science 270 941 (1995) (cover article).

[66] “Numerical Relativity and Black Hole Horizons,” Joan Masso, in Inhomogenous Models and other topics (ERE94), A. Molina and J.M. Senovilla (eds.), World Scientific (1995).

[67] “Event Horizons of Numerical Black Holes,” P. Anninos, D. Bernstein, S. Brandt, J. Libson, J. Masso, E. Seidel, L. Smarr, W-M Suen, and P. Walker, to appear in General Relativity (MG7 proceedings), R. Ruffini and M. Keiser (eds.), World Scientific (1995).

[68] “Event Horizons in Numerical Relativity: Methods and Tests,” J. Libson, J. Masso, E. Seidel, W-M Suen, and P. Walker, Physical Review D53 4335-4350 (1996).

[69] “Collisions of Boosted Black Holes: Perturbation Theory Prediction of Gravitational Radiation,” Andrew M. Abrahams and Gregory B. Cook, Physical Review D50 R2364 (August 15, 1994).

[70] “Black-hole collisions from Brill-Lindquist initial data: Predictions of perturbation theory,”, A. Abrahams, R.H. Price, Physical Review D53-4 1972 (1996).

[71] “Analysis of ‘Gauge Modes’ in Linearized Relativity,” Richard A. Matzner, Mijan Huq, Alonso Botero, Dae Il Choi, Ullar Kask, Juan Lara, Steven Liebling, David Neilsen, Premana Premadi, and Deirdre Shoemaker, Classical and Quantum Gravity 14 L21 (1997).

[72] “Head-On Collision of Two Black Holes: Comparison of Different Approaches,” P. Anninos, R. Price, J. Pullin, E. Seidel, and W-M Suen (June 1995).

[73] “Applying black hole perturbation theory to numerically generated space-times,” A. Abrahams, Physical Review D53-4 1963 (1996).

[74] “Boosted Black Hole Evolutions with Singularity Excision,” The Binary Black Hole Grand Challenge Alliance, Phys. Rev. Lett. (submitted 1997).

[75] “Dynamics of Apparent and Event Horizons,” Peter Anninos, David Bernstein, Steven Brandt, Joseph Libson, Joan Masso, Edward Seidel, Larry Smarr, Wai-Mo Suen, and Paul Walker, Physical Review Letters 74 630 (January 1995).

[76] “Horizon Boundary Condition for Black Hole Spacetimes,” Peter Anninos, Greg Daues, Joan Masso, Edward Seidel, and Wai-Mo Suen, Physical Review D51 5562 (May 15, 1995).

[77] “Finding Apparent Horizons in Dynamic 3D Numerical Spacetimes,” submitted to Physical Review D (1996).

[78] “A 3-D Apparent Horizon Finder,” J. Libson, J. Masso, E. Seidel, and W-M Suen, in General Relativity (MG7 proceedings), R. Ruffini and M. Keiser (eds.), World Scientific (1995).

[79] “Implementing an apparent-horizon finder in three dimensions,” T.W. Baumgarte, G.B. Cook, M.A. Scheel, S.L. Shapiro, S.A. Teukolsky, gr-qc/9606010 (June 6, 1996).

[80] “Apparent Horizon Location in Numerical Spacetimes,” Mijan Huq, PhD dissertation, University of Texas at Austin (1996).

[81] “Curvature Flow Methods and Apparent Horizons”, Deirdre Shoemaker, Mijan F. Huq, and Richard A. Matzner, in preparation (1997).

[82] “Gravitational waveform extraction and outer boundary conditions for hyperbolic numerical relativity simulations,” A. Abrahams, A. Anderson, and C.R. Evans, Preprint (June 28, 1996).

[83] “Schwarzschild-perturbative gravitational wave extraction and outer boundary conditions,” The Binary Black Hole Grand Challenge Alliance, submitted to Physical Review Letters (1997).

[84] “Cauchy-Characteristic evolution of scalar waves in curved spacetime. I. Static, spherically symmetric spacetimes,” P. Papadopoulos, P. Laguna (1995).

[85] “Pittsburgh Group Code: Gravitational Wave Extraction – A Benchmark” (evolution of scalar fields from characteristic data,” Tomasz Haupt and Mark Miller (October 1994).l

[86] “Cauchy-characteristic Evolution of Einstein-Klein-Gordon Systems,” R. Gomez, P. Laguna, P. Papadopoulos, J. Winicour.

[87] “Cauchy-characteristic matching: A new approach to radiation boundary conditions,” Nigel T. Bishop, Roberto Gomez, Paulo R. Holvorcem, Richard A. Matzner, Philippos Papadopoulos, and Jeffrey Winicour, Physical Review Letters 76 4303 (1996).

[88] R. Gómez, R. Marsa and J. Winicour, “Black hole excision with matching”, Phys. Rev. D 56, (Nov. 1997), gr-qc/9708002.

[89] “Cauchy-Characteristic Evolution and Waveforms,” N.T. Bishop, R. Gomez, P.R. Holvorcem, R.A. Matzner, P. Papadopoulos, J. Winicour, J. Comp. Phys. 136 140 (1997).

[90] Nigel T. Bishop,Roberto Gomez,Luis Lehner, Jeffrey Winicour, “ Cauchy-characteristic extraction in numerical relativity,” to appear in Phys. Rev. D (1997).

[91] “Cauchy-characteristic evolution of Einstein-Klein-Gordon Systems,” Roberto Gomez, Pablo Laguna, Philippos Papadopoulos, and Jeffrey Winicour, to appear in Physical Review D.

[92] “Finite Differencing on the Sphere,” R. Gomez, P. Papadopoulos, and J. Winicour, Proceedings of the Seventh Marcel Grossman meeting (1995).

[93] “Computational Application of the eth formalism,” Roberto Gomez, Philip Papadoupolos, J. Winicour, to appear in Proceedings of the VII Marcel Grossman meeting, R. Ruffini and M. Reiser (eds.), World Scientific (1995).

[94] “Gravitational Instability of a Kink,” Williams Barreto, Roberto Gomez, Luis Lehner, and Jeffrey Winicour, to appear in Physical Review D (1998).

[95] C.Bona and J. Masso, Phys. Rev. Lett. 68 1097 (1992).

[96] “A Class of Hyperbolic Gauge Conditions,” C. Bona, J. Stela, J. Masso, and E. Seidel, in General Relativity (MG7 proceedings), R. Ruffini and M. Keiser (eds.), World Scientific (1995).

[97] “New Formalism for Numerical Relativity,” Carles Bona, Joan Masso, Edward Seidel, and Joan Stela, Physical Review Letters 75 600 (July 1995).

[98] “Invariant Algebraic Slicing of the Spacetime,” C. Bona, J. Stela, J. Masso, and E. Seidel, in Inhomogenous Models and other topics (ERE94), A. Molina and J.M. Senovilla (eds.), World Scientific (1995).

[99] “Einstein and Yang-Mills theories in hyperbolic form without gauge-fixing,” A. Abrahams, A. Anderson, Y. Choquet-Bruhat, and J.W. York Phys. Rev. Lett. 75 3377 (1995).

[100] “Dynamics of perturbations of rotating black holes”, w. Krivan, P. Laguna, P. Papadopoulos, and Nils Andersson, Phys. Rev. D56, 3395 (1997)

[101] P.Brady, C. Chambers, W. Krivan and P. Laguna, ”Telling Tails in the Presence of a Cosmological Constant,” submitted to Phys. Rev. D (1996).

[102] W. Krivan, P. Laguna and P. Papadopoulos, ”Dynamics of Scalar Fields in the Background of Rotating Black Holes,” Phys. Rev.D56 (Sept.1997).

[103] W. Krivan, P. Laguna and P. Papadopoulos, ”Dynamics of Spin-2 Fields in Kerr Backgroundgeometries,” to appear Proceedings Texas Symposium for Relativistic Astrophysics (1997).

[104] “Geometrical Hyperbolic Systems for General Relativity and Gauge Theories,” A. Abrahams, A. Anderson, Y. Choquet-Bruhat, and J. York, Jr. submitted to Classical and Quantum Gravity (1995).

[105] “A non-strictly hyperbolic system for the Einstein equations with arbitrary lapse and shift,” A. Abrahams, A. Anderson, Y. Choquet-Bruhat, and J. York, Jr., gr-qc/9607006 (1996).

[106] “3+1 general relativity in hyperbolic form,” A. Abrahams, J.W. York, Jr., paper to appear in Proceedings of Les Houches conference (1995).

[107] “Computing Gravitational Radiation from the Collision of Black Holes: Status Report,” Richard Matzner in Phenomenology of Unification from Present to Future, G. Diambrini-Palazzi et al (eds.), World Scientific (1994).

[108] “Solving Einstein’s Equations on Supercomputers,” Edward Seidel (1994).

[109] “Numerical Relativity,” E. Seidel, in proceedings of ICGC 95, Kluwer Press, to appear (1995).

[110] ”New Developments in Numerical Relativity,” E. Seidel, in Helvetica Physica Acta, to appear (1996).

[111] “A Numerical Approach to Binary Black Hole Coalescence,” L.S. Finn, to appear in proceedings of the GR14 conference, gr-qc/9603004 (1996).

[112] “Calculating Gravitational Waveforms – By Any Means Necessary!,” S.L. Shapiro, invited talk to appear in Proceedings of International Conference on Gravitational Waves: Sources and Detectors, Cascina (Pisa), Italy (March 19-23, 1996).

[113] “Black Hole Collisions: A Computational Grand Challenge,” P. Laguna, in Gravitational Waves: Sources and Detectors, eds. I. Ciufolini and Francesco Fidecaro, World Scientific-Singapore (1997).

[114] P. Anninos, J. Baker, S. Brandt, Hans-P. Nollert,R. Price,J. Pullin, and E. Seidel, ”The head-on collsion of close counter-rotating black holes,” submitted to Phys. Rev. D (1996).

[115] “Unitarity Restoration in the Presence of Closed Timelike Curves,” A. Anderson, Physical Review D51 5705 (May 15, 1995).

[116] “Cauchy-characteristic extraction in numerical relativity,” to appear in Phys. Rev. D (1997).

[117] Matthew W. Choptuik, “ad, An Implementation of the Berger-Oliger Mesh Refinement Algorithm for the Wave Equation in Spherical Symmetry,” Technical Report, (March 1994)

[118] “Black Hole Criticality in the Brans-Dicke Model,” Steven L. Liebling and Matthew W. Choptuik, gr-qc/9606057 (1996).

[119] Joan Masso, “Numerical Relativity and Black Hole Horizons,” in Inhomogeneous Models and Other Topics (ERE94), A. molina and J.M. Senovilla (eds), World Scientific (1995).

[120] “Geometrical Well Posed Systems for the Einstein Equations,” Y. Choquet-Bruhat and J.W. York (August 1995).

[121] A.M. Abrahams, S.L. Shapiro and S. A. Teukolsky, “Disk Collapse in General Relativity,” Physical Review D50 7282 (Dec.1994).

[122] J. Bowen and J.W. York Jr., “Two Black Holes with Axisymmetric parallel spins: Initial data,” Class. Quantum Grav., 1,591-610, (1984).

[123] G.B. Cook, M.W. Choptuik, M. Dubal, S. Klasky, R. Matzner and S.R. Oliveira, Phys. Rev. D, 1471-1490 (1993).

[124] Steven Brandt and Bernd Brügmann, ” A simple construction of initial data for multiple black holes,” Phys.Rev.Lett., 78 3606-3609 (1997). (http://xxx.lanl.gov/find/gr-qc/9703066).

[125] Ann S. Almgren, John B. Bell, Phillip Colella, Louis H. Howell, Michael L. Welcome, “A Conservative Adaptive Projection Method for the Variable Density Incompressible Navier-Stokes Equations,” submitted to J. Comp. Phys. (1997).

[126] P. Anninos, J. Masso, H. E. Seidel, W-M Suen and M. Tobias, “Dynamics of Gravitational Waves in 3D: Formulations, Methods, and Tests,”Phys. Rev. D, 56, 842-858, (1997). grgc/9610057.

[127] J. Balakrishna, G. Daues, E. Seidel, W-M. Suen, M. Tobias and E. Wang, “Coordinate conditions and their implementation in 3-D numerical relativity”, LANL preprint gr-qc/9601027 (1996)

[128] P. Anninos, K. Camarda, J. Masso, E. Seidel, W-M. Suen and J. Towns, “Three-Dimensional Numerical Relativity: The Evolution of Black Holes”, Physical Review D, 52, 2059-2082 (1995)

[129] G. Daues, Ph.D. Dissertation, Washington University, (1996).

[130] “Testing a Simplified Version of Einstein’s Equations for Numerical Relativity,” G.B. Cook, S.L. Shapiro and S.A. Teukolsky, Preprint. (1995)

[131] N. T. Bishop, R. Gómez, L. Lehner, M. Maharaj and J. Winicour, “High-powered Gravitational News”, Phys. Rev. D, 56, (Nov. 1997), gr-qc/9708065.

[132] Nigel T. Bishop, Roberto Gomez, Paulo R. Holvorcem, Richard A. Matzner, Philippos Papadopoulos and Jeffrey Winicour, “Cauchy-characteristic matching: A new approach to radiation boundary condition”, Phys. Rev. Letters 76, 4303 (1996).

[133] C. Clarke, R. d’Inverno, and J. Vickers, Phys. Rev. D, 52, 6863 (1995). M. Dubal, R. d’Inverno, and C. Clarke, Phys. Rev. D, 52, 6868 (1995).

[134] G. Cook and Scheel; http://xxx.lanl.gov/abs/gr-qc/9708067.

[135] Bean, Warming, Yee, J. Comp. Phys. 48 200 (1982).

[136] http://artemis.phyast.pitt.edu/animations.

[137] P. Anninos et al, Phys. Rev. D50 3801 (1994).

[138] http://www.npac.syr.edu/projects/bbh/dagh.html

[139] http://www.npac.syr.edu/tango

[140] http://www.webwisdom.org/papers/sc97/webwisdom3.html

[141] P. D. D’Eath, Black Holes, Gravitational Interactions, Clarendon, OX. (1996).

[142] Peter Anninos, Joan Masso, Edward Seidel and Wai-Mo Suen “Numerical Relativity and Black Holes”, Physics World, v. 9, p. 43 (1996)

[143] Richard H. Price and Jorge Pullin, ”Colliding Black Holes: The Close Limit,” Phys. Rev. Lett. v. 72, p. 3297 (1994). Peter Anninos, Richard H. Price, Jorge Pullin, Ed Seidel, Wai-Mo Suen, ”Head-on collision of two black holes: comparison of different approaches,” Phys. Rev. D v.52, p. 4462 (1995).

[144] M. Huq, “Apparent Horizon Location in Numerical Spacetimes”, The University of Texas at Austin PhD Thesis (1996)

[145] T. W. Baumgarte, G. B. Cook, M. A. Scheel, S. A. Teukolsky and S. L. Shapiro, “Implementing an Apparent Horizon Finder in Three-Dimensions”, Physical Review D, (in press) (1996)

[146] P. Anninos et al, “Finding Apparent Horizons in Dynamic 3-D Numerical Space Times”, LANL preprint gr-qc/9609059 (1996)

[147] E. Seidel and W-M. Suen, “Towards a Singularity Proof Scheme in Numerical Relativity”, Physical Review Letters, 69, 1845-1848 (1992)

[148] M. Alcubierre and B. F. Schutz, “Time-Symmetric ADI and Causal Reconnection: Stable Numerical Techniques for Hyperbolic Systems on Moving Grids”, Journal of Computational Physics, 112, 44-77 (1994)

[149] A. Abrahams, “The No Boundary Proposal”, Talk presented at the NSCA Numerical Relativity Conference, (1994)

[150] M. A. Scheel, S. L. Shapiro and S. A. Teukolsky, “Collapse to Black Holes in Brans-Dicke Theory. 1. Horizon Boundary Conditions for Dynamical Space-times”, Physical Review D, 51, 4208-4235 (1995)

[151] M. A. Scheel, S. L. Shapiro and S. A. Teukolsky, “Collapse to Black Holes in Brans-Dicke Theory. 2. Comparison with General Relativity”, Physical Review D, 51, 4236-4249 (1995)

[152] P. Anninos, G. Daues, J. Masso, E. Seidel and W-M. Suen, “Horizon Boundary Conditions for Black Hole Space-times”, Physical Review D, 51, 5562-5578 (1995)

[153] R. L. Guenther, “A Numerical Study of the Time Dependent Shrödinger Equation Coupled With Newtonian Gravity”, The University of Texas at Austin, PhD Thesis (1995)

[154] A. Brandt, “Multi-level adaptive solutions to boundary-value problems”, Math. Comp., 31, 333-390 (1977)

[155] E. Seidel, ”New developments in Numerical Relativity,” in Helvetica Physica Acta, to apppear, (1996).

[156] http://godel.ph.utexas.edu/Members/marsa/rnpl/refman/refman.html

[157] A. Brandt, “Guide to multigrid development”, in Lecture Notes in Mathematics, 960, (Multigrid Methods), W. Hackbusch and U. Trottenberg, eds., 220-312 (1982)

[158] M. Choptuik and W. G. Unruh, “An Introduction to the Multi-Grid Method for Numerical Relativists”, General Relativity and Gravitation, 18, 813–843 (1986)

[159] M. W. Choptuik,  S. Klasky and M. Parashar,  “Parallel Multi-

grid with the DAGH package: Specifications and Applications” (1995) http://godel.ph.utexas.edu/Members/matt/Doc/mgspecs.ps

[160] Tim Collins Ph.D. dissertation, The University of Texas at Austin, 1995

[161] (van de Geijn, 1997) Robert A. van de Geijn, Using PLAPACK: Parallel Linear Algebra Package, The MIT Press, 1997.

[162] (Edwards, 1996) H. C. Edwards, MMPI: Asynchronous message management for the messagepassing interface. TICAM Report 96-44, Texas Institute for Computational and Applied Mathematics, The University of Texas at Austin, 1996.

[163] Y. Fu, K. Klimkowski, J. Rodin, G. Berger, J.C. Browne, J.C. Singer, R. van de Geijn, and K.S. Vemaganti, ”A Fast Solution Method for Three Dimensional Many-Particle Problems of Linear Elasticity,” Int. J. Num. Meth. Engrg., to appear.

[164] Y. Fu, ”Application of the Fast Multipole Method to Micromechanics,” Ph.D. Dissertation, The University of Texas at Austin.

[165] G.J. Rodin, ”Toward Rapid Evaluation of the Elastic Interactions among Three-Dimensional Dislocations,” Phil. Mag. Letters, Vol 77, No. 4, pp. 187-190, 1998

[166] T.I. Zohdi, J.T. Oden, and G.J. Rodin, ”Hierarchical Modeling of Heterogeneous Bodies,” Computer Methods in Applied Mechanics and Engineering, 138 (1996), pp. 273–298.

[167] J.T. Oden and T. Zohdi, ”Analysis and Adaptive Modeling of Highly Heterogeneous Elastic Structures,” Computer Methods in Applied Mechanics and Engineering, 148 (1997), pp. 267– 381.

[168] E.G. Podnos, I. Babuska, and G. J. Rodin, ”Application of Fictitious Domain Method to Micromechanics of Non-Linear Composite Materials,” submitted to Comp. Mech. in Appl. Mech. Engrg., 1998.

Facilities, Equipment and Other Resources

This is a proposal for computational research. In the past few years, the Center has become quite well-equipped with medium- and high-end graphics workstations. This equipment has been, and will continue to be, absolutely vital to the success of our research. Our current facilities include four Silicon Graphics (SGI) R10000 Impact machines with a total of nearly 1.5 Gbyte main memory and 32 Gbyte of disk storage, as well as two older SGI Crimson R4000 VGXT machines. Together, the six R10000/Crimson machines provide powerful platforms for scientific visualization, code development, and medium-scale production runs for the PIs, our postdocs, and (at most) two graduate students. We also have two entry-level SGI Indigo and two SGI Indy machines which are used heavily by graduate students, but which are rapidly approaching obsolescence and will soon need to be replaced. We have very good access to campus networks, and adequate access to national networks and national supercomputer centers.

Budget Justification

This proposal is to support work by Professor M. Choptuik, and Professor Richard A. Matzner, in computational research on the interaction of black holes. This work will be done in collaboration with colleagues at The University of Pittsburgh, and The Pennsylavania State University, and involves computational science support through the University of Syracuse. The budget principally supports one postdoc, who will be supervised by Matzner and Choptuik. It also modestly supports travel, publications, and very necessary communication to carry out this collaborative research. There is an important piece of cost sharing: The Center for Relativity commits one postdoc (letter attached) supported by State of Texas funds as a match to the funded postdoc position requested here. This amounts to a real costsharing (salary, fringes, and overhead) of $354,775 over the five-year proposed duration of this grant.