Full HTML for
Given by Scott Klasky at Visit by CEWES Team on Mar 25 1997. Foils prepared Mar 24 97
Outside Index
Summary of Material
| This describes DAGH and HPF Application in Binary Black Hole Grand Challenge |
| Also Java visualization Applet for 1D and 2D |
| See MPI HPF and DAGH for Parallelization of Black Hole Codes For more detail |
Denote Foils where Image Critical
Denote Foils where HTML is sufficient
NPAC's Role in the Binary Black Hole Grand Challenge
Quick Overview of the Physics.
Unique Aspects of the BBH Problem
DAGH and HPF's Role in the BBH GC
DAGH Execution.
1D Client Server "Collaborative" Visualization (Ki, Klasky)
1D Visualization continued
Outside Index
Summary of Material
HTML version of Basic Foils prepared Mar 24 97 
Full HTML Index
| T. Haupt, S. Klasky, G. Fox |
| NPAC, Syracuse University |
| haupt@npac.syr.edu, scott@npac.syr.edu, gcf@npac.syr.edu |
| Quick overview of the Physics. |
| Unique aspects of this Problem. |
| DAGH and HPF's role in the BBH GC Problem. |
| 1D-2D Client-Server Visualization (Ki, Klasky). |
HTML version of Basic Foils prepared Mar 24 97
Full HTML Index
| Evolve 2 black holes on a computer. Need to provide controllable convergent algorithms to compute gravitational waveforms which arise from black hole encounters, and which are relevant to astrophysical events and may be used to predict signals which can be used for detection by ground and space based detectors (LIGO). |
| We decompose the four-dimensional covariance of Einstein's equations so that is is possible to pose them as a Cauchy evolution problem. |
| Fundamental variables are: gij , Kij , a, bi . Total # of 3D variables in our code=112. We make use of F90 data structures to easily handle the tensor notation. |
| To achieve long-term stability we cut out the singularity from the computational domain. (At the Apparent Horizon-which we must solve an elliptic equation to find this minimally trapped surface. |
HTML version of Basic Foils prepared Mar 24 97 
Full HTML Index
| Difficulties for H(HPF), H2(HPF-2) and D(DAGH) are given from 1-3, where 1=easy, 2=medium and 3 =difficult implement. |
| Difficulties inherent to the BBH Problem H H2 D Comments |
| Linear wave problem. No "inner boundaries" 1 1 1 lapse=1, shift=0, no holes. |
| Linear wave problem with matching outer |
| Boundary condition for grav. wave extraction. 2 1 3 The outer boundary requires a 1d evolution. |
| Adaptive Mesh Refinements (AMR) 3 2 1 DAGH was designed for this purpose. |
| Multigrid/MLAT + AMR 3 2 1-3 Only a certain class of problems can |
| be solved with DAGH. |
| Spherical "inner boundary" in Cartesian coords 2 2 2 DAGH: large stencils on the inner |
| boundary-expensive communications. |
| HPF: 3D "sparse" array or extrinsic. |
| Moving "inner boundary". 3 1 1 Irregular boundary makes it difficult to implement in HPF. |
| Finding the Apparent Horizon 1 1 1 Expand Spherical Harmonics. |
| Causal Differencing with a "real" shift. 2 2 1 Requires additional storage in HPF. |
| NOTE:The mass curves spacetime, the shift tells us where to |
| interpolate to. |
HTML version of Basic Foils prepared Mar 24 97
Full HTML Index
| Adaptive Mesh Refinement is used to control the memory requirements needed by the physics. Without AMR we would need ~6871 Gigabytes of memory for accurate waveform extraction. |
| AMR determines where to place grid points, and uses a hierarchy of grids of different levels to achieve high accuracy with "low" memory costs. |
| The Grids dynamically adapt to the equations using a truncation error estimate with Finite Difference codes. |
| HPF can handle the majority of problems in the BBH problem, but it is very difficult to implement in HPF 1 or 2. The load balancing will never be ideal. |
| DAGH (Distributed Adaptive Grid Hierarchy) was designed for implementing AMR on parallel computers for finite difference codes. |
HTML version of Basic Foils prepared Mar 24 97
Full HTML Index
| DAGH decomposes all grids into "grid components" and distributes them onto available processors. It's algorithm guarantees the optimal load balance and alignment of grid components on different refinement levels defined on the same computational subdomain. |
| Computations are performed in a data parallel fashion. |
| DAGH is implemented in C++. Thus the driver code must be written in C++ and then the "kernel" routines can be put in any language(Fortran!) |
| DAGH is not a complete language. It has some limitations! If it wasn't implemented in DAGH, then one must code it in MPI communicating with DAGH (which is very tedious!) |
| NPAC developed a PERL script to convert the BBH code from "pure" F90 to DAGH! The users don't have to know the details of DAGH to use it. |
HTML version of Basic Foils prepared Mar 24 97
Full HTML Index
Features:
|
HTML version of Basic Foils prepared Mar 24 97
Full HTML Index
"Fully" hooked up to the blackhole code (very easy!).
|
| Printing capabilities. We have the ability to output to postscript. |
| 2D visualizations, on the way..... (Need more time to implement). |
Collaborative.
|
| Server 1 |
| Server 2 |
| Server 3 |
| Client 1 |