Full HTML for

Scripted foilset CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures

Given by Geoffrey C. Fox at CPS615 Basic Simulation Track for Computational Science on Fall Semester 97. Foils prepared 27 August 1997
Outside Index Summary of Material

This introduces the course which covers the essential programming and algorithmic skills needed for large scale (serious) computing (aka simulation)
We start with an overview of course itself and describe how it compares to other computational science offerings
  • We point out new modules on web based computing
In Introductory material, we describe some large simulations of interest (the so called Grand Challenges)
An Overview of Technology trends driving the computer industry (in web and simulation areas!)
Overview of computational science education program here and elsewhere
Parallel Computing in Society and why it works in computers!

Table of Contents for full HTML of CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures

 Denote Foils where Image Critical
Denote Foils where Image has important information
Denote Foils where HTML is sufficient
Denote Foils where Image is not available
Indicates Available audio which is lightpurpleed out if missing
1 Base Course in Simulation Track of Computational Science CPS615 Fall Semester 97
2 Abstract of CPS615 Introductory Lecture
3 What is Computational Science ?
4 Internetics is Another Way of Thinking
5 Basic CPS615 Contact Points
6 Course Organization
7 Material Covered in this Course
8 Structure of CPS615 - II
9 Basic Structure of Complete CPS615 Base Course on Computational Science Simulation Track -- III
10 Motivating Applications
11 Performance of High End Machines Years 1940-2000
12 Performance of High End Machines Years 1980-2000
13 Peak Supercomputer Performance
14 Some Comments on Simulation and HPCC
15 Some Relevant Technical Trends
16 HPCC Software issues
17 Web Software is the Best!
18 Why use the Web as basis for HPCC Software?
19 Synergy of InterNet and IntraNets
20 We have the Web Tools in Place - I !
21 We have the Web Tools in Place - II!
22 Java for Scientific Computing Resource
23 There are (at least) 3 Major Roles for JAVA in Computation
24 More General 3 Level Software Model Functionality Performance Tradeoff
25 The 3 Roles of Java
26 The Technology
Driving Forces for HPCC
27 Effect of Feature Size on Performance
28 Growing Logic Chip Density
29 Trends in Feature and Die Size as a Function of Time
30 Supercomputer Memory Sizes and trends in RAM Density
31 Three Major Markets -- Logic,ASIC,DRAM
32 Chip and Package Characteristics
33 Fabrication Characteristics
34 Electrical Design and Test Metrics
35 National Roadmap for Semiconductor Technology --1992
36 Status of Parallel Computing and High Speed Networks --
The Grand Challenges and the National Information Infrastructure
37 Parallel Computing Rationale
38 Sequential Memory Structure
39 Parallel Computer Memory Structure
40 The Federal High Performance Computing and Communication Initiative 1992--1996
41 The Federal High Performance Computing and Communication Initiative (HPCCI)
42 The High Performance Computing and Communications Initiative
43 HPCCI Goals
44 Note the Trend from Large Scale Numerical Computing to the Integration of Computing and Communication in the NII
45 The Blue Books
Supplements to the President's Fiscal Year Budget
46 The Blue Book Covers
47 What and Why is Computational Science ?
48 Parallelism Implies Major Changes which have significant educational Implications
49 What is Computational Science?
50 What do we have at Syracuse University?
51 Program in Computational Science
Implemented within current academic framework
52 Methodology for Computation
53 Usefulness of Computational Science Degrees:
54 Syracuse Computational Science Academic Programs -- Masters Degree
55 Syracuse Graduate Computational Science Academic Programs
56 Some Academic Areas and their Relation to Computational Science
57 Program in Information Age Computational Science Implemented Within Current Academic Program
58 Parallel Processing and Society
59 Concurrent Construction of a Wall
Using N = 8 Bricklayers
Decomposition by Vertical Sections
60 Quantitative Speed-Up Analysis for Construction of Hadrian's Wall
61 Amdahl's law for Real World Parallel Processing
62 Pipelining --Another Parallel Processing Strategy for Hadrian's Wall
63 Hadrian's Wall Illustrates that the Topology of Processor Must Include Topology of Problem
64 General Speed Up Analysis
65 Nature's Concurrent Computers
66 Comparison of Concurrent Processing in Society and Computing
67 Back from the Analogy to Parallel Computers!
68 Concurrent Computation as a Mapping Problem -I
69 Concurrent Computation as a Mapping Problem - II
70 Concurrent Computation as a Mapping Problem - III
71 Finite Element Mesh From Nastran
(mesh only shown in upper half)
72 A Simple Equal Area Decomposition
73 Decomposition After Annealing
(one particularly good but nonoptimal decomposition)
74 Comparison of The Complete Problem to the subproblems formed in domain decomposition
75 Hadrian's Wall Illustrating an
Irregular but Homogeneous Problem
76 Some Problems are Inhomogeneous Illustrated by:
An Inhomogeneous Hadrian Wall with Decoration
77 Global and Local Parallelism Illustrated by Hadrian's Wall
78 Parallel I/O Illustrated by
Concurrent Brick Delivery for Hadrian's Wall
Bandwidth of Trucks and Roads
Matches that of Masons

Outside Index Summary of Material

HTML version of Scripted Foils prepared 27 August 1997

Foil 1 Base Course in Simulation Track of Computational Science CPS615 Fall Semester 97

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
http://www.npac.syr.edu/users/gcf/cps615intro97
Geoffrey Fox
Syracuse University NPAC
111 College Place Syracuse NY 13244 4100
3154432163
HTML version of Scripted Foils prepared 27 August 1997

Foil 2 Abstract of CPS615 Introductory Lecture

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
This introduces the course which covers the essential programming and algorithmic skills needed for large scale (serious) computing (aka simulation)
We start with an overview of course itself and describe how it compares to other computational science offerings
  • We point out new modules on web based computing
In Introductory material, we describe some large simulations of interest (the so called Grand Challenges)
An Overview of Technology trends driving the computer industry (in web and simulation areas!)
Overview of computational science education program here and elsewhere
Parallel Computing in Society and why it works in computers!
HTML version of Scripted Foils prepared 27 August 1997

Foil 3 What is Computational Science ?

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Practical use of leading edge computer science technologies to address "real" applications
CPS615/713: Simulation Track
CPS606/616/640/714: Information Track
Large Scale Parallel Computing Low latency Closely Coupled Components
World Wide Distributed Computing Loosely Coupled Components
Parallel Algorithms Fortran90
HPF MPI Interconnection Networks
Transactions
Security
Compression
PERL JavaScript Multimedia
Wide Area Networks
Java
VRML Collaboration Integration (middleware) CORBA Databases
HTML version of Scripted Foils prepared 27 August 1997

Foil 4 Internetics is Another Way of Thinking

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
These ideas come from Xiaoming Li, Peking University
CPS615
HTML version of Scripted Foils prepared 27 August 1997

Foil 5 Basic CPS615 Contact Points

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Instructor: Geoffrey Fox -- gcf@npac.syr.edu, 3154432163 and Room 3-131 CST
Backup: Nancy McCracken -- njm@npac.syr.edu, 3154434687 and Room 3-234 CST
NPAC administrative support: Nora Downey-Easter -- nora@npac.syr.edu, 3154431722 and room 3-206 CST
The above can be reached at cps615ad@npac.syr.edu
Students will be on alias cps615@npac.syr.edu
Homepage is: http://www.npac.syr.edu/projects/cps615fall97
There is a slightly out of date paper "Basic Issues and Current Status of Parallel Computing" by Fox (SCCS736 on NPAC technical reports)
HTML version of Scripted Foils prepared 27 August 1997

Foil 6 Course Organization

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Graded on the basis of approximately 7 Homework sets which will be due Thursday of the week following day (Tuesday or Thursday given out)
There will be two projects -- one will start after message passing (MPI) discussed, the other about 60% through the course
Total grade is 50% homework, 50% the sum of two projects
All homework will be handled through the web and indeed all computer access will be through the VPL or Virtual Programming Laboratory which gives access to compilers, Java visualization etc. through the web
HTML version of Scripted Foils prepared 27 August 1997

Foil 7 Material Covered in this Course

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Status of High Performance Computing and Computation HPCC nationally
  • Including web linked simulation examples
What is Computational Science Nationally (and at Syracuse)
Technology driving forces (until around 2010)
  • Moore's law and exponentially increasing transistors
Elementary discussion of Parallel Computing in Society and why it must obviously work in simulation!
Sequential and Parallel Computer Architectures
Comparison of Architecture of World Wide Web and Parallel Systems
HTML version of Scripted Foils prepared 27 August 1997

Foil 8 Structure of CPS615 - II

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Simple base Example -- Laplace's Equation
  • Illustrate parallel computing
  • Illustrate use of Web
Then we discuss software and algorithms (mini-applications) intermixed
Programming Models -- SPMD and Message Passing (MPI), Software Integration middleware (Java, WebFlow), Data Parallel (HPF, Fortran90)
Visualization -- Java Applets
Other tools: Collaboration , Performance Measurement
HTML version of Scripted Foils prepared 27 August 1997

Foil 9 Basic Structure of Complete CPS615 Base Course on Computational Science Simulation Track -- III

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 96
This introduction is followed by a set of "vignettes" discussing problem classes which illustrate parallel programming and parallel algorithms
Ordinary Differential Equations
  • N body Problem by both O(N^2) and "fast multipole" O(N) method
Numerical Integration including adaptive methods
Floating Point Arithmetic
Monte Carlo Methods including Random Numbers
Full Matrix Algebra as in
  • Computational Electromagnetism
  • Computational Chemistry
Partial Differential Equations implemented as sparse matrix problems (as in Computational Fluid Dynamics)
  • Iterative Algorithms from Gauss Seidel to Conjugate Gradient
  • Finite Element Methods
HTML version of Scripted Foils prepared 27 August 1997

Foil 10 Motivating Applications

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Large Scale Simulations in Engineering
  • Model airflow around an aircraft
  • Study environmental issues -- flow of contaminants
  • Forecast weather
  • Oil Industry: Reservoir Simulation and analysis of Seismic data
Large Scale Academic Simulations (Physics, Chemistry, Biology)
  • Study of Evolution of Universe
  • Study of fundamental particles: Quarks and Gluons
  • Study of protein folding
  • Study of catalysts
"Other Critical Real World Applications"
  • Run optimization and classification algorithms in datamining of Enterprise Information Systems
  • Model Financial Instruments
HTML version of Scripted Foils prepared 27 August 1997

Foil 11 Performance of High End Machines Years 1940-2000

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 72
HTML version of Scripted Foils prepared 27 August 1997

Foil 12 Performance of High End Machines Years 1980-2000

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 33
HTML version of Scripted Foils prepared 27 August 1997

Foil 13 Peak Supercomputer Performance

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
For "Convential" MPP/Distributed Shared Memory Architecture
Now(1996) Peak is 0.1 to 0.2 Teraflops in Production Centers
  • Note both SGI and IBM are changing architectures:
  • IBM Distributed Memory to Distributed Shared Memory
  • SGI Shared Memory to Distributed Shared Memory
In 1999, one will see production 1 Teraflop systems
In 2003, one will see production 10 Teraflop Systems
In 2007, one will see production 50-100 Teraflop Systems
Memory is Roughly 0.25 to 1 Terabyte per 1 Teraflop
If you are lucky/work hard: Realized performance is 30% of Peak
HTML version of Scripted Foils prepared 27 August 1997

Foil 14 Some Comments on Simulation and HPCC

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HPCC is a maturing field with many organizations installing large scale systems
These include NSF (academic computations), DoE (Dept of Energy) and DoD (Defense)
There are new applications with new algorithmic challenges
  • Our work on Black Holes has novel adaptive meshes
  • On earthquake simulation, new "fast multipole" approaches to a problem not tackled this way before
  • On financial modeling, new Monte Carlo methods for complex options
However these are not "qualitatively" new concepts
Software ideas are "sound" but note simulation is only a few percent of total computer market
  • Use where possible software from other sources (such as web)which can spend more money on each feature!
HTML version of Scripted Foils prepared 27 August 1997

Foil 15 Some Relevant Technical Trends

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
PC and workstation clusters are of growing important and this typically distributed memory people's technology is contrasted with distributed shared memory tightly coupled MPP's.
Computational science moving to multidisciplinary (multi-component) applications
Corresponding growing use of databases (for data-intensive applications)
Interoperability between disparate heterogeneous platforms, support of multidisciplinary applications, and metacomputing are three related important areas
"full metacomputing" (decompose general problem on general networked resources) may not be relevant
The Web is delivering a new operating environment (WebWindows) and a rich distributed computing software infrastructure with especially excellent support for software integration
There is a need for a new scalable technical operating system (NT v UNIX v WebWindows)
HTML version of Scripted Foils prepared 27 August 1997

Foil 16 HPCC Software issues

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
We can distinguish Decomposition and Integration
Decomposition is performed by an HPF or other Parallelizing compiler; or by a user writing a Fortran + Message Passing code "by hand"
MPI integrates decomposed parts together with high bandwidth latency constraints
Systems such as AVS integrate larger modules together and much of "software engineering" (modular style of programming) involved with this
Web is a powerful integration model suitable for large coarse modules with modest latency and sometimes modest bandwidth requirements
  • naturally "integrate" data(bases), people, computation together
Collaboration, computational steering, multidisciplinary science are all integration and not decomposition problems!
HTML version of Scripted Foils prepared 27 August 1997

Foil 17 Web Software is the Best!

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
By definition, Web Software will be the "best" software ever built because it has the largest market (and so greatest leverage of investment dollars) and most creative business model (harness the world's best minds together with open interfaces)
  • As Web Software is "distributed computing infrastructure", one will need to customize and integrate for each application
  • Note PC software and perhaps even IBM business OS was high quality software but not so open and not a complete model
One should build upwards from the "democractic Web"
  • e.g. up from POTS --> ISDN/Cable Modem/ADSL --> ...
  • Not down from ..........<--- ATM
This allows you to both deliver your application to the general public (not always required but often desireable) and use the best leveraged software
Note Web Software tends to offer highest functionality as opposed to highest performance and HPCC often requires different trade-offs
HTML version of Scripted Foils prepared 27 August 1997

Foil 18 Why use the Web as basis for HPCC Software?

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HPCC is a small field with limited resources for a very hard problem and must leverage as much software as possible
Web Software provides an excellent pervasive user interface with Java Applets and WebWindows
Web Software provides a potentially excellent high performance object oriented language (Java) for scientific and engineering computation
Web Software provides a high functionality but modest performance distributed computing environment based on either Web Servers or Clients
  • We will choose to use servers as higher functionality than clients although currently less broadly deployed
  • Only addresses Integration of already decomposed parts!
All(!?) we need to do is to add high performance to the Web!
HTML version of Scripted Foils prepared 27 August 1997

Foil 19 Synergy of InterNet and IntraNets

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 20 We have the Web Tools in Place - I !

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Web Technology is still uncertain and there may be major changes but "enough" capabilities are in place to build very general (~all) applications
  • MicroSoft v Sun / Netscape uncertainities
Rapidly evolving Standards and a mechanism to get rapid consensus
Fortran 77 -> Fortran90 --> HPF --> Fortran2000 (23 years)
VRML Idea (1994) --> VRML1 deployed (95) --> VRML2 deployed (early 97) (2.3 years)
  • VRML2 (3D datastructures and Java(Script) enabled methods) is more sophisticated than Fortran ?
  • Java development time was also fast compared to "traditional" standard adoption times
HTML version of Scripted Foils prepared 27 August 1997

Foil 21 We have the Web Tools in Place - II!

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Classic Web: HTTP Mime HTML CGI Perl etc.
Java and JavaScript Compiled to almost compiled (applet) to fully Interpreted Programming Language
VRML2 as a dynamic 3D Datastructure for products and their simulation object
Java Database Connectivity (JDBC) and general Web linked databases
Dynamic Java Servers and Clients
Rich Web Collaboration environment building electronic societies
Security -- still needs maturing as very clumsy or non existent at present in many cases
Compression/ Quality of Service for Web Multimedia
  • Do need higher bandwidth for real video although current POTS (perhaps ISDN needed) adequate for conferencing
Emerging Web Object model including integration of Corba (see JavaBeans and Orblets)
HTML version of Scripted Foils prepared 27 August 1997

Foil 22 Java for Scientific Computing Resource

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
See Original Foil
HTML version of Scripted Foils prepared 27 August 1997

Foil 23 There are (at least) 3 Major Roles for JAVA in Computation

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Java for the User Interface: This is roughly the "WebWindows Philosophy" of building applications to Web Server/Client Standards
Java for Coarse Grain Software Integration: see collaboration and metacomputing
Java as a high performance scientific language: for "inner" (and outer) loops Here parallelism is important but sequential issues also critical and first issues to examine!
HTML version of Scripted Foils prepared 27 August 1997

Foil 24 More General 3 Level Software Model Functionality Performance Tradeoff

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Building from bottom of Computing pyramid starts with high functionality software which has an architecture that can be augmented with high performance
3 Levels of Software
  • Client Interfaces
  • Middleware based on high functionality Web Integration based on network of Web (Java) Servers
  • that can "escape" (as in machine language subroutine) to a high performance layer which is viewed as one of many services -- CORBA, Databases are other services
One universal language -- Java for all layers
HTML version of Scripted Foils prepared 27 August 1997

Foil 25 The 3 Roles of Java

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 26 The Technology
Driving Forces for HPCC

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 14
HTML version of Scripted Foils prepared 27 August 1997

Foil 27 Effect of Feature Size on Performance

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
f is the size of basic lines on chip and is now 0.25nm and will decrease inexorably to 100 nm in 2005-2010 time period
  • 1992 Digital alpha chip was f= 750 nm. feature size and 1.7 x 106 transistors
  • 1995 Sun UltraSparc was f= 500 nm. and 5.2 X 106 transistors
Roughly number of Transistors is proportional to (1/f)2
Speed of chip is proportional to (1/f)
  • This is no longer true as "wires" get so thin that propagation speed decreases and so clock speed levels off around 1 gigaherz
Figure of Merit is (1/f)3
Die size is also increasing like (1/f) and this enhances effect!
HTML version of Scripted Foils prepared 27 August 1997

Foil 28 Growing Logic Chip Density

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 30
HTML version of Scripted Foils prepared 27 August 1997

Foil 29 Trends in Feature and Die Size as a Function of Time

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 25
HTML version of Scripted Foils prepared 27 August 1997

Foil 30 Supercomputer Memory Sizes and trends in RAM Density

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 57
RAM density increases by about a factor of 50 in 8 years
Supercomputers in 1992 have memory sizes around 32 gigabytes (giga = 109)
Supercomputers in year 2000 should have memory sizes around 1.5 terabytes (tera = 1012)
HTML version of Scripted Foils prepared 27 August 1997

Foil 31 Three Major Markets -- Logic,ASIC,DRAM

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 47
Overall Roadmap Technology Characteristics from SIA (Semiconductor Industry Association) Report 1994
L=Logic, D=DRAM, A=ASIC, mP = microprocessor
HTML version of Scripted Foils prepared 27 August 1997

Foil 32 Chip and Package Characteristics

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 37
Overall Roadmap Technology Characteristics from SIA (Semiconductor Industry Association) Report 1994
HTML version of Scripted Foils prepared 27 August 1997

Foil 33 Fabrication Characteristics

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 25
Overall Roadmap Technology Characteristics from SIA (Semiconductor Industry Association) Report 1994
HTML version of Scripted Foils prepared 27 August 1997

Foil 34 Electrical Design and Test Metrics

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 34
Overall Roadmap Technology Characteristics from SIA (Semiconductor Industry Association) Report 1994
HTML version of Scripted Foils prepared 27 August 1997

Foil 35 National Roadmap for Semiconductor Technology --1992

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 56
See Chapter 5 of Petaflops Report -- July 95
HTML version of Scripted Foils prepared 27 August 1997

Foil 36 Status of Parallel Computing and High Speed Networks --
The Grand Challenges and the National Information Infrastructure

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 37 Parallel Computing Rationale

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Transistors are getting cheaper and cheaper and it only takes some 0.5 million transistors to make a very high quality CPU
  • Essentially impossible to increase clock speed and so must exploit increasing transistor density in figure of merit (1/f)2-4
Already we build chips with some factor of ten more transistors than this and this is used for "automatic" instruction level parallelism.
  • This corresponds to parallelism in "innermost loops"
However getting much more speedup than this requires use of "outer loop" or data parallelism.
Actually memory bandwidth is an essential problem in any computer as doing more computations per second requires accessing more memory cells per second!
  • Harder for sequential than parallel computers
  • Data locality is unifying concept!
HTML version of Scripted Foils prepared 27 August 1997

Foil 38 Sequential Memory Structure

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Data locality implies CPU finds information it needs in cache which stores most recently accessed information
This means one reuses a given memory reference in many nearby computations e.g.
A1 = B*C
A2 = B*D + B*B
.... Reuses B
L3 Cache
Main
Memory
Disk
Increasing Memory
Capacity Decreasing
Memory Speed (factor of 100 difference between processor
and main memory
speed)
HTML version of Scripted Foils prepared 27 August 1997

Foil 39 Parallel Computer Memory Structure

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
For both parallel and sequential computers, cost is accessing remote memories with some form of "communication"
Data locality addresses in both cases
Differences are quantitative size of effect and what is done by user and what automatically
HTML version of Scripted Foils prepared 27 August 1997

Foil 40 The Federal High Performance Computing and Communication Initiative 1992--1996

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 41 The Federal High Performance Computing and Communication Initiative (HPCCI)

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 51
Originally $2.9 billion over 5 years starting in 1992 and
  • Rapidly growing Information technology component starting in 1994 and total budget now over $1 billion per year
The Grand Challenges
  • Enabled by teraflop computers and important to economy or fundamental research
    • Global warming - NOAA
    • Oil reservoir and environmental simulation - DOE
    • Structural and aerodynamic calculations - NASA
    • Earth observing satellite - data analysis - NASA
    • Human genome - NIH, DOE
    • Quantum chromodynamics - Fundamental Physics
    • Gravitational waves from black holes - Fundamental Physics
    • Molecular modeling - Fundamental Chemistry
Nearly all grand challenges have industrial payoff but technology transfer NOT funded by HPCCI
HTML version of Scripted Foils prepared 27 August 1997

Foil 42 The High Performance Computing and Communications Initiative

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 38
High Performance Computing Act of 1991
HTML version of Scripted Foils prepared 27 August 1997

Foil 43 HPCCI Goals

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 80
Computational performance of one trillion operations per second on a wide range of important applications
Development of associated system software, tools, and improved algorithms
A national research network capable of one billion bits per second
Sufficient production of PhDs in computational science and engineering
HTML version of Scripted Foils prepared 27 August 1997

Foil 44 Note the Trend from Large Scale Numerical Computing to the Integration of Computing and Communication in the NII

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 45 The Blue Books
Supplements to the President's Fiscal Year Budget

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
1992: Grand Challenges
1993: Grand Challenges
1994: Toward a National Information Infrastructure
1995: Technology for the National Information Infrastructure
1996: Foundation for America's Information Future
HTML version of Scripted Foils prepared 27 August 1997

Foil 46 The Blue Book Covers

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 47 What and Why is Computational Science ?

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 7
HTML version of Scripted Foils prepared 27 August 1997

Foil 48 Parallelism Implies Major Changes which have significant educational Implications

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index Secs 47
Different machines
New types of computers
New libraries
Rewritten Applications
Totally new fields able to use computers .... ==> Need new educational initiatives Computational Science
Will be a nucleus for the phase transition
and accelerate use of parallel computers in the real world
HTML version of Scripted Foils prepared 27 August 1997

Foil 49 What is Computational Science?

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Computational Science is an interdisciplinary field that integrates computer science and applied mathematics with a wide variety of application areas that use significant computation to solve their problems
Includes the study of computational techniques
  • Science and Engineering - Grand Challenges
  • Society and Business - National Challenge
Includes the study of new algorithms, languages and models in computer science and applied mathematics required by the use of high performance computing and communications in any (?) important application
  • At interface of (applied) computer science and applications
Includes computation of complex systems using physical analogies such as neural networks and genetic optimization.
HTML version of Scripted Foils prepared 27 August 1997

Foil 50 What do we have at Syracuse University?

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Formal Master's Program with reasonable curriculum and course material
PhD called Computer and Information Science but can choose computational science research
Certificates(Minors) in Computational Science at both the Masters and PhD Level
Undergraduate Minors in Computational Science
All Programs are open to both computer science and application (computer user) students
Currently have both an "Science and Engineering Track" ("parallel computing") and an "Information oriented Track" ("the web")
HTML version of Scripted Foils prepared 27 August 1997

Foil 51 Program in Computational Science
Implemented within current academic framework

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 52 Methodology for Computation

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 53 Usefulness of Computational Science Degrees:

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Conclusions of DOE Conference on Computational Science Education, Feb 1994
Industry and government laboratories want graduates with Computational Science and Engineering training - don't care what degree is called
Universities - want graduates with Computational Science and Engineering training - want degrees to have traditional names
Premature to have BS Computational Science and Engineering
HTML version of Scripted Foils prepared 27 August 1997

Foil 54 Syracuse Computational Science Academic Programs -- Masters Degree

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Master's Degree in Computational Science Course Requirements:
Core Courses:
  • CPS 615: Introduction to Computational Science
  • CPS 675: Design and analysis of algorithms
  • MAT 683: Methods of numerical analysis I
Application Area:
  • Applications of computational science, including a substantial project. Example: CPS713 Case Studies in Computational Science
It is required to take one course in 3 out of the following 4 areas:
  • 1. Parallel programming, algorithms, and architecture
  • 2. Methodology and techniques Numerical analysis, optimization, simulation
  • 3. High performance software Compilers, languages, visualization, programming environments
  • 4. Advanced computer science and software engineering Structured programming and formal methods
HTML version of Scripted Foils prepared 27 August 1997

Foil 55 Syracuse Graduate Computational Science Academic Programs

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Minors in Computational Science
Masters Level Certificate:
  • Available to graduate students enrolled in any SU masters or Ph.D. program
  • Courses required for certificate are one from each area (15 credits)
    • 1. CPS 615: Introduction to Computational Science -- Simulation or
    • 1. CPS 616: Computational Science for Information Applications
    • (probably CPS 606 can be substituted here)
  • 2. Applications of Computer Science (e.g. CPS713/714)
  • 3. High Performance Parallel Computing
  • 4. Methodology and techniques
  • 5. Computational Science elective -Relevant course chosen by student - (e.g. CPS730)
Doctoral level Certificate:
  • 5 courses as above with one more elective (18 credits)
  • Make a contribution to computational science through the research of the dissertation
Doctoral level Certificate in Computational Neuroscience:
  • Joint program Bioengineering (Institute of Sensory Research, SUNY Health Science Center, Computer Science
HTML version of Scripted Foils prepared 27 August 1997

Foil 56 Some Academic Areas and their Relation to Computational Science

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Computer Science -- Nationally viewed as central activity
  • Congress thinks Computer Science is activities such as NSF Supercomputer Centers i.e. Computational Science
  • Computer Scientists think of the field as less applied
Computer Engineering -- Historically Mathematics and Electrical Engineering have spawned Computer Science programs -- if from electrical engineering, the field is sometimes called computer engineering
Applied Mathematics is a very broad field in U.K. where equivalent to Theoretical Physics. In USA applied mathematics is roughly mathematics associated with fluid flow
  • Field teachs areas such as Scientific Computing even though ignore many issues needed outside differential equation solution
Computational Physics -- Practioners will be judged by their contribtion to physics and not directly by algorithms and software innovations.
  • Similar remarks about Computational Aerospace, Chemistry etc.
HTML version of Scripted Foils prepared 27 August 1997

Foil 57 Program in Information Age Computational Science Implemented Within Current Academic Program

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 58 Parallel Processing and Society

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
The fundamental principles behind the use of concurrent computers are identical to those used in society - in fact they are partly why society exists.
If a problem is too large for one person, one does not hire a SUPERman, but rather puts together a team of ordinary people...
cf. Construction of Hadrians Wall
HTML version of Scripted Foils prepared 27 August 1997

Foil 59 Concurrent Construction of a Wall
Using N = 8 Bricklayers
Decomposition by Vertical Sections

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Domain Decomposition is Key to Parallelism
Need "Large" Subdomains l >> l overlap
HTML version of Scripted Foils prepared 27 August 1997

Foil 60 Quantitative Speed-Up Analysis for Construction of Hadrian's Wall

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 61 Amdahl's law for Real World Parallel Processing

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
AMDAHL"s LAW or
Too many cooks spoil the broth
Says that
Speedup S is small if efficiency e small
or for Hadrian's wall
equivalently S is small if length l small
But this is irrelevant as we do not need parallel processing unless problem big!
HTML version of Scripted Foils prepared 27 August 1997

Foil 62 Pipelining --Another Parallel Processing Strategy for Hadrian's Wall

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
"Pipelining" or decomposition by horizontal section is:
  • In general less effective
  • and leads to less parallelism
  • (N = Number of bricklayers must be < number of layers of bricks)
HTML version of Scripted Foils prepared 27 August 1997

Foil 63 Hadrian's Wall Illustrates that the Topology of Processor Must Include Topology of Problem

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Hadrian's Wall is one dimensional
Humans represent a flexible processor node that can be arranged in different ways for different problems
The lesson for computing is:
Original MIMD machines used a hypercube topology. The hypercube includes several topologies including all meshes. It is a flexible concurrent computer that can tackle a broad range of problems. Current machines use different interconnect structure from hypercube but preserve this capability.
HTML version of Scripted Foils prepared 27 August 1997

Foil 64 General Speed Up Analysis

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Comparing Computer and Hadrian's Wall Cases
HTML version of Scripted Foils prepared 27 August 1997

Foil 65 Nature's Concurrent Computers

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
At the finest resolution, collection of neurons sending and receiving messages by axons and dendrites
At a coarser resolution
Society is a collection of brains sending and receiving messages by sight and sound
Ant Hill is a collection of ants (smaller brains) sending and receiving messages by chemical signals
Lesson: All Nature's Computers Use Message Passing
With several different Architectures
HTML version of Scripted Foils prepared 27 August 1997

Foil 66 Comparison of Concurrent Processing in Society and Computing

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Problems are large - use domain decomposition Overheads are edge effects
Topology of processor matches that of domain - processor with rich flexible node/topology matches most domains
Regular homogeneous problems easiest but
irregular or
Inhomogeneous
Can use local and global parallelism
Can handle concurrent calculation and I/O
Nature always uses message passing as in parallel computers (at lowest level)
HTML version of Scripted Foils prepared 27 August 1997

Foil 67 Back from the Analogy to Parallel Computers!

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Now we go through a set of semi-realistic examples of parallel computing and show they use various forms of data parallelism
Seismic Wave Simulation: Regular mesh of grid points
Cosmology (Universe) Simulation: Irregular collection of stars or galaxies
Computer Chess: "Data" is now parts of a game tree with major complication of pruning algorithms
  • "Deep Blue" uses this global parallelism combined with optimal locally parallel evaluation of "score" of a position using special hardware
Defending the Nation: Tracking multiple missiles achieves parallelism from set of missiles
Finite Element (NASTRAN) structures problem: Irregular collection of nodal (grid) points clustered near a crack
HTML version of Scripted Foils prepared 27 August 1997

Foil 68 Concurrent Computation as a Mapping Problem -I

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
2 Different types of Mappings in Physical Spaces
Both are static
  • a) Seismic Migration with domain decomposition on 4 nodes
  • b)Universe simulation with irregular data but static 16 node decomposition
  • but this problem would be best with dynamic irregular decomposition
HTML version of Scripted Foils prepared 27 August 1997

Foil 69 Concurrent Computation as a Mapping Problem - II

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Different types of Mappings -- A very dynamic case without any underlying Physical Space
c)Computer Chess with dynamic game tree decomposed onto 4 nodes
HTML version of Scripted Foils prepared 27 August 1997

Foil 70 Concurrent Computation as a Mapping Problem - III

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 71 Finite Element Mesh From Nastran
(mesh only shown in upper half)

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
HTML version of Scripted Foils prepared 27 August 1997

Foil 72 A Simple Equal Area Decomposition

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
And the corresponding poor workload balance
HTML version of Scripted Foils prepared 27 August 1997

Foil 73 Decomposition After Annealing
(one particularly good but nonoptimal decomposition)

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
And excellent workload balance
HTML version of Scripted Foils prepared 27 August 1997

Foil 74 Comparison of The Complete Problem to the subproblems formed in domain decomposition

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
The case of Programming a Hypercube
Each node runs software that is similar to sequential code
e.g., FORTRAN with geometry and boundary value sections changed
HTML version of Scripted Foils prepared 27 August 1997

Foil 75 Hadrian's Wall Illustrating an
Irregular but Homogeneous Problem

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Geometry irregular but each brick takes about the same amount of time to lay.
Decomposition of wall for an irregular geometry involves equalizing number of bricks per mason, not length of wall per mason.
HTML version of Scripted Foils prepared 27 August 1997

Foil 76 Some Problems are Inhomogeneous Illustrated by:
An Inhomogeneous Hadrian Wall with Decoration

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Fundamental entities (bricks, gargoyles) are of different complexity
Best decomposition dynamic
Inhomogeneous problems run on concurrent computers but require dynamic assignment of work to nodes and strategies to optimize this
(we use neural networks, simulated annealing, spectral bisection etc.)
HTML version of Scripted Foils prepared 27 August 1997

Foil 77 Global and Local Parallelism Illustrated by Hadrian's Wall

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Global Parallelism
  • Break up domain
  • Amount of Parallelism proportional to size of problem (and is usually large)
  • Unit is Bricklayer or Computer node
Local Parallelism
  • Do in parallel local operations in the processing of basic entities
    • e.g. for Hadrian's problem, use two hands, one for brick and one for mortar while ...
    • for computer case, do addition at same time as multiplication
  • Local Parallelism is limited but useful
Local and Global Parallelism
Should both be Exploited
HTML version of Scripted Foils prepared 27 August 1997

Foil 78 Parallel I/O Illustrated by
Concurrent Brick Delivery for Hadrian's Wall
Bandwidth of Trucks and Roads
Matches that of Masons

From CPS615-Introduction-Course,Driving Technology and HPCC Current Status and Futures CPS615 Basic Simulation Track for Computational Science -- Fall Semester 97. *
Full HTML Index
Disk (input/output) Technology is better matched to several modest power processors than to a single sequential supercomputer
Concurrent Computers natural in databases, transaction analysis
© Northeast Parallel Architectures Center, Syracuse University, npac@npac.syr.edu

If you have any comments about this server, send e-mail to webmaster@npac.syr.edu.

Page produced by wwwfoil on Sun Aug 23 1998