Need for new educational approachs such as Computational Science centered on interdiciplinary border between computer science and applications with both a
-
Science and Engineering and
-
Information (Communications ) Track
-
See for example
 HTML version of Basic Foils prepared April 7,1995 Foil 7 When will Parallel Computing Take Over ?
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *  
  
Full HTML Index
|
Switch from conventional to new types of technology is a phase transition
|
|
Needs headroom (Carver Mead) which is large (factor of 10 ?) due to large new software investment
|
Machines such as the nCUBE-1 and CM-2 were comparable in cost performance to conventional supercomputers
-
Enough to show that "Parallel Computing Works"
-
Not enough to take over!
|
|
Cray T3D, Intel Paragon, CM-5, DECmpp (Maspar MP-2), IBM SP-2, nCUBE-3 have enough headroom to take over from traditional computers ?
|
HTML version of Basic Foils prepared April 7,1995 Foil 8 Some Hardware/Software Trends over next 5 years
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *  
Full HTML Index
|
|
|
ATM networks have rapidly transitioned from research Gigabit networks to commercial deployment
|
See NPAC's work on NYNET
-
ATM likely to be a major force in local area as well as wide area networks
|
Computer Hardware trends imply that all computers (PC's ---> Supercomputers) will be parallel by the year 2000
-
Up to 1993, parallel computers are from small start-up companies (except Intel Supercomputer Division)
-
Now Cray, Convex (HP), Digital, IBM have massively parallel computing systems and Silicon Graphics is becoming a powerful high performance computing vendor
-
Several architectures but only one : Distributed memory MIMD multicomputer is known to scale from one to very many processors
|
Software is challenge and could prevent/delay hardware trend that suggests parallelism will be a mainline computer architecture
-
We must get systems software correct
-
Simultaneously develop applications software in gradually improving parallel programming environment
|
HTML version of Basic Foils prepared April 7,1995 Foil 9 Parallel Computing Works!
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
What have we learnt ?
-
Essentially all large problems parallelize in principle
-
Written from scratch by an expert but
-
Very Hard if need to Port existing codes from ill-documented Fortran or C++
-
See "Parallel Computing Works" Morgan Kaufman, 1994.
|
Experience dominated by academic and research applications as opposed to real = "industry and government" problems
-
Technology (ARPA) will give us (experimental) teraflop (1012 operations per second) machine by 1995; production teraflop machine by 2000.
-
Several hardware architectures "work"
-
Software lagging behind hardware and parallel algorithms
-
Need productive (easy to use) portable (runs on all current machines) scalable ( will run on future parallel machines ) software.
-
e.g. High Performance Fortran addresses these goals.
|
HTML version of Basic Foils prepared April 7,1995 Foil 10 Prospects for use of HPCC in the "Real-World"
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
In simulation arena, HPCC will work in principle but the need to develop or port large complex codes represents a serious handicap (see next foil) which for instance could hold up the "obvious" use of HPCC in Manufacturing by 10 to 20 years
|
|
In High Speed Network arena, most applications such as "video-on-demand" are "new" and so HPCC competes much more favorably with traditional computer technology.
|
In database arena, we can also expect quicker adoption of HPCC than for silmulation
-
Information processing is largest use of computers and so easier to justify investment in new technology
-
A parallel (relational) database is complicated to develop but only has to be done once -- SQL is naturally parallel (unlike Fortran) and so applications can be developed without knowledge of underlying hardware.
-
On the other hand, a parallel Fortran compiler is also very complex but STILL EVERY user must code carefully to exploit parallelism and essentially no existing Fortran codes parallelize without further thought.
|
HTML version of Basic Foils prepared April 7,1995 Foil 11 Data Parallelism is a Universal Source of Scaling Parallelism
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 12 We have learnt that Parallel Computing Works !
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Data Parallelism - universal form of scaling parallelism
|
|
Functional Parallelism - Important but typically modest speedup. - Critical in multidisciplinary applications.
|
On any machine architecture
-
Distributed Memory MIMD
-
Distributed Memory SIMD
-
Shared memory - this affects programming model
-
This affects generality
-
SIMD ~ 50% academic problems
-
but < 50% commercial
|
HTML version of Basic Foils prepared April 7,1995 Foil 13 Methodology of Parallel Computing
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Simple, but general and extensible to many more nodes is domain decomposition
|
All successful concurrent machines with
-
Many nodes
-
High performance (this excludes Dataflow)
|
|
Have obtained parallelism from "Data Parallelism" or "Domain Decomposition"
|
Problem is an algorithm applied to data set
-
Obtain concurrency by acting on data concurrently.
|
The three architectures considered here differ as follows:
-
MIMD Distributed Memory -- Processing and Data Distributed
-
MIMD Shared Memory -- Processing Distributed but memory shared
-
SIMD Distributed Memory -- Synchronous Processing on Distributed Data
|
HTML version of Basic Foils prepared April 7,1995 Foil 14 Concurrent Computation as a Mapping Problem -I
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *  
Full HTML Index
|
2 Different types of Mappings in Physical Spaces
|
Both are static
-
a) Seismic Migration with domain decomposition on 4 nodes
-
b)Universe simulation with irregular data but static 16 node decomposition
-
but this problem would be best with dynamic irregular decomposition
-
|
HTML version of Basic Foils prepared April 7,1995 Foil 15 Concurrent Computation as a Mapping Problem - II
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Different types of Mappings -- A very dynamic case without any underlying Physical Space
|
|
c)Computer Chess with dynamic game tree decomposed onto 4 nodes
|
HTML version of Basic Foils prepared April 7,1995 Foil 16 Concurrent Computation as a Mapping Problem - III
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 17 Structure of Problems and Parallel Software Approaches
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 18 Problem Architectures
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See Parallel Computing Works for general discussion of problem architecture
|
|
Software Maps
|
|
Problem ---> Machine
|
|
Often software designed around machine architecture
|
|
Rather software should be designed around problem architecture
|
|
Only able to persuade application scientist to parallelize
|
HTML version of Basic Foils prepared April 7,1995 Foil 19 5 Categories of Problems
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Synchronous: Data Parallel Tightly coupled and software needs to exploit features of problem structure to get good performance. Comparatively easy as different data elements are essentially identical.
|
Loosely Synchronous:
-
As above but data elements are not identical. Still parallelizes due to macroscopic time synchronization.
|
Asynchronous:
-
Functional (or data) parallelism that is irregular in space and time. Often loosely coupled and so need not worry about optimal decompositions to minimize communication. Hard to parallelize (massively) unless ....
|
Embarrassingly parallel:
-
Essentially independent execution of disconnected components. (can involve reductions)
|
Metaproblems
-
Asynchronous collection of (loosely) synchronous components where these programs themselves can be parallelized
|
HTML version of Basic Foils prepared April 7,1995 Foil 20 Example of Basic Problem Architectures Regular Synchronous Problem Class
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See Parallel Computing Works for QCD and Synchronous problems
|
|
eg. Quantum chromodynamics calculations (calculate proton mass)
|
|
Regular grid in 4D space-time - domain decomposition
|
|
Update algorithm identical at each point
|
|
Time evolution very simple
|
|
The world looked at microscopically in terms of a set of similar fundamental quantities
|
|
Monte Carlo - subtlety that cannot update concurrently points linked in Hamiltonian
|
HTML version of Basic Foils prepared April 7,1995 Foil 21 Example of Basic Problem Architectures Irregular Loosely Synchronous
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See Parallel Computing Works for Loosely Synchronous problems
|
|
The world looked at macroscopically in terms of interactions between irregular inhomogeneous objects evolved as a time synchronized simulation
|
|
eg. circuit simulation for computer or biological applications
|
|
Linked sets of neurons (components) of different types
|
|
Time taken to model one time step of type i depends on nature of node and interconnect between them
|
HTML version of Basic Foils prepared April 7,1995 Foil 22 Example of Basic Problem Architectures The difficult Asynchronous Class
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See Parallel Computing Works Asynchronous problems (including Computer Chess)
|
|
The world looked at macroscopically in terms of interactions between irregular inhomogeneous objects evolved as an event driven simulation
|
|
eg. Battle of Hastings
|
HTML version of Basic Foils prepared April 7,1995 Foil 23 Example of Basic Problem Architectures for MetaProblem Class
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See Parallel Computing Works for Metaproblems including Command and Control
|
|
Metaproblems are of growing importance in general HPCC community
|
|
One important Example is:
|
Manufacturing and Design including Multidisciplinary Optimization which combines many fields to look at total product design, manufacturability etc.
-
Geometry
-
Grid Generation
-
Fluid Flow ----> Performance
-
Acoustics
-
Structural Analysis
-
Optimization Module
-
Visualization
|
Also link in concurrent engineering
-
Design - Manufacturing - Marketing - Service
|
HTML version of Basic Foils prepared April 7,1995 Foil 24 Software Bus Structure of USMADE
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
USMADE is US Multidisciplinary Analysis and Design Environment Project of MADIC Industrial Consortium
|
HTML version of Basic Foils prepared April 7,1995 Foil 25 The Mapping of Heterogeneous Metaproblems onto Heterogeneous Metacomputer Systems
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 26 Further Examples of Metaproblems
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Global Change:
-
Atmospherical Model
-
Ocean Model
-
Land Model
-
Policy
-
Visualize
|
Computational Electromagnetic Scattering (CEM) - A metaproblem of smaller granularity
-
SregionsCalculate Matrix Elements
-
Solve Matrix Equations
-
Visualize
|
|
Note this (CEM) itself could be a part - the electromagnetic signature calculation - in design of military aircraft
|
HTML version of Basic Foils prepared April 7,1995 Foil 27 Heterogeneous Metaproblem Structure for Command and Control
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Si: Satellite with:
-
Image processing
-
Kalman filter
-
Stereo match
|
FCi: Fire Control platform
-
with loosely synchronous target-weapon pairing algorithm
|
|
E: Overall Asynchronous expert system
|
HTML version of Basic Foils prepared April 7,1995 Foil 28 More Remarks on Major Defence MetaProblem BMC3IS - Decision Support (Dual Use !)
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Data gathering
-
Multiple sensors / satellites
|
|
Tracking and other image processing algorithms
|
|
Database
|
|
Geographical Information System
|
|
Spatial reasoning functions (shortest path, visibility, mobility, ....)
|
|
Target - weopen pairing
|
|
Look ahead simulation
|
|
Expert systems and other decision aids
|
HTML version of Basic Foils prepared April 7,1995 Foil 29 Remarks on What are key Global Issues in determining ease of parallelization of particular applications
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 30 Criterion for success in parallelizing a particular problem on a particular machine
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Software powerful enough to express natural parallelism of problem (work in progress)
|
|
Problem is "large"
|
Architecture of computer at least as "rich" as architecture of problem
-
Temporal (control) structure
-
SIMD v. MIMD
-
This is a strong constrain
-
Spatial (topology) structure
-
"Weak" constraint
-
Mismatches "only" affect performance
|
HTML version of Basic Foils prepared April 7,1995 Foil 31 What is the "correct" machine architecture for each problem class
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Problem class Machine
|
|
Synchronous SIMD, MIMD
|
|
Loosely Synchronous MIMD
|
|
Asynchronous Unclear
|
|
Metaproblems Heterogeneous network
|
|
Embarrassingly Parallel Network of workstations
|
HTML version of Basic Foils prepared April 7,1995 Foil 32 Computing as a Mapping Problem
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
A hierarchy of mapping problems
|
|
We would like to optimize the overall mapping
|
HTML version of Basic Foils prepared April 7,1995 Foil 33 Complex Systems to give a Theory of Computing
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Both problems and computer are
|
|
Collections of homogeneous or heterogeneous basic entities with some sort of static or dynamic connection between them
|
|
Modeling involves mapping one "complex problem" into another
|
Simulation or parallel computing involves map
-
Complex problem ---> Complex Computer
-
|
|
What parameters of these two underlying complex systems control effectiveness of map?
|
HTML version of Basic Foils prepared April 7,1995 Foil 34 Parallel Computing is "just" an optimization problem, even if we can't agree on what to optimize
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Execution time
-
main focus of HPCC community?
|
User happiness
-
main focus of software engineering community
|
HTML version of Basic Foils prepared April 7,1995 Foil 35 Complex System Representation of Levels in Computer Software
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 36 Steps of a Parallel Software Engineering Process
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 37 Some Overall Comments on Status of Grand and National Challenges
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 38 Characteristics of Grand Challenges and Comparison with Real World National Challenges
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
see Blue books 94 or 95 or Grand Challenge List or National Challenge List
|
|
Partial Differential Equations
|
|
Particle Dynamics and Multidisciplinary Integration
|
|
Image Processing
|
|
|
|
Some:
|
|
Visualization
|
|
Artificial Intelligence
|
|
|
|
Not Much:
|
|
Network Simulation
|
|
Economic (and other complex system) modeling
|
|
Scheduling
|
|
Manufacturing
|
|
Education
|
|
Entertainment
|
|
Information Processing
|
|
BMC3IS (Command & Control in military war)
|
|
Decision Support in global economic war
|
HTML version of Basic Foils prepared April 7,1995 Foil 39 Prospects for HPCC in Simulation Arena
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Industrial and Government Simulations in "old" areas where computation is well established
-
Are a factor of 10 to 100 larger (up to 106 lines)
-
Have a longer lifetime
-
Less accessible expertise
-
This a problem for both new codes and even more so, for porting of old codes
-
This implies that the parallel hardware must be much better (factor of >ten "headroom" in Carver Mead's model) than sequential competition.
|
Must have better software to port and maintain large parallel codes -- can only convert once!
-
Are software issues the same for academic and industrial production codes?
-
Probably not? So we need to integrate computer science (software) research and development with experience from industrial codes.
|
HTML version of Basic Foils prepared April 7,1995 Foil 40 Opportunities for HPCC in the Science and Engineering Simulation Arena
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
In spite of the large and very succesful national activity, simulation will not be a large "real world" sales opportunity for MPP's
-
Maybe difficulties for Thinking Machines illustrate this
|
However some areas of national endeavor will be customers for MPP's used for simulation
-
Large Scale Academic Calculations
-
Value of Increased Computation demonstrated in many disciplines
-
Codes are sufficiently small that software engineering considerations of adapting 1,000,000 lines not so important
-
Petroleum Industry
-
Resevoir Simulation
-
Siesmic Data Analysis
-
Some Earth and Space Science including
-
Climate and Weather Forecasting
|
HTML version of Basic Foils prepared April 7,1995 Foil 41 Some Simulation Areas where it will be Possible to exploit HPCC in relatively near term
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Some areas which may adopt HPCC for simulation in relatively near future
-
Pharmaceutical Industry
-
Intense and brilliant academic (government research laboratory) effort in biochemical molecular modelling
-
But "Computer Designed Drugs" are not sufficiently promising to clearly justify purchase of large MPP's by drug industry
-
Financial Industry
-
MPP's being used by Prudential but in spite of success, they are not yet being generally adopted
-
Networks of Workstations severe competition as many problems are "embarassingly parallel"
-
Electrical Power Industry
-
Value seems clear for planning and real time control but
-
Industry conservative and faced with growing near term competition
|
HTML version of Basic Foils prepared April 7,1995 Foil 42 Suprisingly Difficult and Suprisingly Promising Areas for HPCC in Simulation
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
The role of HPCC in Manufacturing is quite clear and will be critical to
-
Agile Manufacturing and the year 2010 Manufacturing Industry but for Major fields including
-
HPCC will not have a major impact for simulation in the next few years
|
On the other hand for
-
War Games and Simulations of Complex Scenarios
-
Role of MPP's can be expected to grow especially when coupled as in (old) SIMNET with high speed geographically distributed networks
-
Note this is different basic software technology
-
Event driven -- not time stepped -- simulation
|
HTML version of Basic Foils prepared April 7,1995 Foil 43 Why is it hard to use HPCC in Manufacturing-I?
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Return on Investment Unclear:
-
Amdahl's law for use of HPCC in Industrial Simulation
-
If Simulation was only 10% or less of original design and manufacturing cycle, then can only gain this 10% by speeding up simulation
-
And this speedup comes at huge software engineering cost !
-
Codes are long and expertise to convert to parallelism may no longer exist in new "slim" companies after layoffs , buyouts and freeze on hiring new employees with knowledge of new technologies such as HPCC
-
New codes must be validated by extensive tests before use
-
Remember we can't solve full Navier-Stokes Equations yet and so some approximations necessary
|
|
The Industry is in a very competitive situation and focussed on short term needs
|
HTML version of Basic Foils prepared April 7,1995 Foil 44 Why is it hard to use HPCC in Manufacturing-II?
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
In March 1994 Arpa Meeting in Washington, Boeing(Neves) endorsed parallel databases and not parallel simulation
-
Similar comment made to me by Major Brokerage
-
"Financial Modelling (on MPP) gets the headlines but information services are the critical problem"
|
Aerospace Engineers are just like University Faculty
-
They prefer to use their own workstations and not central Supercomputers
|
There is perhaps some general decline of Supercomputer Industry
-
As performance of technology increases
-
Users don't take full advantage of this performance Increase
-
Rather buy somewhat more powerful computers at somewhat lower cost
|
HTML version of Basic Foils prepared April 7,1995 Foil 45 Multidisciplinary Analysis and Design as a Critical use of HPCC in Manufacturing?
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
MAD (Multidisciplinary Analysis and Design) links:
-
Structural, Fluid flow, electromagnetic signature, manufacturing process computations
-
Design, Manufufacturing, Sales and Support Functions
|
|
(Includes MDO -- Multidisciplinary Optimization)
|
Link Simulation and CAD Processes
-
Technically link CAD databases (using parallel database technology) to MPP simulations
|
|
This is really important application of HPCC as addresses "Amdahl's Law" as we use HPCC to support full manufacturing cycle -- not just one part! Thus large improvements in manufacturers time to market and product quality possible.
|
BUT must change and even harder integrate:
-
ALL software used in Manufacturing and this now comes from different vendors
-
The way of doing business in company
-
New job skills and cultures -- the hardest problem
|
HTML version of Basic Foils prepared April 7,1995 Foil 46 Role of Government and DoD in HPCC Simulation Applications
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
The limited nearterm industrial use of HPCC implies that it is critical for Government and DoD to support and promote
|
DoD Simulation: Dual-Use Philosophy implies
-
Can use Commercial MPP hardware and basic systems software
-
Cannot rely on commercial market for application and sophisticated systems software and indeed hardware targeted at engineering and science simulation
|
Manufacturing Support can lead to future US Industry leadership in advanced HPCC based manufacturing environments 10-20 years from now
-
Industry cannot afford and even consider long term investment needed to integrate HPCC into manufacturing
-
Government should support long-term not short-term needs
-
Government must involve manufacturing Industry in its plans
-
Currently federal Initiatives are correctly involving Industry in more major fashion than before but focussing on short term needs
|
HTML version of Basic Foils prepared April 7,1995 Foil 47 The HPCC Software Industry is not Viable in Simulation Area ?
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
An HPCC Software Industry is essential if HPCC field is to become commercially succesful
|
|
The HPCC Simulation market is small
|
This market is not used to paying true cost for software
-
As cost traditionally bundled with hardware and one can get
-
Federal Grants to develop software yourself ....
|
|
There is a lot of excellent available public domain software (funded by federal government)
|
Small Businesses are natural implementation of HPCC Software Industry
-
Plenty of talented Entrepreneurs
|
Two InfoMall Success Stories
-
Portland Group: Commercializing High Performance Fortran Compiler developed at NPAC. We are not competing with them but adding value
-
Applied Parallel Technologies:Developing with NIST ATP and Venture Capital portable database exploitation tools
-
NPAC provides HPCC facilities, expertise and contact with best of class international software activities
|
HTML version of Basic Foils prepared April 7,1995 Foil 48 Anecdotes from HPCC Software Industry Arena
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Anecdotes from Thinking Machines (TMC) April 94 before the fall
-
"They would like to be a software company but you can only sell software if bundled with hardware"
-
Customers did not buy TMC hardware because TMC's software was too good and so one couldn't then get the federal grants to improve parallel systems software
|
Anecdote from Digital September 94:
-
Digital cannot make money on their scientific software package for alpha workstations
-
If Digital charged true cost of development and maintenance, users would make do with good (but not optimized or as complete) public domain software
-
So if workstation market not viable for simulation software, how can much smaller MPP market lead to viable business plans?
|
HTML version of Basic Foils prepared April 7,1995 Foil 49 National Challenges will drive the adoption of HPCC in the "Real World"
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See National Challenge List and Layers
|
These can be defined simply as those HPCC applications which have sufficient market to sustain a true balanced HPCC computing Industry with viable hardware and software companies
-
With this definition, some "Grand Challenges" such as Oil Exploration are National challenges
|
Alternatively one can define National Challenges by the HPCC technologies exploited
-
High speed geographically distributed (ATM) networks i.e.
-
The National Information Infrastructure (NII) with several hundred million clients and perhaps some 10,000 MPP based high performance multi- media servers
-
Large scale text, Image and Video databases fed by Satellites, Information produced by National Enterprise such as credit card slips etc.
|
HTML version of Basic Foils prepared April 7,1995 Foil 50 The National Challenges Identified by the Federal HPCC Initiative
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Crisis and Emergency Management
-
Civilian words for dual-use analog of Military Command and Control
|
|
Design and Manufacturing
|
|
Education and Lifelong Learning
|
|
Electronic Commerce
|
|
Energy Management
|
|
Environmental Monitoring
|
|
Health Care
|
HTML version of Basic Foils prepared April 7,1995 Foil 51 Program in Computational Science Implemented within current academic framework
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 52 Program in Information Age Computational Science Implemented Within Current Academic Program
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 53 Federal 1994 Blue Book Comparison of National and Grand Challenges
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 54 Some Specific Application Motivators from an Industry Survey done for New York State
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 55 Categories of Industrial and Government Applications of HPCC (with reference to academic applications)
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Define information generally to include both CNN headline news and the insights on QCD gotten from lattice gauge theories
|
Information Production e.g. Simulation
-
Major concentration of MPP and HPCC at present
|
Information Analysis e.g. Extraction of location of oil from seismic data, Extraction of customer preferences from purchase data
-
Growing area of importance and Short term major MPP opportunity in decision support combined with parallel databases
|
Information Access and Dissemination - InfoVision e.g. Transaction Processing, Video-On-Demand
-
Enabled by National Information Infrastructure
-
Very promising medium term market for MPP but need the NII
-
to be reasonably pervasive before area "takes off"
|
Information Integration .
-
Integrates Information Production Analysis and Access e.g.
-
Decision support in business
-
Command and Control for Military
-
Concurrent Engineering and Agile Manufacturing
-
Largest Long Term Market for MPP
|
HTML version of Basic Foils prepared April 7,1995 Foil 56 Tables of Industrial HPCC Applications 1 to 4:SIMULATION
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
|
HTML version of Basic Foils prepared April 7,1995 Foil 57 Tables of Industrial HPCC Applications 5 to 8: SIMULATION
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
|
HTML version of Basic Foils prepared April 7,1995 Foil 58 Tables of Industrial HPCC Applications 9 to 13: SIMULATION
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
|
HTML version of Basic Foils prepared April 7,1995 Foil 59 Tables of Industrial HPCC Applications 14 to 18 Information Analysis -- "DataMining"
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
|
HTML version of Basic Foils prepared April 7,1995 Foil 60 Table of Industrial Applications 19 to 24 for Information Access InfoVision - Information, Video, Imagery and Simulation on Demand
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
|
HTML version of Basic Foils prepared April 7,1995 Foil 61 Information Integration Applications 25 to 28
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
These involve combinations of Information Production, Analysis, Access and Dissemination and thus need the Integration of the various Software and Machines Architecture Issues discussed there
|
|
Sometimes Called System of Systems
|
25: Military and Civilian Command and Control (C2, C3, C4I ....)
-
Battle Management, Command,Control,Communication,Intelligence and Surveillance (BMC3IS)
-
Military Decision Support
-
Crisis Management -- Police and other Government Operations
-
SIMNET simulates this and with people and computers in the loop has many of same issues
|
26 to 28: Applications of InfoVision Services
-
Generalize Compuserve,Prodigy, America Online, Dialog and Other Information Servives
-
26: Decision Support for Society
-
Community Information Systems
-
Travel and Generalized Yellow Page Services
-
27: Business Decision Support -- One example is:
-
Health Care with Image and Video databases supporting telemedicine
-
28: Public Administration and Political Decision Support
-
Government Information Systems
-
Maxwell School at Syracuse University teachs use of realtime video to aid world wide decisions (United Nations)
|
HTML version of Basic Foils prepared April 7,1995 Foil 62 Information Integration Applications 29 to 33
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
29: Real-Time Control Systems
-
Robotics uses Imagery to make decisions(control vehicles)
-
Energy Management controls power use and generation
|
30: Electronic Banking
-
Requires Security, Privacy, Electronic Cash etc.
|
|
31: Electronic Shopping
|
32: Agile Manufacturing -- Multidiscplinary Design and Concurrent Engineering
-
Combines CAD with Applications 1 to 3
-
Requires major changes to Manufacturing Infrastructure and Approach
|
33: Education
-
InfoMall Living Textbook -- 6 Schools on ATM network linked to HPCC InfoVision Servers at NPAC
|
HTML version of Basic Foils prepared April 7,1995 Foil 63 Abbreviations used in tables of Industrial Applications of HPCC
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
See full electronic Table
|
|
PDE Partial Differential Equation
|
|
FEM Finite Element Method
|
|
FD Finite Difference
|
|
ED Event Driven Simulation
|
|
TS Time Stepped Simulation
|
|
CFD Computational Fluid Dynamics
|
|
VR Virtual Reality
|
|
HPF High Performance Fortran
|
Adaptive Software for Irregular Loosely Synchronous Problems
-
handled by pC++, HPF extensions, Message Passing
|
|
Asynchronous Software for asynchronous problems
|
|
(AsyncSoft)
|
|
Integration Software to integrate components of metaproblems
|
|
Software
|
HTML version of Basic Foils prepared April 7,1995 Foil 64 Parallel Computing Software
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
HTML version of Basic Foils prepared April 7,1995 Foil 65 Core Enabling HPCC Software Technologies for Information Production (Simulation)
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
PVM, Express, Linda, MPI
|
|
ISIS (Cornell)
|
|
High Performance Fortran (HPF) Compiler
|
|
High Performance C, C++ Compiler
|
|
HPF Extensions - PARTI
|
|
Parallel / Distributed Computing Runtime Tools
|
|
ADIFOR (Differentiate Fortran Code)
|
|
AVS and Extensions
|
|
High Performance Fortran Interpreter
|
|
Image Processing Software
|
|
Parallel Debugger
|
|
Parallel Performance Visualization
|
|
Parallel Operating Systems
|
|
Virtual Reality
|
|
Event Driven Simulator
|
HTML version of Basic Foils prepared April 7,1995 Foil 66 Core Enabling HPCC Algorithms and Components for Information Production (Simulation)
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Mesh Generation
|
|
SCALAPACK
|
|
Sparse Matrix Solvers - Templates and libraries (Direct and Iterative)
|
Particle Dynamics Kernels - Templates and Libraries
-
( O(N2) to fast multipole)
|
Optimization Methodology and Templates
-
Linear programming
-
Non-linear programming
|
|
Scheduling (neural-net, parallel) Templates
|
HTML version of Basic Foils prepared April 7,1995 Foil 67 Core Enabling HPCC Technologies Information Analysis, Access, Integration
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Parallel (Relational) Database e.g. Oracle 7.0
|
|
Object database
|
|
High Speed Networks
|
|
Multilevel Mass Storage
|
|
Integration Software ("glue")
|
|
Integration of Parallel and Distributed Computing
|
Multimedia Support
-
Video Browsing
-
Image Content
-
Full Text Search
-
Real time I/O (disk ---> network)
|
|
ATM Network Protocols and Management
|
|
Compression
|
|
Parallel Rendering
|
|
Linkage Analysis (between records of database)
|
|
Sorting (large databases)
|
HTML version of Basic Foils prepared April 7,1995 Foil 68 Core Enabling HPCC Technologies Information Analysis, Access, Integration (continued)
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Collaboration Services
-
Multi user video conferencing
-
Electronic whiteboards, etc.
|
|
Security and Privacy
|
|
Usage and Charging Algorithms
|
Televirtuality
-
The world as a metacomputer
-
Naming
-
World Wide Web
|
|
Human-Computer Interfaces
|
|
Information Organization
|
Image Processing
-
Terrain Rendering
-
Kodak Photo-CD
|
|
Geographical Information Systems
|
HTML version of Basic Foils prepared April 7,1995 Foil 69 Pittsburgh Grand Challenges Meeting (May 1993)
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Discussions indicated that very common issues were:
|
Use Multiple Programming Paradigms
-
C++
-
Fortran + Message Passing
-
AVS
-
CMFortran (--> High Performance Fortran)
|
|
Parallel I/O Support
|
HTML version of Basic Foils prepared April 7,1995 Foil 70 What Determines when you can Parallelize an Application
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Each class of problem architectures puts different general constructs on the software
|
One can parallelize an application efficiently if, and only if, you know details of problem architecture
-
Users know structure of their own problems and do this by hand
-
If a computational graph can be extracted from dependency analysis, than one can parallelize "sequential languages" (Fortran 77)
-
If data structures are explicit, as in Fortran 90, then parallelism is clear
|
HTML version of Basic Foils prepared April 7,1995 Foil 71 Software for (Distributed Memory) Parallel Computers -- I/O and O/S
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
General Overall Setup
|
|
|
|
|
I/O, OS Services
-
Typical model: UNIX on the front end with parallel machine as "compute server"
|
Debugging
-
For synchronous or loosely synchronous, similar to sequential
-
See ndb from Parasoft
|
HTML version of Basic Foils prepared April 7,1995 Foil 72 Performance Monitoring and Debugging Load Balancing
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Performance Monitoring
-
Build tracing of messages and events into low level message passing support
-
PICL Oak Ridge
-
PABLO Illinois
|
Load Balancing
-
Decomposing data onto processors is a difficult (NP-complete) optimization problem
-
However do not need exact minima - reasonable decompositions okay
-
So heuristics (approximate minima) effective - described later
-
Spectral Bisection
-
Sorting and Hashing Methods
-
Simulated Annealing
-
Neural Networks
-
Genetic Algorithms
|
HTML version of Basic Foils prepared April 7,1995 Foil 73 Parallel Computing versus Distributed Computing
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Parallel Computing
-
Communication must be high bandwidth and low latency
-
Low flexibility (and overhead) in messages
|
Distributed Computing
-
Communication can be high or low bandwidth
-
Latency typically high ---> can be very flexible messages involving fault tolerance, sophisticated routing, etc.
-
Parallel Computing should on technical grounds define standards as more restrictive (demanding)? However it is smallest field
-
World Wide Web revolutionizing both parallel and distributed approAches and standards
|
HTML version of Basic Foils prepared April 7,1995 Foil 74 Portable Scalable Languages
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
We may have to rewrite our code, but we want the resulting program to run on "All" current and future (anticipated) machines
|
|
No compelling new languages e.g. OCCAM does not solve enough problems to warrant adoption
|
Adapt existing languages
-
Fortran
-
C++
-
ADA
-
C
-
LISP
-
PROLOG
-
BASIC ...... COBOL?
|
|
Can map onto individual sequential or parallel computers or networks of these (distributed computing)
|
HTML version of Basic Foils prepared April 7,1995 Foil 75 Why build on existing languages - especially Fortran/C
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Experience of users
|
|
Good sequential and node compilers exist
|
|
Migration of existing codes
|
Hard to build a new language as need all the features/libraries of Fortran/Cimplemented well.
-
This is why one uses Fortran 90 and not APL
|
|
Fortran and C are good (OK) for a class of applications
|
HTML version of Basic Foils prepared April 7,1995 Foil 76 Characteristics of Fortran (C) Plus Message Passing Programming Model
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
|
Natural MIMD model of programming
|
Used in vast majority of current successful MIMD programmingEach node has program which controls and calculates its data (owner-computes "rule")
-
Do i run over local data
-
CALL READ (for non-local data)
-
CALCULATE ALL i's DATA
|
|
In simplest form one program per node of computer
|
HTML version of Basic Foils prepared April 7,1995 Foil 77 Advantages and Disadvantages of Fortran (C) Plus Message Passing
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Advantages
-
Portable to MIMD distributed and shared memory machines - should scale to future machines with some difficulties (overlapping communication with itself and calculation)
-
Available now
-
MPI is an Industry Standard
-
PVM and Vendor Solutions available now as well as
-
older systems such as Express PICL and
-
Linda with related but different programming model
-
"All Problems" can be written in it
|
Disadvantages
-
Can be "hard-work" for user inserting communication calls
-
Optimizations not portable
-
Only MIMD
|
HTML version of Basic Foils prepared April 7,1995 Foil 78 Characteristics of Data Parallel Fortran Programming Model
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
e.g. Program represented as sequence of functions operating on arrays
-
Dimension: A(100,100), B(100,100), C(100,100)
-
A = B + C
|
Or as another example:
-
A = B + Shift in x, y (C) In APL, Fortran90 - like syntax
-
forall i, j Or we can use explicit forall loops
-
A(i,j) = B (i,j) + C (i-1, j) + ......
|
|
All data owned by a particular processor
|
and we typically use Owner computes rule:
-
For an expression such as A(i,j) = ..........,
-
Processor holding A(i,j) calculates expression after communicating any needed data if r.h.s. contains data not owned by processor.
|
|
Similar concepts in C, C++, LISP, ADA .......
|
HTML version of Basic Foils prepared April 7,1995 Foil 79 Advantages and Disadvantages of Data Parallel Fortran ( C, C++, LISP, ADA )
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Advantages
-
Very portable and scalable to SIMD and MIMD
-
Should be able to eventually handle ~all synchronous and loosely synchronous problems including ones that only run well on MIMD
-
Industry Standard has been adopted
-
High Performance Fortran (HPF)
-
With exception of Cray Research, all key MPP venors and ISV's have adopted
-
Builds on CMFortran, FortranD Vienna Fortran and other commercial and research experience
|
Disadvantages
-
Need to wait for production quality full feature compilers
-
Not all problems can be expressed in it and community is working on HPF extensions
|
HTML version of Basic Foils prepared April 7,1995 Foil 80 Comparison of Programming and Machine Model
From Programming Paradigms A CRPC/MCNC Workshop -- April 10-13 1995. *
Full HTML Index
Note: Distinguish
-
Programming Model ( Architecture of Virtual Machine)
-
Machine Model ( Architecture of Real Machine)
-
We can adopt data parallel Fortran as a programming model for all machines. ( It is specialized to certain problems, not to certain machines )
|
Fortran plus Message Passing
-
naturally maps onto MIMD machines
-
very hard for SIMD real machines
-
can represent SIMD (Synchronous) and MIMD (Loosely Synchronous, Asynchronous) problem
|
Data Parallel Fortran
-
can map onto SIMD and MIMD machines
-
easiest for SIMD (Synchronous) problems
-
can be extended to some MIMD (Loosely Synchronous) problems
|
 Northeast Parallel Architectures Center, Syracuse University, npac@npac.syr.edu
If you have any comments about this server, send e-mail to webmaster@npac.syr.edu.
Page produced by wwwfoil on Sun Feb 22 1998
|
Outside Index
Denote Foils where Image Critical
Denote Foils where Image has important information
Denote Foils where HTML is sufficient