Additional Comments to

Basic Issues and Current Status of Parallel Computing

by Geoffrey Fox

by Ulf Dittmer

• Benchmarks & Performance Comparison

  Benchmarking machines in order to have some kind of "objective measure" to compare it with other machines is especially important with parallel computers. High Peformance is one of the main reasons for their existence (besides having a large memory), so prospective buyers will want to make sure a parallel computer satisfies their performance needs. There are quite a few differences to benchmarking sequential computers, though: Response time is usually not very important, as most production runs will probably be in batch mode anyway. Two machines claiming to have, e.g., 64 nodes each, can (and usually will) have wildly different architectures, as far as the connecting network and the nodes themselves are concerned; so they can't really be compared to each other - which one is "better" can depend strongly on the problem.

  Index at the NHSE

• Vector Computers

  Though vector computers are generally not considered to be parallel machines, they deserve a mention because their primary reason for existence is the same as for parallel computers: performance. Apart from that, todays vector computers, such as the Cray C-90 or T-90, have quite a few vector processors working in parallel (up to 16 or 32, respectively.) Admittedly, these vector processors are mostly used for serving different processes in parallel, but they can be used in the same way as in parallel machines; a lot of the memory and I/O problems raised are also similar.

  Index of Vendors

  Descriptions of Current Machines

• Parallel Algorithms

  A lot of problems that we might consider to be sequential are in fact candidates for at least a partial parallelization. Trying to parallelize a sequential algorithm with the least passible amount of work might not yield an efficient parallel program, though. Quite often a different approach must be pursued, and whether a data-parallel or a message-parallel algorithm works better is also not generally clear from the beginning.
  Also, parallel algorithms are not only measured in their time complexity (like sequential algorithms), but in their time-processor complexity. E.g., a sequential program may have O(n lg n) time complexity, which means a O(n lg n) time-processor complexity (since we have only one processor). If a parallel version of this algorithm runs in O(lg n) time on n processors, it also has O(n lg n) time-processor complexity and is not a real breakthrough (though it is quite a bit faster).

  Algorithms for Irregular Problems

  Chapter 7 of Gregors Introduction gives some examples and compares data-parallel and message-parallel solutions to mathematical problems.

• Parallel Languages

  The paper mentions only data-parallel and message-parallel versions of Fortran, yet there is quite a variety of languages to use for parallel computing, though they are not as pervasive as Fortran (especially not for scientific applications).
  MPI and PVM are message-passing standards that can used be used from any number of languages, for instance from C, one of the most widely used languages.
  Due to the popularity of the now-defunct Connection Machine, its data-parallel version of C, called C*, used to be well-known in the field. C* might survive as part of the GlobalWorks software environment, now that TMC is a software-only company.

  Overview by Guy Blelloch

You can send me email on any topic you see fit.