INSTITUTION NAME: Syracuse University

WORKPAGE TITLE: WebHLA

PROJECT TITLE: SPEEDES + HLA Based Scalable Parallel CMS Federate

POC: David Bernholdt, bernholdt@npac.syr.edu, (315) 443-3857, fax
(315) 443-1973

CTA or PEI: Forces Modeling and Simulation (FMS)

PROJECT DESCRIPTION: 

In Year 3, we ported CMS code to the Origin2000 platforms at CEWES and
ARL. We achieved almost linear speedup i.e. perfect scalability up to
and including 4 processors, but the performance deteriorates for 8 of
more processors. We identified the reason as related to rather complex
object-oriented data structures (including dynamic linked lists of
irregular objects), present in the inner loop of the C++ CMS code that
runs over all mines and vehicle PDUs, tracking for possible hits and
explosions. Our parallelization technique was based on Origin2000 C++
parallel pragmas, i.e. compiler directives that allow to decompose
loop indices over processors.  However, due to the NUMA architecture
of Origin2000 and large memory utilization of the CMS inner loop, we
also need to perform the domain decomposition of the associated
objects over processors to enforce the memory locality. Such data
decomposition pragmas are also available but we discovered that the
proper synchronization of the loop and memory decomposition directives
turned out to be non-optimal for the CMS inner loop. In Year 4, we
propose to enter more deeply into the CMS inner loop and rearrange it
in terms of simpler, more regular arrays rather then complex dynamic
objects, so that one can control better data decomposition and memory
locality. In another ongoing PET FMS project “CHSSI Training
Materials” we are building training materials for the SPEEDES
simulation kernel which is known to offer good scalability for a range
of simulation problems on Origin2000. We therefore propose to explore
in Year 4 the use of SPEEDES as the underlying framework for Scalable
Parallel CMS. We are also participating in CHSSI FMS-5 which develops
SPEEDES based HPC RTI. Using this new SPEEDES capability we will be
able to package our Parallel CMS module as HPC HLA Federate and to
include it in the WebHLA based federation where it will cooperate with
other federates listed in the Year 4 Proposal for the “FMS PET Core
Support”.

PROJECT OBJECTIVES:  

Improve scalability of Parallel CMS over a broad processor range on
Origin2000 by simplifying and regularizing the inner loop data
structures and by using scalable communication libraries of the
SPEEDES simulation kernel.

DELIVERABLES:  

1) Parallel CMS module, scalable over the whole processor range on
   Origin2000; 

2) Analysis of performance and tradeoffs between various communication
   modes, including SPEEDES based scalable communication library;

3) Installation of the scalable Parallel CMS module at CEWES and ARL;

4) Evaluation of total CPU power for Metacomputing CMS on multi-MSRC
   platform.

CUSTOMERS/END USERS: 

1) Night Vision Laboratory at Ft. Belvoir, VA; 2) U.S.  Army
Engineering School (NVL customer and collaborator); 3) Possibly Navy
Simulation System; 4) Possibly JSIMS-Maritime.

BENEFIT TO THE WARFIGHTER: 

Dynamic minefields play a crucial tactical role in the modern
warfare. Minefields of millions of mines can be rapidly deployed today
by the Air Force and they can affect mobility and imply major changes
in the enemy Army or Navy operations due to the rapidly blocked routes
or areas. CMS is the unique M&S system capable of simulating such
advanced mine warfare operations. However, practical simulation of
large minefields with millions of active mines require high
performance computing. Parallel CMS will therefore repesent a useful
advanced simulation module for the next generation realistic
simulation systems such as JSIMS.

PROJECT DEPENDENCIES AND SCOPE: 

This project is related to and needs to be pursued in parallel with
the “WebHLA based Metacomputing CMS for the Million Mines
Challenge” project..

RISK ELEMENT: 

CMS (and other M&S applications) is a large and complex code.  Though
we have spent some time on the analysis, it is possible that
unforeseen problems may crop up with the port or parallelization. This
project anticipates close collaboration with NVL, and additional
support from the CEWES MSRC PET program.

REQUIRED FUNDING LEVEL: