CFD and Workload Management Experimentation

Dr. Steven E. Koonin, DOE Undersecretary for Science, gave a presentation to the National Science Foundation Advisory Committee for Cyberinfrastructure on 27 May 2010 during which he stated that Within the US we do not do a good job of marshalling our public cyberinfrastructure resources as effectively as we should to meet national priorities in collaboration with the private sector and  We need more examples of HPC [High Performance Computing] being used to the benefit of US industry in solving some of our national energy and manufacturing needs. \n\nCummins, Inc., a global manufacturing company with major facilities in Columbus IN, is interested in collaborating with Indiana University to use the FutureGrid system in measuring the efficacy of new methods for designing diesel engine combustion solutions. Cummins is particular interested in testing an automated optimization code called ModeFRONTIER to run computational fluid dynamics (CFD) codes to study combustion strategies to maximize energy efficiency, minimize pollutant emissions, etc. The work being done is experimental in that Cummins is investigating new automated methods for peforming this design process. There is therefore an excellent opportunity to perform a performance analysis of  time to solution' using Cummins' existing cluster facilities as well as FutureGrid systems.

RECOMMENDATION: We request an allocation of computational time on the Cray XT5 and IBM IDataPlex systems within FutureGrid to perform performance analysis on a set of real workflows of interest to Cummins and to the world as a whole because of the potential to reduce oil dependency and pollution emissions through better design of diesel engines.\n\nBy TeraGrid policy, use of TeraGrid resources requires open publication of results. In this particular case, the result that is of interest to publish openly covers two topics:

Development of the partnership and interaction between Cummins and IU.

Improvement in time to solution for the design and optimization process.

Note that Cummins' proprietary IP trade secrets, combustion system designs, etc. stay exclusively Cummins' and will not be published. The interesting thing to publish is not the engine design information, but rather how use of systems in FutureGrid compare in terms of performance with each other and how they may help accelerate the design process. We will particular compare the two FutureGrid systems desribed below against each other and Cummins' current computing cluster: The IBM iDataPlex system is an IBM e1350 distributed shared memory cluster with 1024 processor cores and 3 TB total memory capacity. The compute nodes consist of 128 dx360 M2 iDataPlex servers, each with two quad-core Intel Xeon processors, 24 GB of memory, and a PCIe Mellanox ConnectX 4x DDR InfiniBand adapter for high bandwidth, low-latency MPI applications. The Cray XT5m is a distributed shared memory cluster with 672 processor cores and 1.3 TB total memory capacity. The compute blades consist of 21 XT5 blades, each with eight quad-core AMD Shanghai processors, 64 GB of memory, and an integrated Cray SeaStar adapter for high bandwidth, low-latency MPI applications. The published work will compare performance on these two platforms with each other and analyze the use of automated optimization software in industrial design.