Exhibits, Demos & Posters
Exploiting Latent I/O Asynchrony in Petascale Science Applications
Authors
- Mary Payne, Department of Computer Science, University of New Mexico
- Patrick Widener, Department of Computer Science, University of New Mexico
- Matthew Wolf, College of Computing, Georgia Institute of Technology
- Hasan Abbasi , College of Computing, Georgia Institute of Technology
- Scott McManus, College of Computing, Georgia Institute of Technology
- Patrick G. Bridges, Department of Computer Science, University of New Mexico
- Karsten Schwan, College of Computing, Georgia Institute of Technology
Abstract
We present a collection of techniques for exploiting latent I/O asynchrony which can substantially improve performance in data-intensive parallel applications. Latent asynchrony refers to an application's tolerance for decoupling ancillary operations from its core computation, and is a property of HPC codes not fully explored by current HPC I/O systems. Decoupling operations such as buffering and staging, reorganization, and format conversion in space and in time from core codes can shorten I/O phases, preserving expensive MPP compute cycles. We describe in this poster three software tools—DataTaps, IOgraphs, and Metabots—which allow HPC developers to structure the I/O of their applications in such a decoupled manner. We also show how asynchrony can be exploited by data generators which overlap computation with communication, and by data consumers that perform data conversion and reorganization out-of-band and on-demand. In the context of a data-intensive fusion simulation, we show that exploiting latent asynchrony through decoupling of operations can provide significant performance benefits to HPC applications.
