HPC meets Clouds

Project Information

Discipline

Computer Science (401) 

Orientation

Research 

As Cloud computing emerges as a dominant paradigm in distributed systems, it is important to fully understand the underlying technologies that make Clouds possible. One technology, and perhaps the most important, is virtualization. Recently virtualization, through the use of hypervisors, has become widely used and well understood by many. However, there are a large spread of different hypervisors, each with their own advantages and disadvantages. I hope to provide an in-depth analysis of some of today’s commonly accepted virtualization technologies from feature comparison to performance analysis, focusing on the applicability to High Performance Computing environments using FutureGrid resources.

Through our work we will extensively study the necessary requirements to deploy HPC applications in a virtualized environment complementing exiting research on FutureGrid, taking into account key metrics (e.g. throughput, communication latency, etc.) and more detailed analysis of MPI programs. This will help us understand where Clouds stand in relation to Grids/Supercomputers thus allowing us to focus our efforts on improving the current Cloud infrastructure to perform on-par with conventional supercomputing facilities. Our motivation is to develop a new standard for HPC Clouds where the large scalability, high availability and fault-tolerance would provide scientists with a unique advantage when running simulations. [1] Younge, A. J., R. Henschel, J. Brown, G. von Laszewski, J. Qiu, and G. C. Fox, "Analysis of Virtualization Technologies for High Performance Computing Environments", The 4th International Conference on Cloud Computing (IEEE CLOUD 2011), Washington, DC, IEEE, 07/2011

The current Cloud infrastructure suffers from the overhead of utilizing a virtualized environment. This has lessened the impact of Cloud computing on the high performance computing community and the scientific community, at large. Through our project we will investigate and implement solutions to decrease this overhead and establish Clouds as a viable addition to the Grid and HPC cyber-infrastructures. This process would provide scientists with an extra alternative whose benefits include on-demand provisioning (a problem in conventional Grids/HPC), enhanced fault-tolerance and dynamic scale up/scale down or elasticity.

Project Contact

Project Lead

Li Chunyan (anny) 

Project Manager

Li Chunyan (anny) 

Project Members

lee qingchun, gu genqiang, Xiaolong Wen, Xutao Yang, han qiang, pei xiao  

Resource Requirements

Hardware Systems

• alamo (Dell optiplex at TACC)
• foxtrot (IBM iDataPlex at UF)
• hotel (IBM iDataPlex at U Chicago)
• india (IBM iDataPlex at IU)
• sierra (IBM iDataPlex at SDSC)
• xray (Cray XM5 at IU)
• bravo (large memory machine at IU)
• delta (GPU Cloud)
• Network Impairment Device

 

FutureGrid will be utilized as our distributed, virtualized environment. The FutureGrid infrastructure provides a high performance computing platform where virtual machines can take advantage of the high-speed switches, faster CPUs and buses. We plan to provision a virtualized cluster on FutureGrid and evaluate the performance of the WRF application at large scale. Thus we intend to use FutureGrid resources as a high performance, scalable, virtualized cluster.

We will need a few VMs to setup the environment and prepare the actual experiments. It may take some weeks. Then, we will run a set of run at different scales using different systems and different configurations. Analysis will be performed between experiments in order to evaluate only key scenarios of interest. This process may take some months but the use of the resources will not be continued.

Project Timeline

Submitted

07/15/2012 - 02:30