MapReduce Scheduling in Cloud Environments

Project Information

Discipline
Computer Science (401) 
Orientation
Research 
Abstract

This project aims to develop modules for a MapReduce framework capable of efficiently utilizing the resources in a cloud environment. We will conduct research on the problems associated with existing MapReduce implementations and the components and trade-offs necessary for scheduling and migrating tasks dynamically.

Intellectual Merit

This project will advance the state of the art in scheduling for large-scale data intensive applications by taking into account the heterogeneity in the infrastructure of large grids and clouds.

Broader Impacts

The software modules resulting from this project will be released as open-source to the HPC community.

Project Contact

Project Lead
Renan DelValle (rdelval1) 
Project Manager
Renan DelValle (rdelval1) 
Project Members
Renan DelValle, Jessica Hartog, John Weachock  

Resource Requirements

Hardware System
  • Not sure
 
Use of FutureGrid

Plan to use the FutureGrid resources to increase the scale of our experiments from a few nodes to much larger sizes.

Scale of Use

100-300 nodes, for experiments run a few times a week, for the next 6-12 months.

Project Timeline

Submitted
06/25/2013 - 13:24 
»