Hadoop Testing
Project Information
- Discipline
- Computer Science (401)
- Orientation
- Research
Comparison testing and experimentation for cluster-based Hadoop for comparison with performance on PSC at Blacklight.
Intellectual MeritThis project will enhance our understanding of Hadoop when used on a large shared-memory machine as compared to a traditional cluster installation, delineating use cases where each configuration will be most likely to yield the best results.
Broader ImpactsWith a large shared-memory Hadoop installation, scientific research that involves iterative processing of large data sets should see significant gains.
Project Contact
- Project Lead
- Bryon Gill (bgill)
- Project Manager
- Bryon Gill (bgill)
Resource Requirements
- Hardware System
-
- Not sure
I will use FutureGrid nodes for testing Hadoop performance and stability in configurations that allow a more or less fair comparison with the size of jobs I'm able to run on the Blacklight system.
Scale of UseI will initially want to provision a cluster of 8 nodes, I will likely want to expand that cluster at some point in the future to match the scale of my experiments on Blacklight.
Project Timeline
- Submitted
- 07/26/2011 - 12:02