Karnak Prediction Service

Project Information

Discipline

Computer Science (401) 

Subdiscipline

11.01 Computer and Information Sciences, General 

Orientation

Research 

Karnak is a prediction service that was originally deployed on TeraGrid (http://karnak.teragrid.org) and continues to be available in XSEDE. The service currently provides predictions of how long jobs will wait in scheduler batch queues before they begin to execute. This project will use FutureGrid to evaluate alternative implementations of the service, optimize the accuracy of the machine learning techniques used by the service, and investigate alternative prediction techniques.

This work includes an evaluation of different prediction techniques when used to predict queue wait times (and perhaps other properties). This is novel work and should result in one or more publications.

The Karnak service is available to XSEDE and is being used by XSEDE users. Any improvements to the service (accuracy, reliability, response time) will benefit scientists using XSEDE. The XSEDE Karnak service is available through the XSEDE portal and can also be accessed directly by users and science gateways.

Project Contact

Project Lead

Warren Smith (wsmith) 

Project Manager

Warren Smith (wsmith) 

Resource Requirements

Hardware System

• alamo (Dell optiplex at TACC)

 

I will test alternative implementations of the Karnak service using virtual machines running on FutureGrid. I will optimize the configuration of Karnak prediction techniques by submitting batch jobs to HPC nodes that analyze how different configurations would have performed on historical data.

While evaluating implementations, I expect to run a single virtual machine at a time and will run that virtual machine during business hours a few days a week for approximately 3 months. To optimize prediction techniques, I expect to submit HPC jobs that use approximately 48 node hours every week for approximately 6 months.

Project Timeline

Submitted

09/13/2011 - 20:58