Hygra-Sim: Simulation of a Decentralized protocol for Resource discovery and job Allocation in large computational grids

Project Information

Discipline

Computer Science (401) 

Subdiscipline

11.07 Computer Science 

Orientation

Research 

Hygra (HYperspace-based Grid Resource Allocation) represents a novel approach for resource finding and job allocation in a computational Grid. The technique proposed organises Grid nodes in a peer-to-peer architecture by building an overlay network in which each node is virtually connected to one or more neighbours. HYGRA is strongly based on spatial computing concepts, as the available amount of each resource type is used as a geometric coordinate in a virtual Euclidean hyperspace. Each Grid node is represented as a point in the hyperspace, on the basis of the amount of its free resources. A job request is also represented as a point in this hyperspace, according to the resource amounts needed for job execution. Thus, in this model, finding a Grid node suitable for execution of a job request means navigating the said resource hyperspace. Node search goes from neighbour to neighbour, following the overlay links trying to minimize the Euclidean distance between the current node and the target point representing the job, until the node nearest to it is found. In order to evaluate the proposed approach, we built a software simulator in C language. The simulator makes use of POSIX threads, thus taking benefit from multicore environment. We need to submit a great number of simulations in order to evaluate the emerging properties of the clustering algorithm as the decentralized resource finding algorithm.

The main intellectual merit in this research is the experimental proof that the adoption of a simple p2p organization in a wide-distributed set of heterogeneous resources is possible and event more efficient than the hierarchical approach, like MDS. Our main proposal is twofold. First of all we want to prove that the mapping of each logical container of resources in a virtual Euclidean space and the proposed organizational pattern both represent a key point for any discovering algorithm which makes use of (only) local information (i.e. information of the neighbours). Moreover we want to show that our clustering algorithm is able to build a scale-free network, which is the main organizational pattern in any natural system and social network.

If the results of this experimental campaign confirm our main hypothesis, then the traditional indexing systems would become a secondary concern and no longer they will represent critical points of failure. The adoption of the proposed organization schema and the search algorithms can be adopted both inside big centralized systems (i.e. cloud systems) that across the sites of wide distributed Grid systems.

Project Contact

Project Lead

Fabrizio Messina (fmessina) 

Project Manager

Fabrizio Messina (fmessina) 

Resource Requirements

Hardware System

• I don't care (what I really need is a software environment and I don't care where it runs)

 

I will periodically submit a bag of jobs in a virtual linux cluster. Each bag is generally composed by about a thousound of simulations. The simulations are cpu-intensive but doesn't produce big amount of data.

I need about 100 core provided with at least 512mb of memory per core.

Project Timeline

Submitted

10/25/2011 - 12:56