Resource Definition Files

<PENDING: complete>

Overview

Resource Definition Files inform DOGMA about resources available for its use.  Examples of resources include nodes and applications.

Informing DOGMA of nodes available for its use

The first step in adding a node is to make sure DOGMA is installed correctly on this node (naturally, nodes with a shared filesystem only need one representative node tested.)  Do this by starting up a configuration manager on a master node by typing startDOGMA on the master (the master node should probably not be the node which you are adding as a slave node).  Next go to the Dogma/bin directory on the slave and type startNode <local hostname> <configuration manager hostname>.

If all has worked correctly, the next step is to ensure that you can start the node remotely from the master node.  Do this using your favorite method (e.g. rsh /usr/local/jdoe/Dogma/bin/startNode myslave.foo.bar mymaster.foo.bar).

Next add the node to the resource definition file.  Every node must be defined as both the member of a cluster and a family.  A node's family describes characteristics of the node, while a node's cluster defines which group of nodes it logically belongs to.  The node's family is where the remote starting command is put.  So, nodes which require different startCommands must be put into different families.  See below for the details on the cluster, family, and startCommand format.  (Be sure to use the variables NODE_NAME and CM_NAME when entering startCommands.)

Adding an application to a resource definition file

In order for DOGMA to run an application, it must first be defined in a resource definition file.  The application entry in the resource definition file describes what type of application it use (MPIJ or Dynamic Object Group), what "category" the application is in (where the user will find it to run it), the requirements of the application (e.g. number of nodes required to run the application), and the parameters the application will take.  See the application entry below for the exact format of an application.  Note that an application's applicationReq field must refer to an applicationReq defined in the same resource definition file.

Resource Definition File Format

Resource Definition Files may contain information about several types of resources as shown below by category:

Node Information

cluster
family
nodeContract

Applications

application
applicationReq
nativeApp
parameter

Misc.

aliasList
 

Note that all bolded attributes are required while all non-bolded attributes are optional.


Example resource definition file <pending>


Attribute values notation:

"choice1" | "choice2" Indicates that either choice1 or choice2 should be chosen.  The chosen item should be quoted.
choice1 | choice2 Indicates that either choice1 or choice2 should be chosen.  The chosen item should be unquoted.
<value> Indicates that value should be replaced value with an unquoted value.
<value> Indicates that value should be replaced by a quoted value.


application
Purpose:
    Defines an application which may be run.

Attributes:
    type: "MPIJ" | "dynamic"
    category: <"category1"> <"category2"> ...
    codebase: <"URL(default is DOGMA_HOME/lib on Configuration Manager)">
    applicationReq: <applicationReq name>
    parameters: <parameter2> <parameter2> ...

Example:
application dogma.examples.mpij.Jacobi {
    type       "MPIJ";
    category   "Examples" "MPIJ";
    codebase   "http://myhost.foo.bar/examples/";
    applicationReq  exactlyFourNodes;
    parameters      size;
}


parameter
Purpose:
    Defines a parameter which can be passed to an application when the application is started.

Notes:
    The type of dialog used to input values for this parameter is dependent on the attributes defined.  Parameters with a file attribute will use a file chooser dialogue to enter values.  Parameters with a choices attribute will use a pull down list to select from among those choices.  Parameters without choices or file attributes will use a text field to enter values.  Note that the choices and file attributes are mutually exclusive.

Attributes:
    optional: true | false
    defaultValue: <"default value">
    description: <"description">
    choices <"choice1"> <"choice2"> ...
    file: <"filename">

Examples:

parameter animal {
    description "Animal Name";
    choices "Cow" "Dog" "Cat";
    defaultValue "Cow";
    optional false;
}

parameter end {
    description "End";
    defaultValue "20";
    optional false;
}

parameter InputFile {
    description  "Input File";
    optional     false;
    file         "myDir/Input.txt";
}
 


applicationReq
Purpose:
    Describes requirements of an application which must be met in order for an application to execute.

Notes:
    generationSize and numNodes are mutually exclusive.  nodeContract indicates a contract which contains the minumum duration contract which must be met in order to use a given node.  nativeApps lists the names of applications which must be installed on nodes before this application can execute.  Nodes which do not have a required native application will not be used unless they allow native application installation.  If they allow native appliation installation, the application will be installed if it is not found.  See nativeApp for more information.

Attributes:
    generationSize: <generation 1 size> <generation 2 size> ...
    numNodes: <min> <max>
    nodeContract: <contract name>
    nativeApps:    <myNativeApp> <myOtherNativeApp>

Examples:

applicationReq dynamic1GReq {
    generationSize 150 150;
}

applicationReq Exactly2Nodes {
    numNodes 2 2 2;
    nodeContract    permNodes;
    nativeApps    myNativeApp;
}
 


nativeApp
Purpose:
    Defines a native application.  An application which uses native applications specifies which applications it uses in the nativeApps attribute of the application's applicationReq.  The system administrator defines which nodes have this application by including it in the nativeApps field of a cluster.  The system administrator may also allow nodes to install applications which are not already on the node (see cluster for more information).  If application installation is allowed, applications may list which files constitute the application and the codebase (URL) from  which the application may be read.

Notes:
    nativeApp may be used to define data files which are required by an application.
    Applications do not need define their codebase and filenames.  They can be used simply to designate a particular application.  The codebase and fileNames attributes are used when application installation is allowed.  If application installation is allowed, and codebase and fileNames are set, all application files will be checked to see if they have changed size.  If the size that is installed and the size of the file at the codebase differ, the installed file will be replaced with a copy of the file at the codebase.
 
Attributes:
    codebase: <"codebase">
    fileNames: <"file1"> <"file2"> ...

Examples:
nativeApp myApp {
    codebase "http://foo.bar.com/myAppLoc/";
    fileNames "bin/myApp" "data/data1.txt" "data/data2.txt";
}
 


nodeContract
Purpose:
When nodes join DOGMA, they sign a contract indicating how long they will be part of DOGMA.  (In the future this contract will include additional agreements between the node and DOGMA.)  Applications may specify requirements of nodeContracts which must be met by nodes which run the application.  The scheduler will schedule applications only on nodes which meet the requirements of the contract specified in the applications application requirements.

So, nodeContracts are used in two ways.  When they are referenced from a family definition, they indicate a contract which will be fulfilled by all nodes in the family.  When they are referenced from applicationReq definitions, they refer to a minimum contract which must be met by nodes.  Currently the only defined attribute of the contract is the duration which a node will be part of DOGMA.  In this case "volatile" indicates a node which may leave at any time.  "limited" indicates a node which will be part of DOGMA for a specific amount of time <pending: this attribute is not yet supported in 0.9>.  "permanent" indicates a node which will never be shut down during the middle of an application.

Note that nodes which are defined in the resources file are permanent by default.  Anonymous nodes which join DOGMA are volatile by default.

Notes:

Attributes:
    duration:     volatile | limited | permanent;

Example:

nodeContract {
    duration permanent;
}
 


aliasList
Purpose:
    Defines a list of aliases for DOGMA's http server.  Incoming requests which begin with the alias are modified to look in the designated location.

Notes:
    file and URL are mutually exclusive.

Attributes:
    file: <"alias"> <"actual">
    URL: <"alias"> <"actual">

Example:

aliasList {
    file "DOGMA/codebase/" "c:\\DMaster/lib/";
}


cluster
Purpose:
    Defines which nodes belong to a particular cluster.

Notes:
    The nativeApps attribute defines the names of native applications which are installed on this cluster (see nativeApp for more information).  The allowsNativeAppInstallation attribute defines whether or not new native applications are allowed to be installed on this node.  Proper security policy files must be in place before enabling this feature.  FilePermission should only be granted only to trusted codebases.

Attributes:
    nodes <node1> <node2> ...
    nativeApps  <app1> <app2> ...
    allowsNativeAppInstallation true | false

Example:
cluster Mountains {
    nodes
        everest.cs.byu.edu
        k2.cs.byu.edu;
    nativeApps    myNativeApp myOtherNativeApp;
    allowsNativeAppInstallation    true;
}


family
Purpose:
    Defines which nodes belong to a particular family.  Also defines the command used to start those nodes.

Notes:
    The startCommand is the command which will be used to start the node.  It is highly recommended that this command be tested manually first.  Also note that output from the startCommand is echoed so that its progress may be monitored.
    startCommand supports two variables: NODE_NAME and CM_NAME.  These variables will be replaced by the remote node name and the configuration manager name respectively.
    nodeContract indicates a contract which will be signed by every node in this family.

Attributes:
    nodes <node1> <node2> ...
    startCommand <command> <arg1> <arg2> ...
    nodeContract    <nodeContract name>

Example:
cluster Mountains {
    nodes
        everest.cs.byu.edu
        k2.cs.byu.edu;
    startCommand c:\winnt\system32\rsh.exe NODE_NAME
        "/usr/john/Dogma/bin/startNode"
        NODE_NAME CM_NAME;
}