EMI dCache Tutorial

What is EMI?

European Middleware Initiative (EMI) is a software platform for high performance distributed computing.

It is at the core of grid middleware distributions used by scientific research communities and distributed computing infrastructures all over the world including WLCG--the Worldwide LHC Computing Grid--which supports, for example, the search for the Higgs boson and new types of matter searches of the physicists at LHC, together with other large scientific challenges in astronomy, biology, computational chemistry and other sciences.

Being a close collaboration among well-established grid middleware providers and other specialized software providers, EMI proposes itself as a leading platform for scientific grid computing and looks at expanding outside of its natural environment.

EMI on FutureGrid

EMI have created a number of Virtual Appliances which run various EMI services. This tutorial covers the EMI dCache tools.

The emi2-dcache virtual appliance is available on the following resources

• Sierra
• Hotel
• Foxtrot

dCache provides a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods. The emi2-dcache appliance contains the dCache server and client packages installed on a single node with limited storage. Thus it is made available as a demonstration and training resource only.

Prerequisites

The emi2-dcache appliance supports various access protocols, including srm, dcap and WebDav. It uses x509 certificates with VOMS attributes for authentication and authorisation, thus you will need to obtain an x509 certificate before you can use the virtual appliance. Currently the GILDA Virtual Organisation (VO) is the only one supported on this virtual appliance. GILDA, the Grid INFN Laboratory for Demonstration ..., maintains a training Certificate Authority (CA) which issues short-lived credentials for training purposes. To obtain a certificate, visit http://gilda.ct.infn.it/.

Once your certificate has been issued you will be sent an email by GILDA with instructions on how to download the certificate to your browser. After downloading the certificate you must export or back it up to a file. This can be done through the Manage Certificates function of your browser. For example, in Google Chrome you can find this under Settings - Show advanced settings - HTTPS/SSL - Manage Certificates. In Firefox you can find it under Edit - Preferences - Advanced - Encryption - View Certificates.

Launching the dCache appliance

To launch the emi-unicore VM you must use the Nimbus cloud-client tool. For instructions on how to install and configure Nimbus please see the Nimbus tutorial.

Once you have a working Nimbus client you can search for the emi2-dcache appliance using the following command

$ bin/cloud-client.sh --list

You should see a the emi2-dcache appliance in the list. To instantiate an emi2-dcache appliance use the following command

$ bin/cloud-client.sh --run --name <VM appliance name> --hours <number of hours>

For example, if the emi2-dcache appliance is called emi2-dcache-centos-5.7-x64.gz and you want to run it for three hours, use the following command

$ bin/cloud-client.sh --run --name emi2-dcache-centos-5.7-x64.gz --hours 3

Once the VM is running you should have a working dCache server. You can log in as root using your ssh key and begin using the dCache services.

 

Authenticating to dCache

Once you have launched your emi2-dcache virtual appliance you can access it by logging in as root with your ssh key. To authenticate yourself to dCache, however, you must create a VOMS proxy certificate. This you can only do as a normal (non-root) user. A user called emiuser has been created for you on the virtual appliance.

Copy the GILDA certificate file which you exported from your browser earlier to the /home/emiuser directory, changing the ownership to emiuser:

$ scp mycertbackup.p12 root@myhost.futuregrid.org:
$ ssh root@myhost.futuregrid.org
$ cp mycertbackup.p12 /home/emiuser
$ chown emiuser:emiuser /home/emiuser/mycertbackup.p12

Next you should su to the emiuser and extract your key and certificate from your backup file. These must be saved in the directory .globus in the emiuser's home directory and their permissions set as shown below.

$ su - emiuser
$ mv .globus .globus_ORIGINAL
$ mkdir .globus
$ openssl pkcs12 -nocerts -in mycertbackup.p12 -out .globus/userkey.pem
$ openssl pkcs12 -clcerts -nokeys -in mycertbackup.p12 -out .globus/usercert.pem
$ chmod 0444 .globus/usercert.pem
$ chmod 0400 .globus/userkey.pem

You are now ready to create your VOMS proxy credentials which will allow you to access the dCache services. To create a VOMS proxy run the following command:   $ voms-proxy-init --voms gilda

You can view information about your proxy with the command

$ voms-proxy-info


When you run the dCache commands they will use this proxy certificate to check whether you have permission to use the services and resources. All members of the GILDA VO are permitted to use the services on the emi2-dcache virtual appliance.

Using your dCache System

You should now be ready to get started with the dCache client tools. First you will need to create a test file in your home directory.

$ echo "Hello World" > test.txt

Next use dcap to copy this file to the storage resource

$ dccp -H test.txt dcap://myhost.futuregrid.org/data/world-writable/my-test-file-1

If this worked you should be able to view your file, for example, using the srmls command:

$ srmls -2 srm://myhost.futuregrid.org:8443/data/world-writable/

You can also create and delete files using the srm commands:

$ srmcp -2 file://////home/emiuser/test.txt srm://myhost.futuregrid.org:8443/data/world-writable/my-test-file-2
$ srmrm srm://myhost.futuregrid.org:8443/data/world-writable/my-test-file-2

You should also be able to interact with your files using WebDav, for example in a browser by pointing it to the url of your host: http://myhost.uc.futuregrid.org:2880/