-- AndrewCudd - 26 Feb 2015

Using dCache / dccp for file transfer

This page contains instructions on how to install and use dCache / dccp to facilitate file transfers. These instructions are specifically tailored for accessing the t2ksrm.nd280.org server at TRIUMF. The commands and install were performed on an install of Scientific Linux 7 (SL7), which is a Red Hat 7 based system, using a BASH shell. Adjust your commands accordingly if you are using a different system / shell.

If you are working on the HPCC, dccp is already installed for T2K group use, so you can skip to the section on how to use dccp, following the setup below:

user@host $ cd /mnt/research/T2K
user@host $ source nd280Software/nd280setup.sh


Installing dccp

To install the dccp command you need to install a package called dcap. There are a few ways to install it which mainly depend on the permissions you have available on the machine you are installing on. The easiest way to install dcap is to use the package manager. For SL7 (or other Red Hat based systems) you can use YUM to install dcap as follows:
user@host $ yum install dcap

If YUM complains about not finding the package then either you probably do not have the correct repository loaded or the exact name of the dcap package may be slightly different. For rpm based systems the EPEL repository contains the packages. Note that installing via YUM will probably need root permissions.

The next way to install the dcap package is through installing a rpm (for Red Hat based systems) or deb (for Debian based systems) package. You can download the proper package for your system from here: http://www.dcache.org/downloads/dcap/. The packages you want are the dCap client and the dCap libs, and the version I have installed from the time of writing is dcap-2.47.8. Note to grab the correct files for your operating system architecture. To install the rpm packages, use the following command and make sure to install the dCap libs first:
user@host $ rpm -i /path/to/package.rpm

Installing using a rpm (or deb) package also may require root permisions to work correctly.

Finally you can also install the dcap package from source. The source code can also be found on the previously linked download page as the dcap-2.47.8-1.el6.src file. Unfortunately the source code is packaged in a source rpm file instead of a tar file, which will require an extra step and extra utilities to achieve what we want. The source rpm file contains the tar file that we want, so we need to extract the tar file from the source rpm using a couple commands as follows (be sure to enter them carefully and use the correct name of the src.rpm):
user@host $ rpm2cpio dcap-2.47.8-1.el6.src.rpm | cpio -idmv

Three files should have been extracted from the source rpm including the tar file; the other two files are not needed. Untar the tar file with the following:
user@host $ tar -zxvf 2.47.8.tar.gz

Now cd into the newly created directory and run the following commands:
user@host $ sh bootstrap.sh
user@host $ ./configure --prefix=/path/to/install
user@host $ make
user@host $ make install

That should have now installed everything to run dccp in the directory that you chose in the configure step. If the install did not work or you are having trouble with one or more parts, then I might be able to help. Now that everything is installed, I suggest adding the install directory to your $PATH variable for ease of use.


Running dccp to copy files

Using dccp to copy files is actually quite simple since it functions very similar to the cp(1) command in unix. The dccp command also has no issues with security certificates which makes its use much easier than running globus-url-copy. To copy a file you need to supply dccp a source and destination argument as follows:
user@host $ dccp <source_url> <destination_url>

Where the source is the file to be copied and the destination is the place to copy it to. If the transfer fails, try adding the -A flag (it alters how dccp treats firewalls). For example to copy a file from the t2ksrm.nd280.org server at TRIUMF to a directory the command would look like something this:
user@host $ dccp dcap://t2ksrm.nd280.org/pnfs/nd280.org/data/nd280data/production005/F/mcp/neut/2010-11-air/magnet/beamc/anal/oa_nt_beam_90300000-0000_6rk2dnkozo5s_anal_000_prod005magnet201011airc-bsdv01.root /mnt/research/T2K/oaAnalysisFiles/test.root

Which will copy the file oa_nt_beam_90300000-0000_6rk2dnkozo5s_anal_000_prod005magnet201011airc-bsdv01.root to /mnt/research/T2K/oaAnalysisFiles/ and name it test.root.

More information about the dccp command can be found on its manual page: http://www.dcache.org/manuals/dccp.shtml


Topic revision: r5 - 08 May 2015, AndrewCudd
 

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback