Storage

Description of the storage areas available to users on the HEP Linux cluster. Please feel free to contact Tom Rockwell if you have questions about storage, there are multiple options available.

A trade-off is made between data protection and storage area size --- doing backups etc. places limits on practical storage area size.

Alternatives

For important files, consider making copies outside the cluster. Some options are:

Home area

Each user's home area is stored on a central server and available on any cluster node. This area is backed up to tape weekly. The size of this area is constrained due to the limits of the backup tapes (need to keep total for all users below 200GB).

Quotas are set on this area with individual users allocated 2-6GB.

Use this area for files that are most important to retain and difficult to reproduce. Examples are: source code, documents you create, etc.

RAID areas

RAID stands for Redundant Array of Inexpensive Disks. By combining multiple harddrives and read array protects the stored data against the failure of individual harddrives. This increases reliability, but doesn't eliminate all potential for data loss.

Large (multi-terabyte size) RAID storage is available that is attached to a central server. This is not backed up to tape. Use it for storing data files, etc. that can be reproduced.

Location Size RAID
/ work/raida 8.9TB Yes RAID6

Work areas

Some of the cluster nodes host "work" areas that are available on other cluster nodes. In general, we are moving away from this method as the management overhead is higher. Some of these areas are just single harddrives (so a failure of the disk would result in data loss), and some area RAID, but these areas are statistically less reliable that the centrally hosted RAID areas.

Location Size RAID

Scratch area

Most newer desktop nodes have a directory named /scratch. These are not shared between the nodes (so /scratch on one node is not the same area as /scratch on another). These areas can be used as temporary storage.

-- TomRockwell - 24 Feb 2011
Topic revision: r2 - 24 Feb 2011, TomRockwell
 

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback