Description of the storage areas available to users on the HEP Linux cluster. Please feel free to contact Tom Rockwell if you have questions about storage, there are multiple options available.
A trade-off is made between data protection and storage area size --- doing backups etc. places limits on practical storage area size.
For important files, consider making copies outside the cluster. Some options are:
Each user's home area is stored on a central server and available on any cluster node. This area is backed up to tape weekly. The size of this area is constrained due to the limits of the backup tapes (need to keep total for all users below 200GB).
Quotas are set on this area with individual users allocated 2-6GB.
Use this area for files that are most important to retain and difficult to reproduce. Examples are: source code, documents you create, etc.
stands for Redundant Array of Inexpensive Disks. By combining multiple harddrives and read array protects the stored data against the failure of individual harddrives. This increases reliability, but doesn't eliminate all potential for data loss.
Large (multi-terabyte size) RAID storage is available that is attached to a central server. This is not
backed up to tape. Use it for storing data files, etc. that can be reproduced.
Some of the cluster nodes host "work" areas that are available on other cluster nodes. In general, we are moving away from this method as the management overhead is higher. Some of these areas are just single harddrives (so a failure of the disk would result in data loss), and some area RAID, but these areas are statistically less reliable that the centrally hosted RAID areas.
Most newer desktop nodes have a directory named /scratch. These are not shared between the nodes (so /scratch on one node is not the same area as /scratch on another). These areas can be used as temporary storage.
- 24 Feb 2011