You are here: TWiki > Integration/ITB090 Web>ReleaseDocumentationDocumentationTable?>StorageElementAdmins (04 Jun 2010, SuchandraThapa)

Storage Element Admins

This page contains how-to guides and other necessary information for setting up a Storage Element (SE) or testing a Storage Element Implementation.

For information about CE-Storages ($DATA, ...) and their configuration, please refer to Local Storage Configuration.

Setup of Storage Element (SE)

Is this page needed? Maybe just for the gridftp-based thing and maybe srm-tester? The other setups are just links.

How to Setup SE based on Gridftp-based File System

The easiest way to install a Gridftp server with all the required certificates is using VDT (or even the whole OSG package). There is no special requirement about the directory: it has to be a valid path for the Gridftp server and have the protections set to provide the desired access to Grid users. Sometime sticky (only the creator of a file can remove it) or setgroup (group is maintained in subdirectories) ids may be useful.

Making a Gridftp-based File System that is also a CE-Storage (like OSG_DATA) involves creating a data directory and mounting it on all worker nodes so the storage is accessible to applications running on that site via some environment variables. The SE (host, port, path) has then to point to the same data directory.

To make a concrete example let's say that you have:

  • a host where you have a disk space that you want to use as SE and a Gridftp server installed using VDT (this host may be the gatekeeper or a separate host): ftpserver.domain.
  • the directory on ftpserver.domain that you want to use as SE is /storage/root/dir/
  • on the worker nodes you want to have it visible as /share/osgdata/

Then you have to set accordingly these values in configure-osg:

  • OSG_DATA=/share/osgdata/
  • DEFAULT_SE=gsiftp://ftpserver.domain/storage/root/dir/

And you have to:

  • create the directory in ftpserver.domain, make it worls writable with sticky flag set
  • configure ftpserver.domain to export the directory, e.g. using NFS
  • configure all workernodes and the gatekeeper (or wherever you run your fork jobs) to mount that directory as /share/osgdata/

E.g. To obtain the configuration above you can ececute these commands on ftpserver.domain as root:

mkdir -p /storage/root/dir
chmod a+rwx /storage/root/dir
chmod o+t /storage/root/dir

Then in /etc/exports on ftpserver.domain you'll have a line like (the IPs depend on your subnet): /storage/root/dir 10.0.0.0/255.0.0.0(rw,no_root_squash)

If you have NFS static mounts (the alternative is to use automount), in /etc/fstab of the gatekeeper and all workernodes you'll have an entry like: ftpserver:/storage/root/dir        /share/osgdata                 nfs     bg,intr,noatime

Example is very specific

How to Setup SE based on SRM/dCache

SRM/dCache Installation and Configuration using the VDT-dCache package.

How to Setup SE based on BeSTMAN??!

Need a link here

How to Setup Generic Information Provider (GIP)

To configure this component, see the Generic Information Providers document in this guide.

How to Configure Compute Element Storage

See Configure "CE Storage"

How to Test SE Implementation

SRM-TESTER is available to test SRM based storage sites as well as popular file transfer protocol such as gsiftp, http or ftp, based storage sites. SRM-TESTER is available as a stand-alone tester. SRM-TESTER can be installed through VDT pacman, or downloadable from http://sdm.lbl.gov/srm-dist.

The SRM Client Tools are an optional module in the OSG Cache. They are supported by the SRM-dCACHE team: srm@lbl.gov.

More information

DON'T NEED HERE

  • GlueOSG? - Glue Schema description


Complete: 1
Responsible: TedHesselroth - 17 Oct 2007
Reviewer - date: RobGardner - 08 Nov 2007
Comment: Update?Delete? RobGardner

Topic revision: r22 - 04 Jun 2010 - 18:34:37 - SuchandraThapa
 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..