Install cvmfs

1 About this Document

Here we describe how to install the cvmfs (Cern-VM file system) client. This document is intended for system administrators who wish to install this client to have access to files distributed by cvmfs servers via HTTP.

2 Applicable Versions

This document addresses installation instructions for the latest version of cvmfs, which is only available in OSG 3.3 (supported OS versions: EL6 and EL7). OSG 3.2 (supported OS versions: EL5 and EL6) only receives security updates so it does not have the latest cvmfs version. The installation instructions for the older version are the same, but the verification results will be different as noted below.

3 Requirements

3.1 Host and OS

  • OS is Red Hat Enterprise Linux 6, 7, and variants (see details...). See section about applicable versions above for differences in supported EL versoins.
  • Root access
  • autofs should be installed
  • fuse should be installed (or will be as part of the installation)
  • Sufficient (~20GB+20%) cache space reserved, preferably in a separate filesystem (details below)

3.2 Users and Groups

This installation will create one user unless it already exists:

User Comment
cvmfs CernVM-FS service account

The installation will also create a cvmfs group and default the cvmfs user to that group. In addition, if the fuse rpm is not for some reason already installed, installing cvmfs will also install fuse and that will create another group:

Group Comment Group members
cvmfs CernVM-FS service account none
fuse FUSE service account cvmfs

3.3 Networking

You will need network access to a local squid server such as the squid distributed by OSG. The squid will need out-bound access to cvmfs stratum 1 servers.

3.4 Upgrading

When upgrading from a cvmfs version older than 2.1.20, delete the setting of CVMFS_SERVER_URL in /etc/cvmfs/domain.d/cern.ch.local. If that's the only thing in the file (which is likely) then delete the whole file.

4 Install Instructions

Install the Yum Repositories required by OSG

The OSG RPMs currently support Red Hat Enterprise Linux 6, 7, and variants (see details...).

OSG RPMs are distributed via the OSG yum repositories. Some packages depend on packages distributed via the EPEL repositories. So both repositories must be enabled.

Install EPEL

  • Install the EPEL repository, if not already present. Note: This enables EPEL by default. Choose the right version to match your OS version.
    # EPEL 6 (For RHEL 6, CentOS 6, and SL 6) 
    [root@client ~]$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
    # EPEL 7 (For RHEL 7, CentOS 7, and SL 7) 
    [root@client ~]$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    WARNING: if you have your own mirror or configuration of the EPEL repository, you MUST verify that the OSG repository has a better yum priority than EPEL (details). Otherwise, you will have strange dependency resolution (depsolving) issues.

Install the Yum priorities package

For packages that exist in both OSG and EPEL repositories, it is important to prefer the OSG ones or else OSG software installs may fail. Installing the Yum priorities package enables the repository priority system to work.

  1. Install the Yum priorities package:

    [root@client ~]$ yum install yum-plugin-priorities
  2. Ensure that /etc/yum.conf has the following line in the [main] section (particularly when using ROCKS), thereby enabling Yum plugins, including the priorities one:

    plugins=1
    NOTE: If you do not have a required key you can force the installation using --nogpgcheck; e.g., yum install --nogpgcheck yum-priorities.

Install OSG Repositories

  1. If you are upgrading from one OSG series to another, remove the old OSG repository definition files and clean the Yum cache:

    [root@client ~]$ yum clean all
    [root@client ~]$ rpm -e osg-release

    This step ensures that local changes to *.repo files will not block the installation of the new OSG repositories. After this step, *.repo files that have been changed will exist in /etc/yum.repos.d/ with the *.rpmsave extension. After installing the new OSG repositories (the next step) you may want to apply any changes made in the *.rpmsave files to the new *.repo files.

  2. Install the OSG repositories:

    [root@client ~]$ rpm -Uvh URL

    Where URL is one of the following:

    Series EL6 URL (for RHEL 6, CentOS 6, or SL 6) EL7 URL (for RHEL 7, CentOS 7, or SL 7)
    OSG 3.3 https://repo.grid.iu.edu/osg/3.3/osg-3.3-el6-release-latest.rpm https://repo.grid.iu.edu/osg/3.3/osg-3.3-el7-release-latest.rpm
    OSG 3.4 https://repo.grid.iu.edu/osg/3.4/osg-3.4-el6-release-latest.rpm https://repo.grid.iu.edu/osg/3.4/osg-3.4-el7-release-latest.rpm

For more details, please see our yum repository documentation.

4.1 Installing cvmfs

The following will install cvmfs from the OSG yum repository. It will also install cern public keys as well as fuse and autofs if you do not have them, and it will install the configuration for the OSG cvmfs distribution which is called OASIS.

[root@client ~]$ yum install osg-oasis

4.2 Setup of fuse and automount

Create or edit /etc/fuse.conf. It should contain the following in order to allow fuse to do proper file ownership:

user_allow_other

Create or edit /etc/auto.master. It should contain the following in order to allow cvmfs to automount:

/cvmfs /etc/auto.cvmfs

Restart autofs to make the change take effect:

[root@client ~]$ service autofs restart
Stopping automount:                       [  OK  ]
Starting automount:                       [  OK  ]

4.3 Configuring cvmfs

Create or edit /etc/cvmfs/default.local, a file that controls the cvmfs configuration. Below is a sample configuration, but please note that you will need to edit the parts in red. In particular, the CVMFS_HTTP_PROXY line below must be edited for your site.

CVMFS_REPOSITORIES="`echo $((echo oasis.opensciencegrid.org;echo cms.cern.ch;ls /cvmfs)|sort -u)|tr ' ' ,`"
CVMFS_QUOTA_LIMIT=20000
CVMFS_HTTP_PROXY="http://squid.example.com:3128"

CVMFS by default allows any repository to be mounted. The recommended CVMFS_REPOSITORIES setting is what it is above so that tools such as cvmfs_config and cvmfs_talk that use known repositories will use two common repositories plus any additional that have been mounted. You may want to choose a different set of always-known repositories. A full list of cern.ch repositories is found at http://cernvm.cern.ch/portal/cvmfs/examples.

Set up a list of cvmfs HTTP proxies to retrieve from in CVMFS_HTTP_PROXY. If you do not have any squid at your site follow the instructions to install squid from OSG. Vertical bars separating proxies means to load balance between them and try them all before continuing. A semicolon between proxies means to try that one only after the previous ones have failed. A special proxy called DIRECT can be placed last in the list to indicate directly connecting to servers if all other proxies fail. This is acceptable for small sites but discouraged for large sites because of the potential load that could be put upon the stratum one servers.

Set up the cache limit in CVMFS_QUOTA_LIMIT (in MB). The recommended value for most applications is 20000 MB. This is the combined limit for all but the osgstorage.org repositories. This cache will be stored in /var/lib/cvmfs by default; to override the location, set CVMFS_CACHE_BASE in /etc/cvmfs/default.local. Note that an additional 1000 MB is allocated for a separate osgstorage.org repositories cache in $CVMFS_CACHE_BASE/osgstorage. To be safe, make sure that at least 20% more than $CVMFS_QUOTA_LIMIT + 1000 MB of space stays available for cvmfs in that filesystem. This is very important, since if that space is not available it can cause many I/O errors and application crashes. Many system administrators choose to put the cache space in a separate filesystem, which is a good way to manage it.

ALERT! WARNING!
If you use SELinux and change CVMFS_CACHE_BASE, then the new cache directory must be labelled with SELinux type cvmfs_cache_t. This can be done by executing the following command:
[user@client ~]$ chcon -R -t cvmfs_cache_t $CVMFS_CACHE_BASE

5 Verifying cvmfs

After cvmfs is installed, you should be able to see the /cvmfs directory. But note that it will initially appear to be empty:

[user@client ~]$ ls /cvmfs
[user@client ~]$

Directories within /cvmfs will not be mounted until you examine them. For instance:

[user@client ~]$ ls /cvmfs
[user@client ~]$ ls -l /cvmfs/atlas.cern.ch
total 1
drwxr-xr-x 7 cvmfs cvmfs 3 Jul  7  2015 repo
[user@client ~]$ ls -l /cvmfs/oasis.opensciencegrid.org/cmssoft
total 1
lrwxrwxrwx 1 cvmfs cvmfs 18 May 13  2015 cms -> /cvmfs/cms.cern.ch
[user@client ~]$ ls -l /cvmfs/glast.egi.eu
total 5
drwxr-xr-x 9 cvmfs cvmfs 4096 Feb  7  2014 glast
[user@client ~]$ ls -l /cvmfs/nova.osgstorage.org
total 6
lrwxrwxrwx 1 cvmfs cvmfs   32 Jan 19 11:40 flux -> pnfs/fnal.gov/usr/nova/data/flux
-rw-r--r-- 1 cvmfs cvmfs   49 Jan 19 11:38 new_repository
drwxr-xr-x 3 cvmfs cvmfs 4096 Jan 19 11:39 pnfs
[user@client ~]$ ls /cvmfs
atlas.cern.ch  config-osg.opensciencegrid.org  nova.osgstorage.org
cms.cern.ch    glast.egi.eu                    oasis.opensciencegrid.org

HELP NOTE
cvmfs client versions older than 2.2.0 (for example in OSG 3.2) will not mount the config-osg.opensciencegrid.org or nova.osgstorage.org repositories.

5.1 Troubleshooting problems

If no directories exist under /cvmfs/, you can try the following steps to debug:

  • Mount it manually mkdir /mnt/cvmfs then mount -t cvmfs REPOSITORYNAME /mnt/cvmfs where REPOSITORYNAME is the repository, for example config-osg.opensciencegrid.org (this is the best one to try first because other repositories require it to be mounted). If this works, then cvmfs is working, but there is a problem with automount.

  • If that doesn't work and doesn't give any explanatory errors, try cvmfs_config chksetup or cvmfs_config showconfig REPOSITORYNAME to verify your setup.

  • If chksetup reports access problems to proxies, it may be caused by access control settings in the squids.

  • If you have changed settings in /etc/cvmfs/default.local, and they do not seem to be taking effect, note that there are other configuration files that can override the settings. See the comments at the beginning of /etc/cvmfs/default.conf regarding the order in which configuration files are evaluated and look for old files that may have been left from a previous installation.

  • More things to try are in the upstream documentation.

6 Starting and Stopping services

Once it is set up, cvmfs is always automatically started when one of the repositories are accessed.

cvmfs can be stopped via:

[root@client ~]$  cvmfs_config umount
Unmounting /cvmfs/config-osg.opensciencegrid.org: OK
Unmounting /cvmfs/atlas.cern.ch: OK
Unmounting /cvmfs/oasis.opensciencegrid.org: OK
Unmounting /cvmfs/cms.cern.ch: OK
Unmounting /cvmfs/glast.egi.eu: OK
Unmounting /cvmfs/nova.osgstorage.org: OK

7 Screendump of Install

[root@fermicloud123 ~]# rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
Retrieving http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
Preparing...                ########################################### [100%]
   1:epel-release           ########################################### [100%]
[root@fermicloud123 ~]# yum install yum-plugin-priorities
Loaded plugins: priorities, security
Setting up Install Process
epel/metalink                                            | 9.0 kB     00:00     
epel                                                     | 4.3 kB     00:00     
epel/primary_db                                          | 5.9 MB     00:00     
688 packages excluded due to repository priority protections
Package yum-plugin-priorities-1.1.30-30.el6.noarch already installed and latest version
Nothing to do
[root@fermicloud123 ~]# grep plugins /etc/yum.conf
plugins=1
[root@fermicloud123 ~]# rpm -Uvh https://repo.grid.iu.edu/osg/3.3/osg-3.3-el6-release-latest.rpm
Retrieving https://repo.grid.iu.edu/osg/3.3/osg-3.3-el6-release-latest.rpm
warning: /var/tmp/rpm-tmp.yOhxeG: Header V4 DSA/SHA1 Signature, key ID 824b8603: NOKEY
Preparing...                ########################################### [100%]
   1:osg-release            ########################################### [100%]
[root@fermicloud123 ~]# yum install osg-oasis
Loaded plugins: priorities, security
Setting up Install Process
osg                                                      | 1.9 kB     00:00     
osg/primary_db                                           | 979 kB     00:00     
988 packages excluded due to repository priority protections
Resolving Dependencies
--> Running transaction check
---> Package osg-oasis.noarch 0:6-4.osg33.el6 will be installed
--> Processing Dependency: cvmfs-config-osg >= 1.2-3 for package: osg-oasis-6-4.osg33.el6.noarch
--> Processing Dependency: cvmfs >= 2.2.1 for package: osg-oasis-6-4.osg33.el6.noarch
--> Running transaction check
---> Package cvmfs.x86_64 0:2.2.2-1.osg33.el6 will be installed
--> Processing Dependency: libfuse.so.2(FUSE_2.4)(64bit) for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: libfuse.so.2(FUSE_2.8)(64bit) for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: libfuse.so.2(FUSE_2.6)(64bit) for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: fuse-libs for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: gdb for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: libfuse.so.2(FUSE_2.5)(64bit) for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: fuse for package: cvmfs-2.2.2-1.osg33.el6.x86_64
--> Processing Dependency: libfuse.so.2()(64bit) for package: cvmfs-2.2.2-1.osg33.el6.x86_64
---> Package cvmfs-config-osg.noarch 0:1.2-3.osg33.el6 will be installed
--> Running transaction check
---> Package fuse.x86_64 0:2.8.3-4.el6 will be installed
---> Package fuse-libs.x86_64 0:2.8.3-4.el6 will be installed
---> Package gdb.x86_64 0:7.2-83.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package              Arch       Version                  Repository       Size
================================================================================
Installing:
 osg-oasis            noarch     6-4.osg33.el6            osg             2.8 k
Installing for dependencies:
 cvmfs                x86_64     2.2.2-1.osg33.el6        osg             7.0 M
 cvmfs-config-osg     noarch     1.2-3.osg33.el6          osg             5.6 k
 fuse                 x86_64     2.8.3-4.el6              slf-primary      70 k
 fuse-libs            x86_64     2.8.3-4.el6              slf-primary      73 k
 gdb                  x86_64     7.2-83.el6               slf-primary     2.3 M

Transaction Summary
================================================================================
Install       6 Package(s)

Total download size: 9.4 M
Installed size: 35 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): cvmfs-2.2.2-1.osg33.el6.x86_64.rpm                | 7.0 MB     00:00     
(2/6): cvmfs-config-osg-1.2-3.osg33.el6.noarch.rpm       | 5.6 kB     00:00     
(3/6): fuse-2.8.3-4.el6.x86_64.rpm                       |  70 kB     00:00     
(4/6): fuse-libs-2.8.3-4.el6.x86_64.rpm                  |  73 kB     00:00     
(5/6): gdb-7.2-83.el6.x86_64.rpm                         | 2.3 MB     00:00     
(6/6): osg-oasis-6-4.osg33.el6.noarch.rpm                | 2.8 kB     00:00     
--------------------------------------------------------------------------------
Total                                            14 MB/s | 9.4 MB     00:00     
warning: rpmts_HdrFromFdno: Header V4 DSA/SHA1 Signature, key ID 824b8603: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-OSG
Importing GPG key 0x824B8603:
 Userid : OSG Software Team (RPM Signing Key for Koji Packages) 
 Package: osg-release-3.3-5.osg33.el6.noarch (installed)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-OSG
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : cvmfs-config-osg-1.2-3.osg33.el6.noarch                      1/6 
  Installing : fuse-libs-2.8.3-4.el6.x86_64                                 2/6 
  Installing : fuse-2.8.3-4.el6.x86_64                                      3/6 
  Installing : gdb-7.2-83.el6.x86_64                                        4/6 
  Installing : cvmfs-2.2.2-1.osg33.el6.x86_64                               5/6 
  Installing : osg-oasis-6-4.osg33.el6.noarch                               6/6 
  Verifying  : cvmfs-2.2.2-1.osg33.el6.x86_64                               1/6 
  Verifying  : gdb-7.2-83.el6.x86_64                                        2/6 
  Verifying  : fuse-2.8.3-4.el6.x86_64                                      3/6 
  Verifying  : cvmfs-config-osg-1.2-3.osg33.el6.noarch                      4/6 
  Verifying  : osg-oasis-6-4.osg33.el6.noarch                               5/6 
  Verifying  : fuse-libs-2.8.3-4.el6.x86_64                                 6/6 

Installed:
  osg-oasis.noarch 0:6-4.osg33.el6                                              

Dependency Installed:
  cvmfs.x86_64 0:2.2.2-1.osg33.el6   cvmfs-config-osg.noarch 0:1.2-3.osg33.el6  
  fuse.x86_64 0:2.8.3-4.el6          fuse-libs.x86_64 0:2.8.3-4.el6             
  gdb.x86_64 0:7.2-83.el6           

Complete!
[root@fermicloud123 ~]# echo user_allow_other >>/etc/fuse.conf
[root@fermicloud123 ~]# echo "/cvmfs /etc/auto.cvmfs" >>/etc/auto.master
[root@fermicloud123 ~]# service autofs restart
Stopping automount:                                        [  OK  ]
Starting automount:                                        [  OK  ]
[root@fermicloud123 ~]# cat >/etc/cvmfs/default.local
CVMFS_REPOSITORIES="`echo $((echo oasis.opensciencegrid.org;echo cms.cern.ch;ls /cvmfs)|sort -u)|tr ' ' ,`"
CVMFS_QUOTA_LIMIT=20000
CVMFS_HTTP_PROXY="http://squid.fnal.gov:3128"
[root@fermicloud123 ~]# ls /cvmfs
[root@fermicloud123 ~]# ls -l /cvmfs/atlas.cern.ch
total 1
drwxr-xr-x 7 cvmfs cvmfs 3 Jul  7  2015 repo
[root@fermicloud123 ~]# ls -l /cvmfs/oasis.opensciencegrid.org/cmssoft
total 1
lrwxrwxrwx 1 cvmfs cvmfs 18 May 13  2015 cms -> /cvmfs/cms.cern.ch
[root@fermicloud123 ~]# ls -l /cvmfs/glast.egi.eu
total 5
drwxr-xr-x 9 cvmfs cvmfs 4096 Feb  7  2014 glast
[root@fermicloud123 ~]# ls -l /cvmfs/nova.osgstorage.org
total 6
lrwxrwxrwx 1 cvmfs cvmfs   32 Jan 19 11:40 flux -> pnfs/fnal.gov/usr/nova/data/flux
-rw-r--r-- 1 cvmfs cvmfs   49 Jan 19 11:38 new_repository
drwxr-xr-x 3 cvmfs cvmfs 4096 Jan 19 11:39 pnfs
[root@fermicloud123 ~]# ls /cvmfs
atlas.cern.ch  config-osg.opensciencegrid.org  nova.osgstorage.org
cms.cern.ch    glast.egi.eu                    oasis.opensciencegrid.org
[root@fermicloud123 cvmfs]# cvmfs_config umount
Unmounting /cvmfs/config-osg.opensciencegrid.org: OK
Unmounting /cvmfs/atlas.cern.ch: OK
Unmounting /cvmfs/oasis.opensciencegrid.org: OK
Unmounting /cvmfs/cms.cern.ch: OK
Unmounting /cvmfs/glast.egi.eu: OK
Unmounting /cvmfs/nova.osgstorage.org: OK
[root@fermicloud123 ~]# 

8 File Locations

Service/Process Configuration File Description
cvmfs /etc/cvmfs/default.local cvmfs environment settings and repository setup
fuse /etc/fuse.conf fuse settings
automount /etc/auto.master automount settings

9 How to get Help?

If you cannot resolve the problem, there are several ways to receive help:

For a full set of help options, see Help Procedure.

10 References

11 Comments

Topic revision: r47 - 07 Jun 2017 - 20:12:07 - DaveDykstra
Hello, TWikiGuest!
Register

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..