Upgrade Voms Service


This document describes how to upgrade from VOMS-Admin installed from osg-1.2 (pacman chache) to new osg-voms package (installed via yum): We assume that you are currently running:

  • VOMS Admin 2.0.15-1
  • VOMS Server 1.8.8-2p1
The new version of VOMS-Admin and VOMS-Server, the OSG 3 VOMS, will be installed during the upgrade procedure. You will install:
  • VOMS-Admin 2.6.1
  • VOMS-Server 2.0.0

If you want to do a fresh installation please consult VOMS Installation Guide instead.

You can install the OSG 3 VOMS-Admin and VOMS-Server on the same node where your OSG 1.2 servers are running or on a new node. Either ways OSG 3 is installed using yum, so you need root access on the node.

Preparing For Upgrade

warning WARNING: VOMS-Admin assumes hostname resolves to the fully-qualified domain name; if it does not, the upgrade will fail in mysterious ways. Type hostname and verify it is of the form voms.example.com and not voms. If your pacman-based install was installed with the wrong hostname, you may have to contact osg-software for expert help if the upgrade fails.

In order to prepare for upgrade you will need to stop all the services and create complete myslqdump of your voms databases. You have to perform the following steps:

  1. Login on the node where voms is install
  2. setup VDT_LOCATION
    [root@voms ~]$ cd VDT_LOCATION
    [root@voms ~]$. setup.sh
  3. Stop all the services:
    [root@voms ~]$vdt-control -off
  4. Restart MySql server:
    [root@voms ~]$vdt-control -on mysql5
  5. Create database dump:
    [root@voms ~]$  mysqldump --all-databases --flush-privileges >  ~/voms_database_dump.sql
  6. Stop MySql server
    [root@voms ~]$vdt-control -off mysql5
  7. The VDT pollutes the environment of your shell. Log out of this one.

For the remainder of this document, all pacman-based services should remain off.

Requirements for the OSG 3 VOMS server

Wether you install on the same node or on a new one, you must make sure that the node wehre you install the OSG 3 VOMS server satisfies all the following requirements.

If you are using the same node you can re-use the service certificates. If not you have to get new ones. In doubt, get new ones (see below).

Host and OS

  • A host to install the VOMS Service (Pristine node)
  • OS is Red Hat Enterprise Linux 5, 6, 7, and variants (see details...). Currently most of our testing has been done on Scientific Linux 5.
  • Time must be synchronized. You can find more in the time synchronization document.
  • Root access


The VOMS and voms-admin installation will create two users unless they are already created.

User Default uid Comment
voms none Runs the VOMS daemons, one per VO.
tomcat 91 Runs tomcat5 (EL5)/tomcat6 (EL6) and owns voms-admin configuration and certificates.

Note that if uid 91ise already taken but not used for the tomcat user, you will experience errors. Details...


You will need two service certificates. Here are instructions to request a host certificate. Optional, you can use the host certificate instead as explained in the section below.

Certificate User that owns certificate Path to certificate
VOMS service certificate voms /etc/grid-security/voms/vomscert.pem
Tomcat service certificate tomcat /etc/grid-security/http/httpcert.pem


For more details on overall Firewall configuration, please see our Firewall documentation.

Service Name Protocol Port Number Inbound Outbound Comment
VOMS tcp 15001+ Y   range of ports, increment by 1 for each VO supported
VOMS Admin tcp 8443 Y   VOMS Admin (which runs within Tomcat) web interface, it must be available to VO administrators and users registering online

* 15001 is the port specified in the configuration command. You can choose a different one or have multiple ones if you support multiple VOs.

Additional Requirements

  • Testing requirements:
    • The Subject and Issuer of a voms admin certificate (it could be your certificate)
    • Access to your own certificate if you want to create your voms proxy
    • Your certificate uploaded into a browser if you want to test voms-admin WEB UI

The configuration and testing require some familiarity with openssl commands, e.g. to extract the subject (DN) and the issuer of a certificate. If you are not familiar please check the guides in the reference section:
# to whom was it issued (subject/DN)
[user@voms ~]$ openssl x509 -noout -in /etc/grid-security/hostcert.pem -subject
subject= /DC=org/DC=doegrids/OU=Services/CN=fermicloud002.fnal.gov
# issuer
[user@voms ~]$ openssl x509 -noout -in /etc/grid-security/hostcert.pem -issuer
issuer= /DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1

OSG 3 VOMS Installation

Open a new window on the node you are doing upgrade to make sure that you start with a clean environment!

Install the Yum Repositories required by OSG

The OSG RPMs currently support Red Hat Enterprise Linux 5, 6, 7, and variants (see details...).

OSG RPMs are distributed via the OSG yum repositories. Some packages depend on packages distributed via the EPEL repositories. So both repositories must be enabled.

Install EPEL

  • Install the EPEL repository, if not already present. Note: This enables EPEL by default. Choose the right version to match your OS version.
    # EPEL 5 (For RHEL 5, CentOS 5, and SL 5) 
    [root@voms ~]$ curl -O https://dl.fedoraproject.org/pub/epel/epel-release-latest-5.noarch.rpm
    [root@voms ~]$ rpm -Uvh epel-release-latest-5.noarch.rpm
    # EPEL 6 (For RHEL 6, CentOS 6, and SL 6) 
    [root@voms ~]$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
    # EPEL 7 (For RHEL 7, CentOS 7, and SL 7) 
    [root@voms ~]$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    WARNING: if you have your own mirror or configuration of the EPEL repository, you MUST verify that the OSG repository has a better yum priority than EPEL (details). Otherwise, you will have strange dependency resolution (depsolving) issues.

Install the Yum priorities package

For packages that exist in both OSG and EPEL repositories, it is important to prefer the OSG ones or else OSG software installs may fail. Installing the Yum priorities package enables the repository priority system to work.

  1. Choose the correct package name based on your operating systemís major version:

    • For EL 5 systems, use yum-priorities
    • For EL 6 and EL 7 systems, use yum-plugin-priorities
  2. Install the Yum priorities package:

    [root@voms ~]$ yum install PACKAGE

    Replace PACKAGE with the package name from the previous step.

  3. Ensure that /etc/yum.conf has the following line in the [main] section (particularly when using ROCKS), thereby enabling Yum plugins, including the priorities one:

    NOTE: If you do not have a required key you can force the installation using --nogpgcheck; e.g., yum install --nogpgcheck yum-priorities.

Install OSG Repositories

  1. If you are upgrading from OSG 3.1 (or 3.2) to OSG 3.2 (or 3.3), remove the old OSG repository definition files and clean the Yum cache:

    [root@voms ~]$ yum clean all
    [root@voms ~]$ rpm -e osg-release

    This step ensures that local changes to *.repo files will not block the installation of the new OSG repositories. After this step, *.repo files that have been changed will exist in /etc/yum.repos.d/ with the *.rpmsave extension. After installing the new OSG repositories (the next step) you may want to apply any changes made in the *.rpmsave files to the new *.repo files.

  2. Install the OSG repositories using one of the following methods depending on your EL version:

    1. For EL versions greater than EL5, install the files directly from repo.grid.iu.edu:

      [root@voms ~]$ rpm -Uvh URL

      Where URL is one of the following:

      Series EL6 URL (for RHEL 6, CentOS 6, or SL 6) EL7 URL (for RHEL 7, CentOS 7, or SL 7)
      OSG 3.2 https://repo.grid.iu.edu/osg/3.2/osg-3.2-el6-release-latest.rpm N/A
      OSG 3.3 https://repo.grid.iu.edu/osg/3.3/osg-3.3-el6-release-latest.rpm https://repo.grid.iu.edu/osg/3.3/osg-3.3-el7-release-latest.rpm
    2. For EL5, download the repo file and install it using the following:

      [root@voms ~]$ curl -O https://repo.grid.iu.edu/osg/3.2/osg-3.2-el5-release-latest.rpm
      [root@voms ~]$ rpm -Uvh osg-3.2-el5-release-latest.rpm

For more details, please see our yum repository documentation.

Install the CA Certificates: A quick guide

You must perform one of the following yum commands below to select this host's CA certificates.

Set of CAs CA certs name Installation command (as root)
OSG osg-ca-certs yum install osg-ca-certs Recommended
IGTF igtf-ca-certs yum install igtf-ca-certs
None* empty-ca-certs yum install empty-ca-certs --enablerepo=osg-empty
Any** Any yum install osg-ca-scripts

* The empty-ca-certs RPM indicates you will be manually installing the CA certificates on the node.
** The osg-ca-scripts RPM provides a cron script that automatically downloads CA updates, and requires further configuration.

If you use options 1 or 2, then you will need to run "yum update" in order to get the latest version of CAs when they are released. With option 4 a cron service is provided which will always download the updated CA package for you.

If you use services like Apache's httpd you must restart them after each update of the CA certificates, otherwise they will continue to use the old version of the CA certificates.
For more details and options, please see our CA certificates documentation.

Install VOMS

  1. Install Java using these instructions
  2. Install OSG-VOMS:
    [root@voms ~]$ yum install osg-voms
    [root@voms ~]$ yum install osg-voms
    Loaded plugins: kernel-module
    Setting up Install Process
    Resolving Dependencies
    --> Running transaction check
    ---> Package osg-voms.noarch 0:3.0.0-1 set to be updated
    --> Processing Dependency: voms-mysql-plugin for package: osg-voms
    --> Processing Dependency: voms-server for package: osg-voms
    --> Processing Dependency: voms-admin-server for package: osg-voms
    --> Processing Dependency: voms-admin-client for package: osg-voms
    --> Processing Dependency: mysql-server for package: osg-voms
    Dependencies Resolved
     Package                                Arch              Version                                Repository                 Size
     osg-voms                               noarch            3.0.0-1                                osg                       1.6 k
    Installing for dependencies:
     PyXML                                  x86_64            0.8.4-4.el5_4.2                        fermi-security            1.0 M
     alsa-lib                               x86_64            1.0.17-1.el5                           sl-base                   414 k
    Transaction Summary
    Install      67 Package(s)
    Upgrade       0 Package(s)
    Total download size: 247 M
    Is this ok [y/N]: y
    Downloading Packages:
    (1/67): osg-voms-3.0.0-1.noarch.rpm                                                                       | 1.6 kB     00:00     
    (2/67): geronimo-specs-compat-1.0-0.M2.2jpp.12.x86_64.rpm                                                 | 5.4 kB     00:00     
    (3/67): libXtst-1.0.1-3.1.x86_64.rpm                                                                      |  16 kB     00:00     
    (4/67): xml-commons-1.3.02-0.b2.7jpp.10.x86_64.rpm                                                        |  19 kB     00:00     
    (67/67): jdk-1.6.0_26-fcs.x86_64.rpm                                                                      |  68 MB     00:02     
    Total                                                                                             14 MB/s | 247 MB     00:17     
    warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID 217521f6
    epel/gpgkey                                                                                               | 1.7 kB     00:00     
    Importing GPG key 0x217521F6 "Fedora EPEL " from /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
    Is this ok [y/N]: y
    Running rpm_check_debug
    Running Transaction Test
    Finished Transaction Test
    Transaction Test Succeeded
    Running Transaction
      Installing     : perl-DBI                                                                                                 1/67 
      Installing     : mysql                                                                                                    2/67 
      Installing     : PyXML                                                                                                    3/67 
      Installing     : osg-voms                                                                                                67/67 
      osg-voms.noarch 0:3.0.0-1                                                                                                      
    Dependency Installed:
      PyXML.x86_64 0:0.8.4-4.el5_4.2                                 alsa-lib.x86_64 0:1.0.17-1.el5                                 
      ant.x86_64 0:1.6.5-2jpp.2                                      antlr.x86_64 0:2.7.6-4jpp.2            
      xerces-j2.x86_64 0:2.7.1-7jpp.2.el5_4.2                        xml-commons.x86_64 0:1.3.02-0.b2.7jpp.10                       
      xml-commons-apis.x86_64 0:1.3.02-0.b2.7jpp.10                  xml-commons-resolver.x86_64 0:1.1-1jpp.12                      

Upgrade and Configuration

The configuration requires some initial steps plus the import or addition of one or more VO. You will not be able to start voms-admin if you don't import or add at least one VO.

Configure MySQL Database

Before you will be able configure VOMS you will need to have access to MySQL database. The MySQL software is installed with the osg-voms package. By default it will be using port 3306 and will not have password for root.

It is not necessary to change the port number if you don't want to. If you want to modify the port you will need to edit /etc/my.cnf file and add a new port:

It is wise to set a password for the root user. Run the following command to set the password:
[root@voms ~]$ /etc/init.d/mysqld start
[root@voms ~]$ mysqladmin -u root password top_secret

Setup the Service Certificates

Make sure that you have the host, VOMS and Tomcat service certificates/keys in the right directories and with the correct owners and permissions (600 for key and 644 the certificate). Host certificate is required and must be in /etc/grid-security/hostcert.pem and hostkey.pem. It is recommended to provide also service certificates for VOMS and Tomcat. If you don't, here are the instructions to copy the host certificate/key and set the correct owner (permissions are preserved). Note that if you did not previously have a voms or tomcat user on your system, they were added during the RPM install:
  1. VOMS-core daemon is running as user voms and requires its service certificate and key in /etc/grid-security/voms. To copy the host certificate/key for this service, you need to do the following:
    [root@voms ~]$ cd /etc/grid-security
    [root@voms ~]$ cp hostcert.pem voms/vomscert.pem
    [root@voms ~]$ cp hostkey.pem voms/vomskey.pem
    [root@voms ~]$ chown -R voms:voms voms
  2. Tomcat service is running as user tomcat and requires its service certificate and key in /etc/grid-security/http. To copy the host certificate/key for this service, you need to do the following:
    [root@voms ~]$ cd /etc/grid-security
    [root@voms ~]$ mkdir /etc/grid-security/http
    [root@voms ~]$ cp hostcert.pem http/httpcert.pem
    [root@voms ~]$ cp hostkey.pem http/httpkey.pem
    [root@voms ~]$ chown -R tomcat:tomcat http

Configure Tomcat options

Edit /etc/tomcat5/tomcat5.conf (EL5) or /etc/tomcat6/tomcat6.conf (EL6) to add the following line to the bottom of the file. This is important for good performance and essential if you run more than one VO.


If you run a large number of VOs, you might need to increase the amount of memory used by Tomcat. You can do this by adding one more line to the bottom of /etc/tomcat5/tomcat5.conf (EL5) or /etc/tomcat6/tomcat6.conf (EL6):

JAVA_OPTS="${JAVA_OPTS} -Xmx2048m"

EL6 only: You must add the following line to /etc/tomcat6/tomcat6.conf to use VOMS-Admin successfully:


Disable SSLv3 in Tomcat

The OSG Security team has concluded that the POODLE SSLv3 vulnerability is not a critical concern to OSG software installations. Most services are not affected, and those that are affected are difficult to exploit in a meaningful way. Nonetheless, the recommendation is to disable support for SSLv3 where reasonable.

OSG software includes VOMS Admin Server (currently, version 2.7.0), which runs within Tomcat. By default Tomcat allows SSLv3 connections, but that is easy to change. To disable SSLv3 support from a Tomcat instance that contains VOMS Admin Server, set protocols="TLSv1.1,TLSv1.2" in /etc/tomcat[56]/server.xml, as shown below.

Note: OSG Software has tested this change only for Tomcat and VOMS Admin Server. But this is a server-level Tomcat configuration change and thus affects all Tomcat web applications running on the same server. For now, we recommend making this change to Tomcat servers that run only VOMS Admin Server. There are known issues with applying this change to a Tomcat instance that runs GUMS, in that dCache clients (at least) fail to work with the changed GUMS server.

--- server.xml.old      2014-10-28 11:27:11.000000000 -0500
+++ server.xml  2014-10-28 11:29:34.000000000 -0500
@@ -20,19 +20,20 @@
     <Connector port="8443" SSLEnabled="true"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" disableUploadTimeout="true"
               acceptCount="100" debug="0" scheme="https" secure="true"
               clientAuth="true" sslProtocol="TLS" sslEnabledProtocols="TLSv1"
+              protocols="TLSv1.1,TLSv1.2"
               crlEnabled="true" crlRequired="true"/>

    <Engine name="Catalina" defaultHost="localhost">

      <Host name="localhost" appBase="webapps" />

Allow Tomcat to have more open files and processes

If you run several VOs, you may need to extend the limits for Tomcat so that it can open more file and create more processes. To do this, add the following lines to /etc/security/limits.conf:

tomcat          soft    nofile  63536
tomcat          hard    nofile  63536

tomcat          soft    nproc   16384
tomcat          hard    nproc   16384

Note that this is not needed if you are running just a single VO, this is only needed if you run many VOs.

Configure Trust Manager

Configure Tomcat's trust manager. This must be done once, before starting up Tomcat for the first time:

  1. You have to configure Tomcat in order to enable EMI trustmanager:
    [root@voms ~]$ /var/lib/trustmanager-tomcat/configure.sh
    Info: using default install root: /
    Info: using default configuration file: //var/lib/trustmanager-tomcat/config.properties
    Info: using default configuration directory: //var/lib/trustmanager-tomcat
    Info: you can clean up using the following commands
          mv -f /etc/tomcat5/server.xml.old-trustmanager /etc/tomcat5/server.xml
          rm -f /usr/share/tomcat5/server/lib/bcprov*.jar
          rm -f /usr/share/tomcat5/server/lib/log4j*.jar
          rm -f /usr/share/tomcat5/server/lib/trustmanager-*.jar
          rm -f /etc/tomcat5/log4j-trustmanager.properties
          rm -f //var/lib/trustmanager-tomcat/server.xml
(the output may look different on EL6)

Restore The Old Database

If you did not already, please start MySQL with /sbin/service mysqld start .

Restore database for the database dump:

[root@voms ~]$  mysql -p < ~/voms_database_dump.sql
When asked at the prompt, you must enter the top_secret password that you choose above. If mysql has no root password (we highly recommend a root password), you will have to drop the -p option.

Configure VOMS and VOMS-Admin

In order to configure database you will need to figure out the password and admin name of the database you have used before. Gather information from the following files:
  • VDT_LOCATION/glite/etc/voms/VO_NAME/voms.conf
    • OLD_USER_NAME: Search for --username attribute (e.g --username=test1_adm)
    • OLD_VOMS_PORT: Seach for --port (e.g --port=15000)
  • VDT_LOCATION/glite/etc/voms/VO_NAME/voms.pass
    • OLD_USER_PASSWORD: This file contains the previous MySQL user password (e.g secret).
  • /etc/my.conf
    • MYSQL_PORT: Look for "port"; if it is not set in this file, it is 3306.

Run voms-admin-configure script:

[root@voms ~]$ voms-admin-configure install --dbtype mysql --vo VO_NAME \
    --dbauser root  --dbapwd MYSQL_ROOT_PASSD --dbport MYSQL_PORT \
    --dbusername  OLD_USER_NAME --dbpassword OLD_USER_PASSWORD \
    --port OLD_VOMS_PORT  --mail-from email --smtp-host smtp.domain \
    --sqlloc /usr/lib64/voms/libvomsmysql.so --cert /etc/grid-security/voms/vomscert.pem \
    --key  /etc/grid-security/voms/vomskey.pem --read-access-for-authenticated-clients 
For example:
[root@voms ~]$ voms-admin-configure install --dbtype mysql --vo test1 \
     --dbauser root --dbapwd  top_secret --dbport 3306 \
     --dbusername test1_adm --dbpassword secret
     --port 15000 --mail-from tlevshin@fnal.gov --smtp-host smtp.fnal.gov \
     --sqlloc /usr/lib64/voms/libvomsmysql.so --cert /etc/grid-security/voms/vomscert.pem \
     --key  /etc/grid-security/voms/vomskey.pem --read-access-for-authenticated-clients 

If your VO:

  • Has Custom-made scripts for managing voms-admin, or
  • Uses VOMRS,
These items might be broken due to new CSRF security checks (Cross Site Request Forgery). Affected VOs may chose to disable these security checks. To do this, edit /etc/voms-admin/VO_NAME/voms.service.properties and include the following line:
#### Add other options after this line
voms.csrf.log_only = true

We recommend that unaffected VOs do not make this change.

Upgrade The Database Schema

For each VO that you imported form the old VOMS server you must run the upgrade command:

[root@voms ~]$voms-admin-configure upgrade --vo VO_NAME
For example:
[root@voms ~]$voms-admin-configure upgrade --vo test1

Remember that before using VOMS or VOMS-Admin you must start VOMS, VOMS-Admin and all the other required services.

Add and configure additional VOs

You can add additional VOs if you like. If you migrated at least one pre-existing VO and do not desire to add additional VOs, then you can skip this section.

You must add at least one VO before you can start VOMS Admin. You must start the services as instructed during the configuration steps. You can stop them at the end if you don't want them running yet.

  1. Run voms-admin-configure script to create a new VO. Note that if you have multiple VOs, each VO must have a unique VOMS_PORT
    [root@voms ~]$ voms-admin-configure install \
        --dbtype mysql \
        --vo VO_NAME \
        --createdb \
        --deploy-database  \
        --dbauser root  \
        --dbapwd MYSQL_ROOT_PASSD \
        --dbusername  admin-VO_NAME \
        --dbpassword secret  \
        --dbport  DB_PORT \
        --port VOMS_PORT  \
        --mail-from email \
        --smtp-host smtp.domain \
        --sqlloc /usr/lib64/voms/libvomsmysql.so \
        --cert /etc/grid-security/voms/vomscert.pem \
        --key  /etc/grid-security/voms/vomskey.pem \
    For example for the test1 VO:
    [root@voms ~]$ voms-admin-configure install --dbtype mysql --vo test1 --createdb \
         --deploy-database --dbauser root --dbapwd  top_secret \
         --dbusername admin_test1 --dbpassword secret --dbport 3306 \
         --port 15001 --mail-from tlevshin@fnal.gov --smtp-host smtp.fnal.gov \
         --sqlloc /usr/lib64/voms/libvomsmysql.so --cert /etc/grid-security/voms/vomscert.pem \
         --key  /etc/grid-security/voms/vomskey.pem --read-access-for-authenticated-clients 
    If the command fails with a IOError: [Errno 32] Broken pipe error, check that you entered the correct database password.
    If the above command is executed correctly a lot of sql stuff will scroll across the screen, followed by
    Deploying voms database...
    Database deployed correctly!
    VO test1 installation finished.
    You can start the voms services using the following commands:
        /etc/init.d/voms start test1
        /etc/init.d/voms-admin start test1
  2. Start VOMS and VOMS Admin for the VO you just added:
    [root@voms ~]$ /sbin/service voms start VO_NAME
    [root@voms ~]$ /sbin/service voms-admin start VO_NAME
    For example:
    [root@voms ~]$ /sbin/service voms start test1
    [root@voms ~]$ /sbin/service voms-admin start test1
    Note that the voms-admin init script really isn't an init script. Instead it deploys a Tomcat webapp into Tomcat. In normal usage, you never need to run this command again, and you should not run /sbin/service voms-admin stop VO_NAME.
  3. Add a local admin:
    [root@voms ~]$ /usr/sbin/voms-db-deploy.py add-admin --vo VO_NAME  \
            --dn  HOST_CERT_SUBJECT \
            --ca HOST_CERT_ISSUER 
    For example:
    [root@voms ~]$ /usr/sbin/voms-db-deploy.py add-admin --vo test1 \
         --dn /DC=org/DC=doegrids/OU=Services/CN=fermicloud002.fnal.gov \
         --ca "/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1"
  4. In order to complete the next step you need to start also Tomcat if it is not already running. And since Tomcat requires valid CRLs you need to fetch the CRLs if it is a fresh installation they are not there:
    [root@voms ~]$ /usr/sbin/fetch-crl  # on EL5 before 3.1.15 or EL6. if there are no CRLs in /etc/grid-security/certificates
    [root@voms ~]$ /usr/sbin/fetch-crl3  # On EL5 3.1.15 or later. If there are no CRLs in /etc/grid-security/certificates
    [root@voms ~]$ /sbin/service tomcat5 start # on EL5 only
    [root@voms ~]$ /sbin/service tomcat6 start # on EL6 only
  5. Configure VOMS Admin to allow GUMS and edg-mkgridmap to query your VO for the set of users. Note that you must have a valid certificate; voms-admin needs a certificate authorized to perform the operation (ACL editing) on the VO:
    [root@voms ~]$ voms-admin --nousercert --vo VO_NAME add-ACL-entry /VO_NAME ANYONE VOMS_CA "CONTAINER_READ,MEMBERSHIP_READ" true
    For example:
    [root@voms ~]$ voms-admin --nousercert --vo test1 add-ACL-entry /test1 ANYONE VOMS_CA "CONTAINER_READ,MEMBERSHIP_READ" true
  6. Allow compatibility with GUMS and edg-mkgridmap. Edit /etc/voms-admin/VO_NAME/voms.service.properties by adding the following line to the file:
    voms.csrf.log_only = true

If you don't want to leave your services running you can stop them after you are done with adding the VO:

[root@voms ~]$ /sbin/service voms stop
[root@voms ~]$ /sbin/service tomcat5 stop # on EL5 only
[root@voms ~]$ /sbin/service tomcat6 stop # on EL6 only


VOMS has its persistent repository in a MySQL database, that requires mysql to be running in order to be queried. All grid interactions require updated CA certificates and CRLs. Then the voms server is a daemon servicing authentication requests and voms-admin, requiring Tomcat, is a Web UI/service to manage the VO membership.

Starting and Enabling Services

To start VOMS you need to start several services:

  1. You need to fetch the latest CA Certificate Revocation Lists (CRLs) and you should enable the fetch-crl service to keep the CRLs up to date:
    # For RHEL 5, CentOS 5, and SL5 
    [root@voms ~]$ /usr/sbin/fetch-crl3   # This fetches the CRLs 
    [root@voms ~]$ /sbin/service fetch-crl3-boot start
    [root@voms ~]$ /sbin/service fetch-crl3-cron start
    # For RHEL 6, CentOS 6, and SL6, or OSG 3 _older_ than 3.1.15 
    [root@voms ~]$ /usr/sbin/fetch-crl   # This fetches the CRLs 
    [root@voms ~]$ /sbin/service fetch-crl-boot start
    [root@voms ~]$ /sbin/service fetch-crl-cron start
    # For RHEL 7, CentOS 7, and SL7 
    [root@voms ~]$ /usr/sbin/fetch-crl   # This fetches the CRLs 
    [root@voms ~]$ systemctl start fetch-crl-boot
    [root@voms ~]$ systemctl start fetch-crl-cron
    For more details and options, please see our CRL documentation.
  2. MySQL server needs to be running if it is not already:
    [root@voms ~]$ /sbin/service mysqld start
  3. Start voms server
    [root@voms ~]$ service voms start
  4. Start Tomcat, required by voms-admin:
    [root@voms ~]$ service tomcat5 start # on EL5 only
    [root@voms ~]$ service tomcat6 start # on EL6 only

You should also enable the appropriate services so that they are automatically started when your system is powered on:

  • To enable the fetch-crl service to keep the CRLs up to date after reboots:
    # For RHEL 5, CentOS 5, and SL5 
    [root@voms ~]$ /sbin/chkconfig fetch-crl3-boot on
    [root@voms ~]$ /sbin/chkconfig fetch-crl3-cron on
    # For RHEL 6, CentOS 6, and SL6, or OSG 3 _older_ than 3.1.15 
    [root@voms ~]$ /sbin/chkconfig fetch-crl-boot on
    [root@voms ~]$ /sbin/chkconfig fetch-crl-cron on
    # For RHEL 7, CentOS 7, and SL7 
    [root@voms ~]$ systemctl enable fetch-crl-boot
    [root@voms ~]$ systemctl enable fetch-crl-cron
  • To enable the other services:
    [root@voms ~]$ /sbin/chkconfig mysqld on
    [root@voms ~]$ /sbin/chkconfig voms on
    [root@voms ~]$ /sbin/chkconfig tomcat5 on # on EL5 only
    [root@voms ~]$ /sbin/chkconfig tomcat6 on # on EL6 only

Note: You do not need to run chkconfig on the voms-admin service. This service is only need to do the initial "deployment" of the VOMS Admin service into Tomcat and is not needed thereafter unless you add another VO.

Stopping and Disabling Services

To stop VOMS you need to stop voms and voms-admin, and also the services they use if nothing else is using them:

  1. Stop voms server
    [root@voms ~]$ service voms stop
  2. , Stop Tomcat, if no other application is using it:
    [root@voms ~]$ service tomcat5 stop # on EL5 only
    [root@voms ~]$ service tomcat6 stop # on EL6 only
  3. Stop MySQL server if no other application is using it:
    [root@voms ~]$ service mysqld stop
  4. To stop fetch-crl:
    # For RHEL 5, CentOS 5, and SL5 
    [root@voms ~]$ /sbin/service fetch-crl3-boot stop
    [root@voms ~]$ /sbin/service fetch-crl3-cron stop
    # For RHEL 6, CentOS 6, and SL6, or OSG 3 _older_ than 3.1.15 
    [root@voms ~]$ /sbin/service fetch-crl-boot stop
    [root@voms ~]$ /sbin/service fetch-crl-cron stop
    # For RHEL 7, CentOS 7, and SL7 
    [root@voms ~]$ systemctl stop fetch-crl-boot
    [root@voms ~]$ systemctl stop fetch-crl-cron
    For more details and options, please see our CRL documentation.

In addition, you can disable services by running the following commands. However, you don't need to do this normally.

  • To disable the fetch-crl service:
    # For RHEL 5, CentOS 5, and SL5 
    [root@voms ~]$ /sbin/chkconfig fetch-crl3-boot off
    [root@voms ~]$ /sbin/chkconfig fetch-crl3-cron off
    # For RHEL 6, CentOS 6, and SL6, or OSG 3 _older_ than 3.1.15 
    [root@voms ~]$ /sbin/chkconfig fetch-crl-boot off
    [root@voms ~]$ /sbin/chkconfig fetch-crl-cron off
    # For RHEL 7, CentOS 7, and SL7 
    [root@voms ~]$ systemctl disable fetch-crl-boot
    [root@voms ~]$ systemctl disable fetch-crl-cron
  • To disable the other services:
    [root@voms ~]$ /sbin/chkconfig mysqld off
    [root@voms ~]$ /sbin/chkconfig voms off
    [root@voms ~]$ /sbin/chkconfig tomcat5 off # on EL5 only
    [root@voms ~]$ /sbin/chkconfig tomcat6 off # on EL6 only

Note: You do not need to run chkconfig on the voms-admin service. See above for more details.

Advertise your VOMS server

If you are installing VOMS on the same node or are replacing the old server with a new node that has the same hostname, then you can skip this section. Otherwise you need to notify everyone using your VOMS server about the new IP address as described in the remaining of this section.

A working VOMS server can be contacted to sign certificates extensions (e.g., by voms-proxy-init) or to verify VO memberships (e.g., by GUMS) only if it is on the list of valid VOMS servers (/etc/vomses) and if there is an LSC file for that VO. Some grid services (glite-FTS and glite-WMS) even require a copy of the VOMS server certificate.

The file /etc/vomses contains a list of VOs and their VOMS servers. A VOMS server may appear on multiple lines if it is serving multiple VOs on different ports. All lines contain:

For example:
"test1" "fermicloud002.fnal.gov" "15001" "/DC=org/DC=doegrids/OU=Services/CN=fermicloud002.fnal.gov" "test1"

The LSC file (LiSt of Certificates) is another description of your VOMS server:

  • It is located in $X509_VOMS_DIR/${VO}, by default /etc/grid-security/vomsdir/${VO}
  • The filename is the fully qualified hostname (FQDN) of your server, followed by .lsc
  • The first line contains the subject DN of the VOMS server host certificate, without quotes
  • The second line contains the subject DN of the CA that issued the VOMS server host certificate, also without quotes

For example, for the VO named test1 hosted on fermicloud002, the LSC file named /etc/grid-security/vomsdir/test1/fermicloud002.fnal.gov.lsc might contain:

/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1

The VOMS server manager must distribute the line for /etc/vomses and the LSC file to all hosts that will contact the VOMS server. OSG distributes with all its client and server installations a default list of VOMS servers (/etc/vomses) and a default set of LSC files. These are part of the vo-client RPM.

To be included, use the VO Registration function at OIM VO Registration. Even if you are already registered as a VO you need to notify the Grid Operations Center if the information in the vomses file or the LSC file has changed.


Before performing any of the tests you must start all the services as described above.

To test VOMS you can add a member (her/his certificate) to a VO using voms-admin, login in the Web UI and then:

  • use the voms server to generate a proxy for that certificate.
  • or generate a grid-mapfile
  • or query with GUMS

Voms-admin test

Add a member(yourself) to the VO:
[root@voms ~]$ voms-admin --vo VO_NAME --host HOST_NAME \
       --nousercert create-user USER_CERT_SUBJECT USER_CERT_ISSUER \
       USER_COMMON_NAME email 
For example:
[root@voms ~]$ voms-admin --vo test1 --host fermicloud002 --nousercert create-user \
   "/DC=org/DC=doegrids/OU=People/CN=Joe Doe 12345" "/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1" 
   "Joe Doe 12345" 'jdoe@fnal.gov'
Assign VOAdmin role to yourself, so you can manage the VO using the Web UI:
voms-admin --vo VO_NAME --host HOST_NAME \
    --nousercert assign-role  /VO_NAME  VO-Admin  \
For example:
[root@voms ~]$ voms-admin --vo test1 --host fermicloud002 --nousercert assign-role /test1   VO-Admin  \
   "/DC=org/DC=doegrids/OU=People/CN=Joe Doe 12345" "/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1" 

Check the Web UI by accessing https://hostname:8443/voms/voname. If you have your certificate loaded into the browser you will be able to manage the VO.

Voms-core test:

To perform this test you need the voms-client RPM that is normally installed with OSG client.

To use your VOMS you need to add a line with its description (similar to the following) to /etc/vomses, a file containing a list of all valid voms servers:

Look at /etc/voms-admin/VO_NAME/vomses to find the correct line for your VO. For example:
"test1" "fermicloud002.fnal.gov" "15001" "/DC=org/DC=doegrids/OU=Services/CN=fermicloud002.fnal.gov" "test1"

If it is not there already, add your certificate to the VO using voms-admin create-user as in the previous section. Being the local admin set with voms-db-deploy.py add-admin does not add the certificate to the VO.

Use voms-proxy-init to generate proxy. Login as user, create vomses file (e.g. ~/.vomses). Generate voms proxy

[user@voms ~]$ voms-proxy-init -voms VO_NAME -vomses ~/vomses

Click here for a more detailed example on using and verifying a VOMS proxy.

Testing edg-mkgridmap

The edg-mkgridmap command is run on a site's Compute Element node to generate a grid-mapfile file. This script is not provided as a part of the VOMS installation, so you will have to go on a CE node to test this. You do not have to be the root user on the CE node, but you will need to have a personal certificate available.

The mkgridmap file syntax/values for each VO you create, can be viewed on the WEB UI for your VO by selecting Configuration information on the left hand menu of the main page. You will get a line like:

 group vomss://HOST_NAME:8443/voms/VO_NAME  group_name
e.g. for the OSG VO :
 group vomss://cms-xen3.fnal.gov:8443/voms/OSG  osg

The osg (3rd token) is an EDG format for the account name. You will likely want to change it to the group unix account you want assigned for your CE node. The example above is for a group account assignment.

To test, you will need to do this on a CE node where the authorization mode is "gridmap" (lcmaps or PRIMA are not enabled):

  1. Populate /etc/edg-mkgridmap.conf with the example configuration file on your VOMS server. It will be the only entry.
    [root@voms ~]$ echo 'group vomss://cms-xen3.fnal.gov:8443/voms/OSG  osg' > /etc/edg-mkgridmap.conf
  2. Execute edg-mkgridmap:
    [root@voms ~]$ edg-mkgridmap
      ... the output will be in /etc/grid-security/grid-mapfile and will look as follows ...
      "/DC=org/DC=doegrids/OU=People/CN=John Weigand 458491"  vdt
      "/DC=org/DC=doegrids/OU=Services/CN=cms-xen3.fnal.gov"  vdt

You can find more in the Compute Element install document.

Testing the GUMS interface

The GUMS authorization server periodically 'pulls' the VO membership data for it's authorization and account assignment functions from the various VO's VOMS servers.

Unfortunately, there is no way of testing this except with an installed GUMS server which this document does not address.

See Install Configure and Manage GUMS for more information.

Removing a VO

If you added the VO just for testing purposes, you can remove it using voms-admin-configure remove --vo VONAME as documented in the user guide, e.g.:
## Stop the service 
[root@voms ~]$ service voms-admin stop test1
[root@voms ~]$ service voms stop test1  
## Remove it.
[root@voms ~]$ voms-admin-configure remove --vo test1 --undeploy-database --dropdb --dbapwd top_secret
voms-admin-configure, version 2.6.1

Removing vo  test1
VO test1 succesfully removed.
## Restart to update the voms-server webpage 
[root@voms ~]$ service tomcat5 restart # on EL5 only
[root@voms ~]$ service tomcat6 restart # on EL6 only

Note that the --undeploy-database and --dropdb will remove the database that stores all the membership information. You cannot undo this option. If you're not quite sure that you really want to remove the database, then do not pass this option. The VO will be unaccessible and unusable, but the membership information will be retained in case you recreate the VO later.


Useful configuration and log files

Configuration Files

Service or Process Configuration File Description
voms /etc/voms/VO_NAME/voms.conf
VOMS server configuration and password for the VO database
voms-admin /etc/voms-admin/voms-siblings.xml Auxiliary voms-admin application configuration
  /etc/voms-admin/VO_NAME/logback.runtime.xml Logging configuration
  /etc/voms-admin/VO_NAME/vo-aup.txt VO acceptable use policy
VOMS Admin configuration
  /etc/voms-admin/VO_NAME/voms.database.properties VO database handler configuration
  /etc/voms-admin/VO_NAME/vomses vomses line, see the advertising section
mysql /var/lib/mysql/VO_NAME VOMS database for the VO
  /etc/my.cnf MySQL configuration, e.g. server port
tomcat /etc/tomcat5/ Tomcat configuration files (EL5)
tomcat /etc/tomcat6/ Tomcat configuration files (EL6)
  /etc/tomcat5/Catalina/localhost/* VOMS related tomcat configuration (EL5)
  /etc/tomcat6/Catalina/localhost/* VOMS related tomcat configuration (EL6)

Log files

Service or Process Log File Description
tomcat (EL5) /var/log/tomcat5/catalina.out This is the Tomcat log file. Problems (and a lot of noise) are reported here.
  /var/log/tomcat5/voms-admin-VO_NAME.log This is the vo-by-vo log for voms-admin Web UI
  /var/log/tomcat5/trustmanager.log The trustmanager handles things related to authentication. Useful errors are sometimes here.
tomcat (EL6) /var/log/tomcat6/catalina.out This is the Tomcat log file. Problems (and a lot of noise) are reported here.
  /var/log/tomcat6/voms-admin-VO_NAME.log This is the vo-by-vo log for voms-admin Web UI
  /var/log/tomcat6/trustmanager.log The trustmanager handles things related to authentication. Useful errors are sometimes here.
voms /var/log/voms/voms.VO_NAME This is the log of the VOMS daemon for each VO

Fix these errors

VOMS Server for VO_NAME not known!

If you saw an error like "VOMS Server for VO_NAME not known!" probably you did not add the VO VO_NAME (e.g. test1) to /etc/vomses

Tomcat is not starting

If Tomcat is not starting and /var/log/tomcat5/catalina.out contains an error like "/usr/bin/tomcat5: line 331: /usr/lib/java/bin/java: No such file or directory", then you may have to uncomment the line JAVA_HOME="/usr/lib/jvm/java" in /etc/tomcat5/tomcat5.conf. The Jpackage repository is known to provide a misconfigured tomcat5 package.

Problems running voms-admin

Verify if voms-admin was configured and started correctly:
  • If voms-admin is not responding and you get a "Socket error 111" try restarting tomcat5 (EL5) or tomcat6 (EL6) and voms-admin
  • If there are BC (bouncycastle) related errors in Tomcat's log file (/var/log/tomcat5/catalina.out on EL5, /var/log/tomcat6/catalina.out on EL6), the trust manager may not be configured correctly. Try to run /var/lib/trustmanager-tomcat/configure.sh.
  • If there are OpenSAML or XML parser related errors in Tomcat's log file on EL6 (/var/log/tomcat6/catalina.out), the JAVA_ENDORSED_DIRS variable may not be set in /etc/tomcat6/tomcat6.conf. See the Configure Tomcat options section above.
  • If you get an error like "Socket error: (1, 'error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown')" probably you have no CRLs for your CA certificates. You can see in /var/log/tomcat5/trustmanager.log (EL5) or /var/log/tomcat6/trustmanager.log (EL6) that Tomcat is discarding CA certificates without CRL. Run fetch-crl.

Wrong URLs in VOMS admin pages/Connection timed out/Host unavailable

If you can reach the first page but all the links are failing, check the URL in the links. Tomcat by default constructs the absolute URLs in the web pages using the hostname/IP and port used for the request. If the server resides behind a Firewall or NAT that enforces port redirection, these may be the server IP or the port on the private network, that likely are not reachable by your Web browser. To fix the problem you must use Tomcat's proxy support by setting proxyName and/or proxyPort in Tomcat's configuration file (/etc/tomcat5/server.xml).

/etc/tomcat5/server.xml. Note the two lines with proxyName and/or proxyPort added to the original file. Replace the name and port with your public name and port:
<Server port="8005" shutdown="SHUTDOWN">
  <Service name="Catalina">

    <Connector port="8443" SSLEnabled="true"
proxyName="public_hostname.domain" proxyPort="public_port"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" disableUploadTimeout="true"
               acceptCount="100" debug="0" scheme="https" secure="true"
               clientAuth="true" sslProtocol="TLS"
               crlEnabled="true" crlRequired="true" />

    <Engine name="Catalina" defaultHost="localhost">

      <Host name="localhost" appBase="webapps" />
After changing server.xml stop and restart Tomcat in order for the changes to take effect.

How to get Help?

To get assistance please use Help Procedure.


Other VOMS documents:

EMI documentation:

Openssl commands:


Topic revision: r18 - 06 Dec 2016 - 18:12:45 - KyleGross
Hello, TWikiGuest!


TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..