Validating Supported VOs

About This Document

An important question you should ask yourself is whether or not your site really supports the Virtual Organizations (VOs) you think it supports. How do you know? There are three files that give you insight into the underlying processes that set up the mappings from global identities (user certificate names) to local accounts, and therefore answer this question. Whether you used GUMS or edg-mkgridmap to handle these mappings, these files will help you verify if you are supporting the VOs you wish you support.

These three files are:

  • /var/lib/osg/user-vo-map
  • /var/lib/osg/undefined-accounts
  • /var/lib/osg/supported-vo-list

VO to User Account Map - user-vo-map

The user-vo-map tells you for each account what VO is associated with that account. (By "each account", we just mean accounts used by GUMS or edg-mkgridmap, not other accounts on your system.) It is a critical component to the following services:

  • OSG Accounting Services (Gratia) - in accurately collecting grid resource usage and metrics by VO for jobs submitted using grid proxies or where voms proxy information is not available.

HELP NOTE
This is the VO name collected by the Gratia probe is not necessarily the one displayed in Gratia reporting services. Refer to the OSG Accounting Services (Gratia) for information on how it uses this value.
  • Generic Information Provider (GIP) - in advertising resource availability on a VO basis.
  • In the configuration validation (using osg-configure ) for your CE of any attributes that reference a VO name in the /etc/osg/config.d/*.ini files.

These files should never be manually modified if either the edg-mkgridmap or gums-host-cron are enabled (both are cron initiated processes).

The user-vo-map file provides the ability to map from a UNIX account (assigned by the CE's authorization service) back to the grid user's VO. There are actually 2 different forms of the VO name for reporting purposes:

  • a case sensitive VO name ( #VOc line in the example below) used only by Trash.ReleaseDocumentationMonALISA

HELP NOTE
Trash.ReleaseDocumentationMonALISA is deprecated and not available in OSG Release 3, the above is just for reference to explain the lines that start with #

  • a lower case VO name ( #voi line in the example below) used by the other OSG services.

Example content: (this example will be referred to by the other subsections on this topic)

#voi cdf fermilab  cms miniboone atlas
#VOc CDF FERMILAB CMS FERMILAB ATLAS
cdf cdf
fnalgrid fermilab
uscms124 cms
uscms132 cms
minboone miniboone
The file is interpreted as follows:
  • The uncommented lines are the mapping of UNIX account (token 1) to #vo VO name (token 2).
  • The mapping from lower case VO name is performed using the tokens on the #voi and #VOc lines for which there is a one-to-one positional relation:
    - The #voi lines represent the lower case VO name.
    - The #VOc lines represent the case sensitive VO name used by Trash.ReleaseDocumentationMonALISA

Thus, from the above example, the mappings would result in:

UNIX
Account
lower case
VO Name
case sensitive
VO Name
cdf cdf CDF
fnalgrid fermilab FERMILAB
uscms124 cms CMS
uscms132 cms CMS
minboone miniboone FERMILAB

As mentioned earlier, those CE nodes using either the edg-mkgridmap or gums-host-cron cron processes will have the 3 files programmatically maintained based on these lines in their respective configuration files.

Using Gridmap authorization mode

For those sites using grid-mapfile authorization mode (using the edg-mkgridmap cron process), the edg-mkgridmap.conf file is used to define the mappings found in the user-vo-map.
# USER-VO-MAP cdf CDF 
group vomss://voms.cnaf.infn.it:8443/voms/cdf cdf
# USER-VO-MAP cms CMS 
group vomss://voms.cnaf.infn.it:8443/voms/cms uscms01

The # USER-VO-MAP identifies this as a candidate for the user-vo-map file
  • the 3rd token (cdf and cms ) is the #voi VO name
  • the 4th token (CDF and CMS) is the #VOc VO name.
Any uncommented line(s) following the # USER-VO-MAP line identify the source of the VO membership (the VOMS server) and account to be assigned
  • the 3rd token (cdf and uscms01) is the UNIX account.

The resulting user-vo-map file would look like this:

#voi cdf cms
#VOc CDF CMS
cdf cdf
uscms01 cms

Using Full Privilege (GUMS) authorization mode

For those CE sites in Full Privilege (GUMS) authorization mode using the gums-host-cron cron process, the gums.config file is used to define these mappings.

In the OSG gums.template , it would look like this

<groupMapping name='cdf-cnaf' accountingVo='cdf' accountingDesc='CDF'>
<userGroup .... 
   <accountMapping className='gov.bnl.gums.GroupAccountMapper'
    groupName='cdf' />
        :
<groupMapping name='cmsuser' accountingVo='cms' accountingDesc='CMS'>
<userGroup .... 
   <accountMapping className='gov.bnl.gums.GroupAccountMapper'
    groupName='uscms01' />

The groupMapping element identifies this as a candidate for the user-vo-map file

  • the accountingVo attribute (cdf and cms) is the #voi VO name.
  • the accountingDesc attribute (CDF and CMS) is the #VOc VO name.
  • the groupName attribute (which is the UNIX account assigned) and the accountingVO attribute provide the detail line mapping UNIX account to #voi VO name. Note: This example is simplified by using group accounts. If pool accounts are used, it will create a detail line for each pool account assigned.

The resulting user-vo-map file would look like this:

#voi cdf cms
#VOc CDF CMS
cdf cdf
uscms01 cms

A couple real important points follow:

  1. If a group mapping is defined on the GUMS server but the group is currently empty, there will be no detail line entry for that UNIX account to #voi VO name in the user-vo-map. The presence of a UNIX account in the user-vo-map as generated by GUMS means that there is at least one DN in the VO that can map to that group.
  2. The above point is especially important when pool accounts are used. If a first time grid user (for the CE node) is assigned a new pool account and the gums-host-cron process has not run, the services that attempt to find the VO name will not be successful. This is a timing issue and should not affect those service significantly if the gums-host-cron is run on a timely basis.
  3. Additionally, a verification is made to determine if the UNIX account exists on that CE node. If the UNIX account does not exist, that UNIX account is removed from the detail line of the user-vo-map file. If a grid user is assigned that account, the service requested will never be able to successfully run. These discarded UNIX accounts are listed in the undefined-accounts files described in a subsequent section.

The points, listed above, have an affect on the set of supported VOs the various services report on.

When all UNIX accounts have been validated, the user-vo-map file is re-evaluated to determine the set of OSG VOs that this CE node will support. By that, it is meant that at least one VO's member can expect to the authorized and successfully assigned an account that will run for this CE node. The supported-vo-list file contains this list of VOs.

This user-vo-map file's timestamp will only change when the contents of the file changed. To assist in verifying the currency of the file, another file called user-vo-map.last_checked is created ('touched') each time the processes are run.

Undefined Account - undefined-accounts

The undefined-accounts file provides a list of those UNIX accounts that can be assigned by your authorization service, but have not been defined on your CE node. You can read this as "the list of accounts you should make, if you want to accept all the VOs you claim to accept".

This file contains a list of those UNIX accounts that have been eliminated from the user-vo-map file because the UNIX account assigned by the authorization service does not exist. This is for gridmap authorization mode using edg-mkgridmap and for Full Privilege authorization mode using gums-host-cron

The test used to determine the UNIX account's existence is 'id -u <account>'. However, there is no verfication performed to determine if a HOME directory exists for the account. This is because the check is done on the CE, but it's hard to check if the home directory really exists on the worker node.

Example content:
# A list of accounts eliminated from the user-vo map.
uscms455
uscms456
usatlas

This file's timestamp will only change when the contents of the file changed. To assist in verifying the currency of the file, another file called undefined-accounts.last_checked is created ('touched') each time the processes are run.

Supported VOs - supported-vo-list

The supported-vo-list file provides an easy means of determining which VOs your sites will support based on your authorization service. If a VO isn't listed here, your site doesn't really support it, even if you think it should.

This file contains a list of all VOs that this CE node will support. It does not mean that all members of the these VOs will be able to run jobs here, but that at least one member will be capable. (Why not all? Perhaps not all members are authorized.) This is for gridmap authorization mode using edg-mkgridmap and for Full Privilege authorization mode using gums-host-cron

Example content:
# List of VOs this site claims to support
CDF
FERMILAB
CMS

HELP NOTE
The VO name currently populated in this file is the case sensitive #VOc VO name. Since this version of the VO name is only used by Trash.ReleaseDocumentationMonALISA and does not use this file directly, this may change to using the #voi name for OSG services. This is possibly a future change.

This file's timestamp will only change when the contents of the file changed. To assist in verifying the currency of the file, another file called supported-vo-list.last_checked is created ('touched') each time the processes are run.

Troubleshooting tips

The accuracy, timeliness and integrity of the user-vo-map file is a critical factor for several OSG services.

There are several things that can be looked at to verify it is being updated in addition to the osg_undefined_accounts_txt and supported-vo-list files.

  1. The files are only updated when there has been a change detected. So if you know your authorization has been changed and as a result affecting any of these files, you can see if it as been effected on the CE node.
  2. Each of the files has a .last_checked suffixed file that is touched whenever the cron scripts are run successfully.
  3. Both cron scripts have log files showing the results of their execution:
    • edg-mkgridmap - /var/log/edg-mkgridmap.log
    • gums-host-cron - /var/log/gums/gums-host-cron.log

You can also run the scripts manually and view the output on the screen:

[root@client ~]$ /usr/bin/gums-host-cron --gumsdebug

[root@client ~]$ /usr/sbin/edg-mkgridmap

-- TerrenceMartin - 25 Oct 2011

Topic revision: r15 - 07 Feb 2017 - 20:22:33 - BrianBockelman
Hello, TWikiGuest!
Register

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..