Preparing Compute Element

This document discusses some preliminary steps before beginning the compute element installation.

Included topic: Before Starting Compute Element Installation

Some suggestions before starting your installation.

• Read the Site Planning Guide and know the other servers that will be necessary to get your site going.
• Most site admins install OSG compute element services as root. Paraphrasing from the VDT site, if you install as root the VDT's programs can be made available to all the users on your system. Additionally, the VDT will be able to configure and start some of the grid services you might be installing. Some services can only be run if you are root. Corollary: If you don't install as root, you cannot do these things. Unless your local policy states otherwise, you will want to install the OSG compute element as root.
• Consult SiteFabricBestPractices which may be helpful, and the Pacman notes for installation and best practices.
• Consult here for supported operating systems.
• You must have a supported batch system (Condor, PBS, LSF, or SGE) installed on your CE host machine, or accessible to it. We provide instructions for installing Condor as a batch system if you'd like to use it.
• You'll need a personal grid certificate to run post-install tests.
• You can apply in advance for Host and Service Certificates needed by your CE host. There are also scripts that come with the installation that simplify the process somewhat.
• Consider the configuration values needed (resource name, server location, local storage locations, and so on); this is discussed in PrepareForComputeElementConfigure and LocalStorageConfiguration.

Included topic: Prepare For Compute Element Configure

Introduction

This section is intended to describe the information you will need available to configure your compute element. It also provides some general guidelines for your physical configuration of hosts and file systems. A more detailed description of all of the OSG attributes (variables) can be found in OSG Configuration Parameters. You will need to fill in this information into the config.ini file, when you run the $VDT_LOCATION/monitoring/configure-osg.py script after installing the OSG CE software. Note also you can re-use configuration settings from previous installs, see ReusingConfigurationInformation.

Requested information

GENERAL INFORMATION
OSG Group	Identifies the monitoring group that your site is participating in. Values are â%u20AC¢ OSG - For Production â%u20AC¢ OSG-ITB - For the integration testbed
Resource Name	See ResourceName
Sponsors	The VO sponsors for your site (e.g., usatlas, ivdgl, ligo, uscms, sdss). You must express the percentage of sponsorship using the following notation: myvo:50 yourvo:10 anothervo:20 local:20
Policy URL	The URL for the document describing the usage policy/agreement for this resource.
CONTACT INFORMATION
Contact name	The site administrator's full name.
Contact email	The site adminstrator's email address.
SERVER LOCATION
City	The city your server is located in or near.
Country	The country your server is located in.
Longitude/Latitude	For your city. This will determine your placement on any world maps used for monitoring. You can find some approximate values for your geographic location from GeoTags or you can search your location on Google. For the USA, latitude is about 29 (South) ... 48 (North) â%u20AC¢ longitude is about -123 (west coast) ... -71 (east coast)
LOCAL STORAGE INFORMATION
	Variables such as GRID, APP, DATA, and so on define various storage locations available on your site for jobs. These are described in detail in the a separate Local Storage Configuration document. It also provides recommendations for the definitions and environment variables for the Local Storage accessible to Compute Elements on OSG. It covers the definition of these spaces and correspondences to external schemas (e.g. GLUE, OSG).

SQUID WEB PROXY SERVER
	The following questions only get asked if you indicate that you want to use squid when running configure-osg.py
Squid Location	The host machine on which squid is running. This is usually a cluster machine with sufficient memory and disk space to cache incoming and outgoing web traffic and store any file that clients may download.
Memory Cache Size	The amount of memory that squid should use for caching downloads and files. This may be temporarily exceeded if squid is proxying a very large download. Setting this to more than 512MB should improve download performance significantly.
Disk Cache Size	The amount of disk space that squid should use for storing cached objects. This should be set to a value larger than 2-3GB for optimum benefits.
GIP GENERIC INFORMATION PROVIDER INFORMATION
SE Sitename	This is the registered OSG sitename for the SE that is being used for the cluster (If an SE is available)
SE Hostname	The hostname for the SE that is being used for the cluster (If an SE is available)
Root directory	The full path to the root directory of the se (e.g. /pnfs/osgsite.edu/data
Local directories	The directory paths relative to the root directory for each vo to use on the SE. This is either a single path if the simplified option is used or a path for each VO will be requested if the simplified option is not choosen.
GSIFTP server	The hostname for a gsiftp server associated with the cluster (the hostname of the CE can be used if there is no dedicated gsiftp server)
Access Path	The access path to use with the gsiftp server.
BATCH QUEUE MANAGER INFORMATION
Condor	The values of VDTSETUP_CONDOR_LOCATION and VDTSETUP_CONDOR_CONFIG are needed and these environment variables should be set before the pacman packages are installed
SGE	The value of SGE_ROOT is needed
PBS and LSF	No environment variable is needed but the PBS and/or LSF executables must be in your $PATH

Included topic: Pre Installation Checklist CE

Introduction

This checklist provides some environment variables you may need to set and some facilities considerations that may need to be addressed before installing and enabling OSG software.

Previous OSG Installations

• ReusingConfigurationInformation describes how you can maintain your OSG configuration when installing a new OSG release. Typically you should pre-set OLD_VDT_LOCATION to location-of-previous-vdt-installation.
• You do not need to shutdown OSG services before running the new install on the same machine. However before you start services from the new installation, you will need to shutdown the old services. See OSG Shutdown Guide for details.

Exporting Condor environment variables for VDT install

Define and export the variables VDTSETUP_CONDOR_LOCATION and VDTSETUP_CONDOR_CONFIG to point to your Condor installation (i.e., the directory directly above bin, etc, lib, and so on) and Condor configuration file.

• Typically:

  export VDTSETUP_CONDOR_LOCATION=$CONDOR_LOCATION
  export VDTSETUP_CONDOR_CONFIG=$CONDOR_CONFIG 
  

where $CONDOR_LOCATION =/my/condor/location and $CONDOR_CONFIG =my condor_config file

Configuring Condor to optimize Gratia probe performance.

For versions of Condor 6.9.0 and above, one can configure Condor such that it writes ClassAds into an area watched by the Gratia accounting system upon job completion. If this is the case, the Gratia Condor probe will get all of its information from this ClassAd file and have no need to execute condor_history. This will relieve your gatekeeper of a potentially time consuming and costly operation (especially if you're running Quill). In your Condor schedd configuration (the machine-local condor config file should be sufficient), add:

PER_JOB_HISTORY_DIR = /path-to-vdt-installation/gratia/var/data

RSV Prerequisites

• Valid Unix account: RSV requires a valid Unix account with a valid shell for running probe jobs using the Condor-Cron scheduling infrastructure
  • e.g.: rsvuser account where su -c "/bin/date" rsvuser returns a date output
  • a locked Unix account is acceptable
• Authorization to run jobs/transfer files: RSV probes require access to a grid proxy authorized to run Globus jobs (for CE monitoring) or move files on a target SE (for SE monitoring). Two options are:
  • An RSV service certificate (like an http service cert) used with RSV's automatic proxy regeneration for monitoring services on the same host as the RSV probes. See here for more information on authorizing an RSV service certificate in GUMS or using gridmap files.
  • A valid user certificate with by hand proxy generation which can also be used to monitor services on external hosts.
  • More information on choosing between a user cert and a service cert is available here.

Condor-cron installation with OSG-RSV

As of VDT 1.10.0, condor-cron (a specially configured installation of condor, previously called condor-devel) is automatically installed with the OSG CE for running OSG-RSV. If, however, sites choose not to run OSG-RSV on the CE (because it's run on separate monitoring system for example) or otherwise have any problems with installing condor-cron, the installation will be skipped if pre-set:

  export VDTSETUP_NO_CONDOR_CRON=y

Setting GUMS environment variables for VDT install

To save yourself some steps later on in the privilege (authorization) configuration, set the variable VDT_GUMS_HOST to your GUMS host now. For example:

# export VDT_GUMS_HOST=mygumshost.domain

Operating System and Networking Requirements

• No Compilers are required to install CE
• Working network connection is required. If your site is inside a firewall, has a host firewall (ipchains/iptables), or is using TCP wrappers (/etc/host.{allow,deny}), please review the Firewall Document
• Disk footprint for the OSG CE is ~900MB or ~1.2 GB if Condor for Managed Fork is included
• Time Synchronization: NTP should be supported on all systems as lack of synchronization complicates troubleshooting and can cause problems with grid security infrastructures.
• Reverse Name Lookups (DNS): both the forward and reverse lookups as configured through a local DNS service are required for the IP address of the system

Local Account Information

• Accounts for OSG VOs: UNIX user accounts need to be created by the system administrator for the VOs. You will need to create at least one local user account (with the appropriate configuration) for each VO to which you wish to provide resources. The uid and name of the account can be locally determined. You will be asked to provide a mapping from local accounts to VOs later during the configuration process. Since VO's are frequently added to the OSG, you should access the current list of registered VO's at http://www.grid.iu.edu/osg-includes/ (click VOs). Any new VO's will also appear in on the CE in the file $VDT_LOCATION/monitoring/osg-user-vo-map.txt.
• Service Accounts:
  • Web Service container will be installed to run as the user globus if that account exist. Otherwise it will run as daemon.
  • CEMon container requires an http service cert and runs as daemon.
  • RSV user account - see RSV Prerequisites above.

Do you have a user certificate?

You will need a personal grid certificate to run validation tests in your $HOME/.globus directory. If you don't have a personal certificate, or don't know how to generate a grid proxy (grid-proxy-init, etc), see User Certificates.

Complete:
Responsible: StevenTimm - 01 Nov 2007
Reviewer - date: RobGardner - 20 Nov 2007