OSG Accounting Services - Gratia

Introduction

The Gratia accounting services consists of three services:

  • The probes; please read the general preliminary notes in addition to those specific to each probe as some things have changed. In particular, note item 3 of the preliminary notes. The RPMs now use /etc/cron.d instead of root's crontab: The VDT packages may benefit from the same change;
  • Gratia reporting (installed via gratia_reporting_v1.00.5.tar), which contains the server side report application;
  • Gratia services (installed via gratia_services_v1.00.5.tar), which contains the gratia services that collect information from the probes and write to the database;
  • See also the Full listing of files. You might want to right-click these links and use a utility like wget to download the files on your Linux node where you'll be performing this installation.

Requirements

  • MySQL >= 5.0 (not tested with MySQL >= 5.1.0; does not work with MySQL 4.x);
  • JRE >= 5 (1.5) (not tested with JRE > 5);
  • Tomcat >= 5.5 (not tested with Tomcat >=6.0; does not work with Tomcat 5.0); to get it, (eg) pacman -get http://vdt.cs.wisc.edu/vdt_1101_cache:Tomcat-5.5
  • Port 8080 under Tomcat 5.5 must be defined as a non-ssl port. To implement this, modify $VDT_LOCATION/tomcat/v55/conf/server.xml:
  <!--  Define a non-SSL HTTP/1.1 Connector on port 8880 -->
   <Connector port="8880" maxHttpHeaderSize="8892"
              maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
              enableLookups="false" redirectPort="8443" acceptCount="100"
              connectionTimeout="20000" disableUploadTimeout="true" />

The Apache configuration under the same VDT installation must include the following JkMounts:

Paste the following in to VDT_LOCATION/apache/conf/httpd.conf

     JkMount /gratia-servlets/* tomcat55
     <Location /gratia-servlets>
        SSLVerifyClient optional
     </Location>

     JkMount /gratia-soap/* tomcat55
     <Location /gratia-soap>
         SSLVerifyClient optional
     </Location>

     JkMount /gratia-services/* tomcat55
     <Location /gratia-services>
         SSLVerifyClient optional
     </Location>

     JkMount /gratia-administration/* tomcat55
     <Location /gratia-administration>
         SSLVerifyClient optional
     </Location>

     JkMount /gratia-reporting/* tomcat55
     <Location /gratia-reporting>
         SSLVerifyClient optional
     </Location>

     JkMount /gratia-reports/* tomcat55
     <Location /gratia-reports>
         SSLVerifyClient optional
     </Location>

Deprecated services:

  • gratia-security with release v0.34

Installing the Services

To begin installation of the services, untar the two tarballs containing the gratia services and gratia reporting packages and copy all *.war files from ./gratia/gratia_reporting and ./gratia/gratia_services to ./tomcat/v55/webapps:
  • gratia-administration.war
  • gratia-reporting.war
  • gratia-reports.war
  • gratia-services.war
  • gratia-servlets.war
  • gratia-soap.war
  • gratia-util.war

Deprecated services:

  • gratia-security with release v0.34

If you are going to manually deploy the services under Tomcat, you must deploy gratia_services.war first; then you may deploy the other war files in any order.

HELP NOTE
If you have configured Tomcat to auto-deploy the war files (not the default under VDT), it will deploy the services in the correct order.

Follow the instructions below for the customization of each package:

gratia_services_v1.00.5.tar

This requires MySQL 5.0, Tomcat v55 and JDK 1.5.

HELP NOTE
Since release v0.38, we rely on files being placed in tomcat/v55/common/lib, tomcat/v55/server/lib and tomcat/v55/common/classes. In addition, log4j.properties should be copied from tomcat/v55/gratia to tomcat/v55/common/classes (or merged if one already exists). Our recommendation has always been that Gratia services be the only thing installed in a given VDT area so contention for libraries between different services in the same tomcat container should not be an issue.

HELP NOTE
Since release v0.38, heavy traffic of dCache records might require the tomcat memory startup parameters to read, " -Xmx1024m -XX:MaxPermSize=256m" or better to avoid out-of-memory errors in the service.

MySql Notes

Effective with v0.32.1, Gratia is using the MySql InnoDB transactional engine. This requires some additional attributes in the my.cnf. The list below shows the recommended my.cnf attributes in addition to the InnoDB attributes. Of all the InnoDB attributes, the most critical is the innodb_file_per_table attribute which allocates 1 tablespace per table and they are stored in the database directory. Without that attribute MySql stores ALL the InnoDB tables for ALL the databases in a single tablespace at the var directory level. This makes disk/space management impossible. Once set up with a single tablespace, it is a very time consuming task to convert, especially for a large database.
[mysqld]
datadir=VDT_LOCATION/vdt-app-data/mysql5/var
user=mysql
port=NNNNN
socket=VDT_LOCATION/vdt-app-data/mysql5/var/mysql.sock

key_buffer = 16M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size=512
net_buffer_length=8
read_buffer_size=256
read_rnd_buffer_size=512
myisam_sort_buffer_size = 8M

# Required for InnoDB
log-bin=mysql-bin
log_bin_trust_function_creators=1
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_files_in_group = 2
innodb_log_buffer_size = 8M
innodb_file_per_table
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
expire_logs_days = 7

[client]
port=49152
socket=VDT_LCOATION/vdt-app-data/mysql5/var/mysql.sock

Location of my.cnf file
Another consideration that should be taken into account is the location of the my.cnf file. It has been stored in the VDT_LOCATION/mysql5/var directory . The VDT_LOCATION/vdt-app-data/mysql5/var directory may be a more suitable location as any changes to the my.cnf for a local environment, may want to be carried forward when upgrades are performed.

Installation Notes

  1. Untar the file under $VDT_LOCATION to populate ./tomcat/v55/gratia:
    tar xf gratia_services_v1.00.5.tar
  2. If and only if it does not already exist, create and configure the DB schema:
    1. Create the empty database schema ( CREATE DATABASE `gratia`; )
    2. Create two users: one that has full control over the gratia schema from localhost and one that has only SELECT and EXECUTE privilege. We suggest using "gratia" and "reader" for the full-control and restricted users, respectively. We suggest using the mysql-administrator utility from the MySQL-GUI-tools package available from the mysql.com website. You'll probably need to run it from the MySQL server and connect to localhost as the DB user root to have the necessary privileges. First create the user, then add hosts. We recommend specifying IPs only (not DNS names). Use the tabs to find the schema privileges and use the selection mechanism to initiate the desired privileges for each user for each host. We recommend against having wild cards, especially for the root user. For the coomand-line mysql client, this would be something like:
      grant all on gratia.* to admin@localhost identified by 'pwd';
      grant select,execute on gratia.* to reader@localhost identified by 'pwd';
      
  3. Modify ./tomcat/v55/gratia/service-configuration.properties to contain the correct database and user information you defined in the previous step, as well as the correct name for the Birt services:
      service.rmi.rmibind=//HOSTNAME:17000
      service.rmi.rmilookup=rmi://HOSTNAME:17000
      service.mysql.driver=com.mysql.jdbc.Driver
      service.open.connection=http://HOSTNAME:8880
      service.secure.connection=https://HOSTNAME:8443
      service.mysql.url=jdbc:mysql://DBHOSTNAME:DBPORT/gratia
      service.mysql.user=DBUSER
      service.mysql.password=DBPASSWORD
    
      service.reporting.user=reader
      service.reporting.password=reader
    
      service.reporting.engine.home=BIRT-HOME

    For the current release, BIRT-HOME should be birt-v20.

  4. Modify the $VDT_LOCATION/tomcat/v55/gratia/post-install.sh making sure that it is mode 0700 and owned by the tomcat-executing user: the line requiring configuration reads:
    cat MAGIC_VDT_LOCATION/tomcat/v55/gratia/${proc} | mysql -B --force --unbuffered --user=root --password=ROOTPASS --host=localhost --port=PORT gratia
    
    ROOTPASS, PORT and MAGIC_VDT_LOCATION must all be substituted correctly for the command to work.

Starting and Stopping Gratia Services

For this version of gratia-services, you must start and stop the service by starting tomcat

  • Start the service using Tomcat:
    service tomcat-55 start
  • In contrast to previous versions, all schema updates are handled from within tomcat in the right order. However, post-install.sh must be configured properly to enable execution of those SQL fragments that require SUPER privilege (see above).

NOTE Database updates may take place during the initial deployment of the collector which may take some time, especially if the schema contains a large amount of data.

gratia_reporting_v1.00.5.tar

  1. Deploy all .war files under Tomcat.

Download shortcuts

Current release

Older Releases


Backlinks

%WHU%

Child Topics

Immediate children of this topic include the following:

Major Updates

-- JohnWeigand - 10 May 2008 - For v0.34 removed references to gratia-security web service. Added InnoDB? attributes for MySql

-- ChrisGreen - 12 Sep 2007 - Update to 0.27.5 for VDT to fix calendar image problems and one report.

-- ChrisGreen - 11 Sep 2007 - Update for major release 0.27.3 to VDT.

-- ChrisGreen - 14 Jun 2007 - Update for major release 0.24

-- ForrestChristian - 03 Jan 2007 (edits only for linking from ITB)

-- ChrisGreen - 11 Oct 2006

-- PenelopeConstanta - 30 Aug 2006

Topic revision: r45 - 17 Nov 2008 - 16:48:27 - ChrisGreen
Accounting.InstallationGuideVDT moved from Accounting.InstallationGuide on 06 Apr 2007 - 19:13 by ChrisGreen - put it back

Google Custom Search
Common links

Accounting

Meta-TWiki links

 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback