Clemson Gratia Probe Development Documentation

This is an article designed to facilitate the general creation of Gratia probes by aggregating the information gathered from the development of one specific probe. The probe in question was made for boinc jobs run via Condor on Clemson University's network. The Gratia program has dependencies that keep it running on Linux systems, but for non-rpm-based package managers (like ones on Arch,) manual setup of the probe is necessary. Since the Condor pool on Clemson consists of mostly windows machines, it was necessary to make a script to upload the logs to a central location.

It is useful to take a look at the Gratia API before starting. There is a minimal example located on that page that will give you a vague outline of how a probe is structured. The general rule of thumb is that the more data the better, but not everything is necessary. In order to create a valid Gratia submission, one needs the following fields:

EndTime
The epoch time that the job ended.
ProbeName
The name of your probe. Typically entered in the ProbeConfig file.
ResourceType
The type of resource that is being logged. Redundant when the UsageRecord? function is called.
Njobs
Number of jobs run.
WallDuration
Wallclock seconds that the job was run
CpuUserDuration
Interchangeable with CpuDuration?(self, time, 'user'), defines amount of processor time spent on the job
CpuSystemDuration
Interchangeable with CpuDuration?(self, time, 'system'), defines amount of processor time spent not working on the job while the job was pending completion.
UsageRecord
Defines the type of record to be sent. In the Clemson probe, this field is set to 'Condor.'
LocalJobId
The job ID for the local computation system. In the Clemson probe, this is is the Condor job ID.
StartTime
The time epoch at the beginning of the job's submission.
LocalUserId
The user responsible for the local job submission. In the Clemson probe, all boinc jobs launched via condor run from the 'labuser' user.
MachineName
The name of the overall compute resource that the probe is reporting statistics from. Set to 'clemson-boinc.'
Host
The actual machine hostname on which the job ran.
VOName
The Virtual Organization in which the computation statistics are to be attributed.

In addition to those fields, it is recommended that this line be added to the very beginning of your submission script:

 rev = "$Revision: 1.2 $" 
 Gratia.RegisterReporterLibrary("",Gratia.ExtractSvnRevision(rev))
** Note: The two fields (CpuUserDuration? and CpuSystemDuration?) are very useful to OSG, but the logs that the Clemson pool is generating do not facilitate this kind of information. Therefore, the user time is set to the walltime and the system time is set to zero. This is a workaround and should be avoided if possible. Also, Some of these fields require either an int or a dateTime string.

The probe code itself can be found here

-- BenjaminSterrett - 19 Jan 2010

Topic attachments
I Attachment Action Size Date Who Comment
pngpng boincprobediagram.png manage 118.8 K 22 Jan 2010 - 18:09 BenjaminSterrett  
Topic revision: r2 - 22 Jan 2010 - 18:22:30 - BenjaminSterrett
Hello, TWikiGuest
Register

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..