Storage Meeting Minutes 2009 Sep 30


  • Andrew Baranovski
  • Brian Bockelman
  • Ted Hesselroth
  • Neha Sharma
  • Alex Sim
  • Tanya Levshina

Status Report



Moved SCEC support forward by encouraging UFL to fix their GIP configuration. 
Though the GIP team will not offer support for their non-hep compute element due to extensive customization, 
YuJun Wu is working on this. So, kudos.

Made and tested several gPlazma patches for dCache 1.9.5 "Golden" release. Instead of returning only the 
authorizations, gPlazma now returns a map of all roles found and their authorizations, including "null" if 
the role is not authorized. Solved a problem of unexpected results when multiple roles mapped to the 
same authorization.


Follow up on tickets.


* Support
  - srm-copy usage for a large file/directory transfer
  - srm-ping and srm-copy usage in RSV probe

* bestman release
   - packaging, testing

* next major release as bestman2 will be towards the end of this year.
   bestman2  uses tomcat for the http container and the latest
gridftp/gsi libraries.
   current bestman will be maintained for a while as bestman1
   until all sites move to bestman2, but all new features will only be
added to the bestman2.

* bestman new features in

1. VOMS validation is supported
   conf entry: validateVomsProxy=true  (false by default)
   With or without gums support in the server, it validates the voms proxy.
2. user space keys are supported
   userSpaceKeywords is to define keys to shorten the path
   format: userSpaceKeywords=(key1=/path1)(key2=/path2)
   e.g.: userSpaceKeywords=(tic=/user/tic)(tac=/proj/tac)
   bestman will disable space management when this is defined
   and SFN=/tic/fpath1 in SURL will be translated to SFN=/user/tic/fpath1
   difference from staticTokens: space descriptions are not supported.
3. checksum callout is supported
   this is effective only if showChecksumWhenListingFile=true
   conf entry: hexChecksumCommand
   e.g. hecChecksumCommand=/usr/local/bin/mychecksum -adler32
   When a file needs checksum, the file path is attached to the call-out
   and the returns can be seperated by spaces, and
   the first value is assumed to be the checksum value output.
4. No directory browsing is supported as a server side enforcement.
   conf entry: NoDirBrowsing=true
   It will handle srmLs on all directory level browsing with level 0
   returning the directory information only.

* bestman changes in

5. (full mode) fixed wrong space token generation after restarting the
6. (full mode) fixed wrong status code return in srmCopy
7. (full mode) fixed truncated directory browsing for a long path name

* setup changes and new options in
1. all command line scripts support dash environment on Ubuntu
2. added --with-container-threads= for web service container threads
   defaults to 64 per 512MB max java heap size
3. added --enable-voms-validation
4. added --with-user-space-key= for user space keys
   format: (key1=/path2)(key2=/path2)
5. added --enable-voms-validation
6. added --with-checksum-callout= to specify path for checksum
callout command
7. added configure.gateway and configure.fullmode
   conf/bestman.rc will be customized and simplied for each mode.

* srm clients lbnl - new features in
1. added clear status tags (request or file) for returned status information
2. added "-connectiontimeout " in srm-copy for soap or gsi
   timeout as well as file transfer timeout.


        Tier-3 liaison group
         OSG VO forum
         OSG Documentation meeting

  Email discussions:
        Xrootd/Xrootd-GridFTP/Xrootdfs release/support policy
        Status of SCEC jobs with Michael, Brian
        VOMS Validation in new BeStMan release
        Comments for standalone gridftp document
        dcache support: upgrade discussion with David Lesny
        comments on request to include details for configuring service certificate for BeStMan in documentation
        discussed  workaround for xrootd problem i've encountered 
  Technical work:
        Applied workaround  suggested by xrootd developers, was able to "restore" file meta data , has recommended
         to release xrootd/xrootdfs for vdt
        Tested new release of BeStMan, BeStMan client and srm tester.
        Started to work on thridparty transfer tests
        Reviewed Andrew's proposal, provided feedback

First agenda item

Andrew's troubleshooting/log collection design doc - any feedback?


Brian has questioned the viability of our decision to look into log collection/troubleshooting. He thought that the similar attempts have failed before (e.g. he has mentioned about apparent reluctance of sys admins to install syslog-ng for log collection). Tanya reminded that we have the following item in our WBS for FY10:

"Investigate tools for storage troubleshooting"

Brian agreed that this activity seems to be relevant to WBS and asked Andrew to add more implementation details into the document. Tanya asked to add real use cases to the document, e.g example how logs and relevant filtering rules could help troubleshooting. Andrew said that he would need help with that.

Second Agenda Item

Storage discovery tools - how far we from VDT release?


Ted started working on improving caching. He will contact STG to report the progress and discuss packaging. Tanya suggested to provide configuration script that would allow to configure discovery tools to access OSG or ITB bdii.

Third agenda item

Progress with SCEC (Ted)


Mats loaded tar files to Caltech, Nebraska and Firefly. Now, he will need to untar them and run jobs. UFlorida is still working on GIP configuration.

Fourth agenda item

New Bestman release and plans for the next major release (Alex)


New BeStMan is released and ready for testing (see status report). BeStMan2? work has started.

Filth agenda item

Hadoop, space tokens & space utilization plugin


Plugin is needed to report space usage for static tokens (lcg-cp failed if this information is not available). Brian said that there is a way to set quota on hadoop fs and use hadoop command to report space utilization. Andrew will try to set quotas on our local test stand and write plugin for BeStMan.

Sixth agenda item

RSV Storage Probes


Arvind has requested that OSG Storage takes responsibilities over RSV Storage Probes. We will need to talk to Alain and understand if this is a reasonable request for us before making any decision.

Seventh agenda item

Tickets (Neha)


++++++++++++++++++++++++ New tickets ++++++++++++++++++++++++


7573 - Hadoop - Root I/O Plugin for hadoop

Follow up in progress:

7571 - Bestman - GSIFTP timeout with srm-copy

7572 - Hadoop - Tuning Tips

++++++++++++++++++++++++ Old tickets ++++++++++++++++++++++++

Closed: 7380 -dCache - gPlazma drops roles when multiple roles map to same username

7447 - dCache - Low throughput observed after upgrading to dcache 1.9.4-3

Follow up in progress:

7505/7574 - dCache - Site losing data due to files being marked as non-sticky

7504 - dCache - Configuring space tokens at user account level instead of VO/Role

7448 - dCache - Poor PNFS/Chimera performance on upgrade from 1.9.2-5 to 1.9.4.-x

6967/7010 - dCache - File replicas not being removed

No update: 7326 - dCache - misleading error message when using dccp to transfer data

6971 - dCache - Problem with pnfs register command

6908 - dCache - log4j errors when restarting dcache 1.9.2-5

Created by:

-- TanyaLevshina - 01 Oct 2009
Topic revision: r2 - 01 Oct 2009 - 21:39:08 - TanyaLevshina
Hello, TWikiGuest


TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..