You are here: TWiki > OSGReports Web>DavidMeyers (22 Feb 2012, KyleGross)

Monthly Reports from David Meyers

The current OSG WBS is located here.

My work falls into these categories from the current OSG WBS:

  • 1.1.2.4 - Coordination of Integration Testbed (0.75 FTE)
    • 1.1.2.4.1 - OSG release candidate 0.5.x integration for ITB
      • Maintain an ITB CE site for validating services and application: LIGO-CIT-ITB
      • Chair the ITB weekly meeting as needed.
      • Support the ITB Validation Twiki pages
    • 1.1.2.4.1.1 - System tests and validation for new services and VOs (VTB)
      • Develop a Validation Test Bed facility to accelerate ITB integration cycles

  • 1.3.2.5 - Integrate and Support Workflow management for LIGO (0.25 FTE)
    • Integrate work of the LIGO Data Analysis teams
    • Port Gravitation Wave analyses to the OSG
    • Design and test the LIGO workflow planner
    • Provide requirements to the Pegasus Planner team and test and integrate new functionality
    • Support the activities of the OSG-Trash/Trash/Trash/Extensions efforts.

Period: May 2007

  • Attended briefing from Inspiral team on new calibrated workflow H(t) and discussed support issues on OSG.
  • Installed and tested new VDS 1.4.8 release on OSG submit host.
  • Attended VTB, VDT and DASWG telecoms.
  • Chaired ITB telecom
  • Installed Condor 6.9.2 on OSG submit host.
  • Provided feedback on VDS 1.4.8 to USC/ISI engineer on compatability issue related to LIGO workflows on the OSG.
  • Completed Pegasus training of Britta Daudert. Britta can now run nanoHIPE and H(t) calibrated LIGO workflows on the OSG.
  • Validated H(t) mini-Inspiral workflow on LIGO-CIT-ITB, OSG_LIGO_PSU and UWMilwaukee OSG sites using 8.1 GB of calibrated H(t) data.
  • Determined that UWMilwaukee has fixed an earlier problem with ws-gram and OSG 0.6.0 on that site.
  • Completed scalability testing of ws-gram on the VTB with Suchandra, Rob, Jeff, Stuart and Martin.

Period: April 2007

  • Configured grid certificate for new LIGO employee.
  • Verified lack of access to CIT data repository for new LIGO employee and contacted appropriate manager.
  • Responded to queries by Globus ws-gram developers for access to LIGO ITB cluster.
  • Responded to request from Pegasus developer for cache configuration on OSG_LIGO_PSU.
  • Provided tutorial on Pegasus planner for new LIGO employee.
  • Attend attributes subcommittee, ITB, Pegasus and VDT telecoms. * Provided engineering support for dynamic cleanup study using the LIGO nanoHIPE analysis on the OSG.
  • Attended the DASWG, Pegasus, WS-GRAM and ITB telecoms
  • Analyzed security incident on LIGO OSG compute resource and had offending site blocked at router.
  • Conducted an analysis of active accounts on LIGO OSG resources and deleted obsolete accounts.
  • Reviewed status of work perfomed by previous LIGO OSG engineer on data movement of gravitational-wave files to SRM repository at UCSD.
  • Chaired WS-GRAM, Pegasus and ITB meetings as Rob Gardner was at OSG BluePrint? meeting.
  • Adjusted value of "containerThreadMax" from 20 to 100 in the ws-gram container at the advice of the Globus developers.
  • Determined with the assistance of Jeff@NERSC that submitting hundreds of jobs to the ws-gram service is no longer bringing down the ws-gram service at LIGO-CIT-ITB.
  • Created a script containing all 70 gravitational-wave files used in the nanoHIPE workflow for future testing of bulk transfer from a posix file system to SRM at UCSD.
  • Installed latest OSG client package on LIGO submit host.
  • Requested Java 5 be added to the OSG client package as part of the OSG 0.8.0 requirements.
  • Responded to a request from a Globus developer for an account on a LIGO submit host to provide a resource for OSG testing of ws-gram.
  • Provided a tutorial for Britta Daudert on Pegasus transformation and site catalogs and the use of the vds-get-sites utility for populating the catalogs from OSG ITB and Production grids. * Provided tutorial support for Britta Daudert on voms-proxy-init and vds-get-sites utility.
  • Working with Martin Feller at ANL, determined that increasing the value of GLOBUS_OPTIONS to Xmx=1024 increased the robustness of ws-gram job submission.
  • Working with Suchandra at UC, configured syslog-ng for the site LIGO-CIT-ITB.
  • Provided engineering support to UWMilwaukee OSG team on ws-gram installation problem.
  • Advocated to the site administrator at LIGO-OSG-PSU to configure MonALISA? for monitoring a future production test of the Insprial analysis with ws-gram submission.
  • Installed srmcp version 1.25 required for testing srmcp between LIGO/Caltech and the SRM at UCSD/CMS.
  • Transferred nanoHIPE gravitational-wave files from LIGO to SRM at UCSD/CMS using srmcp 1.25 with a copyjobfile script.
  • Created a Twiki page documenting a procedure for increasing the stability of ws-gram on OSG 0.6.0 based upon VTB experience.
  • Executed nanoHIPE on TTU_ANTAEUS in 28 minutes via the ws-gram interface.

Period: March 2007

  • Participated in OSG All Hands Meeting at SDSC.
  • Presented tutorial "Hands-On Using OSG (client)"
  • Validated new 0.6.0 installation on OSG_LIGO_PSU with nanoHIPE
  • Fixed OSG Twiki documentation on Locating Storage Elements so that examples are valid with Glue 1.2 schema in the case of Storage URLs.
  • Documented on OSG Twiki how to set site specific attributes to overwrite incorrectly configured attributes.
  • Setup new catalog, scripts and pegasus components for testing new algorithm for producing and supporting dynamic cleanup dags.
  • Documented local cache of gravitational-wave files on LIGO-CIT-ITB for ISI engineer to support testing of dynamic cleanup algorithm on LIGO cluster.
  • Attended weekly attributes, DASWG, ITB telecoms.
  • Tested Java ldap browser as a tool for examining Glue values at aggregation point.
  • Reconfigured LIGO app as a ws-gram workflow.
  • Successfully executed the ws-gram enabled LIGO app on LIGO-CIT-ITB.
  • Determined that the following three 0.6.0 sites are unable to support ws-gram jobs: OSG_LIGO_PSU, TTU_TESTWULF, UWMilwaukee.
  • Restarted globus-ws daemon twice on LIGO-CIT-ITB after NERSC engineer was able to crash the daemon while testing. Stu Martin at Globus is aware of the problem.
  • Worked with ISI/Pegasus engineer on plotting dynamic clean up algorithm performance to determine effectiveness of technique.
  • Updated Burst MatLab? test code DAX for compatibility with VDS 1.4.7.
  • Provided two hour tutorial for new hire on Globus architecture.
  • Investigated problem with the srm testing framework at Indiana related to UCSD SRM.
  • Completed survey requested by FNAL on Glue attributes and cluster configuration.
  • Reviewed list of OSG 0.8.0 deliverables.
  • Analyzed results from Pegasus development efforts to dynamically cleanup storage during LIGO Inspiral workflows.
  • Provided two hour tutorial on LIGO Workflow planner and OSG for new LIGO employee.
  • Setup OSG grid middleware environment on LIGO submit host for new LIGO employee.
  • Attended attributes subcommittee, and Pegasus telecoms.
  • Configured Pegasus workflow planner to create dynamic cleanup DAG nodes for nanoHIPE application.
  • Chaired ITB weekly telecom.

Period: February 2007

  • Downgraded to Condor 6.8.4 on LIGO submit host due to bug in Condor 6.9.1
  • Provided support to Michael Samidi for LIGO OSG milestone.
  • Reviewed documentation on US Daylight Savings Change required on computer systems before March 11, 2007.
  • Provided support for John Rosheck on CEMon/Glue problems in VDT 1.6.1.
  • Installed OSG 0.4.1 on tclproxy for additional submit host capabilities at LIGO.
  • Discovered and reported bug in GlueHostNetworkAdapterOutboundIP? attribute in VDT 1.6.1a
  • Adjusted GRIDMANAGER_MAX_JOBMANAGERS_PER_RESOURCE to value of 50 for Condor-G on LIGO submit hosts.
  • Segmented SiteValidationTableITB052? into three categories: Sites Ready for VO App. Testing, Sites in Preparation for VO testing, and Development sites to better communicate the diverse functionality of sites documented on the Twiki page.
  • Examined a high load average on LIGO cluster head end.
  • Worked with Tanya Levshina on new raw CEMon plugin for supporting BDII information.
  • Tested ws-gram on LIGO-CIT-ITB for Stu Martin.
  • Discovered problem with CVS update of LIGOWorkflowPlanner? on tclproxy. Requested Michael Samidi check in and tag a new release.
  • Updated LIGO-CIT-ITB and LIGO-CIT-VTB clusters and tclproxy host with latest Daylight Savings Time OS patches and Java Daylight Savings Time patch.
  • Documented completion of OSG application milestone running 100 jobs continously for one week on OSG resources using the HIPE gravitational wave analysis.
  • Worked with Alain Roy and Tim Cartwright for two days on pacman -update ce:ITB-052 problem on LIGO-CIT-ITB.
  • Reinstalled ITB-052 from scratch on LIGO-CIT-ITB after completion of pacman -update debugging sessions.
  • Validated UC_T2DEV_ITB with nanoHIPE validation workflow bringing to five the number of ITB sites that have successfully validated wth the LIGO VO app.
  • Provided consultation to Ion at CompBioGrid? to bootstrap his VO validation efforts.
  • Documented OSG efforts of Michael Samidi to provide continuity due to his resignation.
  • Added support for additional VOs: GUGrid, gpn, compbiogrid and engage on LIGO-CIT-ITB.
  • Analyzed preliminary report from Michael Samidi on milestone effort and requested additional documentation to clarify error rate and type of failure at each site.
  • Began new effort to understand open issues related to interoperability between SRM and the Pegaus planning of workflows on the UCSD CMS site and SRM site.
  • Validated TTU_TESTWULF (PBS site) with LIGO app on OSG 0.5.2
  • Six sites now have passed the OSG 0.5.2 validation with the LIGO app.
  • Kept the six "Sites Ready for VO Application Testing" in a high state of readiness to support VO app testing.
  • Worked with Saul Yossef on pacman -update bug in VDT 1.6.1x
  • Worked with Nanohub VO validator to support validation of OSG 0.5.2
  • Prepared slides for OSG All Hands session on "Hands-on Training on the OSG client"
  • Attended weekly DASWG, ITB, Pegasus, VDT and VTD telecoms.

Period: January 2007

  • Installed VDT 1.6.0 on VTB CE
  • Successfully executed site_verify script on VTB CE with VTB 1.6.0
  • Installed Condor 6.8.2 on new node1 of VTB
  • Yum install compat-libstdc++-33.i386
  • Attended meeting on acknowledgments for paper submission.
  • Attended meeting called by Kent Blackburn regarding benchmark testing of Pegasus workflow planning of Inspiral analysis on LIGO CIT cluster.
  • Updated kernel and kernel-smp to 2.6.17-1.2142_FC4 on VTB CE and node1
  • Attended ITB, VDT, VTB, Pegasus and DASWG weekly telecoms.
  • Configured NFS support for VDT and Condor on VTB CE and node1
  • Configured chkconfig for NFS, NTP and Condor daemons on VTB CE and node1
  • Attended telecom with Frank Wuerthwein, Kent Blackburn and Michael Samidi on status of milestone workflow at UCSD.
  • Installed and configured Condor 6.8.3 on VTB CE and node1.
  • Tested condor_submit to Condor 6.8.3 - ok
  • Installed VTB:ce-161 & VTB:Globus-Condor-Setup-161 and configured.
  • Attended weekly Pegasus and VTB telecoms. Registered LIGO-CIT-VTB with GOC.
  • Attended weekly DASWG telecom. Tested VTB:ce-161 components.
  • Attended weekly ITB telecom. Planned validation tests for Gratia & CEMon.
  • Worked with John Roshek to configure LIGO-CIT-VTB monitoring with GridScan?.
  • Reviewed vdt-control and vdt-register-service docs.
  • Configured Gratia & CEMon.
  • Documented Gratia & CEMon validation results on Twiki.
  • Provided Chris Green and Philippe Canal with debug info on Graita config.
  • Worked with LIGO and campus facilities on shutdown and restart of OSG resources due to loss of chilled water in machine room.
  • Reviewed milestone statistics provided by Michael Samidi on LIGO application executing on UCSD/OSG site.
  • Refined draft on IEEE paper on LIGO/OSG efforts for LIGO review process.
  • Examined excess fork jobs executed by Nanohub on LIGO-CIT-VTB.
  • Installed VTB:ce-161 & VTB:Globus-Condor-Setup-161 three times as refinements to osg-configure.sh script were made.
  • Attended LIGO Computing Committee meeting held at CIT.
  • Attended special telecom called by Alain Roy on planning for BDII support in CEMon.
  • Successfully validated installation of CEMon and Gratia Probe with Condor job-manager using VTB:ce-161 on Friday Jan 19 on LIGO-CIT-VTB test bed.
  • Installed ITB:ce-161 & ITB:Globus-Condor-Setup on LIGO-CIT-ITB
  • Confirmed proper operation of Gratia, CEMon and site_verify.pl script on LIGO-CIT-ITB.
  • Attended Virtual Workshop
  • Updated SiteValidationTableITB052? and related pages on Twiki
  • Annotated GridEx? and LIGO validations on SiteValidationTableITB052?.
  • Worked with CEMon developers on memory except thown by Java related to Tomcat.
  • Validated the LIGO app on LIGO-CIT-ITB and began an analysis of problems at sites: BNL_ITB_Test1 and CIT_ITB_1 when attempting to execute the LIGO app.
  • Revised a draft of an IEEE conference paper on Scheduling Data Intensive Workflows.
  • Reviewed status of a problem of using Condor submit scripts on Worker Nodes without home directories as a location for srmconfig.xml file.

Period: December 2006

  • Debugged Gratia Accounting Probe with FNAL Developer P. Canal
  • Installed VDT 1.5.2 on Validation Test Bed
  • Installed VDT 1.5.3 on Validation Test Bed
  • Debugged OSG client with Alain Roy on FC4
  • Chaired ITB weekly meeting (2X).
  • Met with OSG Applications Manager on progress and issues.
  • Met with OSG Integration Manager on progress and issues.
  • Moved ITB cluster and VTB hardware to new position on machine room floor.
  • Worked with LIGO staff on power re-routing required that impacted ITB cluster and VTB hardware.
  • Worked with Abhishek (UCSD) and Karan (ISI) to verify that Karan is properly mapped to OSG VO and that Karan can run jobs as a member of the OSG VO at UCSD (For future application work by ISI).
  • Designed GUI mock up (html) for potential VORS interface for John Roscheck.
  • Attended the scheduled weekly meetings: LIGO DASWG, ISI Pegasus, OSG ITB and Troubleshooting and VDT.
  • Installed FC4 os on node1 of the VTB testbed and configured, cabled and tested network interface.
  • Assisted Michael Samidi in configuring Inspiral HIPE code for testing local cache at UWM, FNAL and UCSD.
  • Tested srmcp between host at CIT and srm server at UCSD. Wrote and distributed engineering notes for staff and ISI Pegasus team.
  • Determined that the path to the SRM directory @ FNAL does not have appropriate permissions for LIGO VO members. Referred to FNAL staff for review.
  • Attended Troubleshooting WG weekly meeting.
  • Replaced expired host cert on LIGO OSG submit host.
  • Reviewed srmcp v1 and v2 docs regarding srmls support.
  • Assisted Michael Samidi with milestone data collecction using ML for application milestone deliverable.
  • Shutdown non-essential engineering computer for the holiday break.

Period: November 2006

  • Attended meeting called by OSG extensions manager.
  • Provided Ewa Deelman at ISI with stats on run-time of nanoHIPE partition 1.
  • Worked with Alain Roy on problem installing OSG:client package related to PyGlobus?.
  • Checked FW rule set on LIGO-CIT-ITB against latest OSG ITB recommendations.
  • Opened trouble ticket with GOC on VORS server down (since fixed).
  • Configured CEMon on LIGO-CIT-ITB
  • Configured GIP on LIGO-CIT-ITB.
  • Requested space allocation for storage of a local cache of 220 GB gravity wave files at UWMilwuakee.
  • Constructed a local cache of 220 GB of gravity wave files at UCSD.
  • Created an account on RT trouble ticket system (requested by Rob Gardner).
  • Provided consulting to Suchandra Thapa, a new OSG member of the VTB team at the University of Chicago.
  • Extended the LIGO workflow planner to incorporate the LDAS-GRID cluster at CIT as an additional site.
  • Benchmarked LDAS-GRID, STAR-BNL, FNAL_GPFARM and STAR-BNL with nanoHIPE workflow.
  • Used the LIGO workflow planner and nanoHIPE to validate OSG 0.5.1.
  • Updated the SiteValidationTable051? to use the new Twiki icons and annotated several validation columns.

Period: October 2006

  • Develop horizontal partitioning technique to manage disk storage reqs. on OSG sites.
  • Meet with Shourov Chatterji of the Burst analysis group to gather requirements for guided/all sky analysis.
  • Work with the Pegasus and VDT teams to incorporate VDS 1.4.7 in VDT 1.5.1.
  • Chair the OSG ITB telecom.
  • Upgrade LIGO-CIT-ITB to OSG ITB release 0.5.1
  • Wrote a grid script to find and remove the gravity wave files in a Pegasus Partition for Michael Samidi.
  • Developed an OSG Twiki page on ITB Documentation Best Practices.
  • Developed a DASWG OSG-LIGO web page to document OSG activities for the LIGO community.
  • yum updated kernel on LIGO-CIT-ITB.
  • Added Problem Reports to web problem report tracking on LIGO workflow planner app.
  • Planned activites with Pegasus development team related to next phase of OSG extensions development.

-- DavidMeyers - 30 Nov 2006

Topic revision: r14 - 22 Feb 2012 - 16:36:19 - KyleGross
Hello, TWikiGuest
Register

OSG Reports
  • add items

Meta-TWiki links

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..