OSG Council Monthly Teleconference

• Teleconference Phone Number: 1-510-665-5437
• Meeting ID: 1111
• Date: December 9th, 2008
• Time: 10:00 AM Pacific Time, 11AM Mountain, Noon Central, 1PM Eastern
• Attending: Chander Sehgal (OSG Project Manager), Jim Weichel (Fermilab), Brad Abbott (DZero), Michael Ernst (US ATLAS), Rob Quick (GOC), Paul Avery (OSG Council Co-Chair), Kent Blackburn (OSG Council Co-Chair)

Agenda/Minutes

Discussion with Lisa Childers of CDIGS on the User Perspective Study

• Report Download Location

Minutes:

Lisa Childers presented to the Council on the report available at the URL above. Lisa has been a member of the GLOBUS team at Argonne for 6 years. The intent of the study is to better understand the requirements of the users for distributed computing, instead of having the technology, developers and implementation drive the requirements. CDIGS as part of the report generation process conducted 30 interviews. These are included in the report. They had hoped to carry out more but the time involved in the interview process was underestimated. The users from the interviews were separated into 4 groups for the purpose of study. OSG is captured in the report via interviews with the Engagement VO which was categorized for the purpose of the report as a general purpose HPC infrastructure provider, see section 4.4. The report conveyed a clear message, that is that all users and communities are strongly interconnected in their issues, e.g., scientists use and work with general purpose infrastructure providers. Didn't have the opportunity to interview users of science portals. Hope that this report will provide awareness that the use of the term "user" can have many meanings with many different needs and goals.

Chander asked what were the goals or hopes for what would be done with the resulting report. Lisa hopes that this will result in people creating technology that is better suited to the users of that technology. Kent asked if technology was defined here, e.g. is it GLOBUS or does it need to be defined? This really depends on the user group in question. There are typically many layers of technology that go into the interface a particular user group faces. Focused on what the users were trying to accomplish to avoid implementation details, features deadlines, etc. Hope developers will read this and be more sensitive to the users requirements in the future. Chander asked if the survey identified the different scales of users as is seen in the OSG's VOs and if this was identified in the survey. Lisa said that it was and that the complexity of dealing with this comes from the focus of developers on features, at least in the case of GLOBUS - its implementation complicates delivering to the users. Domain developers are focusing on general purpose infrastructure, as least for their domain and its user communities. Wasn't able to interview these higher level domain systems (e.g. portals). Domain specific developers have a clearer understanding of the needs of the users, but may have less of an understanding of how to implement. It is a real challenge to bridge these two layers of developers with users. Lisa reported that it is clear from the interviews that OSG is filling a really important place for the advancement of science.

The council thanked Lisa for presenting to the council, for constructing this enormous report and for sharing it with the OSG.

After Lisa signed off Jim Weichel noted that they did a very good job of collecting a huge amount of data and did a very good job of summarizing the results. He also noted that the interviews were well represented in the summary and nothing in the summary wasn't found in interviews. Appendix C, about 40 pages captures the bulk of the content of the report. There is a lot of similarity between what OSG has been struggling with and what is found in the report. Jim has collected a matrix of OSG focused elements from the report. The council hasn't seen this matrix from Jim yet. Kent will provide a copy to the council and the council can respond to the content at a future meeting. The matrix identifies 14 items, all of relevance to the OSG. Chander believes that the OSG knows of all of these and it isn't too much of a surprise. Chander gave as an example the situation with documentation for users as an area of user concern that appears in this document and also applies to OSG.

Status Report from the OSG Executive Director

• Next Report to Coucil will include (Ruth's email from November 12th, 5:23AM PST):
  1. "OSG Chapter for LSU book on production grids"
  2. "Franks talk to Nuclear Physics and Alice Task Force"
  3. "Report to the US LHC Joint Oversight Group"

Minutes:

No report this meeting, see Ruth's emails to the Council.

Discussion of "OSG Value Proposition" document to be distributed to JOT and external reviews

• Jim Weichel is the lead on this document.
• See Executive Report circulated in email on December 8th, 2008 for details
• Jim Weichel's condensed summary of OSG relevant components discussed today are attached at the bottom of this webpage

Minutes:

Jim walked through the value proposition and the plans to use it. Document was initiated from an outsiders point of few to find the major areas of value from the OSG. Rough draft generated of where OSG was providing value and this was shared with members of the OSG and interviews were carried out to discuss content. The document has been restructured and edited a couple of times. Each section now has a dollar estimate captured. OSG is providing a significant amount of head count in excess of what would be needed to accomplish the same goals if the partners and stakeholders were talking on the same work. In fact, it is about a factor of two of equivalent head count for the work over the funded FTEs within the OSG. Chander asked that the experiments agree with this or if they have a better way of doing this Jim and Chander would like to hear about it. Kent asked if the opportunistic install computing value of $3M was per year. Jim said it was. Kent noted that the statements of work care non funded contributions of FTEs. Chander reported that the leverage ratio in the SOWs is about 50%. Kent noted that a lot of these people are at a small fraction of an FTE suggesting that the communities are interested in being involved in the OSG and its directions. Kent also noted that it would be good to have a crisp representation of long term extrapolation over time. The experiments involved with OSG are long term and a statement of their longevity and their involvement with OSG would be useful for demonstrating this. Jim noted that the OSG could be used as experiments spin up and down to bridge across the long term.

Michael Ernst and Brad Abbott offered to read through this document and provide feedback to Jim, Chander and Ruth.

Kent asked about the plans for using this document and if the Council would be given the a preview of what is to be shared with the JOT reviews.

Plans for Joint DOE/NSF Review Next Month

• January 14th, 15th 2009: NSF Building: Arlington VA

Minutes:

Chander reported that Don Petravic provided a draft charge. This has been parsed and 4 sections for reporting have been assigned out of this. The 4 groups covering the 4 sections have begun to work on this. There will be a mock review on January 8th via video conference. There are some indications that the charge will evolve. Chander believes that the reviewers are identified by the agency but according the SciDac? review procedures we are not to be known by us. Most individuals to be involved in the review have been notified. Working with LIGO to identify person from LIGO to be at the review.

Brought up the annual report to the DOE. Chander has started the internal canvas to generate the report. Paul will do the external report among the experiments. This will cover January through December. This is 6 months out of phase with the NSF report. The report will be sent to the DOE and shared with the NSF. Haven't started pulling together list of items of backup items of documentation to provide to the JOT Review. Noted that there ill only be one more council meeting before the JOT Review. This report is due by the end of this year.

Round-Table Virtual Organization Reports and Issues - Ongoing Goals/Milestones

Minutes:

Brad (DZero):

First the good news, the Monte Carlo production has been very good for the past month highest or second highest efficiency. This has been attributed to storage element improvements as well as from internal tuning changes within DZERO. In the past it has been noted that efficiencies can drop and will need to wait to see what happens down the road.

Another issue is that there is a class of jobs that involve high intensity, large cpu usage. There have been issues with keeping track and monitoring of these jobs. It is difficult to keep track of the state of the job and what is typically done, is the person running the jobs calls the system administrator to ask what is going on. They have a limited set of sites they are running on, limited by the fact that they have phone numbers of system administrators of a select set of sites. Mention that the OSG All-Hands will have workshops on application deployment and on data management.

Michael (US ATLAS):

Request from WLCG to provide direct report on installed capacity, mostly in the from of cpu and storage resources onto WLCG as a whole. This is based on a reporting through the Grid Reporting System and requires a lot of implementation additions to what we currently have. A document has been prepared and distributed. Together, US ATLAS and US CMS are working with OSG on this. There was a tight deadline that has already expired (yesterday) on this. OSG has asked for an extension. Working to make sure this is available in time to the LHC stakeholders.

Another issue in the context of ATLAS world-wide is the desire to have direct ticket routing for people running computer shifts at CERN. Rob Quick has taken care of the organization and communications issues and this is well underway.

Use of Local Replica Catalog (LRC) has caused some issues. ATLAS made a fundamental decision to migrate to LHC File Catalog (LFC). OSG was asked to integrate this LFC into VDT. This was done on record time and provides for the same catalog everywhere. A small problem was found in the LFC code was found and VDT immediately updated the code to distribute the fix.

A couple of ATLAS activities coming up. ATLAS has collected 460 million cosmic events (1.2PBytes). This has been distributed to Tier I's world wide according to quotas. US ATLAS has received 300 TBytes of these events to reprocess. ATLAS will begin a world wide reprocessing exercise using these events. This will involve the Tier I and Tier 2 centers as well. Test that Panda is prepared to handle this request. Currently running validation and will start reprocessing in January. This is quite a sizable exercise.

Also critical to ATLAS is Distributed Data Management technology. ATLAS launched today a world wide stress test of DDM involving resources everywhere. Intended to stress test large numbers of transactions needed to replicated large numbers of files.

US ATLAS is involved in simulation production, but are now awaiting a new software release so there is a limited demand world wide. Working to make sure facilities are ready production quality. Shortly this will become very busy.

Analysis will be ramping up as soon as the derived data products from the cosmic reconstruction data is available.

Rob Q. (GOC):

Working with ATLAS to get the ticketing from end point to end point without human intervention so that an ATLAS or possibly a CMS shifter could fill out a single form that makes it through all associated ticketed systems in the chain and onto the doorsteps of the admin that is responsible.

Paul:

Will be working on annual report to DOE.

Kent:

Mostly making arrangements for the All-Hands Meeting to be at LIGO Livingston Observatory. Chander asked about which airport to fly into. This really is a cost decision. New Orleans is about an hours drive away from Baton Rouge but there are direct flights and it is cheaper from the west coast, plus you have to make a stop going into Baton Rouge from the west coast and may face weather problems. On the other hand Paul reported that Baton Rouge is cheaper for him coming from Florida.

Baton Rouge is where the hotels for the meeting are located so if you can go that way you'll be closer to your hotel.

AOB

-- KentBlackburn - 11 Nov 2008

• ANL_Survey_Recommendations-1.doc: OSG relevant recommendations from CDIGS (ANL/GLOBUS) survey

• ANL_Survey-1.xls: Fourteen germane finding overlaping OSG and CDIGS Report