OSG Resource Selection Activity

User and Administrator Documentation

• CEMon Troubleshooting Guide for Administrators
• Useful Links For CEMon Validation
• Information Gatherer Installation and Configuration Notes(May 13-2008)
• How to download and build osg-ce-plugin from ETICS(May 13-2008)
• Glue Schema Syntax (Aug 28-2007)
• ReSS User Interface Tools (Feb22-2007)
• Integrating Condor-G with ReSS
• CEMon And GIP Installation Notes (Obsolete: see OSG installation instructions)

System Deployments and Monitoring

• Historical view of the OSG Integration and Production deployments (Mar 2007).
• Validation page of the classad attributes in ReSS for the OSG Integration and Production deployements (Nov 2007). See also the Validation Mechanism documentation.
• FermiGrid uses ReSS for campus-level cluster selection (Mar 2007).
• OSG Engagement VO uses ReSS for VirtualOrganizations?/VOInfo-oriented resource selection in OSG (May 2007).

System Evaluations

• ReSS Validation Test (Oct06-2006)
• CEMon Performance Evaluation on an OSG test environment (Dec14-2006)
• ReSS Evaluation by USCMS (Mar07-2007)

Publications

• G. Garzoglio, T. Levshina, P. Mhashilkar, S. Timm, "ReSS: a Resource Selection Service for the Open Science Grid.", Proceedings of the International Symposium of Grid Computing (ISGC07), March 2007, Taipei, Taiwan (pdf)

Design & Development

• CEMon Collaboration
• OSG CE Plug-in
• Matching List Elements With Condor (Obsolete)
• TestBed Prototype
• Collaboration Meeting March 2006 (March14-2006)
• Glue Schema And ReSS
• Collaboration Meeting May 2006 (May31-2006)
• Panda Integration With ReSS: a one day meeting discussing how to integrate the Atlas job management system with ReSS (Aug04-2006)

Project Management Documents

• Charter
• DZero Requirements
• Project Plan
• ReSS Architecture
• ReSS Phase1 Project Closure Report

Project Charter

OSG Resource Selection and SAM-Grid interoperability

Date: Sep 1, 2005
Authors: Gabriele Garzoglio, Amber Boehnlein, Adam Lyon, Ruth Pordes, Dane Skow

The Open Science Grid is building a US national grid infrastructure for multiple scientific communities. Dozens of computing centers and universities provide access to computing, storage, and network resources via standard grid interfaces and protocols. Users currently submit jobs directly to these resources, selecting them before job submission and specifying all relevant resource attributes in the job description. The necessity of a human intervention in resource selection and attribute specification hinders external automated job management components from accessing OSG resources. Several Virtual Organizations would benefit from a Resource Selection Service (RSS) that could automatically select OSG resources based on the job requirements. Providing such Resource Selection Service is one of the goals of this project.

The SAM-Grid implements one such job management component for the DZero and CDF experiments VO. The component has two responsibilities: (1) application management, including job environment preparation, and (2) resource management. Currently, the SAM-Grid can only access resources where some of its components have been deployed. Among other things, these components report to the SAM-Grid the necessary information to perform resource selection. The SAM-Grid will be able to delegate such task to the new OSG RSS, thus eliminating the need for deploying SAM-Grid directly on all OSG resources.

The OSG RSS will satisfy the requirements of the SAM-Grid system and it will be offered as a general service for those VirtualOrganizations?/VOInfo that have similar requirements. The project aims at delivering the functionalities needed by the SAM-Grid for resource selection early 2006. These functionalities include basic resource ranking algorithms (such as random selection). The RSS will provide an extensible framework to introduce new ranking algorithms, should VirtualOrganizations?/VOInfo be interested in them.

Another goal for this project is to reduce as much as possible the dependence of the SAM-Grid on the implementation of the RSS. The SAM-Grid will be able to integrate different resource selection systems that meet the SAM-Grid requirements.

In summary, the project has two main goals:

1) deploy a system that makes the OSG resources accessible by the SAM-Grid. The system will have architecture similar to the SAM-Grid / LCG integrated system. The architecture will have provisions to allow different RSS implementations.

2) providing OSG with a general resource selection service. Such service has two main responsibilities: (a) allow users to reference resource attributes from the job description, in order to express job requirements and automate the substitution of the resource attribute values in the job description; (b) select the optimal resource, if more than one resources satisfy the job requirements, using an extensible framework to define selection criteria. The project will focus on (a) and provide basic implementations of the resource selection algorithm (such as "random") for (b).

The program of work and efforts for this project is sponsored by the PPDG Common Project and the Fermilab Computing Division. The project progress will be tracked and discussed through the PPDG Common Project meetings.

The project will deliver:
- a resource selection system integrated with the OSG information system and resources.
- an infrastructure capable of forwarding jobs prepared by the SAM-Grid to the OSG resources

The project will not deliver:
- job scheduling
- maintenance for the services
- "intelligent" resource selection mechanisms

Site Tools of the ResourceSelection? Web

*

*Notes:

• You are currently in the ResourceSelection web. The color code for this web is this background, so you know where you are.
• If you are not familiar with the TWiki collaboration platform, please visit WelcomeGuest first.

*

Web	TWiki Site Map	Use to...
Main	Welcome to TWiki ... Users, Groups	...see who is registered on this TWiki
TWiki	Welcome, Registration, and other StartingPoints; TWiki history & Wiki style; All the docs...	...discover TWiki details, and how to start your own site.
ArchivedDocumentation  ?
ArchivedDocumentation.ITB.ITB050		...collaborate on
ArchivedDocumentation.ITB.ITB070		...collaborate on
ArchivedDocumentation.ITB.ITB090		...collaborate on
ArchivedDocumentation.OSG  ?
ArchivedDocumentation.OSG.OSG041	OSG Release Documentation	Provide current release documentation for the OSG
ArchivedDocumentation.OSG.OSG060		...collaborate on
ArchivedDocumentation.OSG.OSG080		...collaborate on
CampusGrids		...collaborate on
Council		...collaborate on
Deployment	Temporary Deployment Web	...collaborate on
Documentation	OSG Web site development for Production and Public Users	...collaborate on the development of web information and layout
Engagement		...collaborate on
InformationServices		...collaborate on
Integration	OSG Integration Activity development and testing of grid software on the OSG Integration Testbed	...collaborate on OSG Software testing and testbed development
Integration.ITB090		...collaborate on
Integration.ITB110		...collaborate on
Interoperability	Addresses issues of interoperation between OSG and other grids, especially the LHC Computing Grid (LCG) and Tera Grid, including technologies, interfaces, tests and demonstrators.	...collaborate on defining the areas which allow interoperablity
MeasurementsAndMetrics		...collaborate on
OSGRA		...collaborate on
ReleaseDocumentation		...collaborate on
Sandbox	Sandbox test area with all features enabled.	...experiment in an unrestricted hands-on web.
Sandbox.SubWeb		...collaborate on
Security	Security Technical Group	...collaborate on OSG security issues.
SiteCoordination		...collaborate on
SoftwareTools
Troubleshooting		...collaborate on
VO	A Web for VOs	...collaborate on
VirtualOrganizations		...collaborate on
VirtualOrganizations.Forum	Discussion forums (TWiki application)	...discuss various topics
VirtualOrganizations.VOInfo	A Web for VOs	...collaborate on
You can use color coding by web for identification and reference. This table is updated automatically based on WebPreferences settings of the individual webs. Contact osg-twiki@opensciencegrid.org if you need a separate collaboration web for your team. See also AdminToolsCategory?.

Legend of icons:

= Go to the home of the web = Search the web

= See recent changes in the web = Subscribe to get notified of changes by e-mail

-- GabrieleGarzoglio - 30 Sep 2008