Q&A on the Use of the GLUE Schema for Resource Selection

After the collaboration meeting in March, the workload management group and the ReSS? collaboration has raised a series of questions on the use of the GLUE model for resource selection. This page summarizes the questions and the answers of the groups, as of Apr 2006.

Main participants in the discussion are Gabriele Garzoglio, Sergio Andreozzi, Shaowen Wang, Eric Shook, Anand Padmanabhan, Marco Mambelli, Torre Wenaus, Frank Wuerthwein, and Igor Sfiligoi.

  • Q: Are we moving toward the paradigm of 1 CE for many VO, as opposed to 1 CE for 1 VO, which we have today?

A1: Yes. This also means that the CEInfoContactString? is going to be a real GRAM URL, as opposed to <GRAM-URL>-

A2: Yes. More in general, consider the VOView as a flexible concept for describing different viewpoints related to local policies on the same resources assigned to a queue. A VOView can be related to a single VOMS group or role of users instead of a single VO. Furthermore, it can be used to model different QoS? levels within the VO (we [Sergio Andreozzi and the GLUE schema group] are currently investigating this possibility, see this presentation)

  • Q: Do the VOView info providers know of their own VO?

A1: Yes. In principle they can gather info on the batch system status, etc., on a per-VO basis. From the examples that I've seen so far I cannot tell as there is complete info for 1 CE per VO.

A2: My understanding is that the info providers coming with LCG 2.7 are able to compute available resources that can be assigned to a job mapped to a particular UNIX GID (see again the presentation above for an example). By means of EGEE components like LCMAPS, you then map Grid-level credentials to local-level account so that the Grid-level gets the expected service when mapped to the local level. [The Privilege project provides similar mapping possibilities with the PRIMA/GUMS infrastructure for OSG].

  • Q: Are we going to move to an extended-proxy-like syntax for CEAccessControlPolicyBaseRule??

A: This is proposed in the GLUE v1.2 but has not been agreed upon neither in EGEE nor in OSG, yet. The hoope is that in EGEE some agreement should be reached in the next weeks [note: this note is of early April 06].

  • Q: What's the plan for the configuration of multiple subclusters in the GIP?
The default configuration consist of 1 cluster and 1 subcluster. System administrators do not generally fix the defaults. Doing the configuration automatically is challenging, as the information is dynamic AND not always readily available. As a result, clusters are considered homogeneous. What's the plan to move away from this situation?

A1: in reality, the information about subclusters is quasi-static, mainly because all attribute values are supposed to be equal to all hosts that are part of the subcluster.

As regards having multiple subclusters per cluster, the actual limitation is not due to dynamics of data. It is mainly due to the fact that using the GT2 GRAM, you can typically submit a job to a queue Therefore, if you want to enforce requirements given by users about HW/SW characteristics of the execution environment, you have to assume that nodes assigned to a single queue are homogeneous. By selecting the queue, you automatically select the WNs that are behind. [In this model queues are selected through a CE. Definition of a CE: "characteristics, resource set, and policies of a single queue of the underlying management system"].

[Note that this answer applies to batch systems such as pbs, but does not apply to condor, since condor does not have the concept of queue]

GT4 GRAM should solve this limitation by offering the so called "forward requirements" to the batch system. You can place a job in queue, plus you can tell the batch system to send the job to a node with certain characteristics. In this case, you can take advantage of multiple subclusters.

A2: The configuration of GIP does not automatically detect eterogeneous clusters. UIowa is trying to address this problem in the context of the MIP project http://grow.its.uiowa.edu/research/mip/. A solution proposed is an agent pre-installed at the worker nodes. This is probably in conflict with the aministrative policies of many sites.

  • Q: How do we pass job requirements to the local batch system?
A resource selection system using GLUE can select subclusters with specific requirements on the architecture of the worker nodes. How do we make the batch system aware of these requirements? A complication is that "abstract requirements" (e.g. expressed in GLUE syntax) must be translated into batch system requirements (who should do it? The job-managers?).

A: So far, GLUE Schema related attributes are not used to be passed to the batch system, but just to take the decision at an higher level. WS-GRAM is providing some new feature [detailed not known]. Furthermore, I know that in EGEE there was the intention to work on "forward requirements" to the batch system supported by the BLAH abstraction layer.

  • Q: How do we make sure that a single cluster with two CE on different machines is advertised as a single cluster and not as two?
Currently, the cluster name is the GIP node name. This is fine for most installations, but the use case in the question still needs a configuration change for a GRIS to work. In the future we'll probably see more and more CEs on different machines to address scalability limits. Should we ask the GIP installer the name of the cluster?

A1: This is an issue that LCG faced quite a long ago, but no solution was accepted. At the moment in EGEE people model them as if they are different clusters [needs to be checked]

A2: UIowa will try giving the cluster the same dn on the static GIP configuration and test if the GIIS of BDII aggregates the information correctly.

  • Q:How do we publish the information in VOView? Sergio Andreozzi mentioned that in the new classad mapping, first a "virtual" CE per VO is created, then information from VOView replaces the information from CE.
The attributes names in the LDIF representation of VOView are the same as the ones in CE. On the other hand, keeping the information of the CE as well as the VOView seems important. For example, a CE knows that the number of free slots is 10; one VO sees 10, another sees 5. If we do not keep the CE info (10 slots), we may not know how to aggregate the info from the two VO (one is tempted to see 15 slots available).

A1: if you have a CE with X VOViews, then you translate it into X+1 CEs only if there are values in the attribute GlueCEAccessControlBaseRule? that are not present also in at least one VOView, e.g.:

CE: ce1
FreeSlots: 400
ACL: /atlas
ACL: /cms
ACL: /alice

 VOView: vo1
 FreeSlots: 100
 ACL: /atlas

 VOView: vo2
 FreeSlots: 200
 ACL: /cms


CE: ce1-atlas
FreeSlots: 100
ACL: /atlas

CE: ce1-cms
FreeSlots: 200
ACL: /cms

CE: ce1
FreeSlots: 400
ACL: /alice

if all ACL at the CE level are also at the VOView level, then I [Sergio Andreozzi] do not find a valid reason (so far) for having the general CE entry.

[This is in contrast with the 2nd draft of the GLUE to old classad mapping, where VOView attributes are distinct from CE attributes and both are propagated in the same classad]

  • Q: Why there are no policy attributes in VOView? There are no attributes from CE.policy in VOView. On the other hand, the policy (e.g. max wall clock time) chages from VO to VO. What am I missing?

A: the replicated attributes are those that were found to be meaningfully "personalized" in real use cases. There might be probably also other attributes. I [Sergio Andreozzi] would suggest you to replicate what you need and document your use case. From the practical viewpoint, everything (GIP, gLite WMS) should work even though you are violating the schema definition. We can use your experience to propose extensions.

  • Q: Have we thought how to couple GLUE information with policy information. In the examples of the 2 VOs and 10 free slots, by seeing the VO information only I cannot determine how many slots I have. Keeping the CE info is a solution. A similar problem arises with multiple CEs on different machines and a single underlying batch system. How do we keep the info from the CEs consistent. For example, 2 CE are in front of a common batch system with 10 free slots: they each advertise 10 free slots; another CE is in front of another batch system with also 10 free slots. If I want to submit 30 jobs, I'm tempted to send 20 to the first batch system and 10 to the second, which is clearly not optimal. How do we couple the policy on batch system free slots allocation with the information on the CE?

A1: maybe 10 to the first and 10 to the second? this is anyway not optimal and the problem is due to the fact that resources are shared. I would say that it should be avoided to have the same VO enabled to submit to two different CE's offering the same set of resources.

If each CE is enabled to a different VO and the resources are shared without any guarantee, then this is part of the game and you cannot do much unless you have some fancy feature to reserve the slots before submission.

-- GabrieleGarzoglio - 04 Apr 2006

Topic revision: r3 - 16 Dec 2008 - 16:39:21 - KyleGross
Hello, TWikiGuest

Resource selection
  • add items

Meta-TWiki links


TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..