relaxLocality in ContainerRequest

[Maysam Yabandeh <m....@gmail.com>]

Hi,

According to the java docs, this is what relaxLocality supposed to mean:
/*
 * @param relaxLocality
 *          If true, containers for this request may be assigned on hosts
 *          and racks other than the ones explicitly requested.
 */

However, the way it is used in AMRMClientImpl.java implies differently:
{code:java}
      for (String node : dedupedNodes) {
        addResourceRequest(req.getPriority(), node, req.getCapability(),
req,
            true);
      }
{code}
where, relaxLocality for nodes is always set to true, ignoring the original
relaxLocality set by the user. Since this is introduced from the very
beginning at YARN-521, I do not whether it is a bug or the title
relaxLocality is confusing.

Thanks
Maysam

Re: relaxLocality in ContainerRequest

[sa...@cloudera.com]

Hi Maysam,

The code and doc are both correct. A node-level ContainerRequest ends up decomposing into 3 ResourceRequests, one for the node, one for the rack, and one for anywhere. The way scheduling works is that when a node is offered to an application, it checks to see whether the application has outstanding requests in the order node, rack, *. If it hits a ResourceRequest with relaxLocality turned off, it won't use it. Check out the docs on ResourceRequest#setRelaxLocality or the discussion on YARN-392 for more details on how this works.

-sandy

(iphnoe tpying)

On Aug 17, 2013, at 1:07 AM, Maysam Yabandeh <m....@gmail.com> wrote:

> Hi,
> 
> According to the java docs, this is what relaxLocality supposed to mean:
> /*
>  * @param relaxLocality
>  *          If true, containers for this request may be assigned on hosts
>  *          and racks other than the ones explicitly requested.
>  */
> 
> However, the way it is used in AMRMClientImpl.java implies differently:
> {code:java}
>       for (String node : dedupedNodes) {
>         addResourceRequest(req.getPriority(), node, req.getCapability(), req,
>             true);
>       }
> {code}
> where, relaxLocality for nodes is always set to true, ignoring the original relaxLocality set by the user. Since this is introduced from the very beginning at YARN-521, I do not whether it is a bug or the title relaxLocality is confusing.
> 
> Thanks
> Maysam