[jira] [Updated] (YARN-2056) Disable preemption at Queue level

["Eric Payne (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Payne updated YARN-2056:
-----------------------------

    Attachment: YARN-2056.201408202039.txt

This patch keeps the {{yarn.resourcemanager.monitor.capacity.preemption.max_ignored_over_capacity}} property as a global parameter, and then adds a per-queue property in this format: {{yarn.resourcemanager.monitor.capacity.preemption.<queue-path>.max_ignored_over_capacity}}

The preemption code makes two sets of passes through the queues. The first time through, it calculates the ideal resource allocation per queue based on normalized guaranteed capacity, and the second time through, it selects which queue's resources to preempt, taking into consideration the {{max_ignored_over_capacity))

In this patch, the per-queue {{...max_ignored_over_capacity}} is taken into consideration in the first pass to help determine which queues have resources available for preempting. This is necessary because without it, queues that could fulfill the need would otherwise be removed from the list of available resources. Then, for the second pass, the global {{...max_ignored_over_capacity}} setting is used, as before, to determine which resources out of the remaining available resources to use.

This patch still requires an RM restart if the queue properties have changed.

> Disable preemption at Queue level
> ---------------------------------
>
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt
>
>
> We need to be able to disable preemption at individual queue level



--
This message was sent by Atlassian JIRA
(v6.2#6252)