[jira] [Commented] (YARN-2009) Priority support for preemption in ProportionalCapacityPreemptionPolicy

["Eric Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096538#comment-15096538 ] 

Eric Payne commented on YARN-2009:
----------------------------------

{quote}
Sunil G, pardon the long delay... From what you say it seems like the priority issues within queue it is important for you and you observe non-trivial delays.
If that's is the case, I think it is fine to venture in adding within-queue cross-app preemption.
...
if preemptions triggered by cross-queue imbalances it would be good to "spend" them to correct the issue you observed.
{quote}
Thanks a lot [~curino] and [~sunilg]. Good discussion.

I feel that we should separate out these two concepts into separate JIRAs:
- In-queue container preemption based on app priority
- cross queue preemption that takes into consideration priority of the container(s) being preemption compared among all containers in the preempted queue.

They have their own set of challenges, and each is significantly complicated that it seems like a logical separation.

Regarding unnecessary preemption because of locality, labels, etc., I think the preemption monitor now considers labels when determining which containers to preempt, so that should be extendable for priority, but locality is still an issue.

And, of course, all of this will probably have to be re-evaluated in light of HDFS-4108.

> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>
>                 Key: YARN-2009
>                 URL: https://issues.apache.org/jira/browse/YARN-2009
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Devaraj K
>            Assignee: Sunil G
>
> While preempting containers based on the queue ideal assignment, we may need to consider preempting the low priority application containers first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)