[jira] [Commented] (YARN-4945) [Umbrella] Capacity Scheduler Preemption Within a queue

["Eric Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15389828#comment-15389828 ] 

Eric Payne commented on YARN-4945:
----------------------------------

HI [~leftnoteasy]. Sorry for the long delay. I am now starting to have more time to focus on putting a design together and getting a POC working for in-queue preemption.

{quote}
- When intra-queue preemption can happen: in some cases, we need intra-queue preemption happen when queue is under its guaranteed resource,
{quote}
Just for clarification, I think that if resources are available in the cluster and a queue can get more resources by growing the queue's usage, then in-queue preemption shouldn't happen. However, if something in the queue's hieararchy has reached its absonlute max capacity or if the the cluster itself is full, then in-queue preemption should happen, even if the queue is under its guaranteed resource max.

For {{queue X}}, this should happen when all of the following occur:
# some set of resources (memory, vcores, labelled, locality, etc) are all used, either by other queues or apps in {{queue X}}
# any user in {{queue X}} is over its minimum user limit percent
# another user in {{queue X}} is under its minimum user limit percent and asking for resources

Having said that, the question of whether a queue can grow its usage by allocating available resources is complicated by the same issues that plague cross-queue preemption such as labelled resources, locality, fragmented memory, and so forth.


> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
>                 Key: YARN-4945
>                 URL: https://issues.apache.org/jira/browse/YARN-4945
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>
> This is umbrella ticket to track efforts of preemption within a queue to support features like:
> YARN-2009. YARN-2113. YARN-4781.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org