[jira] [Commented] (YARN-2069) CS queue level preemption should respect user-limits

["Wangda Tan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054334#comment-14054334 ] 

Wangda Tan commented on YARN-2069:
----------------------------------

Hi [~mayank_bansal],
Thanks for your comments,
I think the change of title/description should be correct, this patch is targeted to solve cross-queue preemption should respect user-limit.

I think your other comments all make sense to me. Only below one,
bq. We need to maintian the reverse order of application submission which only can be done iterating through applications as we want to preempt applications which are last submitted.
IMHO, this is reasonable but conflict with this JIRA's scope, let me give you an example. 
Assume a queue has 10 apps, each app has 5 containers (1G for each container, so queue has 50G mem used). There're two apps, each app has 5 apps. User-limit is 15G, queue's absolute capacity is 30G.
And first 5 apps belongs to user-A, last 5 apps belongs to user-B.
In your correct method, user-B will be preempted 20 containers and user-A will be preempted nothing.
After preemption, only 5 container left for user-B, and 25 containers left for user-A. User-limit is respected here.

Does this make sense to you?

Thanks,
Wangda

> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>
>                 Key: YARN-2069
>                 URL: https://issues.apache.org/jira/browse/YARN-2069
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch, YARN-2069-trunk-3.patch
>
>
> This is different from (even if related to, and likely share code with) YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed capacity, it's individual users are treated in-line with their limits irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to balance queue capacities.



--
This message was sent by Atlassian JIRA
(v6.2#6252)