[jira] [Commented] (YARN-5892) Capacity Scheduler: Support user-specific minimum user limit percent

["Eric Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998847#comment-15998847 ] 

Eric Payne commented on YARN-5892:
----------------------------------

Sorry [~sunilg]. I missed your comment.

{quote}
I have few doubts related to YARN-2113 and this patch together.
   1. Give user-weight is 0, we can get user-limit for certain users as 0. Hence intra-queue preemption module could preempt all containers of user with 0 weight leaving its AM containers behind.
{quote}
If a user starts out with a weight of 0, their app will only get the AM and then never be given any more containers because the weight is also multiplied by the user limit factor. If a user has a weight of >0 and the user's apps have running containers,  and then a cluster admin "pauses" the user on a preemptable queue, then all non-AM containers will be preempted. I think this is consistent behavior.
{quote}
   2. Similar behavior could occur if there are few users in the cluster whose weights are below 1 (and together the sum of weights is less than 1). 
{quote}
The fact that a user's weight is < 1.0 (or even the sum of multiple users' weights < 1.0) won't cause a problem. If user1 has a weight of 0.5 and its app is consuming the whole queue, and user2 with weight 2.0 submits an app, preemption occurs until user2's app has 4 times the resources of user1. It's the same as if user1 had weight of 1 and user2 had weight of 4.

> Capacity Scheduler: Support user-specific minimum user limit percent
> --------------------------------------------------------------------
>
>                 Key: YARN-5892
>                 URL: https://issues.apache.org/jira/browse/YARN-5892
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>         Attachments: Active users highlighted.jpg, YARN-5892.001.patch, YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch, YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch, YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch, YARN-5892.012.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}} property is per queue. A cluster admin should be able to set the minimum user limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled (YARN-4945 / YARN-2113), some users can be deemed as more important than other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like this:
> {code}
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
>     <value>25</value>
>   </property>
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
>     <value>75</value>
>   </property>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org