[jira] [Assigned] (YARN-9351) user can't use total resources of one partition even yarn.scheduler.capacity..minimum-user-limit-percent is set to 100

["tianjuan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

tianjuan reassigned YARN-9351:
------------------------------

    Assignee: tianjuan

> user can't use total resources of one partition even yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100 
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-9351
>                 URL: https://issues.apache.org/jira/browse/YARN-9351
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 3.1.2
>            Reporter: tianjuan
>            Assignee: tianjuan
>            Priority: Major
>
> if we configure queue capacity in absolute term, users can't use total resource of one partition even yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100 
>  for example there are two partition A,B, partition A has (120G memory,30 vcores), and partition B has (180G memory,60 vcores), and Queue Prod is configured with (75G memory, 25 vcores) partition A resource, like yarn.scheduler.capacity.root.Prod.accessible-node-labels.A.capacity=[memory=75Gi,vcores=25],
> and yarn.scheduler.capacity.root.Prod.accessible-node-labels.A.maximum-capacity=[memory=120Gi,vcores=30]
> yarn.scheduler.capacity.root.Prod.minimum-user-limit-percent=100, and at one point the used resource of queue Prod is (90G memory,10 vcores), at this time even though yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100 , users in queue A can't get more resource.
>  
> the reason for this is that  when {color:#d04437}*computeUserLimit*{color}, partitionResource is used for comparing consumed, queueCapacity, so in the example (75G memory, 25 vcores) is the user limit. 
> Resource currentCapacity = Resources.lessThan(resourceCalculator,
>  partitionResource, consumed, queueCapacity)
>  ? queueCapacity
>  : Resources.add(consumed, required);
> Resource userLimitResource = Resources.max(resourceCalculator, partitionResource,Resources.divideAndCeil(resourceCalculator, resourceUsed,
> usersSummedByWeight),Resources.divideAndCeil(resourceCalculator,Resources.multiplyAndRoundDown(currentCapacity, getUserLimit()),100));
>  
> but when *{color:#d04437}canAssignToUser{color}* = Resources.greaterThan(resourceCalculator, clusterResource,
>  user.getUsed(nodePartition), limit)
> *{color:#d04437}clusterResource{color}* {color:#333333}is used for for comparing  *used and limit, the result is false.*{color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org