[jira] [Updated] (YARN-3388) Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit

["Nathan Roberts (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nathan Roberts updated YARN-3388:
---------------------------------
    Attachment: YARN-3388-v3.patch

[~leftnoteasy], [~eepayne]. Ok, "soon" was extremely relative;) Sorry about that. 

I think I addressed Wangda's comments but I need label partition experts to take a look.

Any ideas why people don't hit this more often? We find it's very easy to get stuck at queueCapacity even though userLimitFactor and maxCapacity say the system should allocate further. Do you think people aren't using DRF and are mostly just using memory as the resource?

> Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3388
>                 URL: https://issues.apache.org/jira/browse/YARN-3388
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.6.0
>            Reporter: Nathan Roberts
>            Assignee: Nathan Roberts
>         Attachments: YARN-3388-v0.patch, YARN-3388-v1.patch, YARN-3388-v2.patch, YARN-3388-v3.patch
>
>
> When there are multiple active users in a queue, it should be possible for those users to make use of capacity up-to max_capacity (or close). The resources should be fairly distributed among the active users in the queue. This works pretty well when there is a single resource being scheduled.   However, when there are multiple resources the situation gets more complex and the current algorithm tends to get stuck at Capacity. 
> Example illustrated in subsequent comment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org