[jira] [Commented] (YARN-1582) Capacity Scheduler: add a maximum-allocation-mb setting per queue

["Thomas Graves (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883238#comment-13883238 ] 

Thomas Graves commented on YARN-1582:
-------------------------------------

Note the patch attached leave the cluster level setting. The per queue settings must be less than or equal to the cluster level setting.  It also allows both the cluster level and per queue to be refreshed (yarn rmadmin -refreshQueues) as long as the value increases.  We can't allow it to decrease since we've told the AM's the max size and letting that decrease could mess them up.

> Capacity Scheduler: add a maximum-allocation-mb setting per queue 
> ------------------------------------------------------------------
>
>                 Key: YARN-1582
>                 URL: https://issues.apache.org/jira/browse/YARN-1582
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>    Affects Versions: 3.0.0, 0.23.10, 2.2.0
>            Reporter: Thomas Graves
>            Assignee: Thomas Graves
>         Attachments: YARN-1582-branch-0.23.patch
>
>
> We want to allow certain queues to use larger container sizes while limiting other queues to smaller container sizes.  Setting it per queue will help prevent abuse, help limit the impact of reservations, and allow changes in the maximum container size to be rolled out more easily.
> One reason this is needed is more application types are becoming available on yarn and certain applications require more memory to run efficiently. While we want to allow for that we don't want other applications to abuse that and start requesting bigger containers then what they really need.  
> Note that we could have this based on application type, but that might not be totally accurate either since for example you might want to allow certain users on MapReduce to use larger containers, while limiting other users of MapReduce to smaller containers.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)