You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Aihua Xu (JIRA)" <ji...@apache.org> on 2018/12/12 22:44:00 UTC
[jira] [Updated] (YARN-9116) Capacity Scheduler: add the default
maximum-allocation-mb and maximum-allocation-vcores for the queues
[ https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aihua Xu updated YARN-9116:
---------------------------
Description:
YARN-1582 adds the support of maximum-allocation-mb configuration per queue which is targeting to support larger container features on dedicated queues (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . While to achieve larger container configuration, we need to increase the global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and then override those configurations with desired values on the queues since queue configuration can't be larger than cluster configuration. If we forget to configure such values when adding a new queue, then such queue gets default 120G/256 which typically is not what we want.
We can come up with a queue-default configuration (set to normal queue configuration like 16G/8), so the leaf queues gets such values by default.
was:
YARN-1582 adds the support of maximum-allocation-mb configuration per queue which is targeting to support larger container features on dedicated queues (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . While to achieve larger container configuration, we need to increase the global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and then override those configurations with desired values on the queues since queue configuration can't be larger than cluster configuration. If we forget to configure such values when adding a new queue, then such queue gets default 120G/256 which typically is not what we want.
We can come up with a top-queue-default configuration (set to normal queue configuration like 16G/8), so top queue and its children gets such values by default.
> Capacity Scheduler: add the default maximum-allocation-mb and maximum-allocation-vcores for the queues
> ------------------------------------------------------------------------------------------------------
>
> Key: YARN-9116
> URL: https://issues.apache.org/jira/browse/YARN-9116
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 2.7.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
>
> YARN-1582 adds the support of maximum-allocation-mb configuration per queue which is targeting to support larger container features on dedicated queues (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . While to achieve larger container configuration, we need to increase the global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and then override those configurations with desired values on the queues since queue configuration can't be larger than cluster configuration. If we forget to configure such values when adding a new queue, then such queue gets default 120G/256 which typically is not what we want.
> We can come up with a queue-default configuration (set to normal queue configuration like 16G/8), so the leaf queues gets such values by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org