You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Takanobu Asanuma (JIRA)" <ji...@apache.org> on 2018/06/12 08:05:00 UTC
[jira] [Commented] (YARN-8191) Fair scheduler: queue deletion
without RM restart
[ https://issues.apache.org/jira/browse/YARN-8191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16509316#comment-16509316 ]
Takanobu Asanuma commented on YARN-8191:
----------------------------------------
Thanks for working on this, but it may create a findbugs warning. Please see YARN-8420.
> Fair scheduler: queue deletion without RM restart
> -------------------------------------------------
>
> Key: YARN-8191
> URL: https://issues.apache.org/jira/browse/YARN-8191
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 3.0.1
> Reporter: Gergo Repas
> Assignee: Gergo Repas
> Priority: Major
> Fix For: 3.2.0
>
> Attachments: Queue Deletion in Fair Scheduler.pdf, YARN-8191.000.patch, YARN-8191.001.patch, YARN-8191.002.patch, YARN-8191.003.patch, YARN-8191.004.patch, YARN-8191.005.patch, YARN-8191.006.patch, YARN-8191.007.patch, YARN-8191.008.patch, YARN-8191.009.patch, YARN-8191.010.patch, YARN-8191.011.patch, YARN-8191.012.patch, YARN-8191.013.patch, YARN-8191.014.patch, YARN-8191.015.patch, YARN-8191.016.patch, YARN-8191.017.patch
>
>
> The Fair Scheduler never cleans up queues even if they are deleted in the allocation file, or were dynamically created and are never going to be used again. Queues always remain in memory which leads to two following issues.
> # Steady fairshares aren’t calculated correctly due to remaining queues
> # WebUI shows deleted queues, which is confusing for users (YARN-4022).
> We want to support proper queue deletion without restarting the Resource Manager:
> # Static queues without any entries that are removed from fair-scheduler.xml should be deleted from memory.
> # Dynamic queues without any entries should be deleted.
> # RM Web UI should only show the queues defined in the scheduler at that point in time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org