You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Wilfred Spiegelenburg (JIRA)" <ji...@apache.org> on 2019/03/03 23:22:00 UTC
[jira] [Commented] (YARN-6487) FairScheduler: remove continuous
scheduling (YARN-1010)
[ https://issues.apache.org/jira/browse/YARN-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782897#comment-16782897 ]
Wilfred Spiegelenburg commented on YARN-6487:
---------------------------------------------
It works both ways:
Before we can schedule we need to update the current usage and shares etc. This runs in an update thread. Continuous scheduling triggers that update. The heartbeats when they are processed do the same. This updating requires a lock of the scheduler as does the scheduling process itself. The extra update demand is the trigger/
So you get into a state that the heartbeat, the updates and the scheduling itself are all waiting for the lock. The larger the number of nodes, the larger the number of applications is (in most cases) and the larger the number of queues (again in most cases). All this combined causes processing to start lagging and the continuous scheduling really loses its function.
Node numbers influence continuous scheduling and the other way around.
> FairScheduler: remove continuous scheduling (YARN-1010)
> -------------------------------------------------------
>
> Key: YARN-6487
> URL: https://issues.apache.org/jira/browse/YARN-6487
> Project: Hadoop YARN
> Issue Type: Task
> Components: fairscheduler
> Affects Versions: 2.7.0
> Reporter: Wilfred Spiegelenburg
> Assignee: Wilfred Spiegelenburg
> Priority: Major
>
> Remove deprecated FairScheduler continuous scheduler code
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org