You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2013/06/13 21:19:20 UTC

[jira] [Created] (MAPREDUCE-5321) Enable better parallelism in the Fair Scheduler

Sandy Ryza created MAPREDUCE-5321:
-------------------------------------

             Summary: Enable better parallelism in the Fair Scheduler
                 Key: MAPREDUCE-5321
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5321
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
            Reporter: Sandy Ryza
            Assignee: Sandy Ryza


Currently, the Fair Scheduler is locked on pretty much every operation, node updates, application additions and removals, every time the update thread runs, and every time the RM queries it for information.  Most of this locking is unnecessary, especially as only the core scheduling operations like application additions, removals, and node updates need a consistent view of scheduler state.

We can probably increase parallelism by using concurrent data structures when applicable, as well as keeping a slightly stale view to serve via the RM APIs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira