You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Robert Chansler (JIRA)" <ji...@apache.org> on 2009/03/03 19:34:56 UTC

[jira] Updated: (HADOOP-4789) Change fair scheduler to share between pools by default, not between invidual jobs

     [ https://issues.apache.org/jira/browse/HADOOP-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Chansler updated HADOOP-4789:
------------------------------------

    Release Note: Changed fair scheduler to divide resources equally between pools, not jobs.  (was: Fair scheduler now divides resources equally between pools, not between individual jobs (HADOOP-4789).)
    Hadoop Flags: [Incompatible change, Reviewed]  (was: [Reviewed, Incompatible change])

Edit release note for publication.

> Change fair scheduler to share between pools by default, not between invidual jobs
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-4789
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4789
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Matei Zaharia
>            Assignee: Matei Zaharia
>            Priority: Minor
>             Fix For: 0.20.0
>
>         Attachments: hadoop-4789-v1.patch
>
>
> The fair scheduler currently treats jobs as equal entities in sharing by default, so that a user who submits 2 jobs gets 2x the share of a user who submits only 1 job. We found that it makes more sense to support equal shares between individual pools instead, and have one pool per user, because users can otherwise game the system by submitting multiple small jobs. This patch will set the scheduler to share between pools by default and set the default pool assignment process to one pool per user. it will also be possible to give weights to pools so that some users/groups/applications get a larger share of the cluster if they really do need to run more jobs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.