[jira] Commented: (MAPREDUCE-1998) Size-based queuing for capacity scheduler

["Allen Wittenauer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896134#action_12896134 ] 

Allen Wittenauer commented on MAPREDUCE-1998:
---------------------------------------------

So the way I imagine this working is:

admin configures 3 queues:

jobs with > 1000 tasks
jobs with <999 >100 tasks
jobs with < 99 tasks

user submits a job to default, which triggers sized-based queuing.

hadoop determines map tasks: lets say 100, and user requested 20 reduces.  Total task count: 120.  

JobTracker places job in the middle tier queue since there are 120 tasks.



> Size-based queuing for capacity scheduler
> -----------------------------------------
>
>                 Key: MAPREDUCE-1998
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1998
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/capacity-sched
>            Reporter: Allen Wittenauer
>
> On job submission, it would be useful if the capacity scheduler could pick a queue based on the # of maps and reduces.  This way one could have queues based on job-size without users having to pick the queue prior to submission.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.