You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Xiao Kang (JIRA)" <ji...@apache.org> on 2010/04/06 04:32:27 UTC
[jira] Created: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Some new features for CapacityTaskScheduler
-------------------------------------------
Key: MAPREDUCE-1674
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
Project: Hadoop Map/Reduce
Issue Type: New Feature
Components: contrib/capacity-sched
Affects Versions: 0.20.2
Reporter: Xiao Kang
Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Posted by "Xiao Kang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853721#action_12853721 ]
Xiao Kang commented on MAPREDUCE-1674:
--------------------------------------
Sorry, wrong wiki syntext used, correct here:
# *support queue priority* : higher priority queue get capacity first
# *support queue refresh* : reload queue configuration at run time, including change queue properties and adding/removing queues
# *queue over capacity controlled* : add a queue property to indicate whether this queue can use more slot than its configured capacity
# *seperate web page for capacity-scheduler* : just as fair-scheduler, a new web page dadicated for capacity-scheduler is added
# *job capacity* : allowing user to specify per job map/reduce capacity both in cluster/tasktracker level
# *job initialization* : do not initialize a job if it will not get a chance to run if initialized
> Some new features for CapacityTaskScheduler
> -------------------------------------------
>
> Key: MAPREDUCE-1674
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
>
> Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853762#action_12853762 ]
Hemanth Yamijala commented on MAPREDUCE-1674:
---------------------------------------------
Since Hadoop 0.21, there has been a good deal of changes in the capacity scheduler, primarily to support hierarchical queues. Since the 'Affects version' was set to an earlier version, I just thought of calling this out so you are aware.
> Some new features for CapacityTaskScheduler
> -------------------------------------------
>
> Key: MAPREDUCE-1674
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
>
> Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Posted by "Xiao Kang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853720#action_12853720 ]
Xiao Kang commented on MAPREDUCE-1674:
--------------------------------------
Details as follows:
# support queue priority : higher priority queue get capacity first
# support queue refresh : reload queue configuration at run time, including change queue properties and adding/removing queues
# queue over capacity controlled : add a queue property to indicate whether this queue can use more slot than its configured capacity
# seperate web page for capacity-scheduler : just as fair-scheduler, a new web page dadicated for capacity-scheduler is added
# job capacity : allowing user to specify per job map/reduce capacity both in cluster/tasktracker level
# job initialization : do not initialize a job if it will not get a chance to run if initialized
> Some new features for CapacityTaskScheduler
> -------------------------------------------
>
> Key: MAPREDUCE-1674
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
>
> Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Posted by "Xiao Kang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854428#action_12854428 ]
Xiao Kang commented on MAPREDUCE-1674:
--------------------------------------
Thank you for remind.
I have read about the interesting features in hadoop-0.21. We will follow the new features. Among the six features I listed above, 2 & 3 have been implemeted in hadoop-0.21.
Since great changes have been made to capacity-scheduler, QueueManager and TaskScheduler, it's difficult to migrate our improvents made against hadoop-0.20.2 to the latest version. If they are still valuable for the earlier version, sub task will be created for each.
> Some new features for CapacityTaskScheduler
> -------------------------------------------
>
> Key: MAPREDUCE-1674
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
>
> Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Posted by "Xiao Kang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853723#action_12853723 ]
Xiao Kang commented on MAPREDUCE-1674:
--------------------------------------
Creating a sub task for each feature may be more convient for discuss.
> Some new features for CapacityTaskScheduler
> -------------------------------------------
>
> Key: MAPREDUCE-1674
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
>
> Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1674) Some new features for
CapacityTaskScheduler
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853719#action_12853719 ]
Arun C Murthy commented on MAPREDUCE-1674:
------------------------------------------
More details please? *smile*
> Some new features for CapacityTaskScheduler
> -------------------------------------------
>
> Key: MAPREDUCE-1674
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1674
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/capacity-sched
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
>
> Some new features for CapacityTaskScheduler developed at Baidu.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.