You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Denny Ye (JIRA)" <ji...@apache.org> on 2011/03/17 06:48:29 UTC
[jira] Created: (MAPREDUCE-2393) No total min share limitation of
all pools
No total min share limitation of all pools
------------------------------------------
Key: MAPREDUCE-2393
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2393
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/fair-share
Affects Versions: 0.21.0
Reporter: Denny Ye
hi, there is no limitation about min share of all pools with cluster total shares. User can define arbitrary amount of min share for each pool. It has such description in <fair scheduler design document>, but no regular code.
It may critical for slot distribution. One pool can hold all cluster slots to meet it's min share that greater than cluster total slots very much.
If that case has happened, we should scaled down proportionally.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2393) No total min share limitation of
all pools
Posted by "Kenji Kikushima (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenji Kikushima updated MAPREDUCE-2393:
---------------------------------------
Status: Patch Available (was: Open)
> No total min share limitation of all pools
> ------------------------------------------
>
> Key: MAPREDUCE-2393
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2393
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.21.0
> Reporter: Denny Ye
> Labels: fair, scheduler
> Attachments: MAPREDUCE-2393.patch
>
>
> hi, there is no limitation about min share of all pools with cluster total shares. User can define arbitrary amount of min share for each pool. It has such description in <fair scheduler design document>, but no regular code.
> It may critical for slot distribution. One pool can hold all cluster slots to meet it's min share that greater than cluster total slots very much.
> If that case has happened, we should scaled down proportionally.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2393) No total min share limitation
of all pools
Posted by "Mostafa Elhemali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508389#comment-13508389 ]
Mostafa Elhemali commented on MAPREDUCE-2393:
---------------------------------------------
Can you please add a test for this case?
> No total min share limitation of all pools
> ------------------------------------------
>
> Key: MAPREDUCE-2393
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2393
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.21.0
> Reporter: Denny Ye
> Labels: fair, scheduler
> Attachments: MAPREDUCE-2393.patch
>
>
> hi, there is no limitation about min share of all pools with cluster total shares. User can define arbitrary amount of min share for each pool. It has such description in <fair scheduler design document>, but no regular code.
> It may critical for slot distribution. One pool can hold all cluster slots to meet it's min share that greater than cluster total slots very much.
> If that case has happened, we should scaled down proportionally.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2393) No total min share limitation of
all pools
Posted by "Kenji Kikushima (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenji Kikushima updated MAPREDUCE-2393:
---------------------------------------
Attachment: MAPREDUCE-2393.patch
FairScheduler uses min share directly, nevertheless total min share exceeds total slots.
I made a patch for this issue to scale down min share by w2sRatio.
> No total min share limitation of all pools
> ------------------------------------------
>
> Key: MAPREDUCE-2393
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2393
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Affects Versions: 0.21.0
> Reporter: Denny Ye
> Labels: fair, scheduler
> Attachments: MAPREDUCE-2393.patch
>
>
> hi, there is no limitation about min share of all pools with cluster total shares. User can define arbitrary amount of min share for each pool. It has such description in <fair scheduler design document>, but no regular code.
> It may critical for slot distribution. One pool can hold all cluster slots to meet it's min share that greater than cluster total slots very much.
> If that case has happened, we should scaled down proportionally.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira