You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2010/08/23 23:25:17 UTC
[jira] Created: (MAPREDUCE-2027) fair share scheduler causes
unnecessary preemption under some conditions
fair share scheduler causes unnecessary preemption under some conditions
------------------------------------------------------------------------
Key: MAPREDUCE-2027
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2027
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/fair-share
Reporter: Joydeep Sen Sarma
When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.
Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).
There may be other reasons for excessive preemption as well - this one stood out clearly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2027) fair share scheduler causes
unnecessary preemption under some conditions
Posted by "Matei Zaharia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902017#action_12902017 ]
Matei Zaharia commented on MAPREDUCE-2027:
------------------------------------------
Some of this could be specific to Facebook, because I believe you guys have patched in LATE speculation (HADOOP-2141) onto your Hadoop 0.20, but may not have updated the scheduler bits that count speculative tasks. However, it's also possible that this issue remains in trunk.
> fair share scheduler causes unnecessary preemption under some conditions
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-2027
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2027
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Joydeep Sen Sarma
>
> When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.
> Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).
> There may be other reasons for excessive preemption as well - this one stood out clearly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2027) fair share scheduler causes
unnecessary preemption under some conditions
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903023#action_12903023 ]
Joydeep Sen Sarma commented on MAPREDUCE-2027:
----------------------------------------------
Scott - do you want to take care of the other issue you found (different logic for starting reducer in JIP vs. FS) in this one as well?
> fair share scheduler causes unnecessary preemption under some conditions
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-2027
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2027
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Joydeep Sen Sarma
>
> When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.
> Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).
> There may be other reasons for excessive preemption as well - this one stood out clearly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2027) fair share scheduler causes
unnecessary preemption under some conditions
Posted by "Matei Zaharia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902020#action_12902020 ]
Matei Zaharia commented on MAPREDUCE-2027:
------------------------------------------
Actually, looking at trunk, the issue seems to be there too. The methods you need to fix are neededSpeculativeMaps and neededSpeculativeReduces in DefaultTaskSelector.
> fair share scheduler causes unnecessary preemption under some conditions
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-2027
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2027
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Joydeep Sen Sarma
>
> When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.
> Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).
> There may be other reasons for excessive preemption as well - this one stood out clearly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2027) fair share scheduler causes
unnecessary preemption under some conditions
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903093#action_12903093 ]
Scott Chen commented on MAPREDUCE-2027:
---------------------------------------
Joydeep: I checked the trunk code. FairScheduler uses JobInProgress.scheduleReduces() instead of its own logic. So there is no problem.
> fair share scheduler causes unnecessary preemption under some conditions
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-2027
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2027
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Joydeep Sen Sarma
>
> When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.
> Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).
> There may be other reasons for excessive preemption as well - this one stood out clearly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2027) fair share scheduler causes
unnecessary preemption under some conditions
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902055#action_12902055 ]
Scott Chen commented on MAPREDUCE-2027:
---------------------------------------
I think it is reasonable that we just don't preempt for speculation execution. What do you think?
> fair share scheduler causes unnecessary preemption under some conditions
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-2027
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2027
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Joydeep Sen Sarma
>
> When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.
> Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).
> There may be other reasons for excessive preemption as well - this one stood out clearly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.