You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Adam Kramer (JIRA)" <ji...@apache.org> on 2010/12/06 07:32:11 UTC
[jira] Created: (MAPREDUCE-2210) Preemption does not obey job
priority
Preemption does not obey job priority
-------------------------------------
Key: MAPREDUCE-2210
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2210
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Reporter: Adam Kramer
Preemption, as it currently works, kills running tasks that are "unfair" in terms of the amount of task attention used by one job compared to another.
The bug is that pre-emption occurs such that HIGH priority jobs have their tasks killed when the cluster is filled with lower-priority jobs. Pre-emption should only be allowed to kill tasks for jobs with equal or lower priority.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2210) Preemption does not obey job
priority
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968902#action_12968902 ]
Scott Chen commented on MAPREDUCE-2210:
---------------------------------------
Hi Adam,
Actually preemption does kill based on FairShare and minimal slots requirement. It doesn't kill tasks beyond your FairShare.
We can discuss this more.
> Preemption does not obey job priority
> -------------------------------------
>
> Key: MAPREDUCE-2210
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2210
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Adam Kramer
>
> Preemption, as it currently works, kills running tasks that are "unfair" in terms of the amount of task attention used by one job compared to another.
> The bug is that pre-emption occurs such that HIGH priority jobs have their tasks killed when the cluster is filled with lower-priority jobs. Pre-emption should only be allowed to kill tasks for jobs with equal or lower priority.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2210) Preemption does not obey job
priority
Posted by "Matei Zaharia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12967106#action_12967106 ]
Matei Zaharia commented on MAPREDUCE-2210:
------------------------------------------
Which job scheduler are you using? The choice of tasks to preempt is entirely up to the scheduler, so you should mark the issue as belonging to the particular scheduler that you're using.
> Preemption does not obey job priority
> -------------------------------------
>
> Key: MAPREDUCE-2210
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2210
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
>
> Preemption, as it currently works, kills running tasks that are "unfair" in terms of the amount of task attention used by one job compared to another.
> The bug is that pre-emption occurs such that HIGH priority jobs have their tasks killed when the cluster is filled with lower-priority jobs. Pre-emption should only be allowed to kill tasks for jobs with equal or lower priority.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-2210) Preemption does not obey job
priority
Posted by "Adam Kramer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kramer resolved MAPREDUCE-2210.
------------------------------------
Resolution: Not A Problem
> Preemption does not obey job priority
> -------------------------------------
>
> Key: MAPREDUCE-2210
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2210
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Adam Kramer
>
> Preemption, as it currently works, kills running tasks that are "unfair" in terms of the amount of task attention used by one job compared to another.
> The bug is that pre-emption occurs such that HIGH priority jobs have their tasks killed when the cluster is filled with lower-priority jobs. Pre-emption should only be allowed to kill tasks for jobs with equal or lower priority.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2210) Preemption does not obey job
priority
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated MAPREDUCE-2210:
----------------------------------------
Component/s: (was: jobtracker)
contrib/fair-share
> Preemption does not obey job priority
> -------------------------------------
>
> Key: MAPREDUCE-2210
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2210
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Adam Kramer
>
> Preemption, as it currently works, kills running tasks that are "unfair" in terms of the amount of task attention used by one job compared to another.
> The bug is that pre-emption occurs such that HIGH priority jobs have their tasks killed when the cluster is filled with lower-priority jobs. Pre-emption should only be allowed to kill tasks for jobs with equal or lower priority.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2210) Preemption does not obey job
priority
Posted by "Matei Zaharia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12967107#action_12967107 ]
Matei Zaharia commented on MAPREDUCE-2210:
------------------------------------------
Actually, looking at your email address, I have a good idea which scheduler you might be using :P. This is an issue that can be fixed through configuration, but you should talk to Joydeep about what the best way to deal with it is. If there is a change you want to make to preemption in trunk let me know though.
> Preemption does not obey job priority
> -------------------------------------
>
> Key: MAPREDUCE-2210
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2210
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Adam Kramer
>
> Preemption, as it currently works, kills running tasks that are "unfair" in terms of the amount of task attention used by one job compared to another.
> The bug is that pre-emption occurs such that HIGH priority jobs have their tasks killed when the cluster is filled with lower-priority jobs. Pre-emption should only be allowed to kill tasks for jobs with equal or lower priority.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.