You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Scott Chen (JIRA)" <ji...@apache.org> on 2010/12/14 01:20:01 UTC
[jira] Created: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
The expire launching task should cover the UNASSIGNED task
----------------------------------------------------------
Key: MAPREDUCE-2217
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Affects Versions: 0.23.0
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.23.0
The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2217) The expire launching task
should cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502374#comment-13502374 ]
Scott Chen commented on MAPREDUCE-2217:
---------------------------------------
Karthik: Thank you for working on this.
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Karthik Kambatla
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-2217.1.txt, MR-2217.patch
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Chen updated MAPREDUCE-2217:
----------------------------------
Attachment: (was: MAPREDUCE-2217.txt)
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.1.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Chen updated MAPREDUCE-2217:
----------------------------------
Attachment: (was: MAPREDUCE-2217.1.txt)
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.1.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Chen updated MAPREDUCE-2217:
----------------------------------
Attachment: MAPREDUCE-2217.1.txt
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.1.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974331#action_12974331 ]
Scott Chen commented on MAPREDUCE-2217:
---------------------------------------
I will try to add a unit test for this.
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Lianhui Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974213#action_12974213 ]
Lianhui Wang commented on MAPREDUCE-2217:
-----------------------------------------
in the ExpireLaunchingTasks,run() may calls the JobTracker.killTask(). that can implement it.
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Chen updated MAPREDUCE-2217:
----------------------------------
Attachment: MAPREDUCE-2217.txt
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Chen updated MAPREDUCE-2217:
----------------------------------
Attachment: MAPREDUCE-2217.1.txt
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.1.txt, MAPREDUCE-2217.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-2217:
----------------------------------------
Attachment: MR-2217.patch
Uploading the same patch generated off of branch-1.
We noticed this issue when one of the task trackers was faulty, and the unassigned tasks on that TaskTracker were not expired leading to job incompletion.
+1 for the patch.
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-2217.1.txt, MR-2217.patch
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974330#action_12974330 ]
Scott Chen commented on MAPREDUCE-2217:
---------------------------------------
Hi Lianhui,
Yes, that's the idea. I have attached a patch for this. Currently we will do ExpireLaunchingTasks.removeTask() for any task that updates.
But we should avoid remove the task that is UNASSIGNED so they can be killed by ExpireLaunchingTasks.run() as you described.
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2217.txt
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Assigned] (MAPREDUCE-2217) The expire launching task should
cover the UNASSIGNED task
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla reassigned MAPREDUCE-2217:
-------------------------------------------
Assignee: Karthik Kambatla (was: Scott Chen)
> The expire launching task should cover the UNASSIGNED task
> ----------------------------------------------------------
>
> Key: MAPREDUCE-2217
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2217
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Karthik Kambatla
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-2217.1.txt, MR-2217.patch
>
>
> The ExpireLaunchingTask thread kills the task that are scheduled but not responded.
> Currently if a task is scheduled on tasktracker and for some reason tasktracker cannot put it to RUNNING.
> The task will just hang in the UNASSIGNED status and JobTracker will keep waiting for it.
> JobTracker.ExpireLaunchingTask should be able to kill this task.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira