You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2007/05/11 23:12:15 UTC
[jira] Created: (HADOOP-1351) Want to kill a particular task or
attempt
Want to kill a particular task or attempt
-----------------------------------------
Key: HADOOP-1351
URL: https://issues.apache.org/jira/browse/HADOOP-1351
Project: Hadoop
Issue Type: New Feature
Components: mapred
Affects Versions: 0.12.3
Reporter: Owen O'Malley
Fix For: 0.14.0
It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
bin/hadoop job -kill-task tip_0001_m_000000
bin/hadoop job -kill-attempt task_0001_m_000000_0
This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.4.patch
Patch updated against latest trunk.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511845 ]
Owen O'Malley commented on HADOOP-1351:
---------------------------------------
Enis,
Can you please add a diagnostic message saying "Killed by user" so that the user can tell why their task was killed from the web/ui? I think that would help a lot.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.2.patch
Added kill tasks from web UI, which can be enabled from the configuration.
renamed {{TaskInProgress.shouldCloseForClosedJob}} to {{TaskInProgress.shouldClose}}, and moved the logic for checking killed taks to this function. The logic for {{incompleteSubTask}} and {{taskKilled}} did not change, since i still believe this is the more intuitive way (smile).
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar reassigned HADOOP-1351:
-------------------------------------
Assignee: Enis Soztutar
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512928 ]
Enis Soztutar commented on HADOOP-1351:
---------------------------------------
>I still think we need to mark user-killed tasks as FAILED rather than KILLED to ensure this penalizes the job & hence the user...
Sorry but I must disagree once more. The use case that motivated me to implement killing tasks was that sometimes some tasks fail to initialize for a while due to network latency, so one must literally connect to the machine that runs the task and literally kill the task. Another use case might be that a large job takes all available maps, but we have a small high priority job with few reduces, so we kill some of the maps, to make room for the new job. Other than that I so not expect some one to track and kill the tasks that seem to run slow.
We need to store the info that some task is killed so that we can send a {{KillTaskAction}} to the TT. Considering current implementation handling status info about the tasks is always done via {{JobInProgress}} 's methods. So the chain of responsibility from JobInProgress to TaskInProgress is already broken. Honestly i intend to retain this structure for this patch( rather than marking the task as killed immediately ), and delegate the much serious work to HADOOP-869.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525816 ]
Arun C Murthy commented on HADOOP-1351:
---------------------------------------
+1 (finally! *smile*)
Thanks for your patience on this one Enis!
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch, killTask_v1.9.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Status: Patch Available (was: Open)
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12510063 ]
Arun C Murthy commented on HADOOP-1351:
---------------------------------------
Enis, with due consideration I'd like to suggest an alternative, simpler approach:
On receipt of {{killTask}} from the user, call {{TaskInProgress.incompleteSubTask}} and mark the task as {{FAILED}} (and not {{KILLED}} since you want this attempt to count against the job).
Then hack {{TaskInProgress.shouldCloseForClosedJob}} (I'd suggest rename that to 'shouldTerminate' or something along those lines...) to return {{true}} for that attempt so as to notify the TaskTracker on the next heartbeat. (Basically you'll have ensure the {{taskReportedClosed}} data-structure is maintained correctly i.e. add succesful/failed/killed taskids to this from {{alreadyCompletedTask}}, {{completedTask}} & {{incompleteSubTask}} ).
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Fix Version/s: (was: 0.14.0)
0.15.0
Affects Version/s: (was: 0.14.0)
0.15.0
Moving to 0.15 since this issue is not a bug fix.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525652 ]
Hadoop QA commented on HADOOP-1351:
-----------------------------------
-1, could not apply patch.
The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12365044/killTask_v1.7.patch as a patch to trunk revision r573492.
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/706/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512202 ]
Arun C Murthy commented on HADOOP-1351:
---------------------------------------
Enis, I still think we need to mark user-killed tasks as {{FAILED}} rather than {{KILLED}} to ensure this penalizes the job & hence the user...
Also, it would make me much happier if you used {{TaskInProgress.incompleteSubTask}} to kill the task as soon as the JT receives the user's instruction rather than queuing it up via the list and acting on it later. I generally prefer to maintain as less state as and when possible...
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.9.patch
After a discussion with Arun in #hadoop, we have decided the following
- It may cause to log killing tasks preemptive, so we log a request to kill the task, than log the task as killed once TT reports so (two log entries per action).
- changed {{JobSubmissionProtocol#killTask()}} to return {{boolean}} to indicate success. Of course since killing task is async, true may not mean the task is really killed if the cluster is in an unstable state.
- adding user name to kill task log/diagnostic message. (not implemented, taking user name as a parameter in JobSubmissionProtocol did not seem good enough ).
- passing the cause to kill the job to TT, in order to log it (not implemented since again, i am not fond of extending function signatures just to improve logging)
- change {{JobSubmissionProtocol#killJob() to return boolean}} - Will open a new issue for this.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch, killTask_v1.9.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Affects Version/s: (was: 0.12.3)
0.14.0
Status: Patch Available (was: In Progress)
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514595 ]
Enis Soztutar commented on HADOOP-1351:
---------------------------------------
It is unfortunate that we cannot agree with Arun on this issue, so i wonder what do the other developers think.
> Perhaps we should give folks the option of either killing or failing tasks?
another configuration option? Hell no! .smile.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Status: Patch Available (was: Open)
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch, killTask_v1.9.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.3.patch
The latest version of the patch. Adds diagnostic message. And complies with the changes in HADOOP-1473. Hope this one makes it to 0.14 release.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.5.patch
At last, here is the final(i hope) version of the patch.
As per Doug's and Arun's comments i have enabled both killing and failing tasks.
{{JobSubmissionProtocol}} includes
{code}
public void killTask(String taskId, boolean shouldFail) throws IOException;
{code}
the shouldFail argument is used to indicate failing or killing the task.
and we have command line options
{noformat}
bin/hadoop job -kill-task <task_id>
bin/hadoop job -fail-task <task_id>
{noformat}
I have observed some issues with failing the task directly, such as that we discard task output, but until the kill action has been send to TT, tasks will output data. So i have sticked with the first design.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.6.patch
Fixed taskid -> JIP path to not include string processing.
Changed synchronization to apply to whole function instead.
added a minor log.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1351) Want to kill a
particular task or attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12510079 ]
Enis Soztutar edited comment on HADOOP-1351 at 7/3/07 11:30 PM:
----------------------------------------------------------------
Thanks Arun for the suggestion,
> mark the task as FAILED (and not KILLED since you want this attempt to count against the job).
Actually i think, we should label the task as KILLED, rather than FAILED, since killing is a task is quite an explicit action done by the user examining the system so she may prefer seeing the task as KILLED (Although in the similar case for killing jobs we do not have KILLED status for a job, so killed jobs appear as FAILED). Also, regarding our use cases, we do not want to "black label" the machine for killed task ({{machinesWhereFailed}}), that's why i have written {{taskKilled}} instead of using {{incompleteSubTask}}.
I will look deeper into using {{TaskInProgress.shouldCloseForClosedJob}} and {{tasksReportedClosed}}. I bet having a generic DFA implementation will benefit us in this.
was:
Thanks Arun for the suggestion,
> mark the task as FAILED (and not KILLED since you want this attempt to count against the job).
Actually i think, we should label the task as KILLED, rather than FAILED, since killing is a task is quite an explicit action done by the user examining the system so she may prefer seeing the task as KILLED (Although in the similar case for killing jobs we do not have KILLED status for a job, so killed jobs appear as FAILED). Also, regarding our use cases, we do not want to "black label" the machine for killed task ({{machinesWhereFailed}}), that's why i have written {{taskKilled}} instead of using {{incompleteSubTask}}.
I will look deeper into using {{TaskInProgress.shouldCloseForClosedJob }} and {{tasksReportedClosed}}. I bet having a generic DFA implementation will benefit us in this.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Status: Patch Available (was: Open)
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Status: Patch Available (was: Open)
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12510079 ]
Enis Soztutar commented on HADOOP-1351:
---------------------------------------
Thanks Arun for the suggestion,
> mark the task as FAILED (and not KILLED since you want this attempt to count against the job).
Actually i think, we should label the task as KILLED, rather than FAILED, since killing is a task is quite an explicit action done by the user examining the system so she may prefer seeing the task as KILLED (Although in the similar case for killing jobs we do not have KILLED status for a job, so killed jobs appear as FAILED). Also, regarding our use cases, we do not want to "black label" the machine for killed task ({{machinesWhereFailed}}), that's why i have written {{taskKilled}} instead of using {{incompleteSubTask}}.
I will look deeper into using {{TaskInProgress.shouldCloseForClosedJob }} and {{tasksReportedClosed}}. I bet having a generic DFA implementation will benefit us in this.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1351:
---------------------------------
Status: Open (was: Patch Available)
I'm re-opening this until Enis and Arun can agree on a solution. Perhaps we should give folks the option of either killing or failing tasks?
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1351:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Enis!
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch, killTask_v1.9.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.0.1.patch
<killTask_v1.0.1.patch>
This patch enables killing tasks from JobSubmissionProtocol, and command line. killing tasks from LocalJobRunner is not supported.
{{bin/hadoop job -kill-task task_0001_m_000000_0}}
will kill the task attempt. Only killing task attempts is possible, since killing a task(i.e. TaskInProgress) will result in job failure.
I have tested the implementation, and it passes all unit tests, however I think it deserves more testing. I will shortly add killing tasks from web interface.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.8.patch
latest patch
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525786 ]
Hadoop QA commented on HADOOP-1351:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12365360/killTask_v1.9.patch applied and successfully tested against trunk revision r573513.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/711/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/711/console
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch, killTask_v1.9.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1351:
----------------------------------
Status: Open (was: Patch Available)
Enis, could you please regenerate the patch with trunk? Thanks.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12503842 ]
Enis Soztutar commented on HADOOP-1351:
---------------------------------------
we will also need a kill task action from the web interface similar to HADOOP-1484
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Fix For: 0.14.0
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1351:
----------------------------------
Status: Open (was: Patch Available)
{noformat}
@@ -189,7 +190,16 @@
}
}
+ /** Returns the Jobid, given job number and jtId.*/
+ public static String formJobId(String jtIdentifier, String jobNumber) {
+ return jtIdentifier + "_" + jobNumber;
+ }
+ /** Returns the full Jobid. Returned string starts with job_ */
+ public static String formFullJobId(String jobId) {
+ return "job_" + jobId;
+ }
+
{noformat}
Uh oh, could you please remove that deadcode from the patch and submit again? Thanks!
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524473 ]
Arun C Murthy commented on HADOOP-1351:
---------------------------------------
Sorry, I should have reviewed this earlier... some comments:
1. {{JobTracker.killTask}} should be a *synchronized* method
2. Please don't introduce the string handling functions {{TaskInProgress.extractJobId}} & {{TaskInProgress.extractTIPId}}, they will break soon i.e. after we fix HADOOP-544.
{noformat}
+ String jobid = TaskInProgress.extractJobId(taskid);
+ String tipid = TaskInProgress.extractTIPId(taskid);
{noformat}
can be:
{noformat}
+ TaskInProgress tip = taskidToTIPMap.get(taskid);
+ String tipId = tip.getTIPId();
+ String jobId = tip.getJob().getStatus().getJobId();
{noformat}
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525694 ]
Hadoop QA commented on HADOOP-1351:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12365334/killTask_v1.8.patch applied and successfully tested against trunk revision r573513.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/708/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/708/console
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Attachment: killTask_v1.7.patch
Removed dead code i forgot.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-1351:
----------------------------------
Status: Open (was: Patch Available)
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch, killTask_v1.5.patch, killTask_v1.6.patch, killTask_v1.7.patch, killTask_v1.8.patch, killTask_v1.9.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495522 ]
Enis Soztutar commented on HADOOP-1351:
---------------------------------------
+1
sometimes(in 0.9.2) we have to manually kill the task since it stays in the initializing phase. Such command will ease the task killing process.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Fix For: 0.14.0
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511432 ]
Hadoop QA commented on HADOOP-1351:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12361483/killTask_v1.2.patch applied and successfully tested against trunk revision r554811.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/387/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/387/console
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1351) Want to kill a particular task or
attempt
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514875 ]
Arun C Murthy commented on HADOOP-1351:
---------------------------------------
Sorry I've been busy with others and forgot about this.... didnt' mean to hold it up.
bq. We need to store the info that some task is killed so that we can send a KillTaskAction to the TT.
No question. I was only suggesting that we use the already existing {{taskReportedClosed}} list rather than introduce a new {{tasksToKill}} set...
bq. It is unfortunate that we cannot agree with Arun on this issue, so i wonder what do the other developers think.
I agree, Doug/Owen your take? We just agree to disagree...
bq. another configuration option? Hell no! .smile.
I believe Doug is suggesting a -kill-task and a -fail-task flag... which might be useful post HADOOP-1144.
I couldn't agree more with your sentiments about config options! *smile*
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.15.0
>
> Attachments: killTask_v1.0.1.patch, killTask_v1.2.patch, killTask_v1.3.patch, killTask_v1.4.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (HADOOP-1351) Want to kill a particular task
or attempt
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-1351 started by Enis Soztutar.
> Want to kill a particular task or attempt
> -----------------------------------------
>
> Key: HADOOP-1351
> URL: https://issues.apache.org/jira/browse/HADOOP-1351
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.12.3
> Reporter: Owen O'Malley
> Assignee: Enis Soztutar
> Fix For: 0.14.0
>
> Attachments: killTask_v1.0.1.patch
>
>
> It would be convenient to be able to kill a particular task or attempt from the command line. It would look like:
> bin/hadoop job -kill-task tip_0001_m_000000
> bin/hadoop job -kill-attempt task_0001_m_000000_0
> This would allow the user to tell the system to stop a particular task or attempt without having to restart a task tracker.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.