You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2008/09/22 06:49:44 UTC
[jira] Created: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
JobTracker.killJob() fails to kill a job if the job is not yet initialized
--------------------------------------------------------------------------
Key: HADOOP-4236
URL: https://issues.apache.org/jira/browse/HADOOP-4236
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Amar Kamat
Priority: Blocker
HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
{code}
public synchronized void killJob(JobID jobid) {
JobInProgress job = jobs.get(jobid);
- job.kill();
+ if (job.inited()) {
+ job.kill();
+ }
}
{code}
This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Attachment: 4236_v3.patch
got away with the extra flag, reused jobKilled flag.
also changed synchronization in obtainCleanupTask and obtainSetupTask. now taskInited flag is checked before taking lock on JobInProgress.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637364#action_12637364 ]
Amareshwari Sriramadasu commented on HADOOP-4236:
-------------------------------------------------
bq. At present, calling JIP.kill() will just mark the job for killing, but job is truly not killed completely. A job is completely killed only when JIP.terminateJob()) is called on it, which does things like logging to JobHistory and marking JobStatus as killed. Currently, this method isn't called until a clean-up task is scheduled and runs to completion, but for uninitialized jobs there are no clean-up tasks and so job-kill remains incomplete for ever.
This will be fixed in HADOOP-4261
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639371#action_12639371 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
test patch returned following:
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
Findbugs warning is :
<a href="#IS2_INCONSISTENT_SYNC">Bug type IS2_INCONSISTENT_SYNC (click for details)</a>
<br/>In class org.apache.hadoop.mapred.JobInProgress<br/>Field org.apache.hadoop.mapred.JobInProgress.initStarted<br/>Synchronized 50% of the time<br/>Unsynchronized access at JobInProgress.java:[line 1980]<br/>Synchronized access at JobInProgress.java:[line 349]<br/>Synchronized access at JobInProgress.java:[line 145]<br/>Synchronized access at JobInProgress.java:[line 145]</p>
However initStarted is synchronized everywhere. Seems like findbugs is getting confused.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637362#action_12637362 ]
Vinod K V commented on HADOOP-4236:
-----------------------------------
This patch should also have a fix for JobInProgress.kill() so that calling this on an uninitialized job kills the job completely.
At present, calling JIP.kill() will just mark the job for killing, but job is truly not killed completely. A job is completely killed only when JIP.terminateJob()) is called on it, which does things like logging to JobHistory and marking JobStatus as killed. Currently, this method isn't called until a clean-up task is scheduled and runs to completion, but for uninitialized jobs there are no clean-up tasks and so job-kill remains incomplete for ever.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Status: Patch Available (was: Open)
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639737#action_12639737 ]
Owen O'Malley commented on HADOOP-4236:
---------------------------------------
You need to explicitly terminateJob to make sure all of the tasks are marked as killed.
Sameer has a good point that since the schedulers may delay initializing jobs for an arbitrary length of time, we should probably kill jobs before they get to init. It would be far better if we could always make the kill method do the kill and block if it is in the middle of initialization.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Status: Patch Available (was: Open)
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639745#action_12639745 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
bq. Sameer has a good point that since the schedulers may delay initializing jobs for an arbitrary length of time, we should probably kill jobs before they get to init.
the current patch is already doing it. As far I understand, Arun and Sameer have suggested to move those flags to a class.
bq. It would be far better if we could always make the kill method do the kill and block if it is in the middle of initialization.
Agreed, that would be ideal and cleaner as well (just make kill as synchronized). But we are trying to deal with the problem of jobtracker locking - see HADOOP-3864. Currently during the kill job, the jobtracker is locked and we can't afford to block the kill job if its in the middle of initTasks().
HADOOP-869 is seemingly becoming more important.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637451#action_12637451 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
+1 for not exposing 2 apis.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4236:
----------------------------------
Status: Open (was: Patch Available)
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Attachment: 4236_v5.patch
added some documentation around terminate and made terminate explicitly re-entrant.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch, 4236_v5.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640196#action_12640196 ]
Hudson commented on HADOOP-4236:
--------------------------------
Integrated in Hadoop-trunk #635 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/635/])
. Ensure un-initialized jobs are killed correctly on user-demand. Contributed by Sharad Agarwal.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch, 4236_v5.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639036#action_12639036 ]
Hadoop QA commented on HADOOP-4236:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391989/4236_v2.patch
against trunk revision 703923.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3449/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3449/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3449/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3449/console
This message is automatically generated.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637445#action_12637445 ]
Amar Kamat commented on HADOOP-4236:
------------------------------------
bq. This patch should also have a fix for JobInProgress.kill() so that calling this on an uninitialized job kills the job completely.
This should have been taken care in HADOOP-4261. The task-initializer should call {{job.kill()}} which should do one of the 3 things
- {{garbageCollect()}} is the job has not _started_ (HADOOP-4261)
- mark for kill if the job is in _init_ (this issue)
- mark for cleanup if the job is _running_
Ideally there should be only one api to kill a job (say {{killJob()}}) which internally does the switching. Having 2 apis ({{killJob()}} and {{terminateJob()}}) is confusing.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat updated HADOOP-4236:
-------------------------------
Affects Version/s: 0.19.0
Fix Version/s: 0.19.0
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638537#action_12638537 ]
Vinod K V commented on HADOOP-4236:
-----------------------------------
JobInProgess.kill() should be made re-entrant. We need this as some schedulers may wish to kill jobs that violate certain resource requirements and we don't want to waste cycles on terminating a job that is already killed.
Additionally, we will somehow need to inform job-clients which try to terminate jobs for which a kill has already been issued. Currently, on re-kill attempts, it just keeps saying "job is killed successfully".
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637437#action_12637437 ]
Vinod K V commented on HADOOP-4236:
-----------------------------------
bq. This will be fixed in HADOOP-4261
HADOOP-4261 didn't solve this issue, but it did facilitate a work around by making JobInProgress.terminateJob() package private. So, callers that wish to kill a job should do something like this:
{code}
if (job.inited()) {
job.kill();
} else {
job.terminateJob(JobStatus.KILLED);
}
{code}
It will be good if we can wrap this into a single useful method for easy use.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639746#action_12639746 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
bq. Sameer:When is a JobInProgress created, as soon as a job is submitted or when it is considered for running?
as soon as a job is submitted
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639748#action_12639748 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
also fixed the code as rightly pointed by Sameer.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639752#action_12639752 ]
Arun C Murthy commented on HADOOP-4236:
---------------------------------------
There still is a race condn. in kill() - it might lead to terminate being called more than once i.e. from initTasks() and kill() ...
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639699#action_12639699 ]
Sameer Paranjpye commented on HADOOP-4236:
------------------------------------------
When is a JobInProgress created, as soon as a job is submitted or when it is considered for running??
If the former then having a 'initStarted' flag would make sense. Because {{if (!initStarted)}} then it could be a long time before a job is considered for running and it'll hang around in the queue, killed but not purged. If so, I think this patch does the right thing.
Regardless of whether or not we need an {{initStarted}} flag, I think it's cleaner to create a {{JobInitKillStatus}} class with the flags as members and synchronizie around an instance of this class.
As far as this patch goes, I think
{code}
synchronized(tasksInited){
if(jobKilled) {
terminateJob(JobStatus.KILLED);
return;
}
initStarted = true;
}
{code}
should just be
{code}
synchronized(tasksInited){
if(jobKilled) {
return;
}
initStarted = true;
}
{code}
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639353#action_12639353 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
>> Shouldn't the change in kill() method be done in terminate() method, because the same is valid for JobInProgress.fail() also.
we don't need for fail() as it is never called from user, also Jobtracker/scheduler should not call fail and initTasks simultaneously.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639309#action_12639309 ]
Amareshwari Sriramadasu commented on HADOOP-4236:
-------------------------------------------------
Shouldn't the change in kill() method be done in terminate() method, because the same is valid for JobInProgress.fail() also.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639902#action_12639902 ]
Hadoop QA commented on HADOOP-4236:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12392172/4236_v5.patch
against trunk revision 704818.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3464/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3464/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3464/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3464/console
This message is automatically generated.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch, 4236_v5.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Attachment: 4236_v4.patch
As suggested by Arun/Sameer, moved the flags to a private class JobInitKillStatus.
I have retained initStarted for the reason that schedulers may take time to get to initTasks for the job if there are lot of jobs or scheduler threads are busy with doing init of other jobs.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4236:
----------------------------------
Status: Open (was: Patch Available)
Ok, I had a long discussion with Sameer and Owen about this and here is a slightly modfied approach we feel is slightly more maintainable:
1. Introduce a new job-status class with 2 booleans: killJob and initTasks.
2. JobInProgress.kill should synchronize on the new status object, set the 'killJob' boolean to true (via a setter) and then check the 'initTasks' and call terminate if necessary while holding the lock on the status object.
3. Similarly JobInProgress.initTasks should synchronize on the status object at the _end_ (not at the beginning, there-by eliminating the need for the 'initStarted' flag) and call terminate if necessary while holding the lock on the status object.
Basically, the proposal is to keep a single object which tracks killJob and initTasks simultaneously rather than relying on too many pieces.
Thoughts?
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Attachment: 4236_v1.patch
this patch tries to solve the problem by setting flags. however, the ideal way to solve would be to have granular and one directional locking in jobtracker - HADOOP-869.
- also it makes the terminateJob api back to private. Now EagerTaskInitializationListener uses fail().
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das reassigned HADOOP-4236:
-----------------------------------
Assignee: Sharad Agarwal
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated HADOOP-4236:
-----------------------------------
Attachment: 4236_v2.patch
kill() in JobInProgress has been made reentrant.
Informing clients about the already issued kill is non-trivial as it will require interface change. Also, we don't do this for others commands like kill task. So I think it shouldn't be done as part of this jira.
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639761#action_12639761 ]
Arun C Murthy commented on HADOOP-4236:
---------------------------------------
I take my previous comment back... I apologise for my brain's comatose state.
One minor comment is that we need to heavily document the fact that 'terminate' is re-entrant...
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4236) JobTracker.killJob() fails to kill a
job if the job is not yet initialized
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy resolved HADOOP-4236.
-----------------------------------
Resolution: Fixed
I just committed this. Thanks, Sharad!
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch, 4236_v5.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4236) JobTracker.killJob() fails to kill
a job if the job is not yet initialized
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639824#action_12639824 ]
Sharad Agarwal commented on HADOOP-4236:
----------------------------------------
test-patch output
{code}
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
{code}
all core tests passed on my machine
> JobTracker.killJob() fails to kill a job if the job is not yet initialized
> --------------------------------------------------------------------------
>
> Key: HADOOP-4236
> URL: https://issues.apache.org/jira/browse/HADOOP-4236
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Amar Kamat
> Assignee: Sharad Agarwal
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4236_v1.patch, 4236_v2.patch, 4236_v3.patch, 4236_v4.patch, 4236_v5.patch
>
>
> HADOOP-3864 made the following changes to {{JobTracker.killJob()}}
> {code}
> public synchronized void killJob(JobID jobid) {
> JobInProgress job = jobs.get(jobid);
> - job.kill();
> + if (job.inited()) {
> + job.kill();
> + }
> }
> {code}
> This is a bug as a job will not get killed if its not yet initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.