You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Karam Singh (JIRA)" <ji...@apache.org> on 2009/09/15 09:05:57 UTC
[jira] Created: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
TaskTracker does not cleans up job directory when LinuxTraskController is used.
-------------------------------------------------------------------------------
Key: MAPREDUCE-981
URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Karam Singh
TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker does not cleans up
job directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829005#action_12829005 ]
Amareshwari Sriramadasu commented on MAPREDUCE-981:
---------------------------------------------------
Whenever INITIALIZE_JOB fails, purgeJob would get NullPointerException in the following line:
{code}
removeJobFiles(rjob.jobConf.getUser(), rjob.getJobID().toString());
{code}
So, the issue is not fixed by MAPREDUCE-899.
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Summary: TaskTracker.purgeJob fails with NPE if the job is partially localized. (was: TaskTracker does not cleans up job directory when LinuxTraskController is used.)
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker.purgeJob fails with
NPE if the job is partially localized.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843542#action_12843542 ]
Hadoop QA commented on MAPREDUCE-981:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12438383/patch-981-2.txt
against trunk revision 921230.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/30/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/30/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/30/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/30/console
This message is automatically generated.
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Status: Patch Available (was: Open)
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Attachment: patch-981-3.txt
Patch updated to trunk
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Status: Patch Available (was: Open)
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-981-1.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker.purgeJob fails with
NPE if the job is partially localized.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865656#action_12865656 ]
Hadoop QA commented on MAPREDUCE-981:
-------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12443822/patch-981-3.txt
against trunk revision 941564.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/534/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/534/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/534/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/534/console
This message is automatically generated.
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Attachment: patch-981-2.txt
Patch updated to trunk and incorporating Amar's comments.
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker does not cleans up
job directory when LinuxTraskController is used.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837040#action_12837040 ]
Hadoop QA commented on MAPREDUCE-981:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12436552/patch-981-1.txt
against trunk revision 912471.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/473/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/473/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/473/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/473/console
This message is automatically generated.
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-981-1.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Status: Open (was: Patch Available)
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Attachment: patch-981.txt
Patch for review
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker does not cleans up
job directory when LinuxTraskController is used.
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837237#action_12837237 ]
Amar Kamat commented on MAPREDUCE-981:
--------------------------------------
Few comments
1) Instead of adding UGI as a constructor parameter to RunningJob, why not do something like
{code}
rJob = new RunningJob(jobId);
rjob.ugi = UserGroupInformation.createRemoteUser(tip.getTask().getUser());
{code}
2) It would be nice to have a getUser() method in RunningJob which might internally return 'getUGI().getShortUserName()'. Given a UGI, which username to use might be confusing.
3) Using UtilsForTests.InlineCleanupQueue, the wait for cleanup can be avoided.
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-981-1.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker.purgeJob fails with
NPE if the job is partially localized.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864806#action_12864806 ]
Hadoop QA commented on MAPREDUCE-981:
-------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12443822/patch-981-3.txt
against trunk revision 941564.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/171/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/171/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/171/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/171/console
This message is automatically generated.
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Attachment: patch-981-1.txt
Changed purgeJob to get the user from runningJob.ugi, instead of storing username separately in RunningJob as in earlier patch. I'm passing the ugi to RunningJob constructor itself, since creating it after copying the files (as in trunk) is too late.
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-981-1.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker does not cleans up
job directory when LinuxTraskController is used.
Posted by "Karam Singh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755395#action_12755395 ]
Karam Singh commented on MAPREDUCE-981:
---------------------------------------
2009-09-15 06:50:41,794 WARN org.apache.hadoop.mapred.LinuxTaskController: Exit code from INITIALIZE_JOB is : 15
2009-09-15 06:50:41,796 WARN org.apache.hadoop.mapred.LinuxTaskController: Exception thrown by INITIALIZE_JOB : org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:243)
at org.apache.hadoop.util.Shell.run(Shell.java:170)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:363)
at org.apache.hadoop.mapred.LinuxTaskController.runCommand(LinuxTaskController.java:169)
at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:381)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:855)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1828)
at org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:102)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1793)
2009-09-15 06:50:41,796 INFO org.apache.hadoop.mapred.LinuxTaskController: Output from LinuxTaskController's INITIALIZE_JOB follows:
2009-09-15 06:50:41,796 INFO org.apache.hadoop.mapred.LinuxTaskController: main : command provided 1
2009-09-15 06:50:41,796 INFO org.apache.hadoop.mapred.LinuxTaskController: main : user is username
2009-09-15 06:50:41,796 INFO org.apache.hadoop.mapred.LinuxTaskController: Invalid file path. <job.xml> not user/group owned by the tasktracker.
2009-09-15 06:50:41,796 INFO org.apache.hadoop.mapred.LinuxTaskController: Failed to secure the job_dir <job-dir>
2009-09-15 06:50:41,796 WARN org.apache.hadoop.mapred.TaskTracker: Error initializing attempt_200909150645_0002_r_000008_1:
java.io.IOException: org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.mapred.LinuxTaskController.runCommand(LinuxTaskController.java:178)
at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:381)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:855)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1828)
at org.apache.hadoop.mapred.TaskTracker.access$1200(TaskTracker.java:102)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:1793)
Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:243)
at org.apache.hadoop.util.Shell.run(Shell.java:170)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:363)
at org.apache.hadoop.mapred.LinuxTaskController.runCommand(LinuxTaskController.java:169)
... 5 more
2009-09-15 06:50:41,797 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2009-09-15 06:50:42,288 WARN org.mortbay.log: /tasklog: java.io.IOException: Closed
2009-09-15 06:50:42,291 WARN org.mortbay.log: /tasklog: java.io.IOException: Closed
2009-09-15 06:50:45,307 WARN org.mortbay.log: /tasklog: java.io.IOException: Closed
2009-09-15 06:50:45,309 WARN org.mortbay.log: /tasklog: java.io.IOException: Closed
2009-09-15 06:50:56,665 INFO org.apache.hadoop.mapred.TaskTracker: Received 'KillJobAction' for job: job_200909150645_0002
2009-09-15 06:50:56,665 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_200909150645_0002_m_000016_1 not found in cache
2009-09-15 06:50:56,665 WARN org.apache.hadoop.mapred.TaskTracker: java.lang.NullPointerException
at org.apache.hadoop.mapred.TaskTracker.purgeJob(TaskTracker.java:1508)
at org.apache.hadoop.mapred.TaskTracker.access$100(TaskTracker.java:102)
at org.apache.hadoop.mapred.TaskTracker$1.run(TaskTracker.java:349)
at java.lang.Thread.run(Thread.java:619)
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Fix Version/s: 0.22.0
Status: Patch Available (was: Open)
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Component/s: tasktracker
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Giridharan Kesavan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Giridharan Kesavan updated MAPREDUCE-981:
-----------------------------------------
Status: Open (was: Patch Available)
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-981) TaskTracker.purgeJob fails with
NPE if the job is partially localized.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843910#action_12843910 ]
Amareshwari Sriramadasu commented on MAPREDUCE-981:
---------------------------------------------------
bq. -1 contrib tests.
TestStreamingBadRecords failed because of ZipException. The same test passed on my machine.
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-981) TaskTracker does not cleans up job
directory when LinuxTraskController is used.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu reassigned MAPREDUCE-981:
-------------------------------------------------
Assignee: Amareshwari Sriramadasu
> TaskTracker does not cleans up job directory when LinuxTraskController is used.
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated MAPREDUCE-981:
----------------------------------------------
Status: Open (was: Patch Available)
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE
if the job is partially localized.
Posted by "Giridharan Kesavan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Giridharan Kesavan updated MAPREDUCE-981:
-----------------------------------------
Status: Patch Available (was: Open)
> TaskTracker.purgeJob fails with NPE if the job is partially localized.
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-981
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Reporter: Karam Singh
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.22.0
>
> Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981.txt
>
>
> TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.