You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Siddharth Seth (Created) (JIRA)" <ji...@apache.org> on 2012/02/04 20:51:53 UTC
[jira] [Created] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
NPE in FileOutputCommitter when running a 0 reduce job
------------------------------------------------------
Key: MAPREDUCE-3808
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Siddharth Seth
Priority: Blocker
This was while running LoadGen.
{noformat}
Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
at org.apache.hadoop.fs.Path.<init>(Path.java:56)
at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
at org.apache.hadoop.mapred.Task.done(Task.java:912)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201597#comment-13201597 ]
Hadoop QA commented on MAPREDUCE-3808:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513481/MR-3808.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1794//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1794//console
This message is automatically generated.
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202326#comment-13202326 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #162 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/162/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
svn merge --ignore-ancestry -c 1241217 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241218
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201763#comment-13201763 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #509 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/509/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
svn merge --ignore-ancestry -c 1241217 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241218
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201719#comment-13201719 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #499 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/499/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
svn merge --ignore-ancestry -c 1241217 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241218
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201713#comment-13201713 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1745 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1745/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241217
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201701#comment-13201701 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #488 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/488/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
svn merge --ignore-ancestry -c 1241217 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241218
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201783#comment-13201783 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1687 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1687/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241217
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201471#comment-13201471 ]
Robert Joseph Evans commented on MAPREDUCE-3808:
------------------------------------------------
Sure I can look into it. It appears that for some reason mapreduce.output.fileoutputformat.outputdir is not set in the config.
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201695#comment-13201695 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1672 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1672/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241217
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202353#comment-13202353 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #184 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/184/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
svn merge --ignore-ancestry -c 1241217 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241218
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3808:
-----------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.1
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Release Note: Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces.
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this to trunk and branch-0.23. Thanks Robert!
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201476#comment-13201476 ]
Robert Joseph Evans commented on MAPREDUCE-3808:
------------------------------------------------
Well looking at the rest of the code it looks like the FileOutputCommitter becomes a NOOP when the output dir is not set, so I need to replicate that here too.
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Vinod Kumar Vavilapalli (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli reassigned MAPREDUCE-3808:
--------------------------------------------------
Assignee: Robert Joseph Evans
Rober, can you please take a crack at this one?
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201474#comment-13201474 ]
Robert Joseph Evans commented on MAPREDUCE-3808:
------------------------------------------------
OK I am a bit confused here. Why is it allowed to run a map only job that does not have an output directory set? The FileOutputFormat blows up if there are any reduces and the output directory is not set, but for a map only job it is fine. What is supposed to happen in that situation? I can replicate it, but I don't know what the previous behavior was.
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201670#comment-13201670 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3808:
----------------------------------------------------
This looks good. Simple test too.
+1. Am pushing this in.
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202390#comment-13202390 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #982 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/982/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241217
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-3808:
-------------------------------------------
Target Version/s: 0.23.1, 0.24.0
Affects Version/s: 0.24.0
Status: Patch Available (was: Open)
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-3808:
-------------------------------------------
Attachment: MR-3808.txt
This patch should fix the issue. It adds in a check for null on the output in several places when building paths and short curcits those methods to return null. It also adds in the a null output check in recoverTask and causes it to be a noop when output is null. Finally it adds in tests to validate that the FileOutputCommitter handles that case correctly.
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3808) NPE in FileOutputCommitter when
running a 0 reduce job
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202309#comment-13202309 ]
Hudson commented on MAPREDUCE-3808:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #949 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/949/])
MAPREDUCE-3808. Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. Contributed by Robert Joseph Evans.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241217
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
> NPE in FileOutputCommitter when running a 0 reduce job
> ------------------------------------------------------
>
> Key: MAPREDUCE-3808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MR-3808.txt
>
>
> This was while running LoadGen.
> {noformat}
> Error: java.lang.NullPointerException at org.apache.hadoop.fs.Path.<init>(Path.java:67)
> at org.apache.hadoop.fs.Path.<init>(Path.java:56)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
> at org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
> at org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
> at org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
> at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
> at org.apache.hadoop.mapred.Task.done(Task.java:912)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira