You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Anupam Seth (Created) (JIRA)" <ji...@apache.org> on 2011/10/13 23:17:12 UTC
[jira] [Created] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Terasort fails with Kerberos exception on secure cluster
--------------------------------------------------------
Key: MAPREDUCE-3181
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Anupam Seth
Assignee: Anupam Seth
We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129741#comment-13129741 ]
Thomas Graves commented on MAPREDUCE-3181:
------------------------------------------
I haven't looked in detail at this but taking a quick look the bug doesn't show up without kerberos enabled because it doesn't ever try to load the yarn-site.xml without security on. Looking at the stack trace:
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
The obtainTokensForNamenodes eventually calls Master.getMasterPrincipal(conf) but only if security is enabled. There is if isSecurityEnabled in obtainTokensForNamenodes. getMasterPrinicipal then expects yarn conf to be loaded because if the framework is yarn it does:
masterAddress = conf.get(YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS);
Its not all jobs that fail. teragen, wordcount, etc. work fine but they don't take the same path.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130377#comment-13130377 ]
Mahadev konar commented on MAPREDUCE-3181:
------------------------------------------
@arun/@anupam,
Make sure ant tests are all fine with this change. I think Hitesh might have tried it already.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127981#comment-13127981 ]
Mahadev konar commented on MAPREDUCE-3181:
------------------------------------------
Anupam,
Thinking abt this patch, how are other secure jobs running without this patch?
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3181:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130991#comment-13130991 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1129 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1129/])
MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and yarn-site.xml.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186458
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130392#comment-13130392 ]
Hadoop QA commented on MAPREDUCE-3181:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12499640/MAPREDUCE-3181.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 160 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1064//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1064//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1064//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1064//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1064//console
This message is automatically generated.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127885#comment-13127885 ]
Mahadev konar commented on MAPREDUCE-3181:
------------------------------------------
+1 looks good to me. Anupam can you give the ant tests a try and see if they are ok with this change?
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Anupam Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth updated MAPREDUCE-3181:
-----------------------------------
Attachment: MAPREDUCE-3181-branch0_23.patch
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Attachments: MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130978#comment-13130978 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #25 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/25/])
Merge -c 1186458 from trunk to branch-0.23 to complete fix for MAPREDUCE-3181.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186459
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130306#comment-13130306 ]
Arun C Murthy commented on MAPREDUCE-3181:
------------------------------------------
Ok, I spent more time on this. I think I finally figured this out.
The reason 'normal' jobs work is that they don't call InputFormat.getSplits until they create a YARNRunner which, in turn, creates a YarnConfiguration for the first time. YarnConfiguration has a static block which adds yarn-(default,site).xml as 'default resources' into the Configuration. Thus any configuration created after loads these.
The fix afaics is to add yarn-(default,site).xml as default resources in JobConf via org.apache.hadoop.mapreduce.utilConfigUtil.loadResources the same time we add mapred-(default,site).xml.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130992#comment-13130992 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #25 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/25/])
Merge -c 1186458 from trunk to branch-0.23 to complete fix for MAPREDUCE-3181.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186459
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3181:
-------------------------------------
Status: Open (was: Patch Available)
Anupam, something is weird. I can run secure jobs without this patch, we still need to investigate... any updates on this? Thanks.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130574#comment-13130574 ]
Thomas Graves commented on MAPREDUCE-3181:
------------------------------------------
doesn't this now make the client yarn specific? should it somehow be conditionalized on mapreduce.framework.name
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130348#comment-13130348 ]
Arun C Murthy commented on MAPREDUCE-3181:
------------------------------------------
Anupam, can you pls check to see if it works for you? Tx
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3181:
-------------------------------------
Status: Open (was: Patch Available)
Anupam - this might be a bug in terasort itself. I can run normal sort. Might be due to the way it tries to construct input splits? I'd like to understand this a bit better before we commit this fix...
Also teragen might be broken with MAPREDUCE-2702, can you pls check?
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127964#comment-13127964 ]
Aaron T. Myers commented on MAPREDUCE-3181:
-------------------------------------------
Perhaps I'm missing something obvious, but I fail to see why this bug would only manifest itself with Kerberos enabled.
Also, can no test be written which would expose this bug and ensure it won't regress?
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Anupam Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129968#comment-13129968 ]
Anupam Seth commented on MAPREDUCE-3181:
----------------------------------------
Apologies - tried to write a unit test in a few different ways, but without using PowerMockito, I cannot seem to be able to make it non-trivial and non-absurd while actually testing the issue at hand (which is difficult IMO because of the way the default yarn-site.xml is set to be an empty configuration).
Thus, my patch remains as it is. Kindly let me know if you would like me to include the PowerMockito stub or if someone has a better way to unit test, else requesting for +1 review and commit.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130957#comment-13130957 ]
Thomas Graves commented on MAPREDUCE-3181:
------------------------------------------
makes sense if it doesn't hurt anything.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130355#comment-13130355 ]
Arun C Murthy commented on MAPREDUCE-3181:
------------------------------------------
I found another bug in TeraGen - TeraOutputFormat is broken, it overrides FileOutputCommitter and does nothing.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130970#comment-13130970 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1110 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1110/])
MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and yarn-site.xml.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186458
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3181:
-------------------------------------
Attachment: MAPREDUCE-3181.patch
Ok, this works - tested on a secure cluster.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Anupam Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129745#comment-13129745 ]
Anupam Seth commented on MAPREDUCE-3181:
----------------------------------------
Thanks Tom. I hope your explanation helps clear up Aaron's and Arun's questions. As Tom pointed out, this problem is seem only on Terasort in a secure cluster because of the conditional to not even try this if security is disabled and because this gets hit before the ResourceManagerDelegate is ever called or created (causing yarn-site.xml to be 'normally' loaded). This does not happen with Teragen, for example, because TeraSort attempts to write the input splits to HDFS before running the actual MR.
Regarding the ant tests asked for by Mahadev, I see numerous failures due to timeout issues, most likely an artifact of MAPREDUCE-3176, as my changes are very simple.
Also, I have patch ready with an added unit test to ensure this does not regress - am running tests on it, will put it up shortly.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130969#comment-13130969 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #24 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/24/])
Merge -c 1186458 from trunk to branch-0.23 to complete fix for MAPREDUCE-3181.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186459
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3181:
-------------------------------------
Assignee: Arun C Murthy (was: Anupam Seth)
Status: Patch Available (was: Open)
Anupam, if you don't mind I'll take this over. Thanks.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131573#comment-13131573 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #45 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/45/])
Merge -c 1186458 from trunk to branch-0.23 to complete fix for MAPREDUCE-3181.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186459
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130973#comment-13130973 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1189 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1189/])
MAPREDUCE-3181. Fixed MapReduce runtime to load yarn-default.xml and yarn-site.xml.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186458
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Anupam Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth updated MAPREDUCE-3181:
-----------------------------------
Attachment: MAPREDUCE-3181-branch0_23.patch
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3181:
-------------------------------------
Attachment: MAPREDUCE-3181.patch
Straight-forward patch.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Anupam Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth updated MAPREDUCE-3181:
-----------------------------------
Fix Version/s: 0.23.0
Status: Patch Available (was: Open)
A straightforward change. Kindly review and let me know if it is acceptable or if there is a better way to do the same.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127900#comment-13127900 ]
Hadoop QA commented on MAPREDUCE-3181:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12499101/MAPREDUCE-3181-branch0_23.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated 20 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1028//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1028//console
This message is automatically generated.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131610#comment-13131610 ]
Hudson commented on MAPREDUCE-3181:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #57 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/57/])
Merge -c 1186458 from trunk to branch-0.23 to complete fix for MAPREDUCE-3181.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1186459
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130860#comment-13130860 ]
Arun C Murthy commented on MAPREDUCE-3181:
------------------------------------------
I thought about it, there isn't a good way to effect that. For now, this seems like a simple, straight-fwd change. Makes sense?
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181.patch, MAPREDUCE-3181.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Anupam Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth updated MAPREDUCE-3181:
-----------------------------------
Status: Patch Available (was: Open)
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch, MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3181) Terasort fails with Kerberos
exception on secure cluster
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3181:
-------------------------------------
Priority: Blocker (was: Major)
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
>
> We are seeing the following Kerberos exception upon trying to run terasort on secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira