You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "chiyang (JIRA)" <ji...@apache.org> on 2013/10/08 04:54:41 UTC
[jira] [Created] (PIG-3507) It fails to run pig in local mode on a
Kerberos enabled Hadoop cluster
chiyang created PIG-3507:
----------------------------
Summary: It fails to run pig in local mode on a Kerberos enabled Hadoop cluster
Key: PIG-3507
URL: https://issues.apache.org/jira/browse/PIG-3507
Project: Pig
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: chiyang
It fails to run pig in local mode on a Kerberos enabled Hadoop cluster
*Command*
pig -x local <pig script>
*Pig script*
A = load '/etc/passwd';
dump A;
*Root cause*
When running pig in local mode, jobConf in HExecutionEngine is initiated with core-default.xml (hadoop.security.authentication = simple), mapred-default.xml, and yarn-default.xml. However, the settings are not passed to UserGroupInformation. That's why obtainTokensForNamenodesInternal() is called in obtainTokensForNamenodes(), and causes the exception to happen.
{noformat}
public static void obtainTokensForNamenodes(Credentials credentials, Path[] ps, Configuration conf) throws IOException {
if (!UserGroupInformation.isSecurityEnabled()) {
return;
}
obtainTokensForNamenodesInternal(credentials, ps, conf);
}
{noformat}
*Error*
Pig Stack Trace
---------------
ERROR 6000: Output Location Validation Failed for: 'file:/tmp/temp-308998488/tmp-2025176494 More info to follow:
Can't get JT Kerberos principal for use as renewer
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias A
at org.apache.pig.PigServer.openIterator(PigServer.java:841)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:604)
at org.apache.pig.Main.main(Main.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias A
at org.apache.pig.PigServer.storeEx(PigServer.java:940)
at org.apache.pig.PigServer.store(PigServer.java:903)
at org.apache.pig.PigServer.openIterator(PigServer.java:816)
... 12 more
Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 6000: Output Location Validation Failed for: 'file:/tmp/temp-308998488/tmp-2025176494 More info to follow:
Can't get JT Kerberos principal for use as renewer
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:95)
at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66)
at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:288)
at org.apache.pig.PigServer.compilePp(PigServer.java:1327)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252)
at org.apache.pig.PigServer.storeEx(PigServer.java:936)
... 14 more
Caused by: java.io.IOException: Can't get JT Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:129)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:111)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:85)
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:127)
at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80)
... 24 more
================================================================================
--
This message was sent by Atlassian JIRA
(v6.1#6144)