You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Araceli Henley (Created) (JIRA)" <ji...@apache.org> on 2011/11/15 21:30:51 UTC
[jira] [Created] (PIG-2370) SkewedParitioner results in Kerberos
error
SkewedParitioner results in Kerberos error
------------------------------------------
Key: PIG-2370
URL: https://issues.apache.org/jira/browse/PIG-2370
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.2
Environment: -bash-3.1$ hadoop version
Hadoop 0.23.0.1111080202
Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
>From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
-bash-3.1$ pig -version
USING: /homes/araceli/pighome/dotNext/current
Apache Pig version 0.9.2.1111101150 (r1200499)
compiled Nov 10 2011, 19:50:15
Reporter: Araceli Henley
Most of the SkewedJoin tests fail.
a = load '/user/user1/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name, age, gpa);
b = load '/user/user1/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
e = join a by name, b by name using 'skewed' parallel 8;
store e into '/user/user1/pig/out/user1.1321044742/SkewedJoin_1.out';
Backend error message
---------------------
AttemptID:attempt_1321041443489_0400_m_000000_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2370) SkewedParitioner results in Kerberos
error
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2370:
----------------------------
Attachment: PIG-2370-2.patch
PIG-2370-2.patch fix a unit test failure.
> SkewedParitioner results in Kerberos error
> ------------------------------------------
>
> Key: PIG-2370
> URL: https://issues.apache.org/jira/browse/PIG-2370
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Environment: -bash-3.1$ hadoop version
> Hadoop 0.23.0.1111080202
> Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
> Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
> From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
> -bash-3.1$ pig -version
> USING: /homes/araceli/pighome/dotNext/current
> Apache Pig version 0.9.2.1111101150 (r1200499)
> compiled Nov 10 2011, 19:50:15
> Reporter: Araceli Henley
> Assignee: Daniel Dai
> Labels: hadoop2.0
> Attachments: PIG-2370-1.patch, PIG-2370-2.patch
>
>
> Most of the SkewedJoin tests fail.
> a = load '/user/user1/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name, age, gpa);
> b = load '/user/user1/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
> e = join a by name, b by name using 'skewed' parallel 8;
> store e into '/user/user1/pig/out/user1.1321044742/SkewedJoin_1.out';
> Backend error message
> ---------------------
> AttemptID:attempt_1321041443489_0400_m_000000_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
> at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
> at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2370) SkewedParitioner results in Kerberos
error
Posted by "Thejas M Nair (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163948#comment-13163948 ]
Thejas M Nair commented on PIG-2370:
------------------------------------
+1
> SkewedParitioner results in Kerberos error
> ------------------------------------------
>
> Key: PIG-2370
> URL: https://issues.apache.org/jira/browse/PIG-2370
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Environment: -bash-3.1$ hadoop version
> Hadoop 0.23.0.1111080202
> Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
> Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
> From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
> -bash-3.1$ pig -version
> USING: /homes/araceli/pighome/dotNext/current
> Apache Pig version 0.9.2.1111101150 (r1200499)
> compiled Nov 10 2011, 19:50:15
> Reporter: Araceli Henley
> Assignee: Daniel Dai
> Attachments: PIG-2370-1.patch
>
>
> Most of the SkewedJoin tests fail.
> a = load '/user/user1/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name, age, gpa);
> b = load '/user/user1/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
> e = join a by name, b by name using 'skewed' parallel 8;
> store e into '/user/user1/pig/out/user1.1321044742/SkewedJoin_1.out';
> Backend error message
> ---------------------
> AttemptID:attempt_1321041443489_0400_m_000000_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
> at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
> at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2370) SkewedParitioner results in Kerberos
error
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2370:
----------------------------
Attachment: PIG-2370-1.patch
> SkewedParitioner results in Kerberos error
> ------------------------------------------
>
> Key: PIG-2370
> URL: https://issues.apache.org/jira/browse/PIG-2370
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Environment: -bash-3.1$ hadoop version
> Hadoop 0.23.0.1111080202
> Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
> Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
> From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
> -bash-3.1$ pig -version
> USING: /homes/araceli/pighome/dotNext/current
> Apache Pig version 0.9.2.1111101150 (r1200499)
> compiled Nov 10 2011, 19:50:15
> Reporter: Araceli Henley
> Assignee: Daniel Dai
> Attachments: PIG-2370-1.patch
>
>
> Most of the SkewedJoin tests fail.
> a = load '/user/user1/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name, age, gpa);
> b = load '/user/user1/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
> e = join a by name, b by name using 'skewed' parallel 8;
> store e into '/user/user1/pig/out/user1.1321044742/SkewedJoin_1.out';
> Backend error message
> ---------------------
> AttemptID:attempt_1321041443489_0400_m_000000_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
> at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
> at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2370) SkewedParitioner results in Kerberos
error
Posted by "Daniel Dai (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-2370.
-----------------------------
Resolution: Fixed
Fix Version/s: 0.11
0.9.2
0.10
Hadoop Flags: Reviewed
Unit tests pass. test-patch:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] -1 release audit. The applied patch generated 465 release audit warnings (more than the trunk's current 456 warnings).
No tests added since this is a regression. No new file added so ignore release audit warning.
Patch committed to trunk/0.10/0.9
> SkewedParitioner results in Kerberos error
> ------------------------------------------
>
> Key: PIG-2370
> URL: https://issues.apache.org/jira/browse/PIG-2370
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Environment: -bash-3.1$ hadoop version
> Hadoop 0.23.0.1111080202
> Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
> Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
> From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
> -bash-3.1$ pig -version
> USING: /homes/araceli/pighome/dotNext/current
> Apache Pig version 0.9.2.1111101150 (r1200499)
> compiled Nov 10 2011, 19:50:15
> Reporter: Araceli Henley
> Assignee: Daniel Dai
> Labels: hadoop2.0
> Fix For: 0.10, 0.9.2, 0.11
>
> Attachments: PIG-2370-1.patch, PIG-2370-2.patch
>
>
> Most of the SkewedJoin tests fail.
> a = load '/user/user1/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name, age, gpa);
> b = load '/user/user1/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
> e = join a by name, b by name using 'skewed' parallel 8;
> store e into '/user/user1/pig/out/user1.1321044742/SkewedJoin_1.out';
> Backend error message
> ---------------------
> AttemptID:attempt_1321041443489_0400_m_000000_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
> at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
> at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PIG-2370) SkewedParitioner results in Kerberos
error
Posted by "Daniel Dai (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai reassigned PIG-2370:
-------------------------------
Assignee: Daniel Dai
> SkewedParitioner results in Kerberos error
> ------------------------------------------
>
> Key: PIG-2370
> URL: https://issues.apache.org/jira/browse/PIG-2370
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Environment: -bash-3.1$ hadoop version
> Hadoop 0.23.0.1111080202
> Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
> Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
> From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
> -bash-3.1$ pig -version
> USING: /homes/araceli/pighome/dotNext/current
> Apache Pig version 0.9.2.1111101150 (r1200499)
> compiled Nov 10 2011, 19:50:15
> Reporter: Araceli Henley
> Assignee: Daniel Dai
>
> Most of the SkewedJoin tests fail.
> a = load '/user/user1/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name, age, gpa);
> b = load '/user/user1/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
> e = join a by name, b by name using 'skewed' parallel 8;
> store e into '/user/user1/pig/out/user1.1321044742/SkewedJoin_1.out';
> Backend error message
> ---------------------
> AttemptID:attempt_1321041443489_0400_m_000000_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
> at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
> at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira