You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Araceli Henley (Created) (JIRA)" <ji...@apache.org> on 2011/11/15 22:13:51 UTC
[jira] [Created] (PIG-2371) attempt to load a non-existen file
results in a stack trace rather an error message
attempt to load a non-existen file results in a stack trace rather an error message
-----------------------------------------------------------------------------------
Key: PIG-2371
URL: https://issues.apache.org/jira/browse/PIG-2371
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.2
Environment: -bash-3.1$ hadoop version
Hadoop 0.23.0.1111080202
Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
>From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
-bash-3.1$ pig -version
Apache Pig version 0.9.2.1111101150 (r1200499)
compiled Nov 10 2011, 19:50:15
Reporter: Araceli Henley
Priority: Trivial
a = load 'does_not_exist' using PigStorage();
dump a;
Failed Jobs:
JobId Alias Feature Message Outputs
job_1321041443489_2010 a MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://gsbl90892.blue.ygrid.yahoo.com:8020/user/hadoopqa/does_not_exist
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:282)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:360)
at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1159)
at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1156)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1156)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:233)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://gsbl90892.blue.ygrid.yahoo.com:8020/user/hadoopqa/does_not_exist
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:270)
... 12 more
hdfs://gsbl90892.blue.ygrid.yahoo.com:8020/tmp/temp1739481333/tmp-502339,
Backend error message
---------------------
AttemptID:attempt_1321041443489_2008_m_000001_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2371) attempt to load a non-existen file
results in a stack trace rather an error message
Posted by "Renato Javier Marroquín Mogrovejo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444708#comment-13444708 ]
Renato Javier Marroquín Mogrovejo commented on PIG-2371:
--------------------------------------------------------
Hi,
So we could fix this within the getSlit method from the PigInputFormat class, my guess is that inputs being loaded from conf.get("pig.inputs") is returning nothing so could we just add a validation for this or let the ReadToEndLoader to verify this, what is the consensus on this one? IMHO ReadToEndLoader should do it while in the init() method. Exactly before this code:
{code}
try {
inpSplits = inputFormat.getSplits(HadoopShims.createJobContext(conf,
new JobID()));
} catch (InterruptedException e) {
throw new IOException(e);
}
{\code}
> attempt to load a non-existen file results in a stack trace rather an error message
> -----------------------------------------------------------------------------------
>
> Key: PIG-2371
> URL: https://issues.apache.org/jira/browse/PIG-2371
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Environment: -bash-3.1$ hadoop version
> Hadoop 0.23.0.1111080202
> Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common -r 1196973
> Compiled by hadoopqa on Tue Nov 8 02:12:04 PST 2011
> From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
> -bash-3.1$ pig -version
> Apache Pig version 0.9.2.1111101150 (r1200499)
> compiled Nov 10 2011, 19:50:15
> Reporter: Araceli Henley
> Priority: Trivial
>
> a = load 'does_not_exist' using PigStorage();
> dump a;
> Failed Jobs:
> JobId Alias Feature Message Outputs
> job_1321041443489_2010 a MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs://gsbl90892.blue.ygrid.yahoo.com:8020/user/hadoopqa/does_not_exist
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:282)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:360)
> at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1159)
> at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1156)
> at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
> at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:233)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://gsbl90892.blue.ygrid.yahoo.com:8020/user/hadoopqa/does_not_exist
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:270)
> ... 12 more
> hdfs://gsbl90892.blue.ygrid.yahoo.com:8020/tmp/temp1739481333/tmp-502339,
> Backend error message
> ---------------------
> AttemptID:attempt_1321041443489_2008_m_000001_0 Info:Error: java.lang.RuntimeException: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:119)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:627)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
> Caused by: java.io.IOException: Can't get JobTracker Kerberos principal for use as renewer
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.pig.impl.io.ReadToEndLoader.init(ReadToEndLoader.java:154)
> at org.apache.pig.impl.io.ReadToEndLoader.<init>(ReadToEndLoader.java:116)
> at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.loadPartitionFileFromLocalCache(MapRedUtil.java:101)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner.setConf(SkewedPartitioner.java:114)
> ... 10 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira