You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/03/22 11:23:00 UTC
[jira] [Commented] (HADOOP-8870) NullPointerException when glob
doesn't return files
[ https://issues.apache.org/jira/browse/HADOOP-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798920#comment-16798920 ]
Steve Loughran commented on HADOOP-8870:
----------------------------------------
possibly HADOOP-15748 found in the older s3n connector too: list inconsistencies breaking the globber code
> NullPointerException when glob doesn't return files
> ---------------------------------------------------
>
> Key: HADOOP-8870
> URL: https://issues.apache.org/jira/browse/HADOOP-8870
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/s3
> Affects Versions: 0.20.205.0, 1.0.3
> Reporter: Jaka Jancar
> Priority: Major
> Fix For: 2.2.0
>
>
> Reading
> {code}s3n://bucket/{a/,b/,c/}{code}
> if one of the globs matches nothing, I get:
> {code}
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:992)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:177)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
> at spark.HadoopRDD.<init>(HadoopRDD.scala:51)
> at spark.SparkContext.hadoopFile(SparkContext.scala:186)
> at spark.SparkContext.textFile(SparkContext.scala:155)
> at com.celtra.analyzer.LogAnalyzer.analyzeSufficientS3Logs(LogAnalyzer.scala:52)
> at com.celtra.analyzer.App$.main(App.scala:164)
> at com.celtra.analyzer.App.main(App.scala)
> {code}
> I'm not sure whether this is specific to S3 or all filesystems.
> This was occuring in 0.20.205 and I confirmed it's still present in 1.0.3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org