You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/07/24 17:30:00 UTC

[jira] [Created] (HADOOP-16458) LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3

Steve Loughran created HADOOP-16458:
---------------------------------------

             Summary: LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
                 Key: HADOOP-16458
                 URL: https://issues.apache.org/jira/browse/HADOOP-16458
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.3.0
         Environment: S3 + S3Guard
            Reporter: Steve Loughran
            Assignee: Steve Loughran


Intermittent failure of LocatedFileStatusFetcher.getFileStatuses(), which is using globStatus to find files.

I'd say "turn s3guard on" except this appears to be the case, and the dataset being read is
over 1h old.

Which means it is harder than I'd like to blame S3 for what would sound like an inconsistency

We're hampered by the number of debug level statements in the globber code being approximately none; there's no debugging to turn on. All we know is that globFiles returns null without any explanation.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org