You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Matthias Pohl (Jira)" <ji...@apache.org> on 2022/02/11 08:17:00 UTC
[jira] [Updated] (FLINK-26061) FileSystem.getFileStatus fails for directories

     [ https://issues.apache.org/jira/browse/FLINK-26061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Pohl updated FLINK-26061:
----------------------------------
    Description: 
{{FileSystem.getFileStatus}} or {{FileSystem.exists}} are not supported for empty directories in object stores (like s3). The s3 filesystem implementations used by Flink are working around that in different ways. {{flink-s3-fs-hadoop}} fixes this issue internally. {{flink-s3-fs-presto}} fails in certain situations. This is covered by already existing tests like [FileSystemBehaviorTestSuite:125|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/test/java/org/apache/flink/core/fs/FileSystemBehaviorTestSuite.java#L125].



  was:
{{FileSystem.getFileStatus}} or {{FileSystem.exists}} are not supported for empty directories in object stores (like s3). The s3 filesystem implementations used by Flink are working around that in different ways. {{flink-s3-fs-hadoop}} fixes this issue internally. {{flink-s3-fs-presto}} fails in certain situations. This is covered by already existing tests like 
 We might want to adjust the interface here. It's not clear based on the contract that for certain {{FileSystem}} implementations. This is especially unintuitive because there is a {{isDir()}} method provided by the resulting {{FileStatus}} instance.


> FileSystem.getFileStatus fails for directories
> ----------------------------------------------
>
>                 Key: FLINK-26061
>                 URL: https://issues.apache.org/jira/browse/FLINK-26061
>             Project: Flink
>          Issue Type: Technical Debt
>          Components: Connectors / FileSystem
>    Affects Versions: 1.15.0
>            Reporter: Matthias Pohl
>            Priority: Major
>
> {{FileSystem.getFileStatus}} or {{FileSystem.exists}} are not supported for empty directories in object stores (like s3). The s3 filesystem implementations used by Flink are working around that in different ways. {{flink-s3-fs-hadoop}} fixes this issue internally. {{flink-s3-fs-presto}} fails in certain situations. This is covered by already existing tests like [FileSystemBehaviorTestSuite:125|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-core/src/test/java/org/apache/flink/core/fs/FileSystemBehaviorTestSuite.java#L125].



--
This message was sent by Atlassian Jira
(v8.20.1#820001)