[jira] [Commented] (ARROW-10473) [Python] FSSpecHandler get_file_info with recursive selector not working with s3fs

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264137#comment-17264137 ] 

Joris Van den Bossche commented on ARROW-10473:
-----------------------------------------------

I am not fully sure anymore what the failure was here, and in the referenced PR, the test that was added and originally was "xfailed", was later on enabled before merging the PR (https://github.com/apache/arrow/pull/8573/commits/3e6c866cdeca549043296129c22a2526ca23bb32). But I don't remember anymore why it stopped failing while writing that PR. 
The current parquet tests we have also pass with both s3fs 0.4.2 and 0.5.2. 

So in any case, a possible remaining failure is currently not known / captured by any test. So closing this issue. 

> [Python] FSSpecHandler get_file_info with recursive selector not working with s3fs
> ----------------------------------------------------------------------------------
>
>                 Key: ARROW-10473
>                 URL: https://issues.apache.org/jira/browse/ARROW-10473
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Joris Van den Bossche
>            Assignee: Joris Van den Bossche
>            Priority: Major
>
> The partitioned ParquetDataset tests are failing when using s3fs filesystem (I am adding tests in https://github.com/apache/arrow/pull/8573). 
> I need to come up with a more minimal test isolating the FileSystem.get_file_info behaviour, but from debugging the parquet tests it seems that it is only listing the first level (and not further nested directories/files).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)