[jira] [Created] (ARROW-10921) `TypeError: 'coroutine' object is not iterable` when reading parquet partitions via s3fs >= 0.5 with pyarrow

["Ivan Necas (Jira)" <ji...@apache.org>]

Ivan Necas created ARROW-10921:
----------------------------------

             Summary: `TypeError: 'coroutine' object is not iterable` when reading parquet partitions via s3fs >= 0.5 with pyarrow
                 Key: ARROW-10921
                 URL: https://issues.apache.org/jira/browse/ARROW-10921
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
            Reporter: Ivan Necas


Trying to read partitioned parquet files using updated s3fs {{>= 0.5 (using via }}{{dask}}), and got this error:

 
{code:python}
410 files = set()
 411 
--> 412 for key in list(self.fs._ls(path, refresh=refresh)):
 413 path = key['Key']
 414 if key['StorageClass'] == 'DIRECTORY':

TypeError: 'coroutine' object is not iterable

{code}
coming from [https://github.com/apache/arrow/blob/9baa123ea38ee9cc1d3a90cfc9347239cd28064c/python/pyarrow/filesystem.py#L415] 

 

Seems related to switching s3fs to asyncio in [https://github.com/dask/s3fs/pull/336.|https://github.com/dask/s3fs/pull/336]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)