[jira] [Assigned] (ARROW-12299) [Python] pq.write_to_dataset does not recognize S3FileSystem

["David Li (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Li reassigned ARROW-12299:
--------------------------------

    Assignee: Joris Van den Bossche

> [Python] pq.write_to_dataset does not recognize S3FileSystem
> ------------------------------------------------------------
>
>                 Key: ARROW-12299
>                 URL: https://issues.apache.org/jira/browse/ARROW-12299
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 3.0.0
>            Reporter: Samuel Sanders
>            Assignee: Joris Van den Bossche
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> {code:java}
>     pq.write_to_dataset(pa.concat_tables(pa_tables),
>   File "C:\venv\*\lib\site-packages\pyarrow\parquet.py", line 1914, in write_to_dataset
>     fs, root_path = legacyfs.resolve_filesystem_and_path(root_path, filesystem)
>   File "C:\venv\*\lib\site-packages\pyarrow\filesystem.py", line 474, in resolve_filesystem_and_path
>     filesystem = _ensure_filesystem(filesystem)
>   File "C:\venv\*\lib\site-packages\pyarrow\filesystem.py", line 457, in _ensure_filesystem
>     raise OSError('Unrecognized filesystem: {}'.format(fs_type))
> OSError: Unrecognized filesystem: <class 'pyarrow._s3fs.S3FileSystem'>
> {code}
> Creating the S3FileSystem these two ways produced the above error when invoking parquet.write_to_dataset with filesystem=s3_filesystem:
> {code:java}
>     s3_filesystem = file_system.S3FileSystem(region='us-east-1')
>     s3_filesystem, path = file_system.FileSystem.from_uri("s3://{0}".format(PARQUET_BUCKET))
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)