You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Samuel Sanders (Jira)" <ji...@apache.org> on 2021/04/08 13:35:00 UTC
[jira] [Created] (ARROW-12299) S3FileSystem is an Unrecognized
filesystem
Samuel Sanders created ARROW-12299:
--------------------------------------
Summary: S3FileSystem is an Unrecognized filesystem
Key: ARROW-12299
URL: https://issues.apache.org/jira/browse/ARROW-12299
Project: Apache Arrow
Issue Type: Bug
Components: Python
Affects Versions: 3.0.0
Reporter: Samuel Sanders
{code:java}
pq.write_to_dataset(pa.concat_tables(pa_tables),
File "C:\venv\*\lib\site-packages\pyarrow\parquet.py", line 1914, in write_to_dataset
fs, root_path = legacyfs.resolve_filesystem_and_path(root_path, filesystem)
File "C:\venv\*\lib\site-packages\pyarrow\filesystem.py", line 474, in resolve_filesystem_and_path
filesystem = _ensure_filesystem(filesystem)
File "C:\venv\*\lib\site-packages\pyarrow\filesystem.py", line 457, in _ensure_filesystem
raise OSError('Unrecognized filesystem: {}'.format(fs_type))
OSError: Unrecognized filesystem: <class 'pyarrow._s3fs.S3FileSystem'>
{code}
Creating the S3FileSystem these two ways produced the above error when invoking parquet.write_to_dataset with filesystem=s3_filesystem:
{code:java}
s3_filesystem = file_system.S3FileSystem(region='us-east-1')
s3_filesystem, path = file_system.FileSystem.from_uri("s3://{0}".format(PARQUET_BUCKET))
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)