You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Alessandro Molina (Jira)" <ji...@apache.org> on 2021/07/01 12:47:00 UTC
[jira] [Created] (ARROW-13237) S3 FileSystem doesn't seem to handle
redirects
Alessandro Molina created ARROW-13237:
-----------------------------------------
Summary: S3 FileSystem doesn't seem to handle redirects
Key: ARROW-13237
URL: https://issues.apache.org/jira/browse/ARROW-13237
Project: Apache Arrow
Issue Type: Bug
Components: C++, Python
Affects Versions: 4.0.1
Reporter: Alessandro Molina
Fix For: 5.0.0
In some conditions AWS S3 seems to respond with a redirect, but Arrow seems to consider it an error instead of following the redirect.
For example see
{code}
s3, _ = fs.S3FileSystem.from_uri("s3://ursa-labs-taxi-data/")
print(s3.get_file_info(fs.FileSelector("2011", recursive=True)))
{code}
The error that you get is
{code}
OSError: When listing objects under key '' in bucket '2011': AWS Error [code 100]: Unable to parse ExceptionName: PermanentRedirect Message: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
{code}
There are two oddities in that error:
1) It should probably follow the `PermanentRedirect`
2) The error states that the bucket name is `2011` while the bucket name was `ursa-labs-taxi-data`
--
This message was sent by Atlassian Jira
(v8.3.4#803005)