[jira] [Commented] (ARROW-17985) [Python] Opaque error code ([code: 100]), when not setting region

["Miles Granger (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-17985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17624393#comment-17624393 ] 

Miles Granger commented on ARROW-17985:
---------------------------------------

Pretty uncomfortable indeed. :( 

It appears the relevant code here in [s3fs.cc |https://github.com/apache/arrow/blob/7f6c5aeb5388936709642e48aed6419d1e2144a6/cpp/src/arrow/filesystem/s3fs.cc#L2305] has determined that the bucket exists, but some other error has happened and doesn't appear it does anything to obfuscate it. Infact the ``GetError`` is right from AWS' C++ library: [Outcome.h | https://github.com/aws/aws-sdk-cpp/blob/b469499e96b957c478d121086272ea496c9b92e8/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h#L147] Maybe just bad error code from AWS? 

Suggest maybe opening an issue with them, feel free to link to this issue/comment here in case I'm mistaken. :) 

> [Python] Opaque error code ([code: 100]), when not setting region
> -----------------------------------------------------------------
>
>                 Key: ARROW-17985
>                 URL: https://issues.apache.org/jira/browse/ARROW-17985
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Vedant Roy
>            Priority: Minor
>
> A few odd things are going on with the Python bindings:
>  # Statefulness. I ran the following code:
> {code:java}
> import os
> import pyarrow.fs as arrow_fs
> def fs_():
>     s3_fs = arrow_fs.S3FileSystem(
>         access_key="<token>",
>         secret_key="<token>",
>         endpoint_override="<cloudflare r2 url>",
>     )
>     return s3_fs
> fs = fs_()
> print(fs.get_file_info("data"))
> {code}
> and it worked on one machine but not the other. Only setting
> {code:java}
> region="auto"
> {code}
>  allowed the code to work consistently on both computers.
> Furthermore, the error message is very opaque:
> {code:java}
> Traceback (most recent call last):
>   File "cluster_scripts/test_s3.py", line 51, in <module>
>     print(fs.get_file_info("data"))
>   File "pyarrow/_fs.pyx", line 439, in pyarrow._fs.FileSystem.get_file_info
>   File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
>   File "pyarrow/error.pxi", line 114, in pyarrow.lib.check_status
> OSError: When getting information for bucket 'data': AWS Error [code 100]: No response body.
> {code}
> Googling this error gives no information whatsoever. I managed to figure out the issue by switching from Cloudflare to S3, and when the issue was still going on, I explicitly set a region, but the experience was pretty painful.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)