You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Nicola Crane (Jira)" <ji...@apache.org> on 2021/12/16 07:41:00 UTC
[jira] [Updated] (ARROW-15127) [R] More visible documentation of AWS_EC2_METADATA_DISABLED=TRUE
[ https://issues.apache.org/jira/browse/ARROW-15127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicola Crane updated ARROW-15127:
---------------------------------
Summary: [R] More visible documentation of AWS_EC2_METADATA_DISABLED=TRUE (was: More visible documentation of AWS_EC2_METADATA_DISABLED=TRUE)
> [R] More visible documentation of AWS_EC2_METADATA_DISABLED=TRUE
> ----------------------------------------------------------------
>
> Key: ARROW-15127
> URL: https://issues.apache.org/jira/browse/ARROW-15127
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Affects Versions: 6.0.1
> Reporter: Carl Boettiger
> Priority: Major
>
> S3Filesystem operations can fail with opaque curl errors that are hard to debug from R, as I could see no way of accessing what URL or header information was being sent even when we enter the R debugger. We are left with relatively opaque errors such as:
> {code:java}
> Error: IOError: When resolving region for bucket 'analysis': AWS Error [code 99]: curlCode: 6, Couldn't resolve host name {code}
> which could have a whole host of underlying causes, from incorrect URL construction (e.g. incorrectly specified endpoint_override, default_region, etc) to other network issues https certificate failure or network connectivity problems.
> In any event, a common source of this seems to be collisions with AWS credentials in environmental variables or elsewhere interfering with the intended credentials (which may be fine with 'default' values – e.g. like when accessing a MINIO bucket we do not need a "region" (actually it's not needed in AWS as it can be attained from the endpoint; or at least that is the argument given by MINIO).
> In a recent case, I'd already made sure I didn't have any AWS_DEFAULT_REGION or AWS_S3_ENDPOINT env var set, but still kept getting this error on a MINIO bucket on one machine , but not others.
> Neal pointed out to me the existence of `AWS_EC2_METADATA_DISABLED`, and indeed, setting that to TRUE resolved the issue. Others may also be unaware of this option, it might be good to highlight in the S3Filesystem vignette and/or docs.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)