[jira] (ARROW-15127) [R] More visible documentation of AWS_EC2_METADATA_DISABLED=TRUE

["Carl Boettiger (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-15127 ]


    Carl Boettiger deleted comment on ARROW-15127:
    ----------------------------------------

was (Author: cboettig):
Also, does this only work (or behave differently) on the dev release (6.0.2)?  Just tried rolling back to the latest release (6.0.1) on that machine, and now I get the curl error again even when setting this env var. 

> [R] More visible documentation of AWS_EC2_METADATA_DISABLED=TRUE
> ----------------------------------------------------------------
>
>                 Key: ARROW-15127
>                 URL: https://issues.apache.org/jira/browse/ARROW-15127
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: R
>    Affects Versions: 6.0.1
>            Reporter: Carl Boettiger
>            Priority: Major
>
> S3Filesystem operations can fail with opaque curl errors that are hard to debug from R, as I could see no way of accessing what URL or header information was being sent even when we enter the R debugger.  We are left with relatively opaque errors such as:
> {code:java}
> Error: IOError: When resolving region for bucket 'analysis': AWS Error [code 99]: curlCode: 6, Couldn't resolve host name {code}
> which could have a whole host of underlying causes, from incorrect URL construction (e.g. incorrectly specified endpoint_override, default_region, etc) to other network issues https certificate failure or network connectivity problems. 
> In any event, a common source of this seems to be collisions with AWS credentials in environmental variables or elsewhere interfering with the intended credentials (which may be fine with 'default' values – e.g. like when accessing a MINIO bucket we do not need a "region" (actually it's not needed in AWS as it can be attained from the endpoint; or at least that is the argument given by MINIO).
> In a recent case, I'd already made sure I didn't have any AWS_DEFAULT_REGION or AWS_S3_ENDPOINT env var set, but still kept getting this error on a MINIO bucket on one machine , but not others. 
> Neal pointed out to me the existence of `AWS_EC2_METADATA_DISABLED`, and indeed, setting that to TRUE resolved the issue.  Others may also be unaware of this option, it might be good to highlight in the S3Filesystem vignette and/or docs.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)