[jira] [Commented] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776607#comment-17776607 ] 

Steve Loughran commented on HADOOP-18945:
-----------------------------------------


looking for that error string "Failed to load credentials from IMDS." it looks like it's a problem with the AWS sdk timing out while trying to connect to the service for credentials

https://github.com/aws/aws-sdk-java-v2/issues/3448
https://github.com/aws/aws-sdk-java-v2/issues/3846

looks like the elimination of the async refresh() call from the sdk to the credential provider means that credential retrieval only happens when the request is made, and its done with a 1s timeout. we need to turn on async credential refresh which looks built into the code. 

> S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS
> ---------------------------------------------------------------------------------
>
>                 Key: HADOOP-18945
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18945
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 7.2.18.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Blocker
>
> Failures in impala test VMs using iAM for auth
> {code}
> Failed to open file as a parquet file: java.net.SocketTimeoutException: re-open s3a://impala-test-uswest2-1/test-warehouse/test_pre_gregorian_date_parquet_2e80ae30.db/hive2_pre_gregorian.parquet at 84 on s3a://impala-test-uswest2-1/test-warehouse/test_pre_gregorian_date_parquet_2e80ae30.db/hive2_pre_gregorian.parquet: org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: +: Failed to load credentials from IMDS
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org