You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2026/02/05 16:40:00 UTC

[jira] [Created] (HADOOP-19804) Switch to adaptive AWS SDK retry policy

Steve Loughran created HADOOP-19804:
---------------------------------------

             Summary: Switch to adaptive AWS SDK retry policy
                 Key: HADOOP-19804
                 URL: https://issues.apache.org/jira/browse/HADOOP-19804
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.5.0, 3.4.3
            Reporter: Steve Loughran


The AWS SDK includes an adaptive retry policy which throttles back on all HTTP requests when 503 responses come in, to stop issuing so many rejected requests when a store is being overloaded.
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/retry-strategy.html

The s3a connector meets the single-service-endpoint requirement of this poiicy.
"The adaptive retry strategy assumes that the client works against a single resource (for example, one DynamoDB table or one Amazon S3 bucket)."




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org