You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Moritz Mack (Jira)" <ji...@apache.org> on 2021/11/09 09:22:00 UTC

[jira] [Created] (BEAM-13206) Consolidate multiple levels of retries for AWS IOs

Moritz Mack created BEAM-13206:
----------------------------------

             Summary: Consolidate multiple levels of retries for AWS IOs
                 Key: BEAM-13206
                 URL: https://issues.apache.org/jira/browse/BEAM-13206
             Project: Beam
          Issue Type: Improvement
          Components: io-java-aws
            Reporter: Moritz Mack


The current implementation of retries at the Beam IO level for AWS IOs is misleading and for that reason potentially harmful.

The AWS SDK automatically performs retries by default. In combination with retries implemented by various AWS IOs the number of requests under error might unexpectedly explode.

Instead of re-implementing retries on the IO level, respective retry configuration could be pushed down into the AWS client, e.g. changing the client providers to accept the RetryConfiguration as parameter.

With that we would get a consistent and transparent behavior. And at the same time we'd be able to remove lots of boilerplate code from the IOs.

 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)