[jira] [Commented] (BEAM-13206) Consolidate multiple levels of retries for AWS IOs

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508941#comment-17508941 ] 

Beam JIRA Bot commented on BEAM-13206:
--------------------------------------

This issue is assigned but has not received an update in 30 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.

> Consolidate multiple levels of retries for AWS IOs
> --------------------------------------------------
>
>                 Key: BEAM-13206
>                 URL: https://issues.apache.org/jira/browse/BEAM-13206
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-aws
>            Reporter: Moritz Mack
>            Assignee: Moritz Mack
>            Priority: P3
>              Labels: stale-assigned
>
> The current implementation of retries at the Beam IO level for AWS IOs is misleading and for that reason potentially harmful.
> The AWS SDK automatically performs retries by default. In combination with retries implemented by various AWS IOs the number of requests under error might unexpectedly explode.
> Instead of re-implementing retries on the IO level, respective retry configuration could be pushed down into the AWS client, e.g. changing the client providers to accept the RetryConfiguration as parameter.
> With that we would get a consistent and transparent behavior. And at the same time we'd be able to remove lots of boilerplate code from the IOs.
> Also, the default retry predicate is simply wrong and does not correctly check for any status code that can be retried. Yet another reason to remove any custom retry logic and push it down into the SDK. E.g. for DynamoDBIO, but same for SnsIO
> {code:java}
> private static class DefaultRetryPredicate implements RetryPredicate {
>   private static final ImmutableSet<Integer> ELIGIBLE_CODES =
>       ImmutableSet.of(HttpStatus.SC_SERVICE_UNAVAILABLE);
>   @Override
>   public boolean test(Throwable throwable) {
>     return (throwable instanceof IOException
>         || (throwable instanceof InternalErrorException)
>         || (throwable instanceof InternalErrorException
>             && ELIGIBLE_CODES.contains(((InternalErrorException) throwable).getStatusCode())));
>   }
> } {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)