You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/09/19 07:30:12 UTC

[GitHub] [beam] mosche commented on issue #19142: Add retry logic to S3FileSystem

mosche commented on issue #19142:
URL: https://github.com/apache/beam/issues/19142#issuecomment-1250665880

   All AWS clients implement automatic retries, this behavior is enabled by default. Adding another level of retries won't do any good... 
   
   While possible, it's currently not straight forward to customize the retry policy for the S3 file system. It can be done by setting a custom `s3ClientFactoryClass` in `S3Options`. This factory would then delegate to `ClientBuilderFactory.defaultFactory` with a `ClientConfiguration` containing a customized `RetryConfiguration`.
   
   [https://github.com/apache/beam/blob/5520fe064fc3b7196998d4597746119691eb6681/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/common/ClientBuilderFactory.java#L151-L155](https://github.com/apache/beam/blob/5520fe064fc3b7196998d4597746119691eb6681/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/common/ClientBuilderFactory.java#L151-L155)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org