You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Lakshmi Manasa Gaduputi (Jira)" <ji...@apache.org> on 2021/10/02 00:58:00 UTC
[jira] [Created] (SAMZA-2699) AzureBlob SystemProducer: remove
retries when uploading block to azure-blob-storage to prevent exceptions
being swallowed till samza retries are over
Lakshmi Manasa Gaduputi created SAMZA-2699:
----------------------------------------------
Summary: AzureBlob SystemProducer: remove retries when uploading block to azure-blob-storage to prevent exceptions being swallowed till samza retries are over
Key: SAMZA-2699
URL: https://issues.apache.org/jira/browse/SAMZA-2699
Project: Samza
Issue Type: Bug
Reporter: Lakshmi Manasa Gaduputi
Assignee: Lakshmi Manasa Gaduputi
Problem: Current samza code to retry catches exceptions (other than InterruptedException) and keeps uploading till retry attempts expire. Problem is when InterruptedException is wrapped by another exception, then InterruptedException is ignored. see https://issues.apache.org/jira/browse/SAMZA-2665 which was fixed by bubbling up InterruptedException immediately but InterruptedException wrapped in another exception.
Solution: remove retries when uploading block to azure-blob-storage as azure sdk internally does retry for 4 times as a default.
Samza uses [BlobServiceClientBuilder|https://github.com/apache/samza/blob/master/samza-azure/src/main/java/org/apache/samza/system/azureblob/AzureBlobClientBuilder.java#L89] which uses [RequestRetryOptions|https://github.com/Azure/azure-sdk-for-java/blob/5d602a2d97d36c1b20fc56863313222897c032ed/sdk/storage/azure-storage-blob/src/main/java/com/azure/storage/blob/BlobServiceClientBuilder.java#L72] as the default retry policy and the [RequestRetryOptions sets max retries to 4 with exponential backoff|https://github.com/Azure/azure-sdk-for-java/blob/3f31d68eed6fbe11516ca3afe3955c8840a6e974/sdk/storage/azure-storage-common/src/main/java/com/azure/storage/common/policy/RequestRetryOptions.java#L97].
hence it is safe to remove retries with samza.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)