You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Anuj Modi (Jira)" <ji...@apache.org> on 2023/07/24 13:10:00 UTC
[jira] [Updated] (HADOOP-18759) [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout failures
[ https://issues.apache.org/jira/browse/HADOOP-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anuj Modi updated HADOOP-18759:
-------------------------------
Summary: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout failures (was: [ABFS][Backoff-Optimization] Have a Linear retry policy for connection timeout failures)
> [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout failures
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-18759
> URL: https://issues.apache.org/jira/browse/HADOOP-18759
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.4
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
>
> Today when a request fails with connection timeout, it falls back into the loop for exponential retry. Unlike Azure Storage, there are no guarantees of success on exponentially retried request or recommendations for ideal retry policies for Azure network or any other general failures. Faster failure and retry might be more beneficial for such generic connection timeout failures.
> This PR introduces a new Static Retry Policy which will currently be used only for Connection Timeout failures. It means all the requests failing with Connection Timeout errors will be retried after a constant retry(sleep) interval independent of how many times that request has failed. Max Retry Count check will still be in place.
> Following Configurations will be introduced in the change:
> # "fs.azure.static.retry.for.connection.timeout.enabled" - default: true, true: static retry will be used for CT, false: Exponential retry will be used.
> # "fs.azure.static.retry.interval" - default: 1000ms.
> This also introduces a new field in x-ms-client-request-id only for the requests that are being retried after connection timeout failure. New filed will tell what retry policy was used to get the sleep interval before making this request.
> Header "x-ms-client-request-id " right now has only the retryCount and retryReason this particular API call is. For ex: :eb06d8f6-5693-461b-b63c-5858fa7655e6:29cb0d19-2b68-4409-bc35-cb7160b90dd8:::CF:1_CT.
> Moving ahead for retryReason "CT" it will have retry policy abbreviation as well.
> For ex: :eb06d8f6-5693-461b-b63c-5858fa7655e6:29cb0d19-2b68-4409-bc35-cb7160b90dd8:::CF:1_CT_E.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org