[jira] [Created] (HADOOP-19992) AliyunOSS: Retry OSS multipart completion on QpsLimitExceeded

["JunboWang (Jira)" <ji...@apache.org>]

JunboWang created HADOOP-19992:
----------------------------------

             Summary: AliyunOSS: Retry OSS multipart completion on QpsLimitExceeded
                 Key: HADOOP-19992
                 URL: https://issues.apache.org/jira/browse/HADOOP-19992
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs/oss
    Affects Versions: 3.4.3
            Reporter: JunboWang


In hadoop-aliyun, CompleteMultipartUpload is not retried by the OSS Java SDK's default policy. A QpsLimitExceeded response can therefore fail an output stream close or multipart copy. Increasing fs.oss.attempts.maximum does not resolve this.

Proposed change:
- Share completion retry logic between multipart uploads and copies.
- Retry only QpsLimitExceeded, with exponential backoff and jitter capped at 10 seconds. Add fs.oss.multipart.complete.retry.limit (default: 3 retries after the initial attempt) and fs.oss.multipart.complete.retry.interval (default: 500 ms). Propagate the failure when retries are exhausted.
- Do not treat NoSuchUpload or an existing object alone as proof of success if a completion response is lost.
- Add regression tests in existing test classes for recovery, retry exhaustion, non-retryable errors, and uncertain completion. No new dependencies.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org