[jira] [Created] (HADOOP-19734) S3A: retry on MPU completion failure "One or more of the specified parts could not be found"

["Steve Loughran (Jira)" <ji...@apache.org>]

Steve Loughran created HADOOP-19734:
---------------------------------------

             Summary: S3A: retry on MPU completion failure "One or more of the specified parts could not be found"
                 Key: HADOOP-19734
                 URL: https://issues.apache.org/jira/browse/HADOOP-19734
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.4.2
         Environment: aws s3 london
            Reporter: Steve Loughran



Experienced transient failure in test run of https://github.com/apache/hadoop/pull/7882 : all MPU complete posts failed because the request or parts were not found...the tests started succeeding 60-90s later *and* a "hadoop s3guards uploads" call listed the outstanding uploads of the failing tests.

Hypothesis: a transient failure meant the server receiving the POST calls to complete the uploads was mistakenly reporting no upload IDs.

Outcome: all active write operations failed, without any retry attempts. This can lose data and fail jobs, even though the store may recover.

Proposed. The multipart uploads, especially block output stream, retry on this error; treat it as a connectivity issue. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org