[jira] [Reopened] (HADOOP-16193) add extra S3A MPU test to see what happens if a file is created during the MPU

["Ewan Higgs (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ewan Higgs reopened HADOOP-16193:
---------------------------------

I must have made a mistake in testing this. Re-reviewing it, it's not possible for this to work and indeed it fails when I test it again.

When writing to S3, the 'winner' will be the last-started-upload-that-completed-successfully. In the case of MPU this start is at init time, not complete time. So when the transient file comes in and is deleted then the MPU becomes moot.

In the case of a versioned bucket, then the transient file doesn't unroll the latest version to the MPU because all the 'latest' knowledge is based on init time, not complete time.

I will revert the commit.

> add extra S3A MPU test to see what happens if a file is created during the MPU
> ------------------------------------------------------------------------------
>
>                 Key: HADOOP-16193
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16193
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>             Fix For: 3.1.3
>
>
> Proposed extra test for the S3A MPU: if you create and then delete a file while an MPU is in progress, when you finally complete the MPU the new data is present.
> This verifies that the other FS operations don't somehow cancel the in-progress upload, and that eventual consistency brings the latest value out.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org