You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/04/29 17:40:48 UTC

[GitHub] [hadoop] steveloughran commented on pull request #1944: HADOOP-16900. Very large files can be truncated when written through S3AFileSystem

steveloughran commented on pull request #1944:
URL: https://github.com/apache/hadoop/pull/1944#issuecomment-621360376


   > I am wondering isn't a bad design in terms for fail fast algorithm letting the clients wait unnecessarily knowing that it will fail eventually thus wasting time and compute both. Ideally S3 should have thrown error during the initial uploads only.
   
   S3 doesn't know the order you actually want to complete the upload -any file could have been the last one. The order is defined in the final post just by the ordering of the etags. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org