[jira] [Commented] (HADOOP-13568) S3AFastOutputStream to implement flush()

["Thomas Demoor (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473830#comment-15473830 ] 

Thomas Demoor commented on HADOOP-13568:
----------------------------------------

This might be hard as only the last part of a MPU can be smaller then 5MB. So I fear that the subsequent complete operation would fail if flush() is called when the buffer contains less than 5MB.

http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html ??Each part must be at least 5 MB in size, except the last part. There is no size limit on the last part of your multipart upload.??


> S3AFastOutputStream to implement flush()
> ----------------------------------------
>
>                 Key: HADOOP-13568
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13568
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Steve Loughran
>            Priority: Minor
>
> {{S3AFastOutputStream}} doesn't implement {{flush()}}, so it's a no-op.
> Really it should trigger a multipart upload of the current buffer.
> Note that simply calling {{uploadBuffer()} isn't enough...do that and things fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org