You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/05/15 11:39:05 UTC
[jira] [Updated] (HADOOP-14423) s3guard will set file length to -1
on a putObjectDirect(stream, -1) call
[ https://issues.apache.org/jira/browse/HADOOP-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-14423:
------------------------------------
Priority: Minor (was: Major)
rating as minor as the output streams don't normally pass in -1 as a length
> s3guard will set file length to -1 on a putObjectDirect(stream, -1) call
> ------------------------------------------------------------------------
>
> Key: HADOOP-14423
> URL: https://issues.apache.org/jira/browse/HADOOP-14423
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-alpha3
> Reporter: Steve Loughran
> Priority: Minor
>
> You can pass a negative number into {{S3AFileSystem.putObjectDirect}}, which means "put until the end of the stream". S3guard has been using this {{len}} argument: it needs to be using the actual number of bytes uploaded. Also relevant with client side encryption, when the amount of data put > the amount of data in the file or stream.
> Noted in the committer branch after I added some more assertions, I've changed it there so making changes to S3AFS.putObjectDirect to pull the content length to pass in to finishedWrite() from the {{PutObjectResult}} instead. This can be picked into the s3guard branch
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org