You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Ted Malaska (JIRA)" <ji...@apache.org> on 2013/09/17 22:03:53 UTC
[jira] [Commented] (FLUME-2128) HDFS Sink rollSize is calculated
based off of uncompressed size of cumulative events.
[ https://issues.apache.org/jira/browse/FLUME-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769860#comment-13769860 ]
Ted Malaska commented on FLUME-2128:
------------------------------------
Any update on this.
Thanks
> HDFS Sink rollSize is calculated based off of uncompressed size of cumulative events.
> -------------------------------------------------------------------------------------
>
> Key: FLUME-2128
> URL: https://issues.apache.org/jira/browse/FLUME-2128
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.4.0, v1.3.1
> Reporter: Jeff Lord
> Assignee: Ted Malaska
> Labels: features
> Attachments: FLUME-2128-0.patch
>
>
> The hdfs sink rollSize parameter is compared against uncompressed event sizes.
> The net of this is that if you are using compression and expect the size of your files on HDFS to be rolled/sized based on the value set for rollSize than your files will be much smaller due to compression.
> We should take into account when compression is set and roll based on the compressed size on hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira