[jira] [Commented] (FLUME-2128) HDFS Sink rollSize is calculated based off of uncompressed size of cumulative events.

["Jeff Lord (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13803483#comment-13803483 ] 

Jeff Lord commented on FLUME-2128:
----------------------------------

How are things going with the rebase Ted?

> HDFS Sink rollSize is calculated based off of uncompressed size of cumulative events.
> -------------------------------------------------------------------------------------
>
>                 Key: FLUME-2128
>                 URL: https://issues.apache.org/jira/browse/FLUME-2128
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.4.0, v1.3.1
>            Reporter: Jeff Lord
>            Assignee: Ted Malaska
>              Labels: features
>         Attachments: FLUME-2128-0.patch, FLUME-2128-1.patch
>
>
> The hdfs sink rollSize parameter is compared against uncompressed event sizes.
> The net of this is that if you are using compression and expect the size of your files on HDFS to be rolled/sized based on the value set for rollSize than your files will be much smaller due to compression.
> We should take into account when compression is set and roll based on the compressed size on hdfs.



--
This message was sent by Atlassian JIRA
(v6.1#6144)