You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Johny Rufus (JIRA)" <ji...@apache.org> on 2015/05/04 23:50:17 UTC
[jira] [Updated] (FLUME-2352) HDFSCompressedDataStream should
support appendBatch
[ https://issues.apache.org/jira/browse/FLUME-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johny Rufus updated FLUME-2352:
-------------------------------
Fix Version/s: (was: v1.6.0)
> HDFSCompressedDataStream should support appendBatch
> ---------------------------------------------------
>
> Key: FLUME-2352
> URL: https://issues.apache.org/jira/browse/FLUME-2352
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.5.0
> Reporter: chenshangan
> Assignee: chenshangan
> Attachments: FLUME-2352.patch
>
>
> compressing events in batch is much more efficient than compressing one by one.
> I set hdfs.batchSize to 200000, when I use appendBatch() in BucketWriter, the append operation cost less than 1 seconds, while one by one might cost 10 seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)