You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flume.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/08/14 04:12:00 UTC
[jira] [Commented] (FLUME-3268) Introducing micro batch processing
to HDFSEventSink
[ https://issues.apache.org/jira/browse/FLUME-3268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579236#comment-16579236 ]
ASF GitHub Bot commented on FLUME-3268:
---------------------------------------
GitHub user JohnZZGithub opened a pull request:
https://github.com/apache/flume/pull/223
FLUME-3268:Introducing micro batch processing to HDFSEventSink
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JohnZZGithub/flume FLUME-3268
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/223.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #223
----
commit ab9e259b648e3368e24cb11b139a4f50fcfca772
Author: JohnZZGithub <jo...@...>
Date: 2018-08-14T03:54:43Z
FLUME-3268:Introducing micro batch processing to HDFSEventSink
----
> Introducing micro batch processing to HDFSEventSink
> ---------------------------------------------------
>
> Key: FLUME-3268
> URL: https://issues.apache.org/jira/browse/FLUME-3268
> Project: Flume
> Issue Type: New Feature
> Reporter: zhenzhao wang
> Priority: Major
> Attachments: FLUME-3268-0.patch
>
>
> In our test with HDFSEvent sink, we found that we could increase the draining speed of HDFSSink up to 4x by introducing micro batch processing. With the micro batch processing feature, we will batch the events written to HDFS instead of one by one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org