You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/06/09 16:35:21 UTC
[jira] [Commented] (FLUME-2920) Kafka Channel Should Not Commit
Offsets When Stopping
[ https://issues.apache.org/jira/browse/FLUME-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322815#comment-15322815 ]
ASF GitHub Bot commented on FLUME-2920:
---------------------------------------
GitHub user kevinconaway opened a pull request:
https://github.com/apache/flume/pull/51
FLUME-2920 Don't commit offsets when shutting down consumer
Also removed some unused test code in `TestKafkaChanel`
@jarcec is this something that you can review?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kevinconaway/flume flume-2920
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/51.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #51
----
commit fc31f14abf7b46680ec94014dd877e2e1f01958a
Author: Kevin Conaway <ke...@walmart.com>
Date: 2016-06-09T16:32:35Z
FLUME-2920 Don't commit offsets when shutting down consumer. Remove some dead test code
----
> Kafka Channel Should Not Commit Offsets When Stopping
> -----------------------------------------------------
>
> Key: FLUME-2920
> URL: https://issues.apache.org/jira/browse/FLUME-2920
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: v1.6.0
> Reporter: Kevin Conaway
> Priority: Critical
>
> The Kafka channel commits the consumer offsets when shutting down (via stop() -> decommissionConsumerAndRecords())
> This can lead to data loss if the channel is shut down while messages have been fetched in a transaction but the transaction has not yet been committed.
> The only time that the offsets should be committed is when a transaction is complete.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)