[jira] [Comment Edited] (FLUME-2495) Kafka Source may miss events when channel is not available

["Hari Shreedharan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162220#comment-14162220 ] 

Hari Shreedharan edited comment on FLUME-2495 at 10/7/14 6:15 PM:
------------------------------------------------------------------

There is a bug in this patch - you'd never clear the list if autocommit is enabled - this means that the list would grow indefinitely causing duplicates.


was (Author: hshreedharan):
There is a bug in this patch - you'd never clear the list if autocommit is not enabled - this means that the list would grow indefinitely causing duplicates.

> Kafka Source may miss events when channel is not available
> ----------------------------------------------------------
>
>                 Key: FLUME-2495
>                 URL: https://issues.apache.org/jira/browse/FLUME-2495
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>            Reporter: Gwen Shapira
>            Assignee: Gwen Shapira
>         Attachments: FLUME-2495.0.patch
>
>
> Because the Kafka consumer itself tracks offsets, and we don't restart the consumer when we get channel errors, the consumer will skip messages that we couldn't write to channel, even though we did not advance offset in Zookeeper.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)