You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Pradeep G (JIRA)" <ji...@apache.org> on 2015/03/17 12:34:38 UTC
[jira] [Commented] (KAFKA-2025) In multi-consumer setup - explicit
commit, commits on all partitions
[ https://issues.apache.org/jira/browse/KAFKA-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365011#comment-14365011 ]
Pradeep G commented on KAFKA-2025:
----------------------------------
Workaround available at --> http://grokbase.com/t/kafka/users/138x3vbfcq/is-it-possible-to-commit-offsets-on-a-per-stream-basis
(changed priority from Blocker to Critical).
-Pradeep
> In multi-consumer setup - explicit commit, commits on all partitions
> --------------------------------------------------------------------
>
> Key: KAFKA-2025
> URL: https://issues.apache.org/jira/browse/KAFKA-2025
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.8.2.0
> Environment: 1. Tested in Windows
> 2. Not tested on Linux
> Reporter: Pradeep G
> Assignee: Neha Narkhede
> Priority: Critical
>
> In a setup where there are two consumers C1 & C2 belonging to consumer group CG, two partitions P1 & P2; with auto-commit disabled.
> An explicit commit on ConsumerConnect commits on all the consumers i.e. a commit called by C1 commits all messages being processed by other consumers too here C2.
> Ideally C1 should be able to commit only those messages it has consumed and not what is being processed by C2. The effect of this behavior is that; suppose C2 crashes while processing message M after C1 commits, is that message M being processed by C2 is not available on recovery and is lost forever; and in kafka M is marked as consumed.
> I read that this would be addressed in the rewrite - https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI
> Any thoughts on which release this would be addressed ?. A quick response would be greatly appreciated.
> Thanks,
> Pradeep
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)