[jira] [Created] (KAFKA-2025) In multi-consumer setup - explicit commit, commits on all partitions

["Pradeep G (JIRA)" <ji...@apache.org>]

Pradeep G created KAFKA-2025:
--------------------------------

             Summary: In multi-consumer setup - explicit commit, commits on all partitions
                 Key: KAFKA-2025
                 URL: https://issues.apache.org/jira/browse/KAFKA-2025
             Project: Kafka
          Issue Type: Bug
          Components: consumer
    Affects Versions: 0.8.2.0
         Environment: 1. Tested in Windows
2. Not tested on Linux

            Reporter: Pradeep G
            Assignee: Neha Narkhede
            Priority: Blocker


In a setup where there are two consumers C1 & C2 belonging to consumer group CG, two partitions P1 & P2; with auto-commit disabled.

An explicit commit on ConsumerConnect commits on all the consumers i.e. a commit called by C1 commits all messages being processed by other consumers too here C2. 

Ideally C1 should be able to commit only those messages it has consumed and not what is being processed by C2.  The effect of this behavior is that; suppose C2 crashes while processing message M after C1 commits, is that message M being processed by C2 is not available on recovery and is lost forever; and in kafka M is marked as consumed.

I read that this would be addressed in the rewrite - https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI

Any thoughts on which release this would be addressed ?.  A quick response would be greatly appreciated.

Thanks,
Pradeep

This 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)