You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Lucas Bradstreet (Jira)" <ji...@apache.org> on 2021/01/11 19:37:00 UTC

[jira] [Created] (KAFKA-12178) Improve guard rails for consumer commit when using EOS

Lucas Bradstreet created KAFKA-12178:
----------------------------------------

             Summary: Improve guard rails for consumer commit when using EOS
                 Key: KAFKA-12178
                 URL: https://issues.apache.org/jira/browse/KAFKA-12178
             Project: Kafka
          Issue Type: Improvement
            Reporter: Lucas Bradstreet


When EOS is in use, offsets are committed via the producer using the sendOffsetsToTransaction API. This is what ensures that a transaction is committed atomically along with the consumer offsets. Unfortunately this does not prevent the consumer from committing, making it easy to achieve non-EOS characteristics by accident. enable.auto.commit = true is the default setting for consumers. If this not set to false, or if commitSync/commitAsync are called manually offsets will no longer be committed correctly for EOS semantics.

We need more guard rails to prevent consumers from being incorrectly used in this way. Currently the consumers have no knowledge that a producer is even committing offsets and this is difficult to achieve.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)