[jira] [Created] (KAFKA-16155) Investigate testAutoCommitIntercept

["Lucas Brutschy (Jira)" <ji...@apache.org>]

Lucas Brutschy created KAFKA-16155:
--------------------------------------

             Summary: Investigate testAutoCommitIntercept
                 Key: KAFKA-16155
                 URL: https://issues.apache.org/jira/browse/KAFKA-16155
             Project: Kafka
          Issue Type: Bug
          Components: clients, consumer
            Reporter: Lucas Brutschy


Even with KAFKA-15942, the test PlaintextConsumerTest.testAutoCommitIntercept flakes on the the initial setup (before using interceptors, so interceptors are unrelated here, except for being used later in the test).

The problem is that we are seeking two topic partitions to offset 10 and 20, respectively, but when we commit, we seem to have lost one of the offsets, likely due to a race condition. 

When I output `subscriptionState.allConsumed` repeatedly, I get this output:

 
{code:java}
allConsumed: {topic-0=OffsetAndMetadata{offset=100, leaderEpoch=0, metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}}
seeking topic-0 to FetchPosition{offset=10, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:58298 (id: 0 rack: null)], epoch=0}}
seeking topic-1 to FetchPosition{offset=20, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:58301 (id: 1 rack: null)], epoch=0}}
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=20, leaderEpoch=null, metadata=''}} 
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}} allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}} autocommit start {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}}
{code}
So we after we seek to 10 / 20, we lose one of the offsets, maybe because we haven't reconciled the assignment yet. Later, we get the second topic partition assigned, but the offset is initialized to 0.

We should investigate whether this can be made more like the behavior in the original consumer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)