[jira] [Commented] (KAFKA-14366) Kafka consumer rebalance issue, offsets points back to very old committed offset

["Chetan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641951#comment-17641951 ] 

Chetan commented on KAFKA-14366:
--------------------------------

Hi [~pnee],


 # We have created a custom Listener from ConsumerRebalanceListener and onPartitionsRevoked() we are doing commitSync offset

 # Each message consumed is logged with partition, offset, and topic information in the application log.



Do you see if there is any issue or if the way we do it is wrong, if so can you kindly guide me here?

Regards,
Chetan N

> Kafka consumer rebalance issue, offsets points back to very old committed offset
> --------------------------------------------------------------------------------
>
>                 Key: KAFKA-14366
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14366
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer, offset manager
>    Affects Versions: 2.8.1
>         Environment: Production
>            Reporter: Chetan
>            Priority: Major
>         Attachments: image-2022-11-30-17-58-15-046.png, image-2022-11-30-18-10-37-180.png, image-2022-11-30-18-29-36-445.png, rebalance issue.docx
>
>
> Hi All,
> We are facing an issue while the client consumer restart (again not all restarts are ending up with this issue) and during the re-balancing scenario, sometimes one of the partition offsets goes back a long way from the committed offset.
> Scenario :
> Assume we have 4 instances of consumer and restarts of consumer one after the other.
>  # At the time of starting restarts assume the offset on partition 10 of a topic being consumed is pointing to 50000. (last offset of the topic and 0 lag)
>  # When restarts start (rebalancing) suddenly the offsets start pointing to 20000.
>  # While all the restarts are going on the consumer who is attached starts reading from 20000 and goes on.
>  # Once all rebalance is completed, and all messages from 20000 to 50000 offset has been read (where it had stopped initially)
> We end up having around 30K duplicates.
> (The numbers here are just an example, in production, we are facing huge duplicates and every two rebalance during restarts of consumer out of 10 restart exercise activity ends up in such duplicates and not all partitions and only one or two partitions behave this way and randomly)
> This seems to be a bug. I am attaching all screenshots for reference as well.
> Can someone kindly help out here?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)