[jira] [Commented] (KAFKA-14739) Kafka consumer reading messages out of order after a rebalance

["Luke Chen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-14739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17692452#comment-17692452 ] 

Luke Chen commented on KAFKA-14739:
-----------------------------------

[~colinshaw] , Have you tried to run with the latest version of kafka client? We did some improvement to it before. Please check if this issue still existed. Thanks.

 

> Kafka consumer reading messages out of order after a rebalance
> --------------------------------------------------------------
>
>                 Key: KAFKA-14739
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14739
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Colin Shaw
>            Priority: Major
>         Attachments: image-2023-02-21-16-45-10-464.png
>
>
> We are seeing often when our consumers are running and a rebalance is triggered, some partitions do not resume at the correct offset causing a large number of events to be skipped.
> Our understanding is that within a partition events should also be read in chronological order.
>  
> These consumers run once a day and we see that when they resume again the next day it does return to the missed events.  
>  
> An example we investigated: 
> {quote}We see for partition 88 starting processing around Offset 14001. Events were processed until Offset 14059 but we could see Offset 14060 was not processed from out logs. 
> Upon investigation we could see a rebalance had occured after offset 14059 and afterwards the next event processed was offset 14190 causing about 130 events to be skipped on this partition.
> The subsequent day we it did resume at Offset 14060
> {quote}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)