You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Jason Gustafson (Jira)" <ji...@apache.org> on 2020/04/17 18:20:00 UTC
[jira] [Comment Edited] (KAFKA-9543) Consumer offset reset after
new segment rolling
[ https://issues.apache.org/jira/browse/KAFKA-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085938#comment-17085938 ]
Jason Gustafson edited comment on KAFKA-9543 at 4/17/20, 6:19 PM:
------------------------------------------------------------------
[~junrao] Good suggestion. I opened https://issues.apache.org/jira/browse/KAFKA-9886 to fix this separately. Note that I'll leave the issue here open until we can confirm whether it is fixed by KAFKA-9838.
was (Author: hachikuji):
[~junrao] Good suggestion. I opened https://issues.apache.org/jira/browse/KAFKA-9886. I'll leave this issue open until we can confirm whether it is fixed by KAFKA-9838.
> Consumer offset reset after new segment rolling
> -----------------------------------------------
>
> Key: KAFKA-9543
> URL: https://issues.apache.org/jira/browse/KAFKA-9543
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Rafał Boniecki
> Priority: Major
> Attachments: Untitled.png, image-2020-04-06-17-10-32-636.png
>
>
> After upgrade from kafka 2.1.1 to 2.4.0, I'm experiencing unexpected consumer offset resets.
> Consumer:
> {code:java}
> 2020-02-12T11:12:58.402+01:00 hostname 4a2a39a35a02 [2020-02-12T11:12:58,402][INFO ][org.apache.kafka.clients.consumer.internals.Fetcher] [Consumer clientId=logstash-1, groupId=logstash] Fetch offset 1632750575 is out of range for partition stats-5, resetting offset
> {code}
> Broker:
> {code:java}
> 2020-02-12 11:12:58:400 CET INFO [data-plane-kafka-request-handler-1][kafka.log.Log] [Log partition=stats-5, dir=/kafka4/data] Rolled new log segment at offset 1632750565 in 2 ms.{code}
> All resets are perfectly correlated to rolling new segments at the broker - segment is rolled first, then, couple of ms later, reset on the consumer occurs. Attached is grafana graph with consumer lag per partition. All sudden spikes in lag are offset resets due to this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)