You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "YaYun Wang (Jira)" <ji...@apache.org> on 2023/06/08 03:31:00 UTC
[jira] [Comment Edited] (KAFKA-15074) offset out of range for partition xxx, resetting offset
[ https://issues.apache.org/jira/browse/KAFKA-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730365#comment-17730365 ]
YaYun Wang edited comment on KAFKA-15074 at 6/8/23 3:30 AM:
------------------------------------------------------------
*And here is the debug logs of kafka-client:*
2023-06-06 09:12:59.576+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574009 for partition myTopic-4
2023-06-06 09:12:59.577+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-1 at position FetchPosition{offset=2776251, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:12:59.577+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-7 at position FetchPosition{offset=1678666, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:12:59.577+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:12:59.963+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574109 for partition myTopic-4
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-8 at position FetchPosition{offset=2559637, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-0 at position FetchPosition{offset=1678355, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-3 at position FetchPosition{offset=2773458, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=2171}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-2 at position FetchPosition{offset=1677672, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-5 at position FetchPosition{offset=1678870, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=1999}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-6 at position FetchPosition{offset=1994802, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-8, myTopic-2, myTopic-5), canUseTopicIds=True) to broker host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.659+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574209 for partition myTopic-4
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-8 at position FetchPosition{offset=2559637, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-1 at position FetchPosition{offset=2776251, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-0 at position FetchPosition{offset=1678355, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-3 at position FetchPosition{offset=2773458, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=2171}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-2 at position FetchPosition{offset=1677672, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-5 at position FetchPosition{offset=1678870, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=1999}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-7 at position FetchPosition{offset=1678666, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-6 at position FetchPosition{offset=1994802, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-4 at position FetchPosition{offset=42574305, offsetEpoch=Optional[2214], currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2214}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-8, myTopic-2, myTopic-5), canUseTopicIds=True) to broker host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(myTopic-4), toForget=(), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.725+0000 [rtbeat-thread | group] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [onSuccess] [Consumer clientId=consumer-group-1, groupId=group] Fetch READ_UNCOMMITTED at offset 42574305 for partition myTopic-4 returned fetch data PartitionData(partitionIndex=4, errorCode=1, highWatermark=-1, lastStableOffset=-1, logStartOffset=-1, divergingEpoch=EpochEndOffset(epoch=-1, endOffset=-1), currentLeader=LeaderIdAndEpoch(leaderId=-1, leaderEpoch=-1), snapshotId=SnapshotId(endOffset=-1, epoch=-1), abortedTransactions=null, preferredReadReplica=-1, records=MemoryRecords(size=0, buffer=java.nio.HeapByteBuffer[pos=0 lim=0 cap=3]))
2023-06-06 09:13:00.926+0000 [rtbeat-thread | group] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [sendHeartbeatRequest] [Consumer clientId=consumer-group-1, groupId=group] Sending Heartbeat request with generation 109 and member id consumer-group-1-c25288aa-c1bd-4e53-a3e7-40ca8e9a7d83 to coordinator host:9093 (id: 2147483645 rack: null)
2023-06-06 09:13:01.026+0000 [rtbeat-thread | group] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Received successful Heartbeat response
2023-06-06 09:13:01.435+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574305 for partition myTopic-4
2023-06-06 09:13:01.435+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574305 for partition myTopic-4
2023-06-06 09:13:01.436+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] INFO o.a.k.c.c.internals.Fetcher - [handleOffsetOutOfRange] [Consumer clientId=consumer-group-1, groupId=group] Fetch position FetchPosition{offset=42574305, offsetEpoch=Optional[2214], currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2214}} is out of range for partition myTopic-4, resetting offset
2023-06-06 09:13:01.436+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-8, myTopic-2, myTopic-5), canUseTopicIds=True) to broker host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:01.436+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(FEQ3dqrdRFGUrbFFO8-Nxw:myTopic-4), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.437+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.438+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendListOffsetRequest] [Consumer clientId=consumer-group-1, groupId=group] Sending ListOffsetRequest ListOffsetsRequestData(replicaId=-1, isolationLevel=0, topics=[ListOffsetsTopic(name='myTopic', partitions=[ListOffsetsPartition(partitionIndex=4, currentLeaderEpoch=2214, timestamp=-1, maxNumOffsets=1)])]) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.937+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-0 at position FetchPosition{offset=1678355, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-3 at position FetchPosition{offset=2773458, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=2171}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-6 at position FetchPosition{offset=1994802, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-1 at position FetchPosition{offset=2776251, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-7 at position FetchPosition{offset=1678666, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.939+0000 [istenerEndpointContainer#0-0-C-1|#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [handleListOffsetResponse] [Consumer clientId=consumer-group-1, groupId=group] Handling ListOffsetResponse response for myTopic-4. Fetched offset 43611279, timestamp -1
was (Author: wangyayun):
And here is my debug logs of kafka-client:
2023-06-06 09:12:59.576+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574009 for partition myTopic-4
2023-06-06 09:12:59.577+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-1 at position FetchPosition\{offset=2776251, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:12:59.577+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-7 at position FetchPosition\{offset=1678666, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:12:59.577+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:12:59.963+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574109 for partition myTopic-4
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-8 at position FetchPosition\{offset=2559637, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-0 at position FetchPosition\{offset=1678355, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-3 at position FetchPosition\{offset=2773458, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=2171}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-2 at position FetchPosition\{offset=1677672, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-5 at position FetchPosition\{offset=1678870, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=1999}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-6 at position FetchPosition\{offset=1994802, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-8, myTopic-2, myTopic-5), canUseTopicIds=True) to broker host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:12:59.964+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.659+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574209 for partition myTopic-4
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-8 at position FetchPosition\{offset=2559637, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-1 at position FetchPosition\{offset=2776251, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-0 at position FetchPosition\{offset=1678355, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-3 at position FetchPosition\{offset=2773458, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=2171}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-2 at position FetchPosition\{offset=1677672, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=2001}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-5 at position FetchPosition\{offset=1678870, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 0 rack: cn-north-1a)], epoch=1999}} to node host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-7 at position FetchPosition\{offset=1678666, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-6 at position FetchPosition\{offset=1994802, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-4 at position FetchPosition\{offset=42574305, offsetEpoch=Optional[2214], currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2214}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-8, myTopic-2, myTopic-5), canUseTopicIds=True) to broker host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(myTopic-4), toForget=(), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:00.660+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:00.725+0000 [rtbeat-thread | group] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [onSuccess] [Consumer clientId=consumer-group-1, groupId=group] Fetch READ_UNCOMMITTED at offset 42574305 for partition myTopic-4 returned fetch data PartitionData(partitionIndex=4, errorCode=1, highWatermark=-1, lastStableOffset=-1, logStartOffset=-1, divergingEpoch=EpochEndOffset(epoch=-1, endOffset=-1), currentLeader=LeaderIdAndEpoch(leaderId=-1, leaderEpoch=-1), snapshotId=SnapshotId(endOffset=-1, epoch=-1), abortedTransactions=null, preferredReadReplica=-1, records=MemoryRecords(size=0, buffer=java.nio.HeapByteBuffer[pos=0 lim=0 cap=3]))
2023-06-06 09:13:00.926+0000 [rtbeat-thread | group] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [sendHeartbeatRequest] [Consumer clientId=consumer-group-1, groupId=group] Sending Heartbeat request with generation 109 and member id consumer-group-1-c25288aa-c1bd-4e53-a3e7-40ca8e9a7d83 to coordinator host:9093 (id: 2147483645 rack: null)
2023-06-06 09:13:01.026+0000 [rtbeat-thread | group] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Received successful Heartbeat response
2023-06-06 09:13:01.435+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574305 for partition myTopic-4
2023-06-06 09:13:01.435+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.i.ConsumerCoordinator - [handle] [Consumer clientId=consumer-group-1, groupId=group] Committed offset 42574305 for partition myTopic-4
2023-06-06 09:13:01.436+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] INFO o.a.k.c.c.internals.Fetcher - [handleOffsetOutOfRange] [Consumer clientId=consumer-group-1, groupId=group] Fetch position FetchPosition\{offset=42574305, offsetEpoch=Optional[2214], currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2214}} is out of range for partition myTopic-4, resetting offset
2023-06-06 09:13:01.436+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-8, myTopic-2, myTopic-5), canUseTopicIds=True) to broker host:9093 (id: 0 rack: cn-north-1a)
2023-06-06 09:13:01.436+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(FEQ3dqrdRFGUrbFFO8-Nxw:myTopic-4), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.437+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.438+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendListOffsetRequest] [Consumer clientId=consumer-group-1, groupId=group] Sending ListOffsetRequest ListOffsetsRequestData(replicaId=-1, isolationLevel=0, topics=[ListOffsetsTopic(name='myTopic', partitions=[ListOffsetsPartition(partitionIndex=4, currentLeaderEpoch=2214, timestamp=-1, maxNumOffsets=1)])]) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.937+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-0 at position FetchPosition\{offset=1678355, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-3 at position FetchPosition\{offset=2773458, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=2171}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-6 at position FetchPosition\{offset=1994802, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 1 rack: cn-north-1b)], epoch=1996}} to node host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-0, myTopic-3, myTopic-6), canUseTopicIds=True) to broker host:9093 (id: 1 rack: cn-north-1b)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-1 at position FetchPosition\{offset=2776251, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [prepareFetchRequests] [Consumer clientId=consumer-group-1, groupId=group] Added READ_UNCOMMITTED fetch request for partition myTopic-7 at position FetchPosition\{offset=1678666, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[host:9093 (id: 2 rack: cn-north-1d)], epoch=2222}} to node host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.938+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [sendFetches] [Consumer clientId=consumer-group-1, groupId=group] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), toReplace=(), implied=(myTopic-1, myTopic-7), canUseTopicIds=True) to broker host:9093 (id: 2 rack: cn-north-1d)
2023-06-06 09:13:01.939+0000 [istenerEndpointContainer#0-0-C-1] [traceId=] DEBUG o.a.k.c.c.internals.Fetcher - [handleListOffsetResponse] [Consumer clientId=consumer-group-1, groupId=group] Handling ListOffsetResponse response for myTopic-4. Fetched offset 43611279, timestamp -1
> offset out of range for partition xxx, resetting offset
> -------------------------------------------------------
>
> Key: KAFKA-15074
> URL: https://issues.apache.org/jira/browse/KAFKA-15074
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.3.2
> Reporter: YaYun Wang
> Priority: Major
>
> I got ?? "Fetch position FetchPosition{offset=42574305, offsetEpoch=Optional[2214], currentLeader=LeaderAndEpoch{leader=Optional[host:port (id: 2 rack: cn-north-1d)], epoch=2214}} is out of range for partition vcc.hdmap.tile.delivery-4, resetting offset " ??when i consumer kafka through @KafkaListener in that case producer publish 100W data to partition 4 in one minute, and one consumer consume the data from the partition.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)