You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Guozhang Wang (Jira)" <ji...@apache.org> on 2020/06/19 19:08:00 UTC
[jira] [Commented] (KAFKA-9995) IllegalStateException when offsets
not found
[ https://issues.apache.org/jira/browse/KAFKA-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140763#comment-17140763 ]
Guozhang Wang commented on KAFKA-9995:
--------------------------------------
Hello James, sorry for the late reply.
Your observed issue reminds of a fixed bug I got a while ago, I tried to find out the exact JIRA (sorry...) Do you mind check out the latest 2.6 branch and see if this issue still reproduces?
> IllegalStateException when offsets not found
> --------------------------------------------
>
> Key: KAFKA-9995
> URL: https://issues.apache.org/jira/browse/KAFKA-9995
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.4.1
> Reporter: James Hay
> Priority: Major
>
>
> I have a recently upgraded Kafka Streams 2.4.1 application and we have started seeing the application periodically crash due to the following error:
> {code:java}
> 2020-05-14T16:53:03.839Z DEBUG <> [chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2] o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2-consumer, groupId=chat] Fetching committed offsets for partitions: [private.chat.endpoint-0, public.chat.message-0] 2020-05-14T16:53:03.841Z INFO <> [chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2] o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2-consumer, groupId=chat] Found no committed offset for partition private.chat.endpoint-0 2020-05-14T16:53:03.842Z ERROR <> [chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2] o.a.k.s.p.internals.StreamThread - stream-thread [chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2] Encountered the following error during processing: java.lang.IllegalStateException: Offset limit should monotonically increase, but was reduced. New limit: 0. Previous limit: 857859 at org.apache.kafka.streams.processor.internals.StandbyTask.updateOffsetLimits(StandbyTask.java:215) at org.apache.kafka.streams.processor.internals.StandbyTask.update(StandbyTask.java:181) at org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:1048) at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:825) at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:698) at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:671)
> {code}
> Is this a known issue? What can cause offsets to not be found?
> Other things to note:
> * We have TOPOLOGY_OPTIMIZATION set to OPTIMIZE
> * We are only seeing this on two of our three environments (at the moment). The two environments we are seeing a problem have little traffic and only seems to impact the machines that are on the whole idle. Our prod environment which consumes regular events is showing no signs of having the same problem.
> * There is some evidence to suggest there is a pattern to the timing of this error. Although not always the case, 24hrs between errors is common.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)