You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Guozhang Wang (Jira)" <ji...@apache.org> on 2020/02/07 01:05:00 UTC
[jira] [Resolved] (KAFKA-8737) TaskMigrated Exception while
rebalancing kafka streams
[ https://issues.apache.org/jira/browse/KAFKA-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guozhang Wang resolved KAFKA-8737.
----------------------------------
Assignee: Guozhang Wang (was: Bill Bejeck)
Resolution: Duplicate
> TaskMigrated Exception while rebalancing kafka streams
> ------------------------------------------------------
>
> Key: KAFKA-8737
> URL: https://issues.apache.org/jira/browse/KAFKA-8737
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 1.0.0, 1.0.1
> Environment: 20 partitions
> 1 topic
> 8 Streamer service
> topic-region-1 9 7841726 8236017 394291 streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer-0276e83d-40b5-4b44-b764-7d29e0dab663/ streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer
> topic-region-1 15 7421710 7467666 45956 streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer-0276e83d-40b5-4b44-b764-7d29e0dab663/ streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer
> topic-region-1 19 7737360 8120611 383251 streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer-0276e83d-40b5-4b44-b764-7d29e0dab663/ streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer
> topic-region-1
> Reporter: KUMAR
> Assignee: Guozhang Wang
> Priority: Major
>
> Kafka streams throws following exception while restart of a stream client service -
> o.a.k.s.p.internals.StreamThread.? - stream-thread [streams-subscriberstopic-region-1-32d968e3-f892-4772-a7a4-6f684d7e43c9-StreamThread-1] Detected a task that got migrated to another thread. This implies that this thread missed a rebalance and dropped out of the consumer group. Trying to rejoin the consumer group now.
> org.apache.kafka.streams.errors.TaskMigratedException: Log end offset of topic-region-1-12 should not change while restoring: old end offset 6286727, current offset 6380997
>
> Kafka version is 1.0.0 and we have back merged the fix for KIP-6269-
> [https://github.com/apache/kafka/pull/4300/files#|https://github.com/apache/kafka/pull/4300/files]
> However we observe that there seems to be an issue in rebalance when "auto.offset.reset" is configured as "latest". Based on log analysis we see following behavior -
> # StreamThread starts a restore consumer
> # While Fetching it gets offset out of range o.a.k.c.consumer.internals.Fetcher.? - [Consumer clientId=streams-subscriberstopic-region-1-11b2d7fb-11ce-4b0b-a40a-388d3c7b6bc9-StreamThread-1-restore-
> consumer, groupId=] Fetch READ_UNCOMMITTED at offset 246431 for partition topic-region-1-12 returned fetch data (error=OFFSET_OUT_OF_RANGE, highWaterMark=-1, lastStableOffset = -1, logStartOffset = -1,
> abortedTransactions = null, recordsSizeInBytes=0)
> # Fetcher tries to reset the offset
> # While reset the offset it appears it is changing the offset position and causing TaskMigrated exception
> Above test repeated with "auto.offset.reset" is configured as "earliest" does not throw any TaskMigrated exception as in earliest case we are not reseting the restore consumer position.
>
> Please let us know if this is possible and if a fix would be needed for the offset reset piece when set to latest.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)