You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Patrik Kleindl (Jira)" <ji...@apache.org> on 2021/08/18 11:46:00 UTC
[jira] [Created] (KAFKA-13213) StreamThread: State transition from
PARTITIONS_ASSIGNED to PARTITIONS_ASSIGNED
Patrik Kleindl created KAFKA-13213:
--------------------------------------
Summary: StreamThread: State transition from PARTITIONS_ASSIGNED to PARTITIONS_ASSIGNED
Key: KAFKA-13213
URL: https://issues.apache.org/jira/browse/KAFKA-13213
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 2.6.0
Reporter: Patrik Kleindl
One of our streaming apps sometimes enters a strange looping behaviour.
It is deployed on 2 pods in a kubernetes cluster, but only one of them shows this:
{code:java}
2021-08-18 11:27:20,402 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.s.p.i.StreamThread - stream-thread [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] State transition from PARTITIONS_ASSIGNED to PARTITIONS_ASSIGNED2021-08-18 11:27:20,402 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.s.p.i.StreamThread - stream-thread [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] State transition from PARTITIONS_ASSIGNED to PARTITIONS_ASSIGNED2021-08-18 11:37:23,516 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1-consumer, groupId=app] Attempt to heartbeat failed since group is rebalancing2021-08-18 11:37:23,518 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1-consumer, groupId=app] (Re-)joining group2021-08-18 11:37:23,676 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1-consumer, groupId=app] Successfully joined group with generation 135...2021-08-18 11:37:23,678 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.s.p.i.TaskManager - stream-thread [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] Handle new assignment with: New active tasks: [0_0, 0_1, 0_2, 0_3, 0_4, 3_1, 0_5, 0_6, 3_3, 0_7, 0_8, 3_5, 0_9, 0_10, 3_7, 0_11, 3_9, 3_11] New standby tasks: [1_3, 1_1] Existing active tasks: [0_0, 0_1, 0_2, 0_3, 0_4, 3_1, 0_5, 0_6, 3_3, 0_7, 0_8, 3_5, 0_9, 0_10, 3_7, 0_11, 3_9, 3_11] Existing standby tasks: [1_1, 1_3]2021-08-18 11:37:23,678 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1-consumer, groupId=app] Adding newly assigned partitions: 2021-08-18 11:37:23,679 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.s.p.i.StreamThread - stream-thread [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] State transition from PARTITIONS_ASSIGNED to PARTITIONS_ASSIGNED2021-08-18 11:47:26,768 [app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=app-53da371b-98fb-4c73-ad08-5b93b9c0d1d4-StreamThread-1-consumer, groupId=app] Attempt to heartbeat failed since group is rebalancing{code}
The other one shows:
{code:java}
2021-08-18 11:37:23,710 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.p.i.StreamsPartitionAssignor - stream-thread [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1-consumer] Requested to schedule probing rebalance for 1629280043706 ms.2021-08-18 11:37:23,710 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.p.i.StreamsPartitionAssignor - stream-thread [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1-consumer] Requested to schedule probing rebalance for 1629280043706 ms.2021-08-18 11:37:23,710 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.p.i.TaskManager - stream-thread [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] Handle new assignment with: New active tasks: [1_0, 2_0, 1_1, 2_1, 3_0, 1_2, 2_2, 1_3, 2_3, 3_2, 1_4, 2_4, 1_5, 2_5, 3_4, 1_6, 2_6, 1_7, 2_7, 3_6, 1_8, 2_8, 1_9, 2_9, 3_8, 1_10, 2_10, 1_11, 2_11, 3_10] New standby tasks: [] Existing active tasks: [1_0, 1_1, 2_0, 1_2, 2_1, 3_0, 1_3, 2_2, 1_4, 2_3, 3_2, 1_5, 2_4, 1_6, 2_5, 3_4, 1_7, 2_6, 1_8, 2_7, 3_6, 1_9, 2_8, 1_10, 2_9, 3_8, 1_11, 2_10, 2_11, 3_10] Existing standby tasks: []2021-08-18 11:37:23,711 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.c.c.i.ConsumerCoordinator - [Consumer clientId=app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1-consumer, groupId=app] Adding newly assigned partitions: 2021-08-18 11:37:23,711 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.p.i.StreamThread - stream-thread [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] State transition from RUNNING to PARTITIONS_ASSIGNED2021-08-18 11:37:23,711 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.KafkaStreams - stream-client [app-1c2d16b0-7456-409d-89c5-792026f08647] State transition from RUNNING to REBALANCING2021-08-18 11:37:23,763 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.p.i.StreamThread - stream-thread [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] State transition from PARTITIONS_ASSIGNED to RUNNING2021-08-18 11:37:23,763 [app-1c2d16b0-7456-409d-89c5-792026f08647-StreamThread-1] INFO o.a.k.s.KafkaStreams - stream-client [app-1c2d16b0-7456-409d-89c5-792026f08647] State transition from REBALANCING to RUNNING{code}
According to https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java
the
State transition from PARTITIONS_ASSIGNED to PARTITIONS_ASSIGNED
should not be possible or at least not logged that way.
We will attempt an upgrade to a later version because of some bugs I found related to 2.6.0, but I wanted to log this behaviour just in case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)