You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Raman Gupta (Jira)" <ji...@apache.org> on 2019/10/01 17:07:00 UTC
[jira] [Updated] (KAFKA-8966) Stream state does not transition to
RUNNING on client, broker consumer group shows RUNNING
[ https://issues.apache.org/jira/browse/KAFKA-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raman Gupta updated KAFKA-8966:
-------------------------------
Description:
I have a Kafka stream that has been running fine until recently. The new behavior I see is that the stream state on the client goes from CREATED to REBALANCING, but never transitions from REBALANCING to RUNNING.
However, at the same time, if I look at the offsets of the corresponding consumer group, the consumer group appears to be consuming from the topic and has no lag. And yet, the client never made a state change to RUNNING. This is confirmed by calling `streams.close` on the stream and noting the state change goes from REBALANCING to PENDING_SHUTDOWN instead of RUNNING to PENDING_SHUTDOWN as expected.
I use the state change to enable queries on the stream store -- if the state change listener never triggers to the RUNNING state, there is no way to know when the client is available for queries.
Yes, I have confirmed its the correct consumer group. Yes, the consumer group has no consumers when I shut down the client stream.
Server logs:
kafka-2 kafka 2019-10-01T16:59:36.348859731Z [2019-10-01 16:59:36,348] INFO [GroupCoordinator 2]: Preparing to rebalance group arena-rg-uiService-fileStatusStore-stream in state PreparingRebalance with old generation 0 (__consumer_offsets-42) (reason: Adding new member arena-rg-uiService-fileStatusStore-stream-0a954f60-f8a3-4f13-8d9e-6caa63773dd2-StreamThread-1-consumer-325a6889-659f-48cb-b308-0d626b573944 with group instanceid None) (kafka.coordinator.group.GroupCoordinator)
kafka-2 kafka 2019-10-01T17:00:06.349171842Z [2019-10-01 17:00:06,348] INFO [GroupCoordinator 2]: Stabilized group arena-rg-uiService-fileStatusStore-stream generation 1 (__consumer_offsets-42) (kafka.coordinator.group.GroupCoordinator)
kafka-2 kafka 2019-10-01T17:00:06.604980028Z [2019-10-01 17:00:06,604] INFO [GroupCoordinator 2]: Assignment received from leader for group arena-rg-uiService-fileStatusStore-stream for generation 1 (kafka.coordinator.group.GroupCoordinator)
was:
I have a Kafka stream that has been running fine until recently. The new behavior I see is that the stream state on the client goes from CREATED to REBALANCING, but never transitions from REBALANCING to RUNNING.
However, at the same time, if I look at the offsets of the corresponding consumer group, the consumer group appears to be consuming from the topic and has no lag. And yet, the client never made a state change to RUNNING. This is confirmed by calling `streams.close` on the stream and noting the state change goes from REBALANCING to PENDING_SHUTDOWN instead of RUNNING to PENDING_SHUTDOWN as expected.
I use the state change to enable queries on the stream store -- if the state change listener never triggers to the RUNNING state, there is no way to know when the client is available for queries.
Yes, I have confirmed its the correct consumer group. Yes, the consumer group has no consumers when I shut down the client stream.
> Stream state does not transition to RUNNING on client, broker consumer group shows RUNNING
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-8966
> URL: https://issues.apache.org/jira/browse/KAFKA-8966
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.3.0
> Reporter: Raman Gupta
> Priority: Critical
>
> I have a Kafka stream that has been running fine until recently. The new behavior I see is that the stream state on the client goes from CREATED to REBALANCING, but never transitions from REBALANCING to RUNNING.
> However, at the same time, if I look at the offsets of the corresponding consumer group, the consumer group appears to be consuming from the topic and has no lag. And yet, the client never made a state change to RUNNING. This is confirmed by calling `streams.close` on the stream and noting the state change goes from REBALANCING to PENDING_SHUTDOWN instead of RUNNING to PENDING_SHUTDOWN as expected.
> I use the state change to enable queries on the stream store -- if the state change listener never triggers to the RUNNING state, there is no way to know when the client is available for queries.
> Yes, I have confirmed its the correct consumer group. Yes, the consumer group has no consumers when I shut down the client stream.
> Server logs:
> kafka-2 kafka 2019-10-01T16:59:36.348859731Z [2019-10-01 16:59:36,348] INFO [GroupCoordinator 2]: Preparing to rebalance group arena-rg-uiService-fileStatusStore-stream in state PreparingRebalance with old generation 0 (__consumer_offsets-42) (reason: Adding new member arena-rg-uiService-fileStatusStore-stream-0a954f60-f8a3-4f13-8d9e-6caa63773dd2-StreamThread-1-consumer-325a6889-659f-48cb-b308-0d626b573944 with group instanceid None) (kafka.coordinator.group.GroupCoordinator)
> kafka-2 kafka 2019-10-01T17:00:06.349171842Z [2019-10-01 17:00:06,348] INFO [GroupCoordinator 2]: Stabilized group arena-rg-uiService-fileStatusStore-stream generation 1 (__consumer_offsets-42) (kafka.coordinator.group.GroupCoordinator)
> kafka-2 kafka 2019-10-01T17:00:06.604980028Z [2019-10-01 17:00:06,604] INFO [GroupCoordinator 2]: Assignment received from leader for group arena-rg-uiService-fileStatusStore-stream for generation 1 (kafka.coordinator.group.GroupCoordinator)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)