You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ashish Surana (JIRA)" <ji...@apache.org> on 2018/07/11 04:58:00 UTC
[jira] [Commented] (KAFKA-6976) Kafka Streams instances going in to
DEAD state
[ https://issues.apache.org/jira/browse/KAFKA-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539565#comment-16539565 ]
Ashish Surana commented on KAFKA-6976:
--------------------------------------
Created below ticket to address this issue the right way:
https://issues.apache.org/jira/browse/KAFKA-7149: Reduce assignment data size to improve kafka streams scalability
> Kafka Streams instances going in to DEAD state
> ----------------------------------------------
>
> Key: KAFKA-6976
> URL: https://issues.apache.org/jira/browse/KAFKA-6976
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Deepak Goyal
> Priority: Blocker
> Attachments: kafkaStreamsDeadState.log
>
>
> We are using Kafka 0.10.2.0, Kafka-Streams 1.1.0. We have Kafka Cluster of 16 machines, and topic that is being consumed by Kafka Streams has 256 partitions. We spawned 400 machines of Kakfa Streams application. We see that all of the StreamThreads go in to DEAD state.
> {quote}{{[2018-05-25 05:59:29,282] INFO stream-thread [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7-StreamThread-1] State transition from PENDING_SHUTDOWN to DEAD (org.apache.kafka.streams.processor.internals.StreamThread) [2018-05-25 05:59:29,282] INFO stream-client [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7] State transition from REBALANCING to ERROR (org.apache.kafka.streams.KafkaStreams) [2018-05-25 05:59:29,282] WARN stream-client [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7] All stream threads have died. The instance will be in error state and should be closed. (org.apache.kafka.streams.KafkaStreams) [2018-05-25 05:59:29,282] INFO stream-thread [ksapp-19f923d7-5f9e-4137-b79f-ee20945a7dd7-StreamThread-1] Shutdown complete (org.apache.kafka.streams.processor.internals.StreamThread)}}
> {quote}
> Please note that when we only have 100 kafka-streams application machines, things are working as expected. We see that instances are consuming messages from topic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)