[jira] [Commented] (KAFKA-12972) KStreams - Rebalance not happening after new consumer member joins group

["Matthias J. Sax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-12972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366745#comment-17366745 ] 

Matthias J. Sax commented on KAFKA-12972:
-----------------------------------------

Is your application stateful? How long did you wait? Kafka Streams does a "lazy" migration for stateful processing. Ie, it will first assign so-called "warmup task" to rebuild the state – only after the state was rebuilt, a new rebalance will be triggered re-assigning the partitions.

The logs should contain more details about it.

> KStreams - Rebalance not happening after new consumer member joins group
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-12972
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12972
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 2.7.0
>            Reporter: NEERAJ VAIDYA
>            Priority: Major
>
> I have a kafka streams application which consumes from a topic with 20 partitions.
> I run the application with
> {code:java}
> num.streams.threads = 10
> {code}
> When I check the JMX MBean _kafka_consumer_coordinator_assigned_partitions_, I can see that each thread has been assigned 2 partitions. That sounds reasonable balancing of partitions across the threads.
> Now, I add another instance of this application on a different machine/VM, but I run it with 4 stream threads.
> I was expecting these 4 threads to be assigned (via rebalancing) atleast 1 partition each, but after multiple rebalancing schedules (every 10 mins), I do not see even 1 partition being assigned to any of these 4 threads in the second instance of my application.
> I would have expected some rebalancing to happen across the 14 (10 + 4) stream threads of my application instances.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)