Kafka streams rebalancing issue

[Aadhil RF <aa...@gmail.com>]

Hi All,

         We have two consumers in a consumer group subscribed to the topic.
Both the consumers are in different servers. The topic consists of 11
partitions and 1 replication. Normally, 5 partitions are consumed in
consumer 1 and remaining in consumer 2. Whenever there is a connection
glitch between consumers and the coordinator, the rebalance procedure is
running on both consumers. During this procedure, all the 11 partitions are
assigned to consumer1 and two of the partitions (which are assigned to
consumer1) are assigned to consumer2. This leads to lockexception in
consumer2 and consumer2 remains in livelock to create state directories for
those two partitons.

Kafka version: 0.11.0.0
Zookeeper: 3.4.10

Re: Kafka streams rebalancing issue

["Matthias J. Sax" <ma...@confluent.io>]

> This leads to lockexception in
>> consumer2 and consumer2 remains in livelock to create state directories for
>> those two partitons.

There is a fix for a `LockException` during rebalance in 0.11.0.1:
https://issues.apache.org/jira/browse/KAFKA-5167

Maybe upgrading helps? Note, that you can upgrade Kafka Streams
independent of your brokers. Btw: I would recommend to upgrade to at
least to 0.11.0.3 what is the latest bug-fix release for 0.11.0; in
general, if there is a bug fix release, its recommended to upgrade.

Besides bug-fix release, I would recommend to upgrade to newer version
anyway (including broker if possible) as 0.11.0 is already 2 years old...



-Matthias


On 6/15/19 12:20 AM, Aadhil RF wrote:
> Hi All,
> 
>          We have two consumers in a consumer group subscribed to the topic.
> Both the consumers are in different servers. The topic consists of 11
> partitions and 1 replication. Normally, 5 partitions are consumed in
> consumer 1 and remaining in consumer 2. Whenever there is a connection
> glitch between consumers and the coordinator, the rebalance procedure is
> running on both consumers. During this procedure, all the 11 partitions are
> assigned to consumer1 and two of the partitions (which are assigned to
> consumer1) are assigned to consumer2. This leads to lockexception in
> consumer2 and consumer2 remains in livelock to create state directories for
> those two partitons.
> 
> Kafka version: 0.11.0.0
> Zookeeper: 3.4.10
>