You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by tao xiao <xi...@gmail.com> on 2016/04/09 08:50:50 UTC

Kafka broker constantly ISR shrinking and expanding

Hi team,

We have a cluster of 40 nodes in production. We observed that one of the
nodes ran into a situation where it constantly shrunk and expanded ISR for
more than 10 hours and unable to recover until the broker was bounced.

Here are the snippet of server.log from the broker and controller log from
controller
https://gist.github.com/xiaotao183/0cd198f6571bf4b85d0005242c1dea78

I'd like to know what would cause this issue and why the broken broker
(node 14) was not kicked out of ISR.

Kafka version is 0.9.0.1 and unclean leader election is turned on