clean leader election on kafka 0.10.2.1

[Henry Cai <hc...@pinterest.com.INVALID>]

We were on kafka 0.10.2.1.  We tried to switch from unclean leader election
to clean leader election and found it became very difficult to start up the
whole cluster.

It seems the hosts went into a deadlock situation during startup
    - broker A was a follower on partition 1 and waits for the leader of
partition 1 (which is broker B) to start up
    - broker B was a follower on partition 2 and waits for the leader of
partition 2 (which is broker A) to start up

We found there are quite a few deadlock related bugs fixed in 0.11 or
later, do we have to upgrade our kafka version to use clean leader election?

Re: clean leader election on kafka 0.10.2.1

[Guozhang Wang <wa...@gmail.com>]

Henry,

I am not sure what do you mean by "waits for the leader of partition to
start up"? Leader election should not affect leader - follower starting up
process.


Guozhang


On Thu, Nov 2, 2017 at 4:56 PM, Henry Cai <hc...@pinterest.com.invalid>
wrote:

> We were on kafka 0.10.2.1.  We tried to switch from unclean leader election
> to clean leader election and found it became very difficult to start up the
> whole cluster.
>
> It seems the hosts went into a deadlock situation during startup
>     - broker A was a follower on partition 1 and waits for the leader of
> partition 1 (which is broker B) to start up
>     - broker B was a follower on partition 2 and waits for the leader of
> partition 2 (which is broker A) to start up
>
> We found there are quite a few deadlock related bugs fixed in 0.11 or
> later, do we have to upgrade our kafka version to use clean leader
> election?
>



-- 
-- Guozhang