Connection to broker which lost Zk connection

[Krzysztof Nawara <kr...@cern.ch>]

Hello!

I've got the following question:
Let's assume broker A is leader for partition P1, to which my producer is sending data. At one point A loses its connection to Zk, but my producer can communicate with A without problems. Will my messages be lost? What'll trigger switching to the new leader since A is no longer aware of what happens in the cluster (because it doesn't have connection to Zk)?

Krzysiek

Re: Connection to broker which lost Zk connection

[Anderson Goulart <an...@boxever.com>]

Hi,

I am not an expert, but as I can say as an ops guy is that when a broker 
loses its connection with Zk, it triggers a reelection and A will be out 
of ISR. So no data will go to broker A anymore and during this timeframe 
(from connection lost to reelection) the producer should receive an 
Exception and you should handle it (like trying again, for example). 
After the reelection (which is really fast), the producers will send 
data to the new leader (AFAIK it is transparent to the producers - the 
client libraries will handle this change)

In other words: you can only read/write messages to/from leaders which 
is managed by Zk. If Zk can't connect to the node, it will remove it 
from your cluster (even if the producer can communicate with the broker).

-- Anderson


On 30/07/2016 22:06, Krzysztof Nawara wrote:
> Hello!
>
> I've got the following question:
> Let's assume broker A is leader for partition P1, to which my producer is sending data. At one point A loses its connection to Zk, but my producer can communicate with A without problems. Will my messages be lost? What'll trigger switching to the new leader since A is no longer aware of what happens in the cluster (because it doesn't have connection to Zk)?
>
> Krzysiek