Flink Kafka Connector behaviour on leader change and broker upgrade

[Janardhan Reddy <ja...@olacabs.com>]

HI,

Does the flink kafka connector 0.8.2 handle broker's leader change
gracefully since simple kafka consumer should be handling leader changes
for a partition.

How would the consumer behave when upgrading the brokers from 0.8 to 0.9.


Thanks

Re: Flink Kafka Connector behaviour on leader change and broker upgrade

[Robert Metzger <rm...@apache.org>]

Hi,
yes, the Flink Kafka connector for Kafka 0.8 handles broker leader changes
without failing. The SimpleConsumer provided by Kafka 0.8 doesn't handle
that.

The 0.9 Flink Kafka consumer also supports broker leader changes
transparently.
If you keep using the Flink Kafka 0.8 connector with a 0.9 broker, you will
need to change the protocol version of the 0.9 broker, so that the old
client can talk to it. I would recommend using matching consumer and broker
versions.

Regards,
Robert


On Fri, Nov 4, 2016 at 11:19 AM, Janardhan Reddy <
janardhan.reddy@olacabs.com> wrote:

> HI,
>
> Does the flink kafka connector 0.8.2 handle broker's leader change
> gracefully since simple kafka consumer should be handling leader changes
> for a partition.
>
> How would the consumer behave when upgrading the brokers from 0.8 to 0.9.
>
>
> Thanks
>