NOT_LEADER_FOR_PARTITION - Kafka Producer

[Devang Shah <de...@gmail.com>]

Dear Kafka Users,

We have recently discovered *NotLeaderForPartition *in our Kafka producer.

*Below are the versions we are using,*
Kafka Version (server) - 2.11.0-0.10.2.0
Kafka Client - 0.11.0.1
Spring Kafka - 1.3.3
Camel Kafka - 2.20.2

*Background*
We have 4 brokers on Kafka cluster. We have one topic with four partitions
and replication count of one. We have one producer which sends to specific
partition (one of the four partitions) based on key settings and four
consumers (one for each partition). On the incident day, two of the brokers
were down (was unnoticed). The other two brokers were running fine but
suddenly we started receiving *NotLeaderForPartition *in the producer logs.
I observed certain logs on kafka server which said "Replication thread
shutdown successfully" and it started the truncation process.

*Temporary issue resolution*
The Producer/Consumer were bounced for our application to start processing
the messages. The Kafka cluster was not bounced and the other two brokers
were still down.

*Questions/Queries,*
1. Does the Truncation of logs on Kafka brokers affect the
application process? Would it make the kafka cluster offline or the
truncation happens in the background?
2. We have the the property *log.retention.check.interval.ms
<http://log.retention.check.interval.ms>* set to *300000 (every 5 mins)*. I
think this is set to default and we can make a daily/weekly activity to
reduce load on the kafka brokers and maximize Kafka brokers dedication to
application processing.
3. Is there any known bug in *kafka-client-0.11.0.1* with respect to
re-connection after a network glitch.

Any pointers to resolve the above will be helpful. Thank you.

Thanks & Regards,
Devang

Re: NOT_LEADER_FOR_PARTITION - Kafka Producer

[Devang Shah <de...@gmail.com>]

Appreciate if anyone could shed some light to this issue.

Thank you.

On Sun, Mar 17, 2019, 12:06 PM Devang Shah <de...@gmail.com> wrote:

> Dear Kafka Users,
>
> We have recently discovered *NotLeaderForPartition *in our Kafka producer.
>
> *Below are the versions we are using,*
> Kafka Version (server) - 2.11.0-0.10.2.0
> Kafka Client - 0.11.0.1
> Spring Kafka - 1.3.3
> Camel Kafka - 2.20.2
>
> *Background*
> We have 4 brokers on Kafka cluster. We have one topic with four partitions
> and replication count of one. We have one producer which sends to specific
> partition (one of the four partitions) based on key settings and four
> consumers (one for each partition). On the incident day, two of the brokers
> were down (was unnoticed). The other two brokers were running fine but
> suddenly we started receiving *NotLeaderForPartition *in the producer
> logs. I observed certain logs on kafka server which said "Replication
> thread shutdown successfully" and it started the truncation process.
>
> *Temporary issue resolution*
> The Producer/Consumer were bounced for our application to start processing
> the messages. The Kafka cluster was not bounced and the other two brokers
> were still down.
>
> *Questions/Queries,*
> 1. Does the Truncation of logs on Kafka brokers affect the
> application process? Would it make the kafka cluster offline or the
> truncation happens in the background?
> 2. We have the the property *log.retention.check.interval.ms
> <http://log.retention.check.interval.ms>* set to *300000 (every 5 mins)*.
> I think this is set to default and we can make a daily/weekly activity to
> reduce load on the kafka brokers and maximize Kafka brokers dedication to
> application processing.
> 3. Is there any known bug in *kafka-client-0.11.0.1* with respect to
> re-connection after a network glitch.
>
> Any pointers to resolve the above will be helpful. Thank you.
>
> Thanks & Regards,
> Devang
>