Doubts regarding KafkaProducer implemetation

[Madhukar Bharti <bh...@gmail.com>]

Hi,

We have three brokers in a cluster with replication factor is 3. We are
using Kafka-0.10.0.1. We see some failures on metadata timeout exceptions
while producing.
We have configured retries=3 and max in flight request=1.
After comparing with the old scala Producer code found that in new Producer
"retries" is not working if there is a failure on meta-data refresh so
an exception is throwing to caller API.

But in older Producer, it was retrying based on configured values and
throwing exception only after all retries completed. Can we achieve this in
new Producer? As we don't see any exception like *failed to produce after 3
retries*.



Regards,
Madhukar Bharti

Re: Doubts regarding KafkaProducer implemetation

[Manikumar <ma...@gmail.com>]

Hi,

java producer waits for the metadata of the given topic to be available. We
will get
TimeoutException if we didn't get metadata within max.block.ms. As observed
by you, this
behavior is different from old producer.  "retries" config is not
applicable to metadata fetch.
We can adjust max.block.ms to avoid metadata Timeout exception.


On Mon, Mar 13, 2017 at 4:04 PM, Madhukar Bharti <bh...@gmail.com>
wrote:

> Hi,
>
> We have three brokers in a cluster with replication factor is 3. We are
> using Kafka-0.10.0.1. We see some failures on metadata timeout exceptions
> while producing.
> We have configured retries=3 and max in flight request=1.
> After comparing with the old scala Producer code found that in new Producer
> "retries" is not working if there is a failure on meta-data refresh so
> an exception is throwing to caller API.
>
> But in older Producer, it was retrying based on configured values and
> throwing exception only after all retries completed. Can we achieve this in
> new Producer? As we don't see any exception like *failed to produce after 3
> retries*.
>
>
>
> Regards,
> Madhukar Bharti
>