Failed to Send Message After 3 Tries

["Leng, Hao" <HA...@dish.com>]

Hi there,

We had error logs for three messages failed to produce to Kafka during last week. All three are failed on the same day within one hour range. We checked Kafka logs (server.log and statechange.log) but found no abnormal behaviors.

The exception is :
        kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
        at kafka.producer.async.DefaultEventHandler.handle(Unknown Source)
        at kafka.producer.Producer.send(Unknown Source)
        at kafka.javaapi.producer.Producer.send(Unknown Source)

The Kafka server we use is 0.8.2-beta and our kafka jar is kafka_2.10.0-0.8.0-beta1.jar. On producer properties, we use producer.type = sync and request.required.acks = 1. The rest are default values. There is no activity needed and the system becomes normal again.

Can you please suggest what may cause this issue and how we can avoid it? Thank you and have a good day.

Sincerely,
Hao Leng

Re: Failed to Send Message After 3 Tries

["Helleren, Erik" <Er...@cmegroup.com>]

So, this is expected behavior on the producer when its unable to
communicate with the kafka broker that is the leader for the message is
being sent.  

First, if the design of your app allows, try to migrate to the new
producer API release is 0.8.1  It is fully asynchronous, and provides
callbacks on completion or error cases.  The documentation makes a clear
case for its use. (Easier and more efficient)  And since your server is
already on 0.8.2, you should run the client jar for 0.8.2 as well.

If the design does not, the app may either manually catch and retry the
message, or you can increase the number of times the kafka library will
try to resend using the producer property.

There is no way to prevent retries, its up to the app to handle them
properly at the risk of message loss, retransmission, or hung processes.
-Erik


On 9/15/15, 1:17 PM, "Leng, Hao" <HA...@dish.com> wrote:

>Hi there,
>
>We had error logs for three messages failed to produce to Kafka during
>last week. All three are failed on the same day within one hour range. We
>checked Kafka logs (server.log and statechange.log) but found no abnormal
>behaviors.
>
>The exception is :
>        kafka.common.FailedToSendMessageException: Failed to send
>messages after 3 tries.
>        at kafka.producer.async.DefaultEventHandler.handle(Unknown Source)
>        at kafka.producer.Producer.send(Unknown Source)
>        at kafka.javaapi.producer.Producer.send(Unknown Source)
>
>The Kafka server we use is 0.8.2-beta and our kafka jar is
>kafka_2.10.0-0.8.0-beta1.jar. On producer properties, we use
>producer.type = sync and request.required.acks = 1. The rest are default
>values. There is no activity needed and the system becomes normal again.
>
>Can you please suggest what may cause this issue and how we can avoid it?
>Thank you and have a good day.
>
>Sincerely,
>Hao Leng
>