You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jeff Gong <j....@gmail.com> on 2015/07/22 20:54:05 UTC

Kafka message queue during broker failure

hi all,

currently working with a team that is interested in replacing scribe with
kafka. currently in scribe's model it is my understanding that when there
is a network failure scribe will attempt to persist messages by queueing
them and resending the queued messages when those issues are resolved.

i understand that for kafka this issue is attempted to be avoided with the
replication factor, but in the case where all brokers are disconnected or
facing network issues is it correct that kafka will simply fail to send
messages in this time? if this is the case, is there any plan to develop a
similar model to handle network issues?

thanks for any help!

Re: Kafka message queue during broker failure

Posted by Gwen Shapira <gs...@cloudera.com>.
If you are using the new Kafka Producer (in org.apache.kafka.clients
package), you can configure number of retries. The Producer will queue
messages and re-attempt to send them as specified.

On Wed, Jul 22, 2015 at 11:54 AM, Jeff Gong <j....@gmail.com> wrote:
> hi all,
>
> currently working with a team that is interested in replacing scribe with
> kafka. currently in scribe's model it is my understanding that when there
> is a network failure scribe will attempt to persist messages by queueing
> them and resending the queued messages when those issues are resolved.
>
> i understand that for kafka this issue is attempted to be avoided with the
> replication factor, but in the case where all brokers are disconnected or
> facing network issues is it correct that kafka will simply fail to send
> messages in this time? if this is the case, is there any plan to develop a
> similar model to handle network issues?
>
> thanks for any help!