You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "David Almroth (JIRA)" <ji...@apache.org> on 2013/03/09 16:45:12 UTC

[jira] [Commented] (KAFKA-789) Producer-side persistence for delivery guarantee

    [ https://issues.apache.org/jira/browse/KAFKA-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597982#comment-13597982 ] 

David Almroth commented on KAFKA-789:
-------------------------------------

Looks like this is a duplicate of https://issues.apache.org/jira/browse/KAFKA-156

                
> Producer-side persistence for delivery guarantee
> ------------------------------------------------
>
>                 Key: KAFKA-789
>                 URL: https://issues.apache.org/jira/browse/KAFKA-789
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer 
>            Reporter: Matan Safriel
>            Assignee: Jun Rao
>            Priority: Minor
>             Fix For: 0.9
>
>
> A suggestion for higher guarantee for the part of entering messages into Kafka through it's producer. It aims to address the case that the entire set of broker replicas for a topic and partition is not available. Currently, in that case, data is lost. When a message set exhausts the send retry counter, the message set will be simply dropped. It would be nice being able to provide higher guarantee that a message passed to the producer would eventually be received by the broker. 
> In an environment with some disk space to spare for this on the producer side, persisting to disk would seem to enable keeping messages for later retry (until defined space limits are exhausted). Thus somewhat elevating the level of guarantee. 
> One way to facilitate this would be capitalizing on https://issues.apache.org/jira/browse/KAFKA-496, as the feedback it will add will enable knowing what needs to be retried again later. Changes to the producer or a wrapper around it (that may require access to the partitioning functions) would be able to persist failed message sets and manage delivery with a nice level of guarantee. As it would affect performance and use disks, should probably be a non-default option.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira