You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Matan Safriel (JIRA)" <ji...@apache.org> on 2013/03/09 18:19:16 UTC

[jira] [Comment Edited] (KAFKA-156) Messages should not be dropped when brokers are unavailable

    [ https://issues.apache.org/jira/browse/KAFKA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13598000#comment-13598000 ] 

Matan Safriel edited comment on KAFKA-156 at 3/9/13 5:17 PM:
-------------------------------------------------------------

I was led to believe this is most probably not planned for 0.8 now. 
Just to comment that currently in 0.8 a message set asynchronously sent may partially fail, in the sense that only some of its destinations (determined by topic & partition) will fail while others succeed. That seems to mean a 'one log spool' for all messages (if all messages are spooled before attempting to send them) requires non-sequential management of such a log, or overheads over-proportional to the proportion of failed messages. Perhaps a log per topic would simplify the algorithm but then there's partitions as well...
                
      was (Author: matan):
    I was led to believe this is most probably not planned for 0.8. 
Just to comment that currently in 0.8 a message set asynchronously sent may partially fail, in the sense that only some of its destinations (determined by topic & partition) will fail while others succeed. That seems to mean a 'one log spool' for all messages (if all messages are spooled before attempting to send them) requires non-sequential management of such a log, or overheads over-proportional to the proportion of failed messages. Perhaps a log per topic would simplify the algorithm but then there's partitions as well...
                  
> Messages should not be dropped when brokers are unavailable
> -----------------------------------------------------------
>
>                 Key: KAFKA-156
>                 URL: https://issues.apache.org/jira/browse/KAFKA-156
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Sharad Agarwal
>             Fix For: 0.8
>
>
> When none of the broker is available, producer should spool the messages to disk and keep retrying for brokers to come back.
> This will also enable brokers upgrade/maintenance without message loss.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira