[jira] [Commented] (KAFKA-3965) Mirror maker sync send fails will lose messages

["Jiangjie Qin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524370#comment-15524370 ] 

Jiangjie Qin commented on KAFKA-3965:
-------------------------------------

[~niew37] Could you confirm the mirror maker version you are running? It seems the configurations specified was for old producer. In Kafka 0.10.0.0 mirror maker only uses new producer. Regarding the configurations to ensure no data loss, you may refer to the following link:
http://www.slideshare.net/JiangjieQin/no-data-loss-pipeline-with-apache-kafka-49753844

By default, mirror maker has already been using those configurations.

> Mirror maker sync send fails will lose messages
> -----------------------------------------------
>
>                 Key: KAFKA-3965
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3965
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.10.0.0
>         Environment: SUSE Linux Enterprise Server 11 (x86_64)
>            Reporter: NieWang
>             Fix For: 0.10.1.0
>
>
> 1、Source kafka cluster storage some messages and message size is 500 bytes.
> 2、Mirror maker producer config 
>      producer.type=sync    
>      max.request.size=400
> 3、Start mirror maker backup message from source kafka cluster to destination kafka cluster, then mirror maker will quit because message larger than 400.
> 4、Check source kafka cluster will find offset have set to 1.
> 5、Check destination kafka cluster, then find have not any message.
> 6、Delete producer config max.request.size=400，start mirror maker again. Wait mirror maker finish backup, then find have lose the first message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)