[jira] [Created] (KAFKA-1633) Data loss if broker is killed

["gautham varada (JIRA)" <ji...@apache.org>]

gautham varada created KAFKA-1633:
-------------------------------------

             Summary: Data loss if broker is killed
                 Key: KAFKA-1633
                 URL: https://issues.apache.org/jira/browse/KAFKA-1633
             Project: Kafka
          Issue Type: Bug
          Components: producer 
    Affects Versions: 0.8.1.1
         Environment: centos 6.3, open jdk 7
            Reporter: gautham varada
            Assignee: Jun Rao


We have a 2 node kafka cluster, we experienced data loss when we did a kill -9 on the brokers.  We also found a work around to prevent this loss.
Replication factor :2, 4 partitions
Steps to reproduce
1. Create a 2 node cluster with replication factor 2, num partitions 4
2. We used Jmeter to pump events
3. We used kafka web console to inspect the log size after the test

During the test, we simultaneously killed the brokers using kill -9 and we tallied the metrics reported by jmeter and the size we observed in the web console, we lost tons of messages.

We went back and set the Producer retry to 1 instead of the default 3 and repeated the above tests and we did not loose a single message.

We repeated the above tests with the Producer retry set to 3 and 1 with a single broker and we observed data loss when the retry was 3 and no loss when the retry was 1





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)