You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "John Fung (JIRA)" <ji...@apache.org> on 2012/11/08 05:02:12 UTC
[jira] [Comment Edited] (KAFKA-603) System Test Data Validation
Failure - Replication Factor less than No. of Broker
[ https://issues.apache.org/jira/browse/KAFKA-603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492926#comment-13492926 ]
John Fung edited comment on KAFKA-603 at 11/8/12 4:00 AM:
----------------------------------------------------------
This test case (testcase_4011) is set up as followings:
1. Set up a 3-broker cluster with 2 topics, 2 partitions, Replication Factor 2
2. Producer to produce messages in Sync mode, Acks = -1, Compression Off
3. The expected results would be to see 3 Simple Consumer message count: one of them is zero, two of them are equal.
4. The results shown below is that all ConsoleConsumer messages are matching. However, simple consumer are showing unexpected messages count
To reproduce the issue:
1. Download the latest 0.8 branch
2. Apply the attached patch and build
3. In <kafka_home>/system_test, execute: python –B system_test_runner.py
validation_status :
Unique messages from consumer on [test_1] : 1000
Unique messages from consumer on [test_1] at simple_consumer_1.log : 1000
Unique messages from consumer on [test_1] at simple_consumer_2.log : 500
Unique messages from consumer on [test_1] at simple_consumer_3.log : 500
Unique messages from consumer on [test_2] : 1000
Unique messages from consumer on [test_2] at simple_consumer_1.log : 1000
Unique messages from consumer on [test_2] at simple_consumer_2.log : 500
Unique messages from consumer on [test_2] at simple_consumer_3.log : 505
Unique messages from producer on [test_1] : 1000
Unique messages from producer on [test_2] : 1000
Validate for data matched on topic [test_1] : PASSED
Validate for data matched on topic [test_1] across replicas : FAILED
Validate for data matched on topic [test_2] : PASSED
Validate for data matched on topic [test_2] across replicas : FAILED
Broker Log Segments:
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_1-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_1-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_1_logs/test_1-0/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_1-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_1-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_1_logs/test_1-1/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_2-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_2-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 57942 Nov 7 16:40 kafka_server_1_logs/test_2-0/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_2-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_2-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_1_logs/test_2-1/00000000000000000394.log
======================================================
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_2_logs/test_1-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_2_logs/test_1-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_2_logs/test_1-1/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_2_logs/test_2-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_2_logs/test_2-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_2_logs/test_2-1/00000000000000000394.log
======================================================
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_3_logs/test_1-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_3_logs/test_1-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_3_logs/test_1-0/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_3_logs/test_2-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_3_logs/test_2-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 57942 Nov 7 16:40 kafka_server_3_logs/test_2-0/00000000000000000394.log
was (Author: jfung):
This test case is set up as followings:
1. Set up a 3-broker cluster with 2 topics, 2 partitions, Replication Factor 2
2. Producer to produce messages in Sync mode, Acks = -1, Compression Off
3. The expected results would be to see 3 Simple Consumer message count: one of them is zero, two of them are equal.
4. The results shown below is that all ConsoleConsumer messages are matching. However, simple consumer are showing unexpected messages count
To reproduce the issue:
1. Download the latest 0.8 branch
2. Apply the attached patch and build
3. In <kafka_home>/system_test, execute: python –B system_test_runner.py
validation_status :
Unique messages from consumer on [test_1] : 1000
Unique messages from consumer on [test_1] at simple_consumer_1.log : 1000
Unique messages from consumer on [test_1] at simple_consumer_2.log : 500
Unique messages from consumer on [test_1] at simple_consumer_3.log : 500
Unique messages from consumer on [test_2] : 1000
Unique messages from consumer on [test_2] at simple_consumer_1.log : 1000
Unique messages from consumer on [test_2] at simple_consumer_2.log : 500
Unique messages from consumer on [test_2] at simple_consumer_3.log : 505
Unique messages from producer on [test_1] : 1000
Unique messages from producer on [test_2] : 1000
Validate for data matched on topic [test_1] : PASSED
Validate for data matched on topic [test_1] across replicas : FAILED
Validate for data matched on topic [test_2] : PASSED
Validate for data matched on topic [test_2] across replicas : FAILED
Broker Log Segments:
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_1-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_1-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_1_logs/test_1-0/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_1-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_1-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_1_logs/test_1-1/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_2-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_2-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 57942 Nov 7 16:40 kafka_server_1_logs/test_2-0/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_1_logs/test_2-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_1_logs/test_2-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_1_logs/test_2-1/00000000000000000394.log
======================================================
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_2_logs/test_1-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_2_logs/test_1-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_2_logs/test_1-1/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_2_logs/test_2-1/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_2_logs/test_2-1/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_2_logs/test_2-1/00000000000000000394.log
======================================================
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_3_logs/test_1-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_3_logs/test_1-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 55332 Nov 7 16:40 kafka_server_3_logs/test_1-0/00000000000000000394.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:39 kafka_server_3_logs/test_2-0/00000000000000000000.log
-rw-r--r-- 1 jfung eng 102834 Nov 7 16:40 kafka_server_3_logs/test_2-0/00000000000000000197.log
-rw-r--r-- 1 jfung eng 57942 Nov 7 16:40 kafka_server_3_logs/test_2-0/00000000000000000394.log
> System Test Data Validation Failure - Replication Factor less than No. of Broker
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-603
> URL: https://issues.apache.org/jira/browse/KAFKA-603
> Project: Kafka
> Issue Type: Bug
> Reporter: John Fung
> Priority: Critical
> Attachments: kafka-603-reproduce-issue.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira