You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "John Fung (Created) (JIRA)" <ji...@apache.org> on 2011/12/16 22:08:30 UTC
[jira] [Created] (KAFKA-227) Add Broker Failure Test for a Single
Host
Add Broker Failure Test for a Single Host
-----------------------------------------
Key: KAFKA-227
URL: https://issues.apache.org/jira/browse/KAFKA-227
Project: Kafka
Issue Type: Test
Components: clients
Affects Versions: 0.7.1
Reporter: John Fung
Assignee: John Fung
Fix For: 0.7.1
Attachments: KAFKA-227.patch
Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single
Host
Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Fung updated KAFKA-227:
----------------------------
Attachment: KAFKA-227.patch
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
> Key: KAFKA-227
> URL: https://issues.apache.org/jira/browse/KAFKA-227
> Project: Kafka
> Issue Type: Test
> Components: clients
> Affects Versions: 0.7.1
> Reporter: John Fung
> Assignee: John Fung
> Fix For: 0.7.1
>
> Attachments: KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-227) Add Broker Failure Test for a Single
Host
Posted by "Neha Narkhede (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180801#comment-13180801 ]
Neha Narkhede commented on KAFKA-227:
-------------------------------------
This is a great patch ! Just one comment. The *sh scripts in the system_test/broker_failure need to be given execute permissions. Apart from that, it works well.
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
> Key: KAFKA-227
> URL: https://issues.apache.org/jira/browse/KAFKA-227
> Project: Kafka
> Issue Type: Test
> Components: clients
> Affects Versions: 0.7.1
> Reporter: John Fung
> Assignee: John Fung
> Fix For: 0.7.1
>
> Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single
Host
Posted by "Neha Narkhede (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-227:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
> Key: KAFKA-227
> URL: https://issues.apache.org/jira/browse/KAFKA-227
> Project: Kafka
> Issue Type: Test
> Components: clients
> Affects Versions: 0.7.1
> Reporter: John Fung
> Assignee: John Fung
> Fix For: 0.7.1
>
> Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single
Host
Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Fung updated KAFKA-227:
----------------------------
Attachment: KAFKA-227-v3.patch
KAFKA-227-v3.patch has the following changes:
1. Terminating processes cleanly
2. Restarted Kafka broker log messages are appended to the log file of a previously terminated broker (instead of overwritten to)
3. Report producer and source / mirror consumer message count for duplicate and missing messages
4. Producer is running in the background
5. Calling ConsumerOffsetChecker to wait for zero consumer offset lagging.
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
> Key: KAFKA-227
> URL: https://issues.apache.org/jira/browse/KAFKA-227
> Project: Kafka
> Issue Type: Test
> Components: clients
> Affects Versions: 0.7.1
> Reporter: John Fung
> Assignee: John Fung
> Fix For: 0.7.1
>
> Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-227) Add Broker Failure Test for a Single
Host
Posted by "Neha Narkhede (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181000#comment-13181000 ]
Neha Narkhede commented on KAFKA-227:
-------------------------------------
I made the svn propset changes and committed this patch. This system test has been instrumental in testing KAFKA-228 and I see how this can be useful in testing more Kafka bug fixes and features
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
> Key: KAFKA-227
> URL: https://issues.apache.org/jira/browse/KAFKA-227
> Project: Kafka
> Issue Type: Test
> Components: clients
> Affects Versions: 0.7.1
> Reporter: John Fung
> Assignee: John Fung
> Fix For: 0.7.1
>
> Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single
Host
Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Fung updated KAFKA-227:
----------------------------
Status: Patch Available (was: Open)
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
> Key: KAFKA-227
> URL: https://issues.apache.org/jira/browse/KAFKA-227
> Project: Kafka
> Issue Type: Test
> Components: clients
> Affects Versions: 0.7.1
> Reporter: John Fung
> Assignee: John Fung
> Fix For: 0.7.1
>
> Attachments: KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira