You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "John Fung (Created) (JIRA)" <ji...@apache.org> on 2011/12/16 22:08:30 UTC

[jira] [Created] (KAFKA-227) Add Broker Failure Test for a Single Host

Add Broker Failure Test for a Single Host
-----------------------------------------

                 Key: KAFKA-227
                 URL: https://issues.apache.org/jira/browse/KAFKA-227
             Project: Kafka
          Issue Type: Test
          Components: clients
    Affects Versions: 0.7.1
            Reporter: John Fung
            Assignee: John Fung
             Fix For: 0.7.1
         Attachments: KAFKA-227.patch

Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.

The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.

Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single Host

Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John Fung updated KAFKA-227:
----------------------------

    Attachment: KAFKA-227.patch
    
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
>                 Key: KAFKA-227
>                 URL: https://issues.apache.org/jira/browse/KAFKA-227
>             Project: Kafka
>          Issue Type: Test
>          Components: clients
>    Affects Versions: 0.7.1
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.7.1
>
>         Attachments: KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (KAFKA-227) Add Broker Failure Test for a Single Host

Posted by "Neha Narkhede (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180801#comment-13180801 ] 

Neha Narkhede commented on KAFKA-227:
-------------------------------------

This is a great patch ! Just one comment. The *sh scripts in the system_test/broker_failure need to be given execute permissions. Apart from that, it works well.
                
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
>                 Key: KAFKA-227
>                 URL: https://issues.apache.org/jira/browse/KAFKA-227
>             Project: Kafka
>          Issue Type: Test
>          Components: clients
>    Affects Versions: 0.7.1
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.7.1
>
>         Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single Host

Posted by "Neha Narkhede (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Neha Narkhede updated KAFKA-227:
--------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)
    
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
>                 Key: KAFKA-227
>                 URL: https://issues.apache.org/jira/browse/KAFKA-227
>             Project: Kafka
>          Issue Type: Test
>          Components: clients
>    Affects Versions: 0.7.1
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.7.1
>
>         Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single Host

Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John Fung updated KAFKA-227:
----------------------------

    Attachment: KAFKA-227-v3.patch

KAFKA-227-v3.patch has the following changes:

1. Terminating processes cleanly
2. Restarted Kafka broker log messages are appended to the log file of a previously terminated broker (instead of overwritten to)
3. Report producer and source / mirror consumer message count for duplicate and missing messages
4. Producer is running in the background
5. Calling ConsumerOffsetChecker to wait for zero consumer offset lagging.
                
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
>                 Key: KAFKA-227
>                 URL: https://issues.apache.org/jira/browse/KAFKA-227
>             Project: Kafka
>          Issue Type: Test
>          Components: clients
>    Affects Versions: 0.7.1
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.7.1
>
>         Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (KAFKA-227) Add Broker Failure Test for a Single Host

Posted by "Neha Narkhede (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181000#comment-13181000 ] 

Neha Narkhede commented on KAFKA-227:
-------------------------------------

I made the svn propset changes and committed this patch. This system test has been instrumental in testing KAFKA-228 and I see how this can be useful in testing more Kafka bug fixes and features
                
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
>                 Key: KAFKA-227
>                 URL: https://issues.apache.org/jira/browse/KAFKA-227
>             Project: Kafka
>          Issue Type: Test
>          Components: clients
>    Affects Versions: 0.7.1
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.7.1
>
>         Attachments: KAFKA-227-v3.patch, KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (KAFKA-227) Add Broker Failure Test for a Single Host

Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/KAFKA-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John Fung updated KAFKA-227:
----------------------------

    Status: Patch Available  (was: Open)
    
> Add Broker Failure Test for a Single Host
> -----------------------------------------
>
>                 Key: KAFKA-227
>                 URL: https://issues.apache.org/jira/browse/KAFKA-227
>             Project: Kafka
>          Issue Type: Test
>          Components: clients
>    Affects Versions: 0.7.1
>            Reporter: John Fung
>            Assignee: John Fung
>             Fix For: 0.7.1
>
>         Attachments: KAFKA-227.patch
>
>
> Add this test suite under "system_test" folder. It performs broker failure test with a mirroring setup in a single machine.
> The setup contains a source brokers cluster and a mirror brokers cluster. The brokers in both clusters will be bounced alternately.
> Message checksum is printed to the corresponding producer and consumer log files. The checksum will be validated at the end of the test. A simple logic will be used to determine how many messages are lost or duplicated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira