You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "John Fung (Created) (JIRA)" <ji...@apache.org> on 2012/02/03 22:42:53 UTC
[jira] [Created] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
-------------------------------------------------------------------------------------------------------
Key: KAFKA-263
URL: https://issues.apache.org/jira/browse/KAFKA-263
Project: Kafka
Issue Type: Improvement
Components: clients
Reporter: John Fung
Assignee: John Fung
The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
source clusters
============
broker1 - topic1, topic2
broker2 - topic1, topic2
broker3 - topic1, topic2
broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Posted by "Neha Narkhede (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-263:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed this, after making the following minor changes -
Changed ProducerPerformance to output the checksums at the DEBUG level
Changed ProducerPerformance to use the new Logging trait.
> Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-263
> URL: https://issues.apache.org/jira/browse/KAFKA-263
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Reporter: John Fung
> Assignee: John Fung
> Attachments: kafka-263-v1.patch, kafka-263-v2.patch
>
>
> The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
> source clusters
> ============
> broker1 - topic1, topic2
> broker2 - topic1, topic2
> broker3 - topic1, topic2
> broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Fung updated KAFKA-263:
----------------------------
Attachment: kafka-263-v1.patch
> Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-263
> URL: https://issues.apache.org/jira/browse/KAFKA-263
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Reporter: John Fung
> Assignee: John Fung
> Attachments: kafka-263-v1.patch
>
>
> The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
> source clusters
> ============
> broker1 - topic1, topic2
> broker2 - topic1, topic2
> broker3 - topic1, topic2
> broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Fung updated KAFKA-263:
----------------------------
Attachment: kafka-263-v2.patch
Hi Neha,
Thanks for your review. I have made the changes in v2 patch:
- init is reverted to its original signature
- topic is passed into ChecksumMessageFormatter from props
John
> Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-263
> URL: https://issues.apache.org/jira/browse/KAFKA-263
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Reporter: John Fung
> Assignee: John Fung
> Attachments: kafka-263-v1.patch, kafka-263-v2.patch
>
>
> The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
> source clusters
> ============
> broker1 - topic1, topic2
> broker2 - topic1, topic2
> broker3 - topic1, topic2
> broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Posted by "Neha Narkhede (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201466#comment-13201466 ]
Neha Narkhede commented on KAFKA-263:
-------------------------------------
I took a quick look at the patch and have one suggestion -
- def init(props: Properties) {}
+ def init(props: Properties, tp: String) {}
The above change seems unnecessary. Since the MessageFormatter takes in a Properties object, you can pass in the topic as part of the --property topic=foo argument for the ConsoleConsumer.
Apart of the above, the patch looks good.
> Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-263
> URL: https://issues.apache.org/jira/browse/KAFKA-263
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Reporter: John Fung
> Assignee: John Fung
> Attachments: kafka-263-v1.patch
>
>
> The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
> source clusters
> ============
> broker1 - topic1, topic2
> broker2 - topic1, topic2
> broker3 - topic1, topic2
> broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Posted by "John Fung (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Fung updated KAFKA-263:
----------------------------
Status: Patch Available (was: Open)
Submitting patch. Please note that the following 2 files are modified to print topic info together with checksum such that the no. of checksum for each topic could be validated.
core/src/main/scala/kafka/consumer/ConsoleConsumer.scala
perf/src/main/scala/kafka/perf/ProducerPerformance.scala
> Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-263
> URL: https://issues.apache.org/jira/browse/KAFKA-263
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Reporter: John Fung
> Assignee: John Fung
> Attachments: kafka-263-v1.patch
>
>
> The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
> source clusters
> ============
> broker1 - topic1, topic2
> broker2 - topic1, topic2
> broker3 - topic1, topic2
> broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-263) Enhance single host broker failure
test to have 2 topics with uneven distribution on the source brokers
Posted by "Neha Narkhede (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202547#comment-13202547 ]
Neha Narkhede commented on KAFKA-263:
-------------------------------------
+1 on v2. Thanks for working on this patch !
> Enhance single host broker failure test to have 2 topics with uneven distribution on the source brokers
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-263
> URL: https://issues.apache.org/jira/browse/KAFKA-263
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Reporter: John Fung
> Assignee: John Fung
> Attachments: kafka-263-v1.patch, kafka-263-v2.patch
>
>
> The change is going to have 1 broker in the source cluster to consume messages of only 1 topic as shown below:
> source clusters
> ============
> broker1 - topic1, topic2
> broker2 - topic1, topic2
> broker3 - topic1, topic2
> broker4 - topic1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira