[jira] [Commented] (KAFKA-959) DefaultEventHandler can send more produce requests than necesary

["Guozhang Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702718#comment-13702718 ] 

Guozhang Wang commented on KAFKA-959:
-------------------------------------

Proposed code change:

1. Add a Map<Topic, PartitionIndex> to DefaultEventHandller.

2. At the start of handle(events: Seq[KeyedMessage[K,V]]) function, clear the map.

3. In getPartition function, if key == null, check if there is already an entry in the map with the given topic:

3.1. If not, choose one partition from the availablePartitions, and add that partition to the map, return this partition index.

3.2. If yes, check if its leaderBrokerIdOpt is defined; if yes, return this partition index directly, otherwise throw an exception.
                
> DefaultEventHandler can send more produce requests than necesary
> ----------------------------------------------------------------
>
>                 Key: KAFKA-959
>                 URL: https://issues.apache.org/jira/browse/KAFKA-959
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Jun Rao
>
> In DefaultEventHandler, for a batch of messages, it picks a random partition per message (when there is no key specified). This means that it can send up to P produce requests where P is the number of partitions in a topic. A better way is probably to pick a single random partition for the whole batch of messages. This will reduce the number of produce requests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira