You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jason Gustafson (JIRA)" <ji...@apache.org> on 2016/10/07 17:03:20 UTC
[jira] [Updated] (KAFKA-1183) DefaultEventHandler causes unbalanced
distribution of messages across partitions
[ https://issues.apache.org/jira/browse/KAFKA-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Gustafson updated KAFKA-1183:
-----------------------------------
Fix Version/s: (was: 0.10.1.0)
(was: 0.8.1)
> DefaultEventHandler causes unbalanced distribution of messages across partitions
> --------------------------------------------------------------------------------
>
> Key: KAFKA-1183
> URL: https://issues.apache.org/jira/browse/KAFKA-1183
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.8.0
> Reporter: Dragos Dena
> Assignee: Jun Rao
> Attachments: KAFKA-1183-trunk.patch
>
>
> KAFKA-959 introduced an optimisation in {{DefaultEventHandler}} that was supposed to have the effect of sending all messages from the same batch to a single partition if no key is specified.
> The problem is that the {{sendPartitionPerTopicCache}} cache, which holds the current selected partition for each topic, isn't actually invalided at the start or end of each batch.
> The observed result is that, after the first request chooses a random partition, all subsequent messages from that producer land in the same partition. If you have a large number of producers, then it should be fine, but if your producer count is comparable to the partition count, then it will get unbalanced.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)