You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jiangjie Qin (JIRA)" <ji...@apache.org> on 2015/03/13 18:08:38 UTC
[jira] [Commented] (KAFKA-2019) RoundRobinAssignor clusters by
consumer
[ https://issues.apache.org/jira/browse/KAFKA-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360659#comment-14360659 ]
Jiangjie Qin commented on KAFKA-2019:
-------------------------------------
In this case is it that each consumer has large amount of consumer threads? Theoretically, in current code, a consumer should not take more than number-of-consumer-thread partitions than any other thread.
Anyway, I think sort threads based on hash code makes sense.
> RoundRobinAssignor clusters by consumer
> ---------------------------------------
>
> Key: KAFKA-2019
> URL: https://issues.apache.org/jira/browse/KAFKA-2019
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Reporter: Joseph Holsten
> Assignee: Neha Narkhede
> Priority: Minor
> Attachments: 0001-sort-consumer-thread-ids-by-hashcode.patch
>
>
> When rolling out a change today, I noticed that some of my consumers are "greedy", taking far more partitions than others.
> The cause is that the RoundRobinAssignor is using a list of ConsumerThreadIds sorted by toString, which is {{ "%s-%d".format(consumer, threadId)}}. This causes each consumer's threads to be adjacent to each other.
> One possible fix would be to define ConsumerThreadId.hashCode, and sort by that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)