[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)

["Jun Rao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908997#comment-14908997 ] 

Jun Rao commented on KAFKA-2434:
--------------------------------

[~andy_k], thanks for the patch. It seems the algorithm is following: during the round-robin iteration of all the partitions, it just skips over consumers not subscribing to a topic. While this seems to guarantee consistent assignment, it doesn't seem to guarantee uniform distribution. For example, consider the following case. There are 3 consumers c1, c2, and c3. c1 and c2 subscribe to topics t1, t2, and t3, and c3 subscribes to topic t1 only. Each topic has 2 partitions. The algorithm will sort the partitions in the following order: t1-p1, t2-p1, t3-p1, t1-p2, t2-p2, t3-p2, and iterate the consumers in the order of c1, c2, c3. In the end, it seems that t1-p1, t3-p1, t2-p2 will be assigned to c1 and t2-p1, t1-p2, t3-p2 will be assigned to c2, and c3 won't get any partitions.

> remove roundrobin identical topic constraint in consumer coordinator (old API)
> ------------------------------------------------------------------------------
>
>                 Key: KAFKA-2434
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2434
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Andrew Olson
>            Assignee: Andrew Olson
>         Attachments: KAFKA-2434.patch
>
>
> The roundrobin strategy algorithm improvement made in KAFKA-2196 should be applied to the original high-level consumer API as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)