[jira] [Comment Edited] (KAFKA-3818) Change Mirror Maker default assignment strategy to round robin

["Jason Gustafson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332798#comment-15332798 ] 

Jason Gustafson edited comment on KAFKA-3818 at 6/15/16 11:04 PM:
------------------------------------------------------------------

[~granthenke] Good question. The round robin assignor used by the new consumer is a little different. It doesn't use the same hashing approach as the old one. It just takes all the partitions from all the subscribed topics and lays them out in order in an array (so that all partitions from the same topic are next to each other). It then round-robins through the consumers and assign partitions one by one. The consumers are sorted by the memberId given them by the coordinator, which is "\{clientId\}-\{uuid\}". If a consumer group is using the same clientId, that the ordering of the consumers should be random and this approach ought to give a good distribution for each topic, but it would be nice to verify whether this is actually true. In any case, even if the distribution is less than ideal, it might still be better than that provided by range, especially when you have a lot of topics with few partitions. 


was (Author: hachikuji):
[~granthenke] Good question. The round robin assignor used by the new consumer is a little different. It doesn't use the same hashing approach as the old one. It just takes all the partitions from all the subscribed topics and lays them out in order in an array (so that all partitions from the same topic are next to each other). It then round-robins through the consumers and assign partitions one by one. The consumers are sorted by the memberId given them by the coordinator, which is "{clientId}-{uuid}". If a consumer group is using the same clientId, that the ordering of the consumers should be random and this approach ought to give a good distribution for each topic, but it would be nice to verify whether this is actually true. In any case, even if the distribution is less than ideal, it might still be better than that provided by range, especially when you have a lot of topics with few partitions. 

> Change Mirror Maker default assignment strategy to round robin
> --------------------------------------------------------------
>
>                 Key: KAFKA-3818
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3818
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Gustafson
>            Assignee: Vahid Hashemian
>
> It might make more sense to use round robin assignment by default for MM since it gives a better balance between the instances, in particular when the number of MM instances exceeds the typical number of partitions per topic. There doesn't seem to be any need to keep range assignment since copartitioning is not an issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)