[jira] [Commented] (KAFKA-13226) Partition expansion may cause uneven distribution

["Guozhang Wang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434516#comment-17434516 ] 

Guozhang Wang commented on KAFKA-13226:
---------------------------------------

Hello! It's a bit unfortunate that not everyone here can understand Chinese :P, but after looking through KafkaZkClient#getAllBrokerAndEpochsInCluster seems indeed have a bug here, where at the end when we call `toMap` the returned broker list "map" is no longer sorted. It could cause the following:

* When we create topics for the first time, the broker list that we are assigning partitions to are unsorted.
* When we later expand partitions of the newly created topic, the broker list that we are assigning new partitions to are sorted.

As a result, we would end up with unbalance reassignment -- I think this is indeed a bug that worth fixing in KafkaZkClient given that we have not fully deprecate this path, and if not fixing it we may well again falls into this scenario later.

cc [~junrao] since you reviewed the original PR https://github.com/apache/kafka/pull/5821, does this make sense?

> Partition expansion may cause uneven distribution
> -------------------------------------------------
>
>                 Key: KAFKA-13226
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13226
>             Project: Kafka
>          Issue Type: Bug
>          Components: controller
>    Affects Versions: 2.5.0, 2.8.0, 2.7.1, 2.6.2
>         Environment: mac  
> kafka-2.5.0
>            Reporter: shizhenzhen
>            Priority: Major
>
>  
>  {color:#ff0000}*Partition expansion may cause uneven distribution*{color}
>  
> 1. Create a Topic  , 3-partition   1-replical
> !https://img-blog.csdnimg.cn/561112064b114acfb03882aa09100e0e.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_55,color_FFFFFF,t_70,g_se,x_16!
>  
> 2. partition expansion to 5 - partiton
> !https://img-blog.csdnimg.cn/f7c3c33b6662457080d9bb5bb190c0c2.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_49,color_FFFFFF,t_70,g_se,x_16!
>  
> 3. Does this meet expectations ？
>  
> !https://img-blog.csdnimg.cn/20cc1007c4214c4ebfcb1b2c2eeb98e4.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_18,color_FFFFFF,t_70,g_se,x_16!
>  
> {color:#ff0000}*so this is a bug ?*{color}
>  
> The problem may arise here ; 
> When we create a new topic .  get the broker list is a Object Map ; 
> *This is disordered*
> you can read the code , first it have sortBy brokerId, but finally it convert to a *disorde Map;*
>  
>  
> !https://img-blog.csdnimg.cn/131b9bf0c19e4753a73512af4c9c5854.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_66,color_FFFFFF,t_70,g_se,x_16!
>  
>  
>  
> The important thing is that it has been sorted when expanding the partition and parition-reassignment ;   
> {color:#ff0000}*So why not sort when creating topics?*{color}
>  
> If the topic is sorted when  create a new topic , this problem will not occur ;
>  
> so it maybe is a tiny bug ?
>  
>  
> if you can read Chinese ,
> You can look at this article. I describe it in detail
>  
> We look forward to receiving your reply
>  
> 如果你能看懂中文,可以看看这篇文章 我描述的很详细! 
>  [This may be a Kafka bug？|[https://shirenchuang.blog.csdn.net/article/details/119912418]]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)