Doc says when doing re-balance, sort by leader then partition, but the code seems sort only on partition

[李响 <wa...@gmail.com>]

Dear Kafka community,

In the doc -> https://kafka.apache.org/documentation/#distributionimpl
4. sort Pt (so partitions on the same broker are clustered together)
and
During rebalancing, we try to assign partitions to consumers in such a way
that reduces the number of broker nodes each consumer has to connect to

If I get it correctly, it means when sorting the partitions, firstly it is
sorted by leader. Then among those partitions which have the same leader,
then it is sorted by partition id in numeric order.

While the code of RangeAssignor of
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/PartitionAssignor.scala
seems tell that it is sorted only on partition, not on leader.

My trial also demonstrated that the code is correct.

Is the doc out of date? Or the doc follows some previous versions of Kafka?
Or my understanding on code has something wrong?

Please kindly advise, thanks!!


-- 

                                               李响 Xiang Li

邮件 e-mail      ：waterlx@gmail.com

Re: Doc says when doing re-balance, sort by leader then partition, but the code seems sort only on partition

[Guozhang Wang <wa...@gmail.com>]

Hello Xiang,

I took a look at the PartitionAssignor and I agree with you that the doc is
referring to an older version of the code now. It could be updated.

BTW, this logic is referring the old ZK-dependent consumer behavior, which
has been deprecated and will likely be removed in the future major
releases; so we'd probably just remove this part of the web docs when we do
that.



Guozhang


On Thu, Nov 30, 2017 at 4:31 AM, 李响 <wa...@gmail.com> wrote:

> Dear Kafka community,
>
> In the doc -> https://kafka.apache.org/documentation/#distributionimpl
> 4. sort Pt (so partitions on the same broker are clustered together)
> and
> During rebalancing, we try to assign partitions to consumers in such a way
> that reduces the number of broker nodes each consumer has to connect to
>
> If I get it correctly, it means when sorting the partitions, firstly it is
> sorted by leader. Then among those partitions which have the same leader,
> then it is sorted by partition id in numeric order.
>
> While the code of RangeAssignor of
> https://github.com/apache/kafka/blob/trunk/core/src/
> main/scala/kafka/consumer/PartitionAssignor.scala
> seems tell that it is sorted only on partition, not on leader.
>
> My trial also demonstrated that the code is correct.
>
> Is the doc out of date? Or the doc follows some previous versions of Kafka?
> Or my understanding on code has something wrong?
>
> Please kindly advise, thanks!!
>
>
> --
>
>                                                李响 Xiang Li
>
> 邮件 e-mail      ：waterlx@gmail.com
>



-- 
-- Guozhang