Re: Kafka Definitive guide v2 states auto.leader.rebalance.enable = true is not recommended

[Alexandre Dupriez <al...@gmail.com>]

Hi Liam,

The property you referred to corresponds to partition leadership, not
ownership from consumers. See
https://issues.apache.org/jira/browse/KAFKA-4084 for a discussion
about why post-incident leader rebalance can sometimes impact
foreground traffic.

Thanks,
Alexandre

Le lun. 12 avr. 2021 à 15:37, Liam Clarke-Hutchinson
<li...@adscale.co.nz> a écrit :
>
> Ah, thanks Todd :)
>
> Was it causing issues back in the day of consumer rebalances always being
> stop the world? I was wondering if the statement had perhaps predated the
> cooperative / stick assignors we're able to run now.
>
> Cheers,
>
> Liam Clarke-Hutchinson
>
>
>
> On Tue, Apr 13, 2021 at 2:34 AM Todd Palino <tp...@gmail.com> wrote:
>
> > As a note, that part of the second edition has not been updated yet. This
> > setting used to cause significant problems, but more recent updates to the
> > controller code have made the auto leader rebalancing usable.
> >
> > -Todd
> >
> > On Mon, Apr 12, 2021 at 10:20 AM Liam Clarke-Hutchinson <
> > liam.clarke@adscale.co.nz> wrote:
> >
> > > Hi all,
> > >
> > > This question arose elsewhere, and I'm also going to fire it off to
> > > O'Reilly in the hopes that they'll clarify, but on page 180 of the
> > > Definitive Guide v2
> > > <
> > >
> > https://assets.confluent.io/m/2849a76e39cda2bd/original/20201119-EB-Kafka_The_Definitive_Guide-Preview-Chapters_1_thru_6.pdf
> > > >
> > > it
> > > states:
> > >
> > > *Kafka brokers do not automatically take partition leadership back
> > (unless
> > > auto leader rebalance is enabled, but this configuration is not
> > > recommended)*
> > >
> > > The original commenter raised the point that this defaults to true, and
> > it
> > > sounds like a good idea to have auto leader rebalancing.
> > >
> > > So I'm curious, in anyone's war stories or experiences, has this property
> > > being enabled been harmful? From the context that the paragraph was
> > written
> > > in, I'm assuming the writers were perhaps intending to emphasise the
> > Cruise
> > > Control or Confluents self-balancing-cluster / auto-balancing features
> > were
> > > preferable, but in my very brief Google didn't see any advice to set
> > > auto.leader.rebalance.enabled to false to use those tools.
> > >
> > > So yeah, just curious if this rings any bells.
> > >
> > > Cheers,
> > >
> > > Liam Clarke-Hutchinson
> > >
> > --
> > *Todd Palino*
> > Senior Staff Engineer, Site Reliability
> > Capacity Engineering
> >
> >
> >
> > linkedin.com/in/toddpalino
> >