You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Luke Chen <sh...@gmail.com> on 2024/08/28 10:45:16 UTC
[DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Hi all,
In v3.9.0, we're planning to support the `unclean.leader.election.enable`
config in KRaft. Currently, the implementation PR
<https://github.com/apache/kafka/pull/16866> is under review now. But I'd
like to raise the discussion to let you know there will be one behavior
change when enabling `unclean.leader.election.enable` config dynamically in
KRaft.
scenario:
1. broker [0, 1] config are all set `unclean.leader.election.enable=false`.
2. topic A is created with 1 partition, 2 replication factors and
`unclean.leader.election.enable=false`. Leader of the partition 0 is 0. ISR
is [0].
3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
4. dynamically alter config using admin API to set
`unclean.leader.election.enable=true`, no matter it changes to the topic A,
or to broker 1, or default broker.
In ZK mode, with this scenario, it'll trigger unclean leader election
immediately, and broker 1 will be elected as the leader, which brings the
availability of the partition back.
In KRaft mode, with this scenario, the unclean leader election will be
triggered in an unclean election thread, which is scheduled every 5 mins by
default. That is, this partition might still be unavailable even if it
enables `unclean.leader.election.enable`. If users want to elect a leader
for this partition, run `kafka-leader-election.sh` with `unclean` option to
trigger the unclean leader election immediately.
The main reason we didn't implement what ZK did is because of this
complication
<https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
faced.
Other behaviors keep the same. Ex: When the topic config, or broker config,
or default broker config set `unclean.leader.election.enable=true`, and
later, a partition has leader down with empty ISR [ ], and having some
alive non-ISR replicas, in this case, the unclean leader election will be
triggered, as long as the config was set earlier.
But since the PR is still under review, any comments or thoughts are
welcomed.
Thank you.
Luke
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Colin McCabe <cm...@apache.org>.
Thanks, Luke. Both the static and dynamic options for configuring unclean leader election now work in 3.9, thanks to your help.
Dima, if you have only a transient need to do an unclean leader election, using the kafka-leader-election.sh script is still the best way. It avoids the risk for forgetting to turn off unclean leader election afterwards, and it's faster. You should only set the configurations for unclean leader election if you want to leave it on forever. (i.e. you genuinely don't care about losing data, and just always want a leader.)
best,
Colin
On Wed, Aug 28, 2024, at 23:11, Dima Brodsky wrote:
> Thank you! This helps alot
>
> Sent from Mobile
>
>
> On Wed, Aug 28, 2024 at 19:48 Luke Chen <sh...@gmail.com> wrote:
>
>> Hi Dima,
>>
>> Before v3.9.0, brokers in KRaft mode didn't honor
>> `unclean.leader.election.enable` config.
>> So, the static config and dynamic config didn't work.
>> The only thing works before v3.9.0 is using he admin API or cli to run
>> `kafka-leader-election.sh` with `unclean` option.
>>
>> Thanks.
>> Luke
>>
>> On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <dd...@gmail.com> wrote:
>>
>> > Thanks Luke,
>> >
>> > I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
>> >
>> > ttyl
>> > Dima
>> >
>> >
>> > On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
>> >
>> > > Hi Dima,
>> > >
>> > > > I assume that unclean leader election exists as a static config in
>> > kafka
>> > > kraft, but is still unavailable dynamically? Or is it unavailable in
>> > > general?
>> > >
>> > > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable`
>> in
>> > > KRaft will work, either static config or dynamic config.
>> > > It's just the behavior changes when dynamically enabling the
>> > > `unclean.leader.election.enable`, which will not trigger unclean leader
>> > > election immediately after config change. The detailed behavior change
>> is
>> > > as described above.
>> > >
>> > > Let me know if it's still not clear.
>> > >
>> > > Thanks.
>> > > Luke
>> > >
>> > >
>> > > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Luke,
>> > > >
>> > > > I assume that unclean leader election exists as a static config in
>> > kafka
>> > > > kraft, but is still unavailable dynamically? Or is it unavailable in
>> > > > general?
>> > > >
>> > > > Thanks!
>> > > > ttyl
>> > > > Dima
>> > > >
>> > > >
>> > > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > In v3.9.0, we're planning to support the
>> > > `unclean.leader.election.enable`
>> > > > > config in KRaft. Currently, the implementation PR
>> > > > > <https://github.com/apache/kafka/pull/16866> is under review now.
>> > But
>> > > > I'd
>> > > > > like to raise the discussion to let you know there will be one
>> > behavior
>> > > > > change when enabling `unclean.leader.election.enable` config
>> > > dynamically
>> > > > in
>> > > > > KRaft.
>> > > > >
>> > > > > scenario:
>> > > > > 1. broker [0, 1] config are all set
>> > > > `unclean.leader.election.enable=false`.
>> > > > > 2. topic A is created with 1 partition, 2 replication factors and
>> > > > > `unclean.leader.election.enable=false`. Leader of the partition 0
>> is
>> > 0.
>> > > > ISR
>> > > > > is [0].
>> > > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader
>> none.
>> > > > > 4. dynamically alter config using admin API to set
>> > > > > `unclean.leader.election.enable=true`, no matter it changes to the
>> > > topic
>> > > > A,
>> > > > > or to broker 1, or default broker.
>> > > > >
>> > > > > In ZK mode, with this scenario, it'll trigger unclean leader
>> election
>> > > > > immediately, and broker 1 will be elected as the leader, which
>> brings
>> > > the
>> > > > > availability of the partition back.
>> > > > > In KRaft mode, with this scenario, the unclean leader election will
>> > be
>> > > > > triggered in an unclean election thread, which is scheduled every 5
>> > > mins
>> > > > by
>> > > > > default. That is, this partition might still be unavailable even if
>> > it
>> > > > > enables `unclean.leader.election.enable`. If users want to elect a
>> > > leader
>> > > > > for this partition, run `kafka-leader-election.sh` with `unclean`
>> > > option
>> > > > to
>> > > > > trigger the unclean leader election immediately.
>> > > > >
>> > > > > The main reason we didn't implement what ZK did is because of this
>> > > > > complication
>> > > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206
>> >
>> > we
>> > > > > faced.
>> > > > >
>> > > > > Other behaviors keep the same. Ex: When the topic config, or broker
>> > > > config,
>> > > > > or default broker config set `unclean.leader.election.enable=true`,
>> > and
>> > > > > later, a partition has leader down with empty ISR [ ], and having
>> > some
>> > > > > alive non-ISR replicas, in this case, the unclean leader election
>> > will
>> > > be
>> > > > > triggered, as long as the config was set earlier.
>> > > > >
>> > > > > But since the PR is still under review, any comments or thoughts
>> are
>> > > > > welcomed.
>> > > > >
>> > > > > Thank you.
>> > > > > Luke
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > ddbrodsky@gmail.com
>> > > >
>> > > > "The price of reliability is the pursuit of the utmost simplicity.
>> > > > It is a price which the very rich find the most hard to pay."
>> > > >
>> (Sir
>> > > > Antony Hoare, 1980)
>> > > >
>> > >
>> >
>> >
>> > --
>> > ddbrodsky@gmail.com
>> >
>> > "The price of reliability is the pursuit of the utmost simplicity.
>> > It is a price which the very rich find the most hard to pay."
>> > (Sir
>> > Antony Hoare, 1980)
>> >
>>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Colin McCabe <cm...@apache.org>.
Thanks, Luke. Both the static and dynamic options for configuring unclean leader election now work in 3.9, thanks to your help.
Dima, if you have only a transient need to do an unclean leader election, using the kafka-leader-election.sh script is still the best way. It avoids the risk for forgetting to turn off unclean leader election afterwards, and it's faster. You should only set the configurations for unclean leader election if you want to leave it on forever. (i.e. you genuinely don't care about losing data, and just always want a leader.)
best,
Colin
On Wed, Aug 28, 2024, at 23:11, Dima Brodsky wrote:
> Thank you! This helps alot
>
> Sent from Mobile
>
>
> On Wed, Aug 28, 2024 at 19:48 Luke Chen <sh...@gmail.com> wrote:
>
>> Hi Dima,
>>
>> Before v3.9.0, brokers in KRaft mode didn't honor
>> `unclean.leader.election.enable` config.
>> So, the static config and dynamic config didn't work.
>> The only thing works before v3.9.0 is using he admin API or cli to run
>> `kafka-leader-election.sh` with `unclean` option.
>>
>> Thanks.
>> Luke
>>
>> On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <dd...@gmail.com> wrote:
>>
>> > Thanks Luke,
>> >
>> > I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
>> >
>> > ttyl
>> > Dima
>> >
>> >
>> > On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
>> >
>> > > Hi Dima,
>> > >
>> > > > I assume that unclean leader election exists as a static config in
>> > kafka
>> > > kraft, but is still unavailable dynamically? Or is it unavailable in
>> > > general?
>> > >
>> > > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable`
>> in
>> > > KRaft will work, either static config or dynamic config.
>> > > It's just the behavior changes when dynamically enabling the
>> > > `unclean.leader.election.enable`, which will not trigger unclean leader
>> > > election immediately after config change. The detailed behavior change
>> is
>> > > as described above.
>> > >
>> > > Let me know if it's still not clear.
>> > >
>> > > Thanks.
>> > > Luke
>> > >
>> > >
>> > > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Luke,
>> > > >
>> > > > I assume that unclean leader election exists as a static config in
>> > kafka
>> > > > kraft, but is still unavailable dynamically? Or is it unavailable in
>> > > > general?
>> > > >
>> > > > Thanks!
>> > > > ttyl
>> > > > Dima
>> > > >
>> > > >
>> > > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > In v3.9.0, we're planning to support the
>> > > `unclean.leader.election.enable`
>> > > > > config in KRaft. Currently, the implementation PR
>> > > > > <https://github.com/apache/kafka/pull/16866> is under review now.
>> > But
>> > > > I'd
>> > > > > like to raise the discussion to let you know there will be one
>> > behavior
>> > > > > change when enabling `unclean.leader.election.enable` config
>> > > dynamically
>> > > > in
>> > > > > KRaft.
>> > > > >
>> > > > > scenario:
>> > > > > 1. broker [0, 1] config are all set
>> > > > `unclean.leader.election.enable=false`.
>> > > > > 2. topic A is created with 1 partition, 2 replication factors and
>> > > > > `unclean.leader.election.enable=false`. Leader of the partition 0
>> is
>> > 0.
>> > > > ISR
>> > > > > is [0].
>> > > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader
>> none.
>> > > > > 4. dynamically alter config using admin API to set
>> > > > > `unclean.leader.election.enable=true`, no matter it changes to the
>> > > topic
>> > > > A,
>> > > > > or to broker 1, or default broker.
>> > > > >
>> > > > > In ZK mode, with this scenario, it'll trigger unclean leader
>> election
>> > > > > immediately, and broker 1 will be elected as the leader, which
>> brings
>> > > the
>> > > > > availability of the partition back.
>> > > > > In KRaft mode, with this scenario, the unclean leader election will
>> > be
>> > > > > triggered in an unclean election thread, which is scheduled every 5
>> > > mins
>> > > > by
>> > > > > default. That is, this partition might still be unavailable even if
>> > it
>> > > > > enables `unclean.leader.election.enable`. If users want to elect a
>> > > leader
>> > > > > for this partition, run `kafka-leader-election.sh` with `unclean`
>> > > option
>> > > > to
>> > > > > trigger the unclean leader election immediately.
>> > > > >
>> > > > > The main reason we didn't implement what ZK did is because of this
>> > > > > complication
>> > > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206
>> >
>> > we
>> > > > > faced.
>> > > > >
>> > > > > Other behaviors keep the same. Ex: When the topic config, or broker
>> > > > config,
>> > > > > or default broker config set `unclean.leader.election.enable=true`,
>> > and
>> > > > > later, a partition has leader down with empty ISR [ ], and having
>> > some
>> > > > > alive non-ISR replicas, in this case, the unclean leader election
>> > will
>> > > be
>> > > > > triggered, as long as the config was set earlier.
>> > > > >
>> > > > > But since the PR is still under review, any comments or thoughts
>> are
>> > > > > welcomed.
>> > > > >
>> > > > > Thank you.
>> > > > > Luke
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > ddbrodsky@gmail.com
>> > > >
>> > > > "The price of reliability is the pursuit of the utmost simplicity.
>> > > > It is a price which the very rich find the most hard to pay."
>> > > >
>> (Sir
>> > > > Antony Hoare, 1980)
>> > > >
>> > >
>> >
>> >
>> > --
>> > ddbrodsky@gmail.com
>> >
>> > "The price of reliability is the pursuit of the utmost simplicity.
>> > It is a price which the very rich find the most hard to pay."
>> > (Sir
>> > Antony Hoare, 1980)
>> >
>>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Dima Brodsky <dd...@gmail.com>.
Thank you! This helps alot
Sent from Mobile
On Wed, Aug 28, 2024 at 19:48 Luke Chen <sh...@gmail.com> wrote:
> Hi Dima,
>
> Before v3.9.0, brokers in KRaft mode didn't honor
> `unclean.leader.election.enable` config.
> So, the static config and dynamic config didn't work.
> The only thing works before v3.9.0 is using he admin API or cli to run
> `kafka-leader-election.sh` with `unclean` option.
>
> Thanks.
> Luke
>
> On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <dd...@gmail.com> wrote:
>
> > Thanks Luke,
> >
> > I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
> >
> > ttyl
> > Dima
> >
> >
> > On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
> >
> > > Hi Dima,
> > >
> > > > I assume that unclean leader election exists as a static config in
> > kafka
> > > kraft, but is still unavailable dynamically? Or is it unavailable in
> > > general?
> > >
> > > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable`
> in
> > > KRaft will work, either static config or dynamic config.
> > > It's just the behavior changes when dynamically enabling the
> > > `unclean.leader.election.enable`, which will not trigger unclean leader
> > > election immediately after config change. The detailed behavior change
> is
> > > as described above.
> > >
> > > Let me know if it's still not clear.
> > >
> > > Thanks.
> > > Luke
> > >
> > >
> > > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com>
> > wrote:
> > >
> > > > Hi Luke,
> > > >
> > > > I assume that unclean leader election exists as a static config in
> > kafka
> > > > kraft, but is still unavailable dynamically? Or is it unavailable in
> > > > general?
> > > >
> > > > Thanks!
> > > > ttyl
> > > > Dima
> > > >
> > > >
> > > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > In v3.9.0, we're planning to support the
> > > `unclean.leader.election.enable`
> > > > > config in KRaft. Currently, the implementation PR
> > > > > <https://github.com/apache/kafka/pull/16866> is under review now.
> > But
> > > > I'd
> > > > > like to raise the discussion to let you know there will be one
> > behavior
> > > > > change when enabling `unclean.leader.election.enable` config
> > > dynamically
> > > > in
> > > > > KRaft.
> > > > >
> > > > > scenario:
> > > > > 1. broker [0, 1] config are all set
> > > > `unclean.leader.election.enable=false`.
> > > > > 2. topic A is created with 1 partition, 2 replication factors and
> > > > > `unclean.leader.election.enable=false`. Leader of the partition 0
> is
> > 0.
> > > > ISR
> > > > > is [0].
> > > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader
> none.
> > > > > 4. dynamically alter config using admin API to set
> > > > > `unclean.leader.election.enable=true`, no matter it changes to the
> > > topic
> > > > A,
> > > > > or to broker 1, or default broker.
> > > > >
> > > > > In ZK mode, with this scenario, it'll trigger unclean leader
> election
> > > > > immediately, and broker 1 will be elected as the leader, which
> brings
> > > the
> > > > > availability of the partition back.
> > > > > In KRaft mode, with this scenario, the unclean leader election will
> > be
> > > > > triggered in an unclean election thread, which is scheduled every 5
> > > mins
> > > > by
> > > > > default. That is, this partition might still be unavailable even if
> > it
> > > > > enables `unclean.leader.election.enable`. If users want to elect a
> > > leader
> > > > > for this partition, run `kafka-leader-election.sh` with `unclean`
> > > option
> > > > to
> > > > > trigger the unclean leader election immediately.
> > > > >
> > > > > The main reason we didn't implement what ZK did is because of this
> > > > > complication
> > > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206
> >
> > we
> > > > > faced.
> > > > >
> > > > > Other behaviors keep the same. Ex: When the topic config, or broker
> > > > config,
> > > > > or default broker config set `unclean.leader.election.enable=true`,
> > and
> > > > > later, a partition has leader down with empty ISR [ ], and having
> > some
> > > > > alive non-ISR replicas, in this case, the unclean leader election
> > will
> > > be
> > > > > triggered, as long as the config was set earlier.
> > > > >
> > > > > But since the PR is still under review, any comments or thoughts
> are
> > > > > welcomed.
> > > > >
> > > > > Thank you.
> > > > > Luke
> > > > >
> > > >
> > > >
> > > > --
> > > > ddbrodsky@gmail.com
> > > >
> > > > "The price of reliability is the pursuit of the utmost simplicity.
> > > > It is a price which the very rich find the most hard to pay."
> > > >
> (Sir
> > > > Antony Hoare, 1980)
> > > >
> > >
> >
> >
> > --
> > ddbrodsky@gmail.com
> >
> > "The price of reliability is the pursuit of the utmost simplicity.
> > It is a price which the very rich find the most hard to pay."
> > (Sir
> > Antony Hoare, 1980)
> >
>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Dima Brodsky <dd...@gmail.com>.
Thank you! This helps alot
Sent from Mobile
On Wed, Aug 28, 2024 at 19:48 Luke Chen <sh...@gmail.com> wrote:
> Hi Dima,
>
> Before v3.9.0, brokers in KRaft mode didn't honor
> `unclean.leader.election.enable` config.
> So, the static config and dynamic config didn't work.
> The only thing works before v3.9.0 is using he admin API or cli to run
> `kafka-leader-election.sh` with `unclean` option.
>
> Thanks.
> Luke
>
> On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <dd...@gmail.com> wrote:
>
> > Thanks Luke,
> >
> > I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
> >
> > ttyl
> > Dima
> >
> >
> > On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
> >
> > > Hi Dima,
> > >
> > > > I assume that unclean leader election exists as a static config in
> > kafka
> > > kraft, but is still unavailable dynamically? Or is it unavailable in
> > > general?
> > >
> > > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable`
> in
> > > KRaft will work, either static config or dynamic config.
> > > It's just the behavior changes when dynamically enabling the
> > > `unclean.leader.election.enable`, which will not trigger unclean leader
> > > election immediately after config change. The detailed behavior change
> is
> > > as described above.
> > >
> > > Let me know if it's still not clear.
> > >
> > > Thanks.
> > > Luke
> > >
> > >
> > > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com>
> > wrote:
> > >
> > > > Hi Luke,
> > > >
> > > > I assume that unclean leader election exists as a static config in
> > kafka
> > > > kraft, but is still unavailable dynamically? Or is it unavailable in
> > > > general?
> > > >
> > > > Thanks!
> > > > ttyl
> > > > Dima
> > > >
> > > >
> > > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > In v3.9.0, we're planning to support the
> > > `unclean.leader.election.enable`
> > > > > config in KRaft. Currently, the implementation PR
> > > > > <https://github.com/apache/kafka/pull/16866> is under review now.
> > But
> > > > I'd
> > > > > like to raise the discussion to let you know there will be one
> > behavior
> > > > > change when enabling `unclean.leader.election.enable` config
> > > dynamically
> > > > in
> > > > > KRaft.
> > > > >
> > > > > scenario:
> > > > > 1. broker [0, 1] config are all set
> > > > `unclean.leader.election.enable=false`.
> > > > > 2. topic A is created with 1 partition, 2 replication factors and
> > > > > `unclean.leader.election.enable=false`. Leader of the partition 0
> is
> > 0.
> > > > ISR
> > > > > is [0].
> > > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader
> none.
> > > > > 4. dynamically alter config using admin API to set
> > > > > `unclean.leader.election.enable=true`, no matter it changes to the
> > > topic
> > > > A,
> > > > > or to broker 1, or default broker.
> > > > >
> > > > > In ZK mode, with this scenario, it'll trigger unclean leader
> election
> > > > > immediately, and broker 1 will be elected as the leader, which
> brings
> > > the
> > > > > availability of the partition back.
> > > > > In KRaft mode, with this scenario, the unclean leader election will
> > be
> > > > > triggered in an unclean election thread, which is scheduled every 5
> > > mins
> > > > by
> > > > > default. That is, this partition might still be unavailable even if
> > it
> > > > > enables `unclean.leader.election.enable`. If users want to elect a
> > > leader
> > > > > for this partition, run `kafka-leader-election.sh` with `unclean`
> > > option
> > > > to
> > > > > trigger the unclean leader election immediately.
> > > > >
> > > > > The main reason we didn't implement what ZK did is because of this
> > > > > complication
> > > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206
> >
> > we
> > > > > faced.
> > > > >
> > > > > Other behaviors keep the same. Ex: When the topic config, or broker
> > > > config,
> > > > > or default broker config set `unclean.leader.election.enable=true`,
> > and
> > > > > later, a partition has leader down with empty ISR [ ], and having
> > some
> > > > > alive non-ISR replicas, in this case, the unclean leader election
> > will
> > > be
> > > > > triggered, as long as the config was set earlier.
> > > > >
> > > > > But since the PR is still under review, any comments or thoughts
> are
> > > > > welcomed.
> > > > >
> > > > > Thank you.
> > > > > Luke
> > > > >
> > > >
> > > >
> > > > --
> > > > ddbrodsky@gmail.com
> > > >
> > > > "The price of reliability is the pursuit of the utmost simplicity.
> > > > It is a price which the very rich find the most hard to pay."
> > > >
> (Sir
> > > > Antony Hoare, 1980)
> > > >
> > >
> >
> >
> > --
> > ddbrodsky@gmail.com
> >
> > "The price of reliability is the pursuit of the utmost simplicity.
> > It is a price which the very rich find the most hard to pay."
> > (Sir
> > Antony Hoare, 1980)
> >
>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Luke Chen <sh...@gmail.com>.
Hi Dima,
Before v3.9.0, brokers in KRaft mode didn't honor
`unclean.leader.election.enable` config.
So, the static config and dynamic config didn't work.
The only thing works before v3.9.0 is using he admin API or cli to run
`kafka-leader-election.sh` with `unclean` option.
Thanks.
Luke
On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <dd...@gmail.com> wrote:
> Thanks Luke,
>
> I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
>
> ttyl
> Dima
>
>
> On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
>
> > Hi Dima,
> >
> > > I assume that unclean leader election exists as a static config in
> kafka
> > kraft, but is still unavailable dynamically? Or is it unavailable in
> > general?
> >
> > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in
> > KRaft will work, either static config or dynamic config.
> > It's just the behavior changes when dynamically enabling the
> > `unclean.leader.election.enable`, which will not trigger unclean leader
> > election immediately after config change. The detailed behavior change is
> > as described above.
> >
> > Let me know if it's still not clear.
> >
> > Thanks.
> > Luke
> >
> >
> > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com>
> wrote:
> >
> > > Hi Luke,
> > >
> > > I assume that unclean leader election exists as a static config in
> kafka
> > > kraft, but is still unavailable dynamically? Or is it unavailable in
> > > general?
> > >
> > > Thanks!
> > > ttyl
> > > Dima
> > >
> > >
> > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > In v3.9.0, we're planning to support the
> > `unclean.leader.election.enable`
> > > > config in KRaft. Currently, the implementation PR
> > > > <https://github.com/apache/kafka/pull/16866> is under review now.
> But
> > > I'd
> > > > like to raise the discussion to let you know there will be one
> behavior
> > > > change when enabling `unclean.leader.election.enable` config
> > dynamically
> > > in
> > > > KRaft.
> > > >
> > > > scenario:
> > > > 1. broker [0, 1] config are all set
> > > `unclean.leader.election.enable=false`.
> > > > 2. topic A is created with 1 partition, 2 replication factors and
> > > > `unclean.leader.election.enable=false`. Leader of the partition 0 is
> 0.
> > > ISR
> > > > is [0].
> > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> > > > 4. dynamically alter config using admin API to set
> > > > `unclean.leader.election.enable=true`, no matter it changes to the
> > topic
> > > A,
> > > > or to broker 1, or default broker.
> > > >
> > > > In ZK mode, with this scenario, it'll trigger unclean leader election
> > > > immediately, and broker 1 will be elected as the leader, which brings
> > the
> > > > availability of the partition back.
> > > > In KRaft mode, with this scenario, the unclean leader election will
> be
> > > > triggered in an unclean election thread, which is scheduled every 5
> > mins
> > > by
> > > > default. That is, this partition might still be unavailable even if
> it
> > > > enables `unclean.leader.election.enable`. If users want to elect a
> > leader
> > > > for this partition, run `kafka-leader-election.sh` with `unclean`
> > option
> > > to
> > > > trigger the unclean leader election immediately.
> > > >
> > > > The main reason we didn't implement what ZK did is because of this
> > > > complication
> > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206>
> we
> > > > faced.
> > > >
> > > > Other behaviors keep the same. Ex: When the topic config, or broker
> > > config,
> > > > or default broker config set `unclean.leader.election.enable=true`,
> and
> > > > later, a partition has leader down with empty ISR [ ], and having
> some
> > > > alive non-ISR replicas, in this case, the unclean leader election
> will
> > be
> > > > triggered, as long as the config was set earlier.
> > > >
> > > > But since the PR is still under review, any comments or thoughts are
> > > > welcomed.
> > > >
> > > > Thank you.
> > > > Luke
> > > >
> > >
> > >
> > > --
> > > ddbrodsky@gmail.com
> > >
> > > "The price of reliability is the pursuit of the utmost simplicity.
> > > It is a price which the very rich find the most hard to pay."
> > > (Sir
> > > Antony Hoare, 1980)
> > >
> >
>
>
> --
> ddbrodsky@gmail.com
>
> "The price of reliability is the pursuit of the utmost simplicity.
> It is a price which the very rich find the most hard to pay."
> (Sir
> Antony Hoare, 1980)
>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Luke Chen <sh...@gmail.com>.
Hi Dima,
Before v3.9.0, brokers in KRaft mode didn't honor
`unclean.leader.election.enable` config.
So, the static config and dynamic config didn't work.
The only thing works before v3.9.0 is using he admin API or cli to run
`kafka-leader-election.sh` with `unclean` option.
Thanks.
Luke
On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <dd...@gmail.com> wrote:
> Thanks Luke,
>
> I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
>
> ttyl
> Dima
>
>
> On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
>
> > Hi Dima,
> >
> > > I assume that unclean leader election exists as a static config in
> kafka
> > kraft, but is still unavailable dynamically? Or is it unavailable in
> > general?
> >
> > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in
> > KRaft will work, either static config or dynamic config.
> > It's just the behavior changes when dynamically enabling the
> > `unclean.leader.election.enable`, which will not trigger unclean leader
> > election immediately after config change. The detailed behavior change is
> > as described above.
> >
> > Let me know if it's still not clear.
> >
> > Thanks.
> > Luke
> >
> >
> > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com>
> wrote:
> >
> > > Hi Luke,
> > >
> > > I assume that unclean leader election exists as a static config in
> kafka
> > > kraft, but is still unavailable dynamically? Or is it unavailable in
> > > general?
> > >
> > > Thanks!
> > > ttyl
> > > Dima
> > >
> > >
> > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > In v3.9.0, we're planning to support the
> > `unclean.leader.election.enable`
> > > > config in KRaft. Currently, the implementation PR
> > > > <https://github.com/apache/kafka/pull/16866> is under review now.
> But
> > > I'd
> > > > like to raise the discussion to let you know there will be one
> behavior
> > > > change when enabling `unclean.leader.election.enable` config
> > dynamically
> > > in
> > > > KRaft.
> > > >
> > > > scenario:
> > > > 1. broker [0, 1] config are all set
> > > `unclean.leader.election.enable=false`.
> > > > 2. topic A is created with 1 partition, 2 replication factors and
> > > > `unclean.leader.election.enable=false`. Leader of the partition 0 is
> 0.
> > > ISR
> > > > is [0].
> > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> > > > 4. dynamically alter config using admin API to set
> > > > `unclean.leader.election.enable=true`, no matter it changes to the
> > topic
> > > A,
> > > > or to broker 1, or default broker.
> > > >
> > > > In ZK mode, with this scenario, it'll trigger unclean leader election
> > > > immediately, and broker 1 will be elected as the leader, which brings
> > the
> > > > availability of the partition back.
> > > > In KRaft mode, with this scenario, the unclean leader election will
> be
> > > > triggered in an unclean election thread, which is scheduled every 5
> > mins
> > > by
> > > > default. That is, this partition might still be unavailable even if
> it
> > > > enables `unclean.leader.election.enable`. If users want to elect a
> > leader
> > > > for this partition, run `kafka-leader-election.sh` with `unclean`
> > option
> > > to
> > > > trigger the unclean leader election immediately.
> > > >
> > > > The main reason we didn't implement what ZK did is because of this
> > > > complication
> > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206>
> we
> > > > faced.
> > > >
> > > > Other behaviors keep the same. Ex: When the topic config, or broker
> > > config,
> > > > or default broker config set `unclean.leader.election.enable=true`,
> and
> > > > later, a partition has leader down with empty ISR [ ], and having
> some
> > > > alive non-ISR replicas, in this case, the unclean leader election
> will
> > be
> > > > triggered, as long as the config was set earlier.
> > > >
> > > > But since the PR is still under review, any comments or thoughts are
> > > > welcomed.
> > > >
> > > > Thank you.
> > > > Luke
> > > >
> > >
> > >
> > > --
> > > ddbrodsky@gmail.com
> > >
> > > "The price of reliability is the pursuit of the utmost simplicity.
> > > It is a price which the very rich find the most hard to pay."
> > > (Sir
> > > Antony Hoare, 1980)
> > >
> >
>
>
> --
> ddbrodsky@gmail.com
>
> "The price of reliability is the pursuit of the utmost simplicity.
> It is a price which the very rich find the most hard to pay."
> (Sir
> Antony Hoare, 1980)
>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Dima Brodsky <dd...@gmail.com>.
Thanks Luke,
I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
ttyl
Dima
On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
> Hi Dima,
>
> > I assume that unclean leader election exists as a static config in kafka
> kraft, but is still unavailable dynamically? Or is it unavailable in
> general?
>
> Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in
> KRaft will work, either static config or dynamic config.
> It's just the behavior changes when dynamically enabling the
> `unclean.leader.election.enable`, which will not trigger unclean leader
> election immediately after config change. The detailed behavior change is
> as described above.
>
> Let me know if it's still not clear.
>
> Thanks.
> Luke
>
>
> On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com> wrote:
>
> > Hi Luke,
> >
> > I assume that unclean leader election exists as a static config in kafka
> > kraft, but is still unavailable dynamically? Or is it unavailable in
> > general?
> >
> > Thanks!
> > ttyl
> > Dima
> >
> >
> > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > In v3.9.0, we're planning to support the
> `unclean.leader.election.enable`
> > > config in KRaft. Currently, the implementation PR
> > > <https://github.com/apache/kafka/pull/16866> is under review now. But
> > I'd
> > > like to raise the discussion to let you know there will be one behavior
> > > change when enabling `unclean.leader.election.enable` config
> dynamically
> > in
> > > KRaft.
> > >
> > > scenario:
> > > 1. broker [0, 1] config are all set
> > `unclean.leader.election.enable=false`.
> > > 2. topic A is created with 1 partition, 2 replication factors and
> > > `unclean.leader.election.enable=false`. Leader of the partition 0 is 0.
> > ISR
> > > is [0].
> > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> > > 4. dynamically alter config using admin API to set
> > > `unclean.leader.election.enable=true`, no matter it changes to the
> topic
> > A,
> > > or to broker 1, or default broker.
> > >
> > > In ZK mode, with this scenario, it'll trigger unclean leader election
> > > immediately, and broker 1 will be elected as the leader, which brings
> the
> > > availability of the partition back.
> > > In KRaft mode, with this scenario, the unclean leader election will be
> > > triggered in an unclean election thread, which is scheduled every 5
> mins
> > by
> > > default. That is, this partition might still be unavailable even if it
> > > enables `unclean.leader.election.enable`. If users want to elect a
> leader
> > > for this partition, run `kafka-leader-election.sh` with `unclean`
> option
> > to
> > > trigger the unclean leader election immediately.
> > >
> > > The main reason we didn't implement what ZK did is because of this
> > > complication
> > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
> > > faced.
> > >
> > > Other behaviors keep the same. Ex: When the topic config, or broker
> > config,
> > > or default broker config set `unclean.leader.election.enable=true`, and
> > > later, a partition has leader down with empty ISR [ ], and having some
> > > alive non-ISR replicas, in this case, the unclean leader election will
> be
> > > triggered, as long as the config was set earlier.
> > >
> > > But since the PR is still under review, any comments or thoughts are
> > > welcomed.
> > >
> > > Thank you.
> > > Luke
> > >
> >
> >
> > --
> > ddbrodsky@gmail.com
> >
> > "The price of reliability is the pursuit of the utmost simplicity.
> > It is a price which the very rich find the most hard to pay."
> > (Sir
> > Antony Hoare, 1980)
> >
>
--
ddbrodsky@gmail.com
"The price of reliability is the pursuit of the utmost simplicity.
It is a price which the very rich find the most hard to pay."
(Sir
Antony Hoare, 1980)
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Dima Brodsky <dd...@gmail.com>.
Thanks Luke,
I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example.
ttyl
Dima
On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <sh...@gmail.com> wrote:
> Hi Dima,
>
> > I assume that unclean leader election exists as a static config in kafka
> kraft, but is still unavailable dynamically? Or is it unavailable in
> general?
>
> Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in
> KRaft will work, either static config or dynamic config.
> It's just the behavior changes when dynamically enabling the
> `unclean.leader.election.enable`, which will not trigger unclean leader
> election immediately after config change. The detailed behavior change is
> as described above.
>
> Let me know if it's still not clear.
>
> Thanks.
> Luke
>
>
> On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com> wrote:
>
> > Hi Luke,
> >
> > I assume that unclean leader election exists as a static config in kafka
> > kraft, but is still unavailable dynamically? Or is it unavailable in
> > general?
> >
> > Thanks!
> > ttyl
> > Dima
> >
> >
> > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > In v3.9.0, we're planning to support the
> `unclean.leader.election.enable`
> > > config in KRaft. Currently, the implementation PR
> > > <https://github.com/apache/kafka/pull/16866> is under review now. But
> > I'd
> > > like to raise the discussion to let you know there will be one behavior
> > > change when enabling `unclean.leader.election.enable` config
> dynamically
> > in
> > > KRaft.
> > >
> > > scenario:
> > > 1. broker [0, 1] config are all set
> > `unclean.leader.election.enable=false`.
> > > 2. topic A is created with 1 partition, 2 replication factors and
> > > `unclean.leader.election.enable=false`. Leader of the partition 0 is 0.
> > ISR
> > > is [0].
> > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> > > 4. dynamically alter config using admin API to set
> > > `unclean.leader.election.enable=true`, no matter it changes to the
> topic
> > A,
> > > or to broker 1, or default broker.
> > >
> > > In ZK mode, with this scenario, it'll trigger unclean leader election
> > > immediately, and broker 1 will be elected as the leader, which brings
> the
> > > availability of the partition back.
> > > In KRaft mode, with this scenario, the unclean leader election will be
> > > triggered in an unclean election thread, which is scheduled every 5
> mins
> > by
> > > default. That is, this partition might still be unavailable even if it
> > > enables `unclean.leader.election.enable`. If users want to elect a
> leader
> > > for this partition, run `kafka-leader-election.sh` with `unclean`
> option
> > to
> > > trigger the unclean leader election immediately.
> > >
> > > The main reason we didn't implement what ZK did is because of this
> > > complication
> > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
> > > faced.
> > >
> > > Other behaviors keep the same. Ex: When the topic config, or broker
> > config,
> > > or default broker config set `unclean.leader.election.enable=true`, and
> > > later, a partition has leader down with empty ISR [ ], and having some
> > > alive non-ISR replicas, in this case, the unclean leader election will
> be
> > > triggered, as long as the config was set earlier.
> > >
> > > But since the PR is still under review, any comments or thoughts are
> > > welcomed.
> > >
> > > Thank you.
> > > Luke
> > >
> >
> >
> > --
> > ddbrodsky@gmail.com
> >
> > "The price of reliability is the pursuit of the utmost simplicity.
> > It is a price which the very rich find the most hard to pay."
> > (Sir
> > Antony Hoare, 1980)
> >
>
--
ddbrodsky@gmail.com
"The price of reliability is the pursuit of the utmost simplicity.
It is a price which the very rich find the most hard to pay."
(Sir
Antony Hoare, 1980)
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Luke Chen <sh...@gmail.com>.
Hi Dima,
> I assume that unclean leader election exists as a static config in kafka
kraft, but is still unavailable dynamically? Or is it unavailable in
general?
Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in
KRaft will work, either static config or dynamic config.
It's just the behavior changes when dynamically enabling the
`unclean.leader.election.enable`, which will not trigger unclean leader
election immediately after config change. The detailed behavior change is
as described above.
Let me know if it's still not clear.
Thanks.
Luke
On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com> wrote:
> Hi Luke,
>
> I assume that unclean leader election exists as a static config in kafka
> kraft, but is still unavailable dynamically? Or is it unavailable in
> general?
>
> Thanks!
> ttyl
> Dima
>
>
> On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
>
> > Hi all,
> >
> > In v3.9.0, we're planning to support the `unclean.leader.election.enable`
> > config in KRaft. Currently, the implementation PR
> > <https://github.com/apache/kafka/pull/16866> is under review now. But
> I'd
> > like to raise the discussion to let you know there will be one behavior
> > change when enabling `unclean.leader.election.enable` config dynamically
> in
> > KRaft.
> >
> > scenario:
> > 1. broker [0, 1] config are all set
> `unclean.leader.election.enable=false`.
> > 2. topic A is created with 1 partition, 2 replication factors and
> > `unclean.leader.election.enable=false`. Leader of the partition 0 is 0.
> ISR
> > is [0].
> > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> > 4. dynamically alter config using admin API to set
> > `unclean.leader.election.enable=true`, no matter it changes to the topic
> A,
> > or to broker 1, or default broker.
> >
> > In ZK mode, with this scenario, it'll trigger unclean leader election
> > immediately, and broker 1 will be elected as the leader, which brings the
> > availability of the partition back.
> > In KRaft mode, with this scenario, the unclean leader election will be
> > triggered in an unclean election thread, which is scheduled every 5 mins
> by
> > default. That is, this partition might still be unavailable even if it
> > enables `unclean.leader.election.enable`. If users want to elect a leader
> > for this partition, run `kafka-leader-election.sh` with `unclean` option
> to
> > trigger the unclean leader election immediately.
> >
> > The main reason we didn't implement what ZK did is because of this
> > complication
> > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
> > faced.
> >
> > Other behaviors keep the same. Ex: When the topic config, or broker
> config,
> > or default broker config set `unclean.leader.election.enable=true`, and
> > later, a partition has leader down with empty ISR [ ], and having some
> > alive non-ISR replicas, in this case, the unclean leader election will be
> > triggered, as long as the config was set earlier.
> >
> > But since the PR is still under review, any comments or thoughts are
> > welcomed.
> >
> > Thank you.
> > Luke
> >
>
>
> --
> ddbrodsky@gmail.com
>
> "The price of reliability is the pursuit of the utmost simplicity.
> It is a price which the very rich find the most hard to pay."
> (Sir
> Antony Hoare, 1980)
>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Luke Chen <sh...@gmail.com>.
Hi Dima,
> I assume that unclean leader election exists as a static config in kafka
kraft, but is still unavailable dynamically? Or is it unavailable in
general?
Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in
KRaft will work, either static config or dynamic config.
It's just the behavior changes when dynamically enabling the
`unclean.leader.election.enable`, which will not trigger unclean leader
election immediately after config change. The detailed behavior change is
as described above.
Let me know if it's still not clear.
Thanks.
Luke
On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <dd...@gmail.com> wrote:
> Hi Luke,
>
> I assume that unclean leader election exists as a static config in kafka
> kraft, but is still unavailable dynamically? Or is it unavailable in
> general?
>
> Thanks!
> ttyl
> Dima
>
>
> On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
>
> > Hi all,
> >
> > In v3.9.0, we're planning to support the `unclean.leader.election.enable`
> > config in KRaft. Currently, the implementation PR
> > <https://github.com/apache/kafka/pull/16866> is under review now. But
> I'd
> > like to raise the discussion to let you know there will be one behavior
> > change when enabling `unclean.leader.election.enable` config dynamically
> in
> > KRaft.
> >
> > scenario:
> > 1. broker [0, 1] config are all set
> `unclean.leader.election.enable=false`.
> > 2. topic A is created with 1 partition, 2 replication factors and
> > `unclean.leader.election.enable=false`. Leader of the partition 0 is 0.
> ISR
> > is [0].
> > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> > 4. dynamically alter config using admin API to set
> > `unclean.leader.election.enable=true`, no matter it changes to the topic
> A,
> > or to broker 1, or default broker.
> >
> > In ZK mode, with this scenario, it'll trigger unclean leader election
> > immediately, and broker 1 will be elected as the leader, which brings the
> > availability of the partition back.
> > In KRaft mode, with this scenario, the unclean leader election will be
> > triggered in an unclean election thread, which is scheduled every 5 mins
> by
> > default. That is, this partition might still be unavailable even if it
> > enables `unclean.leader.election.enable`. If users want to elect a leader
> > for this partition, run `kafka-leader-election.sh` with `unclean` option
> to
> > trigger the unclean leader election immediately.
> >
> > The main reason we didn't implement what ZK did is because of this
> > complication
> > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
> > faced.
> >
> > Other behaviors keep the same. Ex: When the topic config, or broker
> config,
> > or default broker config set `unclean.leader.election.enable=true`, and
> > later, a partition has leader down with empty ISR [ ], and having some
> > alive non-ISR replicas, in this case, the unclean leader election will be
> > triggered, as long as the config was set earlier.
> >
> > But since the PR is still under review, any comments or thoughts are
> > welcomed.
> >
> > Thank you.
> > Luke
> >
>
>
> --
> ddbrodsky@gmail.com
>
> "The price of reliability is the pursuit of the utmost simplicity.
> It is a price which the very rich find the most hard to pay."
> (Sir
> Antony Hoare, 1980)
>
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Dima Brodsky <dd...@gmail.com>.
Hi Luke,
I assume that unclean leader election exists as a static config in kafka
kraft, but is still unavailable dynamically? Or is it unavailable in
general?
Thanks!
ttyl
Dima
On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> Hi all,
>
> In v3.9.0, we're planning to support the `unclean.leader.election.enable`
> config in KRaft. Currently, the implementation PR
> <https://github.com/apache/kafka/pull/16866> is under review now. But I'd
> like to raise the discussion to let you know there will be one behavior
> change when enabling `unclean.leader.election.enable` config dynamically in
> KRaft.
>
> scenario:
> 1. broker [0, 1] config are all set `unclean.leader.election.enable=false`.
> 2. topic A is created with 1 partition, 2 replication factors and
> `unclean.leader.election.enable=false`. Leader of the partition 0 is 0. ISR
> is [0].
> 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> 4. dynamically alter config using admin API to set
> `unclean.leader.election.enable=true`, no matter it changes to the topic A,
> or to broker 1, or default broker.
>
> In ZK mode, with this scenario, it'll trigger unclean leader election
> immediately, and broker 1 will be elected as the leader, which brings the
> availability of the partition back.
> In KRaft mode, with this scenario, the unclean leader election will be
> triggered in an unclean election thread, which is scheduled every 5 mins by
> default. That is, this partition might still be unavailable even if it
> enables `unclean.leader.election.enable`. If users want to elect a leader
> for this partition, run `kafka-leader-election.sh` with `unclean` option to
> trigger the unclean leader election immediately.
>
> The main reason we didn't implement what ZK did is because of this
> complication
> <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
> faced.
>
> Other behaviors keep the same. Ex: When the topic config, or broker config,
> or default broker config set `unclean.leader.election.enable=true`, and
> later, a partition has leader down with empty ISR [ ], and having some
> alive non-ISR replicas, in this case, the unclean leader election will be
> triggered, as long as the config was set earlier.
>
> But since the PR is still under review, any comments or thoughts are
> welcomed.
>
> Thank you.
> Luke
>
--
ddbrodsky@gmail.com
"The price of reliability is the pursuit of the utmost simplicity.
It is a price which the very rich find the most hard to pay."
(Sir
Antony Hoare, 1980)
Re: [DISCUSS] `unclean.leader.election.enable` in KRaft behavior change
Posted by Dima Brodsky <dd...@gmail.com>.
Hi Luke,
I assume that unclean leader election exists as a static config in kafka
kraft, but is still unavailable dynamically? Or is it unavailable in
general?
Thanks!
ttyl
Dima
On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <sh...@gmail.com> wrote:
> Hi all,
>
> In v3.9.0, we're planning to support the `unclean.leader.election.enable`
> config in KRaft. Currently, the implementation PR
> <https://github.com/apache/kafka/pull/16866> is under review now. But I'd
> like to raise the discussion to let you know there will be one behavior
> change when enabling `unclean.leader.election.enable` config dynamically in
> KRaft.
>
> scenario:
> 1. broker [0, 1] config are all set `unclean.leader.election.enable=false`.
> 2. topic A is created with 1 partition, 2 replication factors and
> `unclean.leader.election.enable=false`. Leader of the partition 0 is 0. ISR
> is [0].
> 3. broker 0 is down, which leaves ISR to empty [ ], and leader none.
> 4. dynamically alter config using admin API to set
> `unclean.leader.election.enable=true`, no matter it changes to the topic A,
> or to broker 1, or default broker.
>
> In ZK mode, with this scenario, it'll trigger unclean leader election
> immediately, and broker 1 will be elected as the leader, which brings the
> availability of the partition back.
> In KRaft mode, with this scenario, the unclean leader election will be
> triggered in an unclean election thread, which is scheduled every 5 mins by
> default. That is, this partition might still be unavailable even if it
> enables `unclean.leader.election.enable`. If users want to elect a leader
> for this partition, run `kafka-leader-election.sh` with `unclean` option to
> trigger the unclean leader election immediately.
>
> The main reason we didn't implement what ZK did is because of this
> complication
> <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> we
> faced.
>
> Other behaviors keep the same. Ex: When the topic config, or broker config,
> or default broker config set `unclean.leader.election.enable=true`, and
> later, a partition has leader down with empty ISR [ ], and having some
> alive non-ISR replicas, in this case, the unclean leader election will be
> triggered, as long as the config was set earlier.
>
> But since the PR is still under review, any comments or thoughts are
> welcomed.
>
> Thank you.
> Luke
>
--
ddbrodsky@gmail.com
"The price of reliability is the pursuit of the utmost simplicity.
It is a price which the very rich find the most hard to pay."
(Sir
Antony Hoare, 1980)