You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Mayank Shekhar Narula <ma...@gmail.com> on 2023/09/27 11:38:08 UTC

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Reviving this thread, as the discussion thread has been updated.

On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
mayanks.narula@gmail.com> wrote:

> Thanks Jose.
>
> On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> <js...@confluent.io.invalid> wrote:
>
>> The KIP LGTM. Thanks for the design. I am looking forward to the
>> implementation.
>>
>> +1 (binding).
>>
>> Thanks!
>> --
>> -José
>>
>
>
> --
> Regards,
> Mayank Shekhar Narula
>


-- 
Regards,
Mayank Shekhar Narula

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by Mayank Shekhar Narula <ma...@gmail.com>.
David

Thanks, i have shared it here
https://lists.apache.org/thread/0fd0og8j6gj9b2wtppnpy7fz8pzzv20r

On Tue, Feb 6, 2024 at 4:13 PM David Jacot <dj...@confluent.io.invalid>
wrote:

> Hi,
>
> Thanks for bringing this up. It may be worth bringing it in the 3.7 release
> thread too as it may qualify as a blocker for the release.
>
> Best,
> David
>
> On Tue, Feb 6, 2024 at 5:09 PM Mayank Shekhar Narula <
> mayanks.narula@gmail.com> wrote:
>
> > Hi Folks
> >
> > KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered
> > in 3.6.1, to skip backoff period for a produce batch being retried to new
> > leader i.e. KAFKA-15415.
> >
> > KAFKA-15415 current implementation introduced a performance regression,
> by
> > increasing synchronization on the produce path, especially for high
> > partition counts. The description section of
> > https://issues.apache.org/jira/browse/KAFKA-16226 goes more into details
> > of
> > the regression.
> >
> > I have put up a fix https://github.com/apache/kafka/pull/15323, which
> > removes this synchronization. The fix adds a new public method to
> > Cluster.java, and a public constructor to PartitionInfo.java. Let me know
> > your thoughts.
> >
> > On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula <
> > mayanks.narula@gmail.com> wrote:
> >
> > > Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason),
> > and
> > > 1 non-binding vote(Kirk).
> > >
> > > With the current status of voting, KIP is accepted.
> > >
> > > Thanks again to all reviewers and voters.
> > >
> > >
> > >
> > > On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula <
> > > mayanks.narula@gmail.com> wrote:
> > >
> > >> Thank you all for your votes, Jun, David, and Jason!
> > >>
> > >> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson
> > >> <ja...@confluent.io.invalid> wrote:
> > >>
> > >>> +1 Thanks for the KIP
> > >>>
> > >>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot <da...@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > Thanks for the KIP. +1 from me as well.
> > >>> >
> > >>> > Best,
> > >>> > David
> > >>> >
> > >>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a
> > >>> écrit :
> > >>> >
> > >>> > > Hi, Mayank,
> > >>> > >
> > >>> > > Thanks for the detailed explanation in the KIP. +1 from me.
> > >>> > >
> > >>> > > Jun
> > >>> > >
> > >>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
> > >>> > > mayanks.narula@gmail.com> wrote:
> > >>> > >
> > >>> > > > Reviving this thread, as the discussion thread has been
> updated.
> > >>> > > >
> > >>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
> > >>> > > > mayanks.narula@gmail.com> wrote:
> > >>> > > >
> > >>> > > > > Thanks Jose.
> > >>> > > > >
> > >>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> > >>> > > > > <js...@confluent.io.invalid> wrote:
> > >>> > > > >
> > >>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to
> > the
> > >>> > > > >> implementation.
> > >>> > > > >>
> > >>> > > > >> +1 (binding).
> > >>> > > > >>
> > >>> > > > >> Thanks!
> > >>> > > > >> --
> > >>> > > > >> -José
> > >>> > > > >>
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Regards,
> > >>> > > > > Mayank Shekhar Narula
> > >>> > > > >
> > >>> > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > Regards,
> > >>> > > > Mayank Shekhar Narula
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >> --
> > >> Regards,
> > >> Mayank Shekhar Narula
> > >>
> > >
> > >
> > > --
> > > Regards,
> > > Mayank Shekhar Narula
> > >
> >
> >
> > --
> > Regards,
> > Mayank Shekhar Narula
> >
>


-- 
Regards,
Mayank Shekhar Narula

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by David Jacot <dj...@confluent.io.INVALID>.
Hi,

Thanks for bringing this up. It may be worth bringing it in the 3.7 release
thread too as it may qualify as a blocker for the release.

Best,
David

On Tue, Feb 6, 2024 at 5:09 PM Mayank Shekhar Narula <
mayanks.narula@gmail.com> wrote:

> Hi Folks
>
> KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered
> in 3.6.1, to skip backoff period for a produce batch being retried to new
> leader i.e. KAFKA-15415.
>
> KAFKA-15415 current implementation introduced a performance regression, by
> increasing synchronization on the produce path, especially for high
> partition counts. The description section of
> https://issues.apache.org/jira/browse/KAFKA-16226 goes more into details
> of
> the regression.
>
> I have put up a fix https://github.com/apache/kafka/pull/15323, which
> removes this synchronization. The fix adds a new public method to
> Cluster.java, and a public constructor to PartitionInfo.java. Let me know
> your thoughts.
>
> On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula <
> mayanks.narula@gmail.com> wrote:
>
> > Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason),
> and
> > 1 non-binding vote(Kirk).
> >
> > With the current status of voting, KIP is accepted.
> >
> > Thanks again to all reviewers and voters.
> >
> >
> >
> > On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula <
> > mayanks.narula@gmail.com> wrote:
> >
> >> Thank you all for your votes, Jun, David, and Jason!
> >>
> >> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson
> >> <ja...@confluent.io.invalid> wrote:
> >>
> >>> +1 Thanks for the KIP
> >>>
> >>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot <da...@gmail.com>
> >>> wrote:
> >>>
> >>> > Thanks for the KIP. +1 from me as well.
> >>> >
> >>> > Best,
> >>> > David
> >>> >
> >>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a
> >>> écrit :
> >>> >
> >>> > > Hi, Mayank,
> >>> > >
> >>> > > Thanks for the detailed explanation in the KIP. +1 from me.
> >>> > >
> >>> > > Jun
> >>> > >
> >>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
> >>> > > mayanks.narula@gmail.com> wrote:
> >>> > >
> >>> > > > Reviving this thread, as the discussion thread has been updated.
> >>> > > >
> >>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
> >>> > > > mayanks.narula@gmail.com> wrote:
> >>> > > >
> >>> > > > > Thanks Jose.
> >>> > > > >
> >>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> >>> > > > > <js...@confluent.io.invalid> wrote:
> >>> > > > >
> >>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to
> the
> >>> > > > >> implementation.
> >>> > > > >>
> >>> > > > >> +1 (binding).
> >>> > > > >>
> >>> > > > >> Thanks!
> >>> > > > >> --
> >>> > > > >> -José
> >>> > > > >>
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Regards,
> >>> > > > > Mayank Shekhar Narula
> >>> > > > >
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > Regards,
> >>> > > > Mayank Shekhar Narula
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >> --
> >> Regards,
> >> Mayank Shekhar Narula
> >>
> >
> >
> > --
> > Regards,
> > Mayank Shekhar Narula
> >
>
>
> --
> Regards,
> Mayank Shekhar Narula
>

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by Mayank Shekhar Narula <ma...@gmail.com>.
Hi Folks

KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered
in 3.6.1, to skip backoff period for a produce batch being retried to new
leader i.e. KAFKA-15415.

KAFKA-15415 current implementation introduced a performance regression, by
increasing synchronization on the produce path, especially for high
partition counts. The description section of
https://issues.apache.org/jira/browse/KAFKA-16226 goes more into details of
the regression.

I have put up a fix https://github.com/apache/kafka/pull/15323, which
removes this synchronization. The fix adds a new public method to
Cluster.java, and a public constructor to PartitionInfo.java. Let me know
your thoughts.

On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula <
mayanks.narula@gmail.com> wrote:

> Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), and
> 1 non-binding vote(Kirk).
>
> With the current status of voting, KIP is accepted.
>
> Thanks again to all reviewers and voters.
>
>
>
> On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula <
> mayanks.narula@gmail.com> wrote:
>
>> Thank you all for your votes, Jun, David, and Jason!
>>
>> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson
>> <ja...@confluent.io.invalid> wrote:
>>
>>> +1 Thanks for the KIP
>>>
>>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot <da...@gmail.com>
>>> wrote:
>>>
>>> > Thanks for the KIP. +1 from me as well.
>>> >
>>> > Best,
>>> > David
>>> >
>>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a
>>> écrit :
>>> >
>>> > > Hi, Mayank,
>>> > >
>>> > > Thanks for the detailed explanation in the KIP. +1 from me.
>>> > >
>>> > > Jun
>>> > >
>>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
>>> > > mayanks.narula@gmail.com> wrote:
>>> > >
>>> > > > Reviving this thread, as the discussion thread has been updated.
>>> > > >
>>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
>>> > > > mayanks.narula@gmail.com> wrote:
>>> > > >
>>> > > > > Thanks Jose.
>>> > > > >
>>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
>>> > > > > <js...@confluent.io.invalid> wrote:
>>> > > > >
>>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the
>>> > > > >> implementation.
>>> > > > >>
>>> > > > >> +1 (binding).
>>> > > > >>
>>> > > > >> Thanks!
>>> > > > >> --
>>> > > > >> -José
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Regards,
>>> > > > > Mayank Shekhar Narula
>>> > > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > > Regards,
>>> > > > Mayank Shekhar Narula
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>> --
>> Regards,
>> Mayank Shekhar Narula
>>
>
>
> --
> Regards,
> Mayank Shekhar Narula
>


-- 
Regards,
Mayank Shekhar Narula

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by Mayank Shekhar Narula <ma...@gmail.com>.
Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), and
1 non-binding vote(Kirk).

With the current status of voting, KIP is accepted.

Thanks again to all reviewers and voters.



On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula <
mayanks.narula@gmail.com> wrote:

> Thank you all for your votes, Jun, David, and Jason!
>
> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson <ja...@confluent.io.invalid>
> wrote:
>
>> +1 Thanks for the KIP
>>
>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot <da...@gmail.com>
>> wrote:
>>
>> > Thanks for the KIP. +1 from me as well.
>> >
>> > Best,
>> > David
>> >
>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a
>> écrit :
>> >
>> > > Hi, Mayank,
>> > >
>> > > Thanks for the detailed explanation in the KIP. +1 from me.
>> > >
>> > > Jun
>> > >
>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
>> > > mayanks.narula@gmail.com> wrote:
>> > >
>> > > > Reviving this thread, as the discussion thread has been updated.
>> > > >
>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
>> > > > mayanks.narula@gmail.com> wrote:
>> > > >
>> > > > > Thanks Jose.
>> > > > >
>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
>> > > > > <js...@confluent.io.invalid> wrote:
>> > > > >
>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the
>> > > > >> implementation.
>> > > > >>
>> > > > >> +1 (binding).
>> > > > >>
>> > > > >> Thanks!
>> > > > >> --
>> > > > >> -José
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Regards,
>> > > > > Mayank Shekhar Narula
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > > Mayank Shekhar Narula
>> > > >
>> > >
>> >
>>
>
>
> --
> Regards,
> Mayank Shekhar Narula
>


-- 
Regards,
Mayank Shekhar Narula

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by Mayank Shekhar Narula <ma...@gmail.com>.
Thank you all for your votes, Jun, David, and Jason!

On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson <ja...@confluent.io.invalid>
wrote:

> +1 Thanks for the KIP
>
> On Tue, Oct 3, 2023 at 12:30 PM David Jacot <da...@gmail.com> wrote:
>
> > Thanks for the KIP. +1 from me as well.
> >
> > Best,
> > David
> >
> > Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a écrit
> :
> >
> > > Hi, Mayank,
> > >
> > > Thanks for the detailed explanation in the KIP. +1 from me.
> > >
> > > Jun
> > >
> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
> > > mayanks.narula@gmail.com> wrote:
> > >
> > > > Reviving this thread, as the discussion thread has been updated.
> > > >
> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
> > > > mayanks.narula@gmail.com> wrote:
> > > >
> > > > > Thanks Jose.
> > > > >
> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> > > > > <js...@confluent.io.invalid> wrote:
> > > > >
> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the
> > > > >> implementation.
> > > > >>
> > > > >> +1 (binding).
> > > > >>
> > > > >> Thanks!
> > > > >> --
> > > > >> -José
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Mayank Shekhar Narula
> > > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Mayank Shekhar Narula
> > > >
> > >
> >
>


-- 
Regards,
Mayank Shekhar Narula

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by Jason Gustafson <ja...@confluent.io.INVALID>.
+1 Thanks for the KIP

On Tue, Oct 3, 2023 at 12:30 PM David Jacot <da...@gmail.com> wrote:

> Thanks for the KIP. +1 from me as well.
>
> Best,
> David
>
> Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a écrit :
>
> > Hi, Mayank,
> >
> > Thanks for the detailed explanation in the KIP. +1 from me.
> >
> > Jun
> >
> > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
> > mayanks.narula@gmail.com> wrote:
> >
> > > Reviving this thread, as the discussion thread has been updated.
> > >
> > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
> > > mayanks.narula@gmail.com> wrote:
> > >
> > > > Thanks Jose.
> > > >
> > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> > > > <js...@confluent.io.invalid> wrote:
> > > >
> > > >> The KIP LGTM. Thanks for the design. I am looking forward to the
> > > >> implementation.
> > > >>
> > > >> +1 (binding).
> > > >>
> > > >> Thanks!
> > > >> --
> > > >> -José
> > > >>
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Mayank Shekhar Narula
> > > >
> > >
> > >
> > > --
> > > Regards,
> > > Mayank Shekhar Narula
> > >
> >
>

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by David Jacot <da...@gmail.com>.
Thanks for the KIP. +1 from me as well.

Best,
David

Le mar. 3 oct. 2023 à 20:54, Jun Rao <ju...@confluent.io.invalid> a écrit :

> Hi, Mayank,
>
> Thanks for the detailed explanation in the KIP. +1 from me.
>
> Jun
>
> On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
> mayanks.narula@gmail.com> wrote:
>
> > Reviving this thread, as the discussion thread has been updated.
> >
> > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
> > mayanks.narula@gmail.com> wrote:
> >
> > > Thanks Jose.
> > >
> > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> > > <js...@confluent.io.invalid> wrote:
> > >
> > >> The KIP LGTM. Thanks for the design. I am looking forward to the
> > >> implementation.
> > >>
> > >> +1 (binding).
> > >>
> > >> Thanks!
> > >> --
> > >> -José
> > >>
> > >
> > >
> > > --
> > > Regards,
> > > Mayank Shekhar Narula
> > >
> >
> >
> > --
> > Regards,
> > Mayank Shekhar Narula
> >
>

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, Mayank,

Thanks for the detailed explanation in the KIP. +1 from me.

Jun

On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula <
mayanks.narula@gmail.com> wrote:

> Reviving this thread, as the discussion thread has been updated.
>
> On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula <
> mayanks.narula@gmail.com> wrote:
>
> > Thanks Jose.
> >
> > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio
> > <js...@confluent.io.invalid> wrote:
> >
> >> The KIP LGTM. Thanks for the design. I am looking forward to the
> >> implementation.
> >>
> >> +1 (binding).
> >>
> >> Thanks!
> >> --
> >> -José
> >>
> >
> >
> > --
> > Regards,
> > Mayank Shekhar Narula
> >
>
>
> --
> Regards,
> Mayank Shekhar Narula
>