You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "M. Manna" <ma...@gmail.com> on 2019/05/17 21:18:37 UTC
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Apologies for the delay. As per the original thread, there have been 3
binding votes.
I will be closing this and update the confluence page with the results.
Also, I will be submitting the PR soon.
Regards,
On Fri, 5 Apr 2019 at 00:18, M. Manna <ma...@gmail.com> wrote:
> Thanks Harsha.
>
> As per your comments, I have counted 3 binding votes so far.
>
> Thanks everyone for your comments and support. I’ll update the kip next
> morning and do the needful.
>
> Regards,
>
> On Thu, 4 Apr 2019 at 22:10, Harsha <ka...@harsha.io> wrote:
>
>> Looks like the KIP is passed with 3 binding votes. From Matthias, Bill
>> Bejeck and myself you got 3 binding votes.
>> You can do the full tally of the votes and send out a close of vote
>> thread.
>>
>> Thanks,
>> Harsha
>>
>> On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote:
>> > Hello,
>> >
>> > Trying to revive this thread again. Would anyone be interested in having
>> > this KiP through
>> >
>> >
>> > Thanks,
>> >
>> > On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
>> >
>> > > Hello,
>> > >
>> > > I am trying to revive this thread. I only got 1 binding vote so far.
>> > >
>> > > Please feel free to revisit and comment here.
>> > >
>> > > Thanks,
>> > >
>> > > On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
>> > >
>> > >> Hey IJ,
>> > >>
>> > >> Thanks for your interest in the KIP.
>> > >>
>> > >> My point was simply that the round-robin should happen even if the
>> key is
>> > >> not null. As for the importance of key in our case, we treat the key
>> as
>> > >> metadata. Each key is composed of certain info which are parsed by
>> our
>> > >> consumer thread. We will then determine whether it's an actionable
>> message
>> > >> (e.g. process it), or a loopback(ignore it). You could argue, "Why
>> not
>> > >> append this metadata with the record and parse it there?". But that
>> means
>> > >> the following:
>> > >>
>> > >> 1) I'm always passing null key to achieve this - I would like to pass
>> > >> Null/Not-Null/Other key i.e. flexibility
>> > >> 2) Suppose the message size is 99 KB and and max message bytes
>> allowed is
>> > >> 100K. Now prefixing metadata with message results into the actual
>> message
>> > >> being 101K. This will fail at producer level and cause a retry/log
>> this in
>> > >> our DB for future pickup.
>> > >>
>> > >> To avoid all these, we are simply proposing this new partitioner
>> class.
>> > >> but all Kafka new releases will still have DefaultPartitioner as
>> default,
>> > >> unless they change the prop file to use our new class.
>> > >>
>> > >> Regards,
>> > >>
>> > >> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
>> > >>
>> > >>> Thanks for the KIP. Can you please elaborate on the need for the
>> key in
>> > >>> this case? The KIP simply states that the key is needed for
>> metadata, but
>> > >>> doesn't give any more details.
>> > >>>
>> > >>> Ismael
>> > >>>
>> > >>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>> > >>>
>> > >>> > Hello,
>> > >>> >
>> > >>> > I have made necessary changes as per the original discussion
>> thread,
>> > >>> and
>> > >>> > would like to put it for votes.
>> > >>> >
>> > >>> > Thank you very much for your suggestion and guidance so far.
>> > >>> >
>> > >>> > Regards,
>> > >>> >
>> > >>>
>> > >>
>> >
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
I have now submitted the PR for review. Thanks Matthias for pointing out
that KAFKA-3333 was raised to address the same.
https://github.com/apache/kafka/pull/6771
if someone reviews after Jenkins build is complete, I would appreciate it.
Thanks,
On Fri, 17 May 2019 at 22:18, M. Manna <ma...@gmail.com> wrote:
> Apologies for the delay. As per the original thread, there have been 3
> binding votes.
>
> I will be closing this and update the confluence page with the results.
> Also, I will be submitting the PR soon.
>
> Regards,
>
> On Fri, 5 Apr 2019 at 00:18, M. Manna <ma...@gmail.com> wrote:
>
>> Thanks Harsha.
>>
>> As per your comments, I have counted 3 binding votes so far.
>>
>> Thanks everyone for your comments and support. I’ll update the kip next
>> morning and do the needful.
>>
>> Regards,
>>
>> On Thu, 4 Apr 2019 at 22:10, Harsha <ka...@harsha.io> wrote:
>>
>>> Looks like the KIP is passed with 3 binding votes. From Matthias, Bill
>>> Bejeck and myself you got 3 binding votes.
>>> You can do the full tally of the votes and send out a close of vote
>>> thread.
>>>
>>> Thanks,
>>> Harsha
>>>
>>> On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote:
>>> > Hello,
>>> >
>>> > Trying to revive this thread again. Would anyone be interested in
>>> having
>>> > this KiP through
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
>>> >
>>> > > Hello,
>>> > >
>>> > > I am trying to revive this thread. I only got 1 binding vote so far.
>>> > >
>>> > > Please feel free to revisit and comment here.
>>> > >
>>> > > Thanks,
>>> > >
>>> > > On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
>>> > >
>>> > >> Hey IJ,
>>> > >>
>>> > >> Thanks for your interest in the KIP.
>>> > >>
>>> > >> My point was simply that the round-robin should happen even if the
>>> key is
>>> > >> not null. As for the importance of key in our case, we treat the
>>> key as
>>> > >> metadata. Each key is composed of certain info which are parsed by
>>> our
>>> > >> consumer thread. We will then determine whether it's an actionable
>>> message
>>> > >> (e.g. process it), or a loopback(ignore it). You could argue, "Why
>>> not
>>> > >> append this metadata with the record and parse it there?". But that
>>> means
>>> > >> the following:
>>> > >>
>>> > >> 1) I'm always passing null key to achieve this - I would like to
>>> pass
>>> > >> Null/Not-Null/Other key i.e. flexibility
>>> > >> 2) Suppose the message size is 99 KB and and max message bytes
>>> allowed is
>>> > >> 100K. Now prefixing metadata with message results into the actual
>>> message
>>> > >> being 101K. This will fail at producer level and cause a retry/log
>>> this in
>>> > >> our DB for future pickup.
>>> > >>
>>> > >> To avoid all these, we are simply proposing this new partitioner
>>> class.
>>> > >> but all Kafka new releases will still have DefaultPartitioner as
>>> default,
>>> > >> unless they change the prop file to use our new class.
>>> > >>
>>> > >> Regards,
>>> > >>
>>> > >> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk>
>>> wrote:
>>> > >>
>>> > >>> Thanks for the KIP. Can you please elaborate on the need for the
>>> key in
>>> > >>> this case? The KIP simply states that the key is needed for
>>> metadata, but
>>> > >>> doesn't give any more details.
>>> > >>>
>>> > >>> Ismael
>>> > >>>
>>> > >>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com>
>>> wrote:
>>> > >>>
>>> > >>> > Hello,
>>> > >>> >
>>> > >>> > I have made necessary changes as per the original discussion
>>> thread,
>>> > >>> and
>>> > >>> > would like to put it for votes.
>>> > >>> >
>>> > >>> > Thank you very much for your suggestion and guidance so far.
>>> > >>> >
>>> > >>> > Regards,
>>> > >>> >
>>> > >>>
>>> > >>
>>> >
>>>
>>