You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "M. Manna" <ma...@gmail.com> on 2018/09/04 10:39:18 UTC
[VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Hello,
I have made necessary changes as per the original discussion thread, and
would like to put it for votes.
Thank you very much for your suggestion and guidance so far.
Regards,
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by ha...@fastmail.fm.
Having Round-robin partitioner as default makes sense.
+1 (binding).
Thanks,
Harsha
On Oct 19, 2018, 9:54 AM -0700, Matthias J. Sax <ma...@confluent.io>, wrote:
> People a currently quite busy with 2.1.0 and 2.0.1 releases. That's why
> the vote might go slow.
>
> Just keep bumping the vote thread on a weekly bases... Thanks for your
> patients.
>
>
> -Matthias
>
> On 10/19/18 1:08 AM, M. Manna wrote:
> > Since this has gone quiet, could I prequest 1 more vote here - if anyone
> > thinks it's worth doing?
> >
> > Thanks,
> >
> >
> >
> > On Wed, 3 Oct 2018 at 16:14, M. Manna <ma...@gmail.com> wrote:
> >
> > > Thanks for reminding me about the "Binding" vote Bill. I remember some
> > > people with non-binding vote, so jumped the gun a bit too early.
> > > We will wait for 2 more as you stated.
> > >
> > > Regards,
> > >
> > > On Wed, 3 Oct 2018 at 16:07, Bill Bejeck <bb...@gmail.com> wrote:
> > >
> > > > +1 from me.
> > > >
> > > > As for closing the vote, it needs to be open for a minimum of 72 and
> > > > requires three binding +1 votes. Additionally, there needs more +1
> > > > binding
> > > > votes than -1 votes. The description for the lazy majority vote process
> > > > is
> > > > described here
> > > > https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals
> > > > .
> > > >
> > > > I haven't been tracking the vote results, but from what I can see in the
> > > > voting thread, this KIP still requires two more +1 binding votes.
> > > >
> > > > HTH,
> > > > BIll
> > > >
> > > > On Wed, Oct 3, 2018 at 8:58 AM M. Manna <ma...@gmail.com> wrote:
> > > >
> > > > > Since this has been open for a while, I am assuming that it's good to
> > > > go?
> > > > >
> > > > > if so, I will update the KIP page - and start coding this. I would
> > > > prefer
> > > > > re-using existing tests written for DefaultPartitioner, so that we don't
> > > > > need to write new tests.
> > > > >
> > > > > Regards,
> > > > >
> > > > > On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io>
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > @Abhimanyu: can you please update the table in
> > > > > >
> > > > > >
> > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > > > > and add a link to the KIP. Thanks.
> > > > > >
> > > > > > -Matthias
> > > > > >
> > > > > > On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
> > > > > > > +1
> > > > > > >
> > > > > > > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <
> > > > mageshn@confluent.io
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1 ( non-binding)
> > > > > > > >
> > > > > > > > On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com>
> > > > wrote:
> > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > I have made necessary changes as per the original discussion
> > > > thread,
> > > > > > and
> > > > > > > > > would like to put it for votes.
> > > > > > > > >
> > > > > > > > > Thank you very much for your suggestion and guidance so far.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "Matthias J. Sax" <ma...@confluent.io>.
People a currently quite busy with 2.1.0 and 2.0.1 releases. That's why
the vote might go slow.
Just keep bumping the vote thread on a weekly bases... Thanks for your
patients.
-Matthias
On 10/19/18 1:08 AM, M. Manna wrote:
> Since this has gone quiet, could I prequest 1 more vote here - if anyone
> thinks it's worth doing?
>
> Thanks,
>
>
>
> On Wed, 3 Oct 2018 at 16:14, M. Manna <ma...@gmail.com> wrote:
>
>> Thanks for reminding me about the "Binding" vote Bill. I remember some
>> people with non-binding vote, so jumped the gun a bit too early.
>> We will wait for 2 more as you stated.
>>
>> Regards,
>>
>> On Wed, 3 Oct 2018 at 16:07, Bill Bejeck <bb...@gmail.com> wrote:
>>
>>> +1 from me.
>>>
>>> As for closing the vote, it needs to be open for a minimum of 72 and
>>> requires three binding +1 votes. Additionally, there needs more +1
>>> binding
>>> votes than -1 votes. The description for the lazy majority vote process
>>> is
>>> described here
>>> https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals
>>> .
>>>
>>> I haven't been tracking the vote results, but from what I can see in the
>>> voting thread, this KIP still requires two more +1 binding votes.
>>>
>>> HTH,
>>> BIll
>>>
>>> On Wed, Oct 3, 2018 at 8:58 AM M. Manna <ma...@gmail.com> wrote:
>>>
>>>> Since this has been open for a while, I am assuming that it's good to
>>> go?
>>>>
>>>> if so, I will update the KIP page - and start coding this. I would
>>> prefer
>>>> re-using existing tests written for DefaultPartitioner, so that we don't
>>>> need to write new tests.
>>>>
>>>> Regards,
>>>>
>>>> On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io>
>>>> wrote:
>>>>
>>>>> +1 (binding)
>>>>>
>>>>> @Abhimanyu: can you please update the table in
>>>>>
>>>>>
>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>>>> and add a link to the KIP. Thanks.
>>>>>
>>>>> -Matthias
>>>>>
>>>>> On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
>>>>>> +1
>>>>>>
>>>>>> On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <
>>> mageshn@confluent.io
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 ( non-binding)
>>>>>>>
>>>>>>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com>
>>> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I have made necessary changes as per the original discussion
>>> thread,
>>>>> and
>>>>>>>> would like to put it for votes.
>>>>>>>>
>>>>>>>> Thank you very much for your suggestion and guidance so far.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Since this has gone quiet, could I prequest 1 more vote here - if anyone
thinks it's worth doing?
Thanks,
On Wed, 3 Oct 2018 at 16:14, M. Manna <ma...@gmail.com> wrote:
> Thanks for reminding me about the "Binding" vote Bill. I remember some
> people with non-binding vote, so jumped the gun a bit too early.
> We will wait for 2 more as you stated.
>
> Regards,
>
> On Wed, 3 Oct 2018 at 16:07, Bill Bejeck <bb...@gmail.com> wrote:
>
>> +1 from me.
>>
>> As for closing the vote, it needs to be open for a minimum of 72 and
>> requires three binding +1 votes. Additionally, there needs more +1
>> binding
>> votes than -1 votes. The description for the lazy majority vote process
>> is
>> described here
>> https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals
>> .
>>
>> I haven't been tracking the vote results, but from what I can see in the
>> voting thread, this KIP still requires two more +1 binding votes.
>>
>> HTH,
>> BIll
>>
>> On Wed, Oct 3, 2018 at 8:58 AM M. Manna <ma...@gmail.com> wrote:
>>
>> > Since this has been open for a while, I am assuming that it's good to
>> go?
>> >
>> > if so, I will update the KIP page - and start coding this. I would
>> prefer
>> > re-using existing tests written for DefaultPartitioner, so that we don't
>> > need to write new tests.
>> >
>> > Regards,
>> >
>> > On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io>
>> > wrote:
>> >
>> > > +1 (binding)
>> > >
>> > > @Abhimanyu: can you please update the table in
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > > and add a link to the KIP. Thanks.
>> > >
>> > > -Matthias
>> > >
>> > > On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
>> > > > +1
>> > > >
>> > > > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <
>> mageshn@confluent.io
>> > >
>> > > > wrote:
>> > > >
>> > > >> +1 ( non-binding)
>> > > >>
>> > > >> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com>
>> wrote:
>> > > >>
>> > > >>> Hello,
>> > > >>>
>> > > >>> I have made necessary changes as per the original discussion
>> thread,
>> > > and
>> > > >>> would like to put it for votes.
>> > > >>>
>> > > >>> Thank you very much for your suggestion and guidance so far.
>> > > >>>
>> > > >>> Regards,
>> > > >>>
>> > > >>
>> > > >
>> > >
>> > >
>> >
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Since this has gone quiet, could I prequest 1 more vote here - if anyone
thinks it's worth doing?
Thanks,
On Wed, 3 Oct 2018 at 16:14, M. Manna <ma...@gmail.com> wrote:
> Thanks for reminding me about the "Binding" vote Bill. I remember some
> people with non-binding vote, so jumped the gun a bit too early.
> We will wait for 2 more as you stated.
>
> Regards,
>
> On Wed, 3 Oct 2018 at 16:07, Bill Bejeck <bb...@gmail.com> wrote:
>
>> +1 from me.
>>
>> As for closing the vote, it needs to be open for a minimum of 72 and
>> requires three binding +1 votes. Additionally, there needs more +1
>> binding
>> votes than -1 votes. The description for the lazy majority vote process
>> is
>> described here
>> https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals
>> .
>>
>> I haven't been tracking the vote results, but from what I can see in the
>> voting thread, this KIP still requires two more +1 binding votes.
>>
>> HTH,
>> BIll
>>
>> On Wed, Oct 3, 2018 at 8:58 AM M. Manna <ma...@gmail.com> wrote:
>>
>> > Since this has been open for a while, I am assuming that it's good to
>> go?
>> >
>> > if so, I will update the KIP page - and start coding this. I would
>> prefer
>> > re-using existing tests written for DefaultPartitioner, so that we don't
>> > need to write new tests.
>> >
>> > Regards,
>> >
>> > On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io>
>> > wrote:
>> >
>> > > +1 (binding)
>> > >
>> > > @Abhimanyu: can you please update the table in
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > > and add a link to the KIP. Thanks.
>> > >
>> > > -Matthias
>> > >
>> > > On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
>> > > > +1
>> > > >
>> > > > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <
>> mageshn@confluent.io
>> > >
>> > > > wrote:
>> > > >
>> > > >> +1 ( non-binding)
>> > > >>
>> > > >> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com>
>> wrote:
>> > > >>
>> > > >>> Hello,
>> > > >>>
>> > > >>> I have made necessary changes as per the original discussion
>> thread,
>> > > and
>> > > >>> would like to put it for votes.
>> > > >>>
>> > > >>> Thank you very much for your suggestion and guidance so far.
>> > > >>>
>> > > >>> Regards,
>> > > >>>
>> > > >>
>> > > >
>> > >
>> > >
>> >
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Thanks for reminding me about the "Binding" vote Bill. I remember some
people with non-binding vote, so jumped the gun a bit too early.
We will wait for 2 more as you stated.
Regards,
On Wed, 3 Oct 2018 at 16:07, Bill Bejeck <bb...@gmail.com> wrote:
> +1 from me.
>
> As for closing the vote, it needs to be open for a minimum of 72 and
> requires three binding +1 votes. Additionally, there needs more +1 binding
> votes than -1 votes. The description for the lazy majority vote process is
> described here
> https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals.
>
> I haven't been tracking the vote results, but from what I can see in the
> voting thread, this KIP still requires two more +1 binding votes.
>
> HTH,
> BIll
>
> On Wed, Oct 3, 2018 at 8:58 AM M. Manna <ma...@gmail.com> wrote:
>
> > Since this has been open for a while, I am assuming that it's good to go?
> >
> > if so, I will update the KIP page - and start coding this. I would prefer
> > re-using existing tests written for DefaultPartitioner, so that we don't
> > need to write new tests.
> >
> > Regards,
> >
> > On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > @Abhimanyu: can you please update the table in
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > and add a link to the KIP. Thanks.
> > >
> > > -Matthias
> > >
> > > On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
> > > > +1
> > > >
> > > > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <
> mageshn@confluent.io
> > >
> > > > wrote:
> > > >
> > > >> +1 ( non-binding)
> > > >>
> > > >> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> > > >>
> > > >>> Hello,
> > > >>>
> > > >>> I have made necessary changes as per the original discussion
> thread,
> > > and
> > > >>> would like to put it for votes.
> > > >>>
> > > >>> Thank you very much for your suggestion and guidance so far.
> > > >>>
> > > >>> Regards,
> > > >>>
> > > >>
> > > >
> > >
> > >
> >
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by Bill Bejeck <bb...@gmail.com>.
+1 from me.
As for closing the vote, it needs to be open for a minimum of 72 and
requires three binding +1 votes. Additionally, there needs more +1 binding
votes than -1 votes. The description for the lazy majority vote process is
described here
https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals.
I haven't been tracking the vote results, but from what I can see in the
voting thread, this KIP still requires two more +1 binding votes.
HTH,
BIll
On Wed, Oct 3, 2018 at 8:58 AM M. Manna <ma...@gmail.com> wrote:
> Since this has been open for a while, I am assuming that it's good to go?
>
> if so, I will update the KIP page - and start coding this. I would prefer
> re-using existing tests written for DefaultPartitioner, so that we don't
> need to write new tests.
>
> Regards,
>
> On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io>
> wrote:
>
> > +1 (binding)
> >
> > @Abhimanyu: can you please update the table in
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > and add a link to the KIP. Thanks.
> >
> > -Matthias
> >
> > On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
> > > +1
> > >
> > > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <mageshn@confluent.io
> >
> > > wrote:
> > >
> > >> +1 ( non-binding)
> > >>
> > >> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>> I have made necessary changes as per the original discussion thread,
> > and
> > >>> would like to put it for votes.
> > >>>
> > >>> Thank you very much for your suggestion and guidance so far.
> > >>>
> > >>> Regards,
> > >>>
> > >>
> > >
> >
> >
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Since this has been open for a while, I am assuming that it's good to go?
if so, I will update the KIP page - and start coding this. I would prefer
re-using existing tests written for DefaultPartitioner, so that we don't
need to write new tests.
Regards,
On Sun, 30 Sep 2018 at 19:34, Matthias J. Sax <ma...@confluent.io> wrote:
> +1 (binding)
>
> @Abhimanyu: can you please update the table in
>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> and add a link to the KIP. Thanks.
>
> -Matthias
>
> On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
> > +1
> >
> > On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <ma...@confluent.io>
> > wrote:
> >
> >> +1 ( non-binding)
> >>
> >> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> I have made necessary changes as per the original discussion thread,
> and
> >>> would like to put it for votes.
> >>>
> >>> Thank you very much for your suggestion and guidance so far.
> >>>
> >>> Regards,
> >>>
> >>
> >
>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "Matthias J. Sax" <ma...@confluent.io>.
+1 (binding)
@Abhimanyu: can you please update the table in
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
and add a link to the KIP. Thanks.
-Matthias
On 9/4/18 9:56 PM, Abhimanyu Nagrath wrote:
> +1
>
> On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <ma...@confluent.io>
> wrote:
>
>> +1 ( non-binding)
>>
>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I have made necessary changes as per the original discussion thread, and
>>> would like to put it for votes.
>>>
>>> Thank you very much for your suggestion and guidance so far.
>>>
>>> Regards,
>>>
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by Abhimanyu Nagrath <ab...@gmail.com>.
+1
On Wed, Sep 5, 2018 at 2:39 AM Magesh Nandakumar <ma...@confluent.io>
wrote:
> +1 ( non-binding)
>
> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>
> > Hello,
> >
> > I have made necessary changes as per the original discussion thread, and
> > would like to put it for votes.
> >
> > Thank you very much for your suggestion and guidance so far.
> >
> > Regards,
> >
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by Magesh Nandakumar <ma...@confluent.io>.
+1 ( non-binding)
On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> Hello,
>
> I have made necessary changes as per the original discussion thread, and
> would like to put it for votes.
>
> Thank you very much for your suggestion and guidance so far.
>
> Regards,
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
I have now submitted the PR for review. Thanks Matthias for pointing out
that KAFKA-3333 was raised to address the same.
https://github.com/apache/kafka/pull/6771
if someone reviews after Jenkins build is complete, I would appreciate it.
Thanks,
On Fri, 17 May 2019 at 22:18, M. Manna <ma...@gmail.com> wrote:
> Apologies for the delay. As per the original thread, there have been 3
> binding votes.
>
> I will be closing this and update the confluence page with the results.
> Also, I will be submitting the PR soon.
>
> Regards,
>
> On Fri, 5 Apr 2019 at 00:18, M. Manna <ma...@gmail.com> wrote:
>
>> Thanks Harsha.
>>
>> As per your comments, I have counted 3 binding votes so far.
>>
>> Thanks everyone for your comments and support. I’ll update the kip next
>> morning and do the needful.
>>
>> Regards,
>>
>> On Thu, 4 Apr 2019 at 22:10, Harsha <ka...@harsha.io> wrote:
>>
>>> Looks like the KIP is passed with 3 binding votes. From Matthias, Bill
>>> Bejeck and myself you got 3 binding votes.
>>> You can do the full tally of the votes and send out a close of vote
>>> thread.
>>>
>>> Thanks,
>>> Harsha
>>>
>>> On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote:
>>> > Hello,
>>> >
>>> > Trying to revive this thread again. Would anyone be interested in
>>> having
>>> > this KiP through
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
>>> >
>>> > > Hello,
>>> > >
>>> > > I am trying to revive this thread. I only got 1 binding vote so far.
>>> > >
>>> > > Please feel free to revisit and comment here.
>>> > >
>>> > > Thanks,
>>> > >
>>> > > On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
>>> > >
>>> > >> Hey IJ,
>>> > >>
>>> > >> Thanks for your interest in the KIP.
>>> > >>
>>> > >> My point was simply that the round-robin should happen even if the
>>> key is
>>> > >> not null. As for the importance of key in our case, we treat the
>>> key as
>>> > >> metadata. Each key is composed of certain info which are parsed by
>>> our
>>> > >> consumer thread. We will then determine whether it's an actionable
>>> message
>>> > >> (e.g. process it), or a loopback(ignore it). You could argue, "Why
>>> not
>>> > >> append this metadata with the record and parse it there?". But that
>>> means
>>> > >> the following:
>>> > >>
>>> > >> 1) I'm always passing null key to achieve this - I would like to
>>> pass
>>> > >> Null/Not-Null/Other key i.e. flexibility
>>> > >> 2) Suppose the message size is 99 KB and and max message bytes
>>> allowed is
>>> > >> 100K. Now prefixing metadata with message results into the actual
>>> message
>>> > >> being 101K. This will fail at producer level and cause a retry/log
>>> this in
>>> > >> our DB for future pickup.
>>> > >>
>>> > >> To avoid all these, we are simply proposing this new partitioner
>>> class.
>>> > >> but all Kafka new releases will still have DefaultPartitioner as
>>> default,
>>> > >> unless they change the prop file to use our new class.
>>> > >>
>>> > >> Regards,
>>> > >>
>>> > >> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk>
>>> wrote:
>>> > >>
>>> > >>> Thanks for the KIP. Can you please elaborate on the need for the
>>> key in
>>> > >>> this case? The KIP simply states that the key is needed for
>>> metadata, but
>>> > >>> doesn't give any more details.
>>> > >>>
>>> > >>> Ismael
>>> > >>>
>>> > >>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com>
>>> wrote:
>>> > >>>
>>> > >>> > Hello,
>>> > >>> >
>>> > >>> > I have made necessary changes as per the original discussion
>>> thread,
>>> > >>> and
>>> > >>> > would like to put it for votes.
>>> > >>> >
>>> > >>> > Thank you very much for your suggestion and guidance so far.
>>> > >>> >
>>> > >>> > Regards,
>>> > >>> >
>>> > >>>
>>> > >>
>>> >
>>>
>>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Apologies for the delay. As per the original thread, there have been 3
binding votes.
I will be closing this and update the confluence page with the results.
Also, I will be submitting the PR soon.
Regards,
On Fri, 5 Apr 2019 at 00:18, M. Manna <ma...@gmail.com> wrote:
> Thanks Harsha.
>
> As per your comments, I have counted 3 binding votes so far.
>
> Thanks everyone for your comments and support. I’ll update the kip next
> morning and do the needful.
>
> Regards,
>
> On Thu, 4 Apr 2019 at 22:10, Harsha <ka...@harsha.io> wrote:
>
>> Looks like the KIP is passed with 3 binding votes. From Matthias, Bill
>> Bejeck and myself you got 3 binding votes.
>> You can do the full tally of the votes and send out a close of vote
>> thread.
>>
>> Thanks,
>> Harsha
>>
>> On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote:
>> > Hello,
>> >
>> > Trying to revive this thread again. Would anyone be interested in having
>> > this KiP through
>> >
>> >
>> > Thanks,
>> >
>> > On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
>> >
>> > > Hello,
>> > >
>> > > I am trying to revive this thread. I only got 1 binding vote so far.
>> > >
>> > > Please feel free to revisit and comment here.
>> > >
>> > > Thanks,
>> > >
>> > > On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
>> > >
>> > >> Hey IJ,
>> > >>
>> > >> Thanks for your interest in the KIP.
>> > >>
>> > >> My point was simply that the round-robin should happen even if the
>> key is
>> > >> not null. As for the importance of key in our case, we treat the key
>> as
>> > >> metadata. Each key is composed of certain info which are parsed by
>> our
>> > >> consumer thread. We will then determine whether it's an actionable
>> message
>> > >> (e.g. process it), or a loopback(ignore it). You could argue, "Why
>> not
>> > >> append this metadata with the record and parse it there?". But that
>> means
>> > >> the following:
>> > >>
>> > >> 1) I'm always passing null key to achieve this - I would like to pass
>> > >> Null/Not-Null/Other key i.e. flexibility
>> > >> 2) Suppose the message size is 99 KB and and max message bytes
>> allowed is
>> > >> 100K. Now prefixing metadata with message results into the actual
>> message
>> > >> being 101K. This will fail at producer level and cause a retry/log
>> this in
>> > >> our DB for future pickup.
>> > >>
>> > >> To avoid all these, we are simply proposing this new partitioner
>> class.
>> > >> but all Kafka new releases will still have DefaultPartitioner as
>> default,
>> > >> unless they change the prop file to use our new class.
>> > >>
>> > >> Regards,
>> > >>
>> > >> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
>> > >>
>> > >>> Thanks for the KIP. Can you please elaborate on the need for the
>> key in
>> > >>> this case? The KIP simply states that the key is needed for
>> metadata, but
>> > >>> doesn't give any more details.
>> > >>>
>> > >>> Ismael
>> > >>>
>> > >>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>> > >>>
>> > >>> > Hello,
>> > >>> >
>> > >>> > I have made necessary changes as per the original discussion
>> thread,
>> > >>> and
>> > >>> > would like to put it for votes.
>> > >>> >
>> > >>> > Thank you very much for your suggestion and guidance so far.
>> > >>> >
>> > >>> > Regards,
>> > >>> >
>> > >>>
>> > >>
>> >
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Thanks Harsha.
As per your comments, I have counted 3 binding votes so far.
Thanks everyone for your comments and support. I’ll update the kip next
morning and do the needful.
Regards,
On Thu, 4 Apr 2019 at 22:10, Harsha <ka...@harsha.io> wrote:
> Looks like the KIP is passed with 3 binding votes. From Matthias, Bill
> Bejeck and myself you got 3 binding votes.
> You can do the full tally of the votes and send out a close of vote thread.
>
> Thanks,
> Harsha
>
> On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote:
> > Hello,
> >
> > Trying to revive this thread again. Would anyone be interested in having
> > this KiP through
> >
> >
> > Thanks,
> >
> > On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > I am trying to revive this thread. I only got 1 binding vote so far.
> > >
> > > Please feel free to revisit and comment here.
> > >
> > > Thanks,
> > >
> > > On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
> > >
> > >> Hey IJ,
> > >>
> > >> Thanks for your interest in the KIP.
> > >>
> > >> My point was simply that the round-robin should happen even if the
> key is
> > >> not null. As for the importance of key in our case, we treat the key
> as
> > >> metadata. Each key is composed of certain info which are parsed by our
> > >> consumer thread. We will then determine whether it's an actionable
> message
> > >> (e.g. process it), or a loopback(ignore it). You could argue, "Why not
> > >> append this metadata with the record and parse it there?". But that
> means
> > >> the following:
> > >>
> > >> 1) I'm always passing null key to achieve this - I would like to pass
> > >> Null/Not-Null/Other key i.e. flexibility
> > >> 2) Suppose the message size is 99 KB and and max message bytes
> allowed is
> > >> 100K. Now prefixing metadata with message results into the actual
> message
> > >> being 101K. This will fail at producer level and cause a retry/log
> this in
> > >> our DB for future pickup.
> > >>
> > >> To avoid all these, we are simply proposing this new partitioner
> class.
> > >> but all Kafka new releases will still have DefaultPartitioner as
> default,
> > >> unless they change the prop file to use our new class.
> > >>
> > >> Regards,
> > >>
> > >> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
> > >>
> > >>> Thanks for the KIP. Can you please elaborate on the need for the key
> in
> > >>> this case? The KIP simply states that the key is needed for
> metadata, but
> > >>> doesn't give any more details.
> > >>>
> > >>> Ismael
> > >>>
> > >>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> > >>>
> > >>> > Hello,
> > >>> >
> > >>> > I have made necessary changes as per the original discussion
> thread,
> > >>> and
> > >>> > would like to put it for votes.
> > >>> >
> > >>> > Thank you very much for your suggestion and guidance so far.
> > >>> >
> > >>> > Regards,
> > >>> >
> > >>>
> > >>
> >
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection
Posted by Harsha <ka...@harsha.io>.
Looks like the KIP is passed with 3 binding votes. From Matthias, Bill Bejeck and myself you got 3 binding votes.
You can do the full tally of the votes and send out a close of vote thread.
Thanks,
Harsha
On Thu, Apr 4, 2019, at 12:24 PM, M. Manna wrote:
> Hello,
>
> Trying to revive this thread again. Would anyone be interested in having
> this KiP through
>
>
> Thanks,
>
> On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
>
> > Hello,
> >
> > I am trying to revive this thread. I only got 1 binding vote so far.
> >
> > Please feel free to revisit and comment here.
> >
> > Thanks,
> >
> > On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
> >
> >> Hey IJ,
> >>
> >> Thanks for your interest in the KIP.
> >>
> >> My point was simply that the round-robin should happen even if the key is
> >> not null. As for the importance of key in our case, we treat the key as
> >> metadata. Each key is composed of certain info which are parsed by our
> >> consumer thread. We will then determine whether it's an actionable message
> >> (e.g. process it), or a loopback(ignore it). You could argue, "Why not
> >> append this metadata with the record and parse it there?". But that means
> >> the following:
> >>
> >> 1) I'm always passing null key to achieve this - I would like to pass
> >> Null/Not-Null/Other key i.e. flexibility
> >> 2) Suppose the message size is 99 KB and and max message bytes allowed is
> >> 100K. Now prefixing metadata with message results into the actual message
> >> being 101K. This will fail at producer level and cause a retry/log this in
> >> our DB for future pickup.
> >>
> >> To avoid all these, we are simply proposing this new partitioner class.
> >> but all Kafka new releases will still have DefaultPartitioner as default,
> >> unless they change the prop file to use our new class.
> >>
> >> Regards,
> >>
> >> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
> >>
> >>> Thanks for the KIP. Can you please elaborate on the need for the key in
> >>> this case? The KIP simply states that the key is needed for metadata, but
> >>> doesn't give any more details.
> >>>
> >>> Ismael
> >>>
> >>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> >>>
> >>> > Hello,
> >>> >
> >>> > I have made necessary changes as per the original discussion thread,
> >>> and
> >>> > would like to put it for votes.
> >>> >
> >>> > Thank you very much for your suggestion and guidance so far.
> >>> >
> >>> > Regards,
> >>> >
> >>>
> >>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Hello,
Trying to revive this thread again. Would anyone be interested in having
this KiP through
Thanks,
On Fri, 25 Jan 2019 at 16:44, M. Manna <ma...@gmail.com> wrote:
> Hello,
>
> I am trying to revive this thread. I only got 1 binding vote so far.
>
> Please feel free to revisit and comment here.
>
> Thanks,
>
> On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
>
>> Hey IJ,
>>
>> Thanks for your interest in the KIP.
>>
>> My point was simply that the round-robin should happen even if the key is
>> not null. As for the importance of key in our case, we treat the key as
>> metadata. Each key is composed of certain info which are parsed by our
>> consumer thread. We will then determine whether it's an actionable message
>> (e.g. process it), or a loopback(ignore it). You could argue, "Why not
>> append this metadata with the record and parse it there?". But that means
>> the following:
>>
>> 1) I'm always passing null key to achieve this - I would like to pass
>> Null/Not-Null/Other key i.e. flexibility
>> 2) Suppose the message size is 99 KB and and max message bytes allowed is
>> 100K. Now prefixing metadata with message results into the actual message
>> being 101K. This will fail at producer level and cause a retry/log this in
>> our DB for future pickup.
>>
>> To avoid all these, we are simply proposing this new partitioner class.
>> but all Kafka new releases will still have DefaultPartitioner as default,
>> unless they change the prop file to use our new class.
>>
>> Regards,
>>
>> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
>>
>>> Thanks for the KIP. Can you please elaborate on the need for the key in
>>> this case? The KIP simply states that the key is needed for metadata, but
>>> doesn't give any more details.
>>>
>>> Ismael
>>>
>>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>>>
>>> > Hello,
>>> >
>>> > I have made necessary changes as per the original discussion thread,
>>> and
>>> > would like to put it for votes.
>>> >
>>> > Thank you very much for your suggestion and guidance so far.
>>> >
>>> > Regards,
>>> >
>>>
>>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Hello,
I am trying to revive this thread. I only got 1 binding vote so far.
Please feel free to revisit and comment here.
Thanks,
On Thu, 25 Oct 2018 at 00:15, M. Manna <ma...@gmail.com> wrote:
> Hey IJ,
>
> Thanks for your interest in the KIP.
>
> My point was simply that the round-robin should happen even if the key is
> not null. As for the importance of key in our case, we treat the key as
> metadata. Each key is composed of certain info which are parsed by our
> consumer thread. We will then determine whether it's an actionable message
> (e.g. process it), or a loopback(ignore it). You could argue, "Why not
> append this metadata with the record and parse it there?". But that means
> the following:
>
> 1) I'm always passing null key to achieve this - I would like to pass
> Null/Not-Null/Other key i.e. flexibility
> 2) Suppose the message size is 99 KB and and max message bytes allowed is
> 100K. Now prefixing metadata with message results into the actual message
> being 101K. This will fail at producer level and cause a retry/log this in
> our DB for future pickup.
>
> To avoid all these, we are simply proposing this new partitioner class.
> but all Kafka new releases will still have DefaultPartitioner as default,
> unless they change the prop file to use our new class.
>
> Regards,
>
> On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
>
>> Thanks for the KIP. Can you please elaborate on the need for the key in
>> this case? The KIP simply states that the key is needed for metadata, but
>> doesn't give any more details.
>>
>> Ismael
>>
>> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>>
>> > Hello,
>> >
>> > I have made necessary changes as per the original discussion thread, and
>> > would like to put it for votes.
>> >
>> > Thank you very much for your suggestion and guidance so far.
>> >
>> > Regards,
>> >
>>
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by "M. Manna" <ma...@gmail.com>.
Hey IJ,
Thanks for your interest in the KIP.
My point was simply that the round-robin should happen even if the key is
not null. As for the importance of key in our case, we treat the key as
metadata. Each key is composed of certain info which are parsed by our
consumer thread. We will then determine whether it's an actionable message
(e.g. process it), or a loopback(ignore it). You could argue, "Why not
append this metadata with the record and parse it there?". But that means
the following:
1) I'm always passing null key to achieve this - I would like to pass
Null/Not-Null/Other key i.e. flexibility
2) Suppose the message size is 99 KB and and max message bytes allowed is
100K. Now prefixing metadata with message results into the actual message
being 101K. This will fail at producer level and cause a retry/log this in
our DB for future pickup.
To avoid all these, we are simply proposing this new partitioner class. but
all Kafka new releases will still have DefaultPartitioner as default,
unless they change the prop file to use our new class.
Regards,
On Sun, 21 Oct 2018 at 04:05, Ismael Juma <is...@juma.me.uk> wrote:
> Thanks for the KIP. Can you please elaborate on the need for the key in
> this case? The KIP simply states that the key is needed for metadata, but
> doesn't give any more details.
>
> Ismael
>
> On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
>
> > Hello,
> >
> > I have made necessary changes as per the original discussion thread, and
> > would like to put it for votes.
> >
> > Thank you very much for your suggestion and guidance so far.
> >
> > Regards,
> >
>
Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always
Round-Robin" Selection
Posted by Ismael Juma <is...@juma.me.uk>.
Thanks for the KIP. Can you please elaborate on the need for the key in
this case? The KIP simply states that the key is needed for metadata, but
doesn't give any more details.
Ismael
On Tue, Sep 4, 2018 at 3:39 AM M. Manna <ma...@gmail.com> wrote:
> Hello,
>
> I have made necessary changes as per the original discussion thread, and
> would like to put it for votes.
>
> Thank you very much for your suggestion and guidance so far.
>
> Regards,
>