You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Randall Hauch <rh...@gmail.com> on 2018/05/17 23:15:10 UTC
[DISCUSS] KIP-305: Add Connect primitive number converters
I'd like to start discussion of a very straightforward proposal for Connect
to add converters for the basic primitive number types: integer, short,
long, double, and float. Here is the KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-305%3A+Add+Connect+primitive+number+converters
As mentioned in the KIP, I've created a pull request (
https://github.com/apache/kafka/pull/5034) for those looking for
implementation details.
Any feedback is appreciated.
Best regards,
Randall
Re: [DISCUSS] KIP-305: Add Connect primitive number converters
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Sorry, probably not ideal to just be seeing this now given KIP deadline,
but in general straightforward ones where we expect things to be
uncontroversial, I think its fine to kick off a vote thread. Worst case we
have people re-file votes if something substantial changes in the proposal.
-Ewen
On Sat, May 19, 2018 at 6:20 AM Randall Hauch <rh...@gmail.com> wrote:
> Considering this KIP is straightforward, what do you think about kicking
> off a vote? Or does it need more discussion time?
>
> Regards,
> Randall
>
> > On May 18, 2018, at 4:30 PM, Ewen Cheslack-Postava <ew...@confluent.io>
> wrote:
> >
> > Yeah, the usefulness of short seems questionable, but int is probably a
> > large enough range for some identifiers (e.g. we use an int in schema
> > registry). But yeah, I don't really have a problem with having Converters
> > for each of the existing serdes just for symmetry and since presumably
> > somebody finds them useful for something if they exist.
> >
> > -Ewen
> >
> >> On Fri, May 18, 2018 at 11:55 AM Randall Hauch <rh...@gmail.com>
> wrote:
> >>
> >> Thanks, Ewen.
> >>
> >> You make several good points, and I've updated the KIP to hopefully
> address
> >> your comments. I think the symmetry with the Kafka serdes is useful, so
> >> I've kept all 5 converters in the KIP.
> >>
> >> Interestingly, perhaps the short and int converters (with the reduced
> >> ranges) are not necessarily that useful for keys either.
> >>
> >> Regards,
> >>
> >> Randall
> >>
> >> On Thu, May 17, 2018 at 10:08 PM, Ewen Cheslack-Postava <
> ewen@confluent.io
> >>>
> >> wrote:
> >>
> >>> Just a couple of minor points that don't really affect the
> >> implementation:
> >>>
> >>> * For nulls, let's just mention the underlying serializers already
> >> support
> >>> this. I'm actually not sure why they should/need to, but given they do,
> >>> let's just defer to that implementation.
> >>> * I'm not sure where Float and Double converters are actually useful.
> The
> >>> use cases I know for integer serdes is for keys, but floats seem like a
> >> bad
> >>> choice for keys. These aren't a lot of overhead to build and maintain,
> >> but
> >>> if we don't know use cases for the specific types, it might be silly to
> >>> spend time and effort building and maintaining them.
> >>>
> >>> Otherwise, this seems simple and straightforward. Generally +1 on the
> >>> proposal.
> >>>
> >>> -Ewen
> >>>
> >>> On Thu, May 17, 2018 at 6:04 PM Magesh Nandakumar <
> mageshn@confluent.io>
> >>> wrote:
> >>>
> >>>> Thanks Randall for the KIP. I think it will be super useful and looks
> >>>> pretty straightforward to me.
> >>>>
> >>>> Thanks
> >>>> Magesh
> >>>>
> >>>> On Thu, May 17, 2018 at 4:15 PM, Randall Hauch <rh...@gmail.com>
> >> wrote:
> >>>>
> >>>>> I'd like to start discussion of a very straightforward proposal for
> >>>> Connect
> >>>>> to add converters for the basic primitive number types: integer,
> >> short,
> >>>>> long, double, and float. Here is the KIP:
> >>>>>
> >>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>> 305%3A+Add+Connect+primitive+number+converters
> >>>>>
> >>>>> As mentioned in the KIP, I've created a pull request (
> >>>>> https://github.com/apache/kafka/pull/5034) for those looking for
> >>>>> implementation details.
> >>>>>
> >>>>> Any feedback is appreciated.
> >>>>>
> >>>>> Best regards,
> >>>>>
> >>>>> Randall
> >>>>>
> >>>>
> >>>
> >>
>
Re: [DISCUSS] KIP-305: Add Connect primitive number converters
Posted by Randall Hauch <rh...@gmail.com>.
Considering this KIP is straightforward, what do you think about kicking off a vote? Or does it need more discussion time?
Regards,
Randall
> On May 18, 2018, at 4:30 PM, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
>
> Yeah, the usefulness of short seems questionable, but int is probably a
> large enough range for some identifiers (e.g. we use an int in schema
> registry). But yeah, I don't really have a problem with having Converters
> for each of the existing serdes just for symmetry and since presumably
> somebody finds them useful for something if they exist.
>
> -Ewen
>
>> On Fri, May 18, 2018 at 11:55 AM Randall Hauch <rh...@gmail.com> wrote:
>>
>> Thanks, Ewen.
>>
>> You make several good points, and I've updated the KIP to hopefully address
>> your comments. I think the symmetry with the Kafka serdes is useful, so
>> I've kept all 5 converters in the KIP.
>>
>> Interestingly, perhaps the short and int converters (with the reduced
>> ranges) are not necessarily that useful for keys either.
>>
>> Regards,
>>
>> Randall
>>
>> On Thu, May 17, 2018 at 10:08 PM, Ewen Cheslack-Postava <ewen@confluent.io
>>>
>> wrote:
>>
>>> Just a couple of minor points that don't really affect the
>> implementation:
>>>
>>> * For nulls, let's just mention the underlying serializers already
>> support
>>> this. I'm actually not sure why they should/need to, but given they do,
>>> let's just defer to that implementation.
>>> * I'm not sure where Float and Double converters are actually useful. The
>>> use cases I know for integer serdes is for keys, but floats seem like a
>> bad
>>> choice for keys. These aren't a lot of overhead to build and maintain,
>> but
>>> if we don't know use cases for the specific types, it might be silly to
>>> spend time and effort building and maintaining them.
>>>
>>> Otherwise, this seems simple and straightforward. Generally +1 on the
>>> proposal.
>>>
>>> -Ewen
>>>
>>> On Thu, May 17, 2018 at 6:04 PM Magesh Nandakumar <ma...@confluent.io>
>>> wrote:
>>>
>>>> Thanks Randall for the KIP. I think it will be super useful and looks
>>>> pretty straightforward to me.
>>>>
>>>> Thanks
>>>> Magesh
>>>>
>>>> On Thu, May 17, 2018 at 4:15 PM, Randall Hauch <rh...@gmail.com>
>> wrote:
>>>>
>>>>> I'd like to start discussion of a very straightforward proposal for
>>>> Connect
>>>>> to add converters for the basic primitive number types: integer,
>> short,
>>>>> long, double, and float. Here is the KIP:
>>>>>
>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>> 305%3A+Add+Connect+primitive+number+converters
>>>>>
>>>>> As mentioned in the KIP, I've created a pull request (
>>>>> https://github.com/apache/kafka/pull/5034) for those looking for
>>>>> implementation details.
>>>>>
>>>>> Any feedback is appreciated.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Randall
>>>>>
>>>>
>>>
>>
Re: [DISCUSS] KIP-305: Add Connect primitive number converters
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Yeah, the usefulness of short seems questionable, but int is probably a
large enough range for some identifiers (e.g. we use an int in schema
registry). But yeah, I don't really have a problem with having Converters
for each of the existing serdes just for symmetry and since presumably
somebody finds them useful for something if they exist.
-Ewen
On Fri, May 18, 2018 at 11:55 AM Randall Hauch <rh...@gmail.com> wrote:
> Thanks, Ewen.
>
> You make several good points, and I've updated the KIP to hopefully address
> your comments. I think the symmetry with the Kafka serdes is useful, so
> I've kept all 5 converters in the KIP.
>
> Interestingly, perhaps the short and int converters (with the reduced
> ranges) are not necessarily that useful for keys either.
>
> Regards,
>
> Randall
>
> On Thu, May 17, 2018 at 10:08 PM, Ewen Cheslack-Postava <ewen@confluent.io
> >
> wrote:
>
> > Just a couple of minor points that don't really affect the
> implementation:
> >
> > * For nulls, let's just mention the underlying serializers already
> support
> > this. I'm actually not sure why they should/need to, but given they do,
> > let's just defer to that implementation.
> > * I'm not sure where Float and Double converters are actually useful. The
> > use cases I know for integer serdes is for keys, but floats seem like a
> bad
> > choice for keys. These aren't a lot of overhead to build and maintain,
> but
> > if we don't know use cases for the specific types, it might be silly to
> > spend time and effort building and maintaining them.
> >
> > Otherwise, this seems simple and straightforward. Generally +1 on the
> > proposal.
> >
> > -Ewen
> >
> > On Thu, May 17, 2018 at 6:04 PM Magesh Nandakumar <ma...@confluent.io>
> > wrote:
> >
> > > Thanks Randall for the KIP. I think it will be super useful and looks
> > > pretty straightforward to me.
> > >
> > > Thanks
> > > Magesh
> > >
> > > On Thu, May 17, 2018 at 4:15 PM, Randall Hauch <rh...@gmail.com>
> wrote:
> > >
> > > > I'd like to start discussion of a very straightforward proposal for
> > > Connect
> > > > to add converters for the basic primitive number types: integer,
> short,
> > > > long, double, and float. Here is the KIP:
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 305%3A+Add+Connect+primitive+number+converters
> > > >
> > > > As mentioned in the KIP, I've created a pull request (
> > > > https://github.com/apache/kafka/pull/5034) for those looking for
> > > > implementation details.
> > > >
> > > > Any feedback is appreciated.
> > > >
> > > > Best regards,
> > > >
> > > > Randall
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-305: Add Connect primitive number converters
Posted by Randall Hauch <rh...@gmail.com>.
Thanks, Ewen.
You make several good points, and I've updated the KIP to hopefully address
your comments. I think the symmetry with the Kafka serdes is useful, so
I've kept all 5 converters in the KIP.
Interestingly, perhaps the short and int converters (with the reduced
ranges) are not necessarily that useful for keys either.
Regards,
Randall
On Thu, May 17, 2018 at 10:08 PM, Ewen Cheslack-Postava <ew...@confluent.io>
wrote:
> Just a couple of minor points that don't really affect the implementation:
>
> * For nulls, let's just mention the underlying serializers already support
> this. I'm actually not sure why they should/need to, but given they do,
> let's just defer to that implementation.
> * I'm not sure where Float and Double converters are actually useful. The
> use cases I know for integer serdes is for keys, but floats seem like a bad
> choice for keys. These aren't a lot of overhead to build and maintain, but
> if we don't know use cases for the specific types, it might be silly to
> spend time and effort building and maintaining them.
>
> Otherwise, this seems simple and straightforward. Generally +1 on the
> proposal.
>
> -Ewen
>
> On Thu, May 17, 2018 at 6:04 PM Magesh Nandakumar <ma...@confluent.io>
> wrote:
>
> > Thanks Randall for the KIP. I think it will be super useful and looks
> > pretty straightforward to me.
> >
> > Thanks
> > Magesh
> >
> > On Thu, May 17, 2018 at 4:15 PM, Randall Hauch <rh...@gmail.com> wrote:
> >
> > > I'd like to start discussion of a very straightforward proposal for
> > Connect
> > > to add converters for the basic primitive number types: integer, short,
> > > long, double, and float. Here is the KIP:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 305%3A+Add+Connect+primitive+number+converters
> > >
> > > As mentioned in the KIP, I've created a pull request (
> > > https://github.com/apache/kafka/pull/5034) for those looking for
> > > implementation details.
> > >
> > > Any feedback is appreciated.
> > >
> > > Best regards,
> > >
> > > Randall
> > >
> >
>
Re: [DISCUSS] KIP-305: Add Connect primitive number converters
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Just a couple of minor points that don't really affect the implementation:
* For nulls, let's just mention the underlying serializers already support
this. I'm actually not sure why they should/need to, but given they do,
let's just defer to that implementation.
* I'm not sure where Float and Double converters are actually useful. The
use cases I know for integer serdes is for keys, but floats seem like a bad
choice for keys. These aren't a lot of overhead to build and maintain, but
if we don't know use cases for the specific types, it might be silly to
spend time and effort building and maintaining them.
Otherwise, this seems simple and straightforward. Generally +1 on the
proposal.
-Ewen
On Thu, May 17, 2018 at 6:04 PM Magesh Nandakumar <ma...@confluent.io>
wrote:
> Thanks Randall for the KIP. I think it will be super useful and looks
> pretty straightforward to me.
>
> Thanks
> Magesh
>
> On Thu, May 17, 2018 at 4:15 PM, Randall Hauch <rh...@gmail.com> wrote:
>
> > I'd like to start discussion of a very straightforward proposal for
> Connect
> > to add converters for the basic primitive number types: integer, short,
> > long, double, and float. Here is the KIP:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 305%3A+Add+Connect+primitive+number+converters
> >
> > As mentioned in the KIP, I've created a pull request (
> > https://github.com/apache/kafka/pull/5034) for those looking for
> > implementation details.
> >
> > Any feedback is appreciated.
> >
> > Best regards,
> >
> > Randall
> >
>
Re: [DISCUSS] KIP-305: Add Connect primitive number converters
Posted by Magesh Nandakumar <ma...@confluent.io>.
Thanks Randall for the KIP. I think it will be super useful and looks
pretty straightforward to me.
Thanks
Magesh
On Thu, May 17, 2018 at 4:15 PM, Randall Hauch <rh...@gmail.com> wrote:
> I'd like to start discussion of a very straightforward proposal for Connect
> to add converters for the basic primitive number types: integer, short,
> long, double, and float. Here is the KIP:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 305%3A+Add+Connect+primitive+number+converters
>
> As mentioned in the KIP, I've created a pull request (
> https://github.com/apache/kafka/pull/5034) for those looking for
> implementation details.
>
> Any feedback is appreciated.
>
> Best regards,
>
> Randall
>