You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Chia-Ping Tsai <ch...@apache.org> on 2020/07/22 16:24:19 UTC

[DISCUSS] KIP-646 Serializer API should support ByteBuffer

hi folks,

I would like to discuss KIP-646. The KIP plans to use ByteBuffer to be the return type of Serializer#serialize. It opens the door to manage the memory more effectively and flexible.

https://cwiki.apache.org/confluence/x/RiR4CQ

The change involved by this KIP is huge so it would be better to get enough feedback before starting to work :)

Cheers,
Chia-Ping

Re: [DISCUSS] KIP-646 Serializer API should support ByteBuffer

Posted by Corentin Chary <co...@datadoghq.com.INVALID>.

On 2020/07/23 03:23:58, Chia-Ping Tsai <ch...@apache.org> wrote: 
> Thanks for quick feedback! Ismael 
> 
> > Are there options with lower impact that still help us achieve the goal for those who need it?
> > For example, it could be an opt-in thing instead of forcing the world to change.
> 
> It seems to me there are two alternatives.
> 
> 1. Introduce an new extended serializer (there is an existent but deprecated one). if the serializer is extended type, we call new method to get ByteBuffer. Users who have no interest of ByteBuffer keep using standard Serializer interface.
> 2. Don’t deprecate the existent serialize methods. users are not under the pressure of API migration

I guess the Deserializer could have both `byte[]` and `ByteBuffer` deserialization method.

The code would call the `ByteBuffer` one, its default implementation would be to convert to `byte[]` and call the `byte[]` one with Utils.toArray, as is done in the Fetcher today.

This should stay compatible with existing deserializer while making it possible to deal with ByteBuffer or `byte[]`.

Re: [DISCUSS] KIP-646 Serializer API should support ByteBuffer

Posted by Chia-Ping Tsai <ch...@apache.org>.

Thanks for quick feedback! Ismael 

> Are there options with lower impact that still help us achieve the goal for those who need it?
> For example, it could be an opt-in thing instead of forcing the world to change.

It seems to me there are two alternatives.

1. Introduce an new extended serializer (there is an existent but deprecated one). if the serializer is extended type, we call new method to get ByteBuffer. Users who have no interest of ByteBuffer keep using standard Serializer interface.
2. Don’t deprecate the existent serialize methods. users are not under the pressure of API migration

--
Chia-Ping

On 2020/07/22 17:16:37, Ismael Juma <is...@juma.me.uk> wrote: 
> Hi Chia-Ping,
> 
> Thanks for the KIP. It seems like the path chosen here would cause a
> massive impact to user code. Are there options with lower impact that still
> help us achieve the goal for those who need it? For example, it could be an
> opt-in thing instead of forcing the world to change.
> 
> Ismael
> 
> On Wed, Jul 22, 2020 at 9:43 AM Chia-Ping Tsai <ch...@apache.org> wrote:
> 
> > hi folks,
> >
> > I would like to discuss KIP-646. The KIP plans to use ByteBuffer to be the
> > return type of Serializer#serialize. It opens the door to manage the memory
> > more effectively and flexible.
> >
> > https://cwiki.apache.org/confluence/x/RiR4CQ
> >
> > The change involved by this KIP is huge so it would be better to get
> > enough feedback before starting to work :)
> >
> > Cheers,
> > Chia-Ping
> >
>

Re: [DISCUSS] KIP-646 Serializer API should support ByteBuffer

Posted by Ismael Juma <is...@juma.me.uk>.

Hi Chia-Ping,

Thanks for the KIP. It seems like the path chosen here would cause a
massive impact to user code. Are there options with lower impact that still
help us achieve the goal for those who need it? For example, it could be an
opt-in thing instead of forcing the world to change.

Ismael

On Wed, Jul 22, 2020 at 9:43 AM Chia-Ping Tsai <ch...@apache.org> wrote:

> hi folks,
>
> I would like to discuss KIP-646. The KIP plans to use ByteBuffer to be the
> return type of Serializer#serialize. It opens the door to manage the memory
> more effectively and flexible.
>
> https://cwiki.apache.org/confluence/x/RiR4CQ
>
> The change involved by this KIP is huge so it would be better to get
> enough feedback before starting to work :)
>
> Cheers,
> Chia-Ping
>