You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Pushkar Deole <pd...@gmail.com> on 2020/05/07 12:31:53 UTC
records with key as string and value as java ArrayList in topic
Hi All,
I have a requirement to store a record with key as java String and value as
java's ArrayList in the kafka topic. Kafka has by default provided a
StringSerializer and StringDeserializer, however for java ArrayList, how
can get serializer. Do I need to write my own? Can someone share if someone
already has written one?
Re: records with key as string and value as java ArrayList in topic
Posted by Pushkar Deole <pd...@gmail.com>.
Thanks Liam....essentially, it would be an internal topic that we would be
creating to use as a cache store by accessing topic through a GlobalKTable,
so the problem you mentioned above for storing Hashmap may not apply there
On Tue, May 12, 2020, 21:25 Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:
> Hi Pushkar,
>
> Just wanted to say, as someone with battle scars from ActiveMQ and Camel,
> there's very many good reasons to avoid Java serialization on a messaging
> system. What if you need to tail a topic from the console? What if your
> testers want to access in their pytests? Etc. And that's not even getting
> into the minefield of Java serialization compatibility between one VM and
> another.
>
> That said, if using another serialization format like Avro or JSON isn't
> feasible, you can implement your own serializer/deserializer.
>
> At it's heart, a Kafka producer sends a byte array and a consumer receives
> a byte array. So a custom serialiser to turn a list or map into bytes is
> easily doable using an ObjectInputStream. And a consumer can use
> deserialiser built on an ObjectOutputStream at the other end.
>
> Kind regards,
>
> Liam Clarke-Hutchinson
>
> On Tue, 12 May 2020, 5:13 pm Pushkar Deole, <pd...@gmail.com> wrote:
>
> > And by the way, confluent has provided KafkaAvroSerializer/Deserialier.
> > Can't they be used to do conversion for java types?
> >
> > On Tue, May 12, 2020 at 10:09 AM Pushkar Deole <pd...@gmail.com>
> > wrote:
> >
> > > Ok... so jackson json serialization is the way to go for hashmaps as
> > well?
> > >
> > > On Mon, May 11, 2020 at 7:57 PM John Roesler <vv...@apache.org>
> > wrote:
> > >
> > >> Oh, my mistake. I thought this was a different thread :)
> > >>
> > >> You might want to check, but I don’t think there is a kip for a map
> > >> serde. Of course, you’re welcome to start one.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Mon, May 11, 2020, at 09:14, John Roesler wrote:
> > >> > Hi Pushkar,
> > >> >
> > >> > I don’t think there is. You’re welcome to start one if you think it
> > >> > would be a useful addition.
> > >> >
> > >> > Before worrying about it further, though, you might want to check
> the
> > >> > InMemoryKeyValueStore implementation, since my answer was from
> memory.
> > >> >
> > >> > Thanks,
> > >> > John
> > >> >
> > >> > On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote:
> > >> > > John,
> > >> > > is there KIP in progress for supporting Java HashMap also?
> > >> > >
> > >> > > On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org>
> > wrote:
> > >> > >
> > >> > > > Yes, that’s correct. It’s only for serializing the java type
> > >> ‘byte[]’.
> > >> > > >
> > >> > > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> > >> > > > > Thanks John... I got to finish the work in few days so need to
> > >> get it
> > >> > > > > quick, so looking for something ready. I will take a look at
> > >> jackson
> > >> > > > json.
> > >> > > > >
> > >> > > > > By the way, what is the byteArrayserializer? As the name
> > >> suggests, it is
> > >> > > > > for byte arrays so won't work for java ArrayList, right?
> > >> > > > >
> > >> > > > > On Thu, May 7, 2020 at 8:44 PM John Roesler <
> > vvcephei@apache.org>
> > >> wrote:
> > >> > > > >
> > >> > > > > > Hi Pushkar,
> > >> > > > > >
> > >> > > > > > If you’re not too concerned about compactness, I think
> Jackson
> > >> json
> > >> > > > > > serialization is the easiest way to serialize complex types.
> > >> > > > > >
> > >> > > > > > There’s also a kip in progress to add a list serde. You
> might
> > >> take a
> > >> > > > look
> > >> > > > > > at that proposal for ideas to write your own.
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > John
> > >> > > > > >
> > >> > > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > >> > > > > > > Won't say it's a good idea to use java serialized classes
> > for
> > >> > > > messages,
> > >> > > > > > but
> > >> > > > > > > you should use a byteArraySerializer if you want to do
> such
> > >> things
> > >> > > > > > >
> > >> > > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <
> > >> pdeole2015@gmail.com> a
> > >> > > > > > écrit :
> > >> > > > > > >
> > >> > > > > > > > Hi All,
> > >> > > > > > > >
> > >> > > > > > > > I have a requirement to store a record with key as java
> > >> String and
> > >> > > > > > value as
> > >> > > > > > > > java's ArrayList in the kafka topic. Kafka has by
> default
> > >> provided
> > >> > > > a
> > >> > > > > > > > StringSerializer and StringDeserializer, however for
> java
> > >> > > > ArrayList,
> > >> > > > > > how
> > >> > > > > > > > can get serializer. Do I need to write my own? Can
> someone
> > >> share if
> > >> > > > > > someone
> > >> > > > > > > > already has written one?
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > --
> > >> > > > > > > *Nicolas Carlot*
> > >> > > > > > >
> > >> > > > > > > Lead dev
> > >> > > > > > > | | nicolas.carlot@chronopost.fr
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost
> > >> déménage.
> > >> > > > La
> > >> > > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014
> > Paris*
> > >> > > > > > >
> > >> > > > > > > [image: Logo Chronopost]
> > >> > > > > > > | chronopost.fr <http://www.chronopost.fr/>
> > >> > > > > > > Suivez nous sur Facebook <
> > >> https://fr-fr.facebook.com/chronopost> et
> > >> > > > > > Twitter
> > >> > > > > > > <https://twitter.com/chronopost>.
> > >> > > > > > >
> > >> > > > > > > [image: DPD Group]
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Hi Pushkar,
Just wanted to say, as someone with battle scars from ActiveMQ and Camel,
there's very many good reasons to avoid Java serialization on a messaging
system. What if you need to tail a topic from the console? What if your
testers want to access in their pytests? Etc. And that's not even getting
into the minefield of Java serialization compatibility between one VM and
another.
That said, if using another serialization format like Avro or JSON isn't
feasible, you can implement your own serializer/deserializer.
At it's heart, a Kafka producer sends a byte array and a consumer receives
a byte array. So a custom serialiser to turn a list or map into bytes is
easily doable using an ObjectInputStream. And a consumer can use
deserialiser built on an ObjectOutputStream at the other end.
Kind regards,
Liam Clarke-Hutchinson
On Tue, 12 May 2020, 5:13 pm Pushkar Deole, <pd...@gmail.com> wrote:
> And by the way, confluent has provided KafkaAvroSerializer/Deserialier.
> Can't they be used to do conversion for java types?
>
> On Tue, May 12, 2020 at 10:09 AM Pushkar Deole <pd...@gmail.com>
> wrote:
>
> > Ok... so jackson json serialization is the way to go for hashmaps as
> well?
> >
> > On Mon, May 11, 2020 at 7:57 PM John Roesler <vv...@apache.org>
> wrote:
> >
> >> Oh, my mistake. I thought this was a different thread :)
> >>
> >> You might want to check, but I don’t think there is a kip for a map
> >> serde. Of course, you’re welcome to start one.
> >>
> >> Thanks,
> >> John
> >>
> >> On Mon, May 11, 2020, at 09:14, John Roesler wrote:
> >> > Hi Pushkar,
> >> >
> >> > I don’t think there is. You’re welcome to start one if you think it
> >> > would be a useful addition.
> >> >
> >> > Before worrying about it further, though, you might want to check the
> >> > InMemoryKeyValueStore implementation, since my answer was from memory.
> >> >
> >> > Thanks,
> >> > John
> >> >
> >> > On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote:
> >> > > John,
> >> > > is there KIP in progress for supporting Java HashMap also?
> >> > >
> >> > > On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org>
> wrote:
> >> > >
> >> > > > Yes, that’s correct. It’s only for serializing the java type
> >> ‘byte[]’.
> >> > > >
> >> > > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> >> > > > > Thanks John... I got to finish the work in few days so need to
> >> get it
> >> > > > > quick, so looking for something ready. I will take a look at
> >> jackson
> >> > > > json.
> >> > > > >
> >> > > > > By the way, what is the byteArrayserializer? As the name
> >> suggests, it is
> >> > > > > for byte arrays so won't work for java ArrayList, right?
> >> > > > >
> >> > > > > On Thu, May 7, 2020 at 8:44 PM John Roesler <
> vvcephei@apache.org>
> >> wrote:
> >> > > > >
> >> > > > > > Hi Pushkar,
> >> > > > > >
> >> > > > > > If you’re not too concerned about compactness, I think Jackson
> >> json
> >> > > > > > serialization is the easiest way to serialize complex types.
> >> > > > > >
> >> > > > > > There’s also a kip in progress to add a list serde. You might
> >> take a
> >> > > > look
> >> > > > > > at that proposal for ideas to write your own.
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > John
> >> > > > > >
> >> > > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> >> > > > > > > Won't say it's a good idea to use java serialized classes
> for
> >> > > > messages,
> >> > > > > > but
> >> > > > > > > you should use a byteArraySerializer if you want to do such
> >> things
> >> > > > > > >
> >> > > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <
> >> pdeole2015@gmail.com> a
> >> > > > > > écrit :
> >> > > > > > >
> >> > > > > > > > Hi All,
> >> > > > > > > >
> >> > > > > > > > I have a requirement to store a record with key as java
> >> String and
> >> > > > > > value as
> >> > > > > > > > java's ArrayList in the kafka topic. Kafka has by default
> >> provided
> >> > > > a
> >> > > > > > > > StringSerializer and StringDeserializer, however for java
> >> > > > ArrayList,
> >> > > > > > how
> >> > > > > > > > can get serializer. Do I need to write my own? Can someone
> >> share if
> >> > > > > > someone
> >> > > > > > > > already has written one?
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > *Nicolas Carlot*
> >> > > > > > >
> >> > > > > > > Lead dev
> >> > > > > > > | | nicolas.carlot@chronopost.fr
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost
> >> déménage.
> >> > > > La
> >> > > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014
> Paris*
> >> > > > > > >
> >> > > > > > > [image: Logo Chronopost]
> >> > > > > > > | chronopost.fr <http://www.chronopost.fr/>
> >> > > > > > > Suivez nous sur Facebook <
> >> https://fr-fr.facebook.com/chronopost> et
> >> > > > > > Twitter
> >> > > > > > > <https://twitter.com/chronopost>.
> >> > > > > > >
> >> > > > > > > [image: DPD Group]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by Pushkar Deole <pd...@gmail.com>.
And by the way, confluent has provided KafkaAvroSerializer/Deserialier.
Can't they be used to do conversion for java types?
On Tue, May 12, 2020 at 10:09 AM Pushkar Deole <pd...@gmail.com> wrote:
> Ok... so jackson json serialization is the way to go for hashmaps as well?
>
> On Mon, May 11, 2020 at 7:57 PM John Roesler <vv...@apache.org> wrote:
>
>> Oh, my mistake. I thought this was a different thread :)
>>
>> You might want to check, but I don’t think there is a kip for a map
>> serde. Of course, you’re welcome to start one.
>>
>> Thanks,
>> John
>>
>> On Mon, May 11, 2020, at 09:14, John Roesler wrote:
>> > Hi Pushkar,
>> >
>> > I don’t think there is. You’re welcome to start one if you think it
>> > would be a useful addition.
>> >
>> > Before worrying about it further, though, you might want to check the
>> > InMemoryKeyValueStore implementation, since my answer was from memory.
>> >
>> > Thanks,
>> > John
>> >
>> > On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote:
>> > > John,
>> > > is there KIP in progress for supporting Java HashMap also?
>> > >
>> > > On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org> wrote:
>> > >
>> > > > Yes, that’s correct. It’s only for serializing the java type
>> ‘byte[]’.
>> > > >
>> > > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
>> > > > > Thanks John... I got to finish the work in few days so need to
>> get it
>> > > > > quick, so looking for something ready. I will take a look at
>> jackson
>> > > > json.
>> > > > >
>> > > > > By the way, what is the byteArrayserializer? As the name
>> suggests, it is
>> > > > > for byte arrays so won't work for java ArrayList, right?
>> > > > >
>> > > > > On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org>
>> wrote:
>> > > > >
>> > > > > > Hi Pushkar,
>> > > > > >
>> > > > > > If you’re not too concerned about compactness, I think Jackson
>> json
>> > > > > > serialization is the easiest way to serialize complex types.
>> > > > > >
>> > > > > > There’s also a kip in progress to add a list serde. You might
>> take a
>> > > > look
>> > > > > > at that proposal for ideas to write your own.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > John
>> > > > > >
>> > > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
>> > > > > > > Won't say it's a good idea to use java serialized classes for
>> > > > messages,
>> > > > > > but
>> > > > > > > you should use a byteArraySerializer if you want to do such
>> things
>> > > > > > >
>> > > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <
>> pdeole2015@gmail.com> a
>> > > > > > écrit :
>> > > > > > >
>> > > > > > > > Hi All,
>> > > > > > > >
>> > > > > > > > I have a requirement to store a record with key as java
>> String and
>> > > > > > value as
>> > > > > > > > java's ArrayList in the kafka topic. Kafka has by default
>> provided
>> > > > a
>> > > > > > > > StringSerializer and StringDeserializer, however for java
>> > > > ArrayList,
>> > > > > > how
>> > > > > > > > can get serializer. Do I need to write my own? Can someone
>> share if
>> > > > > > someone
>> > > > > > > > already has written one?
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > *Nicolas Carlot*
>> > > > > > >
>> > > > > > > Lead dev
>> > > > > > > | | nicolas.carlot@chronopost.fr
>> > > > > > >
>> > > > > > >
>> > > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost
>> déménage.
>> > > > La
>> > > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
>> > > > > > >
>> > > > > > > [image: Logo Chronopost]
>> > > > > > > | chronopost.fr <http://www.chronopost.fr/>
>> > > > > > > Suivez nous sur Facebook <
>> https://fr-fr.facebook.com/chronopost> et
>> > > > > > Twitter
>> > > > > > > <https://twitter.com/chronopost>.
>> > > > > > >
>> > > > > > > [image: DPD Group]
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: records with key as string and value as java ArrayList in topic
Posted by Pushkar Deole <pd...@gmail.com>.
Ok... so jackson json serialization is the way to go for hashmaps as well?
On Mon, May 11, 2020 at 7:57 PM John Roesler <vv...@apache.org> wrote:
> Oh, my mistake. I thought this was a different thread :)
>
> You might want to check, but I don’t think there is a kip for a map serde.
> Of course, you’re welcome to start one.
>
> Thanks,
> John
>
> On Mon, May 11, 2020, at 09:14, John Roesler wrote:
> > Hi Pushkar,
> >
> > I don’t think there is. You’re welcome to start one if you think it
> > would be a useful addition.
> >
> > Before worrying about it further, though, you might want to check the
> > InMemoryKeyValueStore implementation, since my answer was from memory.
> >
> > Thanks,
> > John
> >
> > On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote:
> > > John,
> > > is there KIP in progress for supporting Java HashMap also?
> > >
> > > On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org> wrote:
> > >
> > > > Yes, that’s correct. It’s only for serializing the java type
> ‘byte[]’.
> > > >
> > > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> > > > > Thanks John... I got to finish the work in few days so need to get
> it
> > > > > quick, so looking for something ready. I will take a look at
> jackson
> > > > json.
> > > > >
> > > > > By the way, what is the byteArrayserializer? As the name suggests,
> it is
> > > > > for byte arrays so won't work for java ArrayList, right?
> > > > >
> > > > > On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org>
> wrote:
> > > > >
> > > > > > Hi Pushkar,
> > > > > >
> > > > > > If you’re not too concerned about compactness, I think Jackson
> json
> > > > > > serialization is the easiest way to serialize complex types.
> > > > > >
> > > > > > There’s also a kip in progress to add a list serde. You might
> take a
> > > > look
> > > > > > at that proposal for ideas to write your own.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > > > > > > Won't say it's a good idea to use java serialized classes for
> > > > messages,
> > > > > > but
> > > > > > > you should use a byteArraySerializer if you want to do such
> things
> > > > > > >
> > > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <
> pdeole2015@gmail.com> a
> > > > > > écrit :
> > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > > I have a requirement to store a record with key as java
> String and
> > > > > > value as
> > > > > > > > java's ArrayList in the kafka topic. Kafka has by default
> provided
> > > > a
> > > > > > > > StringSerializer and StringDeserializer, however for java
> > > > ArrayList,
> > > > > > how
> > > > > > > > can get serializer. Do I need to write my own? Can someone
> share if
> > > > > > someone
> > > > > > > > already has written one?
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Nicolas Carlot*
> > > > > > >
> > > > > > > Lead dev
> > > > > > > | | nicolas.carlot@chronopost.fr
> > > > > > >
> > > > > > >
> > > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost
> déménage.
> > > > La
> > > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
> > > > > > >
> > > > > > > [image: Logo Chronopost]
> > > > > > > | chronopost.fr <http://www.chronopost.fr/>
> > > > > > > Suivez nous sur Facebook <
> https://fr-fr.facebook.com/chronopost> et
> > > > > > Twitter
> > > > > > > <https://twitter.com/chronopost>.
> > > > > > >
> > > > > > > [image: DPD Group]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by John Roesler <vv...@apache.org>.
Oh, my mistake. I thought this was a different thread :)
You might want to check, but I don’t think there is a kip for a map serde. Of course, you’re welcome to start one.
Thanks,
John
On Mon, May 11, 2020, at 09:14, John Roesler wrote:
> Hi Pushkar,
>
> I don’t think there is. You’re welcome to start one if you think it
> would be a useful addition.
>
> Before worrying about it further, though, you might want to check the
> InMemoryKeyValueStore implementation, since my answer was from memory.
>
> Thanks,
> John
>
> On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote:
> > John,
> > is there KIP in progress for supporting Java HashMap also?
> >
> > On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org> wrote:
> >
> > > Yes, that’s correct. It’s only for serializing the java type ‘byte[]’.
> > >
> > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> > > > Thanks John... I got to finish the work in few days so need to get it
> > > > quick, so looking for something ready. I will take a look at jackson
> > > json.
> > > >
> > > > By the way, what is the byteArrayserializer? As the name suggests, it is
> > > > for byte arrays so won't work for java ArrayList, right?
> > > >
> > > > On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org> wrote:
> > > >
> > > > > Hi Pushkar,
> > > > >
> > > > > If you’re not too concerned about compactness, I think Jackson json
> > > > > serialization is the easiest way to serialize complex types.
> > > > >
> > > > > There’s also a kip in progress to add a list serde. You might take a
> > > look
> > > > > at that proposal for ideas to write your own.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > > > > > Won't say it's a good idea to use java serialized classes for
> > > messages,
> > > > > but
> > > > > > you should use a byteArraySerializer if you want to do such things
> > > > > >
> > > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a
> > > > > écrit :
> > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > I have a requirement to store a record with key as java String and
> > > > > value as
> > > > > > > java's ArrayList in the kafka topic. Kafka has by default provided
> > > a
> > > > > > > StringSerializer and StringDeserializer, however for java
> > > ArrayList,
> > > > > how
> > > > > > > can get serializer. Do I need to write my own? Can someone share if
> > > > > someone
> > > > > > > already has written one?
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Nicolas Carlot*
> > > > > >
> > > > > > Lead dev
> > > > > > | | nicolas.carlot@chronopost.fr
> > > > > >
> > > > > >
> > > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage.
> > > La
> > > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
> > > > > >
> > > > > > [image: Logo Chronopost]
> > > > > > | chronopost.fr <http://www.chronopost.fr/>
> > > > > > Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et
> > > > > Twitter
> > > > > > <https://twitter.com/chronopost>.
> > > > > >
> > > > > > [image: DPD Group]
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by John Roesler <vv...@apache.org>.
Hi Pushkar,
I don’t think there is. You’re welcome to start one if you think it would be a useful addition.
Before worrying about it further, though, you might want to check the InMemoryKeyValueStore implementation, since my answer was from memory.
Thanks,
John
On Mon, May 11, 2020, at 03:47, Pushkar Deole wrote:
> John,
> is there KIP in progress for supporting Java HashMap also?
>
> On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org> wrote:
>
> > Yes, that’s correct. It’s only for serializing the java type ‘byte[]’.
> >
> > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> > > Thanks John... I got to finish the work in few days so need to get it
> > > quick, so looking for something ready. I will take a look at jackson
> > json.
> > >
> > > By the way, what is the byteArrayserializer? As the name suggests, it is
> > > for byte arrays so won't work for java ArrayList, right?
> > >
> > > On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org> wrote:
> > >
> > > > Hi Pushkar,
> > > >
> > > > If you’re not too concerned about compactness, I think Jackson json
> > > > serialization is the easiest way to serialize complex types.
> > > >
> > > > There’s also a kip in progress to add a list serde. You might take a
> > look
> > > > at that proposal for ideas to write your own.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > > > > Won't say it's a good idea to use java serialized classes for
> > messages,
> > > > but
> > > > > you should use a byteArraySerializer if you want to do such things
> > > > >
> > > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a
> > > > écrit :
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I have a requirement to store a record with key as java String and
> > > > value as
> > > > > > java's ArrayList in the kafka topic. Kafka has by default provided
> > a
> > > > > > StringSerializer and StringDeserializer, however for java
> > ArrayList,
> > > > how
> > > > > > can get serializer. Do I need to write my own? Can someone share if
> > > > someone
> > > > > > already has written one?
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Nicolas Carlot*
> > > > >
> > > > > Lead dev
> > > > > | | nicolas.carlot@chronopost.fr
> > > > >
> > > > >
> > > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage.
> > La
> > > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
> > > > >
> > > > > [image: Logo Chronopost]
> > > > > | chronopost.fr <http://www.chronopost.fr/>
> > > > > Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et
> > > > Twitter
> > > > > <https://twitter.com/chronopost>.
> > > > >
> > > > > [image: DPD Group]
> > > > >
> > > >
> > >
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by Pushkar Deole <pd...@gmail.com>.
John,
is there KIP in progress for supporting Java HashMap also?
On Sun, May 10, 2020, 00:47 John Roesler <vv...@apache.org> wrote:
> Yes, that’s correct. It’s only for serializing the java type ‘byte[]’.
>
> On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> > Thanks John... I got to finish the work in few days so need to get it
> > quick, so looking for something ready. I will take a look at jackson
> json.
> >
> > By the way, what is the byteArrayserializer? As the name suggests, it is
> > for byte arrays so won't work for java ArrayList, right?
> >
> > On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org> wrote:
> >
> > > Hi Pushkar,
> > >
> > > If you’re not too concerned about compactness, I think Jackson json
> > > serialization is the easiest way to serialize complex types.
> > >
> > > There’s also a kip in progress to add a list serde. You might take a
> look
> > > at that proposal for ideas to write your own.
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > > > Won't say it's a good idea to use java serialized classes for
> messages,
> > > but
> > > > you should use a byteArraySerializer if you want to do such things
> > > >
> > > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a
> > > écrit :
> > > >
> > > > > Hi All,
> > > > >
> > > > > I have a requirement to store a record with key as java String and
> > > value as
> > > > > java's ArrayList in the kafka topic. Kafka has by default provided
> a
> > > > > StringSerializer and StringDeserializer, however for java
> ArrayList,
> > > how
> > > > > can get serializer. Do I need to write my own? Can someone share if
> > > someone
> > > > > already has written one?
> > > > >
> > > >
> > > >
> > > > --
> > > > *Nicolas Carlot*
> > > >
> > > > Lead dev
> > > > | | nicolas.carlot@chronopost.fr
> > > >
> > > >
> > > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage.
> La
> > > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
> > > >
> > > > [image: Logo Chronopost]
> > > > | chronopost.fr <http://www.chronopost.fr/>
> > > > Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et
> > > Twitter
> > > > <https://twitter.com/chronopost>.
> > > >
> > > > [image: DPD Group]
> > > >
> > >
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by John Roesler <vv...@apache.org>.
Yes, that’s correct. It’s only for serializing the java type ‘byte[]’.
On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote:
> Thanks John... I got to finish the work in few days so need to get it
> quick, so looking for something ready. I will take a look at jackson json.
>
> By the way, what is the byteArrayserializer? As the name suggests, it is
> for byte arrays so won't work for java ArrayList, right?
>
> On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org> wrote:
>
> > Hi Pushkar,
> >
> > If you’re not too concerned about compactness, I think Jackson json
> > serialization is the easiest way to serialize complex types.
> >
> > There’s also a kip in progress to add a list serde. You might take a look
> > at that proposal for ideas to write your own.
> >
> > Thanks,
> > John
> >
> > On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > > Won't say it's a good idea to use java serialized classes for messages,
> > but
> > > you should use a byteArraySerializer if you want to do such things
> > >
> > > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a
> > écrit :
> > >
> > > > Hi All,
> > > >
> > > > I have a requirement to store a record with key as java String and
> > value as
> > > > java's ArrayList in the kafka topic. Kafka has by default provided a
> > > > StringSerializer and StringDeserializer, however for java ArrayList,
> > how
> > > > can get serializer. Do I need to write my own? Can someone share if
> > someone
> > > > already has written one?
> > > >
> > >
> > >
> > > --
> > > *Nicolas Carlot*
> > >
> > > Lead dev
> > > | | nicolas.carlot@chronopost.fr
> > >
> > >
> > > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage. La
> > > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
> > >
> > > [image: Logo Chronopost]
> > > | chronopost.fr <http://www.chronopost.fr/>
> > > Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et
> > Twitter
> > > <https://twitter.com/chronopost>.
> > >
> > > [image: DPD Group]
> > >
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by Pushkar Deole <pd...@gmail.com>.
Thanks John... I got to finish the work in few days so need to get it
quick, so looking for something ready. I will take a look at jackson json.
By the way, what is the byteArrayserializer? As the name suggests, it is
for byte arrays so won't work for java ArrayList, right?
On Thu, May 7, 2020 at 8:44 PM John Roesler <vv...@apache.org> wrote:
> Hi Pushkar,
>
> If you’re not too concerned about compactness, I think Jackson json
> serialization is the easiest way to serialize complex types.
>
> There’s also a kip in progress to add a list serde. You might take a look
> at that proposal for ideas to write your own.
>
> Thanks,
> John
>
> On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> > Won't say it's a good idea to use java serialized classes for messages,
> but
> > you should use a byteArraySerializer if you want to do such things
> >
> > Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a
> écrit :
> >
> > > Hi All,
> > >
> > > I have a requirement to store a record with key as java String and
> value as
> > > java's ArrayList in the kafka topic. Kafka has by default provided a
> > > StringSerializer and StringDeserializer, however for java ArrayList,
> how
> > > can get serializer. Do I need to write my own? Can someone share if
> someone
> > > already has written one?
> > >
> >
> >
> > --
> > *Nicolas Carlot*
> >
> > Lead dev
> > | | nicolas.carlot@chronopost.fr
> >
> >
> > *Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage. La
> > nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
> >
> > [image: Logo Chronopost]
> > | chronopost.fr <http://www.chronopost.fr/>
> > Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et
> Twitter
> > <https://twitter.com/chronopost>.
> >
> > [image: DPD Group]
> >
>
Re: records with key as string and value as java ArrayList in topic
Posted by John Roesler <vv...@apache.org>.
Hi Pushkar,
If you’re not too concerned about compactness, I think Jackson json serialization is the easiest way to serialize complex types.
There’s also a kip in progress to add a list serde. You might take a look at that proposal for ideas to write your own.
Thanks,
John
On Thu, May 7, 2020, at 08:17, Nicolas Carlot wrote:
> Won't say it's a good idea to use java serialized classes for messages, but
> you should use a byteArraySerializer if you want to do such things
>
> Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a écrit :
>
> > Hi All,
> >
> > I have a requirement to store a record with key as java String and value as
> > java's ArrayList in the kafka topic. Kafka has by default provided a
> > StringSerializer and StringDeserializer, however for java ArrayList, how
> > can get serializer. Do I need to write my own? Can someone share if someone
> > already has written one?
> >
>
>
> --
> *Nicolas Carlot*
>
> Lead dev
> | | nicolas.carlot@chronopost.fr
>
>
> *Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage. La
> nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
>
> [image: Logo Chronopost]
> | chronopost.fr <http://www.chronopost.fr/>
> Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et Twitter
> <https://twitter.com/chronopost>.
>
> [image: DPD Group]
>
Re: records with key as string and value as java ArrayList in topic
Posted by Nicolas Carlot <ni...@chronopost.fr.INVALID>.
Won't say it's a good idea to use java serialized classes for messages, but
you should use a byteArraySerializer if you want to do such things
Le jeu. 7 mai 2020 à 14:32, Pushkar Deole <pd...@gmail.com> a écrit :
> Hi All,
>
> I have a requirement to store a record with key as java String and value as
> java's ArrayList in the kafka topic. Kafka has by default provided a
> StringSerializer and StringDeserializer, however for java ArrayList, how
> can get serializer. Do I need to write my own? Can someone share if someone
> already has written one?
>
--
*Nicolas Carlot*
Lead dev
| | nicolas.carlot@chronopost.fr
*Veuillez noter qu'à partir du 20 mai, le siège Chronopost déménage. La
nouvelle adresse est : 3 boulevard Romain Rolland 75014 Paris*
[image: Logo Chronopost]
| chronopost.fr <http://www.chronopost.fr/>
Suivez nous sur Facebook <https://fr-fr.facebook.com/chronopost> et Twitter
<https://twitter.com/chronopost>.
[image: DPD Group]