You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Elliot West <te...@gmail.com> on 2015/07/20 18:39:06 UTC
Encoders/Decoders and Serializers/Deserializers
I'm trying to understand the difference between Encoders/Decoders and
Serializers/Deserializers and when I should be using each. Specifically the
types I'm referring to are:
- kafka.serializer.Encoder
- kafka.serializer.Decoder
- org.apache.kafka.common.serialization.Serializer
- org.apache.kafka.common.serialization.Deserializer
Some background: I'd like to append a standard set of metadata to messages
to a number of topics in a manner that is agnostic to their encoding. I was
hoping to do this by providing [en|de]coders that inject/extract the
metadata and then delegate to user supplied [en|de]coders to handle the
encoding of the users actual payload.
Thanks - Elliot.
Re: Encoders/Decoders and Serializers/Deserializers
Posted by Elliot West <te...@gmail.com>.
Thanks Dan,
My current thinking is that perhaps they do the same job but Encoders are
specific to the Scala API whereas Serializers are the Java API equivalent.
It seems that the version I'm using (0.8.2.1) has a complete Java Producer
API, but not Consumer and that this may have been the source of my
confusion. I now have my test case working using a Serializer on one end
and Decoder on the other!
Elliot.
On 20 July 2015 at 17:59, Dan <da...@gmail.com> wrote:
> Linkedin/Confluent did a similar thing with their avro schema registry
>
> https://github.com/confluentinc/schema-registry/tree/master/avro-serializer/src/main/java/io/confluent/kafka/serializers
> which might help
>
> Not sure of the exact answer the to different between Encoder/Serializer
> though sorry.
>
> - Dan
>
>
> On 20 July 2015 at 17:39, Elliot West <te...@gmail.com> wrote:
>
> > I'm trying to understand the difference between Encoders/Decoders and
> > Serializers/Deserializers and when I should be using each. Specifically
> the
> > types I'm referring to are:
> >
> > - kafka.serializer.Encoder
> > - kafka.serializer.Decoder
> > - org.apache.kafka.common.serialization.Serializer
> > - org.apache.kafka.common.serialization.Deserializer
> >
> > Some background: I'd like to append a standard set of metadata to
> messages
> > to a number of topics in a manner that is agnostic to their encoding. I
> was
> > hoping to do this by providing [en|de]coders that inject/extract the
> > metadata and then delegate to user supplied [en|de]coders to handle the
> > encoding of the users actual payload.
> >
> > Thanks - Elliot.
> >
>
Re: Encoders/Decoders and Serializers/Deserializers
Posted by Dan <da...@gmail.com>.
Linkedin/Confluent did a similar thing with their avro schema registry
https://github.com/confluentinc/schema-registry/tree/master/avro-serializer/src/main/java/io/confluent/kafka/serializers
which might help
Not sure of the exact answer the to different between Encoder/Serializer
though sorry.
- Dan
On 20 July 2015 at 17:39, Elliot West <te...@gmail.com> wrote:
> I'm trying to understand the difference between Encoders/Decoders and
> Serializers/Deserializers and when I should be using each. Specifically the
> types I'm referring to are:
>
> - kafka.serializer.Encoder
> - kafka.serializer.Decoder
> - org.apache.kafka.common.serialization.Serializer
> - org.apache.kafka.common.serialization.Deserializer
>
> Some background: I'd like to append a standard set of metadata to messages
> to a number of topics in a manner that is agnostic to their encoding. I was
> hoping to do this by providing [en|de]coders that inject/extract the
> metadata and then delegate to user supplied [en|de]coders to handle the
> encoding of the users actual payload.
>
> Thanks - Elliot.
>