You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Bart van Deenen <ba...@fastmail.fm> on 2019/10/28 12:34:58 UTC

serdeProps what does that mean

Hi all

I need a custom serde for Kafka. Our serde needs a SchemaStore instance with some state, and I'm trying to figure out how to get this into the serde.
I'm trying to follow along with this example
https://github.com/apache/kafka/blob/ad5f31c3e6868757eab9c8dd71efd5fd65f06ac6/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewTypedDemo.java#L102

and I don't understand what the configure method does. What is it for, and what can I stick in the map? Following along with the example, I see that `serdeProps` sets the `tClass` field to the class that you want to (de)serialize. Why?

Thanks

Bart

Re: serdeProps what does that mean

Posted by "Matthias J. Sax" <ma...@confluent.io>.
I guess that should be possible. Kafka Streams would not (de)serialize
it's configuration `Properties` but forward them as pass-by-reference to
the `Serde#configure()`.


-Matthias


On 10/31/19 5:36 AM, Bart van Deenen wrote:
> Hi I'm sorry for the confusion.
> 
> My question should be 'are there limitations on what you can put in this configuration map' ?
> 
> Could you put objects with state in the map for instance? Will this map be serialized ?
> 
> Thanks
> 
> On Thu, Oct 31, 2019, at 10:35, Matthias J. Sax wrote:
>> Well, the `configure()` method is there to configure the serde :)
>>
>> Hence, it might be a good place to instantiate the state of the serde,
>> ie, to create a `SchemaStore` instance.
>>
>> For the passed in map: the full `StreamsConfig` will be passed into it,
>> hence, you can add any configuration you want to the global
>> `KafkaStreams` configuration and it will be forwarded.
>>
>> Hope this helps.
>>
>>
>> -Matthias
>>
>> On 10/28/19 5:34 AM, Bart van Deenen wrote:
>>> Hi all
>>>
>>> I need a custom serde for Kafka. Our serde needs a SchemaStore instance with some state, and I'm trying to figure out how to get this into the serde.
>>> I'm trying to follow along with this example
>>> https://github.com/apache/kafka/blob/ad5f31c3e6868757eab9c8dd71efd5fd65f06ac6/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewTypedDemo.java#L102
>>>
>>> and I don't understand what the configure method does. What is it for, and what can I stick in the map? Following along with the example, I see that `serdeProps` sets the `tClass` field to the class that you want to (de)serialize. Why?
>>>
>>> Thanks
>>>
>>> Bart
>>>
>>
>>
>> Attachments:
>> * signature.asc

Re: serdeProps what does that mean

Posted by Bart van Deenen <ba...@fastmail.fm>.
Hi I'm sorry for the confusion.

My question should be 'are there limitations on what you can put in this configuration map' ?

Could you put objects with state in the map for instance? Will this map be serialized ?

Thanks

On Thu, Oct 31, 2019, at 10:35, Matthias J. Sax wrote:
> Well, the `configure()` method is there to configure the serde :)
> 
> Hence, it might be a good place to instantiate the state of the serde,
> ie, to create a `SchemaStore` instance.
> 
> For the passed in map: the full `StreamsConfig` will be passed into it,
> hence, you can add any configuration you want to the global
> `KafkaStreams` configuration and it will be forwarded.
> 
> Hope this helps.
> 
> 
> -Matthias
> 
> On 10/28/19 5:34 AM, Bart van Deenen wrote:
> > Hi all
> > 
> > I need a custom serde for Kafka. Our serde needs a SchemaStore instance with some state, and I'm trying to figure out how to get this into the serde.
> > I'm trying to follow along with this example
> > https://github.com/apache/kafka/blob/ad5f31c3e6868757eab9c8dd71efd5fd65f06ac6/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewTypedDemo.java#L102
> > 
> > and I don't understand what the configure method does. What is it for, and what can I stick in the map? Following along with the example, I see that `serdeProps` sets the `tClass` field to the class that you want to (de)serialize. Why?
> > 
> > Thanks
> > 
> > Bart
> > 
> 
> 
> Attachments:
> * signature.asc

Re: serdeProps what does that mean

Posted by "Matthias J. Sax" <ma...@confluent.io>.
Well, the `configure()` method is there to configure the serde :)

Hence, it might be a good place to instantiate the state of the serde,
ie, to create a `SchemaStore` instance.

For the passed in map: the full `StreamsConfig` will be passed into it,
hence, you can add any configuration you want to the global
`KafkaStreams` configuration and it will be forwarded.

Hope this helps.


-Matthias

On 10/28/19 5:34 AM, Bart van Deenen wrote:
> Hi all
> 
> I need a custom serde for Kafka. Our serde needs a SchemaStore instance with some state, and I'm trying to figure out how to get this into the serde.
> I'm trying to follow along with this example
> https://github.com/apache/kafka/blob/ad5f31c3e6868757eab9c8dd71efd5fd65f06ac6/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview/PageViewTypedDemo.java#L102
> 
> and I don't understand what the configure method does. What is it for, and what can I stick in the map? Following along with the example, I see that `serdeProps` sets the `tClass` field to the class that you want to (de)serialize. Why?
> 
> Thanks
> 
> Bart
>