You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "John Roesler (Jira)" <ji...@apache.org> on 2021/02/22 21:57:00 UTC
[jira] [Commented] (KAFKA-12242) Decouple state store
materialization enforcement from name/serde provider
[ https://issues.apache.org/jira/browse/KAFKA-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288646#comment-17288646 ]
John Roesler commented on KAFKA-12242:
--------------------------------------
Thanks for raising this, [~guozhang] , I have seen several people getting tripped up by this issue.
Just to throw it out there, another approach to solve this would be to begin to go down the road I proposed here: [https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+DSL+Grammar]
Specifically, what I have in mind is that attaching serdes to a processor node is logically independent from requesting materialization of that node's view. This ticket highlights the awkwardness of using a cross-cutting "Materialized" config at all. In contrast, if it were just "MapValuesParameters" with independent settings for specifying serdes and for requesting materialization, it wouldn't be an issue.
I definitely don't insist on the proposed grammar, but wanted to document the relationship with that proposal.
> Decouple state store materialization enforcement from name/serde provider
> -------------------------------------------------------------------------
>
> Key: KAFKA-12242
> URL: https://issues.apache.org/jira/browse/KAFKA-12242
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Priority: Major
> Labels: needs-kip
>
> Many users of Streams would want the following: let the Streams runtime to decide whether or not to materialize a state store; AND if it decides to do so, use the store name / serdes I provided ahead of time, if not, then nothing happens (the provided store name and serdes can just be dropped).
> However, Streams today take `Materialized` as an indicator to enforce the materialization. We should think of a way for users to optionally decouple materialization enforcement from name/serde provider.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)