[jira] [Created] (KAFKA-14202) IQv2: Expose binary store schema to store implementations

["John Roesler (Jira)" <ji...@apache.org>]

John Roesler created KAFKA-14202:
------------------------------------

             Summary: IQv2: Expose binary store schema to store implementations
                 Key: KAFKA-14202
                 URL: https://issues.apache.org/jira/browse/KAFKA-14202
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: John Roesler


One feature of IQv2 is that store implementations can handle custom queries. Many custom query handlers will need to process the key or value bytes, for example deserializing them to implement some filter or aggregations, or even performing binary operations on them.

For the most part, this should be straightforward for users, since they provide Streams with the serdes, the store implementation, and the custom queries.

However, Streams will sometimes pack extra data around the data produced by the user-provided serdes. For example, the Timestamped store wrappers add a timestamp on the beginning of the value byte array. And in Windowed stores, we add window timestamps to the key bytes.

It would be nice to have some generic mechanism to communicate those schemas to the user-provided inner store layers to support users who need to write custom queries. For example, perhaps we can add an extractor class to the state store context



--
This message was sent by Atlassian Jira
(v8.20.10#820010)