[jira] [Created] (KAFKA-13526) IQv2: Consider more generic logic for mapping between binary and typed queries

["John Roesler (Jira)" <ji...@apache.org>]

John Roesler created KAFKA-13526:
------------------------------------

             Summary: IQv2: Consider more generic logic for mapping between binary and typed queries
                 Key: KAFKA-13526
                 URL: https://issues.apache.org/jira/browse/KAFKA-13526
             Project: Kafka
          Issue Type: Sub-task
            Reporter: John Roesler


Right now, typed queries (like KeyQuery) need to be specially handled and translated to their binary counterparts (like RawKeyQuery). This happens in the Metered store layers, where the serdes are known. It is necessary because lower store layers are only able to handle binary data (because they don't know the serdes).

This situation is not ideal, since the Metered store layers will grow to host quite a bit of query handling and translation logic, because the relationship between typed queries and binary counterparts is not obvious, and because we can only automatically translate known query types. User-supplied queries and stores will have to work things out using their a-priori knowledge of the serdes.

 

One suggestion (from [~mjsax] ) is to come up with some kind of generic "query mapping" API, which the Metered stores would use to translate back and forth between typed and raw keys and values. Users would be able to supply their own mappings along with their custom queries.

Another option would be to have the Metered stores attach the serdes to the query on the way down and then to the result on the way up. Then, the serdes would be available in the bytes store (as part of the request) and to the users when they get their results (as part of the response).

Other options may also surface once we start playing with ideas.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)