[jira] [Created] (FLINK-11055) Allow Queryable State to be transformed on the TaskManager before being returned to the client

["Galen Warren (JIRA)" <ji...@apache.org>]

Galen Warren created FLINK-11055:
------------------------------------

             Summary: Allow Queryable State to be transformed on the TaskManager before being returned to the client
                 Key: FLINK-11055
                 URL: https://issues.apache.org/jira/browse/FLINK-11055
             Project: Flink
          Issue Type: Improvement
          Components: Queryable State
            Reporter: Galen Warren
             Fix For: 1.7.0


The proposal here is to enhance the way Queryable State works to allow for the state object to be transformed on the TaskManager before being returned to the client. As an example, if some MapState<K,V> were made queryable, such a transform might look up a specific key in the map and return its corresponding value, resulting in only that value being returned to the client instead of the entire map. This could be useful in cases where the client only wants a portion of the state and the state is large (this is my use case).

At a high level, I think this could be accomplished by adding an (optional) serializable Function<TState,TTransformed> into KvStateRequest (and related classes?) and having that transform be applied in the QueryableStateServer (or QueryableStateClientProxy?). I expect some additional TypeInformation would also have to be supplied/used in places. It should be doable in a backwards compatible way such that if the client does not specify a transform it works exactly as it does now.

Would there be any interested in a PR for this? This would help me for something I'm currently working on and I'd be willing to take a crack at it. If there is interest, I'll be happy to do some more research to come up with a more concrete proposal.

Thanks for Flink - it's great!

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)