[jira] [Created] (FLINK-15969) Do not multiplex both PersistedValue and PersistedTable with a single MapState state handle

["Tzu-Li (Gordon) Tai (Jira)" <ji...@apache.org>]

Tzu-Li (Gordon) Tai created FLINK-15969:
-------------------------------------------

             Summary: Do not multiplex both PersistedValue and PersistedTable with a single MapState state handle
                 Key: FLINK-15969
                 URL: https://issues.apache.org/jira/browse/FLINK-15969
             Project: Flink
          Issue Type: Improvement
          Components: Stateful Functions
    Affects Versions: statefun-1.1
            Reporter: Tzu-Li (Gordon) Tai
            Assignee: Tzu-Li (Gordon) Tai


Currently in Stateful Functions, {{PersistedValue}}s and {{PersistedTable}}s are multiplexed under a single {{MapState}}. I propose to split them up, and have them multiplexed with 2 separate {{MapState}}s, for the following reasons:
* There's already a problem with the (to-be-introduced) state reader / analyzer, that to read a single function's persisted state values, you have to iterate through ALL keys (which includes state of other functions) since we multiplex everything into a single handle.
* If you multiplex both tables and values into a single state handle, this will because even more of a problem in the future, say when the user just wants to read table state and not value state.
* If we do decide to separate the handles, we can slim down the {{MultiplexedStateKey}} type a bit, by having a separate {{MultiplexedTableStateKey}} that has a {{ByteString userKey}} field and a {{MultiplexedStateKey prefix}} field. There's already a minor concern with the way we use {{MultiplexedStateKey}}: Does protobuf repeated fields require some extra metadata written? If yes, its a tad bit redundant size-wise in this case since we only ever have 1 user key added.
* When multiplexing both value states and table state under the same state handle, the key is essentially ambiguous - there is a possibility that a value state's key in {{MapState}} can be set up to overwrite another key of a table state.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)