[jira] [Resolved] (FLINK-5590) Create a proper internal state hierarchy

["Stephan Ewen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephan Ewen resolved FLINK-5590.
---------------------------------
    Resolution: Fixed

Fixed in 3b97128f05bacfb80afe4a2a49741c31ff306cd2

> Create a proper internal state hierarchy
> ----------------------------------------
>
>                 Key: FLINK-5590
>                 URL: https://issues.apache.org/jira/browse/FLINK-5590
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.2.0
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>             Fix For: 1.3.0
>
>
> Currently, the state interfaces (like {{ListState}}, {{ValueState}}, {{ReducingState}}) are very sparse and contain only methods exposed to the users. That makes sense to keep the public stable API minimal
> At the same time, the runtime needs more methods for its internal interaction with state, such as:
>   - setting namespaces
>   - accessing raw values
>   - merging namespaces
> These are currently realized by re-creating or re-obtaining the state objects from the KeyedStateBackend. That method causes quite an overhead for each access to the state
> The KeyedStateBackend tries to do some tricks to reduce that overhead, but does it only partially and induces other overhead in the course.
> The root cause of all these issues is a problem in the design: There is no proper "internal state abstraction" in a similar way as there is an external state abstraction (the public state API).
> We should add a similar hierarchy of states for the internal methods. It would look like in the example below:
> {code}
>  *             State
>  *               |
>  *               +-------------------InternalKvState
>  *               |                         |
>  *          MergingState                   |
>  *               |                         |
>  *               +-----------------InternalMergingState
>  *               |                         |
>  *      +--------+------+                  |
>  *      |               |                  |
>  * ReducingState    ListState        +-----+-----------------+
>  *      |               |            |                       |
>  *      +-----------+   +-----------   -----------------InternalListState
>  *                  |                |
>  *                  +---------InternalReducingState
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)