You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Gyula Fora (JIRA)" <ji...@apache.org> on 2015/07/23 21:56:04 UTC

[jira] [Created] (FLINK-2403) Rework partitioned state checkpointing logic

Gyula Fora created FLINK-2403:
---------------------------------

             Summary: Rework partitioned state checkpointing logic
                 Key: FLINK-2403
                 URL: https://issues.apache.org/jira/browse/FLINK-2403
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
    Affects Versions: 0.10
            Reporter: Gyula Fora


In case of partitioned states, states are stored per-key (creating a statehandle for each key) so that we can later use this for repartitioning without fetching all the states. 

While this mechanism might be desired later it is extremely inefficient if we use a state backend such as HDFS as we are creating many small files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)