[jira] [Commented] (FLINK-3756) Introduce state hierarchy in CheckpointCoordinator

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240849#comment-15240849 ] 

ASF GitHub Bot commented on FLINK-3756:
---------------------------------------

GitHub user tillrohrmann opened a pull request:

    https://github.com/apache/flink/pull/1883

    [FLINK-3756] [state] Add state hierarchy to CheckpointCoordinator

    This commit introduces a state hierarchy for the StateForTask objects kept
    at the CheckpointCoordinator. Task states are now grouped together if they
    belong to the same ExecutionJobVertex. The StateForTask objects are now
    stored in so called StateForTaskGroup objects. The StateForTaskGroup object
    can also store the key group state handles associated to a ExecutionJobVertex.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tillrohrmann/flink stateHierarchy

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1883.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1883
    
----
commit c9217e7950397e94faadaec31142538311f8c050
Author: Till Rohrmann <tr...@apache.org>
Date:   2016-04-08T13:20:06Z

    [FLINK-3756] [state] Add state hierarchy to CheckpointCoordinator
    
    This commit introduces a state hierarchy for the StateForTask objects kept
    at the CheckpointCoordinator. Task states are now grouped together if they
    belong to the same ExecutionJobVertex. The StateForTask objects are now
    stored in so called StateForTaskGroup objects. The StateForTaskGroup object
    can also store the key group state handles associated to a ExecutionJobVertex.
    
    Adapt restore methods of CheckpointCoordinator and SavepointCoordinator
    
    Add state size computation
    
    Add comments

----


> Introduce state hierarchy in CheckpointCoordinator
> --------------------------------------------------
>
>                 Key: FLINK-3756
>                 URL: https://issues.apache.org/jira/browse/FLINK-3756
>             Project: Flink
>          Issue Type: Sub-task
>          Components: JobManager
>    Affects Versions: 1.1.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>             Fix For: 1.1.0
>
>
> Currently, the {{CheckpointCoordinator}} stores the state handles for all tasks in a list. This lack of structure does not allow to easily group all tasks belonging to the same {{ExecutionJobVertex}} together. With the introduction of key groups, it will be necessary to store the key-group state handles of the different tasks grouped together to make them easily accessible for redistribution. 
> Therefore, I propose to introduce the notion of a state for task groups which groups the state for all tasks belonging to the same {{ExecutionJobVertex}} together. This task group can then also be used to accumulate the key-group states from the different tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)