You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Nicholas Jiang (Jira)" <ji...@apache.org> on 2021/05/09 23:25:00 UTC
[jira] [Comment Edited] (FLINK-22234) Read savepoint before
creating ExecutionGraph
[ https://issues.apache.org/jira/browse/FLINK-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17337149#comment-17337149 ]
Nicholas Jiang edited comment on FLINK-22234 at 5/9/21, 11:24 PM:
------------------------------------------------------------------
[~trohrmann], [~chesnay], the behavior that read savepoint before creating ExecutionGraph could be that create the CheckpointStorage, create the CheckpointStorageCoordinatorView with the create storage, reading potential savepoints, checkpoints, extract their meta information and then create an ExecutionGraph. After the creation of ExecutionGraph, create the CheckpointCoordinator with the created CheckpointStorageCoordinatorVIew. Because given with the CheckpointStorage and JobID, the coordinator view could be generated, which could pass to CheckpointCoordinator, reduce the time reading the savepoint.
[~trohrmann], [~chesnay], what do you think about this change?
was (Author: nicholasjiang):
[~trohrmann], [~chesnay], the behavior that read savepoint before creating ExecutionGraph could be that create the CheckpointStorage, create the CheckpointStorageCoordinatorView with the create storage, reading potential savepoints, checkpoints, extract their meta information and then create an ExecutionGraph. After the creation of ExecutionGraph, create the CheckpointCoordinator with the created CheckpointStorageCoordinatorVIew. Because given with the CheckpointStorage and JobID, the coordinator view could be generated, which could pass to CheckpointCoordinator, reduce the time reading the savepoint.
[~trohrmann], [~chesnay], what do you think about?
> Read savepoint before creating ExecutionGraph
> ---------------------------------------------
>
> Key: FLINK-22234
> URL: https://issues.apache.org/jira/browse/FLINK-22234
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: Chesnay Schepler
> Priority: Major
> Labels: pull-request-available
>
> Savepoints are currently read when the ExecutionGraph is created.
> This is causing problems when determining the (max) parallelism the job is run with, because the savepoint may impose additional restrictions.
> This caused quite a few headaches in FLINK-21844, and further requires the execution vertices to have a mutable max parallelism.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)