You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yun Tang (Jira)" <ji...@apache.org> on 2021/05/06 03:18:00 UTC
[jira] [Commented] (FLINK-22561) Restore checkpoint from another
cluster
[ https://issues.apache.org/jira/browse/FLINK-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17339975#comment-17339975 ]
Yun Tang commented on FLINK-22561:
----------------------------------
[~haoyan.zhang], could you use [Checkpoints#loadCheckpointMetadata|https://github.com/apache/flink/blob/c688bf3c83e72155ccf5d04fe397b7c0a1274fd1/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/Checkpoints.java#L99] to load your {{_metadata}} to see whether the shared path have been stored as absolute path. I believe that they should store as absolute path with hdfs schema, and if so, the problem might be your local hdfs client cannot download data from other cluster instead of looking into Flink.
BTW, if using incremental checkpoint, please be cautious to restore from another cluster as previous checkpoints might contain files still useful in newer checkpoints.
> Restore checkpoint from another cluster
> ---------------------------------------
>
> Key: FLINK-22561
> URL: https://issues.apache.org/jira/browse/FLINK-22561
> Project: Flink
> Issue Type: Improvement
> Environment: flink-1.12.1
> hadoop 3.0.0-cdh6.3.2
> Reporter: zhang haoyan
> Priority: Major
>
> Hi, we have two hadoop clusters and run some flink streaming job on it
> When we migrate some jobs from one cluster to another, we start the job just like:
> flink - s hdfs://clusterA/xx/xxx/chk-35 xxx.jar, but we will get an error like 'java.io.FileNotFoundException: File does not exist: /xxx/flink-checkpoints/cb46fc9afab25ced935e751dab49bee2/shared/329f6b79-292a-4d5d-b47b-a1cb63d02fe5',
> it seems can't read shared path from another cluster
> We don't want copy the share path manually every time and switch jobs between this two cluster will be very often.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)