You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Matthias Pohl (Jira)" <ji...@apache.org> on 2023/03/28 11:56:00 UTC
[jira] [Comment Edited] (FLINK-31593) Update reference data for Migration Tests
[ https://issues.apache.org/jira/browse/FLINK-31593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705954#comment-17705954 ]
Matthias Pohl edited comment on FLINK-31593 at 3/28/23 11:55 AM:
-----------------------------------------------------------------
I'm moving the conversation with [~Yanfei Lei] (which happened initially in [FLINK-28440|https://issues.apache.org/jira/browse/FLINK-28440?focusedCommentId=17705953&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17705953]) in here:
{quote}
Matthias Pohl It seems that these are two problems. FLINK-28440 is the DSTL file of the changelog that can‘t be found, and FLINK-31593 is the rocksdb's file of the changelog state backend that can‘t be found.
Did the job restored from a native savepoint? currently, changelog does not support native savepoint( https://issues.apache.org/jira/browse/FLINK-29802).
I found some log in https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=47590&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba, and found the following log, it seems that job was restored from native savepoint.
11:35:10,088 [jobmanager-io-thread-1] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Reset the checkpoint ID of job 12f8ba2f80aca97baf6f50f5117c8128 to 3.11:35:10,088 [jobmanager-io-thread-1] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Restoring job 12f8ba2f80aca97baf6f50f5117c8128 from Savepoint 2 @ 0 for 12f8ba2f80aca97baf6f50f5117c8128 located at file:/__w/1/s/flink-tests/target/test-classes/stateful-scala-udf-migration-itcase-flink1.17-rocksdb-checkpoint.
{quote}
was (Author: mapohl):
I'm moving the conversation with [~Yanfei Lei] (which happened initially in FLINK-28440) in here:
{quote}
Matthias Pohl It seems that these are two problems. FLINK-28440 is the DSTL file of the changelog that can‘t be found, and FLINK-31593 is the rocksdb's file of the changelog state backend that can‘t be found.
Did the job restored from a native savepoint? currently, changelog does not support native savepoint( https://issues.apache.org/jira/browse/FLINK-29802).
I found some log in https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=47590&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba, and found the following log, it seems that job was restored from native savepoint.
11:35:10,088 [jobmanager-io-thread-1] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Reset the checkpoint ID of job 12f8ba2f80aca97baf6f50f5117c8128 to 3.11:35:10,088 [jobmanager-io-thread-1] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Restoring job 12f8ba2f80aca97baf6f50f5117c8128 from Savepoint 2 @ 0 for 12f8ba2f80aca97baf6f50f5117c8128 located at file:/__w/1/s/flink-tests/target/test-classes/stateful-scala-udf-migration-itcase-flink1.17-rocksdb-checkpoint.
{quote}
> Update reference data for Migration Tests
> -----------------------------------------
>
> Key: FLINK-31593
> URL: https://issues.apache.org/jira/browse/FLINK-31593
> Project: Flink
> Issue Type: Sub-task
> Reporter: Matthias Pohl
> Assignee: Matthias Pohl
> Priority: Major
> Labels: pull-request-available
>
> # Update {{CURRENT_VERSION in TypeSerializerUpgradeTestBase}} with the new version. This will likely fail some tests because snapshots are missing for that version. Generate them, for example in {{TypeSerializerUpgradeTestBase.}}
> # (major/minor only) Update migration tests in master to cover migration from new version: (search for usages of FlinkV{{{}ersion{}}})
> ** AbstractOperatorRestoreTestBase
> ** CEPMigrationTest
> ** BucketingSinkMigrationTest
> ** FlinkKafkaConsumerBaseMigrationTest
> ** ContinuousFileProcessingMigrationTest
> ** WindowOperatorMigrationTest
> ** StatefulJobSavepointMigrationITCase
> ** StatefulJobWBroadcastStateMigrationITCase
--
This message was sent by Atlassian Jira
(v8.20.10#820010)