You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yun Tang (Jira)" <ji...@apache.org> on 2022/02/11 16:24:00 UTC
[jira] [Resolved] (FLINK-25478) Same materialized state handle should not register multi times
[ https://issues.apache.org/jira/browse/FLINK-25478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yun Tang resolved FLINK-25478.
------------------------------
Resolution: Fixed
Merged in master: ae166080fd1458b6e761abcf18d9302a4dcefbcd and a519ed1b11e7bf085d3eee6c6f39cab5967269d0
> Same materialized state handle should not register multi times
> --------------------------------------------------------------
>
> Key: FLINK-25478
> URL: https://issues.apache.org/jira/browse/FLINK-25478
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing, Runtime / State Backends
> Reporter: Yun Tang
> Assignee: Yun Tang
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> Currently, changelog materialization would call RocksDB state backend's snapshot method to generate {{IncrementalRemoteKeyedStateHandle}} as ChangelogStateBackendHandleImpl's materialized artifacts. And before next materialization, it will always report the same {{IncrementalRemoteKeyedStateHandle}} as before.
> It's fine to register this for the 1st time. However, for the 2nd time to register {{IncrementalRemoteKeyedStateHandle}} (via {{{}ChangelogStateBackendHandleImpl#registerSharedStates{}}}), it will discard the private state artifacts without check the register reference:
> IncrementalRemoteKeyedStateHandle:
> {code:java}
> public void discardState() throws Exception {
> try {
> StateUtil.bestEffortDiscardAllStateObjects(privateState.values());
> } catch (Exception e) {
> LOG.warn("Could not properly discard misc file states.", e);
> }
> }
> {code}
> Thus, this would delete the private state (such as RocksDB's MAINFEST), and once restore, job would not report FileNotFoundException.
>
> {code:java}
> Caused by: org.apache.flink.runtime.state.BackendBuildingException: Caught unexpected exception.
> at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:403)
> at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:477)
> at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:90)
> at org.apache.flink.state.changelog.ChangelogStateBackend.lambda$createKeyedStateBackend$1(ChangelogStateBackend.java:153)
> at org.apache.flink.state.changelog.restore.ChangelogBackendRestoreOperation.restore(ChangelogBackendRestoreOperation.java:68)
> at org.apache.flink.state.changelog.ChangelogStateBackend.restore(ChangelogStateBackend.java:221)
> at org.apache.flink.state.changelog.ChangelogStateBackend.createKeyedStateBackend(ChangelogStateBackend.java:145)
> at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328)
> at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
> at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
> at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
> at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
> ... 10 more
> Caused by: java.io.FileNotFoundException: xxxxx
> at org.apache.flink.core.fs.PluginFileSystemFactory$ClassLoaderFixingFileSystem.open(PluginFileSystemFactory.java:127)
> at org.apache.flink.runtime.state.filesystem.FileStateHandle.openInputStream(FileStateHandle.java:68)
> at org.apache.flink.contrib.streaming.state.RocksDBStateDownloader.downloadDataForStateHandle(RocksDBStateDownloader.java:127)
> at org.apache.flink.contrib.streaming.state.RocksDBStateDownloader.lambda$createDownloadRunnables$0(RocksDBStateDownloader.java:110)
> at org.apache.flink.util.function.ThrowingRunnable.lambda$unchecked$0(ThrowingRunnable.java:49)
> at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) ~[?:1.8.0_102]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) ~[?:1.8.0_102]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) ~[?:1.8.0_102]
> ... 1 more {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)