You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ta...@apache.org on 2022/08/31 08:57:51 UTC
[flink] branch release-1.15 updated: [FLINK-29130][state] Correct the doc description of local-recovery
This is an automated email from the ASF dual-hosted git repository.
tangyun pushed a commit to branch release-1.15
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.15 by this push:
new ad33fe8ef10 [FLINK-29130][state] Correct the doc description of local-recovery
ad33fe8ef10 is described below
commit ad33fe8ef104801a6470b11c792d6663120c172e
Author: Yun Tang <my...@live.com>
AuthorDate: Mon Aug 29 20:49:33 2022 +0800
[FLINK-29130][state] Correct the doc description of local-recovery
---
docs/content.zh/docs/ops/state/large_state_tuning.md | 4 ++--
.../shortcodes/generated/checkpointing_configuration.html | 4 ++--
.../shortcodes/generated/common_state_backends_section.html | 4 ++--
.../apache/flink/configuration/CheckpointingOptions.java | 13 ++++++-------
4 files changed, 12 insertions(+), 13 deletions(-)
diff --git a/docs/content.zh/docs/ops/state/large_state_tuning.md b/docs/content.zh/docs/ops/state/large_state_tuning.md
index 1bfd34c4d7e..39ebbe9e70b 100644
--- a/docs/content.zh/docs/ops/state/large_state_tuning.md
+++ b/docs/content.zh/docs/ops/state/large_state_tuning.md
@@ -292,10 +292,10 @@ also cover operator state and timers.*
The following state backends can support task-local recovery.
-- FsStateBackend: task-local recovery is supported for keyed state. The implementation will duplicate the state to a local file. This can introduce additional write costs
+- **HashMapStateBackend**: task-local recovery is supported for keyed state. The implementation will duplicate the state to a local file. This can introduce additional write costs
and occupy local disk space. In the future, we might also offer an implementation that keeps task-local state in memory.
-- RocksDBStateBackend: task-local recovery is supported for keyed state. For *full checkpoints*, state is duplicated to a local file. This can introduce additional write costs
+- **EmbeddedRocksDBStateBackend**: task-local recovery is supported for keyed state. For *full checkpoints*, state is duplicated to a local file. This can introduce additional write costs
and occupy local disk space. For *incremental snapshots*, the local state is based on RocksDB's native checkpointing mechanism. This mechanism is also used as the first step
to create the primary copy, which means that in this case no additional cost is introduced for creating the secondary copy. We simply keep the native checkpoint directory around
instead of deleting it after uploading to the distributed store. This local copy can share active files with the working directory of RocksDB (via hard links), so for active
diff --git a/docs/layouts/shortcodes/generated/checkpointing_configuration.html b/docs/layouts/shortcodes/generated/checkpointing_configuration.html
index ac90ceb5362..b817d7802f5 100644
--- a/docs/layouts/shortcodes/generated/checkpointing_configuration.html
+++ b/docs/layouts/shortcodes/generated/checkpointing_configuration.html
@@ -18,7 +18,7 @@
<td><h5>state.backend.local-recovery</h5></td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
- <td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends. Currently, the MemoryStateBackend does not support local recovery and ignores this option.</td>
+ <td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends (including both the EmbeddedRocksDBStateBackend and the HashMapStateBackend).</td>
</tr>
<tr>
<td><h5>state.checkpoint-storage</h5></td>
@@ -60,7 +60,7 @@
<td><h5>taskmanager.state.local.root-dirs</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>String</td>
- <td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend does not support local recovery and ignores this option. If not configured it will default to <WORKING_DIR>/localState. The <WORKING_DIR> can be configured via <code class="highlighter-rouge">process.taskmanager.working-dir</code></td>
+ <td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends. If not configured it will default to <WORKING_DIR>/localState. The <WORKING_DIR> can be configured via <code class="highlighter-rouge">process.taskmanager.working-dir</code></td>
</tr>
</tbody>
</table>
diff --git a/docs/layouts/shortcodes/generated/common_state_backends_section.html b/docs/layouts/shortcodes/generated/common_state_backends_section.html
index 2fbb39d993d..2d97f2d151e 100644
--- a/docs/layouts/shortcodes/generated/common_state_backends_section.html
+++ b/docs/layouts/shortcodes/generated/common_state_backends_section.html
@@ -42,7 +42,7 @@
<td><h5>state.backend.local-recovery</h5></td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
- <td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends. Currently, the MemoryStateBackend does not support local recovery and ignores this option.</td>
+ <td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends (including both the EmbeddedRocksDBStateBackend and the HashMapStateBackend).</td>
</tr>
<tr>
<td><h5>state.checkpoints.num-retained</h5></td>
@@ -54,7 +54,7 @@
<td><h5>taskmanager.state.local.root-dirs</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>String</td>
- <td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend does not support local recovery and ignores this option. If not configured it will default to <WORKING_DIR>/localState. The <WORKING_DIR> can be configured via <code class="highlighter-rouge">process.taskmanager.working-dir</code></td>
+ <td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends. If not configured it will default to <WORKING_DIR>/localState. The <WORKING_DIR> can be configured via <code class="highlighter-rouge">process.taskmanager.working-dir</code></td>
</tr>
</tbody>
</table>
diff --git a/flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java b/flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java
index c5ca3010772..b6713032859 100644
--- a/flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java
+++ b/flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java
@@ -142,8 +142,8 @@ public class CheckpointingOptions {
* This option configures local recovery for this state backend. By default, local recovery is
* deactivated.
*
- * <p>Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend
- * and HashMapStateBackend do not support local recovery and ignore this option.
+ * <p>Local recovery currently only covers keyed state backends (including both the
+ * EmbeddedRocksDBStateBackend and the HashMapStateBackend).
*/
@Documentation.Section(Documentation.Sections.COMMON_STATE_BACKENDS)
public static final ConfigOption<Boolean> LOCAL_RECOVERY =
@@ -152,8 +152,8 @@ public class CheckpointingOptions {
.defaultValue(false)
.withDescription(
"This option configures local recovery for this state backend. By default, local recovery is "
- + "deactivated. Local recovery currently only covers keyed state backends. Currently, the MemoryStateBackend "
- + "does not support local recovery and ignores this option.");
+ + "deactivated. Local recovery currently only covers keyed state backends "
+ + "(including both the EmbeddedRocksDBStateBackend and the HashMapStateBackend).");
/**
* The config parameter defining the root directories for storing file-based state for local
@@ -172,9 +172,8 @@ public class CheckpointingOptions {
.text(
"The config parameter defining the root directories for storing file-based "
+ "state for local recovery. Local recovery currently only covers keyed "
- + "state backends. Currently, MemoryStateBackend does not support local "
- + "recovery and ignores this option. If not configured it will default "
- + "to <WORKING_DIR>/localState. The <WORKING_DIR> can be configured via %s",
+ + "state backends. If not configured it will default to <WORKING_DIR>/localState. "
+ + "The <WORKING_DIR> can be configured via %s",
TextElement.code(
ClusterOptions
.TASK_MANAGER_PROCESS_WORKING_DIR_BASE