[flink] branch release-1.15 updated: [FLINK-27327][docs] Add description about changing max parallelism explicitly leads to state incompatibility

[ta...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tangyun pushed a commit to branch release-1.15
in repository https://gitbox.apache.org/repos/asf/flink.git


The following commit(s) were added to refs/heads/release-1.15 by this push:
     new 4b6bfbb68ed [FLINK-27327][docs] Add description about changing max parallelism explicitly leads to state incompatibility
4b6bfbb68ed is described below

commit 4b6bfbb68ed151e416095606859360ee6a144655
Author: Hangxiang Yu <ma...@gmail.com>
AuthorDate: Thu Apr 28 15:06:43 2022 +0800

    [FLINK-27327][docs] Add description about changing max parallelism explicitly leads to state incompatibility
---
 docs/content.zh/docs/dev/datastream/execution/parallel.md            | 5 ++++-
 docs/content/docs/dev/datastream/execution/parallel.md               | 2 ++
 docs/layouts/shortcodes/generated/pipeline_configuration.html        | 2 +-
 .../main/java/org/apache/flink/configuration/PipelineOptions.java    | 3 ++-
 4 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/docs/content.zh/docs/dev/datastream/execution/parallel.md b/docs/content.zh/docs/dev/datastream/execution/parallel.md
index 4999c47d858..216dea0281c 100644
--- a/docs/content.zh/docs/dev/datastream/execution/parallel.md
+++ b/docs/content.zh/docs/dev/datastream/execution/parallel.md
@@ -215,7 +215,10 @@ Python API 中尚不支持该特性。
 
 默认的最大并行度等于将 `operatorParallelism + (operatorParallelism / 2)` 值四舍五入到大于等于该值的一个整型值，并且这个整型值是 `2` 的幂次方，注意默认最大并行度下限为 `128`，上限为 `32768`。
 
-<span class="label label-danger">注意</span> 为最大并行度设置一个非常大的值将会降低性能，因为一些 state backends 需要维持内部的数据结构，而这些数据结构将会随着 key-groups 的数目而扩张（key-group 是状态重新分配的最小单元）。
+{{< hint warning >}} 
+为最大并行度设置一个非常大的值将会降低性能，因为一些 state backends 需要维持内部的数据结构，而这些数据结构将会随着 key-groups 的数目而扩张（key-group 是状态重新分配的最小单元）。
 
+从之前的作业恢复时，改变该作业的最大并发度将会导致状态不兼容。
+{{< /hint >}}
 
 {{< top >}}
diff --git a/docs/content/docs/dev/datastream/execution/parallel.md b/docs/content/docs/dev/datastream/execution/parallel.md
index 8125ec6b95f..0b0b7e67ca8 100644
--- a/docs/content/docs/dev/datastream/execution/parallel.md
+++ b/docs/content/docs/dev/datastream/execution/parallel.md
@@ -239,6 +239,8 @@ Setting the maximum parallelism to a very large
 value can be detrimental to performance because some state backends have to keep internal data
 structures that scale with the number of key-groups (which are the internal implementation mechanism for
 rescalable state).
+
+Changing the maximum parallelism explicitly when recovery from original job will lead to state incompatibility.
 {{< /hint >}}
 
 {{< top >}}
diff --git a/docs/layouts/shortcodes/generated/pipeline_configuration.html b/docs/layouts/shortcodes/generated/pipeline_configuration.html
index 9c30641de02..ae5ad33e214 100644
--- a/docs/layouts/shortcodes/generated/pipeline_configuration.html
+++ b/docs/layouts/shortcodes/generated/pipeline_configuration.html
@@ -84,7 +84,7 @@
             <td><h5>pipeline.max-parallelism</h5></td>
             <td style="word-wrap: break-word;">-1</td>
             <td>Integer</td>
-            <td>The program-wide maximum parallelism used for operators which haven't specified a maximum parallelism. The maximum parallelism specifies the upper limit for dynamic scaling and the number of key groups used for partitioned state.</td>
+            <td>The program-wide maximum parallelism used for operators which haven't specified a maximum parallelism. The maximum parallelism specifies the upper limit for dynamic scaling and the number of key groups used for partitioned state. Changing the value explicitly when recovery from original job will lead to state incompatibility.</td>
         </tr>
         <tr>
             <td><h5>pipeline.name</h5></td>
diff --git a/flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java b/flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java
index 0e04abf2b90..3a4e86d8b59 100644
--- a/flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java
+++ b/flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java
@@ -172,7 +172,8 @@ public class PipelineOptions {
                     .withDescription(
                             "The program-wide maximum parallelism used for operators which haven't specified a"
                                     + " maximum parallelism. The maximum parallelism specifies the upper limit for dynamic scaling and"
-                                    + " the number of key groups used for partitioned state.");
+                                    + " the number of key groups used for partitioned state."
+                                    + " Changing the value explicitly when recovery from original job will lead to state incompatibility.");
 
     public static final ConfigOption<Boolean> OBJECT_REUSE =
             key("pipeline.object-reuse")