You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by xt...@apache.org on 2022/08/05 01:58:12 UTC

[flink] 01/02: [hotfix] Add hybrid shuffle to taskmanager.memory.framework.off-heap.batch-shuffle.size description.

This is an automated email from the ASF dual-hosted git repository.

xtsong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git

commit 97e124116bd1a59d38bff08b733d9cea04748f31
Author: Weijie Guo <re...@163.com>
AuthorDate: Tue Aug 2 00:22:34 2022 +0800

    [hotfix] Add hybrid shuffle to taskmanager.memory.framework.off-heap.batch-shuffle.size description.
---
 docs/layouts/shortcodes/generated/common_memory_section.html          | 2 +-
 .../shortcodes/generated/task_manager_memory_configuration.html       | 2 +-
 .../main/java/org/apache/flink/configuration/TaskManagerOptions.java  | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/layouts/shortcodes/generated/common_memory_section.html b/docs/layouts/shortcodes/generated/common_memory_section.html
index 6e520e26378..75c3312256b 100644
--- a/docs/layouts/shortcodes/generated/common_memory_section.html
+++ b/docs/layouts/shortcodes/generated/common_memory_section.html
@@ -78,7 +78,7 @@
             <td><h5>taskmanager.memory.framework.off-heap.batch-shuffle.size</h5></td>
             <td style="word-wrap: break-word;">64 mb</td>
             <td>MemorySize</td>
-            <td>Size of memory used by blocking shuffle for shuffle data read (currently only used by sort-shuffle). Notes: 1) The memory is cut from 'taskmanager.memory.framework.off-heap.size' so must be smaller than that, which means you may also need to increase 'taskmanager.memory.framework.off-heap.size' after you increase this config value; 2) This memory size can influence the shuffle performance and you can increase this config value for large-scale batch jobs (for example, to 1 [...]
+            <td>Size of memory used by blocking shuffle for shuffle data read (currently only used by sort-shuffle and hybrid shuffle). Notes: 1) The memory is cut from 'taskmanager.memory.framework.off-heap.size' so must be smaller than that, which means you may also need to increase 'taskmanager.memory.framework.off-heap.size' after you increase this config value; 2) This memory size can influence the shuffle performance and you can increase this config value for large-scale batch jobs [...]
         </tr>
         <tr>
             <td><h5>taskmanager.memory.framework.off-heap.size</h5></td>
diff --git a/docs/layouts/shortcodes/generated/task_manager_memory_configuration.html b/docs/layouts/shortcodes/generated/task_manager_memory_configuration.html
index d572e802ced..99b3e9d04ce 100644
--- a/docs/layouts/shortcodes/generated/task_manager_memory_configuration.html
+++ b/docs/layouts/shortcodes/generated/task_manager_memory_configuration.html
@@ -24,7 +24,7 @@
             <td><h5>taskmanager.memory.framework.off-heap.batch-shuffle.size</h5></td>
             <td style="word-wrap: break-word;">64 mb</td>
             <td>MemorySize</td>
-            <td>Size of memory used by blocking shuffle for shuffle data read (currently only used by sort-shuffle). Notes: 1) The memory is cut from 'taskmanager.memory.framework.off-heap.size' so must be smaller than that, which means you may also need to increase 'taskmanager.memory.framework.off-heap.size' after you increase this config value; 2) This memory size can influence the shuffle performance and you can increase this config value for large-scale batch jobs (for example, to 1 [...]
+            <td>Size of memory used by blocking shuffle for shuffle data read (currently only used by sort-shuffle and hybrid shuffle). Notes: 1) The memory is cut from 'taskmanager.memory.framework.off-heap.size' so must be smaller than that, which means you may also need to increase 'taskmanager.memory.framework.off-heap.size' after you increase this config value; 2) This memory size can influence the shuffle performance and you can increase this config value for large-scale batch jobs [...]
         </tr>
         <tr>
             <td><h5>taskmanager.memory.framework.off-heap.size</h5></td>
diff --git a/flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java b/flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
index 2ad73336c95..8e4e67f0402 100644
--- a/flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
+++ b/flink-core/src/main/java/org/apache/flink/configuration/TaskManagerOptions.java
@@ -591,8 +591,8 @@ public class TaskManagerOptions {
                     .withDescription(
                             String.format(
                                     "Size of memory used by blocking shuffle for shuffle data read "
-                                            + "(currently only used by sort-shuffle). Notes: "
-                                            + "1) The memory is cut from '%s' so must be smaller than"
+                                            + "(currently only used by sort-shuffle and hybrid shuffle)."
+                                            + " Notes: 1) The memory is cut from '%s' so must be smaller than"
                                             + " that, which means you may also need to increase '%s' "
                                             + "after you increase this config value; 2) This memory"
                                             + " size can influence the shuffle performance and you "