[flink] branch master updated: [FLINK-25780][network] Reduce the maximum size of data output buffer per result partition for sort-shuffle

[ga...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gaoyunhaii pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git


The following commit(s) were added to refs/heads/master by this push:
     new dfa672f  [FLINK-25780][network] Reduce the maximum size of data output buffer per result partition for sort-shuffle
dfa672f is described below

commit dfa672f48d365ec938e21d0e15816e8d8de6eca4
Author: kevin.cyj <ke...@alibaba-inc.com>
AuthorDate: Mon Jan 24 20:23:13 2022 +0800

    [FLINK-25780][network] Reduce the maximum size of data output buffer per result partition for sort-shuffle
    
    The data output buffer of sort-shuffle is for better disk IO performance and currently, the total data output buffer size is 16M which is pretty big. However, blocking request too many buffers may influence performance. This patch reduces the maximum size of data output buffer from 16M to 8M to reduce the buffer request time. Note that the selected value is an empirical one based on the TPC-DS benchmark results.
    
    This closes #18471.
---
 .../flink/runtime/io/network/partition/SortMergeResultPartition.java  | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java b/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
index 36fde49..9456762 100644
--- a/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
+++ b/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
@@ -68,9 +68,9 @@ public class SortMergeResultPartition extends ResultPartition {
 
     /**
      * Number of expected buffer size to allocate for data writing. Currently, it is an empirical
-     * value (16M) which can not be configured.
+     * value (8M) which can not be configured.
      */
-    private static final int NUM_WRITE_BUFFER_BYTES = 16 * 1024 * 1024;
+    private static final int NUM_WRITE_BUFFER_BYTES = 8 * 1024 * 1024;
 
     private final Object lock = new Object();