[incubator-uniffle] branch master updated: [#585] feat(netty): Add MaxDirectMemorySize option for shuffle Server (#690)

[ro...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git


The following commit(s) were added to refs/heads/master by this push:
     new 8bf4c275 [#585] feat(netty): Add MaxDirectMemorySize option for shuffle Server (#690)
8bf4c275 is described below

commit 8bf4c275906105d17c1efbb48e2458f314ea9dd6
Author: roryqi <ro...@apache.org>
AuthorDate: Wed Mar 8 15:26:30 2023 +0800

    [#585] feat(netty): Add MaxDirectMemorySize option for shuffle Server (#690)
    
    ### What changes were proposed in this pull request?
    Add MaxDirectMemorySize option for shuffle Server. It will be convenient for Netty.
    
    ### Why are the changes needed?
    
    Fix: #585
    
    ### Does this PR introduce _any_ user-facing change?
    Yes. I will add the entire document of rss-env.sh in the future.
    
    ### How was this patch tested?
    Manual test
    
    Co-authored-by: roryqi <ro...@tencent.com>
---
 bin/rss-env.sh              | 1 +
 bin/start-shuffle-server.sh | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/bin/rss-env.sh b/bin/rss-env.sh
index 14824273..b46ec3af 100644
--- a/bin/rss-env.sh
+++ b/bin/rss-env.sh
@@ -31,3 +31,4 @@ XMX_SIZE="80g" # Shuffle Server JVM XMX size
 # RSS_PID_DIR, Where the pid file is stored (Default: ${RSS_HOME})
 # RSS_LOG_DIR, Where log files are stored (Default: ${RSS_HOME}/logs)
 # RSS_IP, IP address Shuffle Server binds to on this node (Default: first non-loopback ipv4)
+# MAX_DIRECT_MEMORY_SIZE Shuffle Server JVM off heap memory size (Default: not set)
diff --git a/bin/start-shuffle-server.sh b/bin/start-shuffle-server.sh
index e0b979e9..829a20db 100644
--- a/bin/start-shuffle-server.sh
+++ b/bin/start-shuffle-server.sh
@@ -66,9 +66,15 @@ JAVA_LIB_PATH="-Djava.library.path=$HADOOP_HOME/lib/native"
 
 echo "class path is $CLASSPATH"
 
+MAX_DIRECT_MEMORY_OPTS=""
+if [ -n "$MAX_DIRECT_MEMORY_SIZE" ]; then
+  MAX_DIRECT_MEMORY_OPTS="-XX:MaxDirectMemorySize=$MAX_DIRECT_MEMORY_SIZE"
+fi
+
 JVM_ARGS=" -server \
           -Xmx${XMX_SIZE} \
           -Xms${XMX_SIZE} \
+          ${MAX_DIRECT_MEMORY_OPTS} \
           -XX:+UseG1GC \
           -XX:MaxGCPauseMillis=200 \
           -XX:ParallelGCThreads=20 \