[spark] branch master updated: [SPARK-36374][FOLLOW-UP] Change config key spark.shuffle.server.mergedShuffleFileManagerImpl to spark.shuffle.push.server.mergedShuffleFileManagerImpl

[mr...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

mridulm80 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 7b2842e  [SPARK-36374][FOLLOW-UP] Change config key spark.shuffle.server.mergedShuffleFileManagerImpl to spark.shuffle.push.server.mergedShuffleFileManagerImpl
7b2842e is described below

commit 7b2842e98680e9231c2fa698fda81de21ccf3b23
Author: Venkata krishnan Sowrirajan <vs...@linkedin.com>
AuthorDate: Sun Aug 22 01:28:31 2021 -0500

    [SPARK-36374][FOLLOW-UP] Change config key spark.shuffle.server.mergedShuffleFileManagerImpl to spark.shuffle.push.server.mergedShuffleFileManagerImpl
    
    ### What changes were proposed in this pull request?
    
    Minor changes to change the config key name from `spark.shuffle.server.mergedShuffleFileManagerImpl` to `spark.shuffle.push.server.mergedShuffleFileManagerImpl`. This is missed out in https://github.com/apache/spark/pull/33615.
    
    ### Why are the changes needed?
    
    To keep the config names consistent
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, this is a change in the config key name. But the new config name changes are yet to be released. Technically there is no user facing change because of this change.
    
    ### How was this patch tested?
    
    Existing tests.
    
    Closes #33799 from venkata91/SPARK-36374-follow-up.
    
    Authored-by: Venkata krishnan Sowrirajan <vs...@linkedin.com>
    Signed-off-by: Mridul Muralidharan <mridul<at>gmail.com>
---
 .../main/java/org/apache/spark/network/util/TransportConf.java    | 2 +-
 .../src/main/scala/org/apache/spark/internal/config/package.scala | 8 ++++----
 docs/configuration.md                                             | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java b/common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java
index ed0ca918..bc507a4 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java
@@ -383,7 +383,7 @@ public class TransportConf {
    * 'org.apache.spark.network.shuffle.RemoteBlockPushResolver'.
    */
   public String mergedShuffleFileManagerImpl() {
-    return conf.get("spark.shuffle.server.mergedShuffleFileManagerImpl",
+    return conf.get("spark.shuffle.push.server.mergedShuffleFileManagerImpl",
       "org.apache.spark.network.shuffle.NoOpMergedShuffleFileManager");
   }
 
diff --git a/core/src/main/scala/org/apache/spark/internal/config/package.scala b/core/src/main/scala/org/apache/spark/internal/config/package.scala
index 7ed1f1d..4b7ba9b 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/package.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/package.scala
@@ -2098,10 +2098,10 @@ package object config {
   private[spark] val PUSH_BASED_SHUFFLE_ENABLED =
     ConfigBuilder("spark.shuffle.push.enabled")
       .doc("Set to true to enable push-based shuffle on the client side and this works in " +
-        "conjunction with the server side flag spark.shuffle.server.mergedShuffleFileManagerImpl " +
-        "which needs to be set with the appropriate " +
-        "org.apache.spark.network.shuffle.MergedShuffleFileManager implementation for push-based " +
-        "shuffle to be enabled")
+        "conjunction with the server side flag" +
+        " spark.shuffle.push.server.mergedShuffleFileManagerImpl which needs to be set with" +
+        " the appropriate org.apache.spark.network.shuffle.MergedShuffleFileManager" +
+        " implementation for push-based shuffle to be enabled")
       .version("3.2.0")
       .booleanConf
       .createWithDefault(false)
diff --git a/docs/configuration.md b/docs/configuration.md
index 0d8acf7..c8de478 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -3214,7 +3214,7 @@ Push-based shuffle helps improve the reliability and performance of spark shuffl
   <td><code>spark.shuffle.push.enabled</code></td>
   <td><code>false</code></td>
   <td>
-    Set to true to enable push-based shuffle on the client side and works in conjunction with the server side flag <code>spark.shuffle.server.mergedShuffleFileManagerImpl</code>.
+    Set to true to enable push-based shuffle on the client side and works in conjunction with the server side flag <code>spark.shuffle.push.server.mergedShuffleFileManagerImpl</code>.
   </td>
   <td>3.2.0</td>
 </tr>

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org