[spark] branch branch-3.0 updated: [SPARK-31055][DOCS] Update config docs for shuffle local host reads to have dep on external shuffle service

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.0 by this push:
     new 9caf009  [SPARK-31055][DOCS] Update config docs for shuffle local host reads to have dep on external shuffle service
9caf009 is described below

commit 9caf009ecd9041e71efe2ef56ca0e75cc94cb56e
Author: Thomas Graves <tg...@nvidia.com>
AuthorDate: Mon Mar 9 12:17:59 2020 -0700

    [SPARK-31055][DOCS] Update config docs for shuffle local host reads to have dep on external shuffle service
    
    ### What changes were proposed in this pull request?
    
    with SPARK-27651 we now support host local reads for shuffle, but only when external shuffle service is enabled. Update the config docs to state that.
    
    ### Why are the changes needed?
    
    clarify dependency
    
    ### Does this PR introduce any user-facing change?
    
    no
    
    ### How was this patch tested?
    
    n/a
    
    Closes #27812 from tgravescs/SPARK-27651-follow.
    
    Authored-by: Thomas Graves <tg...@nvidia.com>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit e807118eef9e0214170ff62c828524d237bd58e3)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 core/src/main/scala/org/apache/spark/internal/config/package.scala | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/core/src/main/scala/org/apache/spark/internal/config/package.scala b/core/src/main/scala/org/apache/spark/internal/config/package.scala
index 23c31a5..1308a46 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/package.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/package.scala
@@ -1135,7 +1135,8 @@ package object config {
 
   private[spark] val SHUFFLE_HOST_LOCAL_DISK_READING_ENABLED =
     ConfigBuilder("spark.shuffle.readHostLocalDisk")
-      .doc(s"If enabled (and `${SHUFFLE_USE_OLD_FETCH_PROTOCOL.key}` is disabled), shuffle " +
+      .doc(s"If enabled (and `${SHUFFLE_USE_OLD_FETCH_PROTOCOL.key}` is disabled and external " +
+        s"shuffle `${SHUFFLE_SERVICE_ENABLED.key}` is enabled), shuffle " +
         "blocks requested from those block managers which are running on the same host are read " +
         "from the disk directly instead of being fetched as remote blocks over the network.")
       .booleanConf


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org