(spark) branch master updated: [SPARK-51165][CORE] Enable `spark.master.rest.enabled` by default

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 6a4e0438582a [SPARK-51165][CORE] Enable `spark.master.rest.enabled` by default
6a4e0438582a is described below

commit 6a4e0438582a746d0ed61bf7c5ab3fea25a5947a
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Tue Feb 11 17:04:37 2025 -0800

    [SPARK-51165][CORE] Enable `spark.master.rest.enabled` by default
    
    ### What changes were proposed in this pull request?
    
    This PR aims to enable `spark.master.rest.enabled` by default for Apache Spark 4.1.0.
    
    ### Why are the changes needed?
    
    Apache Spark is ready to enable this feature by default.
    - Since Apache Spark 1.3.0, `spark.master.rest.enabled` has been used stably.
    - Since Apache Spark 4.0.0, `spark.master.rest.filters` provides a way to serve it securely.
      - #47595
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, the migration guide is updated.
    
    ### How was this patch tested?
    
    Pass the CIs.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #49894 from dongjoon-hyun/SPARK-51165.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 core/src/main/scala/org/apache/spark/internal/config/package.scala | 2 +-
 docs/core-migration-guide.md                                       | 4 ++++
 docs/spark-standalone.md                                           | 2 +-
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/core/src/main/scala/org/apache/spark/internal/config/package.scala b/core/src/main/scala/org/apache/spark/internal/config/package.scala
index 5bf7e5f9536c..3ce374d0477d 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/package.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/package.scala
@@ -1973,7 +1973,7 @@ package object config {
   private[spark] val MASTER_REST_SERVER_ENABLED = ConfigBuilder("spark.master.rest.enabled")
     .version("1.3.0")
     .booleanConf
-    .createWithDefault(false)
+    .createWithDefault(true)
 
   private[spark] val MASTER_REST_SERVER_HOST = ConfigBuilder("spark.master.rest.host")
     .doc("Specifies the host of the Master REST API endpoint")
diff --git a/docs/core-migration-guide.md b/docs/core-migration-guide.md
index 7f92fa14f170..914c48a09582 100644
--- a/docs/core-migration-guide.md
+++ b/docs/core-migration-guide.md
@@ -22,6 +22,10 @@ license: |
 * Table of contents
 {:toc}
 
+## Upgrading from Core 4.0 to 4.1
+
+- Since Spark 4.1, Spark Master deamon provides REST API by default. To restore the behavior before Spark 4.1, you can set `spark.master.rest.enabled` to `false`.
+
 ## Upgrading from Core 3.5 to 4.0
 
 - Since Spark 4.0, Spark migrated all its internal reference of servlet API from `javax` to `jakarta` 
diff --git a/docs/spark-standalone.md b/docs/spark-standalone.md
index 51f6ca977c99..5a8eb3f1e060 100644
--- a/docs/spark-standalone.md
+++ b/docs/spark-standalone.md
@@ -237,7 +237,7 @@ SPARK_MASTER_OPTS supports the following system properties:
 </tr>
 <tr>
   <td><code>spark.master.rest.enabled</code></td>
-  <td><code>false</code></td>
+  <td><code>true</code></td>
   <td>
     Whether to use the Master REST API endpoint or not.
   </td>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org