[spark] branch master updated: [SPARK-45147][CORE] Remove `System.setSecurityManager` usage

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 4f74fc58148 [SPARK-45147][CORE] Remove `System.setSecurityManager` usage
4f74fc58148 is described below

commit 4f74fc581486a1a750b3bb27abc12e1a87215ea6
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Wed Sep 13 02:03:28 2023 -0700

    [SPARK-45147][CORE] Remove `System.setSecurityManager` usage
    
    ### What changes were proposed in this pull request?
    
    This PR aims to remove the deprecate Java `System.setSecurityManager` usage for Apache Spark 4.0.
    
    Note that this is the only usage in Apache Spark AS-IS code.
    ```
    $ git grep setSecurityManager
    core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala:      System.setSecurityManager(sm)
    core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala:      System.setSecurityManager(currentSm)
    ```
    
    This usage was added at `Apache Spark 1.5.0`.
    
    - #5841
    
    Since `Apache Spark 2.4.0`, we don't need `setSecurityManager` due to the following improvement.
    
    - #20925
    
    ### Why are the changes needed?
    
    ```
    $ java -version
    openjdk version "21-ea" 2023-09-19
    OpenJDK Runtime Environment (build 21-ea+32-2482)
    OpenJDK 64-Bit Server VM (build 21-ea+32-2482, mixed mode, sharing)
    ```
    
    ```
    max spark-3.5.0-bin-hadoop3:$ bin/spark-sql --help
    ...
    CLI options:
    Exception in thread "main" java.lang.UnsupportedOperationException:
    The Security Manager is deprecated and will be removed in a future release
      at java.base/java.lang.System.setSecurityManager(System.java:429)
      at org.apache.spark.deploy.SparkSubmitArguments.getSqlShellOptions(SparkSubmitArguments.scala:623)
    ```
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Manual test.
    ```
    $ build/sbt test:package -Phive -Phive-thriftserver
    $ bin/spark-sql --help
    ...
    CLI options:
     -d,--define <key=value>          Variable substitution to apply to Hive
                                      commands. e.g. -d A=B or --define A=B
        --database <databasename>     Specify the database to use
     -e <quoted-query-string>         SQL from command line
     -f <filename>                    SQL from files
     -H,--help                        Print help information
        --hiveconf <property=value>   Use value for given property
        --hivevar <key=value>         Variable substitution to apply to Hive
                                      commands. e.g. --hivevar A=B
     -i <filename>                    Initialization SQL file
     -S,--silent                      Silent mode in interactive shell
     -v,--verbose                     Verbose mode (echo executed SQL to the
                                      console)
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #42901 from dongjoon-hyun/SPARK-45147.
    
    Authored-by: Dongjoon Hyun <dh...@apple.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../apache/spark/deploy/SparkSubmitArguments.scala | 26 +---------------------
 1 file changed, 1 insertion(+), 25 deletions(-)

diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
index a3fe5153bee..867fc05cb8a 100644
--- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
@@ -18,7 +18,6 @@
 package org.apache.spark.deploy
 
 import java.io.{ByteArrayOutputStream, File, PrintStream}
-import java.lang.reflect.InvocationTargetException
 import java.nio.charset.StandardCharsets
 import java.util.{List => JList}
 
@@ -599,39 +598,17 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
   /**
    * Run the Spark SQL CLI main class with the "--help" option and catch its output. Then filter
    * the results to remove unwanted lines.
-   *
-   * Since the CLI will call `System.exit()`, we install a security manager to prevent that call
-   * from working, and restore the original one afterwards.
    */
   private def getSqlShellOptions(): String = {
     val currentOut = System.out
     val currentErr = System.err
-    val currentSm = System.getSecurityManager()
     try {
       val out = new ByteArrayOutputStream()
       val stream = new PrintStream(out)
       System.setOut(stream)
       System.setErr(stream)
 
-      val sm = new SecurityManager() {
-        override def checkExit(status: Int): Unit = {
-          throw new SecurityException()
-        }
-
-        override def checkPermission(perm: java.security.Permission): Unit = {}
-      }
-      System.setSecurityManager(sm)
-
-      try {
-        Utils.classForName(mainClass).getMethod("printUsage").invoke(null)
-      } catch {
-        case e: InvocationTargetException =>
-          // Ignore SecurityException, since we throw it above.
-          if (!e.getCause().isInstanceOf[SecurityException]) {
-            throw e
-          }
-      }
-
+      Utils.classForName(mainClass).getMethod("printUsage").invoke(null)
       stream.flush()
 
       // Get the output and discard any unnecessary lines from it.
@@ -641,7 +618,6 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
         }
         .mkString("\n")
     } finally {
-      System.setSecurityManager(currentSm)
       System.setOut(currentOut)
       System.setErr(currentErr)
     }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org