You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/02/14 00:10:12 UTC

(spark) branch master updated: [SPARK-51209][CORE] Improve `getCurrentUserName` to handle Java 24+

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new ffd58bca521a [SPARK-51209][CORE] Improve `getCurrentUserName` to handle Java 24+
ffd58bca521a is described below

commit ffd58bca521ad2bd3cab98aeeb736c62cab8de43
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Feb 13 16:10:00 2025 -0800

    [SPARK-51209][CORE] Improve `getCurrentUserName` to handle Java 24+
    
    ### What changes were proposed in this pull request?
    
    This PR aims to improve `Utils.getCurrentUserName` to handle Java 24+
    
    ### Why are the changes needed?
    
    To handle [JEP 486: Permanently Disable the Security Manager](https://openjdk.org/jeps/486) from Java 24+
    
    ### Does this PR introduce _any_ user-facing change?
    
    Previously, a user can provide the environment variable `SPARK_USER` as a workaround.
    This PR provides additional automated workaround by providing a fallback user name, `spark`.
    
    ### How was this patch tested?
    
    Without Java 24, it's difficult to make a unit test.
    
    Compiled with Java 21 and manual run on Java 24-ea.
    
    **BEFORE**
    ```
    $ bin/spark-shell
    ...
    25/02/13 14:27:39 ERROR Main: Failed to initialize Spark session.
    java.lang.UnsupportedOperationException: getSubject is not supported
            at java.base/javax.security.auth.Subject.getSubject(Subject.java:277)
            at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
            at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2446)
    ```
    
    **AFTER (Warning messages are omitted)**
    ```
    $ bin/spark-shell
    Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
    Setting default log level to "WARN".
    To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
    Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /___/ .__/\_,_/_/ /_/\_\   version 4.1.0-SNAPSHOT
          /_/
    
    Using Scala version 2.13.16 (OpenJDK 64-Bit Server VM, Java 24-ea)
    Type in expressions to have them evaluated.
    Type :help for more information.
    
    Spark context Web UI available at http://localhost:4040
    Spark context available as 'sc' (master = local[*], app id = local-1739485527653).
    Spark session available as 'spark'.
    
    scala>
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #49943 from dongjoon-hyun/SPARK-51209.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 core/src/main/scala/org/apache/spark/util/Utils.scala | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/core/src/main/scala/org/apache/spark/util/Utils.scala b/core/src/main/scala/org/apache/spark/util/Utils.scala
index c15a1f0d173c..4bcc96d5cf4c 100644
--- a/core/src/main/scala/org/apache/spark/util/Utils.scala
+++ b/core/src/main/scala/org/apache/spark/util/Utils.scala
@@ -2439,11 +2439,17 @@ private[spark] object Utils
 
   /**
    * Returns the current user name. This is the currently logged in user, unless that's been
-   * overridden by the `SPARK_USER` environment variable.
+   * overridden by the `SPARK_USER` environment variable. In case of exceptions, returns 'spark'.
    */
   def getCurrentUserName(): String = {
-    Option(System.getenv("SPARK_USER"))
-      .getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
+    try {
+      Option(System.getenv("SPARK_USER"))
+        .getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
+    } catch {
+      // JEP 486: Permanently Disable the Security Manager
+      case e: UnsupportedOperationException if e.getMessage().contains("getSubject") =>
+        "spark"
+    }
   }
 
   val EMPTY_USER_GROUPS = Set.empty[String]


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org