You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/02/14 00:10:12 UTC
(spark) branch master updated: [SPARK-51209][CORE] Improve `getCurrentUserName` to handle Java 24+
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ffd58bca521a [SPARK-51209][CORE] Improve `getCurrentUserName` to handle Java 24+
ffd58bca521a is described below
commit ffd58bca521ad2bd3cab98aeeb736c62cab8de43
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Feb 13 16:10:00 2025 -0800
[SPARK-51209][CORE] Improve `getCurrentUserName` to handle Java 24+
### What changes were proposed in this pull request?
This PR aims to improve `Utils.getCurrentUserName` to handle Java 24+
### Why are the changes needed?
To handle [JEP 486: Permanently Disable the Security Manager](https://openjdk.org/jeps/486) from Java 24+
### Does this PR introduce _any_ user-facing change?
Previously, a user can provide the environment variable `SPARK_USER` as a workaround.
This PR provides additional automated workaround by providing a fallback user name, `spark`.
### How was this patch tested?
Without Java 24, it's difficult to make a unit test.
Compiled with Java 21 and manual run on Java 24-ea.
**BEFORE**
```
$ bin/spark-shell
...
25/02/13 14:27:39 ERROR Main: Failed to initialize Spark session.
java.lang.UnsupportedOperationException: getSubject is not supported
at java.base/javax.security.auth.Subject.getSubject(Subject.java:277)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2446)
```
**AFTER (Warning messages are omitted)**
```
$ bin/spark-shell
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 4.1.0-SNAPSHOT
/_/
Using Scala version 2.13.16 (OpenJDK 64-Bit Server VM, Java 24-ea)
Type in expressions to have them evaluated.
Type :help for more information.
Spark context Web UI available at http://localhost:4040
Spark context available as 'sc' (master = local[*], app id = local-1739485527653).
Spark session available as 'spark'.
scala>
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #49943 from dongjoon-hyun/SPARK-51209.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
core/src/main/scala/org/apache/spark/util/Utils.scala | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/util/Utils.scala b/core/src/main/scala/org/apache/spark/util/Utils.scala
index c15a1f0d173c..4bcc96d5cf4c 100644
--- a/core/src/main/scala/org/apache/spark/util/Utils.scala
+++ b/core/src/main/scala/org/apache/spark/util/Utils.scala
@@ -2439,11 +2439,17 @@ private[spark] object Utils
/**
* Returns the current user name. This is the currently logged in user, unless that's been
- * overridden by the `SPARK_USER` environment variable.
+ * overridden by the `SPARK_USER` environment variable. In case of exceptions, returns 'spark'.
*/
def getCurrentUserName(): String = {
- Option(System.getenv("SPARK_USER"))
- .getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
+ try {
+ Option(System.getenv("SPARK_USER"))
+ .getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
+ } catch {
+ // JEP 486: Permanently Disable the Security Manager
+ case e: UnsupportedOperationException if e.getMessage().contains("getSubject") =>
+ "spark"
+ }
}
val EMPTY_USER_GROUPS = Set.empty[String]
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org