You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Josh Rosen (Jira)" <ji...@apache.org> on 2023/10/12 01:36:00 UTC
[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Rosen updated SPARK-45508:
-------------------------------
Summary: Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+ (was: org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+)
> Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-45508
> URL: https://issues.apache.org/jira/browse/SPARK-45508
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.5.0
> Reporter: Josh Rosen
> Priority: Major
>
> In JDK >= 9.b110, the code at [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213] hits a fallback path because we are using the wrong cleaner class name: `jdk.internal.ref.Cleaner` was removed in [https://bugs.openjdk.org/browse/JDK-8149925]
> This can be verified via
>
> ```
> val f = classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD")
> f.setAccessible(true)
> f.get(null)
> ```
> returning `null` instead of a method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org