You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/05/12 16:22:24 UTC
(spark) branch master updated: [SPARK-56833][TESTS] Add `-XX:+EnableDynamicAgentLoading` to test JVM options to suppress JEP 451 warnings
This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 08a29819bf40 [SPARK-56833][TESTS] Add `-XX:+EnableDynamicAgentLoading` to test JVM options to suppress JEP 451 warnings
08a29819bf40 is described below
commit 08a29819bf4093fb756f3160b49e5f99500323a5
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Tue May 12 09:22:12 2026 -0700
[SPARK-56833][TESTS] Add `-XX:+EnableDynamicAgentLoading` to test JVM options to suppress JEP 451 warnings
### What changes were proposed in this pull request?
Add `-XX:+EnableDynamicAgentLoading` to the test-only JVM options in both SBT (`project/SparkBuild.scala`) and Maven (`pom.xml`).
### Why are the changes needed?
Per [JEP 451: Prepare to Disallow the Dynamic Loading of Agents](https://openjdk.org/jeps/451), JDK 21+ warns when a Java agent is loaded dynamically (e.g. Mockito self-attaching ByteBuddy), and a future release will disallow it by default. Setting this flag on the test JVM silences the warning today and keeps tests working once the default flips. Production JVM options are intentionally untouched since Spark itself does not dynamically attach agents.
Currently, Apache Spark CIs show the following message at ERROR level.
- Java 21: https://github.com/apache/spark/actions/runs/25592460535/job/75172442863
- Java 25: https://github.com/apache/spark/actions/runs/25664584995/job/75333859273
```
[info] Test run started (JUnit Jupiter)
[info] Test org.apache.spark.network.TransportRequestHandlerSuite#handleMergedBlockMetaRequest() started
[error] Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build as described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org.mockito/org/mockito/Mockito.html#0.3
WARNING: A Java agent has been loaded dynamically (/home/runner/.cache/coursier/v1/https/maven-central.storage-download.googleapis.com/maven2/net/bytebuddy/byte-buddy-agent/1.18.4/byte-buddy-agent-1.18.4.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs and manual tests.
**BEFORE**
```
$ build/sbt "network-common/testOnly *.TransportRequestHandlerSuite" > /dev/null
Using SPARK_LOCAL_IP=localhost
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by scala.runtime.LazyVals$ (file:/Users/dongjoon/APACHE/spark-merge/build/sbt-launch-1.12.8.jar)
WARNING: Please consider reporting this to the maintainers of class scala.runtime.LazyVals$
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: package sun.security.action not in java.base
WARNING: A Java agent has been loaded dynamically (/Users/dongjoon/.m2/repository/net/bytebuddy/byte-buddy-agent/1.18.4/byte-buddy-agent-1.18.4.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::allocateMemory has been called by io.netty.util.internal.PlatformDependent0$2 (file:/Users/dongjoon/.m2/repository/io/netty/netty-common/4.2.12.Final/netty-common-4.2.12.Final.jar)
WARNING: Please consider reporting this to the maintainers of class io.netty.util.internal.PlatformDependent0$2
WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
```
**AFTER**
```
$ build/sbt "network-common/testOnly *.TransportRequestHandlerSuite" > /dev/null
Using SPARK_LOCAL_IP=localhost
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by scala.runtime.LazyVals$ (file:/Users/dongjoon/APACHE/spark-merge/.claude/worktrees/youthful-sanderson-0fc91a/build/sbt-launch-1.12.8.jar)
WARNING: Please consider reporting this to the maintainers of class scala.runtime.LazyVals$
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: package sun.security.action not in java.base
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::allocateMemory has been called by io.netty.util.internal.PlatformDependent0$2 (file:/Users/dongjoon/.m2/repository/io/netty/netty-common/4.2.12.Final/netty-common-4.2.12.Final.jar)
WARNING: Please consider reporting this to the maintainers of class io.netty.util.internal.PlatformDependent0$2
WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.7 (1M context)
Closes #55825 from dongjoon-hyun/SPARK-56833.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
pom.xml | 1 +
project/SparkBuild.scala | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 750b44203944..fd55b9b631c2 100644
--- a/pom.xml
+++ b/pom.xml
@@ -342,6 +342,7 @@
-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE
-Dio.netty.noUnsafe=false
--enable-native-access=ALL-UNNAMED
+ -XX:+EnableDynamicAgentLoading
</extraJavaTestArgs>
<mariadb.java.client.version>3.5.7</mariadb.java.client.version>
<mysql.connector.version>9.6.0</mysql.connector.version>
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 66947a58ac59..49556aa1df18 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -1950,7 +1950,8 @@ object TestSettings {
"-Dio.netty.allocator.type=pooled",
"-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE",
"-Dio.netty.noUnsafe=false",
- "--enable-native-access=ALL-UNNAMED").mkString(" ")
+ "--enable-native-access=ALL-UNNAMED",
+ "-XX:+EnableDynamicAgentLoading").mkString(" ")
s"-Xmx$heapSize -Xss4m -XX:MaxMetaspaceSize=$metaspaceSize -XX:ReservedCodeCacheSize=128m -Dfile.encoding=UTF-8 $extraTestJavaArgs"
.split(" ").toSeq
},
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org