You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2026/06/26 04:45:42 UTC

[PR] [DO-NOT-MERGE][SS][KAFKA] Work around JDK-8298446 C2 miscompilation causing IllegalMonitorStateException in Kafka tests on branch-4.2 [spark]

HyukjinKwon opened a new pull request, #56801:
URL: https://github.com/apache/spark/pull/56801

   ### What changes were proposed in this pull request?
   
   Add a single JVM workaround flag to the test args in `pom.xml`:
   
   ```
   -XX:CompileCommand=exclude,org.apache.kafka.clients.producer.KafkaProducer::waitOnMetadata
   ```
   
   This excludes only `KafkaProducer.waitOnMetadata` from JIT compilation in the test JVM, so it runs in the interpreter. It is preceded in `extraJavaTestArgs` by the pre-existing `-XX:+IgnoreUnrecognizedVMOptions`, so it is a no-op on JDKs that do not understand or do not need it.
   
   ### Why are the changes needed?
   
   The `branch-4.2` Maven (JDK 17) Kafka job fails ~227 tests, nearly the whole Kafka suite, with:
   
   ```
   java.lang.IllegalMonitorStateException: current thread is not owner
     at java.base/java.lang.Object.wait(Native Method)
     at org.apache.kafka.common.utils.SystemTime.waitObject(SystemTime.java:60)
     at org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:123)
     at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:1180)
     at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1027)
     ...
   ```
   
   `SystemTime.waitObject` in kafka-clients 3.9.2 calls `obj.wait()` correctly inside `synchronized (obj)`. Verified from the actual jar bytecode: `monitorenter` -> `Object.wait` (line 60) -> `monitorexit`. A correct JVM cannot throw `IllegalMonitorStateException` here.
   
   This is the HotSpot **C2 JIT bug [JDK-8298446](https://bugs.openjdk.org/browse/JDK-8298446)**: once C2 compiles `KafkaProducer.waitOnMetadata`, the monitor on the synchronized `obj.wait()` is dropped and every producer metadata wait throws "current thread is not owner". It is triggered nondeterministically by whether C2 compiles that hot method during the run, which is why the **identical** JDK (Zulu 17.0.19) and kafka-clients 3.9.2 pass on the master Maven Kafka job but wipe out the Kafka suite on the branch-4.2 run. Excluding that one method from JIT avoids the miscompilation.
   
   ### Does this PR introduce any user-facing change?
   
   No. Test-only change.
   
   ### How was this patch tested?
   
   Maven-on-fork (JDK 17) against `branch-4.2` with the fix, module matrix narrowed to the Kafka modules.
   
   #### CI evidence
   - Before (red): https://github.com/apache/spark/actions/runs/28182892330
   - After (green): https://github.com/HyukjinKwon/spark/actions/runs/28205023510
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Yes, Generated-by: Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [DO-NOT-MERGE][SS][KAFKA] Work around JDK-8298446 C2 miscompilation causing IllegalMonitorStateException in Kafka tests on branch-4.2 [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #56801: [DO-NOT-MERGE][SS][KAFKA] Work around JDK-8298446 C2 miscompilation causing IllegalMonitorStateException in Kafka tests on branch-4.2
URL: https://github.com/apache/spark/pull/56801


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [DO-NOT-MERGE][SS][KAFKA] Work around JDK-8298446 C2 miscompilation causing IllegalMonitorStateException in Kafka tests on branch-4.2 [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #56801:
URL: https://github.com/apache/spark/pull/56801#issuecomment-4808878096

   Closing in favor of #56803 (against `master`). Once that merges, the same fix will be backported to `branch-4.2` from the merged master commit, per the standard Spark master-first flow.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org