You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2026/08/10 01:43:18 UTC
[PR] [SPARK-58681][SQL][TESTS] Fix race in SparkSession job cancellation test [spark]
manuzhang opened a new pull request, #57884:
URL: https://github.com/apache/spark/pull/57884
### What changes were proposed in this pull request?
This pull request updates `SparkSessionJobTaggingAndCancellationSuite` so its test listener increments `jobEnded` before releasing the semaphore that wakes the test thread.
### Why are the changes needed?
`Semaphore.release()` could wake the test thread before `jobEnded.incrementAndGet()` completed. The test could then observe the previous counter value and fail intermittently with `2 did not equal 3`.
Updating the counter before releasing the semaphore establishes the required ordering between the listener callback and the waiting test thread.
JIRA: https://issues.apache.org/jira/browse/SPARK-58681
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The affected existing test passed twice with:
```
build/sbt 'sql/testOnly org.apache.spark.sql.SparkSessionJobTaggingAndCancellationSuite -- -z "Cancellation APIs in SparkSession are isolated"'
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58681][CORE][SQL][TESTS] Fix race in job cancellation tests [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57884:
URL: https://github.com/apache/spark/pull/57884#issuecomment-5463317751
Thank you @manuzhang and @dongjoon-hyun!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58681][CORE][SQL][TESTS] Fix race in job cancellation tests [spark]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #57884:
URL: https://github.com/apache/spark/pull/57884#issuecomment-5351033324
@uros-b @dongjoon-hyun Any more comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58681][SQL][TESTS] Fix race in SparkSession job cancellation test [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on code in PR #57884:
URL: https://github.com/apache/spark/pull/57884#discussion_r3746877119
##########
sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala:
##########
@@ -160,8 +160,8 @@ class SparkSessionJobTaggingAndCancellationSuite
}
override def onJobEnd(jobEnd: SparkListenerJobEnd): Unit = {
- sem.release()
jobEnded.incrementAndGet()
+ sem.release()
Review Comment:
Let's add a short comment on the ordering, to keep someone from swapping them back in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58681][CORE][SQL][TESTS] Fix race in job cancellation tests [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b closed pull request #57884: [SPARK-58681][CORE][SQL][TESTS] Fix race in job cancellation tests
URL: https://github.com/apache/spark/pull/57884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58681][CORE][SQL][TESTS] Fix race in job cancellation tests [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57884:
URL: https://github.com/apache/spark/pull/57884#issuecomment-5463322137
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/099a4534965befba19beca26bc5dfe90255706f3
- merged into branch-4.x https://github.com/apache/spark/commit/fb2360b2dfface0d51579a9e6866c9d9da85c9aa
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org