You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "kckoh (via GitHub)" <gi...@apache.org> on 2026/08/17 05:05:22 UTC
[PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
kckoh opened a new pull request, #58042:
URL: https://github.com/apache/spark/pull/58042
### What changes were proposed in this pull request?
This PR increases the timeout for waiting for the `QueryTerminatedEvent` in
`ClientStreamingQuerySuite."listener events"` from 60 seconds to 180 seconds.
The test verifies that a Spark Connect streaming query listener receives start,
progress, and termination events. Since listener event delivery is asynchronous,
the termination event may take longer to be observed in slower or noisier CI
environments.
This is a minimal test-only change intended to check whether giving the
termination event more time is enough to deflake the test before considering a
larger change to the listener/test synchronization logic.
### Why are the changes needed?
`ClientStreamingQuerySuite."listener events"` is flaky in CI. The reported failure
shows that the test timed out while waiting for `listener.terminate` to become
non-empty. Increasing the timeout gives the asynchronous listener path more time to deliver the terminal event in
slow or noisy CI environments.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
i ran `
for i in {1..100}; do
echo "===== run $i ====="
./build/sbt "connect-client-jvm/testOnly org.apache.spark.sql.connect.streaming.ClientStreamingQuerySuite -- -z \"listener events\"" || break
done
`
and all the tests were successful in my local environment.
<img width="699" height="141" alt="Screenshot 2026-08-17 at 1 53 57 PM" src="https://github.com/user-attachments/assets/51eb20c7-31fd-4d26-8ace-7e9daf33c090" />
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
Posted by "kckoh (via GitHub)" <gi...@apache.org>.
kckoh closed pull request #58042: [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti…
URL: https://github.com/apache/spark/pull/58042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #58042:
URL: https://github.com/apache/spark/pull/58042#issuecomment-5313830819
LGTM, thank you @kckoh!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
Posted by "kckoh (via GitHub)" <gi...@apache.org>.
kckoh commented on PR #58042:
URL: https://github.com/apache/spark/pull/58042#issuecomment-5499758623
Yes! I looked it up on the PR and it seems reasonable to me. Let me close the pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
Posted by "sarutak (via GitHub)" <gi...@apache.org>.
sarutak commented on PR #58042:
URL: https://github.com/apache/spark/pull/58042#issuecomment-5317273480
Hi @kckoh, thank you for trying to fix this issue but I don't think this change will fixe the issue.
Actually, I’ve already identified the root cause of this issue. Under certain conditions, the assertion will never succeed, regardless of the timeout value.
Let me open another PR with a real solution later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #58042:
URL: https://github.com/apache/spark/pull/58042#issuecomment-5477158363
Thank you @sarutak for following up on this https://github.com/apache/spark/pull/58425. @kckoh Shall we close the current PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58000][CONNECT][TESTS] Increase listener events termination ti… [spark]
Posted by "kckoh (via GitHub)" <gi...@apache.org>.
kckoh commented on PR #58042:
URL: https://github.com/apache/spark/pull/58042#issuecomment-5321446001
Hello @sarutak,
i will be happy to see the solution for the flaky test. Let me know when you opened the PR.
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org