You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2026/02/24 23:09:19 UTC
[PR] [SPARK-55666][PYTHON][TEST] Fix flaky test_join_ambiguous_cols due to non-deterministic row order [spark]
Yicong-Huang opened a new pull request, #54460:
URL: https://github.com/apache/spark/pull/54460
### What changes were proposed in this pull request?
Sort the `collect()` results before comparing in `test_join_ambiguous_cols` to avoid non-deterministic row order failures.
### Why are the changes needed?
`test_join_ambiguous_cols` uses `assertEqual(cdf.collect(), sdf.collect())` to compare join results. Since joins do not guarantee row order, this can fail non-deterministically (flaky test).
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Existing test, with assertions fixed to use `sorted()`.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55666][PYTHON][TEST] Fix flaky test_join_ambiguous_cols due to non-deterministic row order [spark]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on PR #54460:
URL: https://github.com/apache/spark/pull/54460#issuecomment-3956069278
cc @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55666][PYTHON][TEST] Fix flaky connect tests due to non-deterministic row order [spark]
Posted by "ueshin (via GitHub)" <gi...@apache.org>.
ueshin commented on PR #54460:
URL: https://github.com/apache/spark/pull/54460#issuecomment-3962248661
Thanks! merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55666][PYTHON][TEST] Fix flaky connect tests due to non-deterministic row order [spark]
Posted by "ueshin (via GitHub)" <gi...@apache.org>.
ueshin closed pull request #54460: [SPARK-55666][PYTHON][TEST] Fix flaky connect tests due to non-deterministic row order
URL: https://github.com/apache/spark/pull/54460
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org