You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "ulysses-you (via GitHub)" <gi...@apache.org> on 2026/09/02 01:24:02 UTC
[PR] [SPARK-59045][SQL][FOLLOWUP] Fix the subset join key test for the branch-4.3 backport [spark]
ulysses-you opened a new pull request, #58460:
URL: https://github.com/apache/spark/pull/58460
### What changes were proposed in this pull request?
Follow-up to the `branch-4.3` backport of #58335 (commit 0d253fef083). Sets `spark.sql.sources.v2.bucketing.requireAllClusterKeysForCoPartition` to false in the `SPARK-59045: compatible transforms reduce data type with subset join keys` test, which is what every other subset test in this suite already does on this branch.
### Why are the changes needed?
The backport broke this test on `branch-4.3`:
```
- SPARK-59045: compatible transforms reduce data type with subset join keys *** FAILED ***
List(Exchange hashpartitioning(id#L, 5), ENSURE_REQUIREMENTS, ...) was not empty
storage-partitioned join should not shuffle
```
On `master` the test plans without the config because SPARK-58558 relaxed `createKeyedShuffleSpec` from requiring the partition keys to exactly match the join keys to requiring them to cover the join keys. SPARK-58558 is not on this branch, so with the config at its default `createKeyedShuffleSpec` refuses the subset shape, both sides are shuffled, and the no-shuffle assertion fails.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The failing test now passes; ran `KeyGroupedPartitioningSuite`.
### Was this patch authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59045][SQL][4.3][FOLLOWUP] Fix the subset join key test [spark]
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #58460:
URL: https://github.com/apache/spark/pull/58460#issuecomment-5510729980
Thanks for the quick turnaround!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59045][SQL][4.3][FOLLOWUP] Fix the subset join key test [spark]
Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth commented on PR #58460:
URL: https://github.com/apache/spark/pull/58460#issuecomment-5505864711
Thank you @ulysses-you for the quick fix and @pan3793 for the review.
@HeartSaVioR, 4.3 CI should be unblocked now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59045][SQL][4.3][FOLLOWUP] Fix the subset join key test [spark]
Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth commented on PR #58460:
URL: https://github.com/apache/spark/pull/58460#issuecomment-5505843808
**Merge Summary:**
- merged into branch-4.3 https://github.com/apache/spark/commit/0ebff3892bfc35403aee4b3196978d9490187c50
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59045][SQL][4.3][FOLLOWUP] Fix the subset join key test [spark]
Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth closed pull request #58460: [SPARK-59045][SQL][4.3][FOLLOWUP] Fix the subset join key test
URL: https://github.com/apache/spark/pull/58460
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org