(spark) branch branch-4.x updated: [SPARK-59248][SQL][FOLLOWUP] Update a stale exchange count in a SPARK-59050 test

[ul...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

ulysses-you pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new c49acd75048b [SPARK-59248][SQL][FOLLOWUP] Update a stale exchange count in a SPARK-59050 test
c49acd75048b is described below

commit c49acd75048b043df312db7eb2d79cf9b28eba2e
Author: Xiduo You <ul...@gmail.com>
AuthorDate: Tue Sep 8 14:26:51 2026 +0800

    [SPARK-59248][SQL][FOLLOWUP] Update a stale exchange count in a SPARK-59050 test
    
    ### What changes were proposed in this pull request?
    
    Update the exchange-count pin in the SPARK-59050 test "genuine unknown keys survive an inner join on a key subset" from 4 to 3, and spell out which exchanges remain.
    
    ### Why are the changes needed?
    
    SPARK-59248 (#58522) broke this test on master. In that query, `x`'s partition key `k` is column-pruned from its scan output (the join matches on `id` alone). Before SPARK-59248, the pruned key dropped `x`'s whole key-grouped partitioning report, so `x`'s side of the `r JOIN x` join shuffled on its own; with SPARK-59248 the surviving `(id)` projection pairs with `r`'s one-side keyed shuffle and `x`'s exchange disappears.
    
    The test's answer check still passes, including the genuine unknown-key row `(1, 'zz')`; only the count was stale. The suite is `ExtendedSQLTest`, and the "sql - extended tests" job that failed on master does not run on PR CI, which is why #58522 itself was green.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    The updated test, plus the full `KeyGroupedPartitioningSuite` locally (172/172).
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (GLM 5.3)
    
    Closes #58586 from ulysses-you/spark-59248-followup.
    
    Authored-by: Xiduo You <ul...@gmail.com>
    Signed-off-by: Xiduo You <ul...@apache.org>
    (cherry picked from commit f957459c9aececb4787d57eac9505189e2cc6768)
    Signed-off-by: Xiduo You <ul...@apache.org>
---
 .../apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala    | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
index 8d20faf1a93b..65982e5eb3ec 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala
@@ -7479,9 +7479,13 @@ class KeyGroupedPartitioningSuite
         checkAnswer(df, expected)
         // Pin the total exchange count so later silent shuffles surface: the marked (id, k)
         // side refuses to pair on the subset key, cascading one-side shuffles around it.
+        // The three exchanges shuffle t onto a's (id, k) layout, r onto x's (id) layout, and
+        // the marked side onto u's (id, k) layout. x itself does not shuffle: its key k is
+        // pruned from its scan output and the surviving (id) projection still pairs
+        // (SPARK-59248).
         val plan = df.queryExecution.executedPlan
         val shuffles = collectAllShuffles(plan)
-        assert(shuffles.size == 4, s"expected 4 exchanges, got ${shuffles.size}:\n$plan")
+        assert(shuffles.size == 3, s"expected 3 exchanges, got ${shuffles.size}:\n$plan")
       }
     }
   }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org