(spark) branch master updated: [SPARK-55289][SQL][FOLLOWUP] Fix flaky test in-order-by.sql by disabling broadcast join

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 411db440a446 [SPARK-55289][SQL][FOLLOWUP] Fix flaky test in-order-by.sql by disabling broadcast join
411db440a446 is described below

commit 411db440a446b9da06eded23a5e9c2af1aa87497
Author: Kent Yao <ke...@microsoft.com>
AuthorDate: Sat Mar 7 10:28:51 2026 -0800

    [SPARK-55289][SQL][FOLLOWUP] Fix flaky test in-order-by.sql by disabling broadcast join
    
    ### What changes were proposed in this pull request?
    
    Same fix as #54072 (SPARK-55289) for `in-set-operations.sql`. Adds `--SET spark.sql.autoBroadcastJoinThreshold=-1` to `in-order-by.sql` to prevent OOM from BroadcastHashJoin accumulating hash tables on memory-constrained CI runners.
    
    ### Why are the changes needed?
    
    `in-order-by.sql` intermittently fails on CI with `SparkOutOfMemoryError` for the same root cause as `in-set-operations.sql` — complex correlated IN-subqueries with multiple BroadcastHashJoin operations exceeding JVM heap under memory pressure.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. Test-only change.
    
    ### How was this patch tested?
    
    Golden files regenerated. Minor row reordering for rows with identical sort keys (expected when switching from BroadcastHashJoin to SortMergeJoin).
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Yes, co-authored with GitHub Copilot.
    
    Closes #54663 from yaooqinn/SPARK-55289-followup.
    
    Authored-by: Kent Yao <ke...@microsoft.com>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../resources/sql-tests/inputs/subquery/in-subquery/in-order-by.sql     | 1 +
 .../sql-tests/results/subquery/in-subquery/in-order-by.sql.out          | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-order-by.sql b/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-order-by.sql
index 8bf49a1c2d99..7fbb1c12924f 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-order-by.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-order-by.sql
@@ -1,5 +1,6 @@
 -- A test suite for ORDER BY in parent side, subquery, and both predicate subquery
 -- It includes correlated cases.
+--SET spark.sql.autoBroadcastJoinThreshold=-1
 
 -- Test sort operator with codegen on and off.
 --CONFIG_DIM1 spark.sql.codegen.wholeStage=true
diff --git a/sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-order-by.sql.out b/sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-order-by.sql.out
index d687b5938834..b06fd3dd6fc2 100644
--- a/sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-order-by.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-order-by.sql.out
@@ -227,8 +227,8 @@ struct<t1a:string,t1b:smallint,t1c:int,t1d:bigint,t1e:float,t1f:double,t1g:decim
 -- !query output
 val1d	NULL	16	22	17.0	25.0	2600	2014-06-04 01:01:00	NULL
 val1d	NULL	16	19	17.0	25.0	2600	2014-07-04 01:02:00.001	NULL
-val1a	16	12	21	15.0	20.0	2000	2014-06-04 01:02:00.001	2014-06-04
 val1a	16	12	10	15.0	20.0	2000	2014-07-04 01:01:00	2014-07-04
+val1a	16	12	21	15.0	20.0	2000	2014-06-04 01:02:00.001	2014-06-04
 
 
 -- !query


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org