You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/07/07 09:17:25 UTC
[spark] branch master updated: [SPARK-32577][SQL][TEST] Fix the
config value for shuffled hash join in test in-joins.sql
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new f3c1159 [SPARK-32577][SQL][TEST] Fix the config value for shuffled hash join in test in-joins.sql
f3c1159 is described below
commit f3c11595ce433db8d950f9e951237b17f40f677f
Author: Cheng Su <ch...@fb.com>
AuthorDate: Wed Jul 7 18:16:23 2021 +0900
[SPARK-32577][SQL][TEST] Fix the config value for shuffled hash join in test in-joins.sql
### What changes were proposed in this pull request?
We found the `in-join.sql` does not test shuffled hash join properly in https://issues.apache.org/jira/browse/SPARK-32577, but didn't find a good way to fix it. Given we now have a test config to enforce shuffled hash join in https://github.com/apache/spark/pull/33182, we can fix the test here now as well.
### Why are the changes needed?
Fix test to have better test coverage.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Reran the test to compare the output, and verified the query plan manually to make sure shuffled hash join being used.
Closes #33236 from c21/join-test.
Authored-by: Cheng Su <ch...@fb.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
.../test/resources/sql-tests/inputs/subquery/in-subquery/in-joins.sql | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-joins.sql b/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-joins.sql
index 2353560..08eeb1d 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-joins.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-joins.sql
@@ -7,7 +7,7 @@
--CONFIG_DIM1 spark.sql.autoBroadcastJoinThreshold=10485760
--CONFIG_DIM1 spark.sql.autoBroadcastJoinThreshold=-1,spark.sql.join.preferSortMergeJoin=true
---CONFIG_DIM1 spark.sql.autoBroadcastJoinThreshold=-1,spark.sql.join.preferSortMergeJoin=false
+--CONFIG_DIM1 spark.sql.autoBroadcastJoinThreshold=-1,spark.sql.join.forceApplyShuffledHashJoin=true
--CONFIG_DIM2 spark.sql.codegen.wholeStage=true
--CONFIG_DIM2 spark.sql.codegen.wholeStage=false,spark.sql.codegen.factoryMode=CODEGEN_ONLY
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org