You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2022/08/27 19:59:22 UTC
[spark] branch branch-3.2 updated: [SPARK-40124][SQL][TEST][3.2] Update TPCDS v1.4 q32 for Plan Stability tests
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 817c2039f4d [SPARK-40124][SQL][TEST][3.2] Update TPCDS v1.4 q32 for Plan Stability tests
817c2039f4d is described below
commit 817c2039f4d2ed888dfd3d2d557d11aa4b9138d8
Author: Kapil Kumar Singh <ka...@microsoft.com>
AuthorDate: Sat Aug 27 12:59:11 2022 -0700
[SPARK-40124][SQL][TEST][3.2] Update TPCDS v1.4 q32 for Plan Stability tests
### What changes were proposed in this pull request?
This is port of SPARK-40124 to Spark 3.2. Fix query 32 for TPCDS v1.4
### Why are the changes needed?
Current q32.sql seems to be wrong. It is just selection `1`.
Reference for query template: https://github.com/databricks/tpcds-kit/blob/eff5de2c30337b71cc0dc1976147742d2c65d378/query_templates/query32.tpl#L41
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Test change only
Closes #37675 from mskapilks/change-q32-3.2.
Authored-by: Kapil Kumar Singh <ka...@microsoft.com>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../approved-plans-v1_4/q32.sf100/explain.txt | 120 ++++++++++++---------
.../approved-plans-v1_4/q32.sf100/simplified.txt | 94 ++++++++--------
.../approved-plans-v1_4/q32/explain.txt | 120 ++++++++++++---------
.../approved-plans-v1_4/q32/simplified.txt | 92 ++++++++--------
.../resources/tpcds-query-results/v1_4/q32.sql.out | 4 +-
sql/core/src/test/resources/tpcds/q32.sql | 2 +-
6 files changed, 236 insertions(+), 196 deletions(-)
diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt
index d67ded6a42b..c1f2b9c8fd8 100644
--- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt
+++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/explain.txt
@@ -1,31 +1,33 @@
== Physical Plan ==
-CollectLimit (27)
-+- * Project (26)
- +- * BroadcastHashJoin Inner BuildRight (25)
- :- * Project (23)
- : +- * BroadcastHashJoin Inner BuildLeft (22)
- : :- BroadcastExchange (18)
- : : +- * Project (17)
- : : +- * BroadcastHashJoin Inner BuildLeft (16)
- : : :- BroadcastExchange (5)
- : : : +- * Project (4)
- : : : +- * Filter (3)
- : : : +- * ColumnarToRow (2)
- : : : +- Scan parquet default.item (1)
- : : +- * Filter (15)
- : : +- * HashAggregate (14)
- : : +- Exchange (13)
- : : +- * HashAggregate (12)
- : : +- * Project (11)
- : : +- * BroadcastHashJoin Inner BuildRight (10)
- : : :- * Filter (8)
- : : : +- * ColumnarToRow (7)
- : : : +- Scan parquet default.catalog_sales (6)
- : : +- ReusedExchange (9)
- : +- * Filter (21)
- : +- * ColumnarToRow (20)
- : +- Scan parquet default.catalog_sales (19)
- +- ReusedExchange (24)
+* HashAggregate (29)
++- Exchange (28)
+ +- * HashAggregate (27)
+ +- * Project (26)
+ +- * BroadcastHashJoin Inner BuildRight (25)
+ :- * Project (23)
+ : +- * BroadcastHashJoin Inner BuildLeft (22)
+ : :- BroadcastExchange (18)
+ : : +- * Project (17)
+ : : +- * BroadcastHashJoin Inner BuildLeft (16)
+ : : :- BroadcastExchange (5)
+ : : : +- * Project (4)
+ : : : +- * Filter (3)
+ : : : +- * ColumnarToRow (2)
+ : : : +- Scan parquet default.item (1)
+ : : +- * Filter (15)
+ : : +- * HashAggregate (14)
+ : : +- Exchange (13)
+ : : +- * HashAggregate (12)
+ : : +- * Project (11)
+ : : +- * BroadcastHashJoin Inner BuildRight (10)
+ : : :- * Filter (8)
+ : : : +- * ColumnarToRow (7)
+ : : : +- Scan parquet default.catalog_sales (6)
+ : : +- ReusedExchange (9)
+ : +- * Filter (21)
+ : +- * ColumnarToRow (20)
+ : +- Scan parquet default.catalog_sales (19)
+ +- ReusedExchange (24)
(1) Scan parquet default.item
@@ -65,7 +67,7 @@ Input [3]: [cs_item_sk#3, cs_ext_discount_amt#4, cs_sold_date_sk#5]
Input [3]: [cs_item_sk#3, cs_ext_discount_amt#4, cs_sold_date_sk#5]
Condition : isnotnull(cs_item_sk#3)
-(9) ReusedExchange [Reuses operator id: 32]
+(9) ReusedExchange [Reuses operator id: 34]
Output [1]: [d_date_sk#7]
(10) BroadcastHashJoin [codegen id : 3]
@@ -133,10 +135,10 @@ Right keys [1]: [cs_item_sk#14]
Join condition: (cast(cs_ext_discount_amt#15 as decimal(14,7)) > (1.3 * avg(cs_ext_discount_amt))#13)
(23) Project [codegen id : 6]
-Output [1]: [cs_sold_date_sk#16]
+Output [2]: [cs_ext_discount_amt#15, cs_sold_date_sk#16]
Input [5]: [i_item_sk#1, (1.3 * avg(cs_ext_discount_amt))#13, cs_item_sk#14, cs_ext_discount_amt#15, cs_sold_date_sk#16]
-(24) ReusedExchange [Reuses operator id: 32]
+(24) ReusedExchange [Reuses operator id: 34]
Output [1]: [d_date_sk#17]
(25) BroadcastHashJoin [codegen id : 6]
@@ -145,44 +147,58 @@ Right keys [1]: [d_date_sk#17]
Join condition: None
(26) Project [codegen id : 6]
-Output [1]: [1 AS excess discount amount #18]
-Input [2]: [cs_sold_date_sk#16, d_date_sk#17]
-
-(27) CollectLimit
-Input [1]: [excess discount amount #18]
-Arguments: 100
+Output [1]: [cs_ext_discount_amt#15]
+Input [3]: [cs_ext_discount_amt#15, cs_sold_date_sk#16, d_date_sk#17]
+
+(27) HashAggregate [codegen id : 6]
+Input [1]: [cs_ext_discount_amt#15]
+Keys: []
+Functions [1]: [partial_sum(UnscaledValue(cs_ext_discount_amt#15))]
+Aggregate Attributes [1]: [sum#18]
+Results [1]: [sum#19]
+
+(28) Exchange
+Input [1]: [sum#19]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4]
+
+(29) HashAggregate [codegen id : 7]
+Input [1]: [sum#19]
+Keys: []
+Functions [1]: [sum(UnscaledValue(cs_ext_discount_amt#15))]
+Aggregate Attributes [1]: [sum(UnscaledValue(cs_ext_discount_amt#15))#20]
+Results [1]: [MakeDecimal(sum(UnscaledValue(cs_ext_discount_amt#15))#20,17,2) AS excess discount amount#21]
===== Subqueries =====
Subquery:1 Hosting operator id = 6 Hosting Expression = cs_sold_date_sk#5 IN dynamicpruning#6
-BroadcastExchange (32)
-+- * Project (31)
- +- * Filter (30)
- +- * ColumnarToRow (29)
- +- Scan parquet default.date_dim (28)
+BroadcastExchange (34)
++- * Project (33)
+ +- * Filter (32)
+ +- * ColumnarToRow (31)
+ +- Scan parquet default.date_dim (30)
-(28) Scan parquet default.date_dim
-Output [2]: [d_date_sk#7, d_date#19]
+(30) Scan parquet default.date_dim
+Output [2]: [d_date_sk#7, d_date#22]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-01-27), LessThanOrEqual(d_date,2000-04-26), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(29) ColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#7, d_date#19]
+(31) ColumnarToRow [codegen id : 1]
+Input [2]: [d_date_sk#7, d_date#22]
-(30) Filter [codegen id : 1]
-Input [2]: [d_date_sk#7, d_date#19]
-Condition : (((isnotnull(d_date#19) AND (d_date#19 >= 2000-01-27)) AND (d_date#19 <= 2000-04-26)) AND isnotnull(d_date_sk#7))
+(32) Filter [codegen id : 1]
+Input [2]: [d_date_sk#7, d_date#22]
+Condition : (((isnotnull(d_date#22) AND (d_date#22 >= 2000-01-27)) AND (d_date#22 <= 2000-04-26)) AND isnotnull(d_date_sk#7))
-(31) Project [codegen id : 1]
+(33) Project [codegen id : 1]
Output [1]: [d_date_sk#7]
-Input [2]: [d_date_sk#7, d_date#19]
+Input [2]: [d_date_sk#7, d_date#22]
-(32) BroadcastExchange
+(34) BroadcastExchange
Input [1]: [d_date_sk#7]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
Subquery:2 Hosting operator id = 19 Hosting Expression = cs_sold_date_sk#16 IN dynamicpruning#6
diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt
index 8ca9bf49029..27e630265a3 100644
--- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt
+++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32.sf100/simplified.txt
@@ -1,48 +1,52 @@
-CollectLimit
- WholeStageCodegen (6)
- Project
- BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
- Project [cs_sold_date_sk]
- BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))]
- InputAdapter
- BroadcastExchange #1
- WholeStageCodegen (4)
- Project [i_item_sk,(1.3 * avg(cs_ext_discount_amt))]
- BroadcastHashJoin [i_item_sk,cs_item_sk]
- InputAdapter
- BroadcastExchange #2
- WholeStageCodegen (1)
- Project [i_item_sk]
- Filter [i_manufact_id,i_item_sk]
- ColumnarToRow
- InputAdapter
- Scan parquet default.item [i_item_sk,i_manufact_id]
- Filter [(1.3 * avg(cs_ext_discount_amt))]
- HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count]
- InputAdapter
- Exchange [cs_item_sk] #3
- WholeStageCodegen (3)
- HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count]
- Project [cs_item_sk,cs_ext_discount_amt]
- BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
- Filter [cs_item_sk]
+WholeStageCodegen (7)
+ HashAggregate [sum] [sum(UnscaledValue(cs_ext_discount_amt)),excess discount amount,sum]
+ InputAdapter
+ Exchange #1
+ WholeStageCodegen (6)
+ HashAggregate [cs_ext_discount_amt] [sum,sum]
+ Project [cs_ext_discount_amt]
+ BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
+ Project [cs_ext_discount_amt,cs_sold_date_sk]
+ BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))]
+ InputAdapter
+ BroadcastExchange #2
+ WholeStageCodegen (4)
+ Project [i_item_sk,(1.3 * avg(cs_ext_discount_amt))]
+ BroadcastHashJoin [i_item_sk,cs_item_sk]
+ InputAdapter
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ Project [i_item_sk]
+ Filter [i_manufact_id,i_item_sk]
ColumnarToRow
InputAdapter
- Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #4
- WholeStageCodegen (1)
- Project [d_date_sk]
- Filter [d_date,d_date_sk]
- ColumnarToRow
- InputAdapter
- Scan parquet default.date_dim [d_date_sk,d_date]
- InputAdapter
- ReusedExchange [d_date_sk] #4
- Filter [cs_item_sk,cs_ext_discount_amt]
- ColumnarToRow
+ Scan parquet default.item [i_item_sk,i_manufact_id]
+ Filter [(1.3 * avg(cs_ext_discount_amt))]
+ HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count]
+ InputAdapter
+ Exchange [cs_item_sk] #4
+ WholeStageCodegen (3)
+ HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count]
+ Project [cs_item_sk,cs_ext_discount_amt]
+ BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
+ Filter [cs_item_sk]
+ ColumnarToRow
+ InputAdapter
+ Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #5
+ WholeStageCodegen (1)
+ Project [d_date_sk]
+ Filter [d_date,d_date_sk]
+ ColumnarToRow
+ InputAdapter
+ Scan parquet default.date_dim [d_date_sk,d_date]
+ InputAdapter
+ ReusedExchange [d_date_sk] #5
+ Filter [cs_item_sk,cs_ext_discount_amt]
+ ColumnarToRow
+ InputAdapter
+ Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
InputAdapter
- Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- InputAdapter
- ReusedExchange [d_date_sk] #4
+ ReusedExchange [d_date_sk] #5
diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt
index f38b3802bac..0e2e356b8cb 100644
--- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt
+++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/explain.txt
@@ -1,31 +1,33 @@
== Physical Plan ==
-CollectLimit (27)
-+- * Project (26)
- +- * BroadcastHashJoin Inner BuildRight (25)
- :- * Project (23)
- : +- * BroadcastHashJoin Inner BuildRight (22)
- : :- * Project (10)
- : : +- * BroadcastHashJoin Inner BuildRight (9)
- : : :- * Filter (3)
- : : : +- * ColumnarToRow (2)
- : : : +- Scan parquet default.catalog_sales (1)
- : : +- BroadcastExchange (8)
- : : +- * Project (7)
- : : +- * Filter (6)
- : : +- * ColumnarToRow (5)
- : : +- Scan parquet default.item (4)
- : +- BroadcastExchange (21)
- : +- * Filter (20)
- : +- * HashAggregate (19)
- : +- Exchange (18)
- : +- * HashAggregate (17)
- : +- * Project (16)
- : +- * BroadcastHashJoin Inner BuildRight (15)
- : :- * Filter (13)
- : : +- * ColumnarToRow (12)
- : : +- Scan parquet default.catalog_sales (11)
- : +- ReusedExchange (14)
- +- ReusedExchange (24)
+* HashAggregate (29)
++- Exchange (28)
+ +- * HashAggregate (27)
+ +- * Project (26)
+ +- * BroadcastHashJoin Inner BuildRight (25)
+ :- * Project (23)
+ : +- * BroadcastHashJoin Inner BuildRight (22)
+ : :- * Project (10)
+ : : +- * BroadcastHashJoin Inner BuildRight (9)
+ : : :- * Filter (3)
+ : : : +- * ColumnarToRow (2)
+ : : : +- Scan parquet default.catalog_sales (1)
+ : : +- BroadcastExchange (8)
+ : : +- * Project (7)
+ : : +- * Filter (6)
+ : : +- * ColumnarToRow (5)
+ : : +- Scan parquet default.item (4)
+ : +- BroadcastExchange (21)
+ : +- * Filter (20)
+ : +- * HashAggregate (19)
+ : +- Exchange (18)
+ : +- * HashAggregate (17)
+ : +- * Project (16)
+ : +- * BroadcastHashJoin Inner BuildRight (15)
+ : :- * Filter (13)
+ : : +- * ColumnarToRow (12)
+ : : +- Scan parquet default.catalog_sales (11)
+ : +- ReusedExchange (14)
+ +- ReusedExchange (24)
(1) Scan parquet default.catalog_sales
@@ -89,7 +91,7 @@ Input [3]: [cs_item_sk#7, cs_ext_discount_amt#8, cs_sold_date_sk#9]
Input [3]: [cs_item_sk#7, cs_ext_discount_amt#8, cs_sold_date_sk#9]
Condition : isnotnull(cs_item_sk#7)
-(14) ReusedExchange [Reuses operator id: 32]
+(14) ReusedExchange [Reuses operator id: 34]
Output [1]: [d_date_sk#10]
(15) BroadcastHashJoin [codegen id : 3]
@@ -133,10 +135,10 @@ Right keys [1]: [cs_item_sk#7]
Join condition: (cast(cs_ext_discount_amt#2 as decimal(14,7)) > (1.3 * avg(cs_ext_discount_amt))#16)
(23) Project [codegen id : 6]
-Output [1]: [cs_sold_date_sk#3]
+Output [2]: [cs_ext_discount_amt#2, cs_sold_date_sk#3]
Input [5]: [cs_ext_discount_amt#2, cs_sold_date_sk#3, i_item_sk#5, (1.3 * avg(cs_ext_discount_amt))#16, cs_item_sk#7]
-(24) ReusedExchange [Reuses operator id: 32]
+(24) ReusedExchange [Reuses operator id: 34]
Output [1]: [d_date_sk#17]
(25) BroadcastHashJoin [codegen id : 6]
@@ -145,44 +147,58 @@ Right keys [1]: [d_date_sk#17]
Join condition: None
(26) Project [codegen id : 6]
-Output [1]: [1 AS excess discount amount #18]
-Input [2]: [cs_sold_date_sk#3, d_date_sk#17]
-
-(27) CollectLimit
-Input [1]: [excess discount amount #18]
-Arguments: 100
+Output [1]: [cs_ext_discount_amt#2]
+Input [3]: [cs_ext_discount_amt#2, cs_sold_date_sk#3, d_date_sk#17]
+
+(27) HashAggregate [codegen id : 6]
+Input [1]: [cs_ext_discount_amt#2]
+Keys: []
+Functions [1]: [partial_sum(UnscaledValue(cs_ext_discount_amt#2))]
+Aggregate Attributes [1]: [sum#18]
+Results [1]: [sum#19]
+
+(28) Exchange
+Input [1]: [sum#19]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4]
+
+(29) HashAggregate [codegen id : 7]
+Input [1]: [sum#19]
+Keys: []
+Functions [1]: [sum(UnscaledValue(cs_ext_discount_amt#2))]
+Aggregate Attributes [1]: [sum(UnscaledValue(cs_ext_discount_amt#2))#20]
+Results [1]: [MakeDecimal(sum(UnscaledValue(cs_ext_discount_amt#2))#20,17,2) AS excess discount amount#21]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (32)
-+- * Project (31)
- +- * Filter (30)
- +- * ColumnarToRow (29)
- +- Scan parquet default.date_dim (28)
+BroadcastExchange (34)
++- * Project (33)
+ +- * Filter (32)
+ +- * ColumnarToRow (31)
+ +- Scan parquet default.date_dim (30)
-(28) Scan parquet default.date_dim
-Output [2]: [d_date_sk#17, d_date#19]
+(30) Scan parquet default.date_dim
+Output [2]: [d_date_sk#17, d_date#22]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-01-27), LessThanOrEqual(d_date,2000-04-26), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(29) ColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#17, d_date#19]
+(31) ColumnarToRow [codegen id : 1]
+Input [2]: [d_date_sk#17, d_date#22]
-(30) Filter [codegen id : 1]
-Input [2]: [d_date_sk#17, d_date#19]
-Condition : (((isnotnull(d_date#19) AND (d_date#19 >= 2000-01-27)) AND (d_date#19 <= 2000-04-26)) AND isnotnull(d_date_sk#17))
+(32) Filter [codegen id : 1]
+Input [2]: [d_date_sk#17, d_date#22]
+Condition : (((isnotnull(d_date#22) AND (d_date#22 >= 2000-01-27)) AND (d_date#22 <= 2000-04-26)) AND isnotnull(d_date_sk#17))
-(31) Project [codegen id : 1]
+(33) Project [codegen id : 1]
Output [1]: [d_date_sk#17]
-Input [2]: [d_date_sk#17, d_date#19]
+Input [2]: [d_date_sk#17, d_date#22]
-(32) BroadcastExchange
+(34) BroadcastExchange
Input [1]: [d_date_sk#17]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
Subquery:2 Hosting operator id = 11 Hosting Expression = cs_sold_date_sk#9 IN dynamicpruning#4
diff --git a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt
index aea77be43cf..0b2410699cf 100644
--- a/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt
+++ b/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q32/simplified.txt
@@ -1,48 +1,52 @@
-CollectLimit
- WholeStageCodegen (6)
- Project
- BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
- Project [cs_sold_date_sk]
- BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))]
- Project [cs_ext_discount_amt,cs_sold_date_sk,i_item_sk]
- BroadcastHashJoin [cs_item_sk,i_item_sk]
- Filter [cs_item_sk,cs_ext_discount_amt]
- ColumnarToRow
- InputAdapter
- Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #1
+WholeStageCodegen (7)
+ HashAggregate [sum] [sum(UnscaledValue(cs_ext_discount_amt)),excess discount amount,sum]
+ InputAdapter
+ Exchange #1
+ WholeStageCodegen (6)
+ HashAggregate [cs_ext_discount_amt] [sum,sum]
+ Project [cs_ext_discount_amt]
+ BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
+ Project [cs_ext_discount_amt,cs_sold_date_sk]
+ BroadcastHashJoin [i_item_sk,cs_item_sk,cs_ext_discount_amt,(1.3 * avg(cs_ext_discount_amt))]
+ Project [cs_ext_discount_amt,cs_sold_date_sk,i_item_sk]
+ BroadcastHashJoin [cs_item_sk,i_item_sk]
+ Filter [cs_item_sk,cs_ext_discount_amt]
+ ColumnarToRow
+ InputAdapter
+ Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #2
+ WholeStageCodegen (1)
+ Project [d_date_sk]
+ Filter [d_date,d_date_sk]
+ ColumnarToRow
+ InputAdapter
+ Scan parquet default.date_dim [d_date_sk,d_date]
+ InputAdapter
+ BroadcastExchange #3
WholeStageCodegen (1)
- Project [d_date_sk]
- Filter [d_date,d_date_sk]
+ Project [i_item_sk]
+ Filter [i_manufact_id,i_item_sk]
ColumnarToRow
InputAdapter
- Scan parquet default.date_dim [d_date_sk,d_date]
+ Scan parquet default.item [i_item_sk,i_manufact_id]
+ InputAdapter
+ BroadcastExchange #4
+ WholeStageCodegen (4)
+ Filter [(1.3 * avg(cs_ext_discount_amt))]
+ HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count]
+ InputAdapter
+ Exchange [cs_item_sk] #5
+ WholeStageCodegen (3)
+ HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count]
+ Project [cs_item_sk,cs_ext_discount_amt]
+ BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
+ Filter [cs_item_sk]
+ ColumnarToRow
+ InputAdapter
+ Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ InputAdapter
+ ReusedExchange [d_date_sk] #2
InputAdapter
- BroadcastExchange #2
- WholeStageCodegen (1)
- Project [i_item_sk]
- Filter [i_manufact_id,i_item_sk]
- ColumnarToRow
- InputAdapter
- Scan parquet default.item [i_item_sk,i_manufact_id]
- InputAdapter
- BroadcastExchange #3
- WholeStageCodegen (4)
- Filter [(1.3 * avg(cs_ext_discount_amt))]
- HashAggregate [cs_item_sk,sum,count] [avg(UnscaledValue(cs_ext_discount_amt)),(1.3 * avg(cs_ext_discount_amt)),sum,count]
- InputAdapter
- Exchange [cs_item_sk] #4
- WholeStageCodegen (3)
- HashAggregate [cs_item_sk,cs_ext_discount_amt] [sum,count,sum,count]
- Project [cs_item_sk,cs_ext_discount_amt]
- BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
- Filter [cs_item_sk]
- ColumnarToRow
- InputAdapter
- Scan parquet default.catalog_sales [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- InputAdapter
- ReusedExchange [d_date_sk] #1
- InputAdapter
- ReusedExchange [d_date_sk] #1
+ ReusedExchange [d_date_sk] #2
diff --git a/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out b/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out
index de81b93b81c..fb73f26cf39 100644
--- a/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out
+++ b/sql/core/src/test/resources/tpcds-query-results/v1_4/q32.sql.out
@@ -1,6 +1,6 @@
-- Automatically generated by TPCDSQueryTestSuite
-- !query schema
-struct<excess discount amount :int>
+struct<excess discount amount:decimal(17,2)>
-- !query output
-1
+9089.28
diff --git a/sql/core/src/test/resources/tpcds/q32.sql b/sql/core/src/test/resources/tpcds/q32.sql
index a6f59ecb873..1d856ca5230 100755
--- a/sql/core/src/test/resources/tpcds/q32.sql
+++ b/sql/core/src/test/resources/tpcds/q32.sql
@@ -1,4 +1,4 @@
-SELECT 1 AS `excess discount amount `
+SELECT sum(cs_ext_discount_amt) AS `excess discount amount`
FROM
catalog_sales, item, date_dim
WHERE
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org