You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ag...@apache.org on 2025/11/19 13:59:54 UTC
(datafusion-comet) branch main updated: chore: Finish refactoring expression serde out of `QueryPlanSerde` (#2791)
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 7f90fc4e1 chore: Finish refactoring expression serde out of `QueryPlanSerde` (#2791)
7f90fc4e1 is described below
commit 7f90fc4e1a9b01a730e6cccef1459e0e2b791969
Author: Andy Grove <ag...@apache.org>
AuthorDate: Wed Nov 19 06:59:50 2025 -0700
chore: Finish refactoring expression serde out of `QueryPlanSerde` (#2791)
---
docs/source/user-guide/latest/configs.md | 5 +
.../comet/serde/CometBloomFilterMightContain.scala | 52 +
.../apache/comet/serde/CometScalarSubquery.scala | 51 +
.../org/apache/comet/serde/QueryPlanSerde.scala | 89 +-
.../apache/comet/serde/collectionOperations.scala | 44 +
.../apache/comet/serde/contraintExpressions.scala | 63 +
.../apache/comet/serde/decimalExpressions.scala | 54 +
.../q1.native_iceberg_compat/explain.txt | 351 ++---
.../q1.native_iceberg_compat/extended.txt | 102 +-
.../q1.native_iceberg_compat/simplified.txt | 83 +-
.../approved-plans-v1_4-spark4_0/q1/explain.txt | 351 ++---
.../approved-plans-v1_4-spark4_0/q1/extended.txt | 102 +-
.../approved-plans-v1_4-spark4_0/q1/simplified.txt | 83 +-
.../q11.native_iceberg_compat/explain.txt | 631 ++++----
.../q11.native_iceberg_compat/extended.txt | 151 +-
.../q11.native_iceberg_compat/simplified.txt | 150 +-
.../approved-plans-v1_4-spark4_0/q11/explain.txt | 631 ++++----
.../approved-plans-v1_4-spark4_0/q11/extended.txt | 151 +-
.../q11/simplified.txt | 150 +-
.../q12.native_iceberg_compat/explain.txt | 108 +-
.../q12.native_iceberg_compat/extended.txt | 51 +-
.../q12.native_iceberg_compat/simplified.txt | 59 +-
.../approved-plans-v1_4-spark4_0/q12/explain.txt | 108 +-
.../approved-plans-v1_4-spark4_0/q12/extended.txt | 51 +-
.../q12/simplified.txt | 59 +-
.../q13.native_iceberg_compat/explain.txt | 14 +-
.../q13.native_iceberg_compat/extended.txt | 6 +-
.../q13.native_iceberg_compat/simplified.txt | 6 +-
.../approved-plans-v1_4-spark4_0/q13/explain.txt | 14 +-
.../approved-plans-v1_4-spark4_0/q13/extended.txt | 6 +-
.../q13/simplified.txt | 6 +-
.../q15.native_iceberg_compat/explain.txt | 22 +-
.../q15.native_iceberg_compat/extended.txt | 8 +-
.../q15.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q15/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q15/extended.txt | 8 +-
.../q15/simplified.txt | 10 +-
.../q16.native_iceberg_compat/explain.txt | 14 +-
.../q16.native_iceberg_compat/extended.txt | 6 +-
.../q16.native_iceberg_compat/simplified.txt | 6 +-
.../approved-plans-v1_4-spark4_0/q16/explain.txt | 14 +-
.../approved-plans-v1_4-spark4_0/q16/extended.txt | 6 +-
.../q16/simplified.txt | 6 +-
.../q19.native_iceberg_compat/explain.txt | 22 +-
.../q19.native_iceberg_compat/extended.txt | 8 +-
.../q19.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q19/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q19/extended.txt | 8 +-
.../q19/simplified.txt | 10 +-
.../q2.native_iceberg_compat/explain.txt | 229 ++-
.../q2.native_iceberg_compat/extended.txt | 94 +-
.../q2.native_iceberg_compat/simplified.txt | 81 +-
.../approved-plans-v1_4-spark4_0/q2/explain.txt | 229 ++-
.../approved-plans-v1_4-spark4_0/q2/extended.txt | 94 +-
.../approved-plans-v1_4-spark4_0/q2/simplified.txt | 81 +-
.../q20.native_iceberg_compat/explain.txt | 108 +-
.../q20.native_iceberg_compat/extended.txt | 51 +-
.../q20.native_iceberg_compat/simplified.txt | 59 +-
.../approved-plans-v1_4-spark4_0/q20/explain.txt | 108 +-
.../approved-plans-v1_4-spark4_0/q20/extended.txt | 51 +-
.../q20/simplified.txt | 59 +-
.../q25.native_iceberg_compat/explain.txt | 22 +-
.../q25.native_iceberg_compat/extended.txt | 8 +-
.../q25.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q25/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q25/extended.txt | 8 +-
.../q25/simplified.txt | 10 +-
.../q3.native_iceberg_compat/explain.txt | 22 +-
.../q3.native_iceberg_compat/extended.txt | 8 +-
.../q3.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q3/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q3/extended.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q3/simplified.txt | 10 +-
.../q30.native_iceberg_compat/explain.txt | 393 +++--
.../q30.native_iceberg_compat/extended.txt | 126 +-
.../q30.native_iceberg_compat/simplified.txt | 89 +-
.../approved-plans-v1_4-spark4_0/q30/explain.txt | 393 +++--
.../approved-plans-v1_4-spark4_0/q30/extended.txt | 126 +-
.../q30/simplified.txt | 89 +-
.../q31.native_iceberg_compat/explain.txt | 859 +++++------
.../q31.native_iceberg_compat/extended.txt | 252 ++--
.../q31.native_iceberg_compat/simplified.txt | 243 ++-
.../approved-plans-v1_4-spark4_0/q31/explain.txt | 859 +++++------
.../approved-plans-v1_4-spark4_0/q31/extended.txt | 252 ++--
.../q31/simplified.txt | 243 ++-
.../q32.native_iceberg_compat/explain.txt | 22 +-
.../q32.native_iceberg_compat/extended.txt | 6 +-
.../q32.native_iceberg_compat/simplified.txt | 6 +-
.../approved-plans-v1_4-spark4_0/q32/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q32/extended.txt | 6 +-
.../q32/simplified.txt | 6 +-
.../q33.native_iceberg_compat/explain.txt | 381 +++--
.../q33.native_iceberg_compat/extended.txt | 195 ++-
.../q33.native_iceberg_compat/simplified.txt | 157 +-
.../approved-plans-v1_4-spark4_0/q33/explain.txt | 381 +++--
.../approved-plans-v1_4-spark4_0/q33/extended.txt | 195 ++-
.../q33/simplified.txt | 157 +-
.../q36.native_iceberg_compat/explain.txt | 122 +-
.../q36.native_iceberg_compat/extended.txt | 65 +-
.../q36.native_iceberg_compat/simplified.txt | 73 +-
.../approved-plans-v1_4-spark4_0/q36/explain.txt | 122 +-
.../approved-plans-v1_4-spark4_0/q36/extended.txt | 65 +-
.../q36/simplified.txt | 73 +-
.../q42.native_iceberg_compat/explain.txt | 22 +-
.../q42.native_iceberg_compat/extended.txt | 8 +-
.../q42.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q42/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q42/extended.txt | 8 +-
.../q42/simplified.txt | 10 +-
.../q43.native_iceberg_compat/explain.txt | 22 +-
.../q43.native_iceberg_compat/extended.txt | 8 +-
.../q43.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q43/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q43/extended.txt | 8 +-
.../q43/simplified.txt | 10 +-
.../q46.native_iceberg_compat/explain.txt | 192 ++-
.../q46.native_iceberg_compat/extended.txt | 94 +-
.../q46.native_iceberg_compat/simplified.txt | 38 +-
.../approved-plans-v1_4-spark4_0/q46/explain.txt | 192 ++-
.../approved-plans-v1_4-spark4_0/q46/extended.txt | 94 +-
.../q46/simplified.txt | 38 +-
.../q47.native_iceberg_compat/explain.txt | 274 ++--
.../q47.native_iceberg_compat/extended.txt | 167 +--
.../q47.native_iceberg_compat/simplified.txt | 84 +-
.../approved-plans-v1_4-spark4_0/q47/explain.txt | 274 ++--
.../approved-plans-v1_4-spark4_0/q47/extended.txt | 167 +--
.../q47/simplified.txt | 84 +-
.../q5.native_iceberg_compat/explain.txt | 513 ++++---
.../q5.native_iceberg_compat/extended.txt | 181 ++-
.../q5.native_iceberg_compat/simplified.txt | 183 ++-
.../approved-plans-v1_4-spark4_0/q5/explain.txt | 513 ++++---
.../approved-plans-v1_4-spark4_0/q5/extended.txt | 181 ++-
.../approved-plans-v1_4-spark4_0/q5/simplified.txt | 183 ++-
.../q51.native_iceberg_compat/explain.txt | 298 ++--
.../q51.native_iceberg_compat/extended.txt | 66 +-
.../q51.native_iceberg_compat/simplified.txt | 78 +-
.../approved-plans-v1_4-spark4_0/q51/explain.txt | 298 ++--
.../approved-plans-v1_4-spark4_0/q51/extended.txt | 66 +-
.../q51/simplified.txt | 78 +-
.../q52.native_iceberg_compat/explain.txt | 22 +-
.../q52.native_iceberg_compat/extended.txt | 8 +-
.../q52.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q52/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q52/extended.txt | 8 +-
.../q52/simplified.txt | 10 +-
.../q53.native_iceberg_compat/explain.txt | 126 +-
.../q53.native_iceberg_compat/extended.txt | 61 +-
.../q53.native_iceberg_compat/simplified.txt | 69 +-
.../approved-plans-v1_4-spark4_0/q53/explain.txt | 126 +-
.../approved-plans-v1_4-spark4_0/q53/extended.txt | 61 +-
.../q53/simplified.txt | 69 +-
.../q54.native_iceberg_compat/explain.txt | 310 ++--
.../q54.native_iceberg_compat/extended.txt | 189 ++-
.../q54.native_iceberg_compat/simplified.txt | 203 ++-
.../approved-plans-v1_4-spark4_0/q54/explain.txt | 310 ++--
.../approved-plans-v1_4-spark4_0/q54/extended.txt | 189 ++-
.../q54/simplified.txt | 203 ++-
.../q55.native_iceberg_compat/explain.txt | 22 +-
.../q55.native_iceberg_compat/extended.txt | 8 +-
.../q55.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q55/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q55/extended.txt | 8 +-
.../q55/simplified.txt | 10 +-
.../q56.native_iceberg_compat/explain.txt | 399 +++--
.../q56.native_iceberg_compat/extended.txt | 201 ++-
.../q56.native_iceberg_compat/simplified.txt | 159 +-
.../approved-plans-v1_4-spark4_0/q56/explain.txt | 399 +++--
.../approved-plans-v1_4-spark4_0/q56/extended.txt | 201 ++-
.../q56/simplified.txt | 159 +-
.../q57.native_iceberg_compat/explain.txt | 274 ++--
.../q57.native_iceberg_compat/extended.txt | 167 +--
.../q57.native_iceberg_compat/simplified.txt | 84 +-
.../approved-plans-v1_4-spark4_0/q57/explain.txt | 274 ++--
.../approved-plans-v1_4-spark4_0/q57/extended.txt | 167 +--
.../q57/simplified.txt | 84 +-
.../q58.native_iceberg_compat/explain.txt | 466 +++---
.../q58.native_iceberg_compat/extended.txt | 176 ++-
.../q58.native_iceberg_compat/simplified.txt | 78 +-
.../approved-plans-v1_4-spark4_0/q58/explain.txt | 466 +++---
.../approved-plans-v1_4-spark4_0/q58/extended.txt | 176 ++-
.../q58/simplified.txt | 78 +-
.../q59.native_iceberg_compat/explain.txt | 301 ++--
.../q59.native_iceberg_compat/extended.txt | 103 +-
.../q59.native_iceberg_compat/simplified.txt | 88 +-
.../approved-plans-v1_4-spark4_0/q59/explain.txt | 301 ++--
.../approved-plans-v1_4-spark4_0/q59/extended.txt | 103 +-
.../q59/simplified.txt | 88 +-
.../q60.native_iceberg_compat/explain.txt | 399 +++--
.../q60.native_iceberg_compat/extended.txt | 201 ++-
.../q60.native_iceberg_compat/simplified.txt | 159 +-
.../approved-plans-v1_4-spark4_0/q60/explain.txt | 399 +++--
.../approved-plans-v1_4-spark4_0/q60/extended.txt | 201 ++-
.../q60/simplified.txt | 159 +-
.../q61.native_iceberg_compat/explain.txt | 114 +-
.../q61.native_iceberg_compat/extended.txt | 10 +-
.../q61.native_iceberg_compat/simplified.txt | 12 +-
.../approved-plans-v1_4-spark4_0/q61/explain.txt | 114 +-
.../approved-plans-v1_4-spark4_0/q61/extended.txt | 10 +-
.../q61/simplified.txt | 12 +-
.../q63.native_iceberg_compat/explain.txt | 126 +-
.../q63.native_iceberg_compat/extended.txt | 61 +-
.../q63.native_iceberg_compat/simplified.txt | 69 +-
.../approved-plans-v1_4-spark4_0/q63/explain.txt | 126 +-
.../approved-plans-v1_4-spark4_0/q63/extended.txt | 61 +-
.../q63/simplified.txt | 69 +-
.../q64.native_iceberg_compat/explain.txt | 1560 ++++++++++----------
.../q64.native_iceberg_compat/extended.txt | 512 +++----
.../q64.native_iceberg_compat/simplified.txt | 439 +++---
.../approved-plans-v1_4-spark4_0/q64/explain.txt | 1560 ++++++++++----------
.../approved-plans-v1_4-spark4_0/q64/extended.txt | 512 +++----
.../q64/simplified.txt | 439 +++---
.../q65.native_iceberg_compat/explain.txt | 343 ++---
.../q65.native_iceberg_compat/extended.txt | 106 +-
.../q65.native_iceberg_compat/simplified.txt | 113 +-
.../approved-plans-v1_4-spark4_0/q65/explain.txt | 343 ++---
.../approved-plans-v1_4-spark4_0/q65/extended.txt | 106 +-
.../q65/simplified.txt | 113 +-
.../q68.native_iceberg_compat/explain.txt | 192 ++-
.../q68.native_iceberg_compat/extended.txt | 94 +-
.../q68.native_iceberg_compat/simplified.txt | 38 +-
.../approved-plans-v1_4-spark4_0/q68/explain.txt | 192 ++-
.../approved-plans-v1_4-spark4_0/q68/extended.txt | 94 +-
.../q68/simplified.txt | 38 +-
.../q70.native_iceberg_compat/explain.txt | 92 +-
.../q70.native_iceberg_compat/extended.txt | 8 +-
.../q70.native_iceberg_compat/simplified.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q70/explain.txt | 92 +-
.../approved-plans-v1_4-spark4_0/q70/extended.txt | 8 +-
.../q70/simplified.txt | 8 +-
.../q71.native_iceberg_compat/explain.txt | 118 +-
.../q71.native_iceberg_compat/extended.txt | 101 +-
.../q71.native_iceberg_compat/simplified.txt | 95 +-
.../approved-plans-v1_4-spark4_0/q71/explain.txt | 118 +-
.../approved-plans-v1_4-spark4_0/q71/extended.txt | 101 +-
.../q71/simplified.txt | 95 +-
.../q74.native_iceberg_compat/explain.txt | 621 ++++----
.../q74.native_iceberg_compat/extended.txt | 149 +-
.../q74.native_iceberg_compat/simplified.txt | 148 +-
.../approved-plans-v1_4-spark4_0/q74/explain.txt | 621 ++++----
.../approved-plans-v1_4-spark4_0/q74/extended.txt | 149 +-
.../q74/simplified.txt | 148 +-
.../q75.native_iceberg_compat/explain.txt | 682 +++++----
.../q75.native_iceberg_compat/extended.txt | 328 ++--
.../q75.native_iceberg_compat/simplified.txt | 280 ++--
.../approved-plans-v1_4-spark4_0/q75/explain.txt | 682 +++++----
.../approved-plans-v1_4-spark4_0/q75/extended.txt | 328 ++--
.../q75/simplified.txt | 280 ++--
.../q76.native_iceberg_compat/explain.txt | 22 +-
.../q76.native_iceberg_compat/extended.txt | 8 +-
.../q76.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q76/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q76/extended.txt | 8 +-
.../q76/simplified.txt | 10 +-
.../q77.native_iceberg_compat/explain.txt | 686 +++++----
.../q77.native_iceberg_compat/extended.txt | 110 +-
.../q77.native_iceberg_compat/simplified.txt | 104 +-
.../approved-plans-v1_4-spark4_0/q77/explain.txt | 686 +++++----
.../approved-plans-v1_4-spark4_0/q77/extended.txt | 110 +-
.../q77/simplified.txt | 104 +-
.../q78.native_iceberg_compat/explain.txt | 512 ++++---
.../q78.native_iceberg_compat/extended.txt | 118 +-
.../q78.native_iceberg_compat/simplified.txt | 158 +-
.../approved-plans-v1_4-spark4_0/q78/explain.txt | 512 ++++---
.../approved-plans-v1_4-spark4_0/q78/extended.txt | 118 +-
.../q78/simplified.txt | 158 +-
.../q79.native_iceberg_compat/explain.txt | 139 +-
.../q79.native_iceberg_compat/extended.txt | 71 +-
.../q79.native_iceberg_compat/simplified.txt | 26 +-
.../approved-plans-v1_4-spark4_0/q79/explain.txt | 139 +-
.../approved-plans-v1_4-spark4_0/q79/extended.txt | 71 +-
.../q79/simplified.txt | 26 +-
.../q8.native_iceberg_compat/explain.txt | 22 +-
.../q8.native_iceberg_compat/extended.txt | 8 +-
.../q8.native_iceberg_compat/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q8/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q8/extended.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q8/simplified.txt | 10 +-
.../q80.native_iceberg_compat/explain.txt | 669 ++++-----
.../q80.native_iceberg_compat/extended.txt | 263 ++--
.../q80.native_iceberg_compat/simplified.txt | 241 ++-
.../approved-plans-v1_4-spark4_0/q80/explain.txt | 669 ++++-----
.../approved-plans-v1_4-spark4_0/q80/extended.txt | 263 ++--
.../q80/simplified.txt | 241 ++-
.../q81.native_iceberg_compat/explain.txt | 393 +++--
.../q81.native_iceberg_compat/extended.txt | 126 +-
.../q81.native_iceberg_compat/simplified.txt | 89 +-
.../approved-plans-v1_4-spark4_0/q81/explain.txt | 393 +++--
.../approved-plans-v1_4-spark4_0/q81/extended.txt | 126 +-
.../q81/simplified.txt | 89 +-
.../q86.native_iceberg_compat/explain.txt | 110 +-
.../q86.native_iceberg_compat/extended.txt | 53 +-
.../q86.native_iceberg_compat/simplified.txt | 61 +-
.../approved-plans-v1_4-spark4_0/q86/explain.txt | 110 +-
.../approved-plans-v1_4-spark4_0/q86/extended.txt | 53 +-
.../q86/simplified.txt | 61 +-
.../q89.native_iceberg_compat/explain.txt | 126 +-
.../q89.native_iceberg_compat/extended.txt | 61 +-
.../q89.native_iceberg_compat/simplified.txt | 69 +-
.../approved-plans-v1_4-spark4_0/q89/explain.txt | 126 +-
.../approved-plans-v1_4-spark4_0/q89/extended.txt | 61 +-
.../q89/simplified.txt | 69 +-
.../q91.native_iceberg_compat/explain.txt | 130 +-
.../q91.native_iceberg_compat/extended.txt | 97 +-
.../q91.native_iceberg_compat/simplified.txt | 103 +-
.../approved-plans-v1_4-spark4_0/q91/explain.txt | 130 +-
.../approved-plans-v1_4-spark4_0/q91/extended.txt | 97 +-
.../q91/simplified.txt | 103 +-
.../q92.native_iceberg_compat/explain.txt | 22 +-
.../q92.native_iceberg_compat/extended.txt | 6 +-
.../q92.native_iceberg_compat/simplified.txt | 6 +-
.../approved-plans-v1_4-spark4_0/q92/explain.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q92/extended.txt | 6 +-
.../q92/simplified.txt | 6 +-
.../q94.native_iceberg_compat/explain.txt | 14 +-
.../q94.native_iceberg_compat/extended.txt | 6 +-
.../q94.native_iceberg_compat/simplified.txt | 6 +-
.../approved-plans-v1_4-spark4_0/q94/explain.txt | 14 +-
.../approved-plans-v1_4-spark4_0/q94/extended.txt | 6 +-
.../q94/simplified.txt | 6 +-
.../q95.native_iceberg_compat/explain.txt | 14 +-
.../q95.native_iceberg_compat/extended.txt | 6 +-
.../q95.native_iceberg_compat/simplified.txt | 6 +-
.../approved-plans-v1_4-spark4_0/q95/explain.txt | 14 +-
.../approved-plans-v1_4-spark4_0/q95/extended.txt | 6 +-
.../q95/simplified.txt | 6 +-
.../q98.native_iceberg_compat/explain.txt | 130 +-
.../q98.native_iceberg_compat/extended.txt | 51 +-
.../q98.native_iceberg_compat/simplified.txt | 61 +-
.../approved-plans-v1_4-spark4_0/q98/explain.txt | 130 +-
.../approved-plans-v1_4-spark4_0/q98/extended.txt | 51 +-
.../q98/simplified.txt | 61 +-
.../q11.native_iceberg_compat/explain.txt | 621 ++++----
.../q11.native_iceberg_compat/extended.txt | 149 +-
.../q11.native_iceberg_compat/simplified.txt | 148 +-
.../approved-plans-v2_7-spark4_0/q11/explain.txt | 621 ++++----
.../approved-plans-v2_7-spark4_0/q11/extended.txt | 149 +-
.../q11/simplified.txt | 148 +-
.../q12.native_iceberg_compat/explain.txt | 108 +-
.../q12.native_iceberg_compat/extended.txt | 51 +-
.../q12.native_iceberg_compat/simplified.txt | 59 +-
.../approved-plans-v2_7-spark4_0/q12/explain.txt | 108 +-
.../approved-plans-v2_7-spark4_0/q12/extended.txt | 51 +-
.../q12/simplified.txt | 59 +-
.../q20.native_iceberg_compat/explain.txt | 108 +-
.../q20.native_iceberg_compat/extended.txt | 51 +-
.../q20.native_iceberg_compat/simplified.txt | 59 +-
.../approved-plans-v2_7-spark4_0/q20/explain.txt | 108 +-
.../approved-plans-v2_7-spark4_0/q20/extended.txt | 51 +-
.../q20/simplified.txt | 59 +-
.../q36a.native_iceberg_compat/explain.txt | 281 ++--
.../q36a.native_iceberg_compat/extended.txt | 197 ++-
.../q36a.native_iceberg_compat/simplified.txt | 109 +-
.../approved-plans-v2_7-spark4_0/q36a/explain.txt | 281 ++--
.../approved-plans-v2_7-spark4_0/q36a/extended.txt | 197 ++-
.../q36a/simplified.txt | 109 +-
.../q47.native_iceberg_compat/explain.txt | 274 ++--
.../q47.native_iceberg_compat/extended.txt | 167 +--
.../q47.native_iceberg_compat/simplified.txt | 84 +-
.../approved-plans-v2_7-spark4_0/q47/explain.txt | 274 ++--
.../approved-plans-v2_7-spark4_0/q47/extended.txt | 167 +--
.../q47/simplified.txt | 84 +-
.../q51a.native_iceberg_compat/explain.txt | 624 ++++----
.../q51a.native_iceberg_compat/extended.txt | 258 ++--
.../q51a.native_iceberg_compat/simplified.txt | 94 +-
.../approved-plans-v2_7-spark4_0/q51a/explain.txt | 624 ++++----
.../approved-plans-v2_7-spark4_0/q51a/extended.txt | 258 ++--
.../q51a/simplified.txt | 94 +-
.../q57.native_iceberg_compat/explain.txt | 274 ++--
.../q57.native_iceberg_compat/extended.txt | 167 +--
.../q57.native_iceberg_compat/simplified.txt | 84 +-
.../approved-plans-v2_7-spark4_0/q57/explain.txt | 274 ++--
.../approved-plans-v2_7-spark4_0/q57/extended.txt | 167 +--
.../q57/simplified.txt | 84 +-
.../q5a.native_iceberg_compat/explain.txt | 664 ++++-----
.../q5a.native_iceberg_compat/extended.txt | 540 ++++---
.../q5a.native_iceberg_compat/simplified.txt | 218 ++-
.../approved-plans-v2_7-spark4_0/q5a/explain.txt | 664 ++++-----
.../approved-plans-v2_7-spark4_0/q5a/extended.txt | 540 ++++---
.../q5a/simplified.txt | 218 ++-
.../q64.native_iceberg_compat/explain.txt | 1560 ++++++++++----------
.../q64.native_iceberg_compat/extended.txt | 512 +++----
.../q64.native_iceberg_compat/simplified.txt | 439 +++---
.../approved-plans-v2_7-spark4_0/q64/explain.txt | 1560 ++++++++++----------
.../approved-plans-v2_7-spark4_0/q64/extended.txt | 512 +++----
.../q64/simplified.txt | 439 +++---
.../q70a.native_iceberg_compat/explain.txt | 176 ++-
.../q70a.native_iceberg_compat/extended.txt | 20 +-
.../q70a.native_iceberg_compat/simplified.txt | 8 +-
.../approved-plans-v2_7-spark4_0/q70a/explain.txt | 176 ++-
.../approved-plans-v2_7-spark4_0/q70a/extended.txt | 20 +-
.../q70a/simplified.txt | 8 +-
.../q74.native_iceberg_compat/explain.txt | 621 ++++----
.../q74.native_iceberg_compat/extended.txt | 149 +-
.../q74.native_iceberg_compat/simplified.txt | 148 +-
.../approved-plans-v2_7-spark4_0/q74/explain.txt | 621 ++++----
.../approved-plans-v2_7-spark4_0/q74/extended.txt | 149 +-
.../q74/simplified.txt | 148 +-
.../q75.native_iceberg_compat/explain.txt | 682 +++++----
.../q75.native_iceberg_compat/extended.txt | 328 ++--
.../q75.native_iceberg_compat/simplified.txt | 280 ++--
.../approved-plans-v2_7-spark4_0/q75/explain.txt | 682 +++++----
.../approved-plans-v2_7-spark4_0/q75/extended.txt | 328 ++--
.../q75/simplified.txt | 280 ++--
.../q77a.native_iceberg_compat/explain.txt | 882 ++++++-----
.../q77a.native_iceberg_compat/extended.txt | 326 ++--
.../q77a.native_iceberg_compat/simplified.txt | 116 +-
.../approved-plans-v2_7-spark4_0/q77a/explain.txt | 882 ++++++-----
.../approved-plans-v2_7-spark4_0/q77a/extended.txt | 326 ++--
.../q77a/simplified.txt | 116 +-
.../q78.native_iceberg_compat/explain.txt | 512 ++++---
.../q78.native_iceberg_compat/extended.txt | 118 +-
.../q78.native_iceberg_compat/simplified.txt | 158 +-
.../approved-plans-v2_7-spark4_0/q78/explain.txt | 512 ++++---
.../approved-plans-v2_7-spark4_0/q78/extended.txt | 118 +-
.../q78/simplified.txt | 158 +-
.../q80a.native_iceberg_compat/explain.txt | 820 +++++-----
.../q80a.native_iceberg_compat/extended.txt | 786 +++++-----
.../q80a.native_iceberg_compat/simplified.txt | 270 ++--
.../approved-plans-v2_7-spark4_0/q80a/explain.txt | 820 +++++-----
.../approved-plans-v2_7-spark4_0/q80a/extended.txt | 786 +++++-----
.../q80a/simplified.txt | 270 ++--
.../q86a.native_iceberg_compat/explain.txt | 253 ++--
.../q86a.native_iceberg_compat/extended.txt | 161 +-
.../q86a.native_iceberg_compat/simplified.txt | 97 +-
.../approved-plans-v2_7-spark4_0/q86a/explain.txt | 253 ++--
.../approved-plans-v2_7-spark4_0/q86a/extended.txt | 161 +-
.../q86a/simplified.txt | 97 +-
.../q98.native_iceberg_compat/explain.txt | 122 +-
.../q98.native_iceberg_compat/extended.txt | 51 +-
.../q98.native_iceberg_compat/simplified.txt | 61 +-
.../approved-plans-v2_7-spark4_0/q98/explain.txt | 122 +-
.../approved-plans-v2_7-spark4_0/q98/extended.txt | 51 +-
.../q98/simplified.txt | 61 +-
433 files changed, 36812 insertions(+), 40420 deletions(-)
diff --git a/docs/source/user-guide/latest/configs.md b/docs/source/user-guide/latest/configs.md
index 1767137b0..03db7c9fb 100644
--- a/docs/source/user-guide/latest/configs.md
+++ b/docs/source/user-guide/latest/configs.md
@@ -214,6 +214,7 @@ These settings can be used to determine which parts of the plan are accelerated
| `spark.comet.expression.BitwiseNot.enabled` | Enable Comet acceleration for `BitwiseNot` | true |
| `spark.comet.expression.BitwiseOr.enabled` | Enable Comet acceleration for `BitwiseOr` | true |
| `spark.comet.expression.BitwiseXor.enabled` | Enable Comet acceleration for `BitwiseXor` | true |
+| `spark.comet.expression.BloomFilterMightContain.enabled` | Enable Comet acceleration for `BloomFilterMightContain` | true |
| `spark.comet.expression.CaseWhen.enabled` | Enable Comet acceleration for `CaseWhen` | true |
| `spark.comet.expression.Cast.enabled` | Enable Comet acceleration for `Cast` | true |
| `spark.comet.expression.Ceil.enabled` | Enable Comet acceleration for `Ceil` | true |
@@ -258,6 +259,7 @@ These settings can be used to determine which parts of the plan are accelerated
| `spark.comet.expression.IsNaN.enabled` | Enable Comet acceleration for `IsNaN` | true |
| `spark.comet.expression.IsNotNull.enabled` | Enable Comet acceleration for `IsNotNull` | true |
| `spark.comet.expression.IsNull.enabled` | Enable Comet acceleration for `IsNull` | true |
+| `spark.comet.expression.KnownFloatingPointNormalized.enabled` | Enable Comet acceleration for `KnownFloatingPointNormalized` | true |
| `spark.comet.expression.Length.enabled` | Enable Comet acceleration for `Length` | true |
| `spark.comet.expression.LessThan.enabled` | Enable Comet acceleration for `LessThan` | true |
| `spark.comet.expression.LessThanOrEqual.enabled` | Enable Comet acceleration for `LessThanOrEqual` | true |
@@ -267,6 +269,7 @@ These settings can be used to determine which parts of the plan are accelerated
| `spark.comet.expression.Log10.enabled` | Enable Comet acceleration for `Log10` | true |
| `spark.comet.expression.Log2.enabled` | Enable Comet acceleration for `Log2` | true |
| `spark.comet.expression.Lower.enabled` | Enable Comet acceleration for `Lower` | true |
+| `spark.comet.expression.MakeDecimal.enabled` | Enable Comet acceleration for `MakeDecimal` | true |
| `spark.comet.expression.MapEntries.enabled` | Enable Comet acceleration for `MapEntries` | true |
| `spark.comet.expression.MapFromArrays.enabled` | Enable Comet acceleration for `MapFromArrays` | true |
| `spark.comet.expression.MapKeys.enabled` | Enable Comet acceleration for `MapKeys` | true |
@@ -289,6 +292,7 @@ These settings can be used to determine which parts of the plan are accelerated
| `spark.comet.expression.Remainder.enabled` | Enable Comet acceleration for `Remainder` | true |
| `spark.comet.expression.Reverse.enabled` | Enable Comet acceleration for `Reverse` | true |
| `spark.comet.expression.Round.enabled` | Enable Comet acceleration for `Round` | true |
+| `spark.comet.expression.ScalarSubquery.enabled` | Enable Comet acceleration for `ScalarSubquery` | true |
| `spark.comet.expression.Second.enabled` | Enable Comet acceleration for `Second` | true |
| `spark.comet.expression.Sha1.enabled` | Enable Comet acceleration for `Sha1` | true |
| `spark.comet.expression.Sha2.enabled` | Enable Comet acceleration for `Sha2` | true |
@@ -320,6 +324,7 @@ These settings can be used to determine which parts of the plan are accelerated
| `spark.comet.expression.TruncTimestamp.enabled` | Enable Comet acceleration for `TruncTimestamp` | true |
| `spark.comet.expression.UnaryMinus.enabled` | Enable Comet acceleration for `UnaryMinus` | true |
| `spark.comet.expression.Unhex.enabled` | Enable Comet acceleration for `Unhex` | true |
+| `spark.comet.expression.UnscaledValue.enabled` | Enable Comet acceleration for `UnscaledValue` | true |
| `spark.comet.expression.Upper.enabled` | Enable Comet acceleration for `Upper` | true |
| `spark.comet.expression.WeekDay.enabled` | Enable Comet acceleration for `WeekDay` | true |
| `spark.comet.expression.WeekOfYear.enabled` | Enable Comet acceleration for `WeekOfYear` | true |
diff --git a/spark/src/main/scala/org/apache/comet/serde/CometBloomFilterMightContain.scala b/spark/src/main/scala/org/apache/comet/serde/CometBloomFilterMightContain.scala
new file mode 100644
index 000000000..a13b685ea
--- /dev/null
+++ b/spark/src/main/scala/org/apache/comet/serde/CometBloomFilterMightContain.scala
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.comet.serde
+
+import org.apache.spark.sql.catalyst.expressions.{Attribute, BloomFilterMightContain}
+
+import org.apache.comet.CometSparkSessionExtensions.withInfo
+import org.apache.comet.serde.QueryPlanSerde.exprToProtoInternal
+
+object CometBloomFilterMightContain extends CometExpressionSerde[BloomFilterMightContain] {
+
+ override def convert(
+ expr: BloomFilterMightContain,
+ inputs: Seq[Attribute],
+ binding: Boolean): Option[ExprOuterClass.Expr] = {
+
+ val bloomFilter = expr.left
+ val value = expr.right
+ val bloomFilterExpr = exprToProtoInternal(bloomFilter, inputs, binding)
+ val valueExpr = exprToProtoInternal(value, inputs, binding)
+ if (bloomFilterExpr.isDefined && valueExpr.isDefined) {
+ val builder = ExprOuterClass.BloomFilterMightContain.newBuilder()
+ builder.setBloomFilter(bloomFilterExpr.get)
+ builder.setValue(valueExpr.get)
+ Some(
+ ExprOuterClass.Expr
+ .newBuilder()
+ .setBloomFilterMightContain(builder)
+ .build())
+ } else {
+ withInfo(expr, bloomFilter, value)
+ None
+ }
+ }
+}
diff --git a/spark/src/main/scala/org/apache/comet/serde/CometScalarSubquery.scala b/spark/src/main/scala/org/apache/comet/serde/CometScalarSubquery.scala
new file mode 100644
index 000000000..b1f5a2827
--- /dev/null
+++ b/spark/src/main/scala/org/apache/comet/serde/CometScalarSubquery.scala
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.comet.serde
+
+import org.apache.spark.sql.catalyst.expressions.Attribute
+import org.apache.spark.sql.execution.ScalarSubquery
+
+import org.apache.comet.CometSparkSessionExtensions.withInfo
+import org.apache.comet.serde.QueryPlanSerde.{serializeDataType, supportedDataType}
+
+object CometScalarSubquery extends CometExpressionSerde[ScalarSubquery] {
+ override def convert(
+ expr: ScalarSubquery,
+ inputs: Seq[Attribute],
+ binding: Boolean): Option[ExprOuterClass.Expr] = {
+ if (supportedDataType(expr.dataType)) {
+ val dataType = serializeDataType(expr.dataType)
+ if (dataType.isEmpty) {
+ withInfo(expr, s"Failed to serialize datatype ${expr.dataType} for scalar subquery")
+ return None
+ }
+
+ val builder = ExprOuterClass.Subquery
+ .newBuilder()
+ .setId(expr.exprId.id)
+ .setDatatype(dataType.get)
+ Some(ExprOuterClass.Expr.newBuilder().setSubquery(builder).build())
+ } else {
+ withInfo(expr, s"Unsupported data type: ${expr.dataType}")
+ None
+ }
+
+ }
+}
diff --git a/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala b/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
index 6bf3776a2..0bcfb42d0 100644
--- a/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
+++ b/spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
@@ -25,10 +25,8 @@ import org.apache.spark.internal.Logging
import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.expressions.aggregate._
import org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke
-import org.apache.spark.sql.catalyst.optimizer.NormalizeNaNAndZero
-import org.apache.spark.sql.comet._
-import org.apache.spark.sql.execution
-import org.apache.spark.sql.execution._
+import org.apache.spark.sql.comet.DecimalPrecision
+import org.apache.spark.sql.execution.{ScalarSubquery, SparkPlan}
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.types._
@@ -153,7 +151,7 @@ object QueryPlanSerde extends Logging with CometExprShim {
classOf[Lower] -> CometLower,
classOf[OctetLength] -> CometScalarFunction("octet_length"),
classOf[RegExpReplace] -> CometRegExpReplace,
- classOf[Reverse] -> CometScalarFunction("reverse"),
+ classOf[Reverse] -> CometReverse,
classOf[RLike] -> CometRLike,
classOf[StartsWith] -> CometScalarFunction("starts_with"),
classOf[StringInstr] -> CometScalarFunction("instr"),
@@ -203,21 +201,20 @@ object QueryPlanSerde extends Logging with CometExprShim {
private val miscExpressions: Map[Class[_ <: Expression], CometExpressionSerde[_]] = Map(
// TODO PromotePrecision
- // TODO KnownFloatingPointNormalized
- // TODO ScalarSubquery
- // TODO UnscaledValue
- // TODO MakeDecimal
- // TODO BloomFilterMightContain
- // TODO RegExpReplace
classOf[Alias] -> CometAlias,
classOf[AttributeReference] -> CometAttributeReference,
+ classOf[BloomFilterMightContain] -> CometBloomFilterMightContain,
classOf[CheckOverflow] -> CometCheckOverflow,
classOf[Coalesce] -> CometCoalesce,
+ classOf[KnownFloatingPointNormalized] -> CometKnownFloatingPointNormalized,
classOf[Literal] -> CometLiteral,
+ classOf[MakeDecimal] -> CometMakeDecimal,
classOf[MonotonicallyIncreasingID] -> CometMonotonicallyIncreasingId,
+ classOf[ScalarSubquery] -> CometScalarSubquery,
classOf[SparkPartitionID] -> CometSparkPartitionId,
classOf[SortOrder] -> CometSortOrder,
- classOf[StaticInvoke] -> CometStaticInvoke)
+ classOf[StaticInvoke] -> CometStaticInvoke,
+ classOf[UnscaledValue] -> CometUnscaledValue)
/**
* Mapping of Spark expression class to Comet expression handler.
@@ -541,74 +538,6 @@ object QueryPlanSerde extends Logging with CometExprShim {
// `PromotePrecision` is just a wrapper, don't need to serialize it.
exprToProtoInternal(child, inputs, binding)
- case KnownFloatingPointNormalized(NormalizeNaNAndZero(expr)) =>
- val dataType = serializeDataType(expr.dataType)
- if (dataType.isEmpty) {
- withInfo(expr, s"Unsupported datatype ${expr.dataType}")
- return None
- }
- val ex = exprToProtoInternal(expr, inputs, binding)
- ex.map { child =>
- val builder = ExprOuterClass.NormalizeNaNAndZero
- .newBuilder()
- .setChild(child)
- .setDatatype(dataType.get)
- ExprOuterClass.Expr.newBuilder().setNormalizeNanAndZero(builder).build()
- }
-
- case s @ execution.ScalarSubquery(_, _) =>
- if (supportedDataType(s.dataType)) {
- val dataType = serializeDataType(s.dataType)
- if (dataType.isEmpty) {
- withInfo(s, s"Scalar subquery returns unsupported datatype ${s.dataType}")
- return None
- }
-
- val builder = ExprOuterClass.Subquery
- .newBuilder()
- .setId(s.exprId.id)
- .setDatatype(dataType.get)
- Some(ExprOuterClass.Expr.newBuilder().setSubquery(builder).build())
- } else {
- withInfo(s, s"Unsupported data type: ${s.dataType}")
- None
- }
-
- case UnscaledValue(child) =>
- val childExpr = exprToProtoInternal(child, inputs, binding)
- val optExpr =
- scalarFunctionExprToProtoWithReturnType("unscaled_value", LongType, false, childExpr)
- optExprWithInfo(optExpr, expr, child)
-
- case MakeDecimal(child, precision, scale, true) =>
- val childExpr = exprToProtoInternal(child, inputs, binding)
- val optExpr = scalarFunctionExprToProtoWithReturnType(
- "make_decimal",
- DecimalType(precision, scale),
- false,
- childExpr)
- optExprWithInfo(optExpr, expr, child)
-
- case b @ BloomFilterMightContain(_, _) =>
- val bloomFilter = b.left
- val value = b.right
- val bloomFilterExpr = exprToProtoInternal(bloomFilter, inputs, binding)
- val valueExpr = exprToProtoInternal(value, inputs, binding)
- if (bloomFilterExpr.isDefined && valueExpr.isDefined) {
- val builder = ExprOuterClass.BloomFilterMightContain.newBuilder()
- builder.setBloomFilter(bloomFilterExpr.get)
- builder.setValue(valueExpr.get)
- Some(
- ExprOuterClass.Expr
- .newBuilder()
- .setBloomFilterMightContain(builder)
- .build())
- } else {
- withInfo(expr, bloomFilter, value)
- None
- }
- case r @ Reverse(child) if child.dataType.isInstanceOf[ArrayType] =>
- convert(r, CometArrayReverse)
case expr =>
QueryPlanSerde.exprSerdeMap.get(expr.getClass) match {
case Some(handler) =>
diff --git a/spark/src/main/scala/org/apache/comet/serde/collectionOperations.scala b/spark/src/main/scala/org/apache/comet/serde/collectionOperations.scala
new file mode 100644
index 000000000..716241040
--- /dev/null
+++ b/spark/src/main/scala/org/apache/comet/serde/collectionOperations.scala
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.comet.serde
+
+import org.apache.spark.sql.catalyst.expressions.{Attribute, Reverse}
+import org.apache.spark.sql.types.ArrayType
+
+import org.apache.comet.serde.ExprOuterClass.Expr
+
+object CometReverse extends CometScalarFunction[Reverse]("reverse") {
+
+ override def getSupportLevel(expr: Reverse): SupportLevel = {
+ if (expr.child.dataType.isInstanceOf[ArrayType]) {
+ CometArrayReverse.getSupportLevel(expr)
+ } else {
+ Compatible()
+ }
+ }
+
+ override def convert(expr: Reverse, inputs: Seq[Attribute], binding: Boolean): Option[Expr] = {
+ if (expr.child.dataType.isInstanceOf[ArrayType]) {
+ CometArrayReverse.convert(expr, inputs, binding)
+ } else {
+ super.convert(expr, inputs, binding)
+ }
+ }
+}
diff --git a/spark/src/main/scala/org/apache/comet/serde/contraintExpressions.scala b/spark/src/main/scala/org/apache/comet/serde/contraintExpressions.scala
new file mode 100644
index 000000000..62643d6da
--- /dev/null
+++ b/spark/src/main/scala/org/apache/comet/serde/contraintExpressions.scala
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.comet.serde
+
+import org.apache.spark.sql.catalyst.expressions.{Attribute, KnownFloatingPointNormalized}
+import org.apache.spark.sql.catalyst.optimizer.NormalizeNaNAndZero
+
+import org.apache.comet.CometSparkSessionExtensions.withInfo
+import org.apache.comet.serde.QueryPlanSerde.{exprToProtoInternal, optExprWithInfo, serializeDataType}
+
+object CometKnownFloatingPointNormalized
+ extends CometExpressionSerde[KnownFloatingPointNormalized] {
+
+ override def getSupportLevel(expr: KnownFloatingPointNormalized): SupportLevel = {
+ expr.child match {
+ case _: NormalizeNaNAndZero => Compatible()
+ case _ =>
+ Unsupported(
+ Some(
+ "KnownFloatingPointNormalized only supports NormalizeNaNAndZero child expressions"))
+ }
+ }
+
+ override def convert(
+ expr: KnownFloatingPointNormalized,
+ inputs: Seq[Attribute],
+ binding: Boolean): Option[ExprOuterClass.Expr] = {
+
+ val wrapped = expr.child.asInstanceOf[NormalizeNaNAndZero].child
+
+ val dataType = serializeDataType(wrapped.dataType)
+ if (dataType.isEmpty) {
+ withInfo(wrapped, s"Unsupported datatype ${wrapped.dataType}")
+ return None
+ }
+ val ex = exprToProtoInternal(wrapped, inputs, binding)
+ val optExpr = ex.map { child =>
+ val builder = ExprOuterClass.NormalizeNaNAndZero
+ .newBuilder()
+ .setChild(child)
+ .setDatatype(dataType.get)
+ ExprOuterClass.Expr.newBuilder().setNormalizeNanAndZero(builder).build()
+ }
+ optExprWithInfo(optExpr, expr, wrapped)
+ }
+}
diff --git a/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala b/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala
new file mode 100644
index 000000000..c606d1ac5
--- /dev/null
+++ b/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.comet.serde
+
+import org.apache.spark.sql.catalyst.expressions.{Attribute, MakeDecimal, UnscaledValue}
+import org.apache.spark.sql.types.{DecimalType, LongType}
+
+import org.apache.comet.serde.QueryPlanSerde.{exprToProtoInternal, optExprWithInfo, scalarFunctionExprToProtoWithReturnType}
+
+object CometUnscaledValue extends CometExpressionSerde[UnscaledValue] {
+ override def convert(
+ expr: UnscaledValue,
+ inputs: Seq[Attribute],
+ binding: Boolean): Option[ExprOuterClass.Expr] = {
+ val childExpr = exprToProtoInternal(expr.child, inputs, binding)
+ val optExpr =
+ scalarFunctionExprToProtoWithReturnType("unscaled_value", LongType, false, childExpr)
+ optExprWithInfo(optExpr, expr, expr.child)
+
+ }
+}
+
+object CometMakeDecimal extends CometExpressionSerde[MakeDecimal] {
+ override def convert(
+ expr: MakeDecimal,
+ inputs: Seq[Attribute],
+ binding: Boolean): Option[ExprOuterClass.Expr] = {
+ val childExpr = exprToProtoInternal(expr.child, inputs, binding)
+ val optExpr = scalarFunctionExprToProtoWithReturnType(
+ "make_decimal",
+ DecimalType(expr.precision, expr.scale),
+ failOnError = !expr.nullOnOverflow,
+ childExpr)
+ optExprWithInfo(optExpr, expr, expr.child)
+
+ }
+}
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/explain.txt
index 59af19b16..a37054da2 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/explain.txt
@@ -1,49 +1,45 @@
== Physical Plan ==
-TakeOrderedAndProject (45)
-+- * Project (44)
- +- * BroadcastHashJoin Inner BuildRight (43)
- :- * Project (37)
- : +- * BroadcastHashJoin Inner BuildRight (36)
- : :- * Project (30)
- : : +- * BroadcastHashJoin Inner BuildRight (29)
- : : :- * Filter (13)
- : : : +- * HashAggregate (12)
- : : : +- * CometColumnarToRow (11)
- : : : +- CometExchange (10)
- : : : +- CometHashAggregate (9)
- : : : +- CometProject (8)
- : : : +- CometBroadcastHashJoin (7)
- : : : :- CometFilter (2)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (1)
- : : : +- CometBroadcastExchange (6)
- : : : +- CometProject (5)
- : : : +- CometFilter (4)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
- : : +- BroadcastExchange (28)
- : : +- * Filter (27)
- : : +- * HashAggregate (26)
- : : +- * CometColumnarToRow (25)
- : : +- CometColumnarExchange (24)
- : : +- * HashAggregate (23)
- : : +- * HashAggregate (22)
- : : +- * CometColumnarToRow (21)
- : : +- CometExchange (20)
- : : +- CometHashAggregate (19)
- : : +- CometProject (18)
- : : +- CometBroadcastHashJoin (17)
- : : :- CometFilter (15)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (14)
- : : +- ReusedExchange (16)
- : +- BroadcastExchange (35)
- : +- * CometColumnarToRow (34)
- : +- CometProject (33)
- : +- CometFilter (32)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store (31)
- +- BroadcastExchange (42)
- +- * CometColumnarToRow (41)
- +- CometProject (40)
- +- CometFilter (39)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (38)
+* CometColumnarToRow (41)
++- CometTakeOrderedAndProject (40)
+ +- CometProject (39)
+ +- CometBroadcastHashJoin (38)
+ :- CometProject (33)
+ : +- CometBroadcastHashJoin (32)
+ : :- CometProject (27)
+ : : +- CometBroadcastHashJoin (26)
+ : : :- CometFilter (12)
+ : : : +- CometHashAggregate (11)
+ : : : +- CometExchange (10)
+ : : : +- CometHashAggregate (9)
+ : : : +- CometProject (8)
+ : : : +- CometBroadcastHashJoin (7)
+ : : : :- CometFilter (2)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (1)
+ : : : +- CometBroadcastExchange (6)
+ : : : +- CometProject (5)
+ : : : +- CometFilter (4)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
+ : : +- CometBroadcastExchange (25)
+ : : +- CometFilter (24)
+ : : +- CometHashAggregate (23)
+ : : +- CometExchange (22)
+ : : +- CometHashAggregate (21)
+ : : +- CometHashAggregate (20)
+ : : +- CometExchange (19)
+ : : +- CometHashAggregate (18)
+ : : +- CometProject (17)
+ : : +- CometBroadcastHashJoin (16)
+ : : :- CometFilter (14)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (13)
+ : : +- ReusedExchange (15)
+ : +- CometBroadcastExchange (31)
+ : +- CometProject (30)
+ : +- CometFilter (29)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store (28)
+ +- CometBroadcastExchange (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (34)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
@@ -95,202 +91,179 @@ Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#3))]
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
Arguments: hashpartitioning(sr_customer_sk#1, sr_store_sk#2, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(11) CometColumnarToRow [codegen id : 5]
-Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
-
-(12) HashAggregate [codegen id : 5]
+(11) CometHashAggregate
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
Keys [2]: [sr_customer_sk#1, sr_store_sk#2]
Functions [1]: [sum(UnscaledValue(sr_return_amt#3))]
-Aggregate Attributes [1]: [sum(UnscaledValue(sr_return_amt#3))#9]
-Results [3]: [sr_customer_sk#1 AS ctr_customer_sk#10, sr_store_sk#2 AS ctr_store_sk#11, MakeDecimal(sum(UnscaledValue(sr_return_amt#3))#9,17,2) AS ctr_total_return#12]
-(13) Filter [codegen id : 5]
-Input [3]: [ctr_customer_sk#10, ctr_store_sk#11, ctr_total_return#12]
-Condition : isnotnull(ctr_total_return#12)
+(12) CometFilter
+Input [3]: [ctr_customer_sk#9, ctr_store_sk#10, ctr_total_return#11]
+Condition : isnotnull(ctr_total_return#11)
-(14) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
-Output [4]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16]
+(13) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
+Output [4]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(sr_returned_date_sk#16), dynamicpruningexpression(sr_returned_date_sk#16 IN dynamicpruning#17)]
+PartitionFilters: [isnotnull(sr_returned_date_sk#15), dynamicpruningexpression(sr_returned_date_sk#15 IN dynamicpruning#16)]
PushedFilters: [IsNotNull(sr_store_sk)]
ReadSchema: struct<sr_customer_sk:int,sr_store_sk:int,sr_return_amt:decimal(7,2)>
-(15) CometFilter
-Input [4]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16]
-Condition : isnotnull(sr_store_sk#14)
-
-(16) ReusedExchange [Reuses operator id: 6]
-Output [1]: [d_date_sk#18]
-
-(17) CometBroadcastHashJoin
-Left output [4]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16]
-Right output [1]: [d_date_sk#18]
-Arguments: [sr_returned_date_sk#16], [d_date_sk#18], Inner, BuildRight
-
-(18) CometProject
-Input [5]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16, d_date_sk#18]
-Arguments: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15], [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15]
-
-(19) CometHashAggregate
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15]
-Keys [2]: [sr_customer_sk#13, sr_store_sk#14]
-Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#15))]
-
-(20) CometExchange
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sum#19]
-Arguments: hashpartitioning(sr_customer_sk#13, sr_store_sk#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(21) CometColumnarToRow [codegen id : 1]
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sum#19]
-
-(22) HashAggregate [codegen id : 1]
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sum#19]
-Keys [2]: [sr_customer_sk#13, sr_store_sk#14]
-Functions [1]: [sum(UnscaledValue(sr_return_amt#15))]
-Aggregate Attributes [1]: [sum(UnscaledValue(sr_return_amt#15))#9]
-Results [2]: [sr_store_sk#14 AS ctr_store_sk#20, MakeDecimal(sum(UnscaledValue(sr_return_amt#15))#9,17,2) AS ctr_total_return#21]
-
-(23) HashAggregate [codegen id : 1]
-Input [2]: [ctr_store_sk#20, ctr_total_return#21]
-Keys [1]: [ctr_store_sk#20]
-Functions [1]: [partial_avg(ctr_total_return#21)]
-Aggregate Attributes [2]: [sum#22, count#23]
-Results [3]: [ctr_store_sk#20, sum#24, count#25]
-
-(24) CometColumnarExchange
-Input [3]: [ctr_store_sk#20, sum#24, count#25]
-Arguments: hashpartitioning(ctr_store_sk#20, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
-
-(25) CometColumnarToRow [codegen id : 2]
-Input [3]: [ctr_store_sk#20, sum#24, count#25]
-
-(26) HashAggregate [codegen id : 2]
-Input [3]: [ctr_store_sk#20, sum#24, count#25]
-Keys [1]: [ctr_store_sk#20]
-Functions [1]: [avg(ctr_total_return#21)]
-Aggregate Attributes [1]: [avg(ctr_total_return#21)#26]
-Results [2]: [(avg(ctr_total_return#21)#26 * 1.2) AS (avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-
-(27) Filter [codegen id : 2]
-Input [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-Condition : isnotnull((avg(ctr_total_return) * 1.2)#27)
-
-(28) BroadcastExchange
-Input [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-Arguments: HashedRelationBroadcastMode(List(cast(input[1, int, true] as bigint)),false), [plan_id=4]
-
-(29) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_store_sk#11]
-Right keys [1]: [ctr_store_sk#20]
-Join type: Inner
-Join condition: (cast(ctr_total_return#12 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#27)
-
-(30) Project [codegen id : 5]
-Output [2]: [ctr_customer_sk#10, ctr_store_sk#11]
-Input [5]: [ctr_customer_sk#10, ctr_store_sk#11, ctr_total_return#12, (avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-
-(31) CometScan [native_iceberg_compat] parquet spark_catalog.default.store
-Output [2]: [s_store_sk#28, s_state#29]
+(14) CometFilter
+Input [4]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15]
+Condition : isnotnull(sr_store_sk#13)
+
+(15) ReusedExchange [Reuses operator id: 6]
+Output [1]: [d_date_sk#17]
+
+(16) CometBroadcastHashJoin
+Left output [4]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15]
+Right output [1]: [d_date_sk#17]
+Arguments: [sr_returned_date_sk#15], [d_date_sk#17], Inner, BuildRight
+
+(17) CometProject
+Input [5]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15, d_date_sk#17]
+Arguments: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14], [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14]
+
+(18) CometHashAggregate
+Input [3]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14]
+Keys [2]: [sr_customer_sk#12, sr_store_sk#13]
+Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#14))]
+
+(19) CometExchange
+Input [3]: [sr_customer_sk#12, sr_store_sk#13, sum#18]
+Arguments: hashpartitioning(sr_customer_sk#12, sr_store_sk#13, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(20) CometHashAggregate
+Input [3]: [sr_customer_sk#12, sr_store_sk#13, sum#18]
+Keys [2]: [sr_customer_sk#12, sr_store_sk#13]
+Functions [1]: [sum(UnscaledValue(sr_return_amt#14))]
+
+(21) CometHashAggregate
+Input [2]: [ctr_store_sk#19, ctr_total_return#20]
+Keys [1]: [ctr_store_sk#19]
+Functions [1]: [partial_avg(ctr_total_return#20)]
+
+(22) CometExchange
+Input [3]: [ctr_store_sk#19, sum#21, count#22]
+Arguments: hashpartitioning(ctr_store_sk#19, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+
+(23) CometHashAggregate
+Input [3]: [ctr_store_sk#19, sum#21, count#22]
+Keys [1]: [ctr_store_sk#19]
+Functions [1]: [avg(ctr_total_return#20)]
+
+(24) CometFilter
+Input [2]: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Condition : isnotnull((avg(ctr_total_return) * 1.2)#23)
+
+(25) CometBroadcastExchange
+Input [2]: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Arguments: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+
+(26) CometBroadcastHashJoin
+Left output [3]: [ctr_customer_sk#9, ctr_store_sk#10, ctr_total_return#11]
+Right output [2]: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Arguments: [ctr_store_sk#10], [ctr_store_sk#19], Inner, (cast(ctr_total_return#11 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#23), BuildRight
+
+(27) CometProject
+Input [5]: [ctr_customer_sk#9, ctr_store_sk#10, ctr_total_return#11, (avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Arguments: [ctr_customer_sk#9, ctr_store_sk#10], [ctr_customer_sk#9, ctr_store_sk#10]
+
+(28) CometScan [native_iceberg_compat] parquet spark_catalog.default.store
+Output [2]: [s_store_sk#24, s_state#25]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
PushedFilters: [IsNotNull(s_state), IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_state:string>
-(32) CometFilter
-Input [2]: [s_store_sk#28, s_state#29]
-Condition : ((isnotnull(s_state#29) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(s_state#29, 2)) = TN)) AND isnotnull(s_store_sk#28))
+(29) CometFilter
+Input [2]: [s_store_sk#24, s_state#25]
+Condition : ((isnotnull(s_state#25) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(s_state#25, 2)) = TN)) AND isnotnull(s_store_sk#24))
-(33) CometProject
-Input [2]: [s_store_sk#28, s_state#29]
-Arguments: [s_store_sk#28], [s_store_sk#28]
+(30) CometProject
+Input [2]: [s_store_sk#24, s_state#25]
+Arguments: [s_store_sk#24], [s_store_sk#24]
-(34) CometColumnarToRow [codegen id : 3]
-Input [1]: [s_store_sk#28]
+(31) CometBroadcastExchange
+Input [1]: [s_store_sk#24]
+Arguments: [s_store_sk#24]
-(35) BroadcastExchange
-Input [1]: [s_store_sk#28]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
+(32) CometBroadcastHashJoin
+Left output [2]: [ctr_customer_sk#9, ctr_store_sk#10]
+Right output [1]: [s_store_sk#24]
+Arguments: [ctr_store_sk#10], [s_store_sk#24], Inner, BuildRight
-(36) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_store_sk#11]
-Right keys [1]: [s_store_sk#28]
-Join type: Inner
-Join condition: None
-
-(37) Project [codegen id : 5]
-Output [1]: [ctr_customer_sk#10]
-Input [3]: [ctr_customer_sk#10, ctr_store_sk#11, s_store_sk#28]
+(33) CometProject
+Input [3]: [ctr_customer_sk#9, ctr_store_sk#10, s_store_sk#24]
+Arguments: [ctr_customer_sk#9], [ctr_customer_sk#9]
-(38) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [2]: [c_customer_sk#30, c_customer_id#31]
+(34) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [2]: [c_customer_sk#26, c_customer_id#27]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string>
-(39) CometFilter
-Input [2]: [c_customer_sk#30, c_customer_id#31]
-Condition : isnotnull(c_customer_sk#30)
+(35) CometFilter
+Input [2]: [c_customer_sk#26, c_customer_id#27]
+Condition : isnotnull(c_customer_sk#26)
-(40) CometProject
-Input [2]: [c_customer_sk#30, c_customer_id#31]
-Arguments: [c_customer_sk#30, c_customer_id#32], [c_customer_sk#30, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#31, 16)) AS c_customer_id#32]
+(36) CometProject
+Input [2]: [c_customer_sk#26, c_customer_id#27]
+Arguments: [c_customer_sk#26, c_customer_id#28], [c_customer_sk#26, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#27, 16)) AS c_customer_id#28]
-(41) CometColumnarToRow [codegen id : 4]
-Input [2]: [c_customer_sk#30, c_customer_id#32]
+(37) CometBroadcastExchange
+Input [2]: [c_customer_sk#26, c_customer_id#28]
+Arguments: [c_customer_sk#26, c_customer_id#28]
-(42) BroadcastExchange
-Input [2]: [c_customer_sk#30, c_customer_id#32]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
+(38) CometBroadcastHashJoin
+Left output [1]: [ctr_customer_sk#9]
+Right output [2]: [c_customer_sk#26, c_customer_id#28]
+Arguments: [ctr_customer_sk#9], [c_customer_sk#26], Inner, BuildRight
-(43) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_customer_sk#10]
-Right keys [1]: [c_customer_sk#30]
-Join type: Inner
-Join condition: None
+(39) CometProject
+Input [3]: [ctr_customer_sk#9, c_customer_sk#26, c_customer_id#28]
+Arguments: [c_customer_id#28], [c_customer_id#28]
-(44) Project [codegen id : 5]
-Output [1]: [c_customer_id#32]
-Input [3]: [ctr_customer_sk#10, c_customer_sk#30, c_customer_id#32]
+(40) CometTakeOrderedAndProject
+Input [1]: [c_customer_id#28]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#28 ASC NULLS FIRST], output=[c_customer_id#28]), [c_customer_id#28], 100, 0, [c_customer_id#28 ASC NULLS FIRST], [c_customer_id#28]
-(45) TakeOrderedAndProject
-Input [1]: [c_customer_id#32]
-Arguments: 100, [c_customer_id#32 ASC NULLS FIRST], [c_customer_id#32]
+(41) CometColumnarToRow [codegen id : 1]
+Input [1]: [c_customer_id#28]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = sr_returned_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (50)
-+- * CometColumnarToRow (49)
- +- CometProject (48)
- +- CometFilter (47)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (46)
+BroadcastExchange (46)
++- * CometColumnarToRow (45)
+ +- CometProject (44)
+ +- CometFilter (43)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (42)
-(46) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(42) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_year#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(47) CometFilter
+(43) CometFilter
Input [2]: [d_date_sk#6, d_year#7]
Condition : ((isnotnull(d_year#7) AND (d_year#7 = 2000)) AND isnotnull(d_date_sk#6))
-(48) CometProject
+(44) CometProject
Input [2]: [d_date_sk#6, d_year#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(49) CometColumnarToRow [codegen id : 1]
+(45) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(50) BroadcastExchange
+(46) BroadcastExchange
Input [1]: [d_date_sk#6]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=7]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-Subquery:2 Hosting operator id = 14 Hosting Expression = sr_returned_date_sk#16 IN dynamicpruning#5
+Subquery:2 Hosting operator id = 13 Hosting Expression = sr_returned_date_sk#15 IN dynamicpruning#5
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/extended.txt
index d483ace46..8aa14c43d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/extended.txt
@@ -1,57 +1,53 @@
-TakeOrderedAndProject
-+- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- Project
- : : +- BroadcastHashJoin
- : : :- Filter
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(sr_customer_sk#1 AS ctr_customer_sk#2, sr_store_sk#3 AS ctr_store_sk#4, MakeDecimal(sum(UnscaledValue(sr_return_amt#5))#6,17,2) AS ctr_total_return#7)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
- : : : : +- SubqueryBroadcast
- : : : : +- BroadcastExchange
- : : : : +- CometColumnarToRow
- : : : : +- CometProject
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- BroadcastExchange
- : : +- Filter
- : : +- HashAggregate
- : : +- CometColumnarToRow
- : : +- CometColumnarExchange
- : : +- HashAggregate
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(sr_store_sk#8 AS ctr_store_sk#9, MakeDecimal(sum(UnscaledValue(sr_return_amt#10))#6,17,2) AS ctr_total_return#11)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
- : : : +- ReusedSubquery
- : : +- CometBroadcastExchange
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- BroadcastExchange
- : +- CometColumnarToRow
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store
- +- BroadcastExchange
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
+ : : : : +- SubqueryBroadcast
+ : : : : +- BroadcastExchange
+ : : : : +- CometColumnarToRow
+ : : : : +- CometProject
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
+ : : : +- ReusedSubquery
+ : : +- CometBroadcastExchange
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store
+ +- CometBroadcastExchange
+- CometProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Comet accelerated 30 out of 49 eligible operators (61%). Final plan contains 6 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 46 out of 49 eligible operators (93%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/simplified.txt
index 0d694f3f2..3e1d6243f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1.native_iceberg_compat/simplified.txt
@@ -1,15 +1,15 @@
-TakeOrderedAndProject [c_customer_id]
- WholeStageCodegen (5)
- Project [c_customer_id]
- BroadcastHashJoin [ctr_customer_sk,c_customer_sk]
- Project [ctr_customer_sk]
- BroadcastHashJoin [ctr_store_sk,s_store_sk]
- Project [ctr_customer_sk,ctr_store_sk]
- BroadcastHashJoin [ctr_store_sk,ctr_store_sk,ctr_total_return,(avg(ctr_total_return) * 1.2)]
- Filter [ctr_total_return]
- HashAggregate [sr_customer_sk,sr_store_sk,sum] [sum(UnscaledValue(sr_return_amt)),ctr_customer_sk,ctr_store_sk,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [c_customer_id]
+ CometProject [c_customer_id]
+ CometBroadcastHashJoin [ctr_customer_sk,c_customer_sk,c_customer_id]
+ CometProject [ctr_customer_sk]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,s_store_sk]
+ CometProject [ctr_customer_sk,ctr_store_sk]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_store_sk]
+ CometFilter [ctr_customer_sk,ctr_store_sk,ctr_total_return]
+ CometHashAggregate [sum] [ctr_customer_sk,ctr_store_sk,ctr_total_return,sr_customer_sk,sr_store_sk,sum(UnscaledValue(sr_return_amt))]
CometExchange [sr_customer_sk,sr_store_sk] #1
CometHashAggregate [sr_return_amt] [sr_customer_sk,sr_store_sk,sum]
CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
@@ -28,40 +28,25 @@ TakeOrderedAndProject [c_customer_id]
CometProject [d_date_sk]
CometFilter [d_date_sk,d_year]
CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- InputAdapter
- BroadcastExchange #4
- WholeStageCodegen (2)
- Filter [(avg(ctr_total_return) * 1.2)]
- HashAggregate [ctr_store_sk,sum,count] [avg(ctr_total_return),(avg(ctr_total_return) * 1.2),sum,count]
- CometColumnarToRow
- InputAdapter
- CometColumnarExchange [ctr_store_sk] #5
- WholeStageCodegen (1)
- HashAggregate [ctr_store_sk,ctr_total_return] [sum,count,sum,count]
- HashAggregate [sr_customer_sk,sr_store_sk,sum] [sum(UnscaledValue(sr_return_amt)),ctr_store_sk,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [sr_customer_sk,sr_store_sk] #6
- CometHashAggregate [sr_return_amt] [sr_customer_sk,sr_store_sk,sum]
- CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
- CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
- CometFilter [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #3
- InputAdapter
- BroadcastExchange #7
- WholeStageCodegen (3)
- CometColumnarToRow
- InputAdapter
- CometProject [s_store_sk]
- CometFilter [s_store_sk,s_state]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store [s_store_sk,s_state]
- InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (4)
- CometColumnarToRow
- InputAdapter
- CometProject [c_customer_id] [c_customer_sk,c_customer_id]
- CometFilter [c_customer_sk,c_customer_id]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id]
+ CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_store_sk] #4
+ CometFilter [(avg(ctr_total_return) * 1.2),ctr_store_sk]
+ CometHashAggregate [sum,count] [(avg(ctr_total_return) * 1.2),ctr_store_sk,avg(ctr_total_return)]
+ CometExchange [ctr_store_sk] #5
+ CometHashAggregate [ctr_total_return] [ctr_store_sk,sum,count]
+ CometHashAggregate [sr_customer_sk,sum] [ctr_store_sk,ctr_total_return,sr_store_sk,sum(UnscaledValue(sr_return_amt))]
+ CometExchange [sr_customer_sk,sr_store_sk] #6
+ CometHashAggregate [sr_return_amt] [sr_customer_sk,sr_store_sk,sum]
+ CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
+ CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
+ CometFilter [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #3
+ CometBroadcastExchange [s_store_sk] #7
+ CometProject [s_store_sk]
+ CometFilter [s_store_sk,s_state]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.store [s_store_sk,s_state]
+ CometBroadcastExchange [c_customer_sk,c_customer_id] #8
+ CometProject [c_customer_id] [c_customer_sk,c_customer_id]
+ CometFilter [c_customer_sk,c_customer_id]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/explain.txt
index 59af19b16..a37054da2 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/explain.txt
@@ -1,49 +1,45 @@
== Physical Plan ==
-TakeOrderedAndProject (45)
-+- * Project (44)
- +- * BroadcastHashJoin Inner BuildRight (43)
- :- * Project (37)
- : +- * BroadcastHashJoin Inner BuildRight (36)
- : :- * Project (30)
- : : +- * BroadcastHashJoin Inner BuildRight (29)
- : : :- * Filter (13)
- : : : +- * HashAggregate (12)
- : : : +- * CometColumnarToRow (11)
- : : : +- CometExchange (10)
- : : : +- CometHashAggregate (9)
- : : : +- CometProject (8)
- : : : +- CometBroadcastHashJoin (7)
- : : : :- CometFilter (2)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (1)
- : : : +- CometBroadcastExchange (6)
- : : : +- CometProject (5)
- : : : +- CometFilter (4)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
- : : +- BroadcastExchange (28)
- : : +- * Filter (27)
- : : +- * HashAggregate (26)
- : : +- * CometColumnarToRow (25)
- : : +- CometColumnarExchange (24)
- : : +- * HashAggregate (23)
- : : +- * HashAggregate (22)
- : : +- * CometColumnarToRow (21)
- : : +- CometExchange (20)
- : : +- CometHashAggregate (19)
- : : +- CometProject (18)
- : : +- CometBroadcastHashJoin (17)
- : : :- CometFilter (15)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (14)
- : : +- ReusedExchange (16)
- : +- BroadcastExchange (35)
- : +- * CometColumnarToRow (34)
- : +- CometProject (33)
- : +- CometFilter (32)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store (31)
- +- BroadcastExchange (42)
- +- * CometColumnarToRow (41)
- +- CometProject (40)
- +- CometFilter (39)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (38)
+* CometColumnarToRow (41)
++- CometTakeOrderedAndProject (40)
+ +- CometProject (39)
+ +- CometBroadcastHashJoin (38)
+ :- CometProject (33)
+ : +- CometBroadcastHashJoin (32)
+ : :- CometProject (27)
+ : : +- CometBroadcastHashJoin (26)
+ : : :- CometFilter (12)
+ : : : +- CometHashAggregate (11)
+ : : : +- CometExchange (10)
+ : : : +- CometHashAggregate (9)
+ : : : +- CometProject (8)
+ : : : +- CometBroadcastHashJoin (7)
+ : : : :- CometFilter (2)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (1)
+ : : : +- CometBroadcastExchange (6)
+ : : : +- CometProject (5)
+ : : : +- CometFilter (4)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
+ : : +- CometBroadcastExchange (25)
+ : : +- CometFilter (24)
+ : : +- CometHashAggregate (23)
+ : : +- CometExchange (22)
+ : : +- CometHashAggregate (21)
+ : : +- CometHashAggregate (20)
+ : : +- CometExchange (19)
+ : : +- CometHashAggregate (18)
+ : : +- CometProject (17)
+ : : +- CometBroadcastHashJoin (16)
+ : : :- CometFilter (14)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns (13)
+ : : +- ReusedExchange (15)
+ : +- CometBroadcastExchange (31)
+ : +- CometProject (30)
+ : +- CometFilter (29)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store (28)
+ +- CometBroadcastExchange (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (34)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
@@ -95,202 +91,179 @@ Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#3))]
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
Arguments: hashpartitioning(sr_customer_sk#1, sr_store_sk#2, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(11) CometColumnarToRow [codegen id : 5]
-Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
-
-(12) HashAggregate [codegen id : 5]
+(11) CometHashAggregate
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
Keys [2]: [sr_customer_sk#1, sr_store_sk#2]
Functions [1]: [sum(UnscaledValue(sr_return_amt#3))]
-Aggregate Attributes [1]: [sum(UnscaledValue(sr_return_amt#3))#9]
-Results [3]: [sr_customer_sk#1 AS ctr_customer_sk#10, sr_store_sk#2 AS ctr_store_sk#11, MakeDecimal(sum(UnscaledValue(sr_return_amt#3))#9,17,2) AS ctr_total_return#12]
-(13) Filter [codegen id : 5]
-Input [3]: [ctr_customer_sk#10, ctr_store_sk#11, ctr_total_return#12]
-Condition : isnotnull(ctr_total_return#12)
+(12) CometFilter
+Input [3]: [ctr_customer_sk#9, ctr_store_sk#10, ctr_total_return#11]
+Condition : isnotnull(ctr_total_return#11)
-(14) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
-Output [4]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16]
+(13) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
+Output [4]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(sr_returned_date_sk#16), dynamicpruningexpression(sr_returned_date_sk#16 IN dynamicpruning#17)]
+PartitionFilters: [isnotnull(sr_returned_date_sk#15), dynamicpruningexpression(sr_returned_date_sk#15 IN dynamicpruning#16)]
PushedFilters: [IsNotNull(sr_store_sk)]
ReadSchema: struct<sr_customer_sk:int,sr_store_sk:int,sr_return_amt:decimal(7,2)>
-(15) CometFilter
-Input [4]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16]
-Condition : isnotnull(sr_store_sk#14)
-
-(16) ReusedExchange [Reuses operator id: 6]
-Output [1]: [d_date_sk#18]
-
-(17) CometBroadcastHashJoin
-Left output [4]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16]
-Right output [1]: [d_date_sk#18]
-Arguments: [sr_returned_date_sk#16], [d_date_sk#18], Inner, BuildRight
-
-(18) CometProject
-Input [5]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15, sr_returned_date_sk#16, d_date_sk#18]
-Arguments: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15], [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15]
-
-(19) CometHashAggregate
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sr_return_amt#15]
-Keys [2]: [sr_customer_sk#13, sr_store_sk#14]
-Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#15))]
-
-(20) CometExchange
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sum#19]
-Arguments: hashpartitioning(sr_customer_sk#13, sr_store_sk#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(21) CometColumnarToRow [codegen id : 1]
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sum#19]
-
-(22) HashAggregate [codegen id : 1]
-Input [3]: [sr_customer_sk#13, sr_store_sk#14, sum#19]
-Keys [2]: [sr_customer_sk#13, sr_store_sk#14]
-Functions [1]: [sum(UnscaledValue(sr_return_amt#15))]
-Aggregate Attributes [1]: [sum(UnscaledValue(sr_return_amt#15))#9]
-Results [2]: [sr_store_sk#14 AS ctr_store_sk#20, MakeDecimal(sum(UnscaledValue(sr_return_amt#15))#9,17,2) AS ctr_total_return#21]
-
-(23) HashAggregate [codegen id : 1]
-Input [2]: [ctr_store_sk#20, ctr_total_return#21]
-Keys [1]: [ctr_store_sk#20]
-Functions [1]: [partial_avg(ctr_total_return#21)]
-Aggregate Attributes [2]: [sum#22, count#23]
-Results [3]: [ctr_store_sk#20, sum#24, count#25]
-
-(24) CometColumnarExchange
-Input [3]: [ctr_store_sk#20, sum#24, count#25]
-Arguments: hashpartitioning(ctr_store_sk#20, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
-
-(25) CometColumnarToRow [codegen id : 2]
-Input [3]: [ctr_store_sk#20, sum#24, count#25]
-
-(26) HashAggregate [codegen id : 2]
-Input [3]: [ctr_store_sk#20, sum#24, count#25]
-Keys [1]: [ctr_store_sk#20]
-Functions [1]: [avg(ctr_total_return#21)]
-Aggregate Attributes [1]: [avg(ctr_total_return#21)#26]
-Results [2]: [(avg(ctr_total_return#21)#26 * 1.2) AS (avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-
-(27) Filter [codegen id : 2]
-Input [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-Condition : isnotnull((avg(ctr_total_return) * 1.2)#27)
-
-(28) BroadcastExchange
-Input [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-Arguments: HashedRelationBroadcastMode(List(cast(input[1, int, true] as bigint)),false), [plan_id=4]
-
-(29) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_store_sk#11]
-Right keys [1]: [ctr_store_sk#20]
-Join type: Inner
-Join condition: (cast(ctr_total_return#12 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#27)
-
-(30) Project [codegen id : 5]
-Output [2]: [ctr_customer_sk#10, ctr_store_sk#11]
-Input [5]: [ctr_customer_sk#10, ctr_store_sk#11, ctr_total_return#12, (avg(ctr_total_return) * 1.2)#27, ctr_store_sk#20]
-
-(31) CometScan [native_iceberg_compat] parquet spark_catalog.default.store
-Output [2]: [s_store_sk#28, s_state#29]
+(14) CometFilter
+Input [4]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15]
+Condition : isnotnull(sr_store_sk#13)
+
+(15) ReusedExchange [Reuses operator id: 6]
+Output [1]: [d_date_sk#17]
+
+(16) CometBroadcastHashJoin
+Left output [4]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15]
+Right output [1]: [d_date_sk#17]
+Arguments: [sr_returned_date_sk#15], [d_date_sk#17], Inner, BuildRight
+
+(17) CometProject
+Input [5]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14, sr_returned_date_sk#15, d_date_sk#17]
+Arguments: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14], [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14]
+
+(18) CometHashAggregate
+Input [3]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14]
+Keys [2]: [sr_customer_sk#12, sr_store_sk#13]
+Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#14))]
+
+(19) CometExchange
+Input [3]: [sr_customer_sk#12, sr_store_sk#13, sum#18]
+Arguments: hashpartitioning(sr_customer_sk#12, sr_store_sk#13, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(20) CometHashAggregate
+Input [3]: [sr_customer_sk#12, sr_store_sk#13, sum#18]
+Keys [2]: [sr_customer_sk#12, sr_store_sk#13]
+Functions [1]: [sum(UnscaledValue(sr_return_amt#14))]
+
+(21) CometHashAggregate
+Input [2]: [ctr_store_sk#19, ctr_total_return#20]
+Keys [1]: [ctr_store_sk#19]
+Functions [1]: [partial_avg(ctr_total_return#20)]
+
+(22) CometExchange
+Input [3]: [ctr_store_sk#19, sum#21, count#22]
+Arguments: hashpartitioning(ctr_store_sk#19, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+
+(23) CometHashAggregate
+Input [3]: [ctr_store_sk#19, sum#21, count#22]
+Keys [1]: [ctr_store_sk#19]
+Functions [1]: [avg(ctr_total_return#20)]
+
+(24) CometFilter
+Input [2]: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Condition : isnotnull((avg(ctr_total_return) * 1.2)#23)
+
+(25) CometBroadcastExchange
+Input [2]: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Arguments: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+
+(26) CometBroadcastHashJoin
+Left output [3]: [ctr_customer_sk#9, ctr_store_sk#10, ctr_total_return#11]
+Right output [2]: [(avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Arguments: [ctr_store_sk#10], [ctr_store_sk#19], Inner, (cast(ctr_total_return#11 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#23), BuildRight
+
+(27) CometProject
+Input [5]: [ctr_customer_sk#9, ctr_store_sk#10, ctr_total_return#11, (avg(ctr_total_return) * 1.2)#23, ctr_store_sk#19]
+Arguments: [ctr_customer_sk#9, ctr_store_sk#10], [ctr_customer_sk#9, ctr_store_sk#10]
+
+(28) CometScan [native_iceberg_compat] parquet spark_catalog.default.store
+Output [2]: [s_store_sk#24, s_state#25]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
PushedFilters: [IsNotNull(s_state), IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_state:string>
-(32) CometFilter
-Input [2]: [s_store_sk#28, s_state#29]
-Condition : ((isnotnull(s_state#29) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(s_state#29, 2)) = TN)) AND isnotnull(s_store_sk#28))
+(29) CometFilter
+Input [2]: [s_store_sk#24, s_state#25]
+Condition : ((isnotnull(s_state#25) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(s_state#25, 2)) = TN)) AND isnotnull(s_store_sk#24))
-(33) CometProject
-Input [2]: [s_store_sk#28, s_state#29]
-Arguments: [s_store_sk#28], [s_store_sk#28]
+(30) CometProject
+Input [2]: [s_store_sk#24, s_state#25]
+Arguments: [s_store_sk#24], [s_store_sk#24]
-(34) CometColumnarToRow [codegen id : 3]
-Input [1]: [s_store_sk#28]
+(31) CometBroadcastExchange
+Input [1]: [s_store_sk#24]
+Arguments: [s_store_sk#24]
-(35) BroadcastExchange
-Input [1]: [s_store_sk#28]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
+(32) CometBroadcastHashJoin
+Left output [2]: [ctr_customer_sk#9, ctr_store_sk#10]
+Right output [1]: [s_store_sk#24]
+Arguments: [ctr_store_sk#10], [s_store_sk#24], Inner, BuildRight
-(36) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_store_sk#11]
-Right keys [1]: [s_store_sk#28]
-Join type: Inner
-Join condition: None
-
-(37) Project [codegen id : 5]
-Output [1]: [ctr_customer_sk#10]
-Input [3]: [ctr_customer_sk#10, ctr_store_sk#11, s_store_sk#28]
+(33) CometProject
+Input [3]: [ctr_customer_sk#9, ctr_store_sk#10, s_store_sk#24]
+Arguments: [ctr_customer_sk#9], [ctr_customer_sk#9]
-(38) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [2]: [c_customer_sk#30, c_customer_id#31]
+(34) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [2]: [c_customer_sk#26, c_customer_id#27]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string>
-(39) CometFilter
-Input [2]: [c_customer_sk#30, c_customer_id#31]
-Condition : isnotnull(c_customer_sk#30)
+(35) CometFilter
+Input [2]: [c_customer_sk#26, c_customer_id#27]
+Condition : isnotnull(c_customer_sk#26)
-(40) CometProject
-Input [2]: [c_customer_sk#30, c_customer_id#31]
-Arguments: [c_customer_sk#30, c_customer_id#32], [c_customer_sk#30, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#31, 16)) AS c_customer_id#32]
+(36) CometProject
+Input [2]: [c_customer_sk#26, c_customer_id#27]
+Arguments: [c_customer_sk#26, c_customer_id#28], [c_customer_sk#26, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#27, 16)) AS c_customer_id#28]
-(41) CometColumnarToRow [codegen id : 4]
-Input [2]: [c_customer_sk#30, c_customer_id#32]
+(37) CometBroadcastExchange
+Input [2]: [c_customer_sk#26, c_customer_id#28]
+Arguments: [c_customer_sk#26, c_customer_id#28]
-(42) BroadcastExchange
-Input [2]: [c_customer_sk#30, c_customer_id#32]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
+(38) CometBroadcastHashJoin
+Left output [1]: [ctr_customer_sk#9]
+Right output [2]: [c_customer_sk#26, c_customer_id#28]
+Arguments: [ctr_customer_sk#9], [c_customer_sk#26], Inner, BuildRight
-(43) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_customer_sk#10]
-Right keys [1]: [c_customer_sk#30]
-Join type: Inner
-Join condition: None
+(39) CometProject
+Input [3]: [ctr_customer_sk#9, c_customer_sk#26, c_customer_id#28]
+Arguments: [c_customer_id#28], [c_customer_id#28]
-(44) Project [codegen id : 5]
-Output [1]: [c_customer_id#32]
-Input [3]: [ctr_customer_sk#10, c_customer_sk#30, c_customer_id#32]
+(40) CometTakeOrderedAndProject
+Input [1]: [c_customer_id#28]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#28 ASC NULLS FIRST], output=[c_customer_id#28]), [c_customer_id#28], 100, 0, [c_customer_id#28 ASC NULLS FIRST], [c_customer_id#28]
-(45) TakeOrderedAndProject
-Input [1]: [c_customer_id#32]
-Arguments: 100, [c_customer_id#32 ASC NULLS FIRST], [c_customer_id#32]
+(41) CometColumnarToRow [codegen id : 1]
+Input [1]: [c_customer_id#28]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = sr_returned_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (50)
-+- * CometColumnarToRow (49)
- +- CometProject (48)
- +- CometFilter (47)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (46)
+BroadcastExchange (46)
++- * CometColumnarToRow (45)
+ +- CometProject (44)
+ +- CometFilter (43)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (42)
-(46) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(42) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_year#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(47) CometFilter
+(43) CometFilter
Input [2]: [d_date_sk#6, d_year#7]
Condition : ((isnotnull(d_year#7) AND (d_year#7 = 2000)) AND isnotnull(d_date_sk#6))
-(48) CometProject
+(44) CometProject
Input [2]: [d_date_sk#6, d_year#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(49) CometColumnarToRow [codegen id : 1]
+(45) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(50) BroadcastExchange
+(46) BroadcastExchange
Input [1]: [d_date_sk#6]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=7]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-Subquery:2 Hosting operator id = 14 Hosting Expression = sr_returned_date_sk#16 IN dynamicpruning#5
+Subquery:2 Hosting operator id = 13 Hosting Expression = sr_returned_date_sk#15 IN dynamicpruning#5
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/extended.txt
index d483ace46..8aa14c43d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/extended.txt
@@ -1,57 +1,53 @@
-TakeOrderedAndProject
-+- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- Project
- : : +- BroadcastHashJoin
- : : :- Filter
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(sr_customer_sk#1 AS ctr_customer_sk#2, sr_store_sk#3 AS ctr_store_sk#4, MakeDecimal(sum(UnscaledValue(sr_return_amt#5))#6,17,2) AS ctr_total_return#7)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
- : : : : +- SubqueryBroadcast
- : : : : +- BroadcastExchange
- : : : : +- CometColumnarToRow
- : : : : +- CometProject
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- BroadcastExchange
- : : +- Filter
- : : +- HashAggregate
- : : +- CometColumnarToRow
- : : +- CometColumnarExchange
- : : +- HashAggregate
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(sr_store_sk#8 AS ctr_store_sk#9, MakeDecimal(sum(UnscaledValue(sr_return_amt#10))#6,17,2) AS ctr_total_return#11)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
- : : : +- ReusedSubquery
- : : +- CometBroadcastExchange
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- BroadcastExchange
- : +- CometColumnarToRow
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store
- +- BroadcastExchange
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
+ : : : : +- SubqueryBroadcast
+ : : : : +- BroadcastExchange
+ : : : : +- CometColumnarToRow
+ : : : : +- CometProject
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns
+ : : : +- ReusedSubquery
+ : : +- CometBroadcastExchange
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store
+ +- CometBroadcastExchange
+- CometProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Comet accelerated 30 out of 49 eligible operators (61%). Final plan contains 6 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 46 out of 49 eligible operators (93%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/simplified.txt
index 0d694f3f2..3e1d6243f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q1/simplified.txt
@@ -1,15 +1,15 @@
-TakeOrderedAndProject [c_customer_id]
- WholeStageCodegen (5)
- Project [c_customer_id]
- BroadcastHashJoin [ctr_customer_sk,c_customer_sk]
- Project [ctr_customer_sk]
- BroadcastHashJoin [ctr_store_sk,s_store_sk]
- Project [ctr_customer_sk,ctr_store_sk]
- BroadcastHashJoin [ctr_store_sk,ctr_store_sk,ctr_total_return,(avg(ctr_total_return) * 1.2)]
- Filter [ctr_total_return]
- HashAggregate [sr_customer_sk,sr_store_sk,sum] [sum(UnscaledValue(sr_return_amt)),ctr_customer_sk,ctr_store_sk,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [c_customer_id]
+ CometProject [c_customer_id]
+ CometBroadcastHashJoin [ctr_customer_sk,c_customer_sk,c_customer_id]
+ CometProject [ctr_customer_sk]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,s_store_sk]
+ CometProject [ctr_customer_sk,ctr_store_sk]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_store_sk]
+ CometFilter [ctr_customer_sk,ctr_store_sk,ctr_total_return]
+ CometHashAggregate [sum] [ctr_customer_sk,ctr_store_sk,ctr_total_return,sr_customer_sk,sr_store_sk,sum(UnscaledValue(sr_return_amt))]
CometExchange [sr_customer_sk,sr_store_sk] #1
CometHashAggregate [sr_return_amt] [sr_customer_sk,sr_store_sk,sum]
CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
@@ -28,40 +28,25 @@ TakeOrderedAndProject [c_customer_id]
CometProject [d_date_sk]
CometFilter [d_date_sk,d_year]
CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- InputAdapter
- BroadcastExchange #4
- WholeStageCodegen (2)
- Filter [(avg(ctr_total_return) * 1.2)]
- HashAggregate [ctr_store_sk,sum,count] [avg(ctr_total_return),(avg(ctr_total_return) * 1.2),sum,count]
- CometColumnarToRow
- InputAdapter
- CometColumnarExchange [ctr_store_sk] #5
- WholeStageCodegen (1)
- HashAggregate [ctr_store_sk,ctr_total_return] [sum,count,sum,count]
- HashAggregate [sr_customer_sk,sr_store_sk,sum] [sum(UnscaledValue(sr_return_amt)),ctr_store_sk,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [sr_customer_sk,sr_store_sk] #6
- CometHashAggregate [sr_return_amt] [sr_customer_sk,sr_store_sk,sum]
- CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
- CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
- CometFilter [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #3
- InputAdapter
- BroadcastExchange #7
- WholeStageCodegen (3)
- CometColumnarToRow
- InputAdapter
- CometProject [s_store_sk]
- CometFilter [s_store_sk,s_state]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store [s_store_sk,s_state]
- InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (4)
- CometColumnarToRow
- InputAdapter
- CometProject [c_customer_id] [c_customer_sk,c_customer_id]
- CometFilter [c_customer_sk,c_customer_id]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id]
+ CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_store_sk] #4
+ CometFilter [(avg(ctr_total_return) * 1.2),ctr_store_sk]
+ CometHashAggregate [sum,count] [(avg(ctr_total_return) * 1.2),ctr_store_sk,avg(ctr_total_return)]
+ CometExchange [ctr_store_sk] #5
+ CometHashAggregate [ctr_total_return] [ctr_store_sk,sum,count]
+ CometHashAggregate [sr_customer_sk,sum] [ctr_store_sk,ctr_total_return,sr_store_sk,sum(UnscaledValue(sr_return_amt))]
+ CometExchange [sr_customer_sk,sr_store_sk] #6
+ CometHashAggregate [sr_return_amt] [sr_customer_sk,sr_store_sk,sum]
+ CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
+ CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
+ CometFilter [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.store_returns [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #3
+ CometBroadcastExchange [s_store_sk] #7
+ CometProject [s_store_sk]
+ CometFilter [s_store_sk,s_state]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.store [s_store_sk,s_state]
+ CometBroadcastExchange [c_customer_sk,c_customer_id] #8
+ CometProject [c_customer_id] [c_customer_sk,c_customer_id]
+ CometFilter [c_customer_sk,c_customer_id]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/explain.txt
index b9d337c55..b6edfa880 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/explain.txt
@@ -1,80 +1,77 @@
== Physical Plan ==
-TakeOrderedAndProject (76)
-+- * Project (75)
- +- * BroadcastHashJoin Inner BuildRight (74)
- :- * Project (57)
- : +- * BroadcastHashJoin Inner BuildRight (56)
- : :- * Project (38)
- : : +- * BroadcastHashJoin Inner BuildRight (37)
- : : :- * Filter (18)
- : : : +- * HashAggregate (17)
- : : : +- * CometColumnarToRow (16)
- : : : +- CometExchange (15)
- : : : +- CometHashAggregate (14)
- : : : +- CometProject (13)
- : : : +- CometBroadcastHashJoin (12)
- : : : :- CometProject (8)
- : : : : +- CometBroadcastHashJoin (7)
- : : : : :- CometProject (3)
- : : : : : +- CometFilter (2)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (1)
- : : : : +- CometBroadcastExchange (6)
- : : : : +- CometFilter (5)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (4)
- : : : +- CometBroadcastExchange (11)
- : : : +- CometFilter (10)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
- : : +- BroadcastExchange (36)
- : : +- * HashAggregate (35)
- : : +- * CometColumnarToRow (34)
- : : +- CometExchange (33)
- : : +- CometHashAggregate (32)
- : : +- CometProject (31)
- : : +- CometBroadcastHashJoin (30)
- : : :- CometProject (26)
- : : : +- CometBroadcastHashJoin (25)
- : : : :- CometProject (21)
- : : : : +- CometFilter (20)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (19)
- : : : +- CometBroadcastExchange (24)
- : : : +- CometFilter (23)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (22)
- : : +- CometBroadcastExchange (29)
- : : +- CometFilter (28)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (27)
- : +- BroadcastExchange (55)
- : +- * Filter (54)
- : +- * HashAggregate (53)
- : +- * CometColumnarToRow (52)
- : +- CometExchange (51)
- : +- CometHashAggregate (50)
- : +- CometProject (49)
- : +- CometBroadcastHashJoin (48)
- : :- CometProject (46)
- : : +- CometBroadcastHashJoin (45)
- : : :- CometProject (41)
- : : : +- CometFilter (40)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (39)
- : : +- CometBroadcastExchange (44)
- : : +- CometFilter (43)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (42)
- : +- ReusedExchange (47)
- +- BroadcastExchange (73)
- +- * HashAggregate (72)
- +- * CometColumnarToRow (71)
- +- CometExchange (70)
- +- CometHashAggregate (69)
- +- CometProject (68)
- +- CometBroadcastHashJoin (67)
- :- CometProject (65)
- : +- CometBroadcastHashJoin (64)
- : :- CometProject (60)
- : : +- CometFilter (59)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (58)
- : +- CometBroadcastExchange (63)
- : +- CometFilter (62)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (61)
- +- ReusedExchange (66)
+* CometColumnarToRow (73)
++- CometTakeOrderedAndProject (72)
+ +- CometProject (71)
+ +- CometBroadcastHashJoin (70)
+ :- CometProject (54)
+ : +- CometBroadcastHashJoin (53)
+ : :- CometProject (36)
+ : : +- CometBroadcastHashJoin (35)
+ : : :- CometFilter (17)
+ : : : +- CometHashAggregate (16)
+ : : : +- CometExchange (15)
+ : : : +- CometHashAggregate (14)
+ : : : +- CometProject (13)
+ : : : +- CometBroadcastHashJoin (12)
+ : : : :- CometProject (8)
+ : : : : +- CometBroadcastHashJoin (7)
+ : : : : :- CometProject (3)
+ : : : : : +- CometFilter (2)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (1)
+ : : : : +- CometBroadcastExchange (6)
+ : : : : +- CometFilter (5)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (4)
+ : : : +- CometBroadcastExchange (11)
+ : : : +- CometFilter (10)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
+ : : +- CometBroadcastExchange (34)
+ : : +- CometHashAggregate (33)
+ : : +- CometExchange (32)
+ : : +- CometHashAggregate (31)
+ : : +- CometProject (30)
+ : : +- CometBroadcastHashJoin (29)
+ : : :- CometProject (25)
+ : : : +- CometBroadcastHashJoin (24)
+ : : : :- CometProject (20)
+ : : : : +- CometFilter (19)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (18)
+ : : : +- CometBroadcastExchange (23)
+ : : : +- CometFilter (22)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (21)
+ : : +- CometBroadcastExchange (28)
+ : : +- CometFilter (27)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (26)
+ : +- CometBroadcastExchange (52)
+ : +- CometFilter (51)
+ : +- CometHashAggregate (50)
+ : +- CometExchange (49)
+ : +- CometHashAggregate (48)
+ : +- CometProject (47)
+ : +- CometBroadcastHashJoin (46)
+ : :- CometProject (44)
+ : : +- CometBroadcastHashJoin (43)
+ : : :- CometProject (39)
+ : : : +- CometFilter (38)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (37)
+ : : +- CometBroadcastExchange (42)
+ : : +- CometFilter (41)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (40)
+ : +- ReusedExchange (45)
+ +- CometBroadcastExchange (69)
+ +- CometHashAggregate (68)
+ +- CometExchange (67)
+ +- CometHashAggregate (66)
+ +- CometProject (65)
+ +- CometBroadcastHashJoin (64)
+ :- CometProject (62)
+ : +- CometBroadcastHashJoin (61)
+ : :- CometProject (57)
+ : : +- CometFilter (56)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (55)
+ : +- CometBroadcastExchange (60)
+ : +- CometFilter (59)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (58)
+ +- ReusedExchange (63)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
@@ -150,349 +147,329 @@ Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discoun
Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
Arguments: hashpartitioning(c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(16) CometColumnarToRow [codegen id : 4]
-Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
-
-(17) HashAggregate [codegen id : 4]
+(16) CometHashAggregate
Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14]
Functions [1]: [sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))#23]
-Results [2]: [c_customer_id#9 AS customer_id#24, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))#23,18,2) AS year_total#25]
-(18) Filter [codegen id : 4]
-Input [2]: [customer_id#24, year_total#25]
-Condition : (isnotnull(year_total#25) AND (year_total#25 > 0.00))
+(17) CometFilter
+Input [2]: [customer_id#23, year_total#24]
+Condition : (isnotnull(year_total#24) AND (year_total#24 > 0.00))
-(19) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#26, c_customer_id#27, c_first_name#28, c_last_name#29, c_preferred_cust_flag#30, c_birth_country#31, c_login#32, c_email_address#33]
+(18) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(20) CometFilter
-Input [8]: [c_customer_sk#26, c_customer_id#27, c_first_name#28, c_last_name#29, c_preferred_cust_flag#30, c_birth_country#31, c_login#32, c_email_address#33]
-Condition : (isnotnull(c_customer_sk#26) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#27, 16))))
+(19) CometFilter
+Input [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
+Condition : (isnotnull(c_customer_sk#25) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#26, 16))))
-(21) CometProject
-Input [8]: [c_customer_sk#26, c_customer_id#27, c_first_name#28, c_last_name#29, c_preferred_cust_flag#30, c_birth_country#31, c_login#32, c_email_address#33]
-Arguments: [c_customer_sk#26, c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39], [c_customer_sk#26, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#27, 16)) AS c_customer_id#34, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#28, 20)) AS c_first_name#35, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#29, 30)) AS c_last_name#36, static_invoke(CharVarchar [...]
+(20) CometProject
+Input [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
+Arguments: [c_customer_sk#25, c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38], [c_customer_sk#25, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#26, 16)) AS c_customer_id#33, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#27, 20)) AS c_first_name#34, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#28, 30)) AS c_last_name#35, static_invoke(CharVarchar [...]
-(22) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
-Output [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
+(21) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+Output [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#43), dynamicpruningexpression(ss_sold_date_sk#43 IN dynamicpruning#44)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#42), dynamicpruningexpression(ss_sold_date_sk#42 IN dynamicpruning#43)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_ext_discount_amt:decimal(7,2),ss_ext_list_price:decimal(7,2)>
-(23) CometFilter
-Input [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Condition : isnotnull(ss_customer_sk#40)
+(22) CometFilter
+Input [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Condition : isnotnull(ss_customer_sk#39)
-(24) CometBroadcastExchange
-Input [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Arguments: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
+(23) CometBroadcastExchange
+Input [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Arguments: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
-(25) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#26, c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39]
-Right output [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Arguments: [c_customer_sk#26], [ss_customer_sk#40], Inner, BuildRight
+(24) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#25, c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38]
+Right output [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Arguments: [c_customer_sk#25], [ss_customer_sk#39], Inner, BuildRight
-(26) CometProject
-Input [12]: [c_customer_sk#26, c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Arguments: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43], [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
+(25) CometProject
+Input [12]: [c_customer_sk#25, c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Arguments: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42], [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
-(27) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#45, d_year#46]
+(26) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#44, d_year#45]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(28) CometFilter
-Input [2]: [d_date_sk#45, d_year#46]
-Condition : ((isnotnull(d_year#46) AND (d_year#46 = 2002)) AND isnotnull(d_date_sk#45))
-
-(29) CometBroadcastExchange
-Input [2]: [d_date_sk#45, d_year#46]
-Arguments: [d_date_sk#45, d_year#46]
-
-(30) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Right output [2]: [d_date_sk#45, d_year#46]
-Arguments: [ss_sold_date_sk#43], [d_date_sk#45], Inner, BuildRight
-
-(31) CometProject
-Input [12]: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43, d_date_sk#45, d_year#46]
-Arguments: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, d_year#46], [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, d_year#46]
-
-(32) CometHashAggregate
-Input [10]: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, d_year#46]
-Keys [8]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39]
-Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))]
-
-(33) CometExchange
-Input [9]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, sum#47]
-Arguments: hashpartitioning(c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(34) CometColumnarToRow [codegen id : 1]
-Input [9]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, sum#47]
-
-(35) HashAggregate [codegen id : 1]
-Input [9]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, sum#47]
-Keys [8]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39]
-Functions [1]: [sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))#23]
-Results [3]: [c_customer_id#34 AS customer_id#48, c_preferred_cust_flag#37 AS customer_preferred_cust_flag#49, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))#23,18,2) AS year_total#50]
-
-(36) BroadcastExchange
-Input [3]: [customer_id#48, customer_preferred_cust_flag#49, year_total#50]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=3]
-
-(37) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [customer_id#24]
-Right keys [1]: [customer_id#48]
-Join type: Inner
-Join condition: None
-
-(38) Project [codegen id : 4]
-Output [4]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50]
-Input [5]: [customer_id#24, year_total#25, customer_id#48, customer_preferred_cust_flag#49, year_total#50]
-
-(39) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#51, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#56, c_login#57, c_email_address#58]
+(27) CometFilter
+Input [2]: [d_date_sk#44, d_year#45]
+Condition : ((isnotnull(d_year#45) AND (d_year#45 = 2002)) AND isnotnull(d_date_sk#44))
+
+(28) CometBroadcastExchange
+Input [2]: [d_date_sk#44, d_year#45]
+Arguments: [d_date_sk#44, d_year#45]
+
+(29) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Right output [2]: [d_date_sk#44, d_year#45]
+Arguments: [ss_sold_date_sk#42], [d_date_sk#44], Inner, BuildRight
+
+(30) CometProject
+Input [12]: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42, d_date_sk#44, d_year#45]
+Arguments: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, d_year#45], [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, d_year#45]
+
+(31) CometHashAggregate
+Input [10]: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, d_year#45]
+Keys [8]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38]
+Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#41 - ss_ext_discount_amt#40)))]
+
+(32) CometExchange
+Input [9]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, sum#46]
+Arguments: hashpartitioning(c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(33) CometHashAggregate
+Input [9]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, sum#46]
+Keys [8]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38]
+Functions [1]: [sum(UnscaledValue((ss_ext_list_price#41 - ss_ext_discount_amt#40)))]
+
+(34) CometBroadcastExchange
+Input [3]: [customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+Arguments: [customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+
+(35) CometBroadcastHashJoin
+Left output [2]: [customer_id#23, year_total#24]
+Right output [3]: [customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+Arguments: [customer_id#23], [customer_id#47], Inner, BuildRight
+
+(36) CometProject
+Input [5]: [customer_id#23, year_total#24, customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+Arguments: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49], [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49]
+
+(37) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#50, c_customer_id#51, c_first_name#52, c_last_name#53, c_preferred_cust_flag#54, c_birth_country#55, c_login#56, c_email_address#57]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(40) CometFilter
-Input [8]: [c_customer_sk#51, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#56, c_login#57, c_email_address#58]
-Condition : (isnotnull(c_customer_sk#51) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#52, 16))))
+(38) CometFilter
+Input [8]: [c_customer_sk#50, c_customer_id#51, c_first_name#52, c_last_name#53, c_preferred_cust_flag#54, c_birth_country#55, c_login#56, c_email_address#57]
+Condition : (isnotnull(c_customer_sk#50) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#51, 16))))
-(41) CometProject
-Input [8]: [c_customer_sk#51, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#56, c_login#57, c_email_address#58]
-Arguments: [c_customer_sk#51, c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64], [c_customer_sk#51, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#52, 16)) AS c_customer_id#59, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#53, 20)) AS c_first_name#60, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#54, 30)) AS c_last_name#61, static_invoke(CharVarchar [...]
+(39) CometProject
+Input [8]: [c_customer_sk#50, c_customer_id#51, c_first_name#52, c_last_name#53, c_preferred_cust_flag#54, c_birth_country#55, c_login#56, c_email_address#57]
+Arguments: [c_customer_sk#50, c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63], [c_customer_sk#50, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#51, 16)) AS c_customer_id#58, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#52, 20)) AS c_first_name#59, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#53, 30)) AS c_last_name#60, static_invoke(CharVarchar [...]
-(42) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
+(40) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#68), dynamicpruningexpression(ws_sold_date_sk#68 IN dynamicpruning#69)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#67), dynamicpruningexpression(ws_sold_date_sk#67 IN dynamicpruning#68)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int,ws_ext_discount_amt:decimal(7,2),ws_ext_list_price:decimal(7,2)>
-(43) CometFilter
-Input [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Condition : isnotnull(ws_bill_customer_sk#65)
+(41) CometFilter
+Input [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Condition : isnotnull(ws_bill_customer_sk#64)
-(44) CometBroadcastExchange
-Input [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Arguments: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
+(42) CometBroadcastExchange
+Input [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Arguments: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-(45) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#51, c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64]
-Right output [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Arguments: [c_customer_sk#51], [ws_bill_customer_sk#65], Inner, BuildRight
+(43) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#50, c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63]
+Right output [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Arguments: [c_customer_sk#50], [ws_bill_customer_sk#64], Inner, BuildRight
-(46) CometProject
-Input [12]: [c_customer_sk#51, c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Arguments: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68], [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
+(44) CometProject
+Input [12]: [c_customer_sk#50, c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Arguments: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67], [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-(47) ReusedExchange [Reuses operator id: 11]
-Output [2]: [d_date_sk#70, d_year#71]
+(45) ReusedExchange [Reuses operator id: 11]
+Output [2]: [d_date_sk#69, d_year#70]
-(48) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Right output [2]: [d_date_sk#70, d_year#71]
-Arguments: [ws_sold_date_sk#68], [d_date_sk#70], Inner, BuildRight
+(46) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Right output [2]: [d_date_sk#69, d_year#70]
+Arguments: [ws_sold_date_sk#67], [d_date_sk#69], Inner, BuildRight
-(49) CometProject
-Input [12]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68, d_date_sk#70, d_year#71]
-Arguments: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, d_year#71], [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, d_year#71]
+(47) CometProject
+Input [12]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67, d_date_sk#69, d_year#70]
+Arguments: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70], [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70]
+
+(48) CometHashAggregate
+Input [10]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70]
+Keys [8]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70]
+Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
+
+(49) CometExchange
+Input [9]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70, sum#71]
+Arguments: hashpartitioning(c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(50) CometHashAggregate
-Input [10]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, d_year#71]
-Keys [8]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71]
-Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))]
-
-(51) CometExchange
-Input [9]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, sum#72]
-Arguments: hashpartitioning(c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
-
-(52) CometColumnarToRow [codegen id : 2]
-Input [9]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, sum#72]
-
-(53) HashAggregate [codegen id : 2]
-Input [9]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, sum#72]
-Keys [8]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71]
-Functions [1]: [sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))#73]
-Results [2]: [c_customer_id#59 AS customer_id#74, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))#73,18,2) AS year_total#75]
-
-(54) Filter [codegen id : 2]
-Input [2]: [customer_id#74, year_total#75]
-Condition : (isnotnull(year_total#75) AND (year_total#75 > 0.00))
-
-(55) BroadcastExchange
-Input [2]: [customer_id#74, year_total#75]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=5]
-
-(56) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [customer_id#24]
-Right keys [1]: [customer_id#74]
-Join type: Inner
-Join condition: None
-
-(57) Project [codegen id : 4]
-Output [5]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50, year_total#75]
-Input [6]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50, customer_id#74, year_total#75]
-
-(58) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#76, c_customer_id#77, c_first_name#78, c_last_name#79, c_preferred_cust_flag#80, c_birth_country#81, c_login#82, c_email_address#83]
+Input [9]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70, sum#71]
+Keys [8]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70]
+Functions [1]: [sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
+
+(51) CometFilter
+Input [2]: [customer_id#72, year_total#73]
+Condition : (isnotnull(year_total#73) AND (year_total#73 > 0.00))
+
+(52) CometBroadcastExchange
+Input [2]: [customer_id#72, year_total#73]
+Arguments: [customer_id#72, year_total#73]
+
+(53) CometBroadcastHashJoin
+Left output [4]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49]
+Right output [2]: [customer_id#72, year_total#73]
+Arguments: [customer_id#23], [customer_id#72], Inner, BuildRight
+
+(54) CometProject
+Input [6]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, customer_id#72, year_total#73]
+Arguments: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73], [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73]
+
+(55) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#74, c_customer_id#75, c_first_name#76, c_last_name#77, c_preferred_cust_flag#78, c_birth_country#79, c_login#80, c_email_address#81]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(59) CometFilter
-Input [8]: [c_customer_sk#76, c_customer_id#77, c_first_name#78, c_last_name#79, c_preferred_cust_flag#80, c_birth_country#81, c_login#82, c_email_address#83]
-Condition : (isnotnull(c_customer_sk#76) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#77, 16))))
+(56) CometFilter
+Input [8]: [c_customer_sk#74, c_customer_id#75, c_first_name#76, c_last_name#77, c_preferred_cust_flag#78, c_birth_country#79, c_login#80, c_email_address#81]
+Condition : (isnotnull(c_customer_sk#74) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#75, 16))))
-(60) CometProject
-Input [8]: [c_customer_sk#76, c_customer_id#77, c_first_name#78, c_last_name#79, c_preferred_cust_flag#80, c_birth_country#81, c_login#82, c_email_address#83]
-Arguments: [c_customer_sk#76, c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89], [c_customer_sk#76, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#77, 16)) AS c_customer_id#84, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#78, 20)) AS c_first_name#85, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#79, 30)) AS c_last_name#86, static_invoke(CharVarchar [...]
+(57) CometProject
+Input [8]: [c_customer_sk#74, c_customer_id#75, c_first_name#76, c_last_name#77, c_preferred_cust_flag#78, c_birth_country#79, c_login#80, c_email_address#81]
+Arguments: [c_customer_sk#74, c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87], [c_customer_sk#74, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#75, 16)) AS c_customer_id#82, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#76, 20)) AS c_first_name#83, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#77, 30)) AS c_last_name#84, static_invoke(CharVarchar [...]
-(61) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
+(58) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#93), dynamicpruningexpression(ws_sold_date_sk#93 IN dynamicpruning#94)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#91), dynamicpruningexpression(ws_sold_date_sk#91 IN dynamicpruning#92)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int,ws_ext_discount_amt:decimal(7,2),ws_ext_list_price:decimal(7,2)>
-(62) CometFilter
-Input [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Condition : isnotnull(ws_bill_customer_sk#90)
+(59) CometFilter
+Input [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Condition : isnotnull(ws_bill_customer_sk#88)
+
+(60) CometBroadcastExchange
+Input [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Arguments: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+
+(61) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#74, c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87]
+Right output [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Arguments: [c_customer_sk#74], [ws_bill_customer_sk#88], Inner, BuildRight
+
+(62) CometProject
+Input [12]: [c_customer_sk#74, c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Arguments: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91], [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
-(63) CometBroadcastExchange
-Input [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Arguments: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
+(63) ReusedExchange [Reuses operator id: 28]
+Output [2]: [d_date_sk#93, d_year#94]
(64) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#76, c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89]
-Right output [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Arguments: [c_customer_sk#76], [ws_bill_customer_sk#90], Inner, BuildRight
+Left output [10]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Right output [2]: [d_date_sk#93, d_year#94]
+Arguments: [ws_sold_date_sk#91], [d_date_sk#93], Inner, BuildRight
(65) CometProject
-Input [12]: [c_customer_sk#76, c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Arguments: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93], [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-
-(66) ReusedExchange [Reuses operator id: 29]
-Output [2]: [d_date_sk#95, d_year#96]
-
-(67) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Right output [2]: [d_date_sk#95, d_year#96]
-Arguments: [ws_sold_date_sk#93], [d_date_sk#95], Inner, BuildRight
-
-(68) CometProject
-Input [12]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93, d_date_sk#95, d_year#96]
-Arguments: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, d_year#96], [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, d_year#96]
-
-(69) CometHashAggregate
-Input [10]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, d_year#96]
-Keys [8]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96]
-Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))]
-
-(70) CometExchange
-Input [9]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, sum#97]
-Arguments: hashpartitioning(c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
-
-(71) CometColumnarToRow [codegen id : 3]
-Input [9]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, sum#97]
-
-(72) HashAggregate [codegen id : 3]
-Input [9]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, sum#97]
-Keys [8]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96]
-Functions [1]: [sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))#73]
-Results [2]: [c_customer_id#84 AS customer_id#98, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))#73,18,2) AS year_total#99]
-
-(73) BroadcastExchange
-Input [2]: [customer_id#98, year_total#99]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=7]
-
-(74) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [customer_id#24]
-Right keys [1]: [customer_id#98]
-Join type: Inner
-Join condition: (CASE WHEN (year_total#75 > 0.00) THEN (year_total#99 / year_total#75) END > CASE WHEN (year_total#25 > 0.00) THEN (year_total#50 / year_total#25) END)
-
-(75) Project [codegen id : 4]
-Output [1]: [customer_preferred_cust_flag#49]
-Input [7]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50, year_total#75, customer_id#98, year_total#99]
-
-(76) TakeOrderedAndProject
-Input [1]: [customer_preferred_cust_flag#49]
-Arguments: 100, [customer_preferred_cust_flag#49 ASC NULLS FIRST], [customer_preferred_cust_flag#49]
+Input [12]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91, d_date_sk#93, d_year#94]
+Arguments: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, d_year#94], [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, d_year#94]
+
+(66) CometHashAggregate
+Input [10]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, d_year#94]
+Keys [8]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94]
+Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#90 - ws_ext_discount_amt#89)))]
+
+(67) CometExchange
+Input [9]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94, sum#95]
+Arguments: hashpartitioning(c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
+
+(68) CometHashAggregate
+Input [9]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94, sum#95]
+Keys [8]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94]
+Functions [1]: [sum(UnscaledValue((ws_ext_list_price#90 - ws_ext_discount_amt#89)))]
+
+(69) CometBroadcastExchange
+Input [2]: [customer_id#96, year_total#97]
+Arguments: [customer_id#96, year_total#97]
+
+(70) CometBroadcastHashJoin
+Left output [5]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73]
+Right output [2]: [customer_id#96, year_total#97]
+Arguments: [customer_id#23], [customer_id#96], Inner, (CASE WHEN (year_total#73 > 0.00) THEN (year_total#97 / year_total#73) END > CASE WHEN (year_total#24 > 0.00) THEN (year_total#49 / year_total#24) END), BuildRight
+
+(71) CometProject
+Input [7]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73, customer_id#96, year_total#97]
+Arguments: [customer_preferred_cust_flag#48], [customer_preferred_cust_flag#48]
+
+(72) CometTakeOrderedAndProject
+Input [1]: [customer_preferred_cust_flag#48]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[customer_preferred_cust_flag#48 ASC NULLS FIRST], output=[customer_preferred_cust_flag#48]), [customer_preferred_cust_flag#48], 100, 0, [customer_preferred_cust_flag#48 ASC NULLS FIRST], [customer_preferred_cust_flag#48]
+
+(73) CometColumnarToRow [codegen id : 1]
+Input [1]: [customer_preferred_cust_flag#48]
===== Subqueries =====
Subquery:1 Hosting operator id = 4 Hosting Expression = ss_sold_date_sk#18 IN dynamicpruning#19
-BroadcastExchange (80)
-+- * CometColumnarToRow (79)
- +- CometFilter (78)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (77)
+BroadcastExchange (77)
++- * CometColumnarToRow (76)
+ +- CometFilter (75)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (74)
-(77) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(74) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(78) CometFilter
+(75) CometFilter
Input [2]: [d_date_sk#20, d_year#21]
Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_date_sk#20))
-(79) CometColumnarToRow [codegen id : 1]
+(76) CometColumnarToRow [codegen id : 1]
Input [2]: [d_date_sk#20, d_year#21]
-(80) BroadcastExchange
+(77) BroadcastExchange
Input [2]: [d_date_sk#20, d_year#21]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=8]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=5]
-Subquery:2 Hosting operator id = 22 Hosting Expression = ss_sold_date_sk#43 IN dynamicpruning#44
-BroadcastExchange (84)
-+- * CometColumnarToRow (83)
- +- CometFilter (82)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (81)
+Subquery:2 Hosting operator id = 21 Hosting Expression = ss_sold_date_sk#42 IN dynamicpruning#43
+BroadcastExchange (81)
++- * CometColumnarToRow (80)
+ +- CometFilter (79)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (78)
-(81) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#45, d_year#46]
+(78) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#44, d_year#45]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(82) CometFilter
-Input [2]: [d_date_sk#45, d_year#46]
-Condition : ((isnotnull(d_year#46) AND (d_year#46 = 2002)) AND isnotnull(d_date_sk#45))
+(79) CometFilter
+Input [2]: [d_date_sk#44, d_year#45]
+Condition : ((isnotnull(d_year#45) AND (d_year#45 = 2002)) AND isnotnull(d_date_sk#44))
-(83) CometColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#45, d_year#46]
+(80) CometColumnarToRow [codegen id : 1]
+Input [2]: [d_date_sk#44, d_year#45]
-(84) BroadcastExchange
-Input [2]: [d_date_sk#45, d_year#46]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=9]
+(81) BroadcastExchange
+Input [2]: [d_date_sk#44, d_year#45]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=6]
-Subquery:3 Hosting operator id = 42 Hosting Expression = ws_sold_date_sk#68 IN dynamicpruning#19
+Subquery:3 Hosting operator id = 40 Hosting Expression = ws_sold_date_sk#67 IN dynamicpruning#19
-Subquery:4 Hosting operator id = 61 Hosting Expression = ws_sold_date_sk#93 IN dynamicpruning#44
+Subquery:4 Hosting operator id = 58 Hosting Expression = ws_sold_date_sk#91 IN dynamicpruning#43
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/extended.txt
index fdab1c6f0..781eae905 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/extended.txt
@@ -1,79 +1,76 @@
-TakeOrderedAndProject
-+- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- Project
- : : +- BroadcastHashJoin
- : : :- Filter
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(c_customer_id#1 AS customer_id#2, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#3 - ss_ext_discount_amt#4)))#5,18,2) AS year_total#6)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometBroadcastHashJoin
- : : : : :- CometProject
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- : : : : +- CometBroadcastExchange
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : : +- SubqueryBroadcast
- : : : : +- BroadcastExchange
- : : : : +- CometColumnarToRow
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- BroadcastExchange
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: Vector(c_customer_id#7 AS customer_id#8, c_preferred_cust_flag#9 AS customer_preferred_cust_flag#10, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#11 - ss_ext_discount_amt#12)))#5,18,2) AS year_total#13)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- : : : +- CometBroadcastExchange
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : +- SubqueryBroadcast
- : : : +- BroadcastExchange
- : : : +- CometColumnarToRow
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- CometBroadcastExchange
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- BroadcastExchange
- : +- Filter
- : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(c_customer_id#14 AS customer_id#15, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#16 - ws_ext_discount_amt#17)))#18,18,2) AS year_total#19)]
- : +- CometColumnarToRow
- : +- CometExchange
- : +- CometHashAggregate
- : +- CometProject
- : +- CometBroadcastHashJoin
- : :- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- : : +- CometBroadcastExchange
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- ReusedSubquery
- : +- CometBroadcastExchange
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- BroadcastExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: Vector(c_customer_id#20 AS customer_id#21, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#22 - ws_ext_discount_amt#23)))#18,18,2) AS year_total#24)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometBroadcastHashJoin
+ : : : : :- CometProject
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : : +- SubqueryBroadcast
+ : : : : +- BroadcastExchange
+ : : : : +- CometColumnarToRow
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ : : : +- CometBroadcastExchange
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : +- SubqueryBroadcast
+ : : : +- BroadcastExchange
+ : : : +- CometColumnarToRow
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometFilter
+ : +- CometHashAggregate
+ : +- CometExchange
+ : +- CometHashAggregate
+ : +- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- ReusedSubquery
+ : +- CometBroadcastExchange
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -91,4 +88,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 64 out of 86 eligible operators (74%). Final plan contains 6 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 80 out of 86 eligible operators (93%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/simplified.txt
index 0e6712214..4599ecf53 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11.native_iceberg_compat/simplified.txt
@@ -1,15 +1,15 @@
-TakeOrderedAndProject [customer_preferred_cust_flag]
- WholeStageCodegen (4)
- Project [customer_preferred_cust_flag]
- BroadcastHashJoin [customer_id,customer_id,year_total,year_total,year_total,year_total]
- Project [customer_id,year_total,customer_preferred_cust_flag,year_total,year_total]
- BroadcastHashJoin [customer_id,customer_id]
- Project [customer_id,year_total,customer_preferred_cust_flag,year_total]
- BroadcastHashJoin [customer_id,customer_id]
- Filter [year_total]
- HashAggregate [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum] [sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt))),customer_id,year_total,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [customer_preferred_cust_flag]
+ CometProject [customer_preferred_cust_flag]
+ CometBroadcastHashJoin [customer_id,year_total,customer_preferred_cust_flag,year_total,year_total,customer_id,year_total]
+ CometProject [customer_id,year_total,customer_preferred_cust_flag,year_total,year_total]
+ CometBroadcastHashJoin [customer_id,year_total,customer_preferred_cust_flag,year_total,customer_id,year_total]
+ CometProject [customer_id,year_total,customer_preferred_cust_flag,year_total]
+ CometBroadcastHashJoin [customer_id,year_total,customer_id,customer_preferred_cust_flag,year_total]
+ CometFilter [customer_id,year_total]
+ CometHashAggregate [c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum] [customer_id,year_total,c_customer_id,sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt)))]
CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #1
CometHashAggregate [ss_ext_list_price,ss_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
@@ -32,72 +32,60 @@ TakeOrderedAndProject [customer_preferred_cust_flag]
CometBroadcastExchange [d_date_sk,d_year] #4
CometFilter [d_date_sk,d_year]
CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (1)
- HashAggregate [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum] [sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt))),customer_id,customer_preferred_cust_flag,year_total,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #6
- CometHashAggregate [ss_ext_list_price,ss_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
- CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometBroadcastExchange [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk] #7
- CometFilter [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #2
- BroadcastExchange #8
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometFilter [d_date_sk,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- CometBroadcastExchange [d_date_sk,d_year] #9
- CometFilter [d_date_sk,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- InputAdapter
- BroadcastExchange #10
- WholeStageCodegen (2)
- Filter [year_total]
- HashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt))),customer_id,year_total,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
- CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
- CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #12
- CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk,d_year] #4
- InputAdapter
- BroadcastExchange #13
- WholeStageCodegen (3)
- HashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt))),customer_id,year_total,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
- CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
- CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #15
- CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #2
- ReusedExchange [d_date_sk,d_year] #9
+ CometBroadcastExchange [customer_id,customer_preferred_cust_flag,year_total] #5
+ CometHashAggregate [c_first_name,c_last_name,d_year,c_birth_country,c_login,c_email_address,sum] [customer_id,customer_preferred_cust_flag,year_total,c_customer_id,c_preferred_cust_flag,sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt)))]
+ CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #6
+ CometHashAggregate [ss_ext_list_price,ss_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
+ CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometBroadcastExchange [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk] #7
+ CometFilter [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #2
+ BroadcastExchange #8
+ WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometFilter [d_date_sk,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
+ CometBroadcastExchange [d_date_sk,d_year] #9
+ CometFilter [d_date_sk,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
+ CometBroadcastExchange [customer_id,year_total] #10
+ CometFilter [customer_id,year_total]
+ CometHashAggregate [c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [customer_id,year_total,c_customer_id,sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt)))]
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
+ CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
+ CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #12
+ CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk,d_year] #4
+ CometBroadcastExchange [customer_id,year_total] #13
+ CometHashAggregate [c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [customer_id,year_total,c_customer_id,sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt)))]
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
+ CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
+ CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #15
+ CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #2
+ ReusedExchange [d_date_sk,d_year] #9
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/explain.txt
index b9d337c55..b6edfa880 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/explain.txt
@@ -1,80 +1,77 @@
== Physical Plan ==
-TakeOrderedAndProject (76)
-+- * Project (75)
- +- * BroadcastHashJoin Inner BuildRight (74)
- :- * Project (57)
- : +- * BroadcastHashJoin Inner BuildRight (56)
- : :- * Project (38)
- : : +- * BroadcastHashJoin Inner BuildRight (37)
- : : :- * Filter (18)
- : : : +- * HashAggregate (17)
- : : : +- * CometColumnarToRow (16)
- : : : +- CometExchange (15)
- : : : +- CometHashAggregate (14)
- : : : +- CometProject (13)
- : : : +- CometBroadcastHashJoin (12)
- : : : :- CometProject (8)
- : : : : +- CometBroadcastHashJoin (7)
- : : : : :- CometProject (3)
- : : : : : +- CometFilter (2)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (1)
- : : : : +- CometBroadcastExchange (6)
- : : : : +- CometFilter (5)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (4)
- : : : +- CometBroadcastExchange (11)
- : : : +- CometFilter (10)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
- : : +- BroadcastExchange (36)
- : : +- * HashAggregate (35)
- : : +- * CometColumnarToRow (34)
- : : +- CometExchange (33)
- : : +- CometHashAggregate (32)
- : : +- CometProject (31)
- : : +- CometBroadcastHashJoin (30)
- : : :- CometProject (26)
- : : : +- CometBroadcastHashJoin (25)
- : : : :- CometProject (21)
- : : : : +- CometFilter (20)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (19)
- : : : +- CometBroadcastExchange (24)
- : : : +- CometFilter (23)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (22)
- : : +- CometBroadcastExchange (29)
- : : +- CometFilter (28)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (27)
- : +- BroadcastExchange (55)
- : +- * Filter (54)
- : +- * HashAggregate (53)
- : +- * CometColumnarToRow (52)
- : +- CometExchange (51)
- : +- CometHashAggregate (50)
- : +- CometProject (49)
- : +- CometBroadcastHashJoin (48)
- : :- CometProject (46)
- : : +- CometBroadcastHashJoin (45)
- : : :- CometProject (41)
- : : : +- CometFilter (40)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (39)
- : : +- CometBroadcastExchange (44)
- : : +- CometFilter (43)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (42)
- : +- ReusedExchange (47)
- +- BroadcastExchange (73)
- +- * HashAggregate (72)
- +- * CometColumnarToRow (71)
- +- CometExchange (70)
- +- CometHashAggregate (69)
- +- CometProject (68)
- +- CometBroadcastHashJoin (67)
- :- CometProject (65)
- : +- CometBroadcastHashJoin (64)
- : :- CometProject (60)
- : : +- CometFilter (59)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (58)
- : +- CometBroadcastExchange (63)
- : +- CometFilter (62)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (61)
- +- ReusedExchange (66)
+* CometColumnarToRow (73)
++- CometTakeOrderedAndProject (72)
+ +- CometProject (71)
+ +- CometBroadcastHashJoin (70)
+ :- CometProject (54)
+ : +- CometBroadcastHashJoin (53)
+ : :- CometProject (36)
+ : : +- CometBroadcastHashJoin (35)
+ : : :- CometFilter (17)
+ : : : +- CometHashAggregate (16)
+ : : : +- CometExchange (15)
+ : : : +- CometHashAggregate (14)
+ : : : +- CometProject (13)
+ : : : +- CometBroadcastHashJoin (12)
+ : : : :- CometProject (8)
+ : : : : +- CometBroadcastHashJoin (7)
+ : : : : :- CometProject (3)
+ : : : : : +- CometFilter (2)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (1)
+ : : : : +- CometBroadcastExchange (6)
+ : : : : +- CometFilter (5)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (4)
+ : : : +- CometBroadcastExchange (11)
+ : : : +- CometFilter (10)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
+ : : +- CometBroadcastExchange (34)
+ : : +- CometHashAggregate (33)
+ : : +- CometExchange (32)
+ : : +- CometHashAggregate (31)
+ : : +- CometProject (30)
+ : : +- CometBroadcastHashJoin (29)
+ : : :- CometProject (25)
+ : : : +- CometBroadcastHashJoin (24)
+ : : : :- CometProject (20)
+ : : : : +- CometFilter (19)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (18)
+ : : : +- CometBroadcastExchange (23)
+ : : : +- CometFilter (22)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (21)
+ : : +- CometBroadcastExchange (28)
+ : : +- CometFilter (27)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (26)
+ : +- CometBroadcastExchange (52)
+ : +- CometFilter (51)
+ : +- CometHashAggregate (50)
+ : +- CometExchange (49)
+ : +- CometHashAggregate (48)
+ : +- CometProject (47)
+ : +- CometBroadcastHashJoin (46)
+ : :- CometProject (44)
+ : : +- CometBroadcastHashJoin (43)
+ : : :- CometProject (39)
+ : : : +- CometFilter (38)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (37)
+ : : +- CometBroadcastExchange (42)
+ : : +- CometFilter (41)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (40)
+ : +- ReusedExchange (45)
+ +- CometBroadcastExchange (69)
+ +- CometHashAggregate (68)
+ +- CometExchange (67)
+ +- CometHashAggregate (66)
+ +- CometProject (65)
+ +- CometBroadcastHashJoin (64)
+ :- CometProject (62)
+ : +- CometBroadcastHashJoin (61)
+ : :- CometProject (57)
+ : : +- CometFilter (56)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (55)
+ : +- CometBroadcastExchange (60)
+ : +- CometFilter (59)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (58)
+ +- ReusedExchange (63)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
@@ -150,349 +147,329 @@ Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discoun
Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
Arguments: hashpartitioning(c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(16) CometColumnarToRow [codegen id : 4]
-Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
-
-(17) HashAggregate [codegen id : 4]
+(16) CometHashAggregate
Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14]
Functions [1]: [sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))#23]
-Results [2]: [c_customer_id#9 AS customer_id#24, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))#23,18,2) AS year_total#25]
-(18) Filter [codegen id : 4]
-Input [2]: [customer_id#24, year_total#25]
-Condition : (isnotnull(year_total#25) AND (year_total#25 > 0.00))
+(17) CometFilter
+Input [2]: [customer_id#23, year_total#24]
+Condition : (isnotnull(year_total#24) AND (year_total#24 > 0.00))
-(19) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#26, c_customer_id#27, c_first_name#28, c_last_name#29, c_preferred_cust_flag#30, c_birth_country#31, c_login#32, c_email_address#33]
+(18) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(20) CometFilter
-Input [8]: [c_customer_sk#26, c_customer_id#27, c_first_name#28, c_last_name#29, c_preferred_cust_flag#30, c_birth_country#31, c_login#32, c_email_address#33]
-Condition : (isnotnull(c_customer_sk#26) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#27, 16))))
+(19) CometFilter
+Input [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
+Condition : (isnotnull(c_customer_sk#25) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#26, 16))))
-(21) CometProject
-Input [8]: [c_customer_sk#26, c_customer_id#27, c_first_name#28, c_last_name#29, c_preferred_cust_flag#30, c_birth_country#31, c_login#32, c_email_address#33]
-Arguments: [c_customer_sk#26, c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39], [c_customer_sk#26, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#27, 16)) AS c_customer_id#34, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#28, 20)) AS c_first_name#35, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#29, 30)) AS c_last_name#36, static_invoke(CharVarchar [...]
+(20) CometProject
+Input [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
+Arguments: [c_customer_sk#25, c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38], [c_customer_sk#25, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#26, 16)) AS c_customer_id#33, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#27, 20)) AS c_first_name#34, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#28, 30)) AS c_last_name#35, static_invoke(CharVarchar [...]
-(22) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
-Output [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
+(21) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+Output [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#43), dynamicpruningexpression(ss_sold_date_sk#43 IN dynamicpruning#44)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#42), dynamicpruningexpression(ss_sold_date_sk#42 IN dynamicpruning#43)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_ext_discount_amt:decimal(7,2),ss_ext_list_price:decimal(7,2)>
-(23) CometFilter
-Input [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Condition : isnotnull(ss_customer_sk#40)
+(22) CometFilter
+Input [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Condition : isnotnull(ss_customer_sk#39)
-(24) CometBroadcastExchange
-Input [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Arguments: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
+(23) CometBroadcastExchange
+Input [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Arguments: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
-(25) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#26, c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39]
-Right output [4]: [ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Arguments: [c_customer_sk#26], [ss_customer_sk#40], Inner, BuildRight
+(24) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#25, c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38]
+Right output [4]: [ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Arguments: [c_customer_sk#25], [ss_customer_sk#39], Inner, BuildRight
-(26) CometProject
-Input [12]: [c_customer_sk#26, c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_customer_sk#40, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Arguments: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43], [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
+(25) CometProject
+Input [12]: [c_customer_sk#25, c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_customer_sk#39, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Arguments: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42], [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
-(27) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#45, d_year#46]
+(26) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#44, d_year#45]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(28) CometFilter
-Input [2]: [d_date_sk#45, d_year#46]
-Condition : ((isnotnull(d_year#46) AND (d_year#46 = 2002)) AND isnotnull(d_date_sk#45))
-
-(29) CometBroadcastExchange
-Input [2]: [d_date_sk#45, d_year#46]
-Arguments: [d_date_sk#45, d_year#46]
-
-(30) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43]
-Right output [2]: [d_date_sk#45, d_year#46]
-Arguments: [ss_sold_date_sk#43], [d_date_sk#45], Inner, BuildRight
-
-(31) CometProject
-Input [12]: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, ss_sold_date_sk#43, d_date_sk#45, d_year#46]
-Arguments: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, d_year#46], [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, d_year#46]
-
-(32) CometHashAggregate
-Input [10]: [c_customer_id#34, c_first_name#35, c_last_name#36, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, ss_ext_discount_amt#41, ss_ext_list_price#42, d_year#46]
-Keys [8]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39]
-Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))]
-
-(33) CometExchange
-Input [9]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, sum#47]
-Arguments: hashpartitioning(c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(34) CometColumnarToRow [codegen id : 1]
-Input [9]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, sum#47]
-
-(35) HashAggregate [codegen id : 1]
-Input [9]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39, sum#47]
-Keys [8]: [c_customer_id#34, c_first_name#35, c_last_name#36, d_year#46, c_preferred_cust_flag#37, c_birth_country#31, c_login#38, c_email_address#39]
-Functions [1]: [sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))#23]
-Results [3]: [c_customer_id#34 AS customer_id#48, c_preferred_cust_flag#37 AS customer_preferred_cust_flag#49, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#42 - ss_ext_discount_amt#41)))#23,18,2) AS year_total#50]
-
-(36) BroadcastExchange
-Input [3]: [customer_id#48, customer_preferred_cust_flag#49, year_total#50]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=3]
-
-(37) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [customer_id#24]
-Right keys [1]: [customer_id#48]
-Join type: Inner
-Join condition: None
-
-(38) Project [codegen id : 4]
-Output [4]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50]
-Input [5]: [customer_id#24, year_total#25, customer_id#48, customer_preferred_cust_flag#49, year_total#50]
-
-(39) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#51, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#56, c_login#57, c_email_address#58]
+(27) CometFilter
+Input [2]: [d_date_sk#44, d_year#45]
+Condition : ((isnotnull(d_year#45) AND (d_year#45 = 2002)) AND isnotnull(d_date_sk#44))
+
+(28) CometBroadcastExchange
+Input [2]: [d_date_sk#44, d_year#45]
+Arguments: [d_date_sk#44, d_year#45]
+
+(29) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42]
+Right output [2]: [d_date_sk#44, d_year#45]
+Arguments: [ss_sold_date_sk#42], [d_date_sk#44], Inner, BuildRight
+
+(30) CometProject
+Input [12]: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, ss_sold_date_sk#42, d_date_sk#44, d_year#45]
+Arguments: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, d_year#45], [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, d_year#45]
+
+(31) CometHashAggregate
+Input [10]: [c_customer_id#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, ss_ext_discount_amt#40, ss_ext_list_price#41, d_year#45]
+Keys [8]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38]
+Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#41 - ss_ext_discount_amt#40)))]
+
+(32) CometExchange
+Input [9]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, sum#46]
+Arguments: hashpartitioning(c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(33) CometHashAggregate
+Input [9]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38, sum#46]
+Keys [8]: [c_customer_id#33, c_first_name#34, c_last_name#35, d_year#45, c_preferred_cust_flag#36, c_birth_country#30, c_login#37, c_email_address#38]
+Functions [1]: [sum(UnscaledValue((ss_ext_list_price#41 - ss_ext_discount_amt#40)))]
+
+(34) CometBroadcastExchange
+Input [3]: [customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+Arguments: [customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+
+(35) CometBroadcastHashJoin
+Left output [2]: [customer_id#23, year_total#24]
+Right output [3]: [customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+Arguments: [customer_id#23], [customer_id#47], Inner, BuildRight
+
+(36) CometProject
+Input [5]: [customer_id#23, year_total#24, customer_id#47, customer_preferred_cust_flag#48, year_total#49]
+Arguments: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49], [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49]
+
+(37) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#50, c_customer_id#51, c_first_name#52, c_last_name#53, c_preferred_cust_flag#54, c_birth_country#55, c_login#56, c_email_address#57]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(40) CometFilter
-Input [8]: [c_customer_sk#51, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#56, c_login#57, c_email_address#58]
-Condition : (isnotnull(c_customer_sk#51) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#52, 16))))
+(38) CometFilter
+Input [8]: [c_customer_sk#50, c_customer_id#51, c_first_name#52, c_last_name#53, c_preferred_cust_flag#54, c_birth_country#55, c_login#56, c_email_address#57]
+Condition : (isnotnull(c_customer_sk#50) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#51, 16))))
-(41) CometProject
-Input [8]: [c_customer_sk#51, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#56, c_login#57, c_email_address#58]
-Arguments: [c_customer_sk#51, c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64], [c_customer_sk#51, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#52, 16)) AS c_customer_id#59, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#53, 20)) AS c_first_name#60, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#54, 30)) AS c_last_name#61, static_invoke(CharVarchar [...]
+(39) CometProject
+Input [8]: [c_customer_sk#50, c_customer_id#51, c_first_name#52, c_last_name#53, c_preferred_cust_flag#54, c_birth_country#55, c_login#56, c_email_address#57]
+Arguments: [c_customer_sk#50, c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63], [c_customer_sk#50, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#51, 16)) AS c_customer_id#58, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#52, 20)) AS c_first_name#59, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#53, 30)) AS c_last_name#60, static_invoke(CharVarchar [...]
-(42) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
+(40) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#68), dynamicpruningexpression(ws_sold_date_sk#68 IN dynamicpruning#69)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#67), dynamicpruningexpression(ws_sold_date_sk#67 IN dynamicpruning#68)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int,ws_ext_discount_amt:decimal(7,2),ws_ext_list_price:decimal(7,2)>
-(43) CometFilter
-Input [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Condition : isnotnull(ws_bill_customer_sk#65)
+(41) CometFilter
+Input [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Condition : isnotnull(ws_bill_customer_sk#64)
-(44) CometBroadcastExchange
-Input [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Arguments: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
+(42) CometBroadcastExchange
+Input [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Arguments: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-(45) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#51, c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64]
-Right output [4]: [ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Arguments: [c_customer_sk#51], [ws_bill_customer_sk#65], Inner, BuildRight
+(43) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#50, c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63]
+Right output [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Arguments: [c_customer_sk#50], [ws_bill_customer_sk#64], Inner, BuildRight
-(46) CometProject
-Input [12]: [c_customer_sk#51, c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_bill_customer_sk#65, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Arguments: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68], [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
+(44) CometProject
+Input [12]: [c_customer_sk#50, c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Arguments: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67], [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-(47) ReusedExchange [Reuses operator id: 11]
-Output [2]: [d_date_sk#70, d_year#71]
+(45) ReusedExchange [Reuses operator id: 11]
+Output [2]: [d_date_sk#69, d_year#70]
-(48) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68]
-Right output [2]: [d_date_sk#70, d_year#71]
-Arguments: [ws_sold_date_sk#68], [d_date_sk#70], Inner, BuildRight
+(46) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+Right output [2]: [d_date_sk#69, d_year#70]
+Arguments: [ws_sold_date_sk#67], [d_date_sk#69], Inner, BuildRight
-(49) CometProject
-Input [12]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, ws_sold_date_sk#68, d_date_sk#70, d_year#71]
-Arguments: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, d_year#71], [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, d_year#71]
+(47) CometProject
+Input [12]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67, d_date_sk#69, d_year#70]
+Arguments: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70], [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70]
+
+(48) CometHashAggregate
+Input [10]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70]
+Keys [8]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70]
+Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
+
+(49) CometExchange
+Input [9]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70, sum#71]
+Arguments: hashpartitioning(c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(50) CometHashAggregate
-Input [10]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, ws_ext_discount_amt#66, ws_ext_list_price#67, d_year#71]
-Keys [8]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71]
-Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))]
-
-(51) CometExchange
-Input [9]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, sum#72]
-Arguments: hashpartitioning(c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
-
-(52) CometColumnarToRow [codegen id : 2]
-Input [9]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, sum#72]
-
-(53) HashAggregate [codegen id : 2]
-Input [9]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71, sum#72]
-Keys [8]: [c_customer_id#59, c_first_name#60, c_last_name#61, c_preferred_cust_flag#62, c_birth_country#56, c_login#63, c_email_address#64, d_year#71]
-Functions [1]: [sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))#73]
-Results [2]: [c_customer_id#59 AS customer_id#74, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#67 - ws_ext_discount_amt#66)))#73,18,2) AS year_total#75]
-
-(54) Filter [codegen id : 2]
-Input [2]: [customer_id#74, year_total#75]
-Condition : (isnotnull(year_total#75) AND (year_total#75 > 0.00))
-
-(55) BroadcastExchange
-Input [2]: [customer_id#74, year_total#75]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=5]
-
-(56) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [customer_id#24]
-Right keys [1]: [customer_id#74]
-Join type: Inner
-Join condition: None
-
-(57) Project [codegen id : 4]
-Output [5]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50, year_total#75]
-Input [6]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50, customer_id#74, year_total#75]
-
-(58) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#76, c_customer_id#77, c_first_name#78, c_last_name#79, c_preferred_cust_flag#80, c_birth_country#81, c_login#82, c_email_address#83]
+Input [9]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70, sum#71]
+Keys [8]: [c_customer_id#58, c_first_name#59, c_last_name#60, c_preferred_cust_flag#61, c_birth_country#55, c_login#62, c_email_address#63, d_year#70]
+Functions [1]: [sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
+
+(51) CometFilter
+Input [2]: [customer_id#72, year_total#73]
+Condition : (isnotnull(year_total#73) AND (year_total#73 > 0.00))
+
+(52) CometBroadcastExchange
+Input [2]: [customer_id#72, year_total#73]
+Arguments: [customer_id#72, year_total#73]
+
+(53) CometBroadcastHashJoin
+Left output [4]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49]
+Right output [2]: [customer_id#72, year_total#73]
+Arguments: [customer_id#23], [customer_id#72], Inner, BuildRight
+
+(54) CometProject
+Input [6]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, customer_id#72, year_total#73]
+Arguments: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73], [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73]
+
+(55) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#74, c_customer_id#75, c_first_name#76, c_last_name#77, c_preferred_cust_flag#78, c_birth_country#79, c_login#80, c_email_address#81]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(59) CometFilter
-Input [8]: [c_customer_sk#76, c_customer_id#77, c_first_name#78, c_last_name#79, c_preferred_cust_flag#80, c_birth_country#81, c_login#82, c_email_address#83]
-Condition : (isnotnull(c_customer_sk#76) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#77, 16))))
+(56) CometFilter
+Input [8]: [c_customer_sk#74, c_customer_id#75, c_first_name#76, c_last_name#77, c_preferred_cust_flag#78, c_birth_country#79, c_login#80, c_email_address#81]
+Condition : (isnotnull(c_customer_sk#74) AND isnotnull(static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#75, 16))))
-(60) CometProject
-Input [8]: [c_customer_sk#76, c_customer_id#77, c_first_name#78, c_last_name#79, c_preferred_cust_flag#80, c_birth_country#81, c_login#82, c_email_address#83]
-Arguments: [c_customer_sk#76, c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89], [c_customer_sk#76, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#77, 16)) AS c_customer_id#84, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#78, 20)) AS c_first_name#85, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#79, 30)) AS c_last_name#86, static_invoke(CharVarchar [...]
+(57) CometProject
+Input [8]: [c_customer_sk#74, c_customer_id#75, c_first_name#76, c_last_name#77, c_preferred_cust_flag#78, c_birth_country#79, c_login#80, c_email_address#81]
+Arguments: [c_customer_sk#74, c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87], [c_customer_sk#74, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#75, 16)) AS c_customer_id#82, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_first_name#76, 20)) AS c_first_name#83, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_last_name#77, 30)) AS c_last_name#84, static_invoke(CharVarchar [...]
-(61) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
+(58) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#93), dynamicpruningexpression(ws_sold_date_sk#93 IN dynamicpruning#94)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#91), dynamicpruningexpression(ws_sold_date_sk#91 IN dynamicpruning#92)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int,ws_ext_discount_amt:decimal(7,2),ws_ext_list_price:decimal(7,2)>
-(62) CometFilter
-Input [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Condition : isnotnull(ws_bill_customer_sk#90)
+(59) CometFilter
+Input [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Condition : isnotnull(ws_bill_customer_sk#88)
+
+(60) CometBroadcastExchange
+Input [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Arguments: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+
+(61) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#74, c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87]
+Right output [4]: [ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Arguments: [c_customer_sk#74], [ws_bill_customer_sk#88], Inner, BuildRight
+
+(62) CometProject
+Input [12]: [c_customer_sk#74, c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_bill_customer_sk#88, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Arguments: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91], [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
-(63) CometBroadcastExchange
-Input [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Arguments: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
+(63) ReusedExchange [Reuses operator id: 28]
+Output [2]: [d_date_sk#93, d_year#94]
(64) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#76, c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89]
-Right output [4]: [ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Arguments: [c_customer_sk#76], [ws_bill_customer_sk#90], Inner, BuildRight
+Left output [10]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91]
+Right output [2]: [d_date_sk#93, d_year#94]
+Arguments: [ws_sold_date_sk#91], [d_date_sk#93], Inner, BuildRight
(65) CometProject
-Input [12]: [c_customer_sk#76, c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_bill_customer_sk#90, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Arguments: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93], [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-
-(66) ReusedExchange [Reuses operator id: 29]
-Output [2]: [d_date_sk#95, d_year#96]
-
-(67) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93]
-Right output [2]: [d_date_sk#95, d_year#96]
-Arguments: [ws_sold_date_sk#93], [d_date_sk#95], Inner, BuildRight
-
-(68) CometProject
-Input [12]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, ws_sold_date_sk#93, d_date_sk#95, d_year#96]
-Arguments: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, d_year#96], [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, d_year#96]
-
-(69) CometHashAggregate
-Input [10]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, ws_ext_discount_amt#91, ws_ext_list_price#92, d_year#96]
-Keys [8]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96]
-Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))]
-
-(70) CometExchange
-Input [9]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, sum#97]
-Arguments: hashpartitioning(c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
-
-(71) CometColumnarToRow [codegen id : 3]
-Input [9]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, sum#97]
-
-(72) HashAggregate [codegen id : 3]
-Input [9]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96, sum#97]
-Keys [8]: [c_customer_id#84, c_first_name#85, c_last_name#86, c_preferred_cust_flag#87, c_birth_country#81, c_login#88, c_email_address#89, d_year#96]
-Functions [1]: [sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))]
-Aggregate Attributes [1]: [sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))#73]
-Results [2]: [c_customer_id#84 AS customer_id#98, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#92 - ws_ext_discount_amt#91)))#73,18,2) AS year_total#99]
-
-(73) BroadcastExchange
-Input [2]: [customer_id#98, year_total#99]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=7]
-
-(74) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [customer_id#24]
-Right keys [1]: [customer_id#98]
-Join type: Inner
-Join condition: (CASE WHEN (year_total#75 > 0.00) THEN (year_total#99 / year_total#75) END > CASE WHEN (year_total#25 > 0.00) THEN (year_total#50 / year_total#25) END)
-
-(75) Project [codegen id : 4]
-Output [1]: [customer_preferred_cust_flag#49]
-Input [7]: [customer_id#24, year_total#25, customer_preferred_cust_flag#49, year_total#50, year_total#75, customer_id#98, year_total#99]
-
-(76) TakeOrderedAndProject
-Input [1]: [customer_preferred_cust_flag#49]
-Arguments: 100, [customer_preferred_cust_flag#49 ASC NULLS FIRST], [customer_preferred_cust_flag#49]
+Input [12]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, ws_sold_date_sk#91, d_date_sk#93, d_year#94]
+Arguments: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, d_year#94], [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, d_year#94]
+
+(66) CometHashAggregate
+Input [10]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, ws_ext_discount_amt#89, ws_ext_list_price#90, d_year#94]
+Keys [8]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94]
+Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#90 - ws_ext_discount_amt#89)))]
+
+(67) CometExchange
+Input [9]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94, sum#95]
+Arguments: hashpartitioning(c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
+
+(68) CometHashAggregate
+Input [9]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94, sum#95]
+Keys [8]: [c_customer_id#82, c_first_name#83, c_last_name#84, c_preferred_cust_flag#85, c_birth_country#79, c_login#86, c_email_address#87, d_year#94]
+Functions [1]: [sum(UnscaledValue((ws_ext_list_price#90 - ws_ext_discount_amt#89)))]
+
+(69) CometBroadcastExchange
+Input [2]: [customer_id#96, year_total#97]
+Arguments: [customer_id#96, year_total#97]
+
+(70) CometBroadcastHashJoin
+Left output [5]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73]
+Right output [2]: [customer_id#96, year_total#97]
+Arguments: [customer_id#23], [customer_id#96], Inner, (CASE WHEN (year_total#73 > 0.00) THEN (year_total#97 / year_total#73) END > CASE WHEN (year_total#24 > 0.00) THEN (year_total#49 / year_total#24) END), BuildRight
+
+(71) CometProject
+Input [7]: [customer_id#23, year_total#24, customer_preferred_cust_flag#48, year_total#49, year_total#73, customer_id#96, year_total#97]
+Arguments: [customer_preferred_cust_flag#48], [customer_preferred_cust_flag#48]
+
+(72) CometTakeOrderedAndProject
+Input [1]: [customer_preferred_cust_flag#48]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[customer_preferred_cust_flag#48 ASC NULLS FIRST], output=[customer_preferred_cust_flag#48]), [customer_preferred_cust_flag#48], 100, 0, [customer_preferred_cust_flag#48 ASC NULLS FIRST], [customer_preferred_cust_flag#48]
+
+(73) CometColumnarToRow [codegen id : 1]
+Input [1]: [customer_preferred_cust_flag#48]
===== Subqueries =====
Subquery:1 Hosting operator id = 4 Hosting Expression = ss_sold_date_sk#18 IN dynamicpruning#19
-BroadcastExchange (80)
-+- * CometColumnarToRow (79)
- +- CometFilter (78)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (77)
+BroadcastExchange (77)
++- * CometColumnarToRow (76)
+ +- CometFilter (75)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (74)
-(77) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(74) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(78) CometFilter
+(75) CometFilter
Input [2]: [d_date_sk#20, d_year#21]
Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_date_sk#20))
-(79) CometColumnarToRow [codegen id : 1]
+(76) CometColumnarToRow [codegen id : 1]
Input [2]: [d_date_sk#20, d_year#21]
-(80) BroadcastExchange
+(77) BroadcastExchange
Input [2]: [d_date_sk#20, d_year#21]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=8]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=5]
-Subquery:2 Hosting operator id = 22 Hosting Expression = ss_sold_date_sk#43 IN dynamicpruning#44
-BroadcastExchange (84)
-+- * CometColumnarToRow (83)
- +- CometFilter (82)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (81)
+Subquery:2 Hosting operator id = 21 Hosting Expression = ss_sold_date_sk#42 IN dynamicpruning#43
+BroadcastExchange (81)
++- * CometColumnarToRow (80)
+ +- CometFilter (79)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (78)
-(81) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#45, d_year#46]
+(78) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#44, d_year#45]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(82) CometFilter
-Input [2]: [d_date_sk#45, d_year#46]
-Condition : ((isnotnull(d_year#46) AND (d_year#46 = 2002)) AND isnotnull(d_date_sk#45))
+(79) CometFilter
+Input [2]: [d_date_sk#44, d_year#45]
+Condition : ((isnotnull(d_year#45) AND (d_year#45 = 2002)) AND isnotnull(d_date_sk#44))
-(83) CometColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#45, d_year#46]
+(80) CometColumnarToRow [codegen id : 1]
+Input [2]: [d_date_sk#44, d_year#45]
-(84) BroadcastExchange
-Input [2]: [d_date_sk#45, d_year#46]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=9]
+(81) BroadcastExchange
+Input [2]: [d_date_sk#44, d_year#45]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=6]
-Subquery:3 Hosting operator id = 42 Hosting Expression = ws_sold_date_sk#68 IN dynamicpruning#19
+Subquery:3 Hosting operator id = 40 Hosting Expression = ws_sold_date_sk#67 IN dynamicpruning#19
-Subquery:4 Hosting operator id = 61 Hosting Expression = ws_sold_date_sk#93 IN dynamicpruning#44
+Subquery:4 Hosting operator id = 58 Hosting Expression = ws_sold_date_sk#91 IN dynamicpruning#43
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/extended.txt
index fdab1c6f0..781eae905 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/extended.txt
@@ -1,79 +1,76 @@
-TakeOrderedAndProject
-+- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- Project
- : : +- BroadcastHashJoin
- : : :- Filter
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(c_customer_id#1 AS customer_id#2, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#3 - ss_ext_discount_amt#4)))#5,18,2) AS year_total#6)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometBroadcastHashJoin
- : : : : :- CometProject
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- : : : : +- CometBroadcastExchange
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : : +- SubqueryBroadcast
- : : : : +- BroadcastExchange
- : : : : +- CometColumnarToRow
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- BroadcastExchange
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: Vector(c_customer_id#7 AS customer_id#8, c_preferred_cust_flag#9 AS customer_preferred_cust_flag#10, MakeDecimal(sum(UnscaledValue((ss_ext_list_price#11 - ss_ext_discount_amt#12)))#5,18,2) AS year_total#13)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- : : : +- CometBroadcastExchange
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : +- SubqueryBroadcast
- : : : +- BroadcastExchange
- : : : +- CometColumnarToRow
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- CometBroadcastExchange
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- BroadcastExchange
- : +- Filter
- : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(c_customer_id#14 AS customer_id#15, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#16 - ws_ext_discount_amt#17)))#18,18,2) AS year_total#19)]
- : +- CometColumnarToRow
- : +- CometExchange
- : +- CometHashAggregate
- : +- CometProject
- : +- CometBroadcastHashJoin
- : :- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- : : +- CometBroadcastExchange
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- ReusedSubquery
- : +- CometBroadcastExchange
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- BroadcastExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: Vector(c_customer_id#20 AS customer_id#21, MakeDecimal(sum(UnscaledValue((ws_ext_list_price#22 - ws_ext_discount_amt#23)))#18,18,2) AS year_total#24)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometBroadcastHashJoin
+ : : : : :- CometProject
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : : +- SubqueryBroadcast
+ : : : : +- BroadcastExchange
+ : : : : +- CometColumnarToRow
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ : : : +- CometBroadcastExchange
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : +- SubqueryBroadcast
+ : : : +- BroadcastExchange
+ : : : +- CometColumnarToRow
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometFilter
+ : +- CometHashAggregate
+ : +- CometExchange
+ : +- CometHashAggregate
+ : +- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- ReusedSubquery
+ : +- CometBroadcastExchange
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -91,4 +88,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 64 out of 86 eligible operators (74%). Final plan contains 6 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 80 out of 86 eligible operators (93%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/simplified.txt
index 0e6712214..4599ecf53 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q11/simplified.txt
@@ -1,15 +1,15 @@
-TakeOrderedAndProject [customer_preferred_cust_flag]
- WholeStageCodegen (4)
- Project [customer_preferred_cust_flag]
- BroadcastHashJoin [customer_id,customer_id,year_total,year_total,year_total,year_total]
- Project [customer_id,year_total,customer_preferred_cust_flag,year_total,year_total]
- BroadcastHashJoin [customer_id,customer_id]
- Project [customer_id,year_total,customer_preferred_cust_flag,year_total]
- BroadcastHashJoin [customer_id,customer_id]
- Filter [year_total]
- HashAggregate [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum] [sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt))),customer_id,year_total,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [customer_preferred_cust_flag]
+ CometProject [customer_preferred_cust_flag]
+ CometBroadcastHashJoin [customer_id,year_total,customer_preferred_cust_flag,year_total,year_total,customer_id,year_total]
+ CometProject [customer_id,year_total,customer_preferred_cust_flag,year_total,year_total]
+ CometBroadcastHashJoin [customer_id,year_total,customer_preferred_cust_flag,year_total,customer_id,year_total]
+ CometProject [customer_id,year_total,customer_preferred_cust_flag,year_total]
+ CometBroadcastHashJoin [customer_id,year_total,customer_id,customer_preferred_cust_flag,year_total]
+ CometFilter [customer_id,year_total]
+ CometHashAggregate [c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum] [customer_id,year_total,c_customer_id,sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt)))]
CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #1
CometHashAggregate [ss_ext_list_price,ss_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
@@ -32,72 +32,60 @@ TakeOrderedAndProject [customer_preferred_cust_flag]
CometBroadcastExchange [d_date_sk,d_year] #4
CometFilter [d_date_sk,d_year]
CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (1)
- HashAggregate [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum] [sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt))),customer_id,customer_preferred_cust_flag,year_total,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #6
- CometHashAggregate [ss_ext_list_price,ss_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
- CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometBroadcastExchange [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk] #7
- CometFilter [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #2
- BroadcastExchange #8
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometFilter [d_date_sk,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- CometBroadcastExchange [d_date_sk,d_year] #9
- CometFilter [d_date_sk,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
- InputAdapter
- BroadcastExchange #10
- WholeStageCodegen (2)
- Filter [year_total]
- HashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt))),customer_id,year_total,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
- CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
- CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #12
- CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk,d_year] #4
- InputAdapter
- BroadcastExchange #13
- WholeStageCodegen (3)
- HashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt))),customer_id,year_total,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
- CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
- CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #15
- CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #2
- ReusedExchange [d_date_sk,d_year] #9
+ CometBroadcastExchange [customer_id,customer_preferred_cust_flag,year_total] #5
+ CometHashAggregate [c_first_name,c_last_name,d_year,c_birth_country,c_login,c_email_address,sum] [customer_id,customer_preferred_cust_flag,year_total,c_customer_id,c_preferred_cust_flag,sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt)))]
+ CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #6
+ CometHashAggregate [ss_ext_list_price,ss_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
+ CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometBroadcastExchange [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk] #7
+ CometFilter [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #2
+ BroadcastExchange #8
+ WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometFilter [d_date_sk,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
+ CometBroadcastExchange [d_date_sk,d_year] #9
+ CometFilter [d_date_sk,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year]
+ CometBroadcastExchange [customer_id,year_total] #10
+ CometFilter [customer_id,year_total]
+ CometHashAggregate [c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [customer_id,year_total,c_customer_id,sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt)))]
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
+ CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
+ CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #12
+ CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk,d_year] #4
+ CometBroadcastExchange [customer_id,year_total] #13
+ CometHashAggregate [c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum] [customer_id,year_total,c_customer_id,sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt)))]
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
+ CometHashAggregate [ws_ext_list_price,ws_ext_discount_amt] [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
+ CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #15
+ CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #2
+ ReusedExchange [d_date_sk,d_year] #9
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/explain.txt
index 866cc628a..8811d04e5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/explain.txt
@@ -1,28 +1,27 @@
== Physical Plan ==
-TakeOrderedAndProject (24)
-+- * Project (23)
- +- Window (22)
- +- * CometColumnarToRow (21)
- +- CometSort (20)
- +- CometColumnarExchange (19)
- +- * HashAggregate (18)
- +- * CometColumnarToRow (17)
- +- CometExchange (16)
- +- CometHashAggregate (15)
- +- CometProject (14)
- +- CometBroadcastHashJoin (13)
- :- CometProject (8)
- : +- CometBroadcastHashJoin (7)
- : :- CometFilter (2)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
- : +- CometBroadcastExchange (6)
- : +- CometProject (5)
- : +- CometFilter (4)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
- +- CometBroadcastExchange (12)
- +- CometProject (11)
- +- CometFilter (10)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
+TakeOrderedAndProject (23)
++- * Project (22)
+ +- Window (21)
+ +- * CometColumnarToRow (20)
+ +- CometSort (19)
+ +- CometExchange (18)
+ +- CometHashAggregate (17)
+ +- CometExchange (16)
+ +- CometHashAggregate (15)
+ +- CometProject (14)
+ +- CometBroadcastHashJoin (13)
+ :- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
+ : :- CometFilter (2)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
+ +- CometBroadcastExchange (12)
+ +- CometProject (11)
+ +- CometFilter (10)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
@@ -102,68 +101,63 @@ Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#2))]
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Arguments: hashpartitioning(i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 1]
-Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
-
-(18) HashAggregate [codegen id : 1]
+(17) CometHashAggregate
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ws_ext_sales_price#2))#17]
-Results [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#2))#17,17,2) AS itemrevenue#18, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#2))#17,17,2) AS _w0#19, i_item_id#11]
-(19) CometColumnarExchange
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+(18) CometExchange
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(20) CometSort
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11], [i_class#12 ASC NULLS FIRST]
+(19) CometSort
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11], [i_class#12 ASC NULLS FIRST]
-(21) CometColumnarToRow [codegen id : 2]
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
+(20) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
-(22) Window
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [sum(_w0#19) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#20], [i_class#12]
+(21) Window
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [sum(_w0#18) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#19], [i_class#12]
-(23) Project [codegen id : 3]
-Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, ((_w0#19 * 100) / _we0#20) AS revenueratio#21, i_item_id#11]
-Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11, _we0#20]
+(22) Project [codegen id : 2]
+Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, ((_w0#18 * 100) / _we0#19) AS revenueratio#20, i_item_id#11]
+Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11, _we0#19]
-(24) TakeOrderedAndProject
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21, i_item_id#11]
-Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#21 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21]
+(23) TakeOrderedAndProject
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20, i_item_id#11]
+Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#20 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ws_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (29)
-+- * CometColumnarToRow (28)
- +- CometProject (27)
- +- CometFilter (26)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (25)
+BroadcastExchange (28)
++- * CometColumnarToRow (27)
+ +- CometProject (26)
+ +- CometFilter (25)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (24)
-(25) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(24) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(26) CometFilter
+(25) CometFilter
Input [2]: [d_date_sk#14, d_date#15]
Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(27) CometProject
+(26) CometProject
Input [2]: [d_date_sk#14, d_date#15]
Arguments: [d_date_sk#14], [d_date_sk#14]
-(28) CometColumnarToRow [codegen id : 1]
+(27) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#14]
-(29) BroadcastExchange
+(28) BroadcastExchange
Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/extended.txt
index c467ccd3e..3f41c97ff 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/extended.txt
@@ -3,30 +3,29 @@ TakeOrderedAndProject
+- Window [COMET: WindowExec is not fully compatible with Spark (Native WindowExec has known correctness issues). To enable it anyway, set spark.comet.operator.WindowExec.allowIncompatible=true. For more information, refer to the Comet Compatibility Guide (https://datafusion.apache.org/comet/user-guide/compatibility.html).]
+- CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_item_desc#1, i_category#2, i_class#3, i_current_price#4, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#5))#6,17,2) AS itemrevenue#7, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#5))#6,17,2) AS _w0#8, i_item_id#9)]
- +- CometColumnarToRow
- +- CometExchange
- +- CometHashAggregate
- +- CometProject
- +- CometBroadcastHashJoin
- :- CometProject
- : +- CometBroadcastHashJoin
- : :- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- SubqueryBroadcast
- : : +- BroadcastExchange
- : : +- CometColumnarToRow
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- CometBroadcastExchange
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
- +- CometBroadcastExchange
- +- CometProject
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- SubqueryBroadcast
+ : : +- BroadcastExchange
+ : : +- CometColumnarToRow
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 21 out of 27 eligible operators (77%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 22 out of 27 eligible operators (81%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/simplified.txt
index 1fdffc3b7..fb83fd2f9 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12.native_iceberg_compat/simplified.txt
@@ -1,38 +1,35 @@
TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_current_price,itemrevenue]
- WholeStageCodegen (3)
+ WholeStageCodegen (2)
Project [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,_we0,i_item_id]
InputAdapter
Window [_w0,i_class]
- WholeStageCodegen (2)
+ WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id]
- CometColumnarExchange [i_class] #1
- WholeStageCodegen (1)
- HashAggregate [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum] [sum(UnscaledValue(ws_ext_sales_price)),itemrevenue,_w0,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
- CometHashAggregate [ws_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
- CometProject [ws_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
- CometProject [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
- CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
- CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastExchange [d_date_sk] #5
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometExchange [i_class] #1
+ CometHashAggregate [sum] [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id,sum(UnscaledValue(ws_ext_sales_price))]
+ CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
+ CometHashAggregate [ws_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
+ CometProject [ws_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
+ CometProject [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
+ CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastExchange [d_date_sk] #5
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/explain.txt
index 866cc628a..8811d04e5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/explain.txt
@@ -1,28 +1,27 @@
== Physical Plan ==
-TakeOrderedAndProject (24)
-+- * Project (23)
- +- Window (22)
- +- * CometColumnarToRow (21)
- +- CometSort (20)
- +- CometColumnarExchange (19)
- +- * HashAggregate (18)
- +- * CometColumnarToRow (17)
- +- CometExchange (16)
- +- CometHashAggregate (15)
- +- CometProject (14)
- +- CometBroadcastHashJoin (13)
- :- CometProject (8)
- : +- CometBroadcastHashJoin (7)
- : :- CometFilter (2)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
- : +- CometBroadcastExchange (6)
- : +- CometProject (5)
- : +- CometFilter (4)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
- +- CometBroadcastExchange (12)
- +- CometProject (11)
- +- CometFilter (10)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
+TakeOrderedAndProject (23)
++- * Project (22)
+ +- Window (21)
+ +- * CometColumnarToRow (20)
+ +- CometSort (19)
+ +- CometExchange (18)
+ +- CometHashAggregate (17)
+ +- CometExchange (16)
+ +- CometHashAggregate (15)
+ +- CometProject (14)
+ +- CometBroadcastHashJoin (13)
+ :- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
+ : :- CometFilter (2)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
+ +- CometBroadcastExchange (12)
+ +- CometProject (11)
+ +- CometFilter (10)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
@@ -102,68 +101,63 @@ Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#2))]
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Arguments: hashpartitioning(i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 1]
-Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
-
-(18) HashAggregate [codegen id : 1]
+(17) CometHashAggregate
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ws_ext_sales_price#2))#17]
-Results [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#2))#17,17,2) AS itemrevenue#18, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#2))#17,17,2) AS _w0#19, i_item_id#11]
-(19) CometColumnarExchange
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+(18) CometExchange
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(20) CometSort
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11], [i_class#12 ASC NULLS FIRST]
+(19) CometSort
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11], [i_class#12 ASC NULLS FIRST]
-(21) CometColumnarToRow [codegen id : 2]
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
+(20) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
-(22) Window
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [sum(_w0#19) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#20], [i_class#12]
+(21) Window
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [sum(_w0#18) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#19], [i_class#12]
-(23) Project [codegen id : 3]
-Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, ((_w0#19 * 100) / _we0#20) AS revenueratio#21, i_item_id#11]
-Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11, _we0#20]
+(22) Project [codegen id : 2]
+Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, ((_w0#18 * 100) / _we0#19) AS revenueratio#20, i_item_id#11]
+Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11, _we0#19]
-(24) TakeOrderedAndProject
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21, i_item_id#11]
-Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#21 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21]
+(23) TakeOrderedAndProject
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20, i_item_id#11]
+Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#20 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ws_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (29)
-+- * CometColumnarToRow (28)
- +- CometProject (27)
- +- CometFilter (26)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (25)
+BroadcastExchange (28)
++- * CometColumnarToRow (27)
+ +- CometProject (26)
+ +- CometFilter (25)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (24)
-(25) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(24) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(26) CometFilter
+(25) CometFilter
Input [2]: [d_date_sk#14, d_date#15]
Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(27) CometProject
+(26) CometProject
Input [2]: [d_date_sk#14, d_date#15]
Arguments: [d_date_sk#14], [d_date_sk#14]
-(28) CometColumnarToRow [codegen id : 1]
+(27) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#14]
-(29) BroadcastExchange
+(28) BroadcastExchange
Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/extended.txt
index c467ccd3e..3f41c97ff 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/extended.txt
@@ -3,30 +3,29 @@ TakeOrderedAndProject
+- Window [COMET: WindowExec is not fully compatible with Spark (Native WindowExec has known correctness issues). To enable it anyway, set spark.comet.operator.WindowExec.allowIncompatible=true. For more information, refer to the Comet Compatibility Guide (https://datafusion.apache.org/comet/user-guide/compatibility.html).]
+- CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_item_desc#1, i_category#2, i_class#3, i_current_price#4, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#5))#6,17,2) AS itemrevenue#7, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#5))#6,17,2) AS _w0#8, i_item_id#9)]
- +- CometColumnarToRow
- +- CometExchange
- +- CometHashAggregate
- +- CometProject
- +- CometBroadcastHashJoin
- :- CometProject
- : +- CometBroadcastHashJoin
- : :- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- SubqueryBroadcast
- : : +- BroadcastExchange
- : : +- CometColumnarToRow
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- CometBroadcastExchange
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
- +- CometBroadcastExchange
- +- CometProject
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- SubqueryBroadcast
+ : : +- BroadcastExchange
+ : : +- CometColumnarToRow
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 21 out of 27 eligible operators (77%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 22 out of 27 eligible operators (81%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/simplified.txt
index 1fdffc3b7..fb83fd2f9 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q12/simplified.txt
@@ -1,38 +1,35 @@
TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_current_price,itemrevenue]
- WholeStageCodegen (3)
+ WholeStageCodegen (2)
Project [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,_we0,i_item_id]
InputAdapter
Window [_w0,i_class]
- WholeStageCodegen (2)
+ WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id]
- CometColumnarExchange [i_class] #1
- WholeStageCodegen (1)
- HashAggregate [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum] [sum(UnscaledValue(ws_ext_sales_price)),itemrevenue,_w0,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
- CometHashAggregate [ws_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
- CometProject [ws_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
- CometProject [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
- CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
- CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastExchange [d_date_sk] #5
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometExchange [i_class] #1
+ CometHashAggregate [sum] [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id,sum(UnscaledValue(ws_ext_sales_price))]
+ CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
+ CometHashAggregate [ws_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
+ CometProject [ws_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
+ CometProject [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_item_sk,ws_ext_sales_price,ws_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
+ CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastExchange [d_date_sk] #5
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/explain.txt
index a4c1278d0..027d67941 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/explain.txt
@@ -1,6 +1,6 @@
== Physical Plan ==
-* HashAggregate (34)
-+- * CometColumnarToRow (33)
+* CometColumnarToRow (34)
++- CometHashAggregate (33)
+- CometExchange (32)
+- CometHashAggregate (31)
+- CometProject (30)
@@ -188,15 +188,13 @@ Functions [4]: [partial_avg(ss_quantity#5), partial_avg(UnscaledValue(ss_ext_sal
Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(33) CometColumnarToRow [codegen id : 1]
-Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
-
-(34) HashAggregate [codegen id : 1]
+(33) CometHashAggregate
Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
Keys: []
Functions [4]: [avg(ss_quantity#5), avg(UnscaledValue(ss_ext_sales_price#7)), avg(UnscaledValue(ss_ext_wholesale_cost#8)), sum(UnscaledValue(ss_ext_wholesale_cost#8))]
-Aggregate Attributes [4]: [avg(ss_quantity#5)#33, avg(UnscaledValue(ss_ext_sales_price#7))#34, avg(UnscaledValue(ss_ext_wholesale_cost#8))#35, sum(UnscaledValue(ss_ext_wholesale_cost#8))#36]
-Results [4]: [avg(ss_quantity#5)#33 AS avg(ss_quantity)#37, cast((avg(UnscaledValue(ss_ext_sales_price#7))#34 / 100.0) as decimal(11,6)) AS avg(ss_ext_sales_price)#38, cast((avg(UnscaledValue(ss_ext_wholesale_cost#8))#35 / 100.0) as decimal(11,6)) AS avg(ss_ext_wholesale_cost)#39, MakeDecimal(sum(UnscaledValue(ss_ext_wholesale_cost#8))#36,17,2) AS sum(ss_ext_wholesale_cost)#40]
+
+(34) CometColumnarToRow [codegen id : 1]
+Input [4]: [avg(ss_quantity)#33, avg(ss_ext_sales_price)#34, avg(ss_ext_wholesale_cost)#35, sum(ss_ext_wholesale_cost)#36]
===== Subqueries =====
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/extended.txt
index f74c20954..08e9beb69 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/extended.txt
@@ -1,5 +1,5 @@
- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(avg(ss_quantity#1)#2 AS avg(ss_quantity)#3, cast((avg(UnscaledValue(ss_ext_sales_price#4))#5 / 100.0) as decimal(11,6)) AS avg(ss_ext_sales_price)#6, cast((avg(UnscaledValue(ss_ext_wholesale_cost#7))#8 / 100.0) as decimal(11,6)) AS avg(ss_ext_wholesale_cost)#9, MakeDecimal(sum(UnscaledValue(ss_ext_wholesale_cost#7))#10,17,2) AS sum(ss_ext_wholesale_cost)#11)]
-+- CometColumnarToRow
+CometColumnarToRow
++- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -39,4 +39,4 @@
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.household_demographics
-Comet accelerated 35 out of 38 eligible operators (92%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 36 out of 38 eligible operators (94%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/simplified.txt
index f7cfb4ea0..8ef882a43 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13.native_iceberg_compat/simplified.txt
@@ -1,7 +1,7 @@
WholeStageCodegen (1)
- HashAggregate [sum,count,sum,count,sum,count,sum] [avg(ss_quantity),avg(UnscaledValue(ss_ext_sales_price)),avg(UnscaledValue(ss_ext_wholesale_cost)),sum(UnscaledValue(ss_ext_wholesale_cost)),avg(ss_quantity),avg(ss_ext_sales_price),avg(ss_ext_wholesale_cost),sum(ss_ext_wholesale_cost),sum,count,sum,count,sum,count,sum]
- CometColumnarToRow
- InputAdapter
+ CometColumnarToRow
+ InputAdapter
+ CometHashAggregate [sum,count,sum,count,sum,count,sum] [avg(ss_quantity),avg(ss_ext_sales_price),avg(ss_ext_wholesale_cost),sum(ss_ext_wholesale_cost),avg(ss_quantity),avg(UnscaledValue(ss_ext_sales_price)),avg(UnscaledValue(ss_ext_wholesale_cost)),sum(UnscaledValue(ss_ext_wholesale_cost))]
CometExchange #1
CometHashAggregate [ss_quantity,ss_ext_sales_price,ss_ext_wholesale_cost] [sum,count,sum,count,sum,count,sum]
CometProject [ss_quantity,ss_ext_sales_price,ss_ext_wholesale_cost]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/explain.txt
index a4c1278d0..027d67941 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/explain.txt
@@ -1,6 +1,6 @@
== Physical Plan ==
-* HashAggregate (34)
-+- * CometColumnarToRow (33)
+* CometColumnarToRow (34)
++- CometHashAggregate (33)
+- CometExchange (32)
+- CometHashAggregate (31)
+- CometProject (30)
@@ -188,15 +188,13 @@ Functions [4]: [partial_avg(ss_quantity#5), partial_avg(UnscaledValue(ss_ext_sal
Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(33) CometColumnarToRow [codegen id : 1]
-Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
-
-(34) HashAggregate [codegen id : 1]
+(33) CometHashAggregate
Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
Keys: []
Functions [4]: [avg(ss_quantity#5), avg(UnscaledValue(ss_ext_sales_price#7)), avg(UnscaledValue(ss_ext_wholesale_cost#8)), sum(UnscaledValue(ss_ext_wholesale_cost#8))]
-Aggregate Attributes [4]: [avg(ss_quantity#5)#33, avg(UnscaledValue(ss_ext_sales_price#7))#34, avg(UnscaledValue(ss_ext_wholesale_cost#8))#35, sum(UnscaledValue(ss_ext_wholesale_cost#8))#36]
-Results [4]: [avg(ss_quantity#5)#33 AS avg(ss_quantity)#37, cast((avg(UnscaledValue(ss_ext_sales_price#7))#34 / 100.0) as decimal(11,6)) AS avg(ss_ext_sales_price)#38, cast((avg(UnscaledValue(ss_ext_wholesale_cost#8))#35 / 100.0) as decimal(11,6)) AS avg(ss_ext_wholesale_cost)#39, MakeDecimal(sum(UnscaledValue(ss_ext_wholesale_cost#8))#36,17,2) AS sum(ss_ext_wholesale_cost)#40]
+
+(34) CometColumnarToRow [codegen id : 1]
+Input [4]: [avg(ss_quantity)#33, avg(ss_ext_sales_price)#34, avg(ss_ext_wholesale_cost)#35, sum(ss_ext_wholesale_cost)#36]
===== Subqueries =====
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/extended.txt
index f74c20954..08e9beb69 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/extended.txt
@@ -1,5 +1,5 @@
- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(avg(ss_quantity#1)#2 AS avg(ss_quantity)#3, cast((avg(UnscaledValue(ss_ext_sales_price#4))#5 / 100.0) as decimal(11,6)) AS avg(ss_ext_sales_price)#6, cast((avg(UnscaledValue(ss_ext_wholesale_cost#7))#8 / 100.0) as decimal(11,6)) AS avg(ss_ext_wholesale_cost)#9, MakeDecimal(sum(UnscaledValue(ss_ext_wholesale_cost#7))#10,17,2) AS sum(ss_ext_wholesale_cost)#11)]
-+- CometColumnarToRow
+CometColumnarToRow
++- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -39,4 +39,4 @@
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.household_demographics
-Comet accelerated 35 out of 38 eligible operators (92%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 36 out of 38 eligible operators (94%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/simplified.txt
index f7cfb4ea0..8ef882a43 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q13/simplified.txt
@@ -1,7 +1,7 @@
WholeStageCodegen (1)
- HashAggregate [sum,count,sum,count,sum,count,sum] [avg(ss_quantity),avg(UnscaledValue(ss_ext_sales_price)),avg(UnscaledValue(ss_ext_wholesale_cost)),sum(UnscaledValue(ss_ext_wholesale_cost)),avg(ss_quantity),avg(ss_ext_sales_price),avg(ss_ext_wholesale_cost),sum(ss_ext_wholesale_cost),sum,count,sum,count,sum,count,sum]
- CometColumnarToRow
- InputAdapter
+ CometColumnarToRow
+ InputAdapter
+ CometHashAggregate [sum,count,sum,count,sum,count,sum] [avg(ss_quantity),avg(ss_ext_sales_price),avg(ss_ext_wholesale_cost),sum(ss_ext_wholesale_cost),avg(ss_quantity),avg(UnscaledValue(ss_ext_sales_price)),avg(UnscaledValue(ss_ext_wholesale_cost)),sum(UnscaledValue(ss_ext_wholesale_cost))]
CometExchange #1
CometHashAggregate [ss_quantity,ss_ext_sales_price,ss_ext_wholesale_cost] [sum,count,sum,count,sum,count,sum]
CometProject [ss_quantity,ss_ext_sales_price,ss_ext_wholesale_cost]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/explain.txt
index 4527be4e9..8a4fb53ec 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (24)
-+- * HashAggregate (23)
- +- * CometColumnarToRow (22)
+* CometColumnarToRow (24)
++- CometTakeOrderedAndProject (23)
+ +- CometHashAggregate (22)
+- CometExchange (21)
+- CometHashAggregate (20)
+- CometProject (19)
@@ -126,19 +126,17 @@ Functions [1]: [partial_sum(UnscaledValue(cs_sales_price#2))]
Input [2]: [ca_zip#11, sum#15]
Arguments: hashpartitioning(ca_zip#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(22) CometColumnarToRow [codegen id : 1]
-Input [2]: [ca_zip#11, sum#15]
-
-(23) HashAggregate [codegen id : 1]
+(22) CometHashAggregate
Input [2]: [ca_zip#11, sum#15]
Keys [1]: [ca_zip#11]
Functions [1]: [sum(UnscaledValue(cs_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(cs_sales_price#2))#16]
-Results [2]: [ca_zip#11, MakeDecimal(sum(UnscaledValue(cs_sales_price#2))#16,17,2) AS sum(cs_sales_price)#17]
-(24) TakeOrderedAndProject
-Input [2]: [ca_zip#11, sum(cs_sales_price)#17]
-Arguments: 100, [ca_zip#11 ASC NULLS FIRST], [ca_zip#11, sum(cs_sales_price)#17]
+(23) CometTakeOrderedAndProject
+Input [2]: [ca_zip#11, sum(cs_sales_price)#16]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ca_zip#11 ASC NULLS FIRST], output=[ca_zip#11,sum(cs_sales_price)#16]), [ca_zip#11, sum(cs_sales_price)#16], 100, 0, [ca_zip#11 ASC NULLS FIRST], [ca_zip#11, sum(cs_sales_price)#16]
+
+(24) CometColumnarToRow [codegen id : 1]
+Input [2]: [ca_zip#11, sum(cs_sales_price)#16]
===== Subqueries =====
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/extended.txt
index d99c44f9e..6de0c6485 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_zip#1, MakeDecimal(sum(UnscaledValue(cs_sales_price#2))#3,17,2) AS sum(cs_sales_price)#4)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -29,4 +29,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 24 out of 28 eligible operators (85%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 26 out of 28 eligible operators (92%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/simplified.txt
index 6c4d568d4..c39b96efe 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15.native_iceberg_compat/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [ca_zip,sum(cs_sales_price)]
- WholeStageCodegen (1)
- HashAggregate [ca_zip,sum] [sum(UnscaledValue(cs_sales_price)),sum(cs_sales_price),sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [ca_zip,sum(cs_sales_price)]
+ CometHashAggregate [sum] [ca_zip,sum(cs_sales_price),sum(UnscaledValue(cs_sales_price))]
CometExchange [ca_zip] #1
CometHashAggregate [cs_sales_price] [ca_zip,sum]
CometProject [cs_sales_price,ca_zip]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/explain.txt
index 4527be4e9..8a4fb53ec 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (24)
-+- * HashAggregate (23)
- +- * CometColumnarToRow (22)
+* CometColumnarToRow (24)
++- CometTakeOrderedAndProject (23)
+ +- CometHashAggregate (22)
+- CometExchange (21)
+- CometHashAggregate (20)
+- CometProject (19)
@@ -126,19 +126,17 @@ Functions [1]: [partial_sum(UnscaledValue(cs_sales_price#2))]
Input [2]: [ca_zip#11, sum#15]
Arguments: hashpartitioning(ca_zip#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(22) CometColumnarToRow [codegen id : 1]
-Input [2]: [ca_zip#11, sum#15]
-
-(23) HashAggregate [codegen id : 1]
+(22) CometHashAggregate
Input [2]: [ca_zip#11, sum#15]
Keys [1]: [ca_zip#11]
Functions [1]: [sum(UnscaledValue(cs_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(cs_sales_price#2))#16]
-Results [2]: [ca_zip#11, MakeDecimal(sum(UnscaledValue(cs_sales_price#2))#16,17,2) AS sum(cs_sales_price)#17]
-(24) TakeOrderedAndProject
-Input [2]: [ca_zip#11, sum(cs_sales_price)#17]
-Arguments: 100, [ca_zip#11 ASC NULLS FIRST], [ca_zip#11, sum(cs_sales_price)#17]
+(23) CometTakeOrderedAndProject
+Input [2]: [ca_zip#11, sum(cs_sales_price)#16]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ca_zip#11 ASC NULLS FIRST], output=[ca_zip#11,sum(cs_sales_price)#16]), [ca_zip#11, sum(cs_sales_price)#16], 100, 0, [ca_zip#11 ASC NULLS FIRST], [ca_zip#11, sum(cs_sales_price)#16]
+
+(24) CometColumnarToRow [codegen id : 1]
+Input [2]: [ca_zip#11, sum(cs_sales_price)#16]
===== Subqueries =====
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/extended.txt
index d99c44f9e..6de0c6485 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_zip#1, MakeDecimal(sum(UnscaledValue(cs_sales_price#2))#3,17,2) AS sum(cs_sales_price)#4)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -29,4 +29,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 24 out of 28 eligible operators (85%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 26 out of 28 eligible operators (92%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/simplified.txt
index 6c4d568d4..c39b96efe 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q15/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [ca_zip,sum(cs_sales_price)]
- WholeStageCodegen (1)
- HashAggregate [ca_zip,sum] [sum(UnscaledValue(cs_sales_price)),sum(cs_sales_price),sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [ca_zip,sum(cs_sales_price)]
+ CometHashAggregate [sum] [ca_zip,sum(cs_sales_price),sum(UnscaledValue(cs_sales_price))]
CometExchange [ca_zip] #1
CometHashAggregate [cs_sales_price] [ca_zip,sum]
CometProject [cs_sales_price,ca_zip]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/explain.txt
index 678ae3c7c..7a4c0fa40 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/explain.txt
@@ -1,6 +1,6 @@
== Physical Plan ==
-* HashAggregate (41)
-+- * CometColumnarToRow (40)
+* CometColumnarToRow (41)
++- CometHashAggregate (40)
+- CometColumnarExchange (39)
+- * HashAggregate (38)
+- * HashAggregate (37)
@@ -225,13 +225,11 @@ Results [3]: [sum#20, sum#21, count#25]
Input [3]: [sum#20, sum#21, count#25]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-(40) CometColumnarToRow [codegen id : 2]
-Input [3]: [sum#20, sum#21, count#25]
-
-(41) HashAggregate [codegen id : 2]
+(40) CometHashAggregate
Input [3]: [sum#20, sum#21, count#25]
Keys: []
Functions [3]: [sum(UnscaledValue(cs_ext_ship_cost#6)), sum(UnscaledValue(cs_net_profit#7)), count(distinct cs_order_number#5)]
-Aggregate Attributes [3]: [sum(UnscaledValue(cs_ext_ship_cost#6))#22, sum(UnscaledValue(cs_net_profit#7))#23, count(cs_order_number#5)#24]
-Results [3]: [count(cs_order_number#5)#24 AS order count #26, MakeDecimal(sum(UnscaledValue(cs_ext_ship_cost#6))#22,17,2) AS total shipping cost #27, MakeDecimal(sum(UnscaledValue(cs_net_profit#7))#23,17,2) AS total net profit #28]
+
+(41) CometColumnarToRow [codegen id : 2]
+Input [3]: [order count #26, total shipping cost #27, total net profit #28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/extended.txt
index c6125cb26..2ad029e44 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/extended.txt
@@ -1,5 +1,5 @@
- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(count(cs_order_number#1)#2 AS order count #3, MakeDecimal(sum(UnscaledValue(cs_ext_ship_cost#4))#5,17,2) AS total shipping cost #6, MakeDecimal(sum(UnscaledValue(cs_net_profit#7))#8,17,2) AS total net profit #9)]
-+- CometColumnarToRow
+CometColumnarToRow
++- CometHashAggregate
+- CometColumnarExchange
+- HashAggregate
+- HashAggregate [COMET: Unsupported aggregation mode PartialMerge]
@@ -40,4 +40,4 @@
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.call_center
-Comet accelerated 36 out of 39 eligible operators (92%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 37 out of 39 eligible operators (94%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/simplified.txt
index c703a75e7..8427aa49a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16.native_iceberg_compat/simplified.txt
@@ -1,7 +1,7 @@
WholeStageCodegen (2)
- HashAggregate [sum,sum,count] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),order count ,total shipping cost ,total net profit ,sum,sum,count]
- CometColumnarToRow
- InputAdapter
+ CometColumnarToRow
+ InputAdapter
+ CometHashAggregate [sum,sum,count] [order count ,total shipping cost ,total net profit ,count(cs_order_number),sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit))]
CometColumnarExchange #1
WholeStageCodegen (1)
HashAggregate [cs_order_number] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),sum,sum,count,sum,sum,count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/explain.txt
index 678ae3c7c..7a4c0fa40 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/explain.txt
@@ -1,6 +1,6 @@
== Physical Plan ==
-* HashAggregate (41)
-+- * CometColumnarToRow (40)
+* CometColumnarToRow (41)
++- CometHashAggregate (40)
+- CometColumnarExchange (39)
+- * HashAggregate (38)
+- * HashAggregate (37)
@@ -225,13 +225,11 @@ Results [3]: [sum#20, sum#21, count#25]
Input [3]: [sum#20, sum#21, count#25]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-(40) CometColumnarToRow [codegen id : 2]
-Input [3]: [sum#20, sum#21, count#25]
-
-(41) HashAggregate [codegen id : 2]
+(40) CometHashAggregate
Input [3]: [sum#20, sum#21, count#25]
Keys: []
Functions [3]: [sum(UnscaledValue(cs_ext_ship_cost#6)), sum(UnscaledValue(cs_net_profit#7)), count(distinct cs_order_number#5)]
-Aggregate Attributes [3]: [sum(UnscaledValue(cs_ext_ship_cost#6))#22, sum(UnscaledValue(cs_net_profit#7))#23, count(cs_order_number#5)#24]
-Results [3]: [count(cs_order_number#5)#24 AS order count #26, MakeDecimal(sum(UnscaledValue(cs_ext_ship_cost#6))#22,17,2) AS total shipping cost #27, MakeDecimal(sum(UnscaledValue(cs_net_profit#7))#23,17,2) AS total net profit #28]
+
+(41) CometColumnarToRow [codegen id : 2]
+Input [3]: [order count #26, total shipping cost #27, total net profit #28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/extended.txt
index c6125cb26..2ad029e44 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/extended.txt
@@ -1,5 +1,5 @@
- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(count(cs_order_number#1)#2 AS order count #3, MakeDecimal(sum(UnscaledValue(cs_ext_ship_cost#4))#5,17,2) AS total shipping cost #6, MakeDecimal(sum(UnscaledValue(cs_net_profit#7))#8,17,2) AS total net profit #9)]
-+- CometColumnarToRow
+CometColumnarToRow
++- CometHashAggregate
+- CometColumnarExchange
+- HashAggregate
+- HashAggregate [COMET: Unsupported aggregation mode PartialMerge]
@@ -40,4 +40,4 @@
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.call_center
-Comet accelerated 36 out of 39 eligible operators (92%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 37 out of 39 eligible operators (94%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/simplified.txt
index c703a75e7..8427aa49a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q16/simplified.txt
@@ -1,7 +1,7 @@
WholeStageCodegen (2)
- HashAggregate [sum,sum,count] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),order count ,total shipping cost ,total net profit ,sum,sum,count]
- CometColumnarToRow
- InputAdapter
+ CometColumnarToRow
+ InputAdapter
+ CometHashAggregate [sum,sum,count] [order count ,total shipping cost ,total net profit ,count(cs_order_number),sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit))]
CometColumnarExchange #1
WholeStageCodegen (1)
HashAggregate [cs_order_number] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),sum,sum,count,sum,sum,count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/explain.txt
index bab37e51d..056ed8dd1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (36)
-+- * HashAggregate (35)
- +- * CometColumnarToRow (34)
+* CometColumnarToRow (36)
++- CometTakeOrderedAndProject (35)
+ +- CometHashAggregate (34)
+- CometExchange (33)
+- CometHashAggregate (32)
+- CometProject (31)
@@ -194,17 +194,15 @@ Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#7))]
Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
Arguments: hashpartitioning(i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(34) CometColumnarToRow [codegen id : 1]
-Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
-
-(35) HashAggregate [codegen id : 1]
+(34) CometHashAggregate
Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
Keys [4]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#7))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#7))#26]
-Results [5]: [i_brand_id#10 AS brand_id#27, i_brand#15 AS brand#28, i_manufact_id#12, i_manufact#16, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#7))#26,17,2) AS ext_price#29]
-(36) TakeOrderedAndProject
-Input [5]: [brand_id#27, brand#28, i_manufact_id#12, i_manufact#16, ext_price#29]
-Arguments: 100, [ext_price#29 DESC NULLS LAST, brand#28 ASC NULLS FIRST, brand_id#27 ASC NULLS FIRST, i_manufact_id#12 ASC NULLS FIRST, i_manufact#16 ASC NULLS FIRST], [brand_id#27, brand#28, i_manufact_id#12, i_manufact#16, ext_price#29]
+(35) CometTakeOrderedAndProject
+Input [5]: [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ext_price#28 DESC NULLS LAST,brand#27 ASC NULLS FIRST,brand_id#26 ASC NULLS FIRST,i_manufact_id#12 ASC NULLS FIRST,i_manufact#16 ASC NULLS FIRST], output=[brand_id#26,brand#27,i_manufact_id#12,i_manufact#16,ext_price#28]), [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28], 100, 0, [ext_price#28 DESC NULLS LAST, brand#27 ASC NULLS FIRST, brand_id#26 ASC NULLS FIRST, i_manufact_id#12 ASC NULLS FIRST, i_manufact#16 AS [...]
+
+(36) CometColumnarToRow [codegen id : 1]
+Input [5]: [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/extended.txt
index 68fdff283..a8cfde2d3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_brand_id#1 AS brand_id#2, i_brand#3 AS brand#4, i_manufact_id#5, i_manufact#6, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#7))#8,17,2) AS ext_price#9)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -35,4 +35,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.store
-Comet accelerated 33 out of 35 eligible operators (94%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 35 out of 35 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/simplified.txt
index 9b585fcb7..93ab89c14 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19.native_iceberg_compat/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [ext_price,brand,brand_id,i_manufact_id,i_manufact]
- WholeStageCodegen (1)
- HashAggregate [i_brand,i_brand_id,i_manufact_id,i_manufact,sum] [sum(UnscaledValue(ss_ext_sales_price)),brand_id,brand,ext_price,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [brand_id,brand,i_manufact_id,i_manufact,ext_price]
+ CometHashAggregate [sum] [brand_id,brand,i_manufact_id,i_manufact,ext_price,i_brand_id,i_brand,sum(UnscaledValue(ss_ext_sales_price))]
CometExchange [i_brand,i_brand_id,i_manufact_id,i_manufact] #1
CometHashAggregate [ss_ext_sales_price] [i_brand,i_brand_id,i_manufact_id,i_manufact,sum]
CometProject [ss_ext_sales_price,i_brand_id,i_brand,i_manufact_id,i_manufact]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/explain.txt
index bab37e51d..056ed8dd1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (36)
-+- * HashAggregate (35)
- +- * CometColumnarToRow (34)
+* CometColumnarToRow (36)
++- CometTakeOrderedAndProject (35)
+ +- CometHashAggregate (34)
+- CometExchange (33)
+- CometHashAggregate (32)
+- CometProject (31)
@@ -194,17 +194,15 @@ Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#7))]
Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
Arguments: hashpartitioning(i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(34) CometColumnarToRow [codegen id : 1]
-Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
-
-(35) HashAggregate [codegen id : 1]
+(34) CometHashAggregate
Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
Keys [4]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#7))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#7))#26]
-Results [5]: [i_brand_id#10 AS brand_id#27, i_brand#15 AS brand#28, i_manufact_id#12, i_manufact#16, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#7))#26,17,2) AS ext_price#29]
-(36) TakeOrderedAndProject
-Input [5]: [brand_id#27, brand#28, i_manufact_id#12, i_manufact#16, ext_price#29]
-Arguments: 100, [ext_price#29 DESC NULLS LAST, brand#28 ASC NULLS FIRST, brand_id#27 ASC NULLS FIRST, i_manufact_id#12 ASC NULLS FIRST, i_manufact#16 ASC NULLS FIRST], [brand_id#27, brand#28, i_manufact_id#12, i_manufact#16, ext_price#29]
+(35) CometTakeOrderedAndProject
+Input [5]: [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ext_price#28 DESC NULLS LAST,brand#27 ASC NULLS FIRST,brand_id#26 ASC NULLS FIRST,i_manufact_id#12 ASC NULLS FIRST,i_manufact#16 ASC NULLS FIRST], output=[brand_id#26,brand#27,i_manufact_id#12,i_manufact#16,ext_price#28]), [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28], 100, 0, [ext_price#28 DESC NULLS LAST, brand#27 ASC NULLS FIRST, brand_id#26 ASC NULLS FIRST, i_manufact_id#12 ASC NULLS FIRST, i_manufact#16 AS [...]
+
+(36) CometColumnarToRow [codegen id : 1]
+Input [5]: [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/extended.txt
index 68fdff283..a8cfde2d3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_brand_id#1 AS brand_id#2, i_brand#3 AS brand#4, i_manufact_id#5, i_manufact#6, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#7))#8,17,2) AS ext_price#9)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -35,4 +35,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.store
-Comet accelerated 33 out of 35 eligible operators (94%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 35 out of 35 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/simplified.txt
index 9b585fcb7..93ab89c14 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q19/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [ext_price,brand,brand_id,i_manufact_id,i_manufact]
- WholeStageCodegen (1)
- HashAggregate [i_brand,i_brand_id,i_manufact_id,i_manufact,sum] [sum(UnscaledValue(ss_ext_sales_price)),brand_id,brand,ext_price,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [brand_id,brand,i_manufact_id,i_manufact,ext_price]
+ CometHashAggregate [sum] [brand_id,brand,i_manufact_id,i_manufact,ext_price,i_brand_id,i_brand,sum(UnscaledValue(ss_ext_sales_price))]
CometExchange [i_brand,i_brand_id,i_manufact_id,i_manufact] #1
CometHashAggregate [ss_ext_sales_price] [i_brand,i_brand_id,i_manufact_id,i_manufact,sum]
CometProject [ss_ext_sales_price,i_brand_id,i_brand,i_manufact_id,i_manufact]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/explain.txt
index f6648cdf5..a7a7ef7b7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/explain.txt
@@ -1,42 +1,38 @@
== Physical Plan ==
-* CometColumnarToRow (38)
-+- CometSort (37)
- +- CometColumnarExchange (36)
- +- * Project (35)
- +- * BroadcastHashJoin Inner BuildRight (34)
- :- * Project (22)
- : +- * BroadcastHashJoin Inner BuildRight (21)
- : :- * HashAggregate (15)
- : : +- * CometColumnarToRow (14)
- : : +- CometExchange (13)
- : : +- CometHashAggregate (12)
- : : +- CometProject (11)
- : : +- CometBroadcastHashJoin (10)
- : : :- CometUnion (5)
- : : : :- CometProject (2)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
- : : : +- CometProject (4)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (3)
- : : +- CometBroadcastExchange (9)
- : : +- CometProject (8)
- : : +- CometFilter (7)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (6)
- : +- BroadcastExchange (20)
- : +- * CometColumnarToRow (19)
- : +- CometProject (18)
- : +- CometFilter (17)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (16)
- +- BroadcastExchange (33)
- +- * Project (32)
- +- * BroadcastHashJoin Inner BuildRight (31)
- :- * HashAggregate (25)
- : +- * CometColumnarToRow (24)
- : +- ReusedExchange (23)
- +- BroadcastExchange (30)
- +- * CometColumnarToRow (29)
- +- CometProject (28)
- +- CometFilter (27)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (26)
+* CometColumnarToRow (34)
++- CometSort (33)
+ +- CometExchange (32)
+ +- CometProject (31)
+ +- CometBroadcastHashJoin (30)
+ :- CometProject (20)
+ : +- CometBroadcastHashJoin (19)
+ : :- CometHashAggregate (14)
+ : : +- CometExchange (13)
+ : : +- CometHashAggregate (12)
+ : : +- CometProject (11)
+ : : +- CometBroadcastHashJoin (10)
+ : : :- CometUnion (5)
+ : : : :- CometProject (2)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
+ : : : +- CometProject (4)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (3)
+ : : +- CometBroadcastExchange (9)
+ : : +- CometProject (8)
+ : : +- CometFilter (7)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (6)
+ : +- CometBroadcastExchange (18)
+ : +- CometProject (17)
+ : +- CometFilter (16)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (15)
+ +- CometBroadcastExchange (29)
+ +- CometProject (28)
+ +- CometBroadcastHashJoin (27)
+ :- CometHashAggregate (22)
+ : +- ReusedExchange (21)
+ +- CometBroadcastExchange (26)
+ +- CometProject (25)
+ +- CometFilter (24)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (23)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
@@ -102,115 +98,96 @@ Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday )
Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
Arguments: hashpartitioning(d_week_seq#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(14) CometColumnarToRow [codegen id : 4]
-Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
-
-(15) HashAggregate [codegen id : 4]
+(14) CometHashAggregate
Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
Keys [1]: [d_week_seq#10]
Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Friday ) THEN sales [...]
-Aggregate Attributes [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END))#20, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END))#21, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END))#22, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END))#23, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END))#24, sum(UnscaledValue(CASE WHEN (d_day_name# [...]
-Results [8]: [d_week_seq#10, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END))#20,17,2) AS sun_sales#27, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END))#21,17,2) AS mon_sales#28, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END))#22,17,2) AS tue_sales#29, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END))#23,17,2) AS w [...]
-(16) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_week_seq#34, d_year#35]
+(15) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_week_seq#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_week_seq)]
ReadSchema: struct<d_week_seq:int,d_year:int>
-(17) CometFilter
-Input [2]: [d_week_seq#34, d_year#35]
-Condition : ((isnotnull(d_year#35) AND (d_year#35 = 2001)) AND isnotnull(d_week_seq#34))
-
-(18) CometProject
-Input [2]: [d_week_seq#34, d_year#35]
-Arguments: [d_week_seq#34], [d_week_seq#34]
-
-(19) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_week_seq#34]
+(16) CometFilter
+Input [2]: [d_week_seq#20, d_year#21]
+Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_week_seq#20))
-(20) BroadcastExchange
-Input [1]: [d_week_seq#34]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
+(17) CometProject
+Input [2]: [d_week_seq#20, d_year#21]
+Arguments: [d_week_seq#20], [d_week_seq#20]
-(21) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [d_week_seq#10]
-Right keys [1]: [d_week_seq#34]
-Join type: Inner
-Join condition: None
+(18) CometBroadcastExchange
+Input [1]: [d_week_seq#20]
+Arguments: [d_week_seq#20]
-(22) Project [codegen id : 4]
-Output [8]: [d_week_seq#10 AS d_week_seq1#36, sun_sales#27 AS sun_sales1#37, mon_sales#28 AS mon_sales1#38, tue_sales#29 AS tue_sales1#39, wed_sales#30 AS wed_sales1#40, thu_sales#31 AS thu_sales1#41, fri_sales#32 AS fri_sales1#42, sat_sales#33 AS sat_sales1#43]
-Input [9]: [d_week_seq#10, sun_sales#27, mon_sales#28, tue_sales#29, wed_sales#30, thu_sales#31, fri_sales#32, sat_sales#33, d_week_seq#34]
+(19) CometBroadcastHashJoin
+Left output [8]: [d_week_seq#10, sun_sales#22, mon_sales#23, tue_sales#24, wed_sales#25, thu_sales#26, fri_sales#27, sat_sales#28]
+Right output [1]: [d_week_seq#20]
+Arguments: [d_week_seq#10], [d_week_seq#20], Inner, BuildRight
-(23) ReusedExchange [Reuses operator id: 13]
-Output [8]: [d_week_seq#44, sum#45, sum#46, sum#47, sum#48, sum#49, sum#50, sum#51]
+(20) CometProject
+Input [9]: [d_week_seq#10, sun_sales#22, mon_sales#23, tue_sales#24, wed_sales#25, thu_sales#26, fri_sales#27, sat_sales#28, d_week_seq#20]
+Arguments: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36], [d_week_seq#10 AS d_week_seq1#29, sun_sales#22 AS sun_sales1#30, mon_sales#23 AS mon_sales1#31, tue_sales#24 AS tue_sales1#32, wed_sales#25 AS wed_sales1#33, thu_sales#26 AS thu_sales1#34, fri_sales#27 AS fri_sales1#35, sat_sales#28 AS sat_sales1#36]
-(24) CometColumnarToRow [codegen id : 3]
-Input [8]: [d_week_seq#44, sum#45, sum#46, sum#47, sum#48, sum#49, sum#50, sum#51]
+(21) ReusedExchange [Reuses operator id: 13]
+Output [8]: [d_week_seq#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43, sum#44]
-(25) HashAggregate [codegen id : 3]
-Input [8]: [d_week_seq#44, sum#45, sum#46, sum#47, sum#48, sum#49, sum#50, sum#51]
-Keys [1]: [d_week_seq#44]
-Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#52 = Sunday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Monday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Tuesday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Wednesday) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Thursday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Friday ) THEN [...]
-Aggregate Attributes [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#52 = Sunday ) THEN sales_price#53 END))#20, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Monday ) THEN sales_price#53 END))#21, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Tuesday ) THEN sales_price#53 END))#22, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Wednesday) THEN sales_price#53 END))#23, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Thursday ) THEN sales_price#53 END))#24, sum(UnscaledValue(CASE WHEN (d_day_ [...]
-Results [8]: [d_week_seq#44, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Sunday ) THEN sales_price#53 END))#20,17,2) AS sun_sales#54, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Monday ) THEN sales_price#53 END))#21,17,2) AS mon_sales#55, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Tuesday ) THEN sales_price#53 END))#22,17,2) AS tue_sales#56, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Wednesday) THEN sales_price#53 END))#23,17,2) [...]
+(22) CometHashAggregate
+Input [8]: [d_week_seq#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43, sum#44]
+Keys [1]: [d_week_seq#37]
+Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#45 = Sunday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Monday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Tuesday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Wednesday) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Thursday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Friday ) THEN [...]
-(26) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_week_seq#61, d_year#62]
+(23) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_week_seq#47, d_year#48]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_week_seq)]
ReadSchema: struct<d_week_seq:int,d_year:int>
-(27) CometFilter
-Input [2]: [d_week_seq#61, d_year#62]
-Condition : ((isnotnull(d_year#62) AND (d_year#62 = 2002)) AND isnotnull(d_week_seq#61))
+(24) CometFilter
+Input [2]: [d_week_seq#47, d_year#48]
+Condition : ((isnotnull(d_year#48) AND (d_year#48 = 2002)) AND isnotnull(d_week_seq#47))
+
+(25) CometProject
+Input [2]: [d_week_seq#47, d_year#48]
+Arguments: [d_week_seq#47], [d_week_seq#47]
+
+(26) CometBroadcastExchange
+Input [1]: [d_week_seq#47]
+Arguments: [d_week_seq#47]
+
+(27) CometBroadcastHashJoin
+Left output [8]: [d_week_seq#37, sun_sales#49, mon_sales#50, tue_sales#51, wed_sales#52, thu_sales#53, fri_sales#54, sat_sales#55]
+Right output [1]: [d_week_seq#47]
+Arguments: [d_week_seq#37], [d_week_seq#47], Inner, BuildRight
(28) CometProject
-Input [2]: [d_week_seq#61, d_year#62]
-Arguments: [d_week_seq#61], [d_week_seq#61]
-
-(29) CometColumnarToRow [codegen id : 2]
-Input [1]: [d_week_seq#61]
-
-(30) BroadcastExchange
-Input [1]: [d_week_seq#61]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
-
-(31) BroadcastHashJoin [codegen id : 3]
-Left keys [1]: [d_week_seq#44]
-Right keys [1]: [d_week_seq#61]
-Join type: Inner
-Join condition: None
-
-(32) Project [codegen id : 3]
-Output [8]: [d_week_seq#44 AS d_week_seq2#63, sun_sales#54 AS sun_sales2#64, mon_sales#55 AS mon_sales2#65, tue_sales#56 AS tue_sales2#66, wed_sales#57 AS wed_sales2#67, thu_sales#58 AS thu_sales2#68, fri_sales#59 AS fri_sales2#69, sat_sales#60 AS sat_sales2#70]
-Input [9]: [d_week_seq#44, sun_sales#54, mon_sales#55, tue_sales#56, wed_sales#57, thu_sales#58, fri_sales#59, sat_sales#60, d_week_seq#61]
-
-(33) BroadcastExchange
-Input [8]: [d_week_seq2#63, sun_sales2#64, mon_sales2#65, tue_sales2#66, wed_sales2#67, thu_sales2#68, fri_sales2#69, sat_sales2#70]
-Arguments: HashedRelationBroadcastMode(List(cast((input[0, int, true] - 53) as bigint)),false), [plan_id=4]
-
-(34) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [d_week_seq1#36]
-Right keys [1]: [(d_week_seq2#63 - 53)]
-Join type: Inner
-Join condition: None
-
-(35) Project [codegen id : 4]
-Output [8]: [d_week_seq1#36, round((sun_sales1#37 / sun_sales2#64), 2) AS round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1#38 / mon_sales2#65), 2) AS round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1#39 / tue_sales2#66), 2) AS round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1#40 / wed_sales2#67), 2) AS round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1#41 / thu_sales2#68), 2) AS round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1#42 / fri_sales2#69), [...]
-Input [16]: [d_week_seq1#36, sun_sales1#37, mon_sales1#38, tue_sales1#39, wed_sales1#40, thu_sales1#41, fri_sales1#42, sat_sales1#43, d_week_seq2#63, sun_sales2#64, mon_sales2#65, tue_sales2#66, wed_sales2#67, thu_sales2#68, fri_sales2#69, sat_sales2#70]
-
-(36) CometColumnarExchange
-Input [8]: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77]
-Arguments: rangepartitioning(d_week_seq1#36 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(37) CometSort
-Input [8]: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77]
-Arguments: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77], [d_week_seq1#36 ASC NULLS FIRST]
-
-(38) CometColumnarToRow [codegen id : 5]
-Input [8]: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77]
+Input [9]: [d_week_seq#37, sun_sales#49, mon_sales#50, tue_sales#51, wed_sales#52, thu_sales#53, fri_sales#54, sat_sales#55, d_week_seq#47]
+Arguments: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63], [d_week_seq#37 AS d_week_seq2#56, sun_sales#49 AS sun_sales2#57, mon_sales#50 AS mon_sales2#58, tue_sales#51 AS tue_sales2#59, wed_sales#52 AS wed_sales2#60, thu_sales#53 AS thu_sales2#61, fri_sales#54 AS fri_sales2#62, sat_sales#55 AS sat_sales2#63]
+
+(29) CometBroadcastExchange
+Input [8]: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+Arguments: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+
+(30) CometBroadcastHashJoin
+Left output [8]: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36]
+Right output [8]: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+Arguments: [d_week_seq1#29], [(d_week_seq2#56 - 53)], Inner, BuildRight
+
+(31) CometProject
+Input [16]: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36, d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+Arguments: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70], [d_week_seq1#29, round((sun_sales1#30 / sun_sales2#57), 2) AS round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1#31 / mon_sales2#58), 2) AS round((mon_sales1 / mon_sales2), 2)# [...]
+
+(32) CometExchange
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70]
+Arguments: rangepartitioning(d_week_seq1#29 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(33) CometSort
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70]
+Arguments: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70], [d_week_seq1#29 ASC NULLS FIRST]
+
+(34) CometColumnarToRow [codegen id : 1]
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/extended.txt
index 81b96b418..15f5db847 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/extended.txt
@@ -1,52 +1,48 @@
CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(d_week_seq#1, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#2 = Sunday ) THEN sales_price#3 END))#4,17,2) AS sun_sales#5, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#2 = Monday ) THEN sales_price#3 END))#6,17,2) AS mon_sales#7, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#2 = Tuesday ) THEN sales_price#3 END))#8,17,2) AS tue_sales#9, MakeDecima [...]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometUnion
- : : : :- CometProject
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : : +- CometProject
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
- : : +- CometBroadcastExchange
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- BroadcastExchange
- : +- CometColumnarToRow
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- BroadcastExchange
- +- Project
- +- BroadcastHashJoin
- :- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(d_week_seq#18, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#19 = Sunday ) THEN sales_price#20 END))#4,17,2) AS sun_sales#21, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#19 = Monday ) THEN sales_price#20 END))#6,17,2) AS mon_sales#22, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#19 = Tuesday ) THEN sales_price#20 END))#8,17,2) AS tue_sales#2 [...]
- : +- CometColumnarToRow
- : +- CometExchange
- : +- CometHashAggregate
- : +- CometProject
- : +- CometBroadcastHashJoin
- : :- CometUnion
- : : :- CometProject
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- CometProject
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
- : +- CometBroadcastExchange
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- BroadcastExchange
- +- CometColumnarToRow
- +- CometProject
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometExchange
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometUnion
+ : : : :- CometProject
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : : +- CometProject
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
+ : : +- CometBroadcastExchange
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometHashAggregate
+ : +- CometExchange
+ : +- CometHashAggregate
+ : +- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometUnion
+ : : :- CometProject
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- CometProject
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 34 out of 45 eligible operators (75%). Final plan contains 5 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 45 out of 45 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/simplified.txt
index 150fb0164..852c5fca0 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2.native_iceberg_compat/simplified.txt
@@ -1,51 +1,36 @@
-WholeStageCodegen (5)
+WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [d_week_seq1,round((sun_sales1 / sun_sales2), 2),round((mon_sales1 / mon_sales2), 2),round((tue_sales1 / tue_sales2), 2),round((wed_sales1 / wed_sales2), 2),round((thu_sales1 / thu_sales2), 2),round((fri_sales1 / fri_sales2), 2),round((sat_sales1 / sat_sales2), 2)]
- CometColumnarExchange [d_week_seq1] #1
- WholeStageCodegen (4)
- Project [d_week_seq1,sun_sales1,sun_sales2,mon_sales1,mon_sales2,tue_sales1,tue_sales2,wed_sales1,wed_sales2,thu_sales1,thu_sales2,fri_sales1,fri_sales2,sat_sales1,sat_sales2]
- BroadcastHashJoin [d_week_seq1,d_week_seq2]
- Project [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales]
- BroadcastHashJoin [d_week_seq,d_week_seq]
- HashAggregate [d_week_seq,sum,sum,sum,sum,sum,sum,sum] [sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Thursday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_n [...]
- CometColumnarToRow
- InputAdapter
- CometExchange [d_week_seq] #2
- CometHashAggregate [d_day_name,sales_price] [d_week_seq,sum,sum,sum,sum,sum,sum,sum]
- CometProject [sales_price,d_week_seq,d_day_name]
- CometBroadcastHashJoin [sold_date_sk,sales_price,d_date_sk,d_week_seq,d_day_name]
- CometUnion [sold_date_sk,sales_price]
- CometProject [ws_sold_date_sk,ws_ext_sales_price] [sold_date_sk,sales_price]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_ext_sales_price,ws_sold_date_sk]
- CometProject [cs_sold_date_sk,cs_ext_sales_price] [sold_date_sk,sales_price]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_ext_sales_price,cs_sold_date_sk]
- CometBroadcastExchange [d_date_sk,d_week_seq,d_day_name] #3
- CometProject [d_day_name] [d_date_sk,d_week_seq,d_day_name]
- CometFilter [d_date_sk,d_week_seq,d_day_name]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq,d_day_name]
- InputAdapter
- BroadcastExchange #4
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometProject [d_week_seq]
- CometFilter [d_week_seq,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (3)
- Project [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales]
- BroadcastHashJoin [d_week_seq,d_week_seq]
- HashAggregate [d_week_seq,sum,sum,sum,sum,sum,sum,sum] [sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Thursday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d [...]
- CometColumnarToRow
- InputAdapter
- ReusedExchange [d_week_seq,sum,sum,sum,sum,sum,sum,sum] #2
- InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (2)
- CometColumnarToRow
- InputAdapter
- CometProject [d_week_seq]
- CometFilter [d_week_seq,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
+ CometExchange [d_week_seq1] #1
+ CometProject [sun_sales1,sun_sales2,mon_sales1,mon_sales2,tue_sales1,tue_sales2,wed_sales1,wed_sales2,thu_sales1,thu_sales2,fri_sales1,fri_sales2,sat_sales1,sat_sales2] [d_week_seq1,round((sun_sales1 / sun_sales2), 2),round((mon_sales1 / mon_sales2), 2),round((tue_sales1 / tue_sales2), 2),round((wed_sales1 / wed_sales2), 2),round((thu_sales1 / thu_sales2), 2),round((fri_sales1 / fri_sales2), 2),round((sat_sales1 / sat_sales2), 2)]
+ CometBroadcastHashJoin [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1,d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2]
+ CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1]
+ CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
+ CometHashAggregate [sum,sum,sum,sum,sum,sum,sum] [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name [...]
+ CometExchange [d_week_seq] #2
+ CometHashAggregate [d_day_name,sales_price] [d_week_seq,sum,sum,sum,sum,sum,sum,sum]
+ CometProject [sales_price,d_week_seq,d_day_name]
+ CometBroadcastHashJoin [sold_date_sk,sales_price,d_date_sk,d_week_seq,d_day_name]
+ CometUnion [sold_date_sk,sales_price]
+ CometProject [ws_sold_date_sk,ws_ext_sales_price] [sold_date_sk,sales_price]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_ext_sales_price,ws_sold_date_sk]
+ CometProject [cs_sold_date_sk,cs_ext_sales_price] [sold_date_sk,sales_price]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_ext_sales_price,cs_sold_date_sk]
+ CometBroadcastExchange [d_date_sk,d_week_seq,d_day_name] #3
+ CometProject [d_day_name] [d_date_sk,d_week_seq,d_day_name]
+ CometFilter [d_date_sk,d_week_seq,d_day_name]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq,d_day_name]
+ CometBroadcastExchange [d_week_seq] #4
+ CometProject [d_week_seq]
+ CometFilter [d_week_seq,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
+ CometBroadcastExchange [d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2] #5
+ CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2]
+ CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
+ CometHashAggregate [sum,sum,sum,sum,sum,sum,sum] [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_na [...]
+ ReusedExchange [d_week_seq,sum,sum,sum,sum,sum,sum,sum] #2
+ CometBroadcastExchange [d_week_seq] #6
+ CometProject [d_week_seq]
+ CometFilter [d_week_seq,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/explain.txt
index f6648cdf5..a7a7ef7b7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/explain.txt
@@ -1,42 +1,38 @@
== Physical Plan ==
-* CometColumnarToRow (38)
-+- CometSort (37)
- +- CometColumnarExchange (36)
- +- * Project (35)
- +- * BroadcastHashJoin Inner BuildRight (34)
- :- * Project (22)
- : +- * BroadcastHashJoin Inner BuildRight (21)
- : :- * HashAggregate (15)
- : : +- * CometColumnarToRow (14)
- : : +- CometExchange (13)
- : : +- CometHashAggregate (12)
- : : +- CometProject (11)
- : : +- CometBroadcastHashJoin (10)
- : : :- CometUnion (5)
- : : : :- CometProject (2)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
- : : : +- CometProject (4)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (3)
- : : +- CometBroadcastExchange (9)
- : : +- CometProject (8)
- : : +- CometFilter (7)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (6)
- : +- BroadcastExchange (20)
- : +- * CometColumnarToRow (19)
- : +- CometProject (18)
- : +- CometFilter (17)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (16)
- +- BroadcastExchange (33)
- +- * Project (32)
- +- * BroadcastHashJoin Inner BuildRight (31)
- :- * HashAggregate (25)
- : +- * CometColumnarToRow (24)
- : +- ReusedExchange (23)
- +- BroadcastExchange (30)
- +- * CometColumnarToRow (29)
- +- CometProject (28)
- +- CometFilter (27)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (26)
+* CometColumnarToRow (34)
++- CometSort (33)
+ +- CometExchange (32)
+ +- CometProject (31)
+ +- CometBroadcastHashJoin (30)
+ :- CometProject (20)
+ : +- CometBroadcastHashJoin (19)
+ : :- CometHashAggregate (14)
+ : : +- CometExchange (13)
+ : : +- CometHashAggregate (12)
+ : : +- CometProject (11)
+ : : +- CometBroadcastHashJoin (10)
+ : : :- CometUnion (5)
+ : : : :- CometProject (2)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (1)
+ : : : +- CometProject (4)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (3)
+ : : +- CometBroadcastExchange (9)
+ : : +- CometProject (8)
+ : : +- CometFilter (7)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (6)
+ : +- CometBroadcastExchange (18)
+ : +- CometProject (17)
+ : +- CometFilter (16)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (15)
+ +- CometBroadcastExchange (29)
+ +- CometProject (28)
+ +- CometBroadcastHashJoin (27)
+ :- CometHashAggregate (22)
+ : +- ReusedExchange (21)
+ +- CometBroadcastExchange (26)
+ +- CometProject (25)
+ +- CometFilter (24)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (23)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
@@ -102,115 +98,96 @@ Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday )
Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
Arguments: hashpartitioning(d_week_seq#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(14) CometColumnarToRow [codegen id : 4]
-Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
-
-(15) HashAggregate [codegen id : 4]
+(14) CometHashAggregate
Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
Keys [1]: [d_week_seq#10]
Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Friday ) THEN sales [...]
-Aggregate Attributes [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END))#20, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END))#21, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END))#22, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END))#23, sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END))#24, sum(UnscaledValue(CASE WHEN (d_day_name# [...]
-Results [8]: [d_week_seq#10, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END))#20,17,2) AS sun_sales#27, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END))#21,17,2) AS mon_sales#28, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END))#22,17,2) AS tue_sales#29, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END))#23,17,2) AS w [...]
-(16) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_week_seq#34, d_year#35]
+(15) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_week_seq#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_week_seq)]
ReadSchema: struct<d_week_seq:int,d_year:int>
-(17) CometFilter
-Input [2]: [d_week_seq#34, d_year#35]
-Condition : ((isnotnull(d_year#35) AND (d_year#35 = 2001)) AND isnotnull(d_week_seq#34))
-
-(18) CometProject
-Input [2]: [d_week_seq#34, d_year#35]
-Arguments: [d_week_seq#34], [d_week_seq#34]
-
-(19) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_week_seq#34]
+(16) CometFilter
+Input [2]: [d_week_seq#20, d_year#21]
+Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_week_seq#20))
-(20) BroadcastExchange
-Input [1]: [d_week_seq#34]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
+(17) CometProject
+Input [2]: [d_week_seq#20, d_year#21]
+Arguments: [d_week_seq#20], [d_week_seq#20]
-(21) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [d_week_seq#10]
-Right keys [1]: [d_week_seq#34]
-Join type: Inner
-Join condition: None
+(18) CometBroadcastExchange
+Input [1]: [d_week_seq#20]
+Arguments: [d_week_seq#20]
-(22) Project [codegen id : 4]
-Output [8]: [d_week_seq#10 AS d_week_seq1#36, sun_sales#27 AS sun_sales1#37, mon_sales#28 AS mon_sales1#38, tue_sales#29 AS tue_sales1#39, wed_sales#30 AS wed_sales1#40, thu_sales#31 AS thu_sales1#41, fri_sales#32 AS fri_sales1#42, sat_sales#33 AS sat_sales1#43]
-Input [9]: [d_week_seq#10, sun_sales#27, mon_sales#28, tue_sales#29, wed_sales#30, thu_sales#31, fri_sales#32, sat_sales#33, d_week_seq#34]
+(19) CometBroadcastHashJoin
+Left output [8]: [d_week_seq#10, sun_sales#22, mon_sales#23, tue_sales#24, wed_sales#25, thu_sales#26, fri_sales#27, sat_sales#28]
+Right output [1]: [d_week_seq#20]
+Arguments: [d_week_seq#10], [d_week_seq#20], Inner, BuildRight
-(23) ReusedExchange [Reuses operator id: 13]
-Output [8]: [d_week_seq#44, sum#45, sum#46, sum#47, sum#48, sum#49, sum#50, sum#51]
+(20) CometProject
+Input [9]: [d_week_seq#10, sun_sales#22, mon_sales#23, tue_sales#24, wed_sales#25, thu_sales#26, fri_sales#27, sat_sales#28, d_week_seq#20]
+Arguments: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36], [d_week_seq#10 AS d_week_seq1#29, sun_sales#22 AS sun_sales1#30, mon_sales#23 AS mon_sales1#31, tue_sales#24 AS tue_sales1#32, wed_sales#25 AS wed_sales1#33, thu_sales#26 AS thu_sales1#34, fri_sales#27 AS fri_sales1#35, sat_sales#28 AS sat_sales1#36]
-(24) CometColumnarToRow [codegen id : 3]
-Input [8]: [d_week_seq#44, sum#45, sum#46, sum#47, sum#48, sum#49, sum#50, sum#51]
+(21) ReusedExchange [Reuses operator id: 13]
+Output [8]: [d_week_seq#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43, sum#44]
-(25) HashAggregate [codegen id : 3]
-Input [8]: [d_week_seq#44, sum#45, sum#46, sum#47, sum#48, sum#49, sum#50, sum#51]
-Keys [1]: [d_week_seq#44]
-Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#52 = Sunday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Monday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Tuesday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Wednesday) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Thursday ) THEN sales_price#53 END)), sum(UnscaledValue(CASE WHEN (d_day_name#52 = Friday ) THEN [...]
-Aggregate Attributes [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#52 = Sunday ) THEN sales_price#53 END))#20, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Monday ) THEN sales_price#53 END))#21, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Tuesday ) THEN sales_price#53 END))#22, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Wednesday) THEN sales_price#53 END))#23, sum(UnscaledValue(CASE WHEN (d_day_name#52 = Thursday ) THEN sales_price#53 END))#24, sum(UnscaledValue(CASE WHEN (d_day_ [...]
-Results [8]: [d_week_seq#44, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Sunday ) THEN sales_price#53 END))#20,17,2) AS sun_sales#54, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Monday ) THEN sales_price#53 END))#21,17,2) AS mon_sales#55, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Tuesday ) THEN sales_price#53 END))#22,17,2) AS tue_sales#56, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#52 = Wednesday) THEN sales_price#53 END))#23,17,2) [...]
+(22) CometHashAggregate
+Input [8]: [d_week_seq#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43, sum#44]
+Keys [1]: [d_week_seq#37]
+Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#45 = Sunday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Monday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Tuesday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Wednesday) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Thursday ) THEN sales_price#46 END)), sum(UnscaledValue(CASE WHEN (d_day_name#45 = Friday ) THEN [...]
-(26) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [2]: [d_week_seq#61, d_year#62]
+(23) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [2]: [d_week_seq#47, d_year#48]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_week_seq)]
ReadSchema: struct<d_week_seq:int,d_year:int>
-(27) CometFilter
-Input [2]: [d_week_seq#61, d_year#62]
-Condition : ((isnotnull(d_year#62) AND (d_year#62 = 2002)) AND isnotnull(d_week_seq#61))
+(24) CometFilter
+Input [2]: [d_week_seq#47, d_year#48]
+Condition : ((isnotnull(d_year#48) AND (d_year#48 = 2002)) AND isnotnull(d_week_seq#47))
+
+(25) CometProject
+Input [2]: [d_week_seq#47, d_year#48]
+Arguments: [d_week_seq#47], [d_week_seq#47]
+
+(26) CometBroadcastExchange
+Input [1]: [d_week_seq#47]
+Arguments: [d_week_seq#47]
+
+(27) CometBroadcastHashJoin
+Left output [8]: [d_week_seq#37, sun_sales#49, mon_sales#50, tue_sales#51, wed_sales#52, thu_sales#53, fri_sales#54, sat_sales#55]
+Right output [1]: [d_week_seq#47]
+Arguments: [d_week_seq#37], [d_week_seq#47], Inner, BuildRight
(28) CometProject
-Input [2]: [d_week_seq#61, d_year#62]
-Arguments: [d_week_seq#61], [d_week_seq#61]
-
-(29) CometColumnarToRow [codegen id : 2]
-Input [1]: [d_week_seq#61]
-
-(30) BroadcastExchange
-Input [1]: [d_week_seq#61]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
-
-(31) BroadcastHashJoin [codegen id : 3]
-Left keys [1]: [d_week_seq#44]
-Right keys [1]: [d_week_seq#61]
-Join type: Inner
-Join condition: None
-
-(32) Project [codegen id : 3]
-Output [8]: [d_week_seq#44 AS d_week_seq2#63, sun_sales#54 AS sun_sales2#64, mon_sales#55 AS mon_sales2#65, tue_sales#56 AS tue_sales2#66, wed_sales#57 AS wed_sales2#67, thu_sales#58 AS thu_sales2#68, fri_sales#59 AS fri_sales2#69, sat_sales#60 AS sat_sales2#70]
-Input [9]: [d_week_seq#44, sun_sales#54, mon_sales#55, tue_sales#56, wed_sales#57, thu_sales#58, fri_sales#59, sat_sales#60, d_week_seq#61]
-
-(33) BroadcastExchange
-Input [8]: [d_week_seq2#63, sun_sales2#64, mon_sales2#65, tue_sales2#66, wed_sales2#67, thu_sales2#68, fri_sales2#69, sat_sales2#70]
-Arguments: HashedRelationBroadcastMode(List(cast((input[0, int, true] - 53) as bigint)),false), [plan_id=4]
-
-(34) BroadcastHashJoin [codegen id : 4]
-Left keys [1]: [d_week_seq1#36]
-Right keys [1]: [(d_week_seq2#63 - 53)]
-Join type: Inner
-Join condition: None
-
-(35) Project [codegen id : 4]
-Output [8]: [d_week_seq1#36, round((sun_sales1#37 / sun_sales2#64), 2) AS round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1#38 / mon_sales2#65), 2) AS round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1#39 / tue_sales2#66), 2) AS round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1#40 / wed_sales2#67), 2) AS round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1#41 / thu_sales2#68), 2) AS round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1#42 / fri_sales2#69), [...]
-Input [16]: [d_week_seq1#36, sun_sales1#37, mon_sales1#38, tue_sales1#39, wed_sales1#40, thu_sales1#41, fri_sales1#42, sat_sales1#43, d_week_seq2#63, sun_sales2#64, mon_sales2#65, tue_sales2#66, wed_sales2#67, thu_sales2#68, fri_sales2#69, sat_sales2#70]
-
-(36) CometColumnarExchange
-Input [8]: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77]
-Arguments: rangepartitioning(d_week_seq1#36 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(37) CometSort
-Input [8]: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77]
-Arguments: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77], [d_week_seq1#36 ASC NULLS FIRST]
-
-(38) CometColumnarToRow [codegen id : 5]
-Input [8]: [d_week_seq1#36, round((sun_sales1 / sun_sales2), 2)#71, round((mon_sales1 / mon_sales2), 2)#72, round((tue_sales1 / tue_sales2), 2)#73, round((wed_sales1 / wed_sales2), 2)#74, round((thu_sales1 / thu_sales2), 2)#75, round((fri_sales1 / fri_sales2), 2)#76, round((sat_sales1 / sat_sales2), 2)#77]
+Input [9]: [d_week_seq#37, sun_sales#49, mon_sales#50, tue_sales#51, wed_sales#52, thu_sales#53, fri_sales#54, sat_sales#55, d_week_seq#47]
+Arguments: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63], [d_week_seq#37 AS d_week_seq2#56, sun_sales#49 AS sun_sales2#57, mon_sales#50 AS mon_sales2#58, tue_sales#51 AS tue_sales2#59, wed_sales#52 AS wed_sales2#60, thu_sales#53 AS thu_sales2#61, fri_sales#54 AS fri_sales2#62, sat_sales#55 AS sat_sales2#63]
+
+(29) CometBroadcastExchange
+Input [8]: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+Arguments: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+
+(30) CometBroadcastHashJoin
+Left output [8]: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36]
+Right output [8]: [d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+Arguments: [d_week_seq1#29], [(d_week_seq2#56 - 53)], Inner, BuildRight
+
+(31) CometProject
+Input [16]: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36, d_week_seq2#56, sun_sales2#57, mon_sales2#58, tue_sales2#59, wed_sales2#60, thu_sales2#61, fri_sales2#62, sat_sales2#63]
+Arguments: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70], [d_week_seq1#29, round((sun_sales1#30 / sun_sales2#57), 2) AS round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1#31 / mon_sales2#58), 2) AS round((mon_sales1 / mon_sales2), 2)# [...]
+
+(32) CometExchange
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70]
+Arguments: rangepartitioning(d_week_seq1#29 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(33) CometSort
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70]
+Arguments: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70], [d_week_seq1#29 ASC NULLS FIRST]
+
+(34) CometColumnarToRow [codegen id : 1]
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#64, round((mon_sales1 / mon_sales2), 2)#65, round((tue_sales1 / tue_sales2), 2)#66, round((wed_sales1 / wed_sales2), 2)#67, round((thu_sales1 / thu_sales2), 2)#68, round((fri_sales1 / fri_sales2), 2)#69, round((sat_sales1 / sat_sales2), 2)#70]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/extended.txt
index 81b96b418..15f5db847 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/extended.txt
@@ -1,52 +1,48 @@
CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(d_week_seq#1, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#2 = Sunday ) THEN sales_price#3 END))#4,17,2) AS sun_sales#5, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#2 = Monday ) THEN sales_price#3 END))#6,17,2) AS mon_sales#7, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#2 = Tuesday ) THEN sales_price#3 END))#8,17,2) AS tue_sales#9, MakeDecima [...]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometUnion
- : : : :- CometProject
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : : +- CometProject
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
- : : +- CometBroadcastExchange
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- BroadcastExchange
- : +- CometColumnarToRow
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- BroadcastExchange
- +- Project
- +- BroadcastHashJoin
- :- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(d_week_seq#18, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#19 = Sunday ) THEN sales_price#20 END))#4,17,2) AS sun_sales#21, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#19 = Monday ) THEN sales_price#20 END))#6,17,2) AS mon_sales#22, MakeDecimal(sum(UnscaledValue(CASE WHEN (d_day_name#19 = Tuesday ) THEN sales_price#20 END))#8,17,2) AS tue_sales#2 [...]
- : +- CometColumnarToRow
- : +- CometExchange
- : +- CometHashAggregate
- : +- CometProject
- : +- CometBroadcastHashJoin
- : :- CometUnion
- : : :- CometProject
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- CometProject
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
- : +- CometBroadcastExchange
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- BroadcastExchange
- +- CometColumnarToRow
- +- CometProject
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometExchange
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometUnion
+ : : : :- CometProject
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : : +- CometProject
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
+ : : +- CometBroadcastExchange
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometHashAggregate
+ : +- CometExchange
+ : +- CometHashAggregate
+ : +- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometUnion
+ : : :- CometProject
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- CometProject
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 34 out of 45 eligible operators (75%). Final plan contains 5 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 45 out of 45 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/simplified.txt
index 150fb0164..852c5fca0 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q2/simplified.txt
@@ -1,51 +1,36 @@
-WholeStageCodegen (5)
+WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [d_week_seq1,round((sun_sales1 / sun_sales2), 2),round((mon_sales1 / mon_sales2), 2),round((tue_sales1 / tue_sales2), 2),round((wed_sales1 / wed_sales2), 2),round((thu_sales1 / thu_sales2), 2),round((fri_sales1 / fri_sales2), 2),round((sat_sales1 / sat_sales2), 2)]
- CometColumnarExchange [d_week_seq1] #1
- WholeStageCodegen (4)
- Project [d_week_seq1,sun_sales1,sun_sales2,mon_sales1,mon_sales2,tue_sales1,tue_sales2,wed_sales1,wed_sales2,thu_sales1,thu_sales2,fri_sales1,fri_sales2,sat_sales1,sat_sales2]
- BroadcastHashJoin [d_week_seq1,d_week_seq2]
- Project [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales]
- BroadcastHashJoin [d_week_seq,d_week_seq]
- HashAggregate [d_week_seq,sum,sum,sum,sum,sum,sum,sum] [sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Thursday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_n [...]
- CometColumnarToRow
- InputAdapter
- CometExchange [d_week_seq] #2
- CometHashAggregate [d_day_name,sales_price] [d_week_seq,sum,sum,sum,sum,sum,sum,sum]
- CometProject [sales_price,d_week_seq,d_day_name]
- CometBroadcastHashJoin [sold_date_sk,sales_price,d_date_sk,d_week_seq,d_day_name]
- CometUnion [sold_date_sk,sales_price]
- CometProject [ws_sold_date_sk,ws_ext_sales_price] [sold_date_sk,sales_price]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_ext_sales_price,ws_sold_date_sk]
- CometProject [cs_sold_date_sk,cs_ext_sales_price] [sold_date_sk,sales_price]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_ext_sales_price,cs_sold_date_sk]
- CometBroadcastExchange [d_date_sk,d_week_seq,d_day_name] #3
- CometProject [d_day_name] [d_date_sk,d_week_seq,d_day_name]
- CometFilter [d_date_sk,d_week_seq,d_day_name]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq,d_day_name]
- InputAdapter
- BroadcastExchange #4
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometProject [d_week_seq]
- CometFilter [d_week_seq,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (3)
- Project [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales]
- BroadcastHashJoin [d_week_seq,d_week_seq]
- HashAggregate [d_week_seq,sum,sum,sum,sum,sum,sum,sum] [sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Thursday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d [...]
- CometColumnarToRow
- InputAdapter
- ReusedExchange [d_week_seq,sum,sum,sum,sum,sum,sum,sum] #2
- InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (2)
- CometColumnarToRow
- InputAdapter
- CometProject [d_week_seq]
- CometFilter [d_week_seq,d_year]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
+ CometExchange [d_week_seq1] #1
+ CometProject [sun_sales1,sun_sales2,mon_sales1,mon_sales2,tue_sales1,tue_sales2,wed_sales1,wed_sales2,thu_sales1,thu_sales2,fri_sales1,fri_sales2,sat_sales1,sat_sales2] [d_week_seq1,round((sun_sales1 / sun_sales2), 2),round((mon_sales1 / mon_sales2), 2),round((tue_sales1 / tue_sales2), 2),round((wed_sales1 / wed_sales2), 2),round((thu_sales1 / thu_sales2), 2),round((fri_sales1 / fri_sales2), 2),round((sat_sales1 / sat_sales2), 2)]
+ CometBroadcastHashJoin [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1,d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2]
+ CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1]
+ CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
+ CometHashAggregate [sum,sum,sum,sum,sum,sum,sum] [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name [...]
+ CometExchange [d_week_seq] #2
+ CometHashAggregate [d_day_name,sales_price] [d_week_seq,sum,sum,sum,sum,sum,sum,sum]
+ CometProject [sales_price,d_week_seq,d_day_name]
+ CometBroadcastHashJoin [sold_date_sk,sales_price,d_date_sk,d_week_seq,d_day_name]
+ CometUnion [sold_date_sk,sales_price]
+ CometProject [ws_sold_date_sk,ws_ext_sales_price] [sold_date_sk,sales_price]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales [ws_ext_sales_price,ws_sold_date_sk]
+ CometProject [cs_sold_date_sk,cs_ext_sales_price] [sold_date_sk,sales_price]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_ext_sales_price,cs_sold_date_sk]
+ CometBroadcastExchange [d_date_sk,d_week_seq,d_day_name] #3
+ CometProject [d_day_name] [d_date_sk,d_week_seq,d_day_name]
+ CometFilter [d_date_sk,d_week_seq,d_day_name]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq,d_day_name]
+ CometBroadcastExchange [d_week_seq] #4
+ CometProject [d_week_seq]
+ CometFilter [d_week_seq,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
+ CometBroadcastExchange [d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2] #5
+ CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2]
+ CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
+ CometHashAggregate [sum,sum,sum,sum,sum,sum,sum] [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_na [...]
+ ReusedExchange [d_week_seq,sum,sum,sum,sum,sum,sum,sum] #2
+ CometBroadcastExchange [d_week_seq] #6
+ CometProject [d_week_seq]
+ CometFilter [d_week_seq,d_year]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_week_seq,d_year]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/explain.txt
index 88752d6e1..e6612e757 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/explain.txt
@@ -1,28 +1,27 @@
== Physical Plan ==
-TakeOrderedAndProject (24)
-+- * Project (23)
- +- Window (22)
- +- * CometColumnarToRow (21)
- +- CometSort (20)
- +- CometColumnarExchange (19)
- +- * HashAggregate (18)
- +- * CometColumnarToRow (17)
- +- CometExchange (16)
- +- CometHashAggregate (15)
- +- CometProject (14)
- +- CometBroadcastHashJoin (13)
- :- CometProject (8)
- : +- CometBroadcastHashJoin (7)
- : :- CometFilter (2)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (1)
- : +- CometBroadcastExchange (6)
- : +- CometProject (5)
- : +- CometFilter (4)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
- +- CometBroadcastExchange (12)
- +- CometProject (11)
- +- CometFilter (10)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
+TakeOrderedAndProject (23)
++- * Project (22)
+ +- Window (21)
+ +- * CometColumnarToRow (20)
+ +- CometSort (19)
+ +- CometExchange (18)
+ +- CometHashAggregate (17)
+ +- CometExchange (16)
+ +- CometHashAggregate (15)
+ +- CometProject (14)
+ +- CometBroadcastHashJoin (13)
+ :- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
+ : :- CometFilter (2)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (1)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
+ +- CometBroadcastExchange (12)
+ +- CometProject (11)
+ +- CometFilter (10)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
@@ -102,68 +101,63 @@ Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#2))]
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Arguments: hashpartitioning(i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 1]
-Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
-
-(18) HashAggregate [codegen id : 1]
+(17) CometHashAggregate
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [sum(UnscaledValue(cs_ext_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(cs_ext_sales_price#2))#17]
-Results [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#2))#17,17,2) AS itemrevenue#18, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#2))#17,17,2) AS _w0#19, i_item_id#11]
-(19) CometColumnarExchange
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+(18) CometExchange
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(20) CometSort
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11], [i_class#12 ASC NULLS FIRST]
+(19) CometSort
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11], [i_class#12 ASC NULLS FIRST]
-(21) CometColumnarToRow [codegen id : 2]
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
+(20) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
-(22) Window
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [sum(_w0#19) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#20], [i_class#12]
+(21) Window
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [sum(_w0#18) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#19], [i_class#12]
-(23) Project [codegen id : 3]
-Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, ((_w0#19 * 100) / _we0#20) AS revenueratio#21, i_item_id#11]
-Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11, _we0#20]
+(22) Project [codegen id : 2]
+Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, ((_w0#18 * 100) / _we0#19) AS revenueratio#20, i_item_id#11]
+Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11, _we0#19]
-(24) TakeOrderedAndProject
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21, i_item_id#11]
-Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#21 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21]
+(23) TakeOrderedAndProject
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20, i_item_id#11]
+Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#20 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (29)
-+- * CometColumnarToRow (28)
- +- CometProject (27)
- +- CometFilter (26)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (25)
+BroadcastExchange (28)
++- * CometColumnarToRow (27)
+ +- CometProject (26)
+ +- CometFilter (25)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (24)
-(25) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(24) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(26) CometFilter
+(25) CometFilter
Input [2]: [d_date_sk#14, d_date#15]
Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(27) CometProject
+(26) CometProject
Input [2]: [d_date_sk#14, d_date#15]
Arguments: [d_date_sk#14], [d_date_sk#14]
-(28) CometColumnarToRow [codegen id : 1]
+(27) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#14]
-(29) BroadcastExchange
+(28) BroadcastExchange
Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/extended.txt
index a2d8b16a2..cd52b2cd1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/extended.txt
@@ -3,30 +3,29 @@ TakeOrderedAndProject
+- Window [COMET: WindowExec is not fully compatible with Spark (Native WindowExec has known correctness issues). To enable it anyway, set spark.comet.operator.WindowExec.allowIncompatible=true. For more information, refer to the Comet Compatibility Guide (https://datafusion.apache.org/comet/user-guide/compatibility.html).]
+- CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_item_desc#1, i_category#2, i_class#3, i_current_price#4, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#5))#6,17,2) AS itemrevenue#7, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#5))#6,17,2) AS _w0#8, i_item_id#9)]
- +- CometColumnarToRow
- +- CometExchange
- +- CometHashAggregate
- +- CometProject
- +- CometBroadcastHashJoin
- :- CometProject
- : +- CometBroadcastHashJoin
- : :- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
- : : +- SubqueryBroadcast
- : : +- BroadcastExchange
- : : +- CometColumnarToRow
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- CometBroadcastExchange
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
- +- CometBroadcastExchange
- +- CometProject
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
+ : : +- SubqueryBroadcast
+ : : +- BroadcastExchange
+ : : +- CometColumnarToRow
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 21 out of 27 eligible operators (77%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 22 out of 27 eligible operators (81%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/simplified.txt
index db00a01c2..2958d060f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20.native_iceberg_compat/simplified.txt
@@ -1,38 +1,35 @@
TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_current_price,itemrevenue]
- WholeStageCodegen (3)
+ WholeStageCodegen (2)
Project [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,_we0,i_item_id]
InputAdapter
Window [_w0,i_class]
- WholeStageCodegen (2)
+ WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id]
- CometColumnarExchange [i_class] #1
- WholeStageCodegen (1)
- HashAggregate [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum] [sum(UnscaledValue(cs_ext_sales_price)),itemrevenue,_w0,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
- CometHashAggregate [cs_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
- CometProject [cs_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
- CometProject [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
- CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
- CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastExchange [d_date_sk] #5
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometExchange [i_class] #1
+ CometHashAggregate [sum] [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id,sum(UnscaledValue(cs_ext_sales_price))]
+ CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
+ CometHashAggregate [cs_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
+ CometProject [cs_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
+ CometProject [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
+ CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastExchange [d_date_sk] #5
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/explain.txt
index 88752d6e1..e6612e757 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/explain.txt
@@ -1,28 +1,27 @@
== Physical Plan ==
-TakeOrderedAndProject (24)
-+- * Project (23)
- +- Window (22)
- +- * CometColumnarToRow (21)
- +- CometSort (20)
- +- CometColumnarExchange (19)
- +- * HashAggregate (18)
- +- * CometColumnarToRow (17)
- +- CometExchange (16)
- +- CometHashAggregate (15)
- +- CometProject (14)
- +- CometBroadcastHashJoin (13)
- :- CometProject (8)
- : +- CometBroadcastHashJoin (7)
- : :- CometFilter (2)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (1)
- : +- CometBroadcastExchange (6)
- : +- CometProject (5)
- : +- CometFilter (4)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
- +- CometBroadcastExchange (12)
- +- CometProject (11)
- +- CometFilter (10)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
+TakeOrderedAndProject (23)
++- * Project (22)
+ +- Window (21)
+ +- * CometColumnarToRow (20)
+ +- CometSort (19)
+ +- CometExchange (18)
+ +- CometHashAggregate (17)
+ +- CometExchange (16)
+ +- CometHashAggregate (15)
+ +- CometProject (14)
+ +- CometBroadcastHashJoin (13)
+ :- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
+ : :- CometFilter (2)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales (1)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item (3)
+ +- CometBroadcastExchange (12)
+ +- CometProject (11)
+ +- CometFilter (10)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (9)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
@@ -102,68 +101,63 @@ Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#2))]
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Arguments: hashpartitioning(i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 1]
-Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
-
-(18) HashAggregate [codegen id : 1]
+(17) CometHashAggregate
Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [sum(UnscaledValue(cs_ext_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(cs_ext_sales_price#2))#17]
-Results [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#2))#17,17,2) AS itemrevenue#18, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#2))#17,17,2) AS _w0#19, i_item_id#11]
-(19) CometColumnarExchange
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+(18) CometExchange
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(20) CometSort
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11], [i_class#12 ASC NULLS FIRST]
+(19) CometSort
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11], [i_class#12 ASC NULLS FIRST]
-(21) CometColumnarToRow [codegen id : 2]
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
+(20) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
-(22) Window
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11]
-Arguments: [sum(_w0#19) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#20], [i_class#12]
+(21) Window
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [sum(_w0#18) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#19], [i_class#12]
-(23) Project [codegen id : 3]
-Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, ((_w0#19 * 100) / _we0#20) AS revenueratio#21, i_item_id#11]
-Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, _w0#19, i_item_id#11, _we0#20]
+(22) Project [codegen id : 2]
+Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, ((_w0#18 * 100) / _we0#19) AS revenueratio#20, i_item_id#11]
+Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11, _we0#19]
-(24) TakeOrderedAndProject
-Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21, i_item_id#11]
-Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#21 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#18, revenueratio#21]
+(23) TakeOrderedAndProject
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20, i_item_id#11]
+Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#20 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (29)
-+- * CometColumnarToRow (28)
- +- CometProject (27)
- +- CometFilter (26)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (25)
+BroadcastExchange (28)
++- * CometColumnarToRow (27)
+ +- CometProject (26)
+ +- CometFilter (25)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (24)
-(25) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(24) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(26) CometFilter
+(25) CometFilter
Input [2]: [d_date_sk#14, d_date#15]
Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(27) CometProject
+(26) CometProject
Input [2]: [d_date_sk#14, d_date#15]
Arguments: [d_date_sk#14], [d_date_sk#14]
-(28) CometColumnarToRow [codegen id : 1]
+(27) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#14]
-(29) BroadcastExchange
+(28) BroadcastExchange
Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/extended.txt
index a2d8b16a2..cd52b2cd1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/extended.txt
@@ -3,30 +3,29 @@ TakeOrderedAndProject
+- Window [COMET: WindowExec is not fully compatible with Spark (Native WindowExec has known correctness issues). To enable it anyway, set spark.comet.operator.WindowExec.allowIncompatible=true. For more information, refer to the Comet Compatibility Guide (https://datafusion.apache.org/comet/user-guide/compatibility.html).]
+- CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_item_desc#1, i_category#2, i_class#3, i_current_price#4, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#5))#6,17,2) AS itemrevenue#7, MakeDecimal(sum(UnscaledValue(cs_ext_sales_price#5))#6,17,2) AS _w0#8, i_item_id#9)]
- +- CometColumnarToRow
- +- CometExchange
- +- CometHashAggregate
- +- CometProject
- +- CometBroadcastHashJoin
- :- CometProject
- : +- CometBroadcastHashJoin
- : :- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
- : : +- SubqueryBroadcast
- : : +- BroadcastExchange
- : : +- CometColumnarToRow
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- CometBroadcastExchange
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
- +- CometBroadcastExchange
- +- CometProject
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales
+ : : +- SubqueryBroadcast
+ : : +- BroadcastExchange
+ : : +- CometColumnarToRow
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
+ +- CometBroadcastExchange
+ +- CometProject
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Comet accelerated 21 out of 27 eligible operators (77%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 22 out of 27 eligible operators (81%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/simplified.txt
index db00a01c2..2958d060f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q20/simplified.txt
@@ -1,38 +1,35 @@
TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_current_price,itemrevenue]
- WholeStageCodegen (3)
+ WholeStageCodegen (2)
Project [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,_we0,i_item_id]
InputAdapter
Window [_w0,i_class]
- WholeStageCodegen (2)
+ WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id]
- CometColumnarExchange [i_class] #1
- WholeStageCodegen (1)
- HashAggregate [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum] [sum(UnscaledValue(cs_ext_sales_price)),itemrevenue,_w0,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
- CometHashAggregate [cs_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
- CometProject [cs_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
- CometProject [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
- CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
- CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometBroadcastExchange [d_date_sk] #5
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometExchange [i_class] #1
+ CometHashAggregate [sum] [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id,sum(UnscaledValue(cs_ext_sales_price))]
+ CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
+ CometHashAggregate [cs_ext_sales_price] [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum]
+ CometProject [cs_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
+ CometProject [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk,i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_ext_sales_price,cs_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
+ CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometBroadcastExchange [d_date_sk] #5
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/explain.txt
index f2c8b8a99..f076c7886 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (44)
-+- * HashAggregate (43)
- +- * CometColumnarToRow (42)
+* CometColumnarToRow (44)
++- CometTakeOrderedAndProject (43)
+ +- CometHashAggregate (42)
+- CometExchange (41)
+- CometHashAggregate (40)
+- CometProject (39)
@@ -240,19 +240,17 @@ Functions [3]: [partial_sum(UnscaledValue(ss_net_profit#5)), partial_sum(Unscale
Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
Arguments: hashpartitioning(i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(42) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
-
-(43) HashAggregate [codegen id : 1]
+(42) CometHashAggregate
Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
Keys [4]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28]
Functions [3]: [sum(UnscaledValue(ss_net_profit#5)), sum(UnscaledValue(sr_net_loss#11)), sum(UnscaledValue(cs_net_profit#16))]
-Aggregate Attributes [3]: [sum(UnscaledValue(ss_net_profit#5))#37, sum(UnscaledValue(sr_net_loss#11))#38, sum(UnscaledValue(cs_net_profit#16))#39]
-Results [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, MakeDecimal(sum(UnscaledValue(ss_net_profit#5))#37,17,2) AS store_sales_profit#40, MakeDecimal(sum(UnscaledValue(sr_net_loss#11))#38,17,2) AS store_returns_loss#41, MakeDecimal(sum(UnscaledValue(cs_net_profit#16))#39,17,2) AS catalog_sales_profit#42]
-(44) TakeOrderedAndProject
-Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#40, store_returns_loss#41, catalog_sales_profit#42]
-Arguments: 100, [i_item_id#33 ASC NULLS FIRST, i_item_desc#32 ASC NULLS FIRST, s_store_id#29 ASC NULLS FIRST, s_store_name#28 ASC NULLS FIRST], [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#40, store_returns_loss#41, catalog_sales_profit#42]
+(43) CometTakeOrderedAndProject
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#33 ASC NULLS FIRST,i_item_desc#32 ASC NULLS FIRST,s_store_id#29 ASC NULLS FIRST,s_store_name#28 ASC NULLS FIRST], output=[i_item_id#33,i_item_desc#32,s_store_id#29,s_store_name#28,store_sales_profit#37,store_returns_loss#38,catalog_sales_profit#39]), [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39], 100, 0, [i_item_id#33 ASC NULLS FIRST, i_it [...]
+
+(44) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39]
===== Subqueries =====
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/extended.txt
index cded28ce8..4e1a4f3ec 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_item_id#1, i_item_desc#2, s_store_id#3, s_store_name#4, MakeDecimal(sum(UnscaledValue(ss_net_profit#5))#6,17,2) AS store_sales_profit#7, MakeDecimal(sum(UnscaledValue(sr_net_loss#8))#9,17,2) AS store_returns_loss#10, MakeDecimal(sum(UnscaledValue(cs_net_profit#11))#12,17,2) AS catalog_sales_profit#13)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -59,4 +59,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
-Comet accelerated 50 out of 57 eligible operators (87%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 52 out of 57 eligible operators (91%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/simplified.txt
index 2140d7239..dcad30445 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25.native_iceberg_compat/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
- WholeStageCodegen (1)
- HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
+ CometHashAggregate [sum,sum,sum] [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit,sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit))]
CometExchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
CometHashAggregate [ss_net_profit,sr_net_loss,cs_net_profit] [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum]
CometProject [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/explain.txt
index f2c8b8a99..f076c7886 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (44)
-+- * HashAggregate (43)
- +- * CometColumnarToRow (42)
+* CometColumnarToRow (44)
++- CometTakeOrderedAndProject (43)
+ +- CometHashAggregate (42)
+- CometExchange (41)
+- CometHashAggregate (40)
+- CometProject (39)
@@ -240,19 +240,17 @@ Functions [3]: [partial_sum(UnscaledValue(ss_net_profit#5)), partial_sum(Unscale
Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
Arguments: hashpartitioning(i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(42) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
-
-(43) HashAggregate [codegen id : 1]
+(42) CometHashAggregate
Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
Keys [4]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28]
Functions [3]: [sum(UnscaledValue(ss_net_profit#5)), sum(UnscaledValue(sr_net_loss#11)), sum(UnscaledValue(cs_net_profit#16))]
-Aggregate Attributes [3]: [sum(UnscaledValue(ss_net_profit#5))#37, sum(UnscaledValue(sr_net_loss#11))#38, sum(UnscaledValue(cs_net_profit#16))#39]
-Results [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, MakeDecimal(sum(UnscaledValue(ss_net_profit#5))#37,17,2) AS store_sales_profit#40, MakeDecimal(sum(UnscaledValue(sr_net_loss#11))#38,17,2) AS store_returns_loss#41, MakeDecimal(sum(UnscaledValue(cs_net_profit#16))#39,17,2) AS catalog_sales_profit#42]
-(44) TakeOrderedAndProject
-Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#40, store_returns_loss#41, catalog_sales_profit#42]
-Arguments: 100, [i_item_id#33 ASC NULLS FIRST, i_item_desc#32 ASC NULLS FIRST, s_store_id#29 ASC NULLS FIRST, s_store_name#28 ASC NULLS FIRST], [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#40, store_returns_loss#41, catalog_sales_profit#42]
+(43) CometTakeOrderedAndProject
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#33 ASC NULLS FIRST,i_item_desc#32 ASC NULLS FIRST,s_store_id#29 ASC NULLS FIRST,s_store_name#28 ASC NULLS FIRST], output=[i_item_id#33,i_item_desc#32,s_store_id#29,s_store_name#28,store_sales_profit#37,store_returns_loss#38,catalog_sales_profit#39]), [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39], 100, 0, [i_item_id#33 ASC NULLS FIRST, i_it [...]
+
+(44) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39]
===== Subqueries =====
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/extended.txt
index cded28ce8..4e1a4f3ec 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(i_item_id#1, i_item_desc#2, s_store_id#3, s_store_name#4, MakeDecimal(sum(UnscaledValue(ss_net_profit#5))#6,17,2) AS store_sales_profit#7, MakeDecimal(sum(UnscaledValue(sr_net_loss#8))#9,17,2) AS store_returns_loss#10, MakeDecimal(sum(UnscaledValue(cs_net_profit#11))#12,17,2) AS catalog_sales_profit#13)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -59,4 +59,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
-Comet accelerated 50 out of 57 eligible operators (87%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 52 out of 57 eligible operators (91%). Final plan contains 3 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/simplified.txt
index 2140d7239..dcad30445 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q25/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
- WholeStageCodegen (1)
- HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
+ CometHashAggregate [sum,sum,sum] [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit,sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit))]
CometExchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
CometHashAggregate [ss_net_profit,sr_net_loss,cs_net_profit] [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum]
CometProject [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/explain.txt
index 59dc80719..2d4f57a9f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (19)
-+- * HashAggregate (18)
- +- * CometColumnarToRow (17)
+* CometColumnarToRow (19)
++- CometTakeOrderedAndProject (18)
+ +- CometHashAggregate (17)
+- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
@@ -97,17 +97,15 @@ Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
Arguments: hashpartitioning(d_year#2, i_brand#11, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 1]
-Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
-
-(18) HashAggregate [codegen id : 1]
+(17) CometHashAggregate
Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
Keys [3]: [d_year#2, i_brand#11, i_brand_id#8]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#5))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#5))#13]
-Results [4]: [d_year#2, i_brand_id#8 AS brand_id#14, i_brand#11 AS brand#15, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#5))#13,17,2) AS sum_agg#16]
-(19) TakeOrderedAndProject
-Input [4]: [d_year#2, brand_id#14, brand#15, sum_agg#16]
-Arguments: 100, [d_year#2 ASC NULLS FIRST, sum_agg#16 DESC NULLS LAST, brand_id#14 ASC NULLS FIRST], [d_year#2, brand_id#14, brand#15, sum_agg#16]
+(18) CometTakeOrderedAndProject
+Input [4]: [d_year#2, brand_id#13, brand#14, sum_agg#15]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[d_year#2 ASC NULLS FIRST,sum_agg#15 DESC NULLS LAST,brand_id#13 ASC NULLS FIRST], output=[d_year#2,brand_id#13,brand#14,sum_agg#15]), [d_year#2, brand_id#13, brand#14, sum_agg#15], 100, 0, [d_year#2 ASC NULLS FIRST, sum_agg#15 DESC NULLS LAST, brand_id#13 ASC NULLS FIRST], [d_year#2, brand_id#13, brand#14, sum_agg#15]
+
+(19) CometColumnarToRow [codegen id : 1]
+Input [4]: [d_year#2, brand_id#13, brand#14, sum_agg#15]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/extended.txt
index 58ab22b65..a292badf5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(d_year#1, i_brand_id#2 AS brand_id#3, i_brand#4 AS brand#5, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#6))#7,17,2) AS sum_agg#8)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -18,4 +18,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
-Comet accelerated 16 out of 18 eligible operators (88%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 18 out of 18 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/simplified.txt
index 84c696334..7bdad4b94 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3.native_iceberg_compat/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [d_year,sum_agg,brand_id,brand]
- WholeStageCodegen (1)
- HashAggregate [d_year,i_brand,i_brand_id,sum] [sum(UnscaledValue(ss_ext_sales_price)),brand_id,brand,sum_agg,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [d_year,brand_id,brand,sum_agg]
+ CometHashAggregate [sum] [d_year,brand_id,brand,sum_agg,i_brand_id,i_brand,sum(UnscaledValue(ss_ext_sales_price))]
CometExchange [d_year,i_brand,i_brand_id] #1
CometHashAggregate [ss_ext_sales_price] [d_year,i_brand,i_brand_id,sum]
CometProject [d_year,ss_ext_sales_price,i_brand_id,i_brand]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/explain.txt
index 59dc80719..2d4f57a9f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
-TakeOrderedAndProject (19)
-+- * HashAggregate (18)
- +- * CometColumnarToRow (17)
+* CometColumnarToRow (19)
++- CometTakeOrderedAndProject (18)
+ +- CometHashAggregate (17)
+- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
@@ -97,17 +97,15 @@ Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
Arguments: hashpartitioning(d_year#2, i_brand#11, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 1]
-Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
-
-(18) HashAggregate [codegen id : 1]
+(17) CometHashAggregate
Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
Keys [3]: [d_year#2, i_brand#11, i_brand_id#8]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#5))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#5))#13]
-Results [4]: [d_year#2, i_brand_id#8 AS brand_id#14, i_brand#11 AS brand#15, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#5))#13,17,2) AS sum_agg#16]
-(19) TakeOrderedAndProject
-Input [4]: [d_year#2, brand_id#14, brand#15, sum_agg#16]
-Arguments: 100, [d_year#2 ASC NULLS FIRST, sum_agg#16 DESC NULLS LAST, brand_id#14 ASC NULLS FIRST], [d_year#2, brand_id#14, brand#15, sum_agg#16]
+(18) CometTakeOrderedAndProject
+Input [4]: [d_year#2, brand_id#13, brand#14, sum_agg#15]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[d_year#2 ASC NULLS FIRST,sum_agg#15 DESC NULLS LAST,brand_id#13 ASC NULLS FIRST], output=[d_year#2,brand_id#13,brand#14,sum_agg#15]), [d_year#2, brand_id#13, brand#14, sum_agg#15], 100, 0, [d_year#2 ASC NULLS FIRST, sum_agg#15 DESC NULLS LAST, brand_id#13 ASC NULLS FIRST], [d_year#2, brand_id#13, brand#14, sum_agg#15]
+
+(19) CometColumnarToRow [codegen id : 1]
+Input [4]: [d_year#2, brand_id#13, brand#14, sum_agg#15]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/extended.txt
index 58ab22b65..a292badf5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/extended.txt
@@ -1,6 +1,6 @@
-TakeOrderedAndProject
-+- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(d_year#1, i_brand_id#2 AS brand_id#3, i_brand#4 AS brand#5, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#6))#7,17,2) AS sum_agg#8)]
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometHashAggregate
+- CometExchange
+- CometHashAggregate
+- CometProject
@@ -18,4 +18,4 @@ TakeOrderedAndProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.item
-Comet accelerated 16 out of 18 eligible operators (88%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 18 out of 18 eligible operators (100%). Final plan contains 1 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/simplified.txt
index 84c696334..7bdad4b94 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q3/simplified.txt
@@ -1,8 +1,8 @@
-TakeOrderedAndProject [d_year,sum_agg,brand_id,brand]
- WholeStageCodegen (1)
- HashAggregate [d_year,i_brand,i_brand_id,sum] [sum(UnscaledValue(ss_ext_sales_price)),brand_id,brand,sum_agg,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [d_year,brand_id,brand,sum_agg]
+ CometHashAggregate [sum] [d_year,brand_id,brand,sum_agg,i_brand_id,i_brand,sum(UnscaledValue(ss_ext_sales_price))]
CometExchange [d_year,i_brand,i_brand_id] #1
CometHashAggregate [ss_ext_sales_price] [d_year,i_brand,i_brand_id,sum]
CometProject [d_year,ss_ext_sales_price,i_brand_id,i_brand]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/explain.txt
index c97d45cc8..dac625d85 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/explain.txt
@@ -1,58 +1,54 @@
== Physical Plan ==
-TakeOrderedAndProject (54)
-+- * Project (53)
- +- * BroadcastHashJoin Inner BuildRight (52)
- :- * Project (46)
- : +- * BroadcastHashJoin Inner BuildRight (45)
- : :- * Project (39)
- : : +- * BroadcastHashJoin Inner BuildRight (38)
- : : :- * Filter (19)
- : : : +- * HashAggregate (18)
- : : : +- * CometColumnarToRow (17)
- : : : +- CometExchange (16)
- : : : +- CometHashAggregate (15)
- : : : +- CometProject (14)
- : : : +- CometBroadcastHashJoin (13)
- : : : :- CometProject (8)
- : : : : +- CometBroadcastHashJoin (7)
- : : : : :- CometFilter (2)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (1)
- : : : : +- CometBroadcastExchange (6)
- : : : : +- CometProject (5)
- : : : : +- CometFilter (4)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
- : : : +- CometBroadcastExchange (12)
- : : : +- CometProject (11)
- : : : +- CometFilter (10)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (9)
- : : +- BroadcastExchange (37)
- : : +- * Filter (36)
- : : +- * HashAggregate (35)
- : : +- * CometColumnarToRow (34)
- : : +- CometColumnarExchange (33)
- : : +- * HashAggregate (32)
- : : +- * HashAggregate (31)
- : : +- * CometColumnarToRow (30)
- : : +- CometExchange (29)
- : : +- CometHashAggregate (28)
- : : +- CometProject (27)
- : : +- CometBroadcastHashJoin (26)
- : : :- CometProject (24)
- : : : +- CometBroadcastHashJoin (23)
- : : : :- CometFilter (21)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (20)
- : : : +- ReusedExchange (22)
- : : +- ReusedExchange (25)
- : +- BroadcastExchange (44)
- : +- * CometColumnarToRow (43)
- : +- CometProject (42)
- : +- CometFilter (41)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (40)
- +- BroadcastExchange (51)
- +- * CometColumnarToRow (50)
- +- CometProject (49)
- +- CometFilter (48)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (47)
+* CometColumnarToRow (50)
++- CometTakeOrderedAndProject (49)
+ +- CometProject (48)
+ +- CometBroadcastHashJoin (47)
+ :- CometProject (42)
+ : +- CometBroadcastHashJoin (41)
+ : :- CometProject (36)
+ : : +- CometBroadcastHashJoin (35)
+ : : :- CometFilter (18)
+ : : : +- CometHashAggregate (17)
+ : : : +- CometExchange (16)
+ : : : +- CometHashAggregate (15)
+ : : : +- CometProject (14)
+ : : : +- CometBroadcastHashJoin (13)
+ : : : :- CometProject (8)
+ : : : : +- CometBroadcastHashJoin (7)
+ : : : : :- CometFilter (2)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (1)
+ : : : : +- CometBroadcastExchange (6)
+ : : : : +- CometProject (5)
+ : : : : +- CometFilter (4)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
+ : : : +- CometBroadcastExchange (12)
+ : : : +- CometProject (11)
+ : : : +- CometFilter (10)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (9)
+ : : +- CometBroadcastExchange (34)
+ : : +- CometFilter (33)
+ : : +- CometHashAggregate (32)
+ : : +- CometExchange (31)
+ : : +- CometHashAggregate (30)
+ : : +- CometHashAggregate (29)
+ : : +- CometExchange (28)
+ : : +- CometHashAggregate (27)
+ : : +- CometProject (26)
+ : : +- CometBroadcastHashJoin (25)
+ : : :- CometProject (23)
+ : : : +- CometBroadcastHashJoin (22)
+ : : : :- CometFilter (20)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (19)
+ : : : +- ReusedExchange (21)
+ : : +- ReusedExchange (24)
+ : +- CometBroadcastExchange (40)
+ : +- CometProject (39)
+ : +- CometFilter (38)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (37)
+ +- CometBroadcastExchange (46)
+ +- CometProject (45)
+ +- CometFilter (44)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (43)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
@@ -132,214 +128,191 @@ Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#3))]
Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
Arguments: hashpartitioning(wr_returning_customer_sk#1, ca_state#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 5]
-Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
-
-(18) HashAggregate [codegen id : 5]
+(17) CometHashAggregate
Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
Keys [2]: [wr_returning_customer_sk#1, ca_state#10]
Functions [1]: [sum(UnscaledValue(wr_return_amt#3))]
-Aggregate Attributes [1]: [sum(UnscaledValue(wr_return_amt#3))#12]
-Results [3]: [wr_returning_customer_sk#1 AS ctr_customer_sk#13, ca_state#10 AS ctr_state#14, MakeDecimal(sum(UnscaledValue(wr_return_amt#3))#12,17,2) AS ctr_total_return#15]
-(19) Filter [codegen id : 5]
-Input [3]: [ctr_customer_sk#13, ctr_state#14, ctr_total_return#15]
-Condition : isnotnull(ctr_total_return#15)
+(18) CometFilter
+Input [3]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14]
+Condition : isnotnull(ctr_total_return#14)
-(20) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
-Output [4]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19]
+(19) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
+Output [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(wr_returned_date_sk#19), dynamicpruningexpression(wr_returned_date_sk#19 IN dynamicpruning#20)]
+PartitionFilters: [isnotnull(wr_returned_date_sk#18), dynamicpruningexpression(wr_returned_date_sk#18 IN dynamicpruning#19)]
PushedFilters: [IsNotNull(wr_returning_addr_sk)]
ReadSchema: struct<wr_returning_customer_sk:int,wr_returning_addr_sk:int,wr_return_amt:decimal(7,2)>
-(21) CometFilter
-Input [4]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19]
-Condition : isnotnull(wr_returning_addr_sk#17)
-
-(22) ReusedExchange [Reuses operator id: 6]
-Output [1]: [d_date_sk#21]
-
-(23) CometBroadcastHashJoin
-Left output [4]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19]
-Right output [1]: [d_date_sk#21]
-Arguments: [wr_returned_date_sk#19], [d_date_sk#21], Inner, BuildRight
-
-(24) CometProject
-Input [5]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19, d_date_sk#21]
-Arguments: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18], [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18]
-
-(25) ReusedExchange [Reuses operator id: 12]
-Output [2]: [ca_address_sk#22, ca_state#23]
-
-(26) CometBroadcastHashJoin
-Left output [3]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18]
-Right output [2]: [ca_address_sk#22, ca_state#23]
-Arguments: [wr_returning_addr_sk#17], [ca_address_sk#22], Inner, BuildRight
-
-(27) CometProject
-Input [5]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, ca_address_sk#22, ca_state#23]
-Arguments: [wr_returning_customer_sk#16, wr_return_amt#18, ca_state#23], [wr_returning_customer_sk#16, wr_return_amt#18, ca_state#23]
-
-(28) CometHashAggregate
-Input [3]: [wr_returning_customer_sk#16, wr_return_amt#18, ca_state#23]
-Keys [2]: [wr_returning_customer_sk#16, ca_state#23]
-Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#18))]
-
-(29) CometExchange
-Input [3]: [wr_returning_customer_sk#16, ca_state#23, sum#24]
-Arguments: hashpartitioning(wr_returning_customer_sk#16, ca_state#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(30) CometColumnarToRow [codegen id : 1]
-Input [3]: [wr_returning_customer_sk#16, ca_state#23, sum#24]
-
-(31) HashAggregate [codegen id : 1]
-Input [3]: [wr_returning_customer_sk#16, ca_state#23, sum#24]
-Keys [2]: [wr_returning_customer_sk#16, ca_state#23]
-Functions [1]: [sum(UnscaledValue(wr_return_amt#18))]
-Aggregate Attributes [1]: [sum(UnscaledValue(wr_return_amt#18))#12]
-Results [2]: [ca_state#23 AS ctr_state#25, MakeDecimal(sum(UnscaledValue(wr_return_amt#18))#12,17,2) AS ctr_total_return#26]
-
-(32) HashAggregate [codegen id : 1]
-Input [2]: [ctr_state#25, ctr_total_return#26]
-Keys [1]: [ctr_state#25]
-Functions [1]: [partial_avg(ctr_total_return#26)]
-Aggregate Attributes [2]: [sum#27, count#28]
-Results [3]: [ctr_state#25, sum#29, count#30]
-
-(33) CometColumnarExchange
-Input [3]: [ctr_state#25, sum#29, count#30]
-Arguments: hashpartitioning(ctr_state#25, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
-
-(34) CometColumnarToRow [codegen id : 2]
-Input [3]: [ctr_state#25, sum#29, count#30]
-
-(35) HashAggregate [codegen id : 2]
-Input [3]: [ctr_state#25, sum#29, count#30]
-Keys [1]: [ctr_state#25]
-Functions [1]: [avg(ctr_total_return#26)]
-Aggregate Attributes [1]: [avg(ctr_total_return#26)#31]
-Results [2]: [(avg(ctr_total_return#26)#31 * 1.2) AS (avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-
-(36) Filter [codegen id : 2]
-Input [2]: [(avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-Condition : isnotnull((avg(ctr_total_return) * 1.2)#32)
-
-(37) BroadcastExchange
-Input [2]: [(avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-Arguments: HashedRelationBroadcastMode(List(input[1, string, true]),false), [plan_id=4]
-
-(38) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_state#14]
-Right keys [1]: [ctr_state#25]
-Join type: Inner
-Join condition: (cast(ctr_total_return#15 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#32)
-
-(39) Project [codegen id : 5]
-Output [2]: [ctr_customer_sk#13, ctr_total_return#15]
-Input [5]: [ctr_customer_sk#13, ctr_state#14, ctr_total_return#15, (avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-
-(40) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [14]: [c_customer_sk#33, c_customer_id#34, c_current_addr_sk#35, c_salutation#36, c_first_name#37, c_last_name#38, c_preferred_cust_flag#39, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#44, c_email_address#45, c_last_review_date#46]
+(20) CometFilter
+Input [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
+Condition : isnotnull(wr_returning_addr_sk#16)
+
+(21) ReusedExchange [Reuses operator id: 6]
+Output [1]: [d_date_sk#20]
+
+(22) CometBroadcastHashJoin
+Left output [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
+Right output [1]: [d_date_sk#20]
+Arguments: [wr_returned_date_sk#18], [d_date_sk#20], Inner, BuildRight
+
+(23) CometProject
+Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18, d_date_sk#20]
+Arguments: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17], [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17]
+
+(24) ReusedExchange [Reuses operator id: 12]
+Output [2]: [ca_address_sk#21, ca_state#22]
+
+(25) CometBroadcastHashJoin
+Left output [3]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17]
+Right output [2]: [ca_address_sk#21, ca_state#22]
+Arguments: [wr_returning_addr_sk#16], [ca_address_sk#21], Inner, BuildRight
+
+(26) CometProject
+Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, ca_address_sk#21, ca_state#22]
+Arguments: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22], [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22]
+
+(27) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22]
+Keys [2]: [wr_returning_customer_sk#15, ca_state#22]
+Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#17))]
+
+(28) CometExchange
+Input [3]: [wr_returning_customer_sk#15, ca_state#22, sum#23]
+Arguments: hashpartitioning(wr_returning_customer_sk#15, ca_state#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(29) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#15, ca_state#22, sum#23]
+Keys [2]: [wr_returning_customer_sk#15, ca_state#22]
+Functions [1]: [sum(UnscaledValue(wr_return_amt#17))]
+
+(30) CometHashAggregate
+Input [2]: [ctr_state#24, ctr_total_return#25]
+Keys [1]: [ctr_state#24]
+Functions [1]: [partial_avg(ctr_total_return#25)]
+
+(31) CometExchange
+Input [3]: [ctr_state#24, sum#26, count#27]
+Arguments: hashpartitioning(ctr_state#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+
+(32) CometHashAggregate
+Input [3]: [ctr_state#24, sum#26, count#27]
+Keys [1]: [ctr_state#24]
+Functions [1]: [avg(ctr_total_return#25)]
+
+(33) CometFilter
+Input [2]: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Condition : isnotnull((avg(ctr_total_return) * 1.2)#28)
+
+(34) CometBroadcastExchange
+Input [2]: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Arguments: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+
+(35) CometBroadcastHashJoin
+Left output [3]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14]
+Right output [2]: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Arguments: [ctr_state#13], [ctr_state#24], Inner, (cast(ctr_total_return#14 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#28), BuildRight
+
+(36) CometProject
+Input [5]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14, (avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Arguments: [ctr_customer_sk#12, ctr_total_return#14], [ctr_customer_sk#12, ctr_total_return#14]
+
+(37) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [14]: [c_customer_sk#29, c_customer_id#30, c_current_addr_sk#31, c_salutation#32, c_first_name#33, c_last_name#34, c_preferred_cust_flag#35, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#40, c_email_address#41, c_last_review_date#42]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_current_addr_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_current_addr_sk:int,c_salutation:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_day:int,c_birth_month:int,c_birth_year:int,c_birth_country:string,c_login:string,c_email_address:string,c_last_review_date:int>
-(41) CometFilter
-Input [14]: [c_customer_sk#33, c_customer_id#34, c_current_addr_sk#35, c_salutation#36, c_first_name#37, c_last_name#38, c_preferred_cust_flag#39, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#44, c_email_address#45, c_last_review_date#46]
-Condition : (isnotnull(c_customer_sk#33) AND isnotnull(c_current_addr_sk#35))
+(38) CometFilter
+Input [14]: [c_customer_sk#29, c_customer_id#30, c_current_addr_sk#31, c_salutation#32, c_first_name#33, c_last_name#34, c_preferred_cust_flag#35, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#40, c_email_address#41, c_last_review_date#42]
+Condition : (isnotnull(c_customer_sk#29) AND isnotnull(c_current_addr_sk#31))
-(42) CometProject
-Input [14]: [c_customer_sk#33, c_customer_id#34, c_current_addr_sk#35, c_salutation#36, c_first_name#37, c_last_name#38, c_preferred_cust_flag#39, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#44, c_email_address#45, c_last_review_date#46]
-Arguments: [c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46], [c_customer_sk#33, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#34, 16)) AS c_customer_id#47, c_current_addr_sk#35, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_salutation#36, 10)) AS c [...]
+(39) CometProject
+Input [14]: [c_customer_sk#29, c_customer_id#30, c_current_addr_sk#31, c_salutation#32, c_first_name#33, c_last_name#34, c_preferred_cust_flag#35, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#40, c_email_address#41, c_last_review_date#42]
+Arguments: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42], [c_customer_sk#29, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#30, 16)) AS c_customer_id#43, c_current_addr_sk#31, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_salutation#32, 10)) AS c [...]
-(43) CometColumnarToRow [codegen id : 3]
-Input [14]: [c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
+(40) CometBroadcastExchange
+Input [14]: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Arguments: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
-(44) BroadcastExchange
-Input [14]: [c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
+(41) CometBroadcastHashJoin
+Left output [2]: [ctr_customer_sk#12, ctr_total_return#14]
+Right output [14]: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Arguments: [ctr_customer_sk#12], [c_customer_sk#29], Inner, BuildRight
-(45) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_customer_sk#13]
-Right keys [1]: [c_customer_sk#33]
-Join type: Inner
-Join condition: None
-
-(46) Project [codegen id : 5]
-Output [14]: [ctr_total_return#15, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
-Input [16]: [ctr_customer_sk#13, ctr_total_return#15, c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
+(42) CometProject
+Input [16]: [ctr_customer_sk#12, ctr_total_return#14, c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Arguments: [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42], [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login [...]
-(47) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
-Output [2]: [ca_address_sk#54, ca_state#55]
+(43) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+Output [2]: [ca_address_sk#50, ca_state#51]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
PushedFilters: [IsNotNull(ca_state), IsNotNull(ca_address_sk)]
ReadSchema: struct<ca_address_sk:int,ca_state:string>
-(48) CometFilter
-Input [2]: [ca_address_sk#54, ca_state#55]
-Condition : ((isnotnull(ca_state#55) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(ca_state#55, 2)) = GA)) AND isnotnull(ca_address_sk#54))
+(44) CometFilter
+Input [2]: [ca_address_sk#50, ca_state#51]
+Condition : ((isnotnull(ca_state#51) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(ca_state#51, 2)) = GA)) AND isnotnull(ca_address_sk#50))
-(49) CometProject
-Input [2]: [ca_address_sk#54, ca_state#55]
-Arguments: [ca_address_sk#54], [ca_address_sk#54]
+(45) CometProject
+Input [2]: [ca_address_sk#50, ca_state#51]
+Arguments: [ca_address_sk#50], [ca_address_sk#50]
-(50) CometColumnarToRow [codegen id : 4]
-Input [1]: [ca_address_sk#54]
+(46) CometBroadcastExchange
+Input [1]: [ca_address_sk#50]
+Arguments: [ca_address_sk#50]
-(51) BroadcastExchange
-Input [1]: [ca_address_sk#54]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
+(47) CometBroadcastHashJoin
+Left output [14]: [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Right output [1]: [ca_address_sk#50]
+Arguments: [c_current_addr_sk#31], [ca_address_sk#50], Inner, BuildRight
-(52) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [c_current_addr_sk#35]
-Right keys [1]: [ca_address_sk#54]
-Join type: Inner
-Join condition: None
+(48) CometProject
+Input [15]: [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ca_address_sk#50]
+Arguments: [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14], [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14]
-(53) Project [codegen id : 5]
-Output [13]: [c_customer_id#47, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46, ctr_total_return#15]
-Input [15]: [ctr_total_return#15, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46, ca_address_sk#54]
+(49) CometTakeOrderedAndProject
+Input [13]: [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#43 ASC NULLS FIRST,c_salutation#44 ASC NULLS FIRST,c_first_name#45 ASC NULLS FIRST,c_last_name#46 ASC NULLS FIRST,c_preferred_cust_flag#47 ASC NULLS FIRST,c_birth_day#36 ASC NULLS FIRST,c_birth_month#37 ASC NULLS FIRST,c_birth_year#38 ASC NULLS FIRST,c_birth_country#39 ASC NULLS FIRST,c_login#48 ASC NULLS FIRST,c_email_address#49 ASC NULLS FIRST,c_last_review_date#42 ASC NULLS FIRST,ctr_total_return#14 ASC NULLS FIRST], o [...]
-(54) TakeOrderedAndProject
-Input [13]: [c_customer_id#47, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46, ctr_total_return#15]
-Arguments: 100, [c_customer_id#47 ASC NULLS FIRST, c_salutation#48 ASC NULLS FIRST, c_first_name#49 ASC NULLS FIRST, c_last_name#50 ASC NULLS FIRST, c_preferred_cust_flag#51 ASC NULLS FIRST, c_birth_day#40 ASC NULLS FIRST, c_birth_month#41 ASC NULLS FIRST, c_birth_year#42 ASC NULLS FIRST, c_birth_country#43 ASC NULLS FIRST, c_login#52 ASC NULLS FIRST, c_email_address#53 ASC NULLS FIRST, c_last_review_date#46 ASC NULLS FIRST, ctr_total_return#15 ASC NULLS FIRST], [c_customer_id#47, c_salu [...]
+(50) CometColumnarToRow [codegen id : 1]
+Input [13]: [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = wr_returned_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (59)
-+- * CometColumnarToRow (58)
- +- CometProject (57)
- +- CometFilter (56)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (55)
+BroadcastExchange (55)
++- * CometColumnarToRow (54)
+ +- CometProject (53)
+ +- CometFilter (52)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (51)
-(55) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(51) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_year#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(56) CometFilter
+(52) CometFilter
Input [2]: [d_date_sk#6, d_year#7]
Condition : ((isnotnull(d_year#7) AND (d_year#7 = 2002)) AND isnotnull(d_date_sk#6))
-(57) CometProject
+(53) CometProject
Input [2]: [d_date_sk#6, d_year#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(58) CometColumnarToRow [codegen id : 1]
+(54) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(59) BroadcastExchange
+(55) BroadcastExchange
Input [1]: [d_date_sk#6]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=7]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-Subquery:2 Hosting operator id = 20 Hosting Expression = wr_returned_date_sk#19 IN dynamicpruning#5
+Subquery:2 Hosting operator id = 19 Hosting Expression = wr_returned_date_sk#18 IN dynamicpruning#5
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/extended.txt
index 38b08219c..7ebd69d34 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/extended.txt
@@ -1,69 +1,65 @@
-TakeOrderedAndProject
-+- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- Project
- : : +- BroadcastHashJoin
- : : :- Filter
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(wr_returning_customer_sk#1 AS ctr_customer_sk#2, ca_state#3 AS ctr_state#4, MakeDecimal(sum(UnscaledValue(wr_return_amt#5))#6,17,2) AS ctr_total_return#7)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometBroadcastHashJoin
- : : : : :- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
- : : : : : +- SubqueryBroadcast
- : : : : : +- BroadcastExchange
- : : : : : +- CometColumnarToRow
- : : : : : +- CometProject
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : +- CometBroadcastExchange
- : : : : +- CometProject
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : : +- BroadcastExchange
- : : +- Filter
- : : +- HashAggregate
- : : +- CometColumnarToRow
- : : +- CometColumnarExchange
- : : +- HashAggregate
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_state#8 AS ctr_state#9, MakeDecimal(sum(UnscaledValue(wr_return_amt#10))#6,17,2) AS ctr_total_return#11)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
- : : : : +- ReusedSubquery
- : : : +- CometBroadcastExchange
- : : : +- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- CometBroadcastExchange
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : +- BroadcastExchange
- : +- CometColumnarToRow
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- +- BroadcastExchange
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometBroadcastHashJoin
+ : : : : :- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
+ : : : : : +- SubqueryBroadcast
+ : : : : : +- BroadcastExchange
+ : : : : : +- CometColumnarToRow
+ : : : : : +- CometProject
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometProject
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
+ : : : : +- ReusedSubquery
+ : : : +- CometBroadcastExchange
+ : : : +- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ +- CometBroadcastExchange
+- CometProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
-Comet accelerated 42 out of 61 eligible operators (68%). Final plan contains 6 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 58 out of 61 eligible operators (95%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/simplified.txt
index 112d86a05..0227534fa 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30.native_iceberg_compat/simplified.txt
@@ -1,15 +1,15 @@
-TakeOrderedAndProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
- WholeStageCodegen (5)
- Project [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
- BroadcastHashJoin [c_current_addr_sk,ca_address_sk]
- Project [ctr_total_return,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- BroadcastHashJoin [ctr_customer_sk,c_customer_sk]
- Project [ctr_customer_sk,ctr_total_return]
- BroadcastHashJoin [ctr_state,ctr_state,ctr_total_return,(avg(ctr_total_return) * 1.2)]
- Filter [ctr_total_return]
- HashAggregate [wr_returning_customer_sk,ca_state,sum] [sum(UnscaledValue(wr_return_amt)),ctr_customer_sk,ctr_state,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
+ CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
+ CometBroadcastHashJoin [ctr_total_return,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ca_address_sk]
+ CometProject [ctr_total_return,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_total_return,c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometProject [ctr_customer_sk,ctr_total_return]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_state,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_state]
+ CometFilter [ctr_customer_sk,ctr_state,ctr_total_return]
+ CometHashAggregate [sum] [ctr_customer_sk,ctr_state,ctr_total_return,wr_returning_customer_sk,ca_state,sum(UnscaledValue(wr_return_amt))]
CometExchange [wr_returning_customer_sk,ca_state] #1
CometHashAggregate [wr_return_amt] [wr_returning_customer_sk,ca_state,sum]
CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
@@ -34,43 +34,28 @@ TakeOrderedAndProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_pre
CometProject [ca_state] [ca_address_sk,ca_state]
CometFilter [ca_address_sk,ca_state]
CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (2)
- Filter [(avg(ctr_total_return) * 1.2)]
- HashAggregate [ctr_state,sum,count] [avg(ctr_total_return),(avg(ctr_total_return) * 1.2),sum,count]
- CometColumnarToRow
- InputAdapter
- CometColumnarExchange [ctr_state] #6
- WholeStageCodegen (1)
- HashAggregate [ctr_state,ctr_total_return] [sum,count,sum,count]
- HashAggregate [wr_returning_customer_sk,ca_state,sum] [sum(UnscaledValue(wr_return_amt)),ctr_state,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [wr_returning_customer_sk,ca_state] #7
- CometHashAggregate [wr_return_amt] [wr_returning_customer_sk,ca_state,sum]
- CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
- CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
- CometProject [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt]
- CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk,d_date_sk]
- CometFilter [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #3
- ReusedExchange [ca_address_sk,ca_state] #4
- InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (3)
- CometColumnarToRow
- InputAdapter
- CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- CometFilter [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- InputAdapter
- BroadcastExchange #9
- WholeStageCodegen (4)
- CometColumnarToRow
- InputAdapter
- CometProject [ca_address_sk]
- CometFilter [ca_address_sk,ca_state]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_state] #5
+ CometFilter [(avg(ctr_total_return) * 1.2),ctr_state]
+ CometHashAggregate [sum,count] [(avg(ctr_total_return) * 1.2),ctr_state,avg(ctr_total_return)]
+ CometExchange [ctr_state] #6
+ CometHashAggregate [ctr_total_return] [ctr_state,sum,count]
+ CometHashAggregate [wr_returning_customer_sk,sum] [ctr_state,ctr_total_return,ca_state,sum(UnscaledValue(wr_return_amt))]
+ CometExchange [wr_returning_customer_sk,ca_state] #7
+ CometHashAggregate [wr_return_amt] [wr_returning_customer_sk,ca_state,sum]
+ CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
+ CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
+ CometProject [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt]
+ CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk,d_date_sk]
+ CometFilter [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #3
+ ReusedExchange [ca_address_sk,ca_state] #4
+ CometBroadcastExchange [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date] #8
+ CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometFilter [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometBroadcastExchange [ca_address_sk] #9
+ CometProject [ca_address_sk]
+ CometFilter [ca_address_sk,ca_state]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/explain.txt
index c97d45cc8..dac625d85 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/explain.txt
@@ -1,58 +1,54 @@
== Physical Plan ==
-TakeOrderedAndProject (54)
-+- * Project (53)
- +- * BroadcastHashJoin Inner BuildRight (52)
- :- * Project (46)
- : +- * BroadcastHashJoin Inner BuildRight (45)
- : :- * Project (39)
- : : +- * BroadcastHashJoin Inner BuildRight (38)
- : : :- * Filter (19)
- : : : +- * HashAggregate (18)
- : : : +- * CometColumnarToRow (17)
- : : : +- CometExchange (16)
- : : : +- CometHashAggregate (15)
- : : : +- CometProject (14)
- : : : +- CometBroadcastHashJoin (13)
- : : : :- CometProject (8)
- : : : : +- CometBroadcastHashJoin (7)
- : : : : :- CometFilter (2)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (1)
- : : : : +- CometBroadcastExchange (6)
- : : : : +- CometProject (5)
- : : : : +- CometFilter (4)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
- : : : +- CometBroadcastExchange (12)
- : : : +- CometProject (11)
- : : : +- CometFilter (10)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (9)
- : : +- BroadcastExchange (37)
- : : +- * Filter (36)
- : : +- * HashAggregate (35)
- : : +- * CometColumnarToRow (34)
- : : +- CometColumnarExchange (33)
- : : +- * HashAggregate (32)
- : : +- * HashAggregate (31)
- : : +- * CometColumnarToRow (30)
- : : +- CometExchange (29)
- : : +- CometHashAggregate (28)
- : : +- CometProject (27)
- : : +- CometBroadcastHashJoin (26)
- : : :- CometProject (24)
- : : : +- CometBroadcastHashJoin (23)
- : : : :- CometFilter (21)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (20)
- : : : +- ReusedExchange (22)
- : : +- ReusedExchange (25)
- : +- BroadcastExchange (44)
- : +- * CometColumnarToRow (43)
- : +- CometProject (42)
- : +- CometFilter (41)
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (40)
- +- BroadcastExchange (51)
- +- * CometColumnarToRow (50)
- +- CometProject (49)
- +- CometFilter (48)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (47)
+* CometColumnarToRow (50)
++- CometTakeOrderedAndProject (49)
+ +- CometProject (48)
+ +- CometBroadcastHashJoin (47)
+ :- CometProject (42)
+ : +- CometBroadcastHashJoin (41)
+ : :- CometProject (36)
+ : : +- CometBroadcastHashJoin (35)
+ : : :- CometFilter (18)
+ : : : +- CometHashAggregate (17)
+ : : : +- CometExchange (16)
+ : : : +- CometHashAggregate (15)
+ : : : +- CometProject (14)
+ : : : +- CometBroadcastHashJoin (13)
+ : : : :- CometProject (8)
+ : : : : +- CometBroadcastHashJoin (7)
+ : : : : :- CometFilter (2)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (1)
+ : : : : +- CometBroadcastExchange (6)
+ : : : : +- CometProject (5)
+ : : : : +- CometFilter (4)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
+ : : : +- CometBroadcastExchange (12)
+ : : : +- CometProject (11)
+ : : : +- CometFilter (10)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (9)
+ : : +- CometBroadcastExchange (34)
+ : : +- CometFilter (33)
+ : : +- CometHashAggregate (32)
+ : : +- CometExchange (31)
+ : : +- CometHashAggregate (30)
+ : : +- CometHashAggregate (29)
+ : : +- CometExchange (28)
+ : : +- CometHashAggregate (27)
+ : : +- CometProject (26)
+ : : +- CometBroadcastHashJoin (25)
+ : : :- CometProject (23)
+ : : : +- CometBroadcastHashJoin (22)
+ : : : :- CometFilter (20)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns (19)
+ : : : +- ReusedExchange (21)
+ : : +- ReusedExchange (24)
+ : +- CometBroadcastExchange (40)
+ : +- CometProject (39)
+ : +- CometFilter (38)
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer (37)
+ +- CometBroadcastExchange (46)
+ +- CometProject (45)
+ +- CometFilter (44)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (43)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
@@ -132,214 +128,191 @@ Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#3))]
Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
Arguments: hashpartitioning(wr_returning_customer_sk#1, ca_state#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(17) CometColumnarToRow [codegen id : 5]
-Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
-
-(18) HashAggregate [codegen id : 5]
+(17) CometHashAggregate
Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
Keys [2]: [wr_returning_customer_sk#1, ca_state#10]
Functions [1]: [sum(UnscaledValue(wr_return_amt#3))]
-Aggregate Attributes [1]: [sum(UnscaledValue(wr_return_amt#3))#12]
-Results [3]: [wr_returning_customer_sk#1 AS ctr_customer_sk#13, ca_state#10 AS ctr_state#14, MakeDecimal(sum(UnscaledValue(wr_return_amt#3))#12,17,2) AS ctr_total_return#15]
-(19) Filter [codegen id : 5]
-Input [3]: [ctr_customer_sk#13, ctr_state#14, ctr_total_return#15]
-Condition : isnotnull(ctr_total_return#15)
+(18) CometFilter
+Input [3]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14]
+Condition : isnotnull(ctr_total_return#14)
-(20) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
-Output [4]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19]
+(19) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
+Output [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(wr_returned_date_sk#19), dynamicpruningexpression(wr_returned_date_sk#19 IN dynamicpruning#20)]
+PartitionFilters: [isnotnull(wr_returned_date_sk#18), dynamicpruningexpression(wr_returned_date_sk#18 IN dynamicpruning#19)]
PushedFilters: [IsNotNull(wr_returning_addr_sk)]
ReadSchema: struct<wr_returning_customer_sk:int,wr_returning_addr_sk:int,wr_return_amt:decimal(7,2)>
-(21) CometFilter
-Input [4]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19]
-Condition : isnotnull(wr_returning_addr_sk#17)
-
-(22) ReusedExchange [Reuses operator id: 6]
-Output [1]: [d_date_sk#21]
-
-(23) CometBroadcastHashJoin
-Left output [4]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19]
-Right output [1]: [d_date_sk#21]
-Arguments: [wr_returned_date_sk#19], [d_date_sk#21], Inner, BuildRight
-
-(24) CometProject
-Input [5]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, wr_returned_date_sk#19, d_date_sk#21]
-Arguments: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18], [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18]
-
-(25) ReusedExchange [Reuses operator id: 12]
-Output [2]: [ca_address_sk#22, ca_state#23]
-
-(26) CometBroadcastHashJoin
-Left output [3]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18]
-Right output [2]: [ca_address_sk#22, ca_state#23]
-Arguments: [wr_returning_addr_sk#17], [ca_address_sk#22], Inner, BuildRight
-
-(27) CometProject
-Input [5]: [wr_returning_customer_sk#16, wr_returning_addr_sk#17, wr_return_amt#18, ca_address_sk#22, ca_state#23]
-Arguments: [wr_returning_customer_sk#16, wr_return_amt#18, ca_state#23], [wr_returning_customer_sk#16, wr_return_amt#18, ca_state#23]
-
-(28) CometHashAggregate
-Input [3]: [wr_returning_customer_sk#16, wr_return_amt#18, ca_state#23]
-Keys [2]: [wr_returning_customer_sk#16, ca_state#23]
-Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#18))]
-
-(29) CometExchange
-Input [3]: [wr_returning_customer_sk#16, ca_state#23, sum#24]
-Arguments: hashpartitioning(wr_returning_customer_sk#16, ca_state#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(30) CometColumnarToRow [codegen id : 1]
-Input [3]: [wr_returning_customer_sk#16, ca_state#23, sum#24]
-
-(31) HashAggregate [codegen id : 1]
-Input [3]: [wr_returning_customer_sk#16, ca_state#23, sum#24]
-Keys [2]: [wr_returning_customer_sk#16, ca_state#23]
-Functions [1]: [sum(UnscaledValue(wr_return_amt#18))]
-Aggregate Attributes [1]: [sum(UnscaledValue(wr_return_amt#18))#12]
-Results [2]: [ca_state#23 AS ctr_state#25, MakeDecimal(sum(UnscaledValue(wr_return_amt#18))#12,17,2) AS ctr_total_return#26]
-
-(32) HashAggregate [codegen id : 1]
-Input [2]: [ctr_state#25, ctr_total_return#26]
-Keys [1]: [ctr_state#25]
-Functions [1]: [partial_avg(ctr_total_return#26)]
-Aggregate Attributes [2]: [sum#27, count#28]
-Results [3]: [ctr_state#25, sum#29, count#30]
-
-(33) CometColumnarExchange
-Input [3]: [ctr_state#25, sum#29, count#30]
-Arguments: hashpartitioning(ctr_state#25, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
-
-(34) CometColumnarToRow [codegen id : 2]
-Input [3]: [ctr_state#25, sum#29, count#30]
-
-(35) HashAggregate [codegen id : 2]
-Input [3]: [ctr_state#25, sum#29, count#30]
-Keys [1]: [ctr_state#25]
-Functions [1]: [avg(ctr_total_return#26)]
-Aggregate Attributes [1]: [avg(ctr_total_return#26)#31]
-Results [2]: [(avg(ctr_total_return#26)#31 * 1.2) AS (avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-
-(36) Filter [codegen id : 2]
-Input [2]: [(avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-Condition : isnotnull((avg(ctr_total_return) * 1.2)#32)
-
-(37) BroadcastExchange
-Input [2]: [(avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-Arguments: HashedRelationBroadcastMode(List(input[1, string, true]),false), [plan_id=4]
-
-(38) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_state#14]
-Right keys [1]: [ctr_state#25]
-Join type: Inner
-Join condition: (cast(ctr_total_return#15 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#32)
-
-(39) Project [codegen id : 5]
-Output [2]: [ctr_customer_sk#13, ctr_total_return#15]
-Input [5]: [ctr_customer_sk#13, ctr_state#14, ctr_total_return#15, (avg(ctr_total_return) * 1.2)#32, ctr_state#25]
-
-(40) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
-Output [14]: [c_customer_sk#33, c_customer_id#34, c_current_addr_sk#35, c_salutation#36, c_first_name#37, c_last_name#38, c_preferred_cust_flag#39, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#44, c_email_address#45, c_last_review_date#46]
+(20) CometFilter
+Input [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
+Condition : isnotnull(wr_returning_addr_sk#16)
+
+(21) ReusedExchange [Reuses operator id: 6]
+Output [1]: [d_date_sk#20]
+
+(22) CometBroadcastHashJoin
+Left output [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
+Right output [1]: [d_date_sk#20]
+Arguments: [wr_returned_date_sk#18], [d_date_sk#20], Inner, BuildRight
+
+(23) CometProject
+Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18, d_date_sk#20]
+Arguments: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17], [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17]
+
+(24) ReusedExchange [Reuses operator id: 12]
+Output [2]: [ca_address_sk#21, ca_state#22]
+
+(25) CometBroadcastHashJoin
+Left output [3]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17]
+Right output [2]: [ca_address_sk#21, ca_state#22]
+Arguments: [wr_returning_addr_sk#16], [ca_address_sk#21], Inner, BuildRight
+
+(26) CometProject
+Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, ca_address_sk#21, ca_state#22]
+Arguments: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22], [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22]
+
+(27) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22]
+Keys [2]: [wr_returning_customer_sk#15, ca_state#22]
+Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#17))]
+
+(28) CometExchange
+Input [3]: [wr_returning_customer_sk#15, ca_state#22, sum#23]
+Arguments: hashpartitioning(wr_returning_customer_sk#15, ca_state#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(29) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#15, ca_state#22, sum#23]
+Keys [2]: [wr_returning_customer_sk#15, ca_state#22]
+Functions [1]: [sum(UnscaledValue(wr_return_amt#17))]
+
+(30) CometHashAggregate
+Input [2]: [ctr_state#24, ctr_total_return#25]
+Keys [1]: [ctr_state#24]
+Functions [1]: [partial_avg(ctr_total_return#25)]
+
+(31) CometExchange
+Input [3]: [ctr_state#24, sum#26, count#27]
+Arguments: hashpartitioning(ctr_state#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+
+(32) CometHashAggregate
+Input [3]: [ctr_state#24, sum#26, count#27]
+Keys [1]: [ctr_state#24]
+Functions [1]: [avg(ctr_total_return#25)]
+
+(33) CometFilter
+Input [2]: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Condition : isnotnull((avg(ctr_total_return) * 1.2)#28)
+
+(34) CometBroadcastExchange
+Input [2]: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Arguments: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+
+(35) CometBroadcastHashJoin
+Left output [3]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14]
+Right output [2]: [(avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Arguments: [ctr_state#13], [ctr_state#24], Inner, (cast(ctr_total_return#14 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#28), BuildRight
+
+(36) CometProject
+Input [5]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14, (avg(ctr_total_return) * 1.2)#28, ctr_state#24]
+Arguments: [ctr_customer_sk#12, ctr_total_return#14], [ctr_customer_sk#12, ctr_total_return#14]
+
+(37) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+Output [14]: [c_customer_sk#29, c_customer_id#30, c_current_addr_sk#31, c_salutation#32, c_first_name#33, c_last_name#34, c_preferred_cust_flag#35, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#40, c_email_address#41, c_last_review_date#42]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_current_addr_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_current_addr_sk:int,c_salutation:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_day:int,c_birth_month:int,c_birth_year:int,c_birth_country:string,c_login:string,c_email_address:string,c_last_review_date:int>
-(41) CometFilter
-Input [14]: [c_customer_sk#33, c_customer_id#34, c_current_addr_sk#35, c_salutation#36, c_first_name#37, c_last_name#38, c_preferred_cust_flag#39, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#44, c_email_address#45, c_last_review_date#46]
-Condition : (isnotnull(c_customer_sk#33) AND isnotnull(c_current_addr_sk#35))
+(38) CometFilter
+Input [14]: [c_customer_sk#29, c_customer_id#30, c_current_addr_sk#31, c_salutation#32, c_first_name#33, c_last_name#34, c_preferred_cust_flag#35, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#40, c_email_address#41, c_last_review_date#42]
+Condition : (isnotnull(c_customer_sk#29) AND isnotnull(c_current_addr_sk#31))
-(42) CometProject
-Input [14]: [c_customer_sk#33, c_customer_id#34, c_current_addr_sk#35, c_salutation#36, c_first_name#37, c_last_name#38, c_preferred_cust_flag#39, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#44, c_email_address#45, c_last_review_date#46]
-Arguments: [c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46], [c_customer_sk#33, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#34, 16)) AS c_customer_id#47, c_current_addr_sk#35, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_salutation#36, 10)) AS c [...]
+(39) CometProject
+Input [14]: [c_customer_sk#29, c_customer_id#30, c_current_addr_sk#31, c_salutation#32, c_first_name#33, c_last_name#34, c_preferred_cust_flag#35, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#40, c_email_address#41, c_last_review_date#42]
+Arguments: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42], [c_customer_sk#29, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_customer_id#30, 16)) AS c_customer_id#43, c_current_addr_sk#31, static_invoke(CharVarcharCodegenUtils.readSidePadding(c_salutation#32, 10)) AS c [...]
-(43) CometColumnarToRow [codegen id : 3]
-Input [14]: [c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
+(40) CometBroadcastExchange
+Input [14]: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Arguments: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
-(44) BroadcastExchange
-Input [14]: [c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
+(41) CometBroadcastHashJoin
+Left output [2]: [ctr_customer_sk#12, ctr_total_return#14]
+Right output [14]: [c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Arguments: [ctr_customer_sk#12], [c_customer_sk#29], Inner, BuildRight
-(45) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [ctr_customer_sk#13]
-Right keys [1]: [c_customer_sk#33]
-Join type: Inner
-Join condition: None
-
-(46) Project [codegen id : 5]
-Output [14]: [ctr_total_return#15, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
-Input [16]: [ctr_customer_sk#13, ctr_total_return#15, c_customer_sk#33, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46]
+(42) CometProject
+Input [16]: [ctr_customer_sk#12, ctr_total_return#14, c_customer_sk#29, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Arguments: [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42], [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login [...]
-(47) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
-Output [2]: [ca_address_sk#54, ca_state#55]
+(43) CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+Output [2]: [ca_address_sk#50, ca_state#51]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
PushedFilters: [IsNotNull(ca_state), IsNotNull(ca_address_sk)]
ReadSchema: struct<ca_address_sk:int,ca_state:string>
-(48) CometFilter
-Input [2]: [ca_address_sk#54, ca_state#55]
-Condition : ((isnotnull(ca_state#55) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(ca_state#55, 2)) = GA)) AND isnotnull(ca_address_sk#54))
+(44) CometFilter
+Input [2]: [ca_address_sk#50, ca_state#51]
+Condition : ((isnotnull(ca_state#51) AND (static_invoke(CharVarcharCodegenUtils.readSidePadding(ca_state#51, 2)) = GA)) AND isnotnull(ca_address_sk#50))
-(49) CometProject
-Input [2]: [ca_address_sk#54, ca_state#55]
-Arguments: [ca_address_sk#54], [ca_address_sk#54]
+(45) CometProject
+Input [2]: [ca_address_sk#50, ca_state#51]
+Arguments: [ca_address_sk#50], [ca_address_sk#50]
-(50) CometColumnarToRow [codegen id : 4]
-Input [1]: [ca_address_sk#54]
+(46) CometBroadcastExchange
+Input [1]: [ca_address_sk#50]
+Arguments: [ca_address_sk#50]
-(51) BroadcastExchange
-Input [1]: [ca_address_sk#54]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
+(47) CometBroadcastHashJoin
+Left output [14]: [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42]
+Right output [1]: [ca_address_sk#50]
+Arguments: [c_current_addr_sk#31], [ca_address_sk#50], Inner, BuildRight
-(52) BroadcastHashJoin [codegen id : 5]
-Left keys [1]: [c_current_addr_sk#35]
-Right keys [1]: [ca_address_sk#54]
-Join type: Inner
-Join condition: None
+(48) CometProject
+Input [15]: [ctr_total_return#14, c_customer_id#43, c_current_addr_sk#31, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ca_address_sk#50]
+Arguments: [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14], [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14]
-(53) Project [codegen id : 5]
-Output [13]: [c_customer_id#47, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46, ctr_total_return#15]
-Input [15]: [ctr_total_return#15, c_customer_id#47, c_current_addr_sk#35, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46, ca_address_sk#54]
+(49) CometTakeOrderedAndProject
+Input [13]: [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#43 ASC NULLS FIRST,c_salutation#44 ASC NULLS FIRST,c_first_name#45 ASC NULLS FIRST,c_last_name#46 ASC NULLS FIRST,c_preferred_cust_flag#47 ASC NULLS FIRST,c_birth_day#36 ASC NULLS FIRST,c_birth_month#37 ASC NULLS FIRST,c_birth_year#38 ASC NULLS FIRST,c_birth_country#39 ASC NULLS FIRST,c_login#48 ASC NULLS FIRST,c_email_address#49 ASC NULLS FIRST,c_last_review_date#42 ASC NULLS FIRST,ctr_total_return#14 ASC NULLS FIRST], o [...]
-(54) TakeOrderedAndProject
-Input [13]: [c_customer_id#47, c_salutation#48, c_first_name#49, c_last_name#50, c_preferred_cust_flag#51, c_birth_day#40, c_birth_month#41, c_birth_year#42, c_birth_country#43, c_login#52, c_email_address#53, c_last_review_date#46, ctr_total_return#15]
-Arguments: 100, [c_customer_id#47 ASC NULLS FIRST, c_salutation#48 ASC NULLS FIRST, c_first_name#49 ASC NULLS FIRST, c_last_name#50 ASC NULLS FIRST, c_preferred_cust_flag#51 ASC NULLS FIRST, c_birth_day#40 ASC NULLS FIRST, c_birth_month#41 ASC NULLS FIRST, c_birth_year#42 ASC NULLS FIRST, c_birth_country#43 ASC NULLS FIRST, c_login#52 ASC NULLS FIRST, c_email_address#53 ASC NULLS FIRST, c_last_review_date#46 ASC NULLS FIRST, ctr_total_return#15 ASC NULLS FIRST], [c_customer_id#47, c_salu [...]
+(50) CometColumnarToRow [codegen id : 1]
+Input [13]: [c_customer_id#43, c_salutation#44, c_first_name#45, c_last_name#46, c_preferred_cust_flag#47, c_birth_day#36, c_birth_month#37, c_birth_year#38, c_birth_country#39, c_login#48, c_email_address#49, c_last_review_date#42, ctr_total_return#14]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = wr_returned_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (59)
-+- * CometColumnarToRow (58)
- +- CometProject (57)
- +- CometFilter (56)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (55)
+BroadcastExchange (55)
++- * CometColumnarToRow (54)
+ +- CometProject (53)
+ +- CometFilter (52)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (51)
-(55) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(51) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_year#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(56) CometFilter
+(52) CometFilter
Input [2]: [d_date_sk#6, d_year#7]
Condition : ((isnotnull(d_year#7) AND (d_year#7 = 2002)) AND isnotnull(d_date_sk#6))
-(57) CometProject
+(53) CometProject
Input [2]: [d_date_sk#6, d_year#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(58) CometColumnarToRow [codegen id : 1]
+(54) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(59) BroadcastExchange
+(55) BroadcastExchange
Input [1]: [d_date_sk#6]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=7]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-Subquery:2 Hosting operator id = 20 Hosting Expression = wr_returned_date_sk#19 IN dynamicpruning#5
+Subquery:2 Hosting operator id = 19 Hosting Expression = wr_returned_date_sk#18 IN dynamicpruning#5
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/extended.txt
index 38b08219c..7ebd69d34 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/extended.txt
@@ -1,69 +1,65 @@
-TakeOrderedAndProject
-+- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- Project
- : : +- BroadcastHashJoin
- : : :- Filter
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(wr_returning_customer_sk#1 AS ctr_customer_sk#2, ca_state#3 AS ctr_state#4, MakeDecimal(sum(UnscaledValue(wr_return_amt#5))#6,17,2) AS ctr_total_return#7)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometBroadcastHashJoin
- : : : : :- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
- : : : : : +- SubqueryBroadcast
- : : : : : +- BroadcastExchange
- : : : : : +- CometColumnarToRow
- : : : : : +- CometProject
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : +- CometBroadcastExchange
- : : : : +- CometProject
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : : +- BroadcastExchange
- : : +- Filter
- : : +- HashAggregate
- : : +- CometColumnarToRow
- : : +- CometColumnarExchange
- : : +- HashAggregate
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_state#8 AS ctr_state#9, MakeDecimal(sum(UnscaledValue(wr_return_amt#10))#6,17,2) AS ctr_total_return#11)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
- : : : : +- ReusedSubquery
- : : : +- CometBroadcastExchange
- : : : +- CometProject
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- CometBroadcastExchange
- : : +- CometProject
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : +- BroadcastExchange
- : +- CometColumnarToRow
- : +- CometProject
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
- +- BroadcastExchange
- +- CometColumnarToRow
+CometColumnarToRow
++- CometTakeOrderedAndProject
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometBroadcastHashJoin
+ : : : : :- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
+ : : : : : +- SubqueryBroadcast
+ : : : : : +- BroadcastExchange
+ : : : : : +- CometColumnarToRow
+ : : : : : +- CometProject
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometProject
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns
+ : : : : +- ReusedSubquery
+ : : : +- CometBroadcastExchange
+ : : : +- CometProject
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometProject
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : +- CometBroadcastExchange
+ : +- CometProject
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer
+ +- CometBroadcastExchange
+- CometProject
+- CometFilter
+- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
-Comet accelerated 42 out of 61 eligible operators (68%). Final plan contains 6 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 58 out of 61 eligible operators (95%). Final plan contains 2 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/simplified.txt
index 112d86a05..0227534fa 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q30/simplified.txt
@@ -1,15 +1,15 @@
-TakeOrderedAndProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
- WholeStageCodegen (5)
- Project [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
- BroadcastHashJoin [c_current_addr_sk,ca_address_sk]
- Project [ctr_total_return,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- BroadcastHashJoin [ctr_customer_sk,c_customer_sk]
- Project [ctr_customer_sk,ctr_total_return]
- BroadcastHashJoin [ctr_state,ctr_state,ctr_total_return,(avg(ctr_total_return) * 1.2)]
- Filter [ctr_total_return]
- HashAggregate [wr_returning_customer_sk,ca_state,sum] [sum(UnscaledValue(wr_return_amt)),ctr_customer_sk,ctr_state,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
+WholeStageCodegen (1)
+ CometColumnarToRow
+ InputAdapter
+ CometTakeOrderedAndProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
+ CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ctr_total_return]
+ CometBroadcastHashJoin [ctr_total_return,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date,ca_address_sk]
+ CometProject [ctr_total_return,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_total_return,c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometProject [ctr_customer_sk,ctr_total_return]
+ CometBroadcastHashJoin [ctr_customer_sk,ctr_state,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_state]
+ CometFilter [ctr_customer_sk,ctr_state,ctr_total_return]
+ CometHashAggregate [sum] [ctr_customer_sk,ctr_state,ctr_total_return,wr_returning_customer_sk,ca_state,sum(UnscaledValue(wr_return_amt))]
CometExchange [wr_returning_customer_sk,ca_state] #1
CometHashAggregate [wr_return_amt] [wr_returning_customer_sk,ca_state,sum]
CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
@@ -34,43 +34,28 @@ TakeOrderedAndProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_pre
CometProject [ca_state] [ca_address_sk,ca_state]
CometFilter [ca_address_sk,ca_state]
CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (2)
- Filter [(avg(ctr_total_return) * 1.2)]
- HashAggregate [ctr_state,sum,count] [avg(ctr_total_return),(avg(ctr_total_return) * 1.2),sum,count]
- CometColumnarToRow
- InputAdapter
- CometColumnarExchange [ctr_state] #6
- WholeStageCodegen (1)
- HashAggregate [ctr_state,ctr_total_return] [sum,count,sum,count]
- HashAggregate [wr_returning_customer_sk,ca_state,sum] [sum(UnscaledValue(wr_return_amt)),ctr_state,ctr_total_return,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [wr_returning_customer_sk,ca_state] #7
- CometHashAggregate [wr_return_amt] [wr_returning_customer_sk,ca_state,sum]
- CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
- CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
- CometProject [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt]
- CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk,d_date_sk]
- CometFilter [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #3
- ReusedExchange [ca_address_sk,ca_state] #4
- InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (3)
- CometColumnarToRow
- InputAdapter
- CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- CometFilter [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- InputAdapter
- BroadcastExchange #9
- WholeStageCodegen (4)
- CometColumnarToRow
- InputAdapter
- CometProject [ca_address_sk]
- CometFilter [ca_address_sk,ca_state]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_state] #5
+ CometFilter [(avg(ctr_total_return) * 1.2),ctr_state]
+ CometHashAggregate [sum,count] [(avg(ctr_total_return) * 1.2),ctr_state,avg(ctr_total_return)]
+ CometExchange [ctr_state] #6
+ CometHashAggregate [ctr_total_return] [ctr_state,sum,count]
+ CometHashAggregate [wr_returning_customer_sk,sum] [ctr_state,ctr_total_return,ca_state,sum(UnscaledValue(wr_return_amt))]
+ CometExchange [wr_returning_customer_sk,ca_state] #7
+ CometHashAggregate [wr_return_amt] [wr_returning_customer_sk,ca_state,sum]
+ CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
+ CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
+ CometProject [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt]
+ CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk,d_date_sk]
+ CometFilter [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.web_returns [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #3
+ ReusedExchange [ca_address_sk,ca_state] #4
+ CometBroadcastExchange [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date] #8
+ CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometFilter [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometBroadcastExchange [ca_address_sk] #9
+ CometProject [ca_address_sk]
+ CometFilter [ca_address_sk,ca_state]
+ CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/explain.txt
index 97da1fbdf..83d717e29 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/explain.txt
@@ -1,100 +1,94 @@
== Physical Plan ==
-* CometColumnarToRow (96)
-+- CometSort (95)
- +- CometColumnarExchange (94)
- +- * Project (93)
- +- * BroadcastHashJoin Inner BuildRight (92)
- :- * Project (78)
- : +- * BroadcastHashJoin Inner BuildRight (77)
- : :- * BroadcastHashJoin Inner BuildRight (63)
- : : :- * Project (49)
- : : : +- * BroadcastHashJoin Inner BuildRight (48)
- : : : :- * BroadcastHashJoin Inner BuildRight (32)
- : : : : :- * HashAggregate (16)
- : : : : : +- * CometColumnarToRow (15)
- : : : : : +- CometExchange (14)
- : : : : : +- CometHashAggregate (13)
- : : : : : +- CometProject (12)
- : : : : : +- CometBroadcastHashJoin (11)
- : : : : : :- CometProject (7)
- : : : : : : +- CometBroadcastHashJoin (6)
- : : : : : : :- CometFilter (2)
- : : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (1)
- : : : : : : +- CometBroadcastExchange (5)
- : : : : : : +- CometFilter (4)
- : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
- : : : : : +- CometBroadcastExchange (10)
- : : : : : +- CometFilter (9)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (8)
- : : : : +- BroadcastExchange (31)
- : : : : +- * HashAggregate (30)
- : : : : +- * CometColumnarToRow (29)
- : : : : +- CometExchange (28)
- : : : : +- CometHashAggregate (27)
- : : : : +- CometProject (26)
- : : : : +- CometBroadcastHashJoin (25)
- : : : : :- CometProject (23)
- : : : : : +- CometBroadcastHashJoin (22)
- : : : : : :- CometFilter (18)
- : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (17)
- : : : : : +- CometBroadcastExchange (21)
- : : : : : +- CometFilter (20)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (19)
- : : : : +- ReusedExchange (24)
- : : : +- BroadcastExchange (47)
- : : : +- * HashAggregate (46)
- : : : +- * CometColumnarToRow (45)
- : : : +- CometExchange (44)
- : : : +- CometHashAggregate (43)
- : : : +- CometProject (42)
- : : : +- CometBroadcastHashJoin (41)
- : : : :- CometProject (39)
- : : : : +- CometBroadcastHashJoin (38)
- : : : : :- CometFilter (34)
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (33)
- : : : : +- CometBroadcastExchange (37)
- : : : : +- CometFilter (36)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (35)
- : : : +- ReusedExchange (40)
- : : +- BroadcastExchange (62)
- : : +- * HashAggregate (61)
- : : +- * CometColumnarToRow (60)
- : : +- CometExchange (59)
- : : +- CometHashAggregate (58)
- : : +- CometProject (57)
- : : +- CometBroadcastHashJoin (56)
- : : :- CometProject (54)
- : : : +- CometBroadcastHashJoin (53)
- : : : :- CometFilter (51)
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (50)
- : : : +- ReusedExchange (52)
- : : +- ReusedExchange (55)
- : +- BroadcastExchange (76)
- : +- * HashAggregate (75)
- : +- * CometColumnarToRow (74)
- : +- CometExchange (73)
- : +- CometHashAggregate (72)
- : +- CometProject (71)
- : +- CometBroadcastHashJoin (70)
- : :- CometProject (68)
- : : +- CometBroadcastHashJoin (67)
- : : :- CometFilter (65)
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (64)
- : : +- ReusedExchange (66)
- : +- ReusedExchange (69)
- +- BroadcastExchange (91)
- +- * HashAggregate (90)
- +- * CometColumnarToRow (89)
- +- CometExchange (88)
- +- CometHashAggregate (87)
- +- CometProject (86)
- +- CometBroadcastHashJoin (85)
- :- CometProject (83)
- : +- CometBroadcastHashJoin (82)
- : :- CometFilter (80)
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (79)
- : +- ReusedExchange (81)
- +- ReusedExchange (84)
+* CometColumnarToRow (90)
++- CometSort (89)
+ +- CometExchange (88)
+ +- CometProject (87)
+ +- CometBroadcastHashJoin (86)
+ :- CometProject (73)
+ : +- CometBroadcastHashJoin (72)
+ : :- CometBroadcastHashJoin (59)
+ : : :- CometProject (46)
+ : : : +- CometBroadcastHashJoin (45)
+ : : : :- CometBroadcastHashJoin (30)
+ : : : : :- CometHashAggregate (15)
+ : : : : : +- CometExchange (14)
+ : : : : : +- CometHashAggregate (13)
+ : : : : : +- CometProject (12)
+ : : : : : +- CometBroadcastHashJoin (11)
+ : : : : : :- CometProject (7)
+ : : : : : : +- CometBroadcastHashJoin (6)
+ : : : : : : :- CometFilter (2)
+ : : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (1)
+ : : : : : : +- CometBroadcastExchange (5)
+ : : : : : : +- CometFilter (4)
+ : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (3)
+ : : : : : +- CometBroadcastExchange (10)
+ : : : : : +- CometFilter (9)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address (8)
+ : : : : +- CometBroadcastExchange (29)
+ : : : : +- CometHashAggregate (28)
+ : : : : +- CometExchange (27)
+ : : : : +- CometHashAggregate (26)
+ : : : : +- CometProject (25)
+ : : : : +- CometBroadcastHashJoin (24)
+ : : : : :- CometProject (22)
+ : : : : : +- CometBroadcastHashJoin (21)
+ : : : : : :- CometFilter (17)
+ : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (16)
+ : : : : : +- CometBroadcastExchange (20)
+ : : : : : +- CometFilter (19)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (18)
+ : : : : +- ReusedExchange (23)
+ : : : +- CometBroadcastExchange (44)
+ : : : +- CometHashAggregate (43)
+ : : : +- CometExchange (42)
+ : : : +- CometHashAggregate (41)
+ : : : +- CometProject (40)
+ : : : +- CometBroadcastHashJoin (39)
+ : : : :- CometProject (37)
+ : : : : +- CometBroadcastHashJoin (36)
+ : : : : :- CometFilter (32)
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales (31)
+ : : : : +- CometBroadcastExchange (35)
+ : : : : +- CometFilter (34)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (33)
+ : : : +- ReusedExchange (38)
+ : : +- CometBroadcastExchange (58)
+ : : +- CometHashAggregate (57)
+ : : +- CometExchange (56)
+ : : +- CometHashAggregate (55)
+ : : +- CometProject (54)
+ : : +- CometBroadcastHashJoin (53)
+ : : :- CometProject (51)
+ : : : +- CometBroadcastHashJoin (50)
+ : : : :- CometFilter (48)
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (47)
+ : : : +- ReusedExchange (49)
+ : : +- ReusedExchange (52)
+ : +- CometBroadcastExchange (71)
+ : +- CometHashAggregate (70)
+ : +- CometExchange (69)
+ : +- CometHashAggregate (68)
+ : +- CometProject (67)
+ : +- CometBroadcastHashJoin (66)
+ : :- CometProject (64)
+ : : +- CometBroadcastHashJoin (63)
+ : : :- CometFilter (61)
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (60)
+ : : +- ReusedExchange (62)
+ : +- ReusedExchange (65)
+ +- CometBroadcastExchange (85)
+ +- CometHashAggregate (84)
+ +- CometExchange (83)
+ +- CometHashAggregate (82)
+ +- CometProject (81)
+ +- CometBroadcastHashJoin (80)
+ :- CometProject (78)
+ : +- CometBroadcastHashJoin (77)
+ : :- CometFilter (75)
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales (74)
+ : +- ReusedExchange (76)
+ +- ReusedExchange (79)
(1) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
@@ -166,469 +160,434 @@ Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#2))]
Input [4]: [ca_county#9, d_qoy#7, d_year#6, sum#10]
Arguments: hashpartitioning(ca_county#9, d_qoy#7, d_year#6, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(15) CometColumnarToRow [codegen id : 6]
-Input [4]: [ca_county#9, d_qoy#7, d_year#6, sum#10]
-
-(16) HashAggregate [codegen id : 6]
+(15) CometHashAggregate
Input [4]: [ca_county#9, d_qoy#7, d_year#6, sum#10]
Keys [3]: [ca_county#9, d_qoy#7, d_year#6]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#2))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#2))#11]
-Results [3]: [ca_county#9, d_year#6, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#2))#11,17,2) AS store_sales#12]
-(17) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
-Output [3]: [ss_addr_sk#13, ss_ext_sales_price#14, ss_sold_date_sk#15]
+(16) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+Output [3]: [ss_addr_sk#11, ss_ext_sales_price#12, ss_sold_date_sk#13]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#15), dynamicpruningexpression(ss_sold_date_sk#15 IN dynamicpruning#16)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#13), dynamicpruningexpression(ss_sold_date_sk#13 IN dynamicpruning#14)]
PushedFilters: [IsNotNull(ss_addr_sk)]
ReadSchema: struct<ss_addr_sk:int,ss_ext_sales_price:decimal(7,2)>
-(18) CometFilter
-Input [3]: [ss_addr_sk#13, ss_ext_sales_price#14, ss_sold_date_sk#15]
-Condition : isnotnull(ss_addr_sk#13)
+(17) CometFilter
+Input [3]: [ss_addr_sk#11, ss_ext_sales_price#12, ss_sold_date_sk#13]
+Condition : isnotnull(ss_addr_sk#11)
-(19) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#17, d_year#18, d_qoy#19]
+(18) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#15, d_year#16, d_qoy#17]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,2), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(20) CometFilter
-Input [3]: [d_date_sk#17, d_year#18, d_qoy#19]
-Condition : ((((isnotnull(d_qoy#19) AND isnotnull(d_year#18)) AND (d_qoy#19 = 2)) AND (d_year#18 = 2000)) AND isnotnull(d_date_sk#17))
-
-(21) CometBroadcastExchange
-Input [3]: [d_date_sk#17, d_year#18, d_qoy#19]
-Arguments: [d_date_sk#17, d_year#18, d_qoy#19]
-
-(22) CometBroadcastHashJoin
-Left output [3]: [ss_addr_sk#13, ss_ext_sales_price#14, ss_sold_date_sk#15]
-Right output [3]: [d_date_sk#17, d_year#18, d_qoy#19]
-Arguments: [ss_sold_date_sk#15], [d_date_sk#17], Inner, BuildRight
-
-(23) CometProject
-Input [6]: [ss_addr_sk#13, ss_ext_sales_price#14, ss_sold_date_sk#15, d_date_sk#17, d_year#18, d_qoy#19]
-Arguments: [ss_addr_sk#13, ss_ext_sales_price#14, d_year#18, d_qoy#19], [ss_addr_sk#13, ss_ext_sales_price#14, d_year#18, d_qoy#19]
-
-(24) ReusedExchange [Reuses operator id: 10]
-Output [2]: [ca_address_sk#20, ca_county#21]
-
-(25) CometBroadcastHashJoin
-Left output [4]: [ss_addr_sk#13, ss_ext_sales_price#14, d_year#18, d_qoy#19]
-Right output [2]: [ca_address_sk#20, ca_county#21]
-Arguments: [ss_addr_sk#13], [ca_address_sk#20], Inner, BuildRight
-
-(26) CometProject
-Input [6]: [ss_addr_sk#13, ss_ext_sales_price#14, d_year#18, d_qoy#19, ca_address_sk#20, ca_county#21]
-Arguments: [ss_ext_sales_price#14, d_year#18, d_qoy#19, ca_county#21], [ss_ext_sales_price#14, d_year#18, d_qoy#19, ca_county#21]
-
-(27) CometHashAggregate
-Input [4]: [ss_ext_sales_price#14, d_year#18, d_qoy#19, ca_county#21]
-Keys [3]: [ca_county#21, d_qoy#19, d_year#18]
-Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#14))]
-
-(28) CometExchange
-Input [4]: [ca_county#21, d_qoy#19, d_year#18, sum#22]
-Arguments: hashpartitioning(ca_county#21, d_qoy#19, d_year#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(29) CometColumnarToRow [codegen id : 1]
-Input [4]: [ca_county#21, d_qoy#19, d_year#18, sum#22]
-
-(30) HashAggregate [codegen id : 1]
-Input [4]: [ca_county#21, d_qoy#19, d_year#18, sum#22]
-Keys [3]: [ca_county#21, d_qoy#19, d_year#18]
-Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#14))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#14))#11]
-Results [2]: [ca_county#21, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#14))#11,17,2) AS store_sales#23]
-
-(31) BroadcastExchange
-Input [2]: [ca_county#21, store_sales#23]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=3]
-
-(32) BroadcastHashJoin [codegen id : 6]
-Left keys [1]: [ca_county#9]
-Right keys [1]: [ca_county#21]
-Join type: Inner
-Join condition: None
-
-(33) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
-Output [3]: [ss_addr_sk#24, ss_ext_sales_price#25, ss_sold_date_sk#26]
+(19) CometFilter
+Input [3]: [d_date_sk#15, d_year#16, d_qoy#17]
+Condition : ((((isnotnull(d_qoy#17) AND isnotnull(d_year#16)) AND (d_qoy#17 = 2)) AND (d_year#16 = 2000)) AND isnotnull(d_date_sk#15))
+
+(20) CometBroadcastExchange
+Input [3]: [d_date_sk#15, d_year#16, d_qoy#17]
+Arguments: [d_date_sk#15, d_year#16, d_qoy#17]
+
+(21) CometBroadcastHashJoin
+Left output [3]: [ss_addr_sk#11, ss_ext_sales_price#12, ss_sold_date_sk#13]
+Right output [3]: [d_date_sk#15, d_year#16, d_qoy#17]
+Arguments: [ss_sold_date_sk#13], [d_date_sk#15], Inner, BuildRight
+
+(22) CometProject
+Input [6]: [ss_addr_sk#11, ss_ext_sales_price#12, ss_sold_date_sk#13, d_date_sk#15, d_year#16, d_qoy#17]
+Arguments: [ss_addr_sk#11, ss_ext_sales_price#12, d_year#16, d_qoy#17], [ss_addr_sk#11, ss_ext_sales_price#12, d_year#16, d_qoy#17]
+
+(23) ReusedExchange [Reuses operator id: 10]
+Output [2]: [ca_address_sk#18, ca_county#19]
+
+(24) CometBroadcastHashJoin
+Left output [4]: [ss_addr_sk#11, ss_ext_sales_price#12, d_year#16, d_qoy#17]
+Right output [2]: [ca_address_sk#18, ca_county#19]
+Arguments: [ss_addr_sk#11], [ca_address_sk#18], Inner, BuildRight
+
+(25) CometProject
+Input [6]: [ss_addr_sk#11, ss_ext_sales_price#12, d_year#16, d_qoy#17, ca_address_sk#18, ca_county#19]
+Arguments: [ss_ext_sales_price#12, d_year#16, d_qoy#17, ca_county#19], [ss_ext_sales_price#12, d_year#16, d_qoy#17, ca_county#19]
+
+(26) CometHashAggregate
+Input [4]: [ss_ext_sales_price#12, d_year#16, d_qoy#17, ca_county#19]
+Keys [3]: [ca_county#19, d_qoy#17, d_year#16]
+Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#12))]
+
+(27) CometExchange
+Input [4]: [ca_county#19, d_qoy#17, d_year#16, sum#20]
+Arguments: hashpartitioning(ca_county#19, d_qoy#17, d_year#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(28) CometHashAggregate
+Input [4]: [ca_county#19, d_qoy#17, d_year#16, sum#20]
+Keys [3]: [ca_county#19, d_qoy#17, d_year#16]
+Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#12))]
+
+(29) CometBroadcastExchange
+Input [2]: [ca_county#19, store_sales#21]
+Arguments: [ca_county#19, store_sales#21]
+
+(30) CometBroadcastHashJoin
+Left output [3]: [ca_county#9, d_year#6, store_sales#22]
+Right output [2]: [ca_county#19, store_sales#21]
+Arguments: [ca_county#9], [ca_county#19], Inner, BuildRight
+
+(31) CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+Output [3]: [ss_addr_sk#23, ss_ext_sales_price#24, ss_sold_date_sk#25]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#26), dynamicpruningexpression(ss_sold_date_sk#26 IN dynamicpruning#27)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#25), dynamicpruningexpression(ss_sold_date_sk#25 IN dynamicpruning#26)]
PushedFilters: [IsNotNull(ss_addr_sk)]
ReadSchema: struct<ss_addr_sk:int,ss_ext_sales_price:decimal(7,2)>
-(34) CometFilter
-Input [3]: [ss_addr_sk#24, ss_ext_sales_price#25, ss_sold_date_sk#26]
-Condition : isnotnull(ss_addr_sk#24)
+(32) CometFilter
+Input [3]: [ss_addr_sk#23, ss_ext_sales_price#24, ss_sold_date_sk#25]
+Condition : isnotnull(ss_addr_sk#23)
-(35) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#28, d_year#29, d_qoy#30]
+(33) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#27, d_year#28, d_qoy#29]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,3), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(36) CometFilter
-Input [3]: [d_date_sk#28, d_year#29, d_qoy#30]
-Condition : ((((isnotnull(d_qoy#30) AND isnotnull(d_year#29)) AND (d_qoy#30 = 3)) AND (d_year#29 = 2000)) AND isnotnull(d_date_sk#28))
+(34) CometFilter
+Input [3]: [d_date_sk#27, d_year#28, d_qoy#29]
+Condition : ((((isnotnull(d_qoy#29) AND isnotnull(d_year#28)) AND (d_qoy#29 = 3)) AND (d_year#28 = 2000)) AND isnotnull(d_date_sk#27))
+
+(35) CometBroadcastExchange
+Input [3]: [d_date_sk#27, d_year#28, d_qoy#29]
+Arguments: [d_date_sk#27, d_year#28, d_qoy#29]
+
+(36) CometBroadcastHashJoin
+Left output [3]: [ss_addr_sk#23, ss_ext_sales_price#24, ss_sold_date_sk#25]
+Right output [3]: [d_date_sk#27, d_year#28, d_qoy#29]
+Arguments: [ss_sold_date_sk#25], [d_date_sk#27], Inner, BuildRight
-(37) CometBroadcastExchange
-Input [3]: [d_date_sk#28, d_year#29, d_qoy#30]
-Arguments: [d_date_sk#28, d_year#29, d_qoy#30]
+(37) CometProject
+Input [6]: [ss_addr_sk#23, ss_ext_sales_price#24, ss_sold_date_sk#25, d_date_sk#27, d_year#28, d_qoy#29]
+Arguments: [ss_addr_sk#23, ss_ext_sales_price#24, d_year#28, d_qoy#29], [ss_addr_sk#23, ss_ext_sales_price#24, d_year#28, d_qoy#29]
-(38) CometBroadcastHashJoin
-Left output [3]: [ss_addr_sk#24, ss_ext_sales_price#25, ss_sold_date_sk#26]
-Right output [3]: [d_date_sk#28, d_year#29, d_qoy#30]
-Arguments: [ss_sold_date_sk#26], [d_date_sk#28], Inner, BuildRight
+(38) ReusedExchange [Reuses operator id: 10]
+Output [2]: [ca_address_sk#30, ca_county#31]
-(39) CometProject
-Input [6]: [ss_addr_sk#24, ss_ext_sales_price#25, ss_sold_date_sk#26, d_date_sk#28, d_year#29, d_qoy#30]
-Arguments: [ss_addr_sk#24, ss_ext_sales_price#25, d_year#29, d_qoy#30], [ss_addr_sk#24, ss_ext_sales_price#25, d_year#29, d_qoy#30]
+(39) CometBroadcastHashJoin
+Left output [4]: [ss_addr_sk#23, ss_ext_sales_price#24, d_year#28, d_qoy#29]
+Right output [2]: [ca_address_sk#30, ca_county#31]
+Arguments: [ss_addr_sk#23], [ca_address_sk#30], Inner, BuildRight
-(40) ReusedExchange [Reuses operator id: 10]
-Output [2]: [ca_address_sk#31, ca_county#32]
+(40) CometProject
+Input [6]: [ss_addr_sk#23, ss_ext_sales_price#24, d_year#28, d_qoy#29, ca_address_sk#30, ca_county#31]
+Arguments: [ss_ext_sales_price#24, d_year#28, d_qoy#29, ca_county#31], [ss_ext_sales_price#24, d_year#28, d_qoy#29, ca_county#31]
-(41) CometBroadcastHashJoin
-Left output [4]: [ss_addr_sk#24, ss_ext_sales_price#25, d_year#29, d_qoy#30]
-Right output [2]: [ca_address_sk#31, ca_county#32]
-Arguments: [ss_addr_sk#24], [ca_address_sk#31], Inner, BuildRight
+(41) CometHashAggregate
+Input [4]: [ss_ext_sales_price#24, d_year#28, d_qoy#29, ca_county#31]
+Keys [3]: [ca_county#31, d_qoy#29, d_year#28]
+Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#24))]
-(42) CometProject
-Input [6]: [ss_addr_sk#24, ss_ext_sales_price#25, d_year#29, d_qoy#30, ca_address_sk#31, ca_county#32]
-Arguments: [ss_ext_sales_price#25, d_year#29, d_qoy#30, ca_county#32], [ss_ext_sales_price#25, d_year#29, d_qoy#30, ca_county#32]
+(42) CometExchange
+Input [4]: [ca_county#31, d_qoy#29, d_year#28, sum#32]
+Arguments: hashpartitioning(ca_county#31, d_qoy#29, d_year#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(43) CometHashAggregate
-Input [4]: [ss_ext_sales_price#25, d_year#29, d_qoy#30, ca_county#32]
-Keys [3]: [ca_county#32, d_qoy#30, d_year#29]
-Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#25))]
-
-(44) CometExchange
-Input [4]: [ca_county#32, d_qoy#30, d_year#29, sum#33]
-Arguments: hashpartitioning(ca_county#32, d_qoy#30, d_year#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
-
-(45) CometColumnarToRow [codegen id : 2]
-Input [4]: [ca_county#32, d_qoy#30, d_year#29, sum#33]
-
-(46) HashAggregate [codegen id : 2]
-Input [4]: [ca_county#32, d_qoy#30, d_year#29, sum#33]
-Keys [3]: [ca_county#32, d_qoy#30, d_year#29]
-Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#25))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_ext_sales_price#25))#11]
-Results [2]: [ca_county#32, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#25))#11,17,2) AS store_sales#34]
-
-(47) BroadcastExchange
-Input [2]: [ca_county#32, store_sales#34]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=5]
-
-(48) BroadcastHashJoin [codegen id : 6]
-Left keys [1]: [ca_county#21]
-Right keys [1]: [ca_county#32]
-Join type: Inner
-Join condition: None
-
-(49) Project [codegen id : 6]
-Output [5]: [ca_county#9, d_year#6, store_sales#12, store_sales#23, store_sales#34]
-Input [7]: [ca_county#9, d_year#6, store_sales#12, ca_county#21, store_sales#23, ca_county#32, store_sales#34]
-
-(50) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [3]: [ws_bill_addr_sk#35, ws_ext_sales_price#36, ws_sold_date_sk#37]
+Input [4]: [ca_county#31, d_qoy#29, d_year#28, sum#32]
+Keys [3]: [ca_county#31, d_qoy#29, d_year#28]
+Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#24))]
+
+(44) CometBroadcastExchange
+Input [2]: [ca_county#31, store_sales#33]
+Arguments: [ca_county#31, store_sales#33]
+
+(45) CometBroadcastHashJoin
+Left output [5]: [ca_county#9, d_year#6, store_sales#22, ca_county#19, store_sales#21]
+Right output [2]: [ca_county#31, store_sales#33]
+Arguments: [ca_county#19], [ca_county#31], Inner, BuildRight
+
+(46) CometProject
+Input [7]: [ca_county#9, d_year#6, store_sales#22, ca_county#19, store_sales#21, ca_county#31, store_sales#33]
+Arguments: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33], [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33]
+
+(47) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [3]: [ws_bill_addr_sk#34, ws_ext_sales_price#35, ws_sold_date_sk#36]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#37), dynamicpruningexpression(ws_sold_date_sk#37 IN dynamicpruning#38)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#36), dynamicpruningexpression(ws_sold_date_sk#36 IN dynamicpruning#37)]
PushedFilters: [IsNotNull(ws_bill_addr_sk)]
ReadSchema: struct<ws_bill_addr_sk:int,ws_ext_sales_price:decimal(7,2)>
-(51) CometFilter
-Input [3]: [ws_bill_addr_sk#35, ws_ext_sales_price#36, ws_sold_date_sk#37]
-Condition : isnotnull(ws_bill_addr_sk#35)
+(48) CometFilter
+Input [3]: [ws_bill_addr_sk#34, ws_ext_sales_price#35, ws_sold_date_sk#36]
+Condition : isnotnull(ws_bill_addr_sk#34)
+
+(49) ReusedExchange [Reuses operator id: 5]
+Output [3]: [d_date_sk#38, d_year#39, d_qoy#40]
+
+(50) CometBroadcastHashJoin
+Left output [3]: [ws_bill_addr_sk#34, ws_ext_sales_price#35, ws_sold_date_sk#36]
+Right output [3]: [d_date_sk#38, d_year#39, d_qoy#40]
+Arguments: [ws_sold_date_sk#36], [d_date_sk#38], Inner, BuildRight
-(52) ReusedExchange [Reuses operator id: 5]
-Output [3]: [d_date_sk#39, d_year#40, d_qoy#41]
+(51) CometProject
+Input [6]: [ws_bill_addr_sk#34, ws_ext_sales_price#35, ws_sold_date_sk#36, d_date_sk#38, d_year#39, d_qoy#40]
+Arguments: [ws_bill_addr_sk#34, ws_ext_sales_price#35, d_year#39, d_qoy#40], [ws_bill_addr_sk#34, ws_ext_sales_price#35, d_year#39, d_qoy#40]
+
+(52) ReusedExchange [Reuses operator id: 10]
+Output [2]: [ca_address_sk#41, ca_county#42]
(53) CometBroadcastHashJoin
-Left output [3]: [ws_bill_addr_sk#35, ws_ext_sales_price#36, ws_sold_date_sk#37]
-Right output [3]: [d_date_sk#39, d_year#40, d_qoy#41]
-Arguments: [ws_sold_date_sk#37], [d_date_sk#39], Inner, BuildRight
+Left output [4]: [ws_bill_addr_sk#34, ws_ext_sales_price#35, d_year#39, d_qoy#40]
+Right output [2]: [ca_address_sk#41, ca_county#42]
+Arguments: [ws_bill_addr_sk#34], [ca_address_sk#41], Inner, BuildRight
(54) CometProject
-Input [6]: [ws_bill_addr_sk#35, ws_ext_sales_price#36, ws_sold_date_sk#37, d_date_sk#39, d_year#40, d_qoy#41]
-Arguments: [ws_bill_addr_sk#35, ws_ext_sales_price#36, d_year#40, d_qoy#41], [ws_bill_addr_sk#35, ws_ext_sales_price#36, d_year#40, d_qoy#41]
-
-(55) ReusedExchange [Reuses operator id: 10]
-Output [2]: [ca_address_sk#42, ca_county#43]
-
-(56) CometBroadcastHashJoin
-Left output [4]: [ws_bill_addr_sk#35, ws_ext_sales_price#36, d_year#40, d_qoy#41]
-Right output [2]: [ca_address_sk#42, ca_county#43]
-Arguments: [ws_bill_addr_sk#35], [ca_address_sk#42], Inner, BuildRight
-
-(57) CometProject
-Input [6]: [ws_bill_addr_sk#35, ws_ext_sales_price#36, d_year#40, d_qoy#41, ca_address_sk#42, ca_county#43]
-Arguments: [ws_ext_sales_price#36, d_year#40, d_qoy#41, ca_county#43], [ws_ext_sales_price#36, d_year#40, d_qoy#41, ca_county#43]
-
-(58) CometHashAggregate
-Input [4]: [ws_ext_sales_price#36, d_year#40, d_qoy#41, ca_county#43]
-Keys [3]: [ca_county#43, d_qoy#41, d_year#40]
-Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#36))]
-
-(59) CometExchange
-Input [4]: [ca_county#43, d_qoy#41, d_year#40, sum#44]
-Arguments: hashpartitioning(ca_county#43, d_qoy#41, d_year#40, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
-
-(60) CometColumnarToRow [codegen id : 3]
-Input [4]: [ca_county#43, d_qoy#41, d_year#40, sum#44]
-
-(61) HashAggregate [codegen id : 3]
-Input [4]: [ca_county#43, d_qoy#41, d_year#40, sum#44]
-Keys [3]: [ca_county#43, d_qoy#41, d_year#40]
-Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#36))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ws_ext_sales_price#36))#45]
-Results [2]: [ca_county#43, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#36))#45,17,2) AS web_sales#46]
-
-(62) BroadcastExchange
-Input [2]: [ca_county#43, web_sales#46]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=7]
-
-(63) BroadcastHashJoin [codegen id : 6]
-Left keys [1]: [ca_county#9]
-Right keys [1]: [ca_county#43]
-Join type: Inner
-Join condition: None
-
-(64) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [3]: [ws_bill_addr_sk#47, ws_ext_sales_price#48, ws_sold_date_sk#49]
+Input [6]: [ws_bill_addr_sk#34, ws_ext_sales_price#35, d_year#39, d_qoy#40, ca_address_sk#41, ca_county#42]
+Arguments: [ws_ext_sales_price#35, d_year#39, d_qoy#40, ca_county#42], [ws_ext_sales_price#35, d_year#39, d_qoy#40, ca_county#42]
+
+(55) CometHashAggregate
+Input [4]: [ws_ext_sales_price#35, d_year#39, d_qoy#40, ca_county#42]
+Keys [3]: [ca_county#42, d_qoy#40, d_year#39]
+Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#35))]
+
+(56) CometExchange
+Input [4]: [ca_county#42, d_qoy#40, d_year#39, sum#43]
+Arguments: hashpartitioning(ca_county#42, d_qoy#40, d_year#39, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
+
+(57) CometHashAggregate
+Input [4]: [ca_county#42, d_qoy#40, d_year#39, sum#43]
+Keys [3]: [ca_county#42, d_qoy#40, d_year#39]
+Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#35))]
+
+(58) CometBroadcastExchange
+Input [2]: [ca_county#42, web_sales#44]
+Arguments: [ca_county#42, web_sales#44]
+
+(59) CometBroadcastHashJoin
+Left output [5]: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33]
+Right output [2]: [ca_county#42, web_sales#44]
+Arguments: [ca_county#9], [ca_county#42], Inner, BuildRight
+
+(60) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [3]: [ws_bill_addr_sk#45, ws_ext_sales_price#46, ws_sold_date_sk#47]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#49), dynamicpruningexpression(ws_sold_date_sk#49 IN dynamicpruning#50)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#47), dynamicpruningexpression(ws_sold_date_sk#47 IN dynamicpruning#48)]
PushedFilters: [IsNotNull(ws_bill_addr_sk)]
ReadSchema: struct<ws_bill_addr_sk:int,ws_ext_sales_price:decimal(7,2)>
-(65) CometFilter
-Input [3]: [ws_bill_addr_sk#47, ws_ext_sales_price#48, ws_sold_date_sk#49]
-Condition : isnotnull(ws_bill_addr_sk#47)
-
-(66) ReusedExchange [Reuses operator id: 21]
-Output [3]: [d_date_sk#51, d_year#52, d_qoy#53]
-
-(67) CometBroadcastHashJoin
-Left output [3]: [ws_bill_addr_sk#47, ws_ext_sales_price#48, ws_sold_date_sk#49]
-Right output [3]: [d_date_sk#51, d_year#52, d_qoy#53]
-Arguments: [ws_sold_date_sk#49], [d_date_sk#51], Inner, BuildRight
-
-(68) CometProject
-Input [6]: [ws_bill_addr_sk#47, ws_ext_sales_price#48, ws_sold_date_sk#49, d_date_sk#51, d_year#52, d_qoy#53]
-Arguments: [ws_bill_addr_sk#47, ws_ext_sales_price#48, d_year#52, d_qoy#53], [ws_bill_addr_sk#47, ws_ext_sales_price#48, d_year#52, d_qoy#53]
-
-(69) ReusedExchange [Reuses operator id: 10]
-Output [2]: [ca_address_sk#54, ca_county#55]
-
-(70) CometBroadcastHashJoin
-Left output [4]: [ws_bill_addr_sk#47, ws_ext_sales_price#48, d_year#52, d_qoy#53]
-Right output [2]: [ca_address_sk#54, ca_county#55]
-Arguments: [ws_bill_addr_sk#47], [ca_address_sk#54], Inner, BuildRight
-
-(71) CometProject
-Input [6]: [ws_bill_addr_sk#47, ws_ext_sales_price#48, d_year#52, d_qoy#53, ca_address_sk#54, ca_county#55]
-Arguments: [ws_ext_sales_price#48, d_year#52, d_qoy#53, ca_county#55], [ws_ext_sales_price#48, d_year#52, d_qoy#53, ca_county#55]
-
-(72) CometHashAggregate
-Input [4]: [ws_ext_sales_price#48, d_year#52, d_qoy#53, ca_county#55]
-Keys [3]: [ca_county#55, d_qoy#53, d_year#52]
-Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#48))]
-
-(73) CometExchange
-Input [4]: [ca_county#55, d_qoy#53, d_year#52, sum#56]
-Arguments: hashpartitioning(ca_county#55, d_qoy#53, d_year#52, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=8]
-
-(74) CometColumnarToRow [codegen id : 4]
-Input [4]: [ca_county#55, d_qoy#53, d_year#52, sum#56]
-
-(75) HashAggregate [codegen id : 4]
-Input [4]: [ca_county#55, d_qoy#53, d_year#52, sum#56]
-Keys [3]: [ca_county#55, d_qoy#53, d_year#52]
-Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#48))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ws_ext_sales_price#48))#45]
-Results [2]: [ca_county#55, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#48))#45,17,2) AS web_sales#57]
-
-(76) BroadcastExchange
-Input [2]: [ca_county#55, web_sales#57]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=9]
-
-(77) BroadcastHashJoin [codegen id : 6]
-Left keys [1]: [ca_county#43]
-Right keys [1]: [ca_county#55]
-Join type: Inner
-Join condition: (CASE WHEN (web_sales#46 > 0.00) THEN (web_sales#57 / web_sales#46) END > CASE WHEN (store_sales#12 > 0.00) THEN (store_sales#23 / store_sales#12) END)
-
-(78) Project [codegen id : 6]
-Output [8]: [ca_county#9, d_year#6, store_sales#12, store_sales#23, store_sales#34, ca_county#43, web_sales#46, web_sales#57]
-Input [9]: [ca_county#9, d_year#6, store_sales#12, store_sales#23, store_sales#34, ca_county#43, web_sales#46, ca_county#55, web_sales#57]
-
-(79) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
-Output [3]: [ws_bill_addr_sk#58, ws_ext_sales_price#59, ws_sold_date_sk#60]
+(61) CometFilter
+Input [3]: [ws_bill_addr_sk#45, ws_ext_sales_price#46, ws_sold_date_sk#47]
+Condition : isnotnull(ws_bill_addr_sk#45)
+
+(62) ReusedExchange [Reuses operator id: 20]
+Output [3]: [d_date_sk#49, d_year#50, d_qoy#51]
+
+(63) CometBroadcastHashJoin
+Left output [3]: [ws_bill_addr_sk#45, ws_ext_sales_price#46, ws_sold_date_sk#47]
+Right output [3]: [d_date_sk#49, d_year#50, d_qoy#51]
+Arguments: [ws_sold_date_sk#47], [d_date_sk#49], Inner, BuildRight
+
+(64) CometProject
+Input [6]: [ws_bill_addr_sk#45, ws_ext_sales_price#46, ws_sold_date_sk#47, d_date_sk#49, d_year#50, d_qoy#51]
+Arguments: [ws_bill_addr_sk#45, ws_ext_sales_price#46, d_year#50, d_qoy#51], [ws_bill_addr_sk#45, ws_ext_sales_price#46, d_year#50, d_qoy#51]
+
+(65) ReusedExchange [Reuses operator id: 10]
+Output [2]: [ca_address_sk#52, ca_county#53]
+
+(66) CometBroadcastHashJoin
+Left output [4]: [ws_bill_addr_sk#45, ws_ext_sales_price#46, d_year#50, d_qoy#51]
+Right output [2]: [ca_address_sk#52, ca_county#53]
+Arguments: [ws_bill_addr_sk#45], [ca_address_sk#52], Inner, BuildRight
+
+(67) CometProject
+Input [6]: [ws_bill_addr_sk#45, ws_ext_sales_price#46, d_year#50, d_qoy#51, ca_address_sk#52, ca_county#53]
+Arguments: [ws_ext_sales_price#46, d_year#50, d_qoy#51, ca_county#53], [ws_ext_sales_price#46, d_year#50, d_qoy#51, ca_county#53]
+
+(68) CometHashAggregate
+Input [4]: [ws_ext_sales_price#46, d_year#50, d_qoy#51, ca_county#53]
+Keys [3]: [ca_county#53, d_qoy#51, d_year#50]
+Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#46))]
+
+(69) CometExchange
+Input [4]: [ca_county#53, d_qoy#51, d_year#50, sum#54]
+Arguments: hashpartitioning(ca_county#53, d_qoy#51, d_year#50, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
+
+(70) CometHashAggregate
+Input [4]: [ca_county#53, d_qoy#51, d_year#50, sum#54]
+Keys [3]: [ca_county#53, d_qoy#51, d_year#50]
+Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#46))]
+
+(71) CometBroadcastExchange
+Input [2]: [ca_county#53, web_sales#55]
+Arguments: [ca_county#53, web_sales#55]
+
+(72) CometBroadcastHashJoin
+Left output [7]: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33, ca_county#42, web_sales#44]
+Right output [2]: [ca_county#53, web_sales#55]
+Arguments: [ca_county#42], [ca_county#53], Inner, (CASE WHEN (web_sales#44 > 0.00) THEN (web_sales#55 / web_sales#44) END > CASE WHEN (store_sales#22 > 0.00) THEN (store_sales#21 / store_sales#22) END), BuildRight
+
+(73) CometProject
+Input [9]: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33, ca_county#42, web_sales#44, ca_county#53, web_sales#55]
+Arguments: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33, ca_county#42, web_sales#44, web_sales#55], [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33, ca_county#42, web_sales#44, web_sales#55]
+
+(74) CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+Output [3]: [ws_bill_addr_sk#56, ws_ext_sales_price#57, ws_sold_date_sk#58]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#60), dynamicpruningexpression(ws_sold_date_sk#60 IN dynamicpruning#61)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#58), dynamicpruningexpression(ws_sold_date_sk#58 IN dynamicpruning#59)]
PushedFilters: [IsNotNull(ws_bill_addr_sk)]
ReadSchema: struct<ws_bill_addr_sk:int,ws_ext_sales_price:decimal(7,2)>
-(80) CometFilter
-Input [3]: [ws_bill_addr_sk#58, ws_ext_sales_price#59, ws_sold_date_sk#60]
-Condition : isnotnull(ws_bill_addr_sk#58)
+(75) CometFilter
+Input [3]: [ws_bill_addr_sk#56, ws_ext_sales_price#57, ws_sold_date_sk#58]
+Condition : isnotnull(ws_bill_addr_sk#56)
-(81) ReusedExchange [Reuses operator id: 37]
-Output [3]: [d_date_sk#62, d_year#63, d_qoy#64]
+(76) ReusedExchange [Reuses operator id: 35]
+Output [3]: [d_date_sk#60, d_year#61, d_qoy#62]
-(82) CometBroadcastHashJoin
-Left output [3]: [ws_bill_addr_sk#58, ws_ext_sales_price#59, ws_sold_date_sk#60]
-Right output [3]: [d_date_sk#62, d_year#63, d_qoy#64]
-Arguments: [ws_sold_date_sk#60], [d_date_sk#62], Inner, BuildRight
+(77) CometBroadcastHashJoin
+Left output [3]: [ws_bill_addr_sk#56, ws_ext_sales_price#57, ws_sold_date_sk#58]
+Right output [3]: [d_date_sk#60, d_year#61, d_qoy#62]
+Arguments: [ws_sold_date_sk#58], [d_date_sk#60], Inner, BuildRight
-(83) CometProject
-Input [6]: [ws_bill_addr_sk#58, ws_ext_sales_price#59, ws_sold_date_sk#60, d_date_sk#62, d_year#63, d_qoy#64]
-Arguments: [ws_bill_addr_sk#58, ws_ext_sales_price#59, d_year#63, d_qoy#64], [ws_bill_addr_sk#58, ws_ext_sales_price#59, d_year#63, d_qoy#64]
+(78) CometProject
+Input [6]: [ws_bill_addr_sk#56, ws_ext_sales_price#57, ws_sold_date_sk#58, d_date_sk#60, d_year#61, d_qoy#62]
+Arguments: [ws_bill_addr_sk#56, ws_ext_sales_price#57, d_year#61, d_qoy#62], [ws_bill_addr_sk#56, ws_ext_sales_price#57, d_year#61, d_qoy#62]
-(84) ReusedExchange [Reuses operator id: 10]
-Output [2]: [ca_address_sk#65, ca_county#66]
+(79) ReusedExchange [Reuses operator id: 10]
+Output [2]: [ca_address_sk#63, ca_county#64]
-(85) CometBroadcastHashJoin
-Left output [4]: [ws_bill_addr_sk#58, ws_ext_sales_price#59, d_year#63, d_qoy#64]
-Right output [2]: [ca_address_sk#65, ca_county#66]
-Arguments: [ws_bill_addr_sk#58], [ca_address_sk#65], Inner, BuildRight
+(80) CometBroadcastHashJoin
+Left output [4]: [ws_bill_addr_sk#56, ws_ext_sales_price#57, d_year#61, d_qoy#62]
+Right output [2]: [ca_address_sk#63, ca_county#64]
+Arguments: [ws_bill_addr_sk#56], [ca_address_sk#63], Inner, BuildRight
-(86) CometProject
-Input [6]: [ws_bill_addr_sk#58, ws_ext_sales_price#59, d_year#63, d_qoy#64, ca_address_sk#65, ca_county#66]
-Arguments: [ws_ext_sales_price#59, d_year#63, d_qoy#64, ca_county#66], [ws_ext_sales_price#59, d_year#63, d_qoy#64, ca_county#66]
+(81) CometProject
+Input [6]: [ws_bill_addr_sk#56, ws_ext_sales_price#57, d_year#61, d_qoy#62, ca_address_sk#63, ca_county#64]
+Arguments: [ws_ext_sales_price#57, d_year#61, d_qoy#62, ca_county#64], [ws_ext_sales_price#57, d_year#61, d_qoy#62, ca_county#64]
-(87) CometHashAggregate
-Input [4]: [ws_ext_sales_price#59, d_year#63, d_qoy#64, ca_county#66]
-Keys [3]: [ca_county#66, d_qoy#64, d_year#63]
-Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#59))]
+(82) CometHashAggregate
+Input [4]: [ws_ext_sales_price#57, d_year#61, d_qoy#62, ca_county#64]
+Keys [3]: [ca_county#64, d_qoy#62, d_year#61]
+Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#57))]
-(88) CometExchange
-Input [4]: [ca_county#66, d_qoy#64, d_year#63, sum#67]
-Arguments: hashpartitioning(ca_county#66, d_qoy#64, d_year#63, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=10]
+(83) CometExchange
+Input [4]: [ca_county#64, d_qoy#62, d_year#61, sum#65]
+Arguments: hashpartitioning(ca_county#64, d_qoy#62, d_year#61, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
-(89) CometColumnarToRow [codegen id : 5]
-Input [4]: [ca_county#66, d_qoy#64, d_year#63, sum#67]
+(84) CometHashAggregate
+Input [4]: [ca_county#64, d_qoy#62, d_year#61, sum#65]
+Keys [3]: [ca_county#64, d_qoy#62, d_year#61]
+Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#57))]
-(90) HashAggregate [codegen id : 5]
-Input [4]: [ca_county#66, d_qoy#64, d_year#63, sum#67]
-Keys [3]: [ca_county#66, d_qoy#64, d_year#63]
-Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#59))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ws_ext_sales_price#59))#45]
-Results [2]: [ca_county#66, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#59))#45,17,2) AS web_sales#68]
+(85) CometBroadcastExchange
+Input [2]: [ca_county#64, web_sales#66]
+Arguments: [ca_county#64, web_sales#66]
-(91) BroadcastExchange
-Input [2]: [ca_county#66, web_sales#68]
-Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=11]
+(86) CometBroadcastHashJoin
+Left output [8]: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33, ca_county#42, web_sales#44, web_sales#55]
+Right output [2]: [ca_county#64, web_sales#66]
+Arguments: [ca_county#42], [ca_county#64], Inner, (CASE WHEN (web_sales#55 > 0.00) THEN (web_sales#66 / web_sales#55) END > CASE WHEN (store_sales#21 > 0.00) THEN (store_sales#33 / store_sales#21) END), BuildRight
-(92) BroadcastHashJoin [codegen id : 6]
-Left keys [1]: [ca_county#43]
-Right keys [1]: [ca_county#66]
-Join type: Inner
-Join condition: (CASE WHEN (web_sales#57 > 0.00) THEN (web_sales#68 / web_sales#57) END > CASE WHEN (store_sales#23 > 0.00) THEN (store_sales#34 / store_sales#23) END)
+(87) CometProject
+Input [10]: [ca_county#9, d_year#6, store_sales#22, store_sales#21, store_sales#33, ca_county#42, web_sales#44, web_sales#55, ca_county#64, web_sales#66]
+Arguments: [ca_county#9, d_year#6, web_q1_q2_increase#67, store_q1_q2_increase#68, web_q2_q3_increase#69, store_q2_q3_increase#70], [ca_county#9, d_year#6, (web_sales#55 / web_sales#44) AS web_q1_q2_increase#67, (store_sales#21 / store_sales#22) AS store_q1_q2_increase#68, (web_sales#66 / web_sales#55) AS web_q2_q3_increase#69, (store_sales#33 / store_sales#21) AS store_q2_q3_increase#70]
-(93) Project [codegen id : 6]
-Output [6]: [ca_county#9, d_year#6, (web_sales#57 / web_sales#46) AS web_q1_q2_increase#69, (store_sales#23 / store_sales#12) AS store_q1_q2_increase#70, (web_sales#68 / web_sales#57) AS web_q2_q3_increase#71, (store_sales#34 / store_sales#23) AS store_q2_q3_increase#72]
-Input [10]: [ca_county#9, d_year#6, store_sales#12, store_sales#23, store_sales#34, ca_county#43, web_sales#46, web_sales#57, ca_county#66, web_sales#68]
-
-(94) CometColumnarExchange
-Input [6]: [ca_county#9, d_year#6, web_q1_q2_increase#69, store_q1_q2_increase#70, web_q2_q3_increase#71, store_q2_q3_increase#72]
-Arguments: rangepartitioning(ca_county#9 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=12]
+(88) CometExchange
+Input [6]: [ca_county#9, d_year#6, web_q1_q2_increase#67, store_q1_q2_increase#68, web_q2_q3_increase#69, store_q2_q3_increase#70]
+Arguments: rangepartitioning(ca_county#9 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=7]
-(95) CometSort
-Input [6]: [ca_county#9, d_year#6, web_q1_q2_increase#69, store_q1_q2_increase#70, web_q2_q3_increase#71, store_q2_q3_increase#72]
-Arguments: [ca_county#9, d_year#6, web_q1_q2_increase#69, store_q1_q2_increase#70, web_q2_q3_increase#71, store_q2_q3_increase#72], [ca_county#9 ASC NULLS FIRST]
+(89) CometSort
+Input [6]: [ca_county#9, d_year#6, web_q1_q2_increase#67, store_q1_q2_increase#68, web_q2_q3_increase#69, store_q2_q3_increase#70]
+Arguments: [ca_county#9, d_year#6, web_q1_q2_increase#67, store_q1_q2_increase#68, web_q2_q3_increase#69, store_q2_q3_increase#70], [ca_county#9 ASC NULLS FIRST]
-(96) CometColumnarToRow [codegen id : 7]
-Input [6]: [ca_county#9, d_year#6, web_q1_q2_increase#69, store_q1_q2_increase#70, web_q2_q3_increase#71, store_q2_q3_increase#72]
+(90) CometColumnarToRow [codegen id : 1]
+Input [6]: [ca_county#9, d_year#6, web_q1_q2_increase#67, store_q1_q2_increase#68, web_q2_q3_increase#69, store_q2_q3_increase#70]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (100)
-+- * CometColumnarToRow (99)
- +- CometFilter (98)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (97)
+BroadcastExchange (94)
++- * CometColumnarToRow (93)
+ +- CometFilter (92)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (91)
-(97) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+(91) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#5, d_year#6, d_qoy#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,1), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(98) CometFilter
+(92) CometFilter
Input [3]: [d_date_sk#5, d_year#6, d_qoy#7]
Condition : ((((isnotnull(d_qoy#7) AND isnotnull(d_year#6)) AND (d_qoy#7 = 1)) AND (d_year#6 = 2000)) AND isnotnull(d_date_sk#5))
-(99) CometColumnarToRow [codegen id : 1]
+(93) CometColumnarToRow [codegen id : 1]
Input [3]: [d_date_sk#5, d_year#6, d_qoy#7]
-(100) BroadcastExchange
+(94) BroadcastExchange
Input [3]: [d_date_sk#5, d_year#6, d_qoy#7]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=13]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=8]
-Subquery:2 Hosting operator id = 17 Hosting Expression = ss_sold_date_sk#15 IN dynamicpruning#16
-BroadcastExchange (104)
-+- * CometColumnarToRow (103)
- +- CometFilter (102)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (101)
+Subquery:2 Hosting operator id = 16 Hosting Expression = ss_sold_date_sk#13 IN dynamicpruning#14
+BroadcastExchange (98)
++- * CometColumnarToRow (97)
+ +- CometFilter (96)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (95)
-(101) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#17, d_year#18, d_qoy#19]
+(95) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#15, d_year#16, d_qoy#17]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,2), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(102) CometFilter
-Input [3]: [d_date_sk#17, d_year#18, d_qoy#19]
-Condition : ((((isnotnull(d_qoy#19) AND isnotnull(d_year#18)) AND (d_qoy#19 = 2)) AND (d_year#18 = 2000)) AND isnotnull(d_date_sk#17))
+(96) CometFilter
+Input [3]: [d_date_sk#15, d_year#16, d_qoy#17]
+Condition : ((((isnotnull(d_qoy#17) AND isnotnull(d_year#16)) AND (d_qoy#17 = 2)) AND (d_year#16 = 2000)) AND isnotnull(d_date_sk#15))
-(103) CometColumnarToRow [codegen id : 1]
-Input [3]: [d_date_sk#17, d_year#18, d_qoy#19]
+(97) CometColumnarToRow [codegen id : 1]
+Input [3]: [d_date_sk#15, d_year#16, d_qoy#17]
-(104) BroadcastExchange
-Input [3]: [d_date_sk#17, d_year#18, d_qoy#19]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=14]
+(98) BroadcastExchange
+Input [3]: [d_date_sk#15, d_year#16, d_qoy#17]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=9]
-Subquery:3 Hosting operator id = 33 Hosting Expression = ss_sold_date_sk#26 IN dynamicpruning#27
-BroadcastExchange (108)
-+- * CometColumnarToRow (107)
- +- CometFilter (106)
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (105)
+Subquery:3 Hosting operator id = 31 Hosting Expression = ss_sold_date_sk#25 IN dynamicpruning#26
+BroadcastExchange (102)
++- * CometColumnarToRow (101)
+ +- CometFilter (100)
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim (99)
-(105) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#28, d_year#29, d_qoy#30]
+(99) CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#27, d_year#28, d_qoy#29]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,3), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(106) CometFilter
-Input [3]: [d_date_sk#28, d_year#29, d_qoy#30]
-Condition : ((((isnotnull(d_qoy#30) AND isnotnull(d_year#29)) AND (d_qoy#30 = 3)) AND (d_year#29 = 2000)) AND isnotnull(d_date_sk#28))
+(100) CometFilter
+Input [3]: [d_date_sk#27, d_year#28, d_qoy#29]
+Condition : ((((isnotnull(d_qoy#29) AND isnotnull(d_year#28)) AND (d_qoy#29 = 3)) AND (d_year#28 = 2000)) AND isnotnull(d_date_sk#27))
-(107) CometColumnarToRow [codegen id : 1]
-Input [3]: [d_date_sk#28, d_year#29, d_qoy#30]
+(101) CometColumnarToRow [codegen id : 1]
+Input [3]: [d_date_sk#27, d_year#28, d_qoy#29]
-(108) BroadcastExchange
-Input [3]: [d_date_sk#28, d_year#29, d_qoy#30]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=15]
+(102) BroadcastExchange
+Input [3]: [d_date_sk#27, d_year#28, d_qoy#29]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=10]
-Subquery:4 Hosting operator id = 50 Hosting Expression = ws_sold_date_sk#37 IN dynamicpruning#4
+Subquery:4 Hosting operator id = 47 Hosting Expression = ws_sold_date_sk#36 IN dynamicpruning#4
-Subquery:5 Hosting operator id = 64 Hosting Expression = ws_sold_date_sk#49 IN dynamicpruning#16
+Subquery:5 Hosting operator id = 60 Hosting Expression = ws_sold_date_sk#47 IN dynamicpruning#14
-Subquery:6 Hosting operator id = 79 Hosting Expression = ws_sold_date_sk#60 IN dynamicpruning#27
+Subquery:6 Hosting operator id = 74 Hosting Expression = ws_sold_date_sk#58 IN dynamicpruning#26
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/extended.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/extended.txt
index aef97a206..b4e7aa4db 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/extended.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/extended.txt
@@ -1,132 +1,126 @@
CometColumnarToRow
+- CometSort
- +- CometColumnarExchange
- +- Project
- +- BroadcastHashJoin
- :- Project
- : +- BroadcastHashJoin
- : :- BroadcastHashJoin
- : : :- Project
- : : : +- BroadcastHashJoin
- : : : :- BroadcastHashJoin
- : : : : :- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_county#1, d_year#2, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#3))#4,17,2) AS store_sales#5)]
- : : : : : +- CometColumnarToRow
- : : : : : +- CometExchange
- : : : : : +- CometHashAggregate
- : : : : : +- CometProject
- : : : : : +- CometBroadcastHashJoin
- : : : : : :- CometProject
- : : : : : : +- CometBroadcastHashJoin
- : : : : : : :- CometFilter
- : : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : : : : : +- SubqueryBroadcast
- : : : : : : : +- BroadcastExchange
- : : : : : : : +- CometColumnarToRow
- : : : : : : : +- CometFilter
- : : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : : : +- CometBroadcastExchange
- : : : : : : +- CometFilter
- : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : : +- CometBroadcastExchange
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : : : : +- BroadcastExchange
- : : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_county#6, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#7))#4,17,2) AS store_sales#8)]
- : : : : +- CometColumnarToRow
- : : : : +- CometExchange
- : : : : +- CometHashAggregate
- : : : : +- CometProject
- : : : : +- CometBroadcastHashJoin
- : : : : :- CometProject
- : : : : : +- CometBroadcastHashJoin
- : : : : : :- CometFilter
- : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : : : : +- SubqueryBroadcast
- : : : : : : +- BroadcastExchange
- : : : : : : +- CometColumnarToRow
- : : : : : : +- CometFilter
- : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : : +- CometBroadcastExchange
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : +- CometBroadcastExchange
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : : : +- BroadcastExchange
- : : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_county#9, MakeDecimal(sum(UnscaledValue(ss_ext_sales_price#10))#4,17,2) AS store_sales#11)]
- : : : +- CometColumnarToRow
- : : : +- CometExchange
- : : : +- CometHashAggregate
- : : : +- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometProject
- : : : : +- CometBroadcastHashJoin
- : : : : :- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
- : : : : : +- SubqueryBroadcast
- : : : : : +- BroadcastExchange
- : : : : : +- CometColumnarToRow
- : : : : : +- CometFilter
- : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : : +- CometBroadcastExchange
- : : : : +- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : : +- CometBroadcastExchange
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : : +- BroadcastExchange
- : : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_county#12, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#13))#14,17,2) AS web_sales#15)]
- : : +- CometColumnarToRow
- : : +- CometExchange
- : : +- CometHashAggregate
- : : +- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometProject
- : : : +- CometBroadcastHashJoin
- : : : :- CometFilter
- : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : : : +- ReusedSubquery
- : : : +- CometBroadcastExchange
- : : : +- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : : +- CometBroadcastExchange
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- : +- BroadcastExchange
- : +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_county#16, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#17))#14,17,2) AS web_sales#18)]
- : +- CometColumnarToRow
- : +- CometExchange
- : +- CometHashAggregate
- : +- CometProject
- : +- CometBroadcastHashJoin
- : :- CometProject
- : : +- CometBroadcastHashJoin
- : : :- CometFilter
- : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : : +- ReusedSubquery
- : : +- CometBroadcastExchange
- : : +- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- : +- CometBroadcastExchange
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
- +- BroadcastExchange
- +- HashAggregate [COMET: makedecimal is not supported, Unsupported result expressions found in: List(ca_county#19, MakeDecimal(sum(UnscaledValue(ws_ext_sales_price#20))#14,17,2) AS web_sales#21)]
- +- CometColumnarToRow
- +- CometExchange
- +- CometHashAggregate
- +- CometProject
- +- CometBroadcastHashJoin
- :- CometProject
- : +- CometBroadcastHashJoin
- : :- CometFilter
- : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
- : : +- ReusedSubquery
- : +- CometBroadcastExchange
- : +- CometFilter
- : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
- +- CometBroadcastExchange
- +- CometFilter
- +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ +- CometExchange
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometBroadcastHashJoin
+ : : : : :- CometHashAggregate
+ : : : : : +- CometExchange
+ : : : : : +- CometHashAggregate
+ : : : : : +- CometProject
+ : : : : : +- CometBroadcastHashJoin
+ : : : : : :- CometProject
+ : : : : : : +- CometBroadcastHashJoin
+ : : : : : : :- CometFilter
+ : : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : : : : : +- SubqueryBroadcast
+ : : : : : : : +- BroadcastExchange
+ : : : : : : : +- CometColumnarToRow
+ : : : : : : : +- CometFilter
+ : : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : : : +- CometBroadcastExchange
+ : : : : : : +- CometFilter
+ : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : : +- CometBroadcastExchange
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometHashAggregate
+ : : : : +- CometExchange
+ : : : : +- CometHashAggregate
+ : : : : +- CometProject
+ : : : : +- CometBroadcastHashJoin
+ : : : : :- CometProject
+ : : : : : +- CometBroadcastHashJoin
+ : : : : : :- CometFilter
+ : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : : : : +- SubqueryBroadcast
+ : : : : : : +- BroadcastExchange
+ : : : : : : +- CometColumnarToRow
+ : : : : : : +- CometFilter
+ : : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : : +- CometBroadcastExchange
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : : : +- CometBroadcastExchange
+ : : : +- CometHashAggregate
+ : : : +- CometExchange
+ : : : +- CometHashAggregate
+ : : : +- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometProject
+ : : : : +- CometBroadcastHashJoin
+ : : : : :- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales
+ : : : : : +- SubqueryBroadcast
+ : : : : : +- BroadcastExchange
+ : : : : : +- CometColumnarToRow
+ : : : : : +- CometFilter
+ : : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : : +- CometBroadcastExchange
+ : : : : +- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : : +- CometBroadcastExchange
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : : +- CometBroadcastExchange
+ : : +- CometHashAggregate
+ : : +- CometExchange
+ : : +- CometHashAggregate
+ : : +- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometProject
+ : : : +- CometBroadcastHashJoin
+ : : : :- CometFilter
+ : : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : : : +- ReusedSubquery
+ : : : +- CometBroadcastExchange
+ : : : +- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ : +- CometBroadcastExchange
+ : +- CometHashAggregate
+ : +- CometExchange
+ : +- CometHashAggregate
+ : +- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometProject
+ : : +- CometBroadcastHashJoin
+ : : :- CometFilter
+ : : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : : +- ReusedSubquery
+ : : +- CometBroadcastExchange
+ : : +- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ : +- CometBroadcastExchange
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
+ +- CometBroadcastExchange
+ +- CometHashAggregate
+ +- CometExchange
+ +- CometHashAggregate
+ +- CometProject
+ +- CometBroadcastHashJoin
+ :- CometProject
+ : +- CometBroadcastHashJoin
+ : :- CometFilter
+ : : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.web_sales
+ : : +- ReusedSubquery
+ : +- CometBroadcastExchange
+ : +- CometFilter
+ : +- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim
+ +- CometBroadcastExchange
+ +- CometFilter
+ +- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address
-Comet accelerated 92 out of 120 eligible operators (76%). Final plan contains 10 transitions between Spark and Comet.
\ No newline at end of file
+Comet accelerated 111 out of 120 eligible operators (92%). Final plan contains 4 transitions between Spark and Comet.
\ No newline at end of file
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/simplified.txt
index fcea2cb24..2f9738432 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark4_0/q31.native_iceberg_compat/simplified.txt
@@ -1,139 +1,116 @@
-WholeStageCodegen (7)
+WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
CometSort [ca_county,d_year,web_q1_q2_increase,store_q1_q2_increase,web_q2_q3_increase,store_q2_q3_increase]
- CometColumnarExchange [ca_county] #1
- WholeStageCodegen (6)
- Project [ca_county,d_year,web_sales,web_sales,store_sales,store_sales,web_sales,store_sales]
- BroadcastHashJoin [ca_county,ca_county,web_sales,web_sales,store_sales,store_sales]
- Project [ca_county,d_year,store_sales,store_sales,store_sales,ca_county,web_sales,web_sales]
- BroadcastHashJoin [ca_county,ca_county,web_sales,web_sales,store_sales,store_sales]
- BroadcastHashJoin [ca_county,ca_county]
- Project [ca_county,d_year,store_sales,store_sales,store_sales]
- BroadcastHashJoin [ca_county,ca_county]
- BroadcastHashJoin [ca_county,ca_county]
- HashAggregate [ca_county,d_qoy,d_year,sum] [sum(UnscaledValue(ss_ext_sales_price)),store_sales,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [ca_county,d_qoy,d_year] #2
- CometHashAggregate [ss_ext_sales_price] [ca_county,d_qoy,d_year,sum]
- CometProject [ss_ext_sales_price,d_year,d_qoy,ca_county]
- CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
- CometProject [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy]
- CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,ss_sold_date_sk,d_date_sk,d_year,d_qoy]
- CometFilter [ss_addr_sk,ss_ext_sales_price,ss_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales [ss_addr_sk,ss_ext_sales_price,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- CometBroadcastExchange [d_date_sk,d_year,d_qoy] #4
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- CometBroadcastExchange [ca_address_sk,ca_county] #5
- CometFilter [ca_address_sk,ca_county]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.customer_address [ca_address_sk,ca_county]
- InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (1)
- HashAggregate [ca_county,d_qoy,d_year,sum] [sum(UnscaledValue(ss_ext_sales_price)),store_sales,sum]
- CometColumnarToRow
- InputAdapter
- CometExchange [ca_county,d_qoy,d_year] #7
- CometHashAggregate [ss_ext_sales_price] [ca_county,d_qoy,d_year,sum]
- CometProject [ss_ext_sales_price,d_year,d_qoy,ca_county]
- CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
- CometProject [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy]
- CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,ss_sold_date_sk,d_date_sk,d_year,d_qoy]
- CometFilter [ss_addr_sk,ss_ext_sales_price,ss_sold_date_sk]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.store_sales [ss_addr_sk,ss_ext_sales_price,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #2
- BroadcastExchange #8
- WholeStageCodegen (1)
- CometColumnarToRow
- InputAdapter
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan [native_iceberg_compat] parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- CometBroadcastExchange [d_date_sk,d_year,d_qoy] #9
+ CometExchange [ca_county] #1
+ CometProject [web_sales,web_sales,store_sales,store_sales,web_sales,store_sales] [ca_county,d_year,web_q1_q2_increase,store_q1_q2_increase,web_q2_q3_increase,store_q2_q3_increase]
+ CometBroadcastHashJoin [ca_county,d_year,store_sales,store_sales,store_sales,ca_county,web_sales,web_sales,ca_county,web_sales]
+ CometProject [ca_county,d_year,store_sales,store_sales,store_sales,ca_county,web_sales,web_sales]
+ CometBroadcastHashJoin [ca_county,d_year,store_sales,store_sales,store_sales,ca_county,web_sales,ca_county,web_sales]
+ CometBroadcastHashJoin [ca_county,d_year,store_sales,store_sales,store_sales,ca_county,web_sales]
+ CometProject [ca_county,d_year,store_sales,store_sales,store_sales]
+ CometBroadcastHashJoin [ca_county,d_year,store_sales,ca_county,store_sales,ca_county,store_sales]
+ CometBroadcastHashJoin [ca_county,d_year,store_sales,ca_county,store_sales]
+ CometHashAggregate [d_qoy,sum] [ca_county,d_year,store_sales,sum(UnscaledValue(ss_ext_sales_price))]
+ CometExchange [ca_county,d_qoy,d_year] #2
+ CometHashAggregate [ss_ext_sales_price] [ca_county,d_qoy,d_year,sum]
+ CometProject [ss_ext_sales_price,d_year,d_qoy,ca_county]
+ CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
... 81848 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org