You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ag...@apache.org on 2024/08/28 23:27:03 UTC
(datafusion-comet) branch main updated: chore: change shuffle mode default from jvm to auto (#877)
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new fe352242 chore: change shuffle mode default from jvm to auto (#877)
fe352242 is described below
commit fe35224256cce28964df0a622aabb93f8e9525a6
Author: Andy Grove <ag...@apache.org>
AuthorDate: Wed Aug 28 17:27:00 2024 -0600
chore: change shuffle mode default from jvm to auto (#877)
* change shuffle mode default from jvm to auto
* update expected plans for Spark 3.4
* update expected plans for Spark 3.5
* fix docs for default value
* update expected plans for Spark 4.0
* update expected plans for Spark 4.0
* update 3.4 plans
---
.../main/scala/org/apache/comet/CometConf.scala | 4 +-
docs/source/user-guide/configs.md | 2 +-
.../approved-plans-v1_4-spark3_5/q1/explain.txt | 18 +-
.../approved-plans-v1_4-spark3_5/q1/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q10/explain.txt | 126 ++--
.../q10/simplified.txt | 128 ++--
.../approved-plans-v1_4-spark3_5/q11/explain.txt | 24 +-
.../q11/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q12/explain.txt | 12 +-
.../q12/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q13/explain.txt | 6 +-
.../q13/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q14a/explain.txt | 36 +-
.../q14a/simplified.txt | 12 +-
.../approved-plans-v1_4-spark3_5/q14b/explain.txt | 24 +-
.../q14b/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q15/explain.txt | 6 +-
.../q15/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q16/explain.txt | 116 ++--
.../q16/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark3_5/q17/explain.txt | 6 +-
.../q17/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q18/explain.txt | 6 +-
.../q18/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q19/explain.txt | 6 +-
.../q19/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q2/explain.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q2/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q20/explain.txt | 12 +-
.../q20/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q21/explain.txt | 6 +-
.../q21/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q22/explain.txt | 6 +-
.../q22/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q23a/explain.txt | 42 +-
.../q23a/simplified.txt | 14 +-
.../approved-plans-v1_4-spark3_5/q23b/explain.txt | 54 +-
.../q23b/simplified.txt | 18 +-
.../approved-plans-v1_4-spark3_5/q24a/explain.txt | 258 +++----
.../q24a/simplified.txt | 164 +++--
.../approved-plans-v1_4-spark3_5/q24b/explain.txt | 258 +++----
.../q24b/simplified.txt | 164 +++--
.../approved-plans-v1_4-spark3_5/q25/explain.txt | 6 +-
.../q25/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q26/explain.txt | 6 +-
.../q26/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q27/explain.txt | 6 +-
.../q27/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q28/explain.txt | 350 +++++-----
.../q28/simplified.txt | 140 ++--
.../approved-plans-v1_4-spark3_5/q29/explain.txt | 6 +-
.../q29/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q3/explain.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q3/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q30/explain.txt | 18 +-
.../q30/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q31/explain.txt | 36 +-
.../q31/simplified.txt | 12 +-
.../approved-plans-v1_4-spark3_5/q32/explain.txt | 12 +-
.../q32/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q33/explain.txt | 24 +-
.../q33/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q34/explain.txt | 6 +-
.../q34/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q35/explain.txt | 124 ++--
.../q35/simplified.txt | 126 ++--
.../approved-plans-v1_4-spark3_5/q36/explain.txt | 12 +-
.../q36/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q37/explain.txt | 6 +-
.../q37/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q38/explain.txt | 24 +-
.../q38/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q39a/explain.txt | 12 +-
.../q39a/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q39b/explain.txt | 12 +-
.../q39b/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q4/explain.txt | 36 +-
.../approved-plans-v1_4-spark3_5/q4/simplified.txt | 12 +-
.../approved-plans-v1_4-spark3_5/q40/explain.txt | 18 +-
.../q40/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q41/explain.txt | 12 +-
.../q41/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q42/explain.txt | 6 +-
.../q42/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q43/explain.txt | 6 +-
.../q43/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q44/explain.txt | 208 +++---
.../q44/simplified.txt | 74 +-
.../approved-plans-v1_4-spark3_5/q45/explain.txt | 108 ++-
.../q45/simplified.txt | 98 ++-
.../approved-plans-v1_4-spark3_5/q46/explain.txt | 6 +-
.../q46/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q47/explain.txt | 12 +-
.../q47/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q48/explain.txt | 6 +-
.../q48/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q49/explain.txt | 218 +++---
.../q49/simplified.txt | 220 +++---
.../approved-plans-v1_4-spark3_5/q5/explain.txt | 188 +++--
.../approved-plans-v1_4-spark3_5/q5/simplified.txt | 182 +++--
.../approved-plans-v1_4-spark3_5/q50/explain.txt | 6 +-
.../q50/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q51/explain.txt | 199 +++---
.../q51/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark3_5/q52/explain.txt | 6 +-
.../q52/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q53/explain.txt | 12 +-
.../q53/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q54/explain.txt | 30 +-
.../q54/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q55/explain.txt | 6 +-
.../q55/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q56/explain.txt | 24 +-
.../q56/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q57/explain.txt | 12 +-
.../q57/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q58/explain.txt | 18 +-
.../q58/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q59/explain.txt | 6 +-
.../q59/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q6/explain.txt | 18 +-
.../approved-plans-v1_4-spark3_5/q6/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q60/explain.txt | 24 +-
.../q60/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q61/explain.txt | 12 +-
.../q61/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q62/explain.txt | 6 +-
.../q62/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q63/explain.txt | 12 +-
.../q63/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q64/explain.txt | 42 +-
.../q64/simplified.txt | 14 +-
.../approved-plans-v1_4-spark3_5/q65/explain.txt | 18 +-
.../q65/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q66/explain.txt | 18 +-
.../q66/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q67/explain.txt | 110 ++-
.../q67/simplified.txt | 76 +-
.../approved-plans-v1_4-spark3_5/q68/explain.txt | 6 +-
.../q68/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q69/explain.txt | 124 ++--
.../q69/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark3_5/q7/explain.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q7/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q70/explain.txt | 150 ++--
.../q70/simplified.txt | 120 ++--
.../approved-plans-v1_4-spark3_5/q71/explain.txt | 6 +-
.../q71/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q72/explain.txt | 219 +++---
.../q72/simplified.txt | 172 ++---
.../approved-plans-v1_4-spark3_5/q73/explain.txt | 6 +-
.../q73/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q74/explain.txt | 24 +-
.../q74/simplified.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q75/explain.txt | 90 +--
.../q75/simplified.txt | 30 +-
.../approved-plans-v1_4-spark3_5/q76/explain.txt | 6 +-
.../q76/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q77/explain.txt | 228 +++---
.../q77/simplified.txt | 210 +++---
.../approved-plans-v1_4-spark3_5/q78/explain.txt | 54 +-
.../q78/simplified.txt | 18 +-
.../approved-plans-v1_4-spark3_5/q79/explain.txt | 6 +-
.../q79/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q8/explain.txt | 18 +-
.../approved-plans-v1_4-spark3_5/q8/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q80/explain.txt | 276 ++++----
.../q80/simplified.txt | 240 ++++---
.../approved-plans-v1_4-spark3_5/q81/explain.txt | 18 +-
.../q81/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q82/explain.txt | 6 +-
.../q82/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q83/explain.txt | 18 +-
.../q83/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q85/explain.txt | 6 +-
.../q85/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q86/explain.txt | 12 +-
.../q86/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q87/explain.txt | 144 ++--
.../q87/simplified.txt | 126 ++--
.../approved-plans-v1_4-spark3_5/q88/explain.txt | 48 +-
.../q88/simplified.txt | 16 +-
.../approved-plans-v1_4-spark3_5/q89/explain.txt | 12 +-
.../q89/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q9/explain.txt | 30 +-
.../approved-plans-v1_4-spark3_5/q9/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q90/explain.txt | 12 +-
.../q90/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q91/explain.txt | 6 +-
.../q91/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q92/explain.txt | 12 +-
.../q92/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q93/explain.txt | 18 +-
.../q93/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q94/explain.txt | 116 ++--
.../q94/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark3_5/q95/explain.txt | 148 ++--
.../q95/simplified.txt | 178 +++--
.../approved-plans-v1_4-spark3_5/q96/explain.txt | 6 +-
.../q96/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q97/explain.txt | 18 +-
.../q97/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q98/explain.txt | 12 +-
.../q98/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q99/explain.txt | 6 +-
.../q99/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q1/explain.txt | 204 +++---
.../approved-plans-v1_4-spark4_0/q1/simplified.txt | 84 ++-
.../approved-plans-v1_4-spark4_0/q10/explain.txt | 126 ++--
.../q10/simplified.txt | 128 ++--
.../approved-plans-v1_4-spark4_0/q11/explain.txt | 338 +++++----
.../q11/simplified.txt | 148 ++--
.../approved-plans-v1_4-spark4_0/q12/explain.txt | 100 ++-
.../q12/simplified.txt | 66 +-
.../approved-plans-v1_4-spark4_0/q13/explain.txt | 100 ++-
.../q13/simplified.txt | 88 ++-
.../approved-plans-v1_4-spark4_0/q14a/explain.txt | 474 ++++++-------
.../q14a/simplified.txt | 334 +++++----
.../approved-plans-v1_4-spark4_0/q14b/explain.txt | 442 ++++++------
.../q14b/simplified.txt | 304 ++++----
.../approved-plans-v1_4-spark4_0/q15/explain.txt | 82 +--
.../q15/simplified.txt | 66 +-
.../approved-plans-v1_4-spark4_0/q16/explain.txt | 116 ++--
.../q16/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark4_0/q17/explain.txt | 140 ++--
.../q17/simplified.txt | 116 ++--
.../approved-plans-v1_4-spark4_0/q18/explain.txt | 118 ++--
.../q18/simplified.txt | 102 ++-
.../approved-plans-v1_4-spark4_0/q19/explain.txt | 84 +--
.../q19/simplified.txt | 72 +-
.../approved-plans-v1_4-spark4_0/q2/explain.txt | 138 ++--
.../approved-plans-v1_4-spark4_0/q2/simplified.txt | 39 +-
.../approved-plans-v1_4-spark4_0/q20/explain.txt | 100 ++-
.../q20/simplified.txt | 66 +-
.../approved-plans-v1_4-spark4_0/q21/explain.txt | 82 +--
.../q21/simplified.txt | 64 +-
.../approved-plans-v1_4-spark4_0/q22/explain.txt | 84 +--
.../q22/simplified.txt | 68 +-
.../approved-plans-v1_4-spark4_0/q23a/explain.txt | 392 +++++------
.../q23a/simplified.txt | 275 ++++----
.../approved-plans-v1_4-spark4_0/q23b/explain.txt | 492 ++++++-------
.../q23b/simplified.txt | 330 +++++----
.../approved-plans-v1_4-spark4_0/q24a/explain.txt | 258 +++----
.../q24a/simplified.txt | 164 +++--
.../approved-plans-v1_4-spark4_0/q24b/explain.txt | 258 +++----
.../q24b/simplified.txt | 164 +++--
.../approved-plans-v1_4-spark4_0/q25/explain.txt | 140 ++--
.../q25/simplified.txt | 116 ++--
.../approved-plans-v1_4-spark4_0/q26/explain.txt | 96 ++-
.../q26/simplified.txt | 80 ++-
.../approved-plans-v1_4-spark4_0/q27/explain.txt | 96 ++-
.../q27/simplified.txt | 80 ++-
.../approved-plans-v1_4-spark4_0/q28/explain.txt | 390 ++++-------
.../q28/simplified.txt | 192 +++---
.../approved-plans-v1_4-spark4_0/q29/explain.txt | 166 +++--
.../q29/simplified.txt | 126 ++--
.../approved-plans-v1_4-spark4_0/q3/explain.txt | 54 +-
.../approved-plans-v1_4-spark4_0/q3/simplified.txt | 42 +-
.../approved-plans-v1_4-spark4_0/q30/explain.txt | 226 +++---
.../q30/simplified.txt | 100 ++-
.../approved-plans-v1_4-spark4_0/q31/explain.txt | 478 ++++++-------
.../q31/simplified.txt | 216 +++---
.../approved-plans-v1_4-spark4_0/q32/explain.txt | 126 ++--
.../q32/simplified.txt | 98 ++-
.../approved-plans-v1_4-spark4_0/q33/explain.txt | 264 ++++---
.../q33/simplified.txt | 180 +++--
.../approved-plans-v1_4-spark4_0/q34/explain.txt | 6 +-
.../q34/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q35/explain.txt | 124 ++--
.../q35/simplified.txt | 126 ++--
.../approved-plans-v1_4-spark4_0/q36/explain.txt | 114 ++-
.../q36/simplified.txt | 80 +--
.../approved-plans-v1_4-spark4_0/q37/explain.txt | 6 +-
.../q37/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q38/explain.txt | 24 +-
.../q38/simplified.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q39a/explain.txt | 222 +++---
.../q39a/simplified.txt | 124 ++--
.../approved-plans-v1_4-spark4_0/q39b/explain.txt | 222 +++---
.../q39b/simplified.txt | 124 ++--
.../approved-plans-v1_4-spark4_0/q4/explain.txt | 498 ++++++--------
.../approved-plans-v1_4-spark4_0/q4/simplified.txt | 220 +++---
.../approved-plans-v1_4-spark4_0/q40/explain.txt | 104 ++-
.../q40/simplified.txt | 82 ++-
.../approved-plans-v1_4-spark4_0/q41/explain.txt | 12 +-
.../q41/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q42/explain.txt | 54 +-
.../q42/simplified.txt | 42 +-
.../approved-plans-v1_4-spark4_0/q43/explain.txt | 54 +-
.../q43/simplified.txt | 42 +-
.../approved-plans-v1_4-spark4_0/q44/explain.txt | 293 ++++----
.../q44/simplified.txt | 132 ++--
.../approved-plans-v1_4-spark4_0/q45/explain.txt | 108 ++-
.../q45/simplified.txt | 98 ++-
.../approved-plans-v1_4-spark4_0/q46/explain.txt | 144 ++--
.../q46/simplified.txt | 80 ++-
.../approved-plans-v1_4-spark4_0/q47/explain.txt | 202 +++---
.../q47/simplified.txt | 92 ++-
.../approved-plans-v1_4-spark4_0/q48/explain.txt | 90 ++-
.../q48/simplified.txt | 78 +--
.../approved-plans-v1_4-spark4_0/q49/explain.txt | 378 +++++-----
.../q49/simplified.txt | 244 ++++---
.../approved-plans-v1_4-spark4_0/q5/explain.txt | 308 ++++-----
.../approved-plans-v1_4-spark4_0/q5/simplified.txt | 200 +++---
.../approved-plans-v1_4-spark4_0/q50/explain.txt | 92 ++-
.../q50/simplified.txt | 70 +-
.../approved-plans-v1_4-spark4_0/q51/explain.txt | 243 +++----
.../q51/simplified.txt | 88 ++-
.../approved-plans-v1_4-spark4_0/q52/explain.txt | 54 +-
.../q52/simplified.txt | 42 +-
.../approved-plans-v1_4-spark4_0/q53/explain.txt | 116 ++--
.../q53/simplified.txt | 78 +--
.../approved-plans-v1_4-spark4_0/q54/explain.txt | 256 ++++---
.../q54/simplified.txt | 208 +++---
.../approved-plans-v1_4-spark4_0/q55/explain.txt | 54 +-
.../q55/simplified.txt | 42 +-
.../approved-plans-v1_4-spark4_0/q56/explain.txt | 264 ++++---
.../q56/simplified.txt | 180 +++--
.../approved-plans-v1_4-spark4_0/q57/explain.txt | 202 +++---
.../q57/simplified.txt | 92 ++-
.../approved-plans-v1_4-spark4_0/q58/explain.txt | 278 ++++----
.../q58/simplified.txt | 150 ++--
.../approved-plans-v1_4-spark4_0/q59/explain.txt | 168 +++--
.../q59/simplified.txt | 33 +-
.../approved-plans-v1_4-spark4_0/q6/explain.txt | 182 +++--
.../approved-plans-v1_4-spark4_0/q6/simplified.txt | 138 ++--
.../approved-plans-v1_4-spark4_0/q60/explain.txt | 264 ++++---
.../q60/simplified.txt | 180 +++--
.../approved-plans-v1_4-spark4_0/q61/explain.txt | 224 +++---
.../q61/simplified.txt | 154 ++---
.../approved-plans-v1_4-spark4_0/q62/explain.txt | 72 +-
.../q62/simplified.txt | 60 +-
.../approved-plans-v1_4-spark4_0/q63/explain.txt | 116 ++--
.../q63/simplified.txt | 78 +--
.../approved-plans-v1_4-spark4_0/q64/explain.txt | 764 ++++++++++-----------
.../q64/simplified.txt | 425 ++++++------
.../approved-plans-v1_4-spark4_0/q65/explain.txt | 188 +++--
.../q65/simplified.txt | 84 ++-
.../approved-plans-v1_4-spark4_0/q66/explain.txt | 200 +++---
.../q66/simplified.txt | 142 ++--
.../approved-plans-v1_4-spark4_0/q67/explain.txt | 124 ++--
.../q67/simplified.txt | 82 ++-
.../approved-plans-v1_4-spark4_0/q68/explain.txt | 144 ++--
.../q68/simplified.txt | 80 ++-
.../approved-plans-v1_4-spark4_0/q69/explain.txt | 124 ++--
.../q69/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark4_0/q7/explain.txt | 96 ++-
.../approved-plans-v1_4-spark4_0/q7/simplified.txt | 80 ++-
.../approved-plans-v1_4-spark4_0/q70/explain.txt | 184 +++--
.../q70/simplified.txt | 124 ++--
.../approved-plans-v1_4-spark4_0/q71/explain.txt | 120 ++--
.../q71/simplified.txt | 96 ++-
.../approved-plans-v1_4-spark4_0/q72/explain.txt | 219 +++---
.../q72/simplified.txt | 172 ++---
.../approved-plans-v1_4-spark4_0/q73/explain.txt | 6 +-
.../q73/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q74/explain.txt | 334 ++++-----
.../q74/simplified.txt | 148 ++--
.../approved-plans-v1_4-spark4_0/q75/explain.txt | 494 +++++++------
.../q75/simplified.txt | 304 ++++----
.../approved-plans-v1_4-spark4_0/q76/explain.txt | 84 +--
.../q76/simplified.txt | 72 +-
.../approved-plans-v1_4-spark4_0/q77/explain.txt | 414 +++++------
.../q77/simplified.txt | 262 ++++---
.../approved-plans-v1_4-spark4_0/q78/explain.txt | 304 ++++----
.../q78/simplified.txt | 154 ++---
.../approved-plans-v1_4-spark4_0/q79/explain.txt | 110 ++-
.../q79/simplified.txt | 70 +-
.../approved-plans-v1_4-spark4_0/q8/explain.txt | 128 ++--
.../approved-plans-v1_4-spark4_0/q8/simplified.txt | 102 ++-
.../approved-plans-v1_4-spark4_0/q80/explain.txt | 416 ++++++-----
.../q80/simplified.txt | 258 ++++---
.../approved-plans-v1_4-spark4_0/q81/explain.txt | 222 +++---
.../q81/simplified.txt | 100 ++-
.../approved-plans-v1_4-spark4_0/q82/explain.txt | 6 +-
.../q82/simplified.txt | 2 +-
.../q83.ansi/explain.txt | 262 ++++---
.../q83.ansi/simplified.txt | 140 ++--
.../approved-plans-v1_4-spark4_0/q85/explain.txt | 126 ++--
.../q85/simplified.txt | 106 ++-
.../approved-plans-v1_4-spark4_0/q86/explain.txt | 102 ++-
.../q86/simplified.txt | 68 +-
.../approved-plans-v1_4-spark4_0/q87/explain.txt | 144 ++--
.../q87/simplified.txt | 126 ++--
.../approved-plans-v1_4-spark4_0/q88/explain.txt | 48 +-
.../q88/simplified.txt | 16 +-
.../approved-plans-v1_4-spark4_0/q89/explain.txt | 114 ++-
.../q89/simplified.txt | 76 +-
.../approved-plans-v1_4-spark4_0/q9/explain.txt | 212 +++---
.../approved-plans-v1_4-spark4_0/q9/simplified.txt | 100 ++-
.../approved-plans-v1_4-spark4_0/q90/explain.txt | 12 +-
.../q90/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q91/explain.txt | 128 ++--
.../q91/simplified.txt | 100 ++-
.../approved-plans-v1_4-spark4_0/q92/explain.txt | 126 ++--
.../q92/simplified.txt | 98 ++-
.../approved-plans-v1_4-spark4_0/q93/explain.txt | 68 +-
.../q93/simplified.txt | 48 +-
.../approved-plans-v1_4-spark4_0/q94/explain.txt | 116 ++--
.../q94/simplified.txt | 122 ++--
.../approved-plans-v1_4-spark4_0/q95/explain.txt | 148 ++--
.../q95/simplified.txt | 178 +++--
.../approved-plans-v1_4-spark4_0/q96/explain.txt | 6 +-
.../q96/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q97/explain.txt | 92 ++-
.../q97/simplified.txt | 74 +-
.../approved-plans-v1_4-spark4_0/q98/explain.txt | 116 ++--
.../q98/simplified.txt | 66 +-
.../approved-plans-v1_4-spark4_0/q99/explain.txt | 72 +-
.../q99/simplified.txt | 60 +-
.../approved-plans-v1_4/q1/explain.txt | 18 +-
.../approved-plans-v1_4/q1/simplified.txt | 6 +-
.../approved-plans-v1_4/q10/explain.txt | 126 ++--
.../approved-plans-v1_4/q10/simplified.txt | 128 ++--
.../approved-plans-v1_4/q11/explain.txt | 24 +-
.../approved-plans-v1_4/q11/simplified.txt | 8 +-
.../approved-plans-v1_4/q12/explain.txt | 12 +-
.../approved-plans-v1_4/q12/simplified.txt | 4 +-
.../approved-plans-v1_4/q13/explain.txt | 6 +-
.../approved-plans-v1_4/q13/simplified.txt | 2 +-
.../approved-plans-v1_4/q14a/explain.txt | 36 +-
.../approved-plans-v1_4/q14a/simplified.txt | 12 +-
.../approved-plans-v1_4/q14b/explain.txt | 24 +-
.../approved-plans-v1_4/q14b/simplified.txt | 8 +-
.../approved-plans-v1_4/q15/explain.txt | 6 +-
.../approved-plans-v1_4/q15/simplified.txt | 2 +-
.../approved-plans-v1_4/q16/explain.txt | 116 ++--
.../approved-plans-v1_4/q16/simplified.txt | 122 ++--
.../approved-plans-v1_4/q17/explain.txt | 6 +-
.../approved-plans-v1_4/q17/simplified.txt | 2 +-
.../approved-plans-v1_4/q18/explain.txt | 6 +-
.../approved-plans-v1_4/q18/simplified.txt | 2 +-
.../approved-plans-v1_4/q19/explain.txt | 6 +-
.../approved-plans-v1_4/q19/simplified.txt | 2 +-
.../approved-plans-v1_4/q2/explain.txt | 6 +-
.../approved-plans-v1_4/q2/simplified.txt | 2 +-
.../approved-plans-v1_4/q20/explain.txt | 12 +-
.../approved-plans-v1_4/q20/simplified.txt | 4 +-
.../approved-plans-v1_4/q21/explain.txt | 6 +-
.../approved-plans-v1_4/q21/simplified.txt | 2 +-
.../approved-plans-v1_4/q22/explain.txt | 6 +-
.../approved-plans-v1_4/q22/simplified.txt | 2 +-
.../approved-plans-v1_4/q23a/explain.txt | 42 +-
.../approved-plans-v1_4/q23a/simplified.txt | 14 +-
.../approved-plans-v1_4/q23b/explain.txt | 54 +-
.../approved-plans-v1_4/q23b/simplified.txt | 18 +-
.../approved-plans-v1_4/q24a/explain.txt | 258 +++----
.../approved-plans-v1_4/q24a/simplified.txt | 164 +++--
.../approved-plans-v1_4/q24b/explain.txt | 258 +++----
.../approved-plans-v1_4/q24b/simplified.txt | 164 +++--
.../approved-plans-v1_4/q25/explain.txt | 6 +-
.../approved-plans-v1_4/q25/simplified.txt | 2 +-
.../approved-plans-v1_4/q26/explain.txt | 6 +-
.../approved-plans-v1_4/q26/simplified.txt | 2 +-
.../approved-plans-v1_4/q27/explain.txt | 6 +-
.../approved-plans-v1_4/q27/simplified.txt | 2 +-
.../approved-plans-v1_4/q28/explain.txt | 350 +++++-----
.../approved-plans-v1_4/q28/simplified.txt | 140 ++--
.../approved-plans-v1_4/q29/explain.txt | 6 +-
.../approved-plans-v1_4/q29/simplified.txt | 2 +-
.../approved-plans-v1_4/q3/explain.txt | 6 +-
.../approved-plans-v1_4/q3/simplified.txt | 2 +-
.../approved-plans-v1_4/q30/explain.txt | 18 +-
.../approved-plans-v1_4/q30/simplified.txt | 6 +-
.../approved-plans-v1_4/q31/explain.txt | 36 +-
.../approved-plans-v1_4/q31/simplified.txt | 12 +-
.../approved-plans-v1_4/q32/explain.txt | 12 +-
.../approved-plans-v1_4/q32/simplified.txt | 4 +-
.../approved-plans-v1_4/q33/explain.txt | 24 +-
.../approved-plans-v1_4/q33/simplified.txt | 8 +-
.../approved-plans-v1_4/q34/explain.txt | 6 +-
.../approved-plans-v1_4/q34/simplified.txt | 2 +-
.../approved-plans-v1_4/q35/explain.txt | 124 ++--
.../approved-plans-v1_4/q35/simplified.txt | 126 ++--
.../approved-plans-v1_4/q36/explain.txt | 12 +-
.../approved-plans-v1_4/q36/simplified.txt | 4 +-
.../approved-plans-v1_4/q37/explain.txt | 6 +-
.../approved-plans-v1_4/q37/simplified.txt | 2 +-
.../approved-plans-v1_4/q38/explain.txt | 24 +-
.../approved-plans-v1_4/q38/simplified.txt | 8 +-
.../approved-plans-v1_4/q39a/explain.txt | 12 +-
.../approved-plans-v1_4/q39a/simplified.txt | 4 +-
.../approved-plans-v1_4/q39b/explain.txt | 12 +-
.../approved-plans-v1_4/q39b/simplified.txt | 4 +-
.../approved-plans-v1_4/q4/explain.txt | 36 +-
.../approved-plans-v1_4/q4/simplified.txt | 12 +-
.../approved-plans-v1_4/q40/explain.txt | 18 +-
.../approved-plans-v1_4/q40/simplified.txt | 6 +-
.../approved-plans-v1_4/q41/explain.txt | 12 +-
.../approved-plans-v1_4/q41/simplified.txt | 4 +-
.../approved-plans-v1_4/q42/explain.txt | 6 +-
.../approved-plans-v1_4/q42/simplified.txt | 2 +-
.../approved-plans-v1_4/q43/explain.txt | 6 +-
.../approved-plans-v1_4/q43/simplified.txt | 2 +-
.../approved-plans-v1_4/q44/explain.txt | 18 +-
.../approved-plans-v1_4/q44/simplified.txt | 6 +-
.../approved-plans-v1_4/q45/explain.txt | 108 ++-
.../approved-plans-v1_4/q45/simplified.txt | 98 ++-
.../approved-plans-v1_4/q46/explain.txt | 6 +-
.../approved-plans-v1_4/q46/simplified.txt | 2 +-
.../approved-plans-v1_4/q47/explain.txt | 12 +-
.../approved-plans-v1_4/q47/simplified.txt | 4 +-
.../approved-plans-v1_4/q48/explain.txt | 6 +-
.../approved-plans-v1_4/q48/simplified.txt | 2 +-
.../approved-plans-v1_4/q49/explain.txt | 218 +++---
.../approved-plans-v1_4/q49/simplified.txt | 220 +++---
.../approved-plans-v1_4/q5/explain.txt | 188 +++--
.../approved-plans-v1_4/q5/simplified.txt | 182 +++--
.../approved-plans-v1_4/q50/explain.txt | 6 +-
.../approved-plans-v1_4/q50/simplified.txt | 2 +-
.../approved-plans-v1_4/q51/explain.txt | 199 +++---
.../approved-plans-v1_4/q51/simplified.txt | 122 ++--
.../approved-plans-v1_4/q52/explain.txt | 6 +-
.../approved-plans-v1_4/q52/simplified.txt | 2 +-
.../approved-plans-v1_4/q53/explain.txt | 12 +-
.../approved-plans-v1_4/q53/simplified.txt | 4 +-
.../approved-plans-v1_4/q54/explain.txt | 30 +-
.../approved-plans-v1_4/q54/simplified.txt | 10 +-
.../approved-plans-v1_4/q55/explain.txt | 6 +-
.../approved-plans-v1_4/q55/simplified.txt | 2 +-
.../approved-plans-v1_4/q56/explain.txt | 24 +-
.../approved-plans-v1_4/q56/simplified.txt | 8 +-
.../approved-plans-v1_4/q57/explain.txt | 12 +-
.../approved-plans-v1_4/q57/simplified.txt | 4 +-
.../approved-plans-v1_4/q58/explain.txt | 18 +-
.../approved-plans-v1_4/q58/simplified.txt | 6 +-
.../approved-plans-v1_4/q59/explain.txt | 6 +-
.../approved-plans-v1_4/q59/simplified.txt | 2 +-
.../approved-plans-v1_4/q6/explain.txt | 18 +-
.../approved-plans-v1_4/q6/simplified.txt | 6 +-
.../approved-plans-v1_4/q60/explain.txt | 24 +-
.../approved-plans-v1_4/q60/simplified.txt | 8 +-
.../approved-plans-v1_4/q61/explain.txt | 12 +-
.../approved-plans-v1_4/q61/simplified.txt | 4 +-
.../approved-plans-v1_4/q62/explain.txt | 6 +-
.../approved-plans-v1_4/q62/simplified.txt | 2 +-
.../approved-plans-v1_4/q63/explain.txt | 12 +-
.../approved-plans-v1_4/q63/simplified.txt | 4 +-
.../approved-plans-v1_4/q64/explain.txt | 42 +-
.../approved-plans-v1_4/q64/simplified.txt | 14 +-
.../approved-plans-v1_4/q65/explain.txt | 18 +-
.../approved-plans-v1_4/q65/simplified.txt | 6 +-
.../approved-plans-v1_4/q66/explain.txt | 18 +-
.../approved-plans-v1_4/q66/simplified.txt | 6 +-
.../approved-plans-v1_4/q67/explain.txt | 12 +-
.../approved-plans-v1_4/q67/simplified.txt | 4 +-
.../approved-plans-v1_4/q68/explain.txt | 6 +-
.../approved-plans-v1_4/q68/simplified.txt | 2 +-
.../approved-plans-v1_4/q69/explain.txt | 124 ++--
.../approved-plans-v1_4/q69/simplified.txt | 122 ++--
.../approved-plans-v1_4/q7/explain.txt | 6 +-
.../approved-plans-v1_4/q7/simplified.txt | 2 +-
.../approved-plans-v1_4/q70/explain.txt | 148 ++--
.../approved-plans-v1_4/q70/simplified.txt | 118 ++--
.../approved-plans-v1_4/q71/explain.txt | 6 +-
.../approved-plans-v1_4/q71/simplified.txt | 2 +-
.../approved-plans-v1_4/q72/explain.txt | 219 +++---
.../approved-plans-v1_4/q72/simplified.txt | 172 ++---
.../approved-plans-v1_4/q73/explain.txt | 6 +-
.../approved-plans-v1_4/q73/simplified.txt | 2 +-
.../approved-plans-v1_4/q74/explain.txt | 24 +-
.../approved-plans-v1_4/q74/simplified.txt | 8 +-
.../approved-plans-v1_4/q75/explain.txt | 90 +--
.../approved-plans-v1_4/q75/simplified.txt | 30 +-
.../approved-plans-v1_4/q76/explain.txt | 6 +-
.../approved-plans-v1_4/q76/simplified.txt | 2 +-
.../approved-plans-v1_4/q77/explain.txt | 228 +++---
.../approved-plans-v1_4/q77/simplified.txt | 210 +++---
.../approved-plans-v1_4/q78/explain.txt | 54 +-
.../approved-plans-v1_4/q78/simplified.txt | 18 +-
.../approved-plans-v1_4/q79/explain.txt | 6 +-
.../approved-plans-v1_4/q79/simplified.txt | 2 +-
.../approved-plans-v1_4/q8/explain.txt | 18 +-
.../approved-plans-v1_4/q8/simplified.txt | 6 +-
.../approved-plans-v1_4/q80/explain.txt | 276 ++++----
.../approved-plans-v1_4/q80/simplified.txt | 240 ++++---
.../approved-plans-v1_4/q81/explain.txt | 18 +-
.../approved-plans-v1_4/q81/simplified.txt | 6 +-
.../approved-plans-v1_4/q82/explain.txt | 6 +-
.../approved-plans-v1_4/q82/simplified.txt | 2 +-
.../approved-plans-v1_4/q83/explain.txt | 18 +-
.../approved-plans-v1_4/q83/simplified.txt | 6 +-
.../approved-plans-v1_4/q85/explain.txt | 6 +-
.../approved-plans-v1_4/q85/simplified.txt | 2 +-
.../approved-plans-v1_4/q86/explain.txt | 12 +-
.../approved-plans-v1_4/q86/simplified.txt | 4 +-
.../approved-plans-v1_4/q87/explain.txt | 144 ++--
.../approved-plans-v1_4/q87/simplified.txt | 126 ++--
.../approved-plans-v1_4/q88/explain.txt | 48 +-
.../approved-plans-v1_4/q88/simplified.txt | 16 +-
.../approved-plans-v1_4/q89/explain.txt | 12 +-
.../approved-plans-v1_4/q89/simplified.txt | 4 +-
.../approved-plans-v1_4/q9/explain.txt | 30 +-
.../approved-plans-v1_4/q9/simplified.txt | 10 +-
.../approved-plans-v1_4/q90/explain.txt | 12 +-
.../approved-plans-v1_4/q90/simplified.txt | 4 +-
.../approved-plans-v1_4/q91/explain.txt | 6 +-
.../approved-plans-v1_4/q91/simplified.txt | 2 +-
.../approved-plans-v1_4/q92/explain.txt | 12 +-
.../approved-plans-v1_4/q92/simplified.txt | 4 +-
.../approved-plans-v1_4/q93/explain.txt | 18 +-
.../approved-plans-v1_4/q93/simplified.txt | 6 +-
.../approved-plans-v1_4/q94/explain.txt | 116 ++--
.../approved-plans-v1_4/q94/simplified.txt | 122 ++--
.../approved-plans-v1_4/q95/explain.txt | 148 ++--
.../approved-plans-v1_4/q95/simplified.txt | 178 +++--
.../approved-plans-v1_4/q96/explain.txt | 6 +-
.../approved-plans-v1_4/q96/simplified.txt | 2 +-
.../approved-plans-v1_4/q97/explain.txt | 18 +-
.../approved-plans-v1_4/q97/simplified.txt | 6 +-
.../approved-plans-v1_4/q98/explain.txt | 12 +-
.../approved-plans-v1_4/q98/simplified.txt | 4 +-
.../approved-plans-v1_4/q99/explain.txt | 6 +-
.../approved-plans-v1_4/q99/simplified.txt | 2 +-
.../approved-plans-v2_7-spark3_5/q10a/explain.txt | 6 +-
.../q10a/simplified.txt | 2 +-
.../approved-plans-v2_7-spark3_5/q11/explain.txt | 24 +-
.../q11/simplified.txt | 8 +-
.../approved-plans-v2_7-spark3_5/q12/explain.txt | 12 +-
.../q12/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q14/explain.txt | 24 +-
.../q14/simplified.txt | 8 +-
.../approved-plans-v2_7-spark3_5/q14a/explain.txt | 66 +-
.../q14a/simplified.txt | 22 +-
.../approved-plans-v2_7-spark3_5/q18a/explain.txt | 30 +-
.../q18a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q20/explain.txt | 12 +-
.../q20/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q22/explain.txt | 84 +--
.../q22/simplified.txt | 74 +-
.../approved-plans-v2_7-spark3_5/q22a/explain.txt | 30 +-
.../q22a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q24/explain.txt | 274 ++++----
.../q24/simplified.txt | 164 +++--
.../approved-plans-v2_7-spark3_5/q27a/explain.txt | 18 +-
.../q27a/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q34/explain.txt | 6 +-
.../q34/simplified.txt | 2 +-
.../approved-plans-v2_7-spark3_5/q35/explain.txt | 124 ++--
.../q35/simplified.txt | 126 ++--
.../approved-plans-v2_7-spark3_5/q35a/explain.txt | 6 +-
.../q35a/simplified.txt | 2 +-
.../approved-plans-v2_7-spark3_5/q36a/explain.txt | 30 +-
.../q36a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q47/explain.txt | 12 +-
.../q47/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q49/explain.txt | 218 +++---
.../q49/simplified.txt | 220 +++---
.../approved-plans-v2_7-spark3_5/q51a/explain.txt | 331 ++++-----
.../q51a/simplified.txt | 77 +--
.../approved-plans-v2_7-spark3_5/q57/explain.txt | 12 +-
.../q57/simplified.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q5a/explain.txt | 284 ++++----
.../q5a/simplified.txt | 246 ++++---
.../approved-plans-v2_7-spark3_5/q6/explain.txt | 18 +-
.../approved-plans-v2_7-spark3_5/q6/simplified.txt | 6 +-
.../approved-plans-v2_7-spark3_5/q64/explain.txt | 42 +-
.../q64/simplified.txt | 14 +-
.../approved-plans-v2_7-spark3_5/q67a/explain.txt | 222 +++---
.../q67a/simplified.txt | 156 +++--
.../approved-plans-v2_7-spark3_5/q70a/explain.txt | 236 +++----
.../q70a/simplified.txt | 168 +++--
.../approved-plans-v2_7-spark3_5/q72/explain.txt | 219 +++---
.../q72/simplified.txt | 172 ++---
.../approved-plans-v2_7-spark3_5/q74/explain.txt | 24 +-
.../q74/simplified.txt | 8 +-
.../approved-plans-v2_7-spark3_5/q75/explain.txt | 90 +--
.../q75/simplified.txt | 30 +-
.../approved-plans-v2_7-spark3_5/q77a/explain.txt | 324 ++++-----
.../q77a/simplified.txt | 272 ++++----
.../approved-plans-v2_7-spark3_5/q78/explain.txt | 54 +-
.../q78/simplified.txt | 18 +-
.../approved-plans-v2_7-spark3_5/q80a/explain.txt | 372 +++++-----
.../q80a/simplified.txt | 304 ++++----
.../approved-plans-v2_7-spark3_5/q86a/explain.txt | 30 +-
.../q86a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q98/explain.txt | 12 +-
.../q98/simplified.txt | 4 +-
.../approved-plans-v2_7-spark4_0/q10a/explain.txt | 6 +-
.../q10a/simplified.txt | 2 +-
.../approved-plans-v2_7-spark4_0/q11/explain.txt | 334 ++++-----
.../q11/simplified.txt | 148 ++--
.../approved-plans-v2_7-spark4_0/q12/explain.txt | 100 ++-
.../q12/simplified.txt | 66 +-
.../approved-plans-v2_7-spark4_0/q14/explain.txt | 442 ++++++------
.../q14/simplified.txt | 304 ++++----
.../approved-plans-v2_7-spark4_0/q14a/explain.txt | 656 ++++++++----------
.../q14a/simplified.txt | 438 ++++++------
.../approved-plans-v2_7-spark4_0/q18a/explain.txt | 592 ++++++++--------
.../q18a/simplified.txt | 344 +++++-----
.../approved-plans-v2_7-spark4_0/q20/explain.txt | 100 ++-
.../q20/simplified.txt | 66 +-
.../approved-plans-v2_7-spark4_0/q22/explain.txt | 84 +--
.../q22/simplified.txt | 74 +-
.../approved-plans-v2_7-spark4_0/q22a/explain.txt | 228 +++---
.../q22a/simplified.txt | 134 ++--
.../approved-plans-v2_7-spark4_0/q24/explain.txt | 274 ++++----
.../q24/simplified.txt | 164 +++--
.../approved-plans-v2_7-spark4_0/q27a/explain.txt | 286 ++++----
.../q27a/simplified.txt | 176 +++--
.../approved-plans-v2_7-spark4_0/q34/explain.txt | 6 +-
.../q34/simplified.txt | 2 +-
.../approved-plans-v2_7-spark4_0/q35/explain.txt | 124 ++--
.../q35/simplified.txt | 126 ++--
.../approved-plans-v2_7-spark4_0/q35a/explain.txt | 112 ++-
.../q35a/simplified.txt | 100 ++-
.../approved-plans-v2_7-spark4_0/q36a/explain.txt | 200 +++---
.../q36a/simplified.txt | 118 ++--
.../approved-plans-v2_7-spark4_0/q47/explain.txt | 202 +++---
.../q47/simplified.txt | 92 ++-
.../approved-plans-v2_7-spark4_0/q49/explain.txt | 378 +++++-----
.../q49/simplified.txt | 244 ++++---
.../approved-plans-v2_7-spark4_0/q51a/explain.txt | 397 +++++------
.../q51a/simplified.txt | 171 +++--
.../approved-plans-v2_7-spark4_0/q57/explain.txt | 202 +++---
.../q57/simplified.txt | 92 ++-
.../approved-plans-v2_7-spark4_0/q5a/explain.txt | 402 +++++------
.../q5a/simplified.txt | 250 ++++---
.../approved-plans-v2_7-spark4_0/q6/explain.txt | 182 +++--
.../approved-plans-v2_7-spark4_0/q6/simplified.txt | 138 ++--
.../approved-plans-v2_7-spark4_0/q64/explain.txt | 764 ++++++++++-----------
.../q64/simplified.txt | 425 ++++++------
.../approved-plans-v2_7-spark4_0/q67a/explain.txt | 398 ++++-------
.../q67a/simplified.txt | 254 +++----
.../approved-plans-v2_7-spark4_0/q70a/explain.txt | 268 +++-----
.../q70a/simplified.txt | 174 +++--
.../approved-plans-v2_7-spark4_0/q72/explain.txt | 219 +++---
.../q72/simplified.txt | 172 ++---
.../approved-plans-v2_7-spark4_0/q74/explain.txt | 334 ++++-----
.../q74/simplified.txt | 148 ++--
.../approved-plans-v2_7-spark4_0/q75/explain.txt | 494 +++++++------
.../q75/simplified.txt | 304 ++++----
.../approved-plans-v2_7-spark4_0/q77a/explain.txt | 508 ++++++--------
.../q77a/simplified.txt | 296 ++++----
.../approved-plans-v2_7-spark4_0/q78/explain.txt | 304 ++++----
.../q78/simplified.txt | 154 ++---
.../approved-plans-v2_7-spark4_0/q80a/explain.txt | 510 ++++++--------
.../q80a/simplified.txt | 308 ++++-----
.../approved-plans-v2_7-spark4_0/q86a/explain.txt | 188 ++---
.../q86a/simplified.txt | 106 ++-
.../approved-plans-v2_7-spark4_0/q98/explain.txt | 112 ++-
.../q98/simplified.txt | 66 +-
.../approved-plans-v2_7/q10a/explain.txt | 6 +-
.../approved-plans-v2_7/q10a/simplified.txt | 2 +-
.../approved-plans-v2_7/q11/explain.txt | 24 +-
.../approved-plans-v2_7/q11/simplified.txt | 8 +-
.../approved-plans-v2_7/q12/explain.txt | 12 +-
.../approved-plans-v2_7/q12/simplified.txt | 4 +-
.../approved-plans-v2_7/q14/explain.txt | 24 +-
.../approved-plans-v2_7/q14/simplified.txt | 8 +-
.../approved-plans-v2_7/q14a/explain.txt | 66 +-
.../approved-plans-v2_7/q14a/simplified.txt | 22 +-
.../approved-plans-v2_7/q18a/explain.txt | 30 +-
.../approved-plans-v2_7/q18a/simplified.txt | 10 +-
.../approved-plans-v2_7/q20/explain.txt | 12 +-
.../approved-plans-v2_7/q20/simplified.txt | 4 +-
.../approved-plans-v2_7/q22/explain.txt | 84 +--
.../approved-plans-v2_7/q22/simplified.txt | 74 +-
.../approved-plans-v2_7/q22a/explain.txt | 30 +-
.../approved-plans-v2_7/q22a/simplified.txt | 10 +-
.../approved-plans-v2_7/q24/explain.txt | 274 ++++----
.../approved-plans-v2_7/q24/simplified.txt | 164 +++--
.../approved-plans-v2_7/q27a/explain.txt | 18 +-
.../approved-plans-v2_7/q27a/simplified.txt | 6 +-
.../approved-plans-v2_7/q34/explain.txt | 6 +-
.../approved-plans-v2_7/q34/simplified.txt | 2 +-
.../approved-plans-v2_7/q35/explain.txt | 124 ++--
.../approved-plans-v2_7/q35/simplified.txt | 126 ++--
.../approved-plans-v2_7/q35a/explain.txt | 6 +-
.../approved-plans-v2_7/q35a/simplified.txt | 2 +-
.../approved-plans-v2_7/q36a/explain.txt | 30 +-
.../approved-plans-v2_7/q36a/simplified.txt | 10 +-
.../approved-plans-v2_7/q47/explain.txt | 12 +-
.../approved-plans-v2_7/q47/simplified.txt | 4 +-
.../approved-plans-v2_7/q49/explain.txt | 218 +++---
.../approved-plans-v2_7/q49/simplified.txt | 220 +++---
.../approved-plans-v2_7/q51a/explain.txt | 331 ++++-----
.../approved-plans-v2_7/q51a/simplified.txt | 77 +--
.../approved-plans-v2_7/q57/explain.txt | 12 +-
.../approved-plans-v2_7/q57/simplified.txt | 4 +-
.../approved-plans-v2_7/q5a/explain.txt | 284 ++++----
.../approved-plans-v2_7/q5a/simplified.txt | 246 ++++---
.../approved-plans-v2_7/q6/explain.txt | 18 +-
.../approved-plans-v2_7/q6/simplified.txt | 6 +-
.../approved-plans-v2_7/q64/explain.txt | 42 +-
.../approved-plans-v2_7/q64/simplified.txt | 14 +-
.../approved-plans-v2_7/q67a/explain.txt | 60 +-
.../approved-plans-v2_7/q67a/simplified.txt | 20 +-
.../approved-plans-v2_7/q70a/explain.txt | 234 +++----
.../approved-plans-v2_7/q70a/simplified.txt | 166 +++--
.../approved-plans-v2_7/q72/explain.txt | 219 +++---
.../approved-plans-v2_7/q72/simplified.txt | 172 ++---
.../approved-plans-v2_7/q74/explain.txt | 24 +-
.../approved-plans-v2_7/q74/simplified.txt | 8 +-
.../approved-plans-v2_7/q75/explain.txt | 90 +--
.../approved-plans-v2_7/q75/simplified.txt | 30 +-
.../approved-plans-v2_7/q77a/explain.txt | 324 ++++-----
.../approved-plans-v2_7/q77a/simplified.txt | 272 ++++----
.../approved-plans-v2_7/q78/explain.txt | 54 +-
.../approved-plans-v2_7/q78/simplified.txt | 18 +-
.../approved-plans-v2_7/q80a/explain.txt | 372 +++++-----
.../approved-plans-v2_7/q80a/simplified.txt | 304 ++++----
.../approved-plans-v2_7/q86a/explain.txt | 30 +-
.../approved-plans-v2_7/q86a/simplified.txt | 10 +-
.../approved-plans-v2_7/q98/explain.txt | 12 +-
.../approved-plans-v2_7/q98/simplified.txt | 4 +-
806 files changed, 33471 insertions(+), 38242 deletions(-)
diff --git a/common/src/main/scala/org/apache/comet/CometConf.scala b/common/src/main/scala/org/apache/comet/CometConf.scala
index b02f526e..b8b4799d 100644
--- a/common/src/main/scala/org/apache/comet/CometConf.scala
+++ b/common/src/main/scala/org/apache/comet/CometConf.scala
@@ -197,11 +197,11 @@ object CometConf extends ShimCometConf {
"'native' is for native shuffle which has best performance in general. " +
"'jvm' is for jvm-based columnar shuffle which has higher coverage than native shuffle. " +
"'auto' is for Comet to choose the best shuffle mode based on the query plan. " +
- "By default, this config is 'jvm'.")
+ "By default, this config is 'auto'.")
.stringConf
.transform(_.toLowerCase(Locale.ROOT))
.checkValues(Set("native", "jvm", "auto"))
- .createWithDefault("jvm")
+ .createWithDefault("auto")
val COMET_EXEC_BROADCAST_FORCE_ENABLED: ConfigEntry[Boolean] =
conf(s"$COMET_EXEC_CONFIG_PREFIX.broadcast.enabled")
diff --git a/docs/source/user-guide/configs.md b/docs/source/user-guide/configs.md
index 0679bacb..b45d2588 100644
--- a/docs/source/user-guide/configs.md
+++ b/docs/source/user-guide/configs.md
@@ -51,7 +51,7 @@ Comet provides the following configuration settings.
| spark.comet.exec.project.enabled | Whether to enable project by default. | true |
| spark.comet.exec.shuffle.codec | The codec of Comet native shuffle used to compress shuffle data. Only zstd is supported. | zstd |
| spark.comet.exec.shuffle.enabled | Whether to enable Comet native shuffle. By default, this config is false. Note that this requires setting 'spark.shuffle.manager' to 'org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager'. 'spark.shuffle.manager' must be set before starting the Spark application and cannot be changed during the application. | false |
-| spark.comet.exec.shuffle.mode | The mode of Comet shuffle. This config is only effective if Comet shuffle is enabled. Available modes are 'native', 'jvm', and 'auto'. 'native' is for native shuffle which has best performance in general. 'jvm' is for jvm-based columnar shuffle which has higher coverage than native shuffle. 'auto' is for Comet to choose the best shuffle mode based on the query plan. By default, this config is 'jvm'. | jvm |
+| spark.comet.exec.shuffle.mode | The mode of Comet shuffle. This config is only effective if Comet shuffle is enabled. Available modes are 'native', 'jvm', and 'auto'. 'native' is for native shuffle which has best performance in general. 'jvm' is for jvm-based columnar shuffle which has higher coverage than native shuffle. 'auto' is for Comet to choose the best shuffle mode based on the query plan. By default, this config is 'auto'. | auto |
| spark.comet.exec.sort.enabled | Whether to enable sort by default. | true |
| spark.comet.exec.sortMergeJoin.enabled | Whether to enable sortMergeJoin by default. | true |
| spark.comet.exec.stddev.enabled | Whether to enable stddev by default. stddev is slower than Spark's implementation. | true |
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt
index 3752ee15..707707a7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt
@@ -9,7 +9,7 @@
: : +- CometBroadcastHashJoin (26)
: : :- CometFilter (12)
: : : +- CometHashAggregate (11)
- : : : +- CometColumnarExchange (10)
+ : : : +- CometExchange (10)
: : : +- CometHashAggregate (9)
: : : +- CometProject (8)
: : : +- CometBroadcastHashJoin (7)
@@ -22,10 +22,10 @@
: : +- CometBroadcastExchange (25)
: : +- CometFilter (24)
: : +- CometHashAggregate (23)
- : : +- CometColumnarExchange (22)
+ : : +- CometExchange (22)
: : +- CometHashAggregate (21)
: : +- CometHashAggregate (20)
- : : +- CometColumnarExchange (19)
+ : : +- CometExchange (19)
: : +- CometHashAggregate (18)
: : +- CometProject (17)
: : +- CometBroadcastHashJoin (16)
@@ -86,9 +86,9 @@ Input [3]: [sr_customer_sk#1, sr_store_sk#2, sr_return_amt#3]
Keys [2]: [sr_customer_sk#1, sr_store_sk#2]
Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#3))]
-(10) CometColumnarExchange
+(10) CometExchange
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
-Arguments: hashpartitioning(sr_customer_sk#1, sr_store_sk#2, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(sr_customer_sk#1, sr_store_sk#2, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(11) CometHashAggregate
Input [3]: [sr_customer_sk#1, sr_store_sk#2, sum#8]
@@ -128,9 +128,9 @@ Input [3]: [sr_customer_sk#12, sr_store_sk#13, sr_return_amt#14]
Keys [2]: [sr_customer_sk#12, sr_store_sk#13]
Functions [1]: [partial_sum(UnscaledValue(sr_return_amt#14))]
-(19) CometColumnarExchange
+(19) CometExchange
Input [3]: [sr_customer_sk#12, sr_store_sk#13, sum#18]
-Arguments: hashpartitioning(sr_customer_sk#12, sr_store_sk#13, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(sr_customer_sk#12, sr_store_sk#13, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(20) CometHashAggregate
Input [3]: [sr_customer_sk#12, sr_store_sk#13, sum#18]
@@ -142,9 +142,9 @@ Input [2]: [ctr_store_sk#19, ctr_total_return#20]
Keys [1]: [ctr_store_sk#19]
Functions [1]: [partial_avg(ctr_total_return#20)]
-(22) CometColumnarExchange
+(22) CometExchange
Input [3]: [ctr_store_sk#19, sum#21, count#22]
-Arguments: hashpartitioning(ctr_store_sk#19, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(ctr_store_sk#19, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(23) CometHashAggregate
Input [3]: [ctr_store_sk#19, sum#21, count#22]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt
index 28b22a2e..5aa8750a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt
@@ -10,7 +10,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [ctr_customer_sk,ctr_store_sk,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_store_sk]
CometFilter [ctr_customer_sk,ctr_store_sk,ctr_total_return]
CometHashAggregate [ctr_customer_sk,ctr_store_sk,ctr_total_return,sr_customer_sk,sr_store_sk,sum,sum(UnscaledValue(sr_return_amt))]
- CometColumnarExchange [sr_customer_sk,sr_store_sk] #1
+ CometExchange [sr_customer_sk,sr_store_sk] #1
CometHashAggregate [sr_customer_sk,sr_store_sk,sum,sr_return_amt]
CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
@@ -31,10 +31,10 @@ WholeStageCodegen (1)
CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_store_sk] #4
CometFilter [(avg(ctr_total_return) * 1.2),ctr_store_sk]
CometHashAggregate [(avg(ctr_total_return) * 1.2),ctr_store_sk,sum,count,avg(ctr_total_return)]
- CometColumnarExchange [ctr_store_sk] #5
+ CometExchange [ctr_store_sk] #5
CometHashAggregate [ctr_store_sk,sum,count,ctr_total_return]
CometHashAggregate [ctr_store_sk,ctr_total_return,sr_customer_sk,sr_store_sk,sum,sum(UnscaledValue(sr_return_amt))]
- CometColumnarExchange [sr_customer_sk,sr_store_sk] #6
+ CometExchange [sr_customer_sk,sr_store_sk] #6
CometHashAggregate [sr_customer_sk,sr_store_sk,sum,sr_return_amt]
CometProject [sr_customer_sk,sr_store_sk,sr_return_amt]
CometBroadcastHashJoin [sr_customer_sk,sr_store_sk,sr_return_amt,sr_returned_date_sk,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt
index 6f8c539f..4a29b726 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt
@@ -1,51 +1,49 @@
== Physical Plan ==
-TakeOrderedAndProject (47)
-+- * HashAggregate (46)
- +- * ColumnarToRow (45)
- +- CometColumnarExchange (44)
- +- RowToColumnar (43)
- +- * HashAggregate (42)
- +- * Project (41)
- +- * BroadcastHashJoin Inner BuildRight (40)
- :- * Project (35)
- : +- * BroadcastHashJoin Inner BuildRight (34)
- : :- * Project (28)
- : : +- * Filter (27)
- : : +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (26)
- : : :- * BroadcastHashJoin ExistenceJoin(exists#2) BuildRight (19)
- : : : :- * ColumnarToRow (12)
- : : : : +- CometBroadcastHashJoin (11)
- : : : : :- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.customer (1)
- : : : : +- CometBroadcastExchange (10)
- : : : : +- CometProject (9)
- : : : : +- CometBroadcastHashJoin (8)
- : : : : :- CometScan parquet spark_catalog.default.store_sales (3)
- : : : : +- CometBroadcastExchange (7)
- : : : : +- CometProject (6)
- : : : : +- CometFilter (5)
- : : : : +- CometScan parquet spark_catalog.default.date_dim (4)
- : : : +- BroadcastExchange (18)
- : : : +- * ColumnarToRow (17)
- : : : +- CometProject (16)
- : : : +- CometBroadcastHashJoin (15)
- : : : :- CometScan parquet spark_catalog.default.web_sales (13)
- : : : +- ReusedExchange (14)
- : : +- BroadcastExchange (25)
- : : +- * ColumnarToRow (24)
- : : +- CometProject (23)
- : : +- CometBroadcastHashJoin (22)
- : : :- CometScan parquet spark_catalog.default.catalog_sales (20)
- : : +- ReusedExchange (21)
- : +- BroadcastExchange (33)
- : +- * ColumnarToRow (32)
- : +- CometProject (31)
- : +- CometFilter (30)
- : +- CometScan parquet spark_catalog.default.customer_address (29)
- +- BroadcastExchange (39)
- +- * ColumnarToRow (38)
- +- CometFilter (37)
- +- CometScan parquet spark_catalog.default.customer_demographics (36)
+TakeOrderedAndProject (45)
++- * HashAggregate (44)
+ +- Exchange (43)
+ +- * HashAggregate (42)
+ +- * Project (41)
+ +- * BroadcastHashJoin Inner BuildRight (40)
+ :- * Project (35)
+ : +- * BroadcastHashJoin Inner BuildRight (34)
+ : :- * Project (28)
+ : : +- * Filter (27)
+ : : +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (26)
+ : : :- * BroadcastHashJoin ExistenceJoin(exists#2) BuildRight (19)
+ : : : :- * ColumnarToRow (12)
+ : : : : +- CometBroadcastHashJoin (11)
+ : : : : :- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.customer (1)
+ : : : : +- CometBroadcastExchange (10)
+ : : : : +- CometProject (9)
+ : : : : +- CometBroadcastHashJoin (8)
+ : : : : :- CometScan parquet spark_catalog.default.store_sales (3)
+ : : : : +- CometBroadcastExchange (7)
+ : : : : +- CometProject (6)
+ : : : : +- CometFilter (5)
+ : : : : +- CometScan parquet spark_catalog.default.date_dim (4)
+ : : : +- BroadcastExchange (18)
+ : : : +- * ColumnarToRow (17)
+ : : : +- CometProject (16)
+ : : : +- CometBroadcastHashJoin (15)
+ : : : :- CometScan parquet spark_catalog.default.web_sales (13)
+ : : : +- ReusedExchange (14)
+ : : +- BroadcastExchange (25)
+ : : +- * ColumnarToRow (24)
+ : : +- CometProject (23)
+ : : +- CometBroadcastHashJoin (22)
+ : : :- CometScan parquet spark_catalog.default.catalog_sales (20)
+ : : +- ReusedExchange (21)
+ : +- BroadcastExchange (33)
+ : +- * ColumnarToRow (32)
+ : +- CometProject (31)
+ : +- CometFilter (30)
+ : +- CometScan parquet spark_catalog.default.customer_address (29)
+ +- BroadcastExchange (39)
+ +- * ColumnarToRow (38)
+ +- CometFilter (37)
+ +- CometScan parquet spark_catalog.default.customer_demographics (36)
(1) Scan parquet spark_catalog.default.customer
@@ -245,56 +243,50 @@ Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#31]
Results [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
-(43) RowToColumnar
+(43) Exchange
Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
+Arguments: hashpartitioning(cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, 5), ENSURE_REQUIREMENTS, [plan_id=5]
-(44) CometColumnarExchange
-Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
-Arguments: hashpartitioning(cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(45) ColumnarToRow [codegen id : 6]
-Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
-
-(46) HashAggregate [codegen id : 6]
+(44) HashAggregate [codegen id : 6]
Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
Keys [8]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#33]
Results [14]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, count(1)#33 AS cnt1#34, cd_purchase_estimate#26, count(1)#33 AS cnt2#35, cd_credit_rating#27, count(1)#33 AS cnt3#36, cd_dep_count#28, count(1)#33 AS cnt4#37, cd_dep_employed_count#29, count(1)#33 AS cnt5#38, cd_dep_college_count#30, count(1)#33 AS cnt6#39]
-(47) TakeOrderedAndProject
+(45) TakeOrderedAndProject
Input [14]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cnt1#34, cd_purchase_estimate#26, cnt2#35, cd_credit_rating#27, cnt3#36, cd_dep_count#28, cnt4#37, cd_dep_employed_count#29, cnt5#38, cd_dep_college_count#30, cnt6#39]
Arguments: 100, [cd_gender#23 ASC NULLS FIRST, cd_marital_status#24 ASC NULLS FIRST, cd_education_status#25 ASC NULLS FIRST, cd_purchase_estimate#26 ASC NULLS FIRST, cd_credit_rating#27 ASC NULLS FIRST, cd_dep_count#28 ASC NULLS FIRST, cd_dep_employed_count#29 ASC NULLS FIRST, cd_dep_college_count#30 ASC NULLS FIRST], [cd_gender#23, cd_marital_status#24, cd_education_status#25, cnt1#34, cd_purchase_estimate#26, cnt2#35, cd_credit_rating#27, cnt3#36, cd_dep_count#28, cnt4#37, cd_dep_emplo [...]
===== Subqueries =====
Subquery:1 Hosting operator id = 3 Hosting Expression = ss_sold_date_sk#7 IN dynamicpruning#8
-BroadcastExchange (52)
-+- * ColumnarToRow (51)
- +- CometProject (50)
- +- CometFilter (49)
- +- CometScan parquet spark_catalog.default.date_dim (48)
+BroadcastExchange (50)
++- * ColumnarToRow (49)
+ +- CometProject (48)
+ +- CometFilter (47)
+ +- CometScan parquet spark_catalog.default.date_dim (46)
-(48) Scan parquet spark_catalog.default.date_dim
+(46) Scan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#9, d_year#10, d_moy#11]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_moy), EqualTo(d_year,2002), GreaterThanOrEqual(d_moy,1), LessThanOrEqual(d_moy,4), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(49) CometFilter
+(47) CometFilter
Input [3]: [d_date_sk#9, d_year#10, d_moy#11]
Condition : (((((isnotnull(d_year#10) AND isnotnull(d_moy#11)) AND (d_year#10 = 2002)) AND (d_moy#11 >= 1)) AND (d_moy#11 <= 4)) AND isnotnull(d_date_sk#9))
-(50) CometProject
+(48) CometProject
Input [3]: [d_date_sk#9, d_year#10, d_moy#11]
Arguments: [d_date_sk#9], [d_date_sk#9]
-(51) ColumnarToRow [codegen id : 1]
+(49) ColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#9]
-(52) BroadcastExchange
+(50) BroadcastExchange
Input [1]: [d_date_sk#9]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt
index a3ab6bbb..efd4b187 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt
@@ -1,73 +1,71 @@
TakeOrderedAndProject [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count,cnt1,cnt2,cnt3,cnt4,cnt5,cnt6]
WholeStageCodegen (6)
HashAggregate [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count,count] [count(1),cnt1,cnt2,cnt3,cnt4,cnt5,cnt6,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] #1
- RowToColumnar
- WholeStageCodegen (5)
- HashAggregate [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] [count,count]
- Project [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
- BroadcastHashJoin [c_current_cdemo_sk,cd_demo_sk]
- Project [c_current_cdemo_sk]
- BroadcastHashJoin [c_current_addr_sk,ca_address_sk]
- Project [c_current_cdemo_sk,c_current_addr_sk]
- Filter [exists,exists]
- BroadcastHashJoin [c_customer_sk,cs_ship_customer_sk]
- BroadcastHashJoin [c_customer_sk,ws_bill_customer_sk]
+ InputAdapter
+ Exchange [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] #1
+ WholeStageCodegen (5)
+ HashAggregate [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] [count,count]
+ Project [cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ BroadcastHashJoin [c_current_cdemo_sk,cd_demo_sk]
+ Project [c_current_cdemo_sk]
+ BroadcastHashJoin [c_current_addr_sk,ca_address_sk]
+ Project [c_current_cdemo_sk,c_current_addr_sk]
+ Filter [exists,exists]
+ BroadcastHashJoin [c_customer_sk,cs_ship_customer_sk]
+ BroadcastHashJoin [c_customer_sk,ws_bill_customer_sk]
+ ColumnarToRow
+ InputAdapter
+ CometBroadcastHashJoin [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk,ss_customer_sk]
+ CometFilter [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
+ CometBroadcastExchange [ss_customer_sk] #2
+ CometProject [ss_customer_sk]
+ CometBroadcastHashJoin [ss_customer_sk,ss_sold_date_sk,d_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_moy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
+ CometBroadcastExchange [d_date_sk] #4
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_moy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
+ InputAdapter
+ BroadcastExchange #5
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [ws_bill_customer_sk]
+ CometBroadcastHashJoin [ws_bill_customer_sk,ws_sold_date_sk,d_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #4
+ InputAdapter
+ BroadcastExchange #6
+ WholeStageCodegen (2)
ColumnarToRow
InputAdapter
- CometBroadcastHashJoin [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk,ss_customer_sk]
- CometFilter [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
- CometBroadcastExchange [ss_customer_sk] #2
- CometProject [ss_customer_sk]
- CometBroadcastHashJoin [ss_customer_sk,ss_sold_date_sk,d_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_moy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
- CometBroadcastExchange [d_date_sk] #4
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_moy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [ws_bill_customer_sk]
- CometBroadcastHashJoin [ws_bill_customer_sk,ws_sold_date_sk,d_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #4
- InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (2)
- ColumnarToRow
- InputAdapter
- CometProject [cs_ship_customer_sk]
- CometBroadcastHashJoin [cs_ship_customer_sk,cs_sold_date_sk,d_date_sk]
- CometScan parquet spark_catalog.default.catalog_sales [cs_ship_customer_sk,cs_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #4
- InputAdapter
- BroadcastExchange #7
- WholeStageCodegen (3)
- ColumnarToRow
- InputAdapter
- CometProject [ca_address_sk]
- CometFilter [ca_address_sk,ca_county]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county]
+ CometProject [cs_ship_customer_sk]
+ CometBroadcastHashJoin [cs_ship_customer_sk,cs_sold_date_sk,d_date_sk]
+ CometScan parquet spark_catalog.default.catalog_sales [cs_ship_customer_sk,cs_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #4
InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (4)
+ BroadcastExchange #7
+ WholeStageCodegen (3)
ColumnarToRow
InputAdapter
- CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
- CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometProject [ca_address_sk]
+ CometFilter [ca_address_sk,ca_county]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county]
+ InputAdapter
+ BroadcastExchange #8
+ WholeStageCodegen (4)
+ ColumnarToRow
+ InputAdapter
+ CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt
index ac0d82f8..0346efa5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt
@@ -9,7 +9,7 @@
: : +- CometBroadcastHashJoin (33)
: : :- CometFilter (16)
: : : +- CometHashAggregate (15)
- : : : +- CometColumnarExchange (14)
+ : : : +- CometExchange (14)
: : : +- CometHashAggregate (13)
: : : +- CometProject (12)
: : : +- CometBroadcastHashJoin (11)
@@ -25,7 +25,7 @@
: : : +- CometScan parquet spark_catalog.default.date_dim (8)
: : +- CometBroadcastExchange (32)
: : +- CometHashAggregate (31)
- : : +- CometColumnarExchange (30)
+ : : +- CometExchange (30)
: : +- CometHashAggregate (29)
: : +- CometProject (28)
: : +- CometBroadcastHashJoin (27)
@@ -42,7 +42,7 @@
: +- CometBroadcastExchange (49)
: +- CometFilter (48)
: +- CometHashAggregate (47)
- : +- CometColumnarExchange (46)
+ : +- CometExchange (46)
: +- CometHashAggregate (45)
: +- CometProject (44)
: +- CometBroadcastHashJoin (43)
@@ -56,7 +56,7 @@
: +- ReusedExchange (42)
+- CometBroadcastExchange (65)
+- CometHashAggregate (64)
- +- CometColumnarExchange (63)
+ +- CometExchange (63)
+- CometHashAggregate (62)
+- CometProject (61)
+- CometBroadcastHashJoin (60)
@@ -135,9 +135,9 @@ Input [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_fl
Keys [8]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#11 - ss_ext_discount_amt#10)))]
-(14) CometColumnarExchange
+(14) CometExchange
Input [9]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, sum#16]
-Arguments: hashpartitioning(c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(15) CometHashAggregate
Input [9]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, sum#16]
@@ -213,9 +213,9 @@ Input [10]: [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust
Keys [8]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26]
Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#29 - ss_ext_discount_amt#28)))]
-(30) CometColumnarExchange
+(30) CometExchange
Input [9]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, sum#34]
-Arguments: hashpartitioning(c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(31) CometHashAggregate
Input [9]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, sum#34]
@@ -288,9 +288,9 @@ Input [10]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust
Keys [8]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52]
Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#48 - ws_ext_discount_amt#47)))]
-(46) CometColumnarExchange
+(46) CometExchange
Input [9]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, sum#53]
-Arguments: hashpartitioning(c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(47) CometHashAggregate
Input [9]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, sum#53]
@@ -367,9 +367,9 @@ Input [10]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust
Keys [8]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70]
Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
-(63) CometColumnarExchange
+(63) CometExchange
Input [9]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, sum#71]
-Arguments: hashpartitioning(c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(64) CometHashAggregate
Input [9]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, sum#71]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt
index 83ffbd0a..e509c8c3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt
@@ -10,7 +10,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [customer_id,year_total,customer_id,customer_preferred_cust_flag,year_total]
CometFilter [customer_id,year_total]
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum,sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt)))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #1
+ CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #1
CometHashAggregate [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum,ss_ext_list_price,ss_ext_discount_amt]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
@@ -33,7 +33,7 @@ WholeStageCodegen (1)
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [customer_id,customer_preferred_cust_flag,year_total] #5
CometHashAggregate [customer_id,customer_preferred_cust_flag,year_total,c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum,sum(UnscaledValue((ss_ext_list_price - ss_ext_discount_amt)))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #6
+ CometExchange [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address] #6
CometHashAggregate [c_customer_id,c_first_name,c_last_name,d_year,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,sum,ss_ext_list_price,ss_ext_discount_amt]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
@@ -57,7 +57,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [customer_id,year_total] #10
CometFilter [customer_id,year_total]
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt)))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,ws_ext_list_price,ws_ext_discount_amt]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
@@ -72,7 +72,7 @@ WholeStageCodegen (1)
ReusedExchange [d_date_sk,d_year] #4
CometBroadcastExchange [customer_id,year_total] #13
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,sum(UnscaledValue((ws_ext_list_price - ws_ext_discount_amt)))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,ws_ext_list_price,ws_ext_discount_amt]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt
index 40c6660a..572f500c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt
@@ -4,9 +4,9 @@ TakeOrderedAndProject (22)
+- Window (20)
+- * ColumnarToRow (19)
+- CometSort (18)
- +- CometColumnarExchange (17)
+ +- CometExchange (17)
+- CometHashAggregate (16)
- +- CometColumnarExchange (15)
+ +- CometExchange (15)
+- CometHashAggregate (14)
+- CometProject (13)
+- CometBroadcastHashJoin (12)
@@ -92,18 +92,18 @@ Input [6]: [ws_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8,
Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#2))]
-(15) CometColumnarExchange
+(15) CometExchange
Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
-Arguments: hashpartitioning(i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(16) CometHashAggregate
Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#2))]
-(17) CometColumnarExchange
+(17) CometExchange
Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: hashpartitioning(i_class#9, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_class#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(18) CometSort
Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt
index 3e0fd16a..e81a5071 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt
@@ -7,9 +7,9 @@ TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_c
ColumnarToRow
InputAdapter
CometSort [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id]
- CometColumnarExchange [i_class] #1
+ CometExchange [i_class] #1
CometHashAggregate [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
+ CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
CometHashAggregate [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
CometBroadcastHashJoin [ws_ext_sales_price,ws_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt
index b05df292..7d035666 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
* ColumnarToRow (33)
+- CometHashAggregate (32)
- +- CometColumnarExchange (31)
+ +- CometExchange (31)
+- CometHashAggregate (30)
+- CometProject (29)
+- CometBroadcastHashJoin (28)
@@ -179,9 +179,9 @@ Input [3]: [ss_quantity#5, ss_ext_sales_price#7, ss_ext_wholesale_cost#8]
Keys: []
Functions [4]: [partial_avg(ss_quantity#5), partial_avg(UnscaledValue(ss_ext_sales_price#7)), partial_avg(UnscaledValue(ss_ext_wholesale_cost#8)), partial_sum(UnscaledValue(ss_ext_wholesale_cost#8))]
-(31) CometColumnarExchange
+(31) CometExchange
Input [7]: [sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(32) CometHashAggregate
Input [7]: [sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt
index 0f438f15..46ed9b63 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt
@@ -2,7 +2,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [avg(ss_quantity),avg(ss_ext_sales_price),avg(ss_ext_wholesale_cost),sum(ss_ext_wholesale_cost),sum,count,sum,count,sum,count,sum,avg(ss_quantity),avg(UnscaledValue(ss_ext_sales_price)),avg(UnscaledValue(ss_ext_wholesale_cost)),sum(UnscaledValue(ss_ext_wholesale_cost))]
- CometColumnarExchange #1
+ CometExchange #1
CometHashAggregate [sum,count,sum,count,sum,count,sum,ss_quantity,ss_ext_sales_price,ss_ext_wholesale_cost]
CometProject [ss_quantity,ss_ext_sales_price,ss_ext_wholesale_cost]
CometBroadcastHashJoin [ss_hdemo_sk,ss_quantity,ss_sales_price,ss_ext_sales_price,ss_ext_wholesale_cost,cd_marital_status,cd_education_status,hd_demo_sk,hd_dep_count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/explain.txt
index 81c7b9c4..c758aeb6 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/explain.txt
@@ -2,14 +2,14 @@
* ColumnarToRow (102)
+- CometTakeOrderedAndProject (101)
+- CometHashAggregate (100)
- +- CometColumnarExchange (99)
+ +- CometExchange (99)
+- CometHashAggregate (98)
+- CometExpand (97)
+- CometUnion (96)
:- CometProject (65)
: +- CometFilter (64)
: +- CometHashAggregate (63)
- : +- CometColumnarExchange (62)
+ : +- CometExchange (62)
: +- CometHashAggregate (61)
: +- CometProject (60)
: +- CometBroadcastHashJoin (59)
@@ -26,7 +26,7 @@
: : : +- CometBroadcastExchange (43)
: : : +- CometBroadcastHashJoin (42)
: : : :- CometHashAggregate (32)
- : : : : +- CometColumnarExchange (31)
+ : : : : +- CometExchange (31)
: : : : +- CometHashAggregate (30)
: : : : +- CometProject (29)
: : : : +- CometBroadcastHashJoin (28)
@@ -74,7 +74,7 @@
:- CometProject (80)
: +- CometFilter (79)
: +- CometHashAggregate (78)
- : +- CometColumnarExchange (77)
+ : +- CometExchange (77)
: +- CometHashAggregate (76)
: +- CometProject (75)
: +- CometBroadcastHashJoin (74)
@@ -89,7 +89,7 @@
+- CometProject (95)
+- CometFilter (94)
+- CometHashAggregate (93)
- +- CometColumnarExchange (92)
+ +- CometExchange (92)
+- CometHashAggregate (91)
+- CometProject (90)
+- CometBroadcastHashJoin (89)
@@ -252,9 +252,9 @@ Input [3]: [brand_id#27, class_id#28, category_id#29]
Keys [3]: [brand_id#27, class_id#28, category_id#29]
Functions: []
-(31) CometColumnarExchange
+(31) CometExchange
Input [3]: [brand_id#27, class_id#28, category_id#29]
-Arguments: hashpartitioning(brand_id#27, class_id#28, category_id#29, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(brand_id#27, class_id#28, category_id#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(32) CometHashAggregate
Input [3]: [brand_id#27, class_id#28, category_id#29]
@@ -393,9 +393,9 @@ Input [5]: [ss_quantity#2, ss_list_price#3, i_brand_id#40, i_class_id#41, i_cate
Keys [3]: [i_brand_id#40, i_class_id#41, i_category_id#42]
Functions [2]: [partial_sum((cast(ss_quantity#2 as decimal(10,0)) * ss_list_price#3)), partial_count(1)]
-(62) CometColumnarExchange
+(62) CometExchange
Input [6]: [i_brand_id#40, i_class_id#41, i_category_id#42, sum#46, isEmpty#47, count#48]
-Arguments: hashpartitioning(i_brand_id#40, i_class_id#41, i_category_id#42, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_brand_id#40, i_class_id#41, i_category_id#42, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(63) CometHashAggregate
Input [6]: [i_brand_id#40, i_class_id#41, i_category_id#42, sum#46, isEmpty#47, count#48]
@@ -459,9 +459,9 @@ Input [5]: [cs_quantity#58, cs_list_price#59, i_brand_id#64, i_class_id#65, i_ca
Keys [3]: [i_brand_id#64, i_class_id#65, i_category_id#66]
Functions [2]: [partial_sum((cast(cs_quantity#58 as decimal(10,0)) * cs_list_price#59)), partial_count(1)]
-(77) CometColumnarExchange
+(77) CometExchange
Input [6]: [i_brand_id#64, i_class_id#65, i_category_id#66, sum#68, isEmpty#69, count#70]
-Arguments: hashpartitioning(i_brand_id#64, i_class_id#65, i_category_id#66, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(i_brand_id#64, i_class_id#65, i_category_id#66, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(78) CometHashAggregate
Input [6]: [i_brand_id#64, i_class_id#65, i_category_id#66, sum#68, isEmpty#69, count#70]
@@ -525,9 +525,9 @@ Input [5]: [ws_quantity#75, ws_list_price#76, i_brand_id#81, i_class_id#82, i_ca
Keys [3]: [i_brand_id#81, i_class_id#82, i_category_id#83]
Functions [2]: [partial_sum((cast(ws_quantity#75 as decimal(10,0)) * ws_list_price#76)), partial_count(1)]
-(92) CometColumnarExchange
+(92) CometExchange
Input [6]: [i_brand_id#81, i_class_id#82, i_category_id#83, sum#85, isEmpty#86, count#87]
-Arguments: hashpartitioning(i_brand_id#81, i_class_id#82, i_category_id#83, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(i_brand_id#81, i_class_id#82, i_category_id#83, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(93) CometHashAggregate
Input [6]: [i_brand_id#81, i_class_id#82, i_category_id#83, sum#85, isEmpty#86, count#87]
@@ -556,9 +556,9 @@ Input [7]: [sales#49, number_sales#50, channel#91, i_brand_id#92, i_class_id#93,
Keys [5]: [channel#91, i_brand_id#92, i_class_id#93, i_category_id#94, spark_grouping_id#95]
Functions [2]: [partial_sum(sales#49), partial_sum(number_sales#50)]
-(99) CometColumnarExchange
+(99) CometExchange
Input [8]: [channel#91, i_brand_id#92, i_class_id#93, i_category_id#94, spark_grouping_id#95, sum#96, isEmpty#97, sum#98]
-Arguments: hashpartitioning(channel#91, i_brand_id#92, i_class_id#93, i_category_id#94, spark_grouping_id#95, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(channel#91, i_brand_id#92, i_class_id#93, i_category_id#94, spark_grouping_id#95, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(100) CometHashAggregate
Input [8]: [channel#91, i_brand_id#92, i_class_id#93, i_category_id#94, spark_grouping_id#95, sum#96, isEmpty#97, sum#98]
@@ -577,7 +577,7 @@ Input [6]: [channel#91, i_brand_id#92, i_class_id#93, i_category_id#94, sum(sale
Subquery:1 Hosting operator id = 64 Hosting Expression = Subquery scalar-subquery#51, [id=#52]
* ColumnarToRow (119)
+- CometHashAggregate (118)
- +- CometColumnarExchange (117)
+ +- CometExchange (117)
+- CometHashAggregate (116)
+- CometUnion (115)
:- CometProject (106)
@@ -661,9 +661,9 @@ Input [2]: [quantity#106, list_price#107]
Keys: []
Functions [1]: [partial_avg((cast(quantity#106 as decimal(10,0)) * list_price#107))]
-(117) CometColumnarExchange
+(117) CometExchange
Input [2]: [sum#122, count#123]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
(118) CometHashAggregate
Input [2]: [sum#122, count#123]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/simplified.txt
index bb69fc92..852af8ad 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14a/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
CometHashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales),spark_grouping_id,sum,isEmpty,sum,sum(sales),sum(number_salesL)]
- CometColumnarExchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
+ CometExchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
CometHashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum,sales,number_sales]
CometExpand [channel,i_brand_id,i_class_id,i_category_id] [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id]
CometUnion [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
@@ -14,7 +14,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [average_sales,sum,count,avg((cast(quantity as decimal(10,0)) * list_price))]
- CometColumnarExchange #15
+ CometExchange #15
CometHashAggregate [sum,count,quantity,list_price]
CometUnion [quantity,list_price]
CometProject [ss_quantity,ss_list_price] [quantity,list_price]
@@ -33,7 +33,7 @@ WholeStageCodegen (1)
ReusedSubquery [d_date_sk] #2
ReusedExchange [d_date_sk] #11
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sales,number_sales,sum,isEmpty,count,sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1)]
- CometColumnarExchange [i_brand_id,i_class_id,i_category_id] #2
+ CometExchange [i_brand_id,i_class_id,i_category_id] #2
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count,ss_quantity,ss_list_price]
CometProject [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
CometBroadcastHashJoin [ss_quantity,ss_list_price,ss_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
@@ -58,7 +58,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [brand_id,class_id,category_id] #5
CometBroadcastHashJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
CometHashAggregate [brand_id,class_id,category_id]
- CometColumnarExchange [brand_id,class_id,category_id] #6
+ CometExchange [brand_id,class_id,category_id] #6
CometHashAggregate [brand_id,class_id,category_id]
CometProject [i_brand_id,i_class_id,i_category_id] [brand_id,class_id,category_id]
CometBroadcastHashJoin [ss_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
@@ -117,7 +117,7 @@ WholeStageCodegen (1)
CometFilter [i_brand_id,i_class_id,i_category_id,sales,number_sales]
ReusedSubquery [average_sales] #3
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sales,number_sales,sum,isEmpty,count,sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1)]
- CometColumnarExchange [i_brand_id,i_class_id,i_category_id] #16
+ CometExchange [i_brand_id,i_class_id,i_category_id] #16
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count,cs_quantity,cs_list_price]
CometProject [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
CometBroadcastHashJoin [cs_quantity,cs_list_price,cs_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
@@ -134,7 +134,7 @@ WholeStageCodegen (1)
CometFilter [i_brand_id,i_class_id,i_category_id,sales,number_sales]
ReusedSubquery [average_sales] #3
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sales,number_sales,sum,isEmpty,count,sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1)]
- CometColumnarExchange [i_brand_id,i_class_id,i_category_id] #17
+ CometExchange [i_brand_id,i_class_id,i_category_id] #17
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count,ws_quantity,ws_list_price]
CometProject [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
CometBroadcastHashJoin [ws_quantity,ws_list_price,ws_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/explain.txt
index 7bb99e44..95ed68f7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/explain.txt
@@ -4,7 +4,7 @@
+- CometBroadcastHashJoin (83)
:- CometFilter (64)
: +- CometHashAggregate (63)
- : +- CometColumnarExchange (62)
+ : +- CometExchange (62)
: +- CometHashAggregate (61)
: +- CometProject (60)
: +- CometBroadcastHashJoin (59)
@@ -21,7 +21,7 @@
: : : +- CometBroadcastExchange (43)
: : : +- CometBroadcastHashJoin (42)
: : : :- CometHashAggregate (32)
- : : : : +- CometColumnarExchange (31)
+ : : : : +- CometExchange (31)
: : : : +- CometHashAggregate (30)
: : : : +- CometProject (29)
: : : : +- CometBroadcastHashJoin (28)
@@ -69,7 +69,7 @@
+- CometBroadcastExchange (82)
+- CometFilter (81)
+- CometHashAggregate (80)
- +- CometColumnarExchange (79)
+ +- CometExchange (79)
+- CometHashAggregate (78)
+- CometProject (77)
+- CometBroadcastHashJoin (76)
@@ -235,9 +235,9 @@ Input [3]: [brand_id#27, class_id#28, category_id#29]
Keys [3]: [brand_id#27, class_id#28, category_id#29]
Functions: []
-(31) CometColumnarExchange
+(31) CometExchange
Input [3]: [brand_id#27, class_id#28, category_id#29]
-Arguments: hashpartitioning(brand_id#27, class_id#28, category_id#29, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(brand_id#27, class_id#28, category_id#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(32) CometHashAggregate
Input [3]: [brand_id#27, class_id#28, category_id#29]
@@ -376,9 +376,9 @@ Input [5]: [ss_quantity#2, ss_list_price#3, i_brand_id#40, i_class_id#41, i_cate
Keys [3]: [i_brand_id#40, i_class_id#41, i_category_id#42]
Functions [2]: [partial_sum((cast(ss_quantity#2 as decimal(10,0)) * ss_list_price#3)), partial_count(1)]
-(62) CometColumnarExchange
+(62) CometExchange
Input [6]: [i_brand_id#40, i_class_id#41, i_category_id#42, sum#47, isEmpty#48, count#49]
-Arguments: hashpartitioning(i_brand_id#40, i_class_id#41, i_category_id#42, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_brand_id#40, i_class_id#41, i_category_id#42, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(63) CometHashAggregate
Input [6]: [i_brand_id#40, i_class_id#41, i_category_id#42, sum#47, isEmpty#48, count#49]
@@ -454,9 +454,9 @@ Input [5]: [ss_quantity#56, ss_list_price#57, i_brand_id#62, i_class_id#63, i_ca
Keys [3]: [i_brand_id#62, i_class_id#63, i_category_id#64]
Functions [2]: [partial_sum((cast(ss_quantity#56 as decimal(10,0)) * ss_list_price#57)), partial_count(1)]
-(79) CometColumnarExchange
+(79) CometExchange
Input [6]: [i_brand_id#62, i_class_id#63, i_category_id#64, sum#69, isEmpty#70, count#71]
-Arguments: hashpartitioning(i_brand_id#62, i_class_id#63, i_category_id#64, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(i_brand_id#62, i_class_id#63, i_category_id#64, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(80) CometHashAggregate
Input [6]: [i_brand_id#62, i_class_id#63, i_category_id#64, sum#69, isEmpty#70, count#71]
@@ -488,7 +488,7 @@ Input [12]: [channel#50, i_brand_id#40, i_class_id#41, i_category_id#42, sales#5
Subquery:1 Hosting operator id = 64 Hosting Expression = Subquery scalar-subquery#53, [id=#54]
* ColumnarToRow (102)
+- CometHashAggregate (101)
- +- CometColumnarExchange (100)
+ +- CometExchange (100)
+- CometHashAggregate (99)
+- CometUnion (98)
:- CometProject (89)
@@ -572,9 +572,9 @@ Input [2]: [quantity#80, list_price#81]
Keys: []
Functions [1]: [partial_avg((cast(quantity#80 as decimal(10,0)) * list_price#81))]
-(100) CometColumnarExchange
+(100) CometExchange
Input [2]: [sum#96, count#97]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(101) CometHashAggregate
Input [2]: [sum#96, count#97]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/simplified.txt
index a51d1d00..bb4be4d3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q14b/simplified.txt
@@ -9,7 +9,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [average_sales,sum,count,avg((cast(quantity as decimal(10,0)) * list_price))]
- CometColumnarExchange #14
+ CometExchange #14
CometHashAggregate [sum,count,quantity,list_price]
CometUnion [quantity,list_price]
CometProject [ss_quantity,ss_list_price] [quantity,list_price]
@@ -28,7 +28,7 @@ WholeStageCodegen (1)
ReusedSubquery [d_date_sk] #3
ReusedExchange [d_date_sk] #10
CometHashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales,sum,isEmpty,count,sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1)]
- CometColumnarExchange [i_brand_id,i_class_id,i_category_id] #1
+ CometExchange [i_brand_id,i_class_id,i_category_id] #1
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count,ss_quantity,ss_list_price]
CometProject [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
CometBroadcastHashJoin [ss_quantity,ss_list_price,ss_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
@@ -60,7 +60,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [brand_id,class_id,category_id] #4
CometBroadcastHashJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
CometHashAggregate [brand_id,class_id,category_id]
- CometColumnarExchange [brand_id,class_id,category_id] #5
+ CometExchange [brand_id,class_id,category_id] #5
CometHashAggregate [brand_id,class_id,category_id]
CometProject [i_brand_id,i_class_id,i_category_id] [brand_id,class_id,category_id]
CometBroadcastHashJoin [ss_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
@@ -120,7 +120,7 @@ WholeStageCodegen (1)
CometFilter [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
ReusedSubquery [average_sales] #4
CometHashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales,sum,isEmpty,count,sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1)]
- CometColumnarExchange [i_brand_id,i_class_id,i_category_id] #16
+ CometExchange [i_brand_id,i_class_id,i_category_id] #16
CometHashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count,ss_quantity,ss_list_price]
CometProject [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
CometBroadcastHashJoin [ss_quantity,ss_list_price,ss_sold_date_sk,i_brand_id,i_class_id,i_category_id,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt
index af9c0909..842c3e57 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (23)
+- CometTakeOrderedAndProject (22)
+- CometHashAggregate (21)
- +- CometColumnarExchange (20)
+ +- CometExchange (20)
+- CometHashAggregate (19)
+- CometProject (18)
+- CometBroadcastHashJoin (17)
@@ -117,9 +117,9 @@ Input [2]: [cs_sales_price#2, ca_zip#9]
Keys [1]: [ca_zip#9]
Functions [1]: [partial_sum(UnscaledValue(cs_sales_price#2))]
-(20) CometColumnarExchange
+(20) CometExchange
Input [2]: [ca_zip#9, sum#13]
-Arguments: hashpartitioning(ca_zip#9, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ca_zip#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(21) CometHashAggregate
Input [2]: [ca_zip#9, sum#13]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt
index 989ea085..d417f1c6 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [ca_zip,sum(cs_sales_price)]
CometHashAggregate [ca_zip,sum(cs_sales_price),sum,sum(UnscaledValue(cs_sales_price))]
- CometColumnarExchange [ca_zip] #1
+ CometExchange [ca_zip] #1
CometHashAggregate [ca_zip,sum,cs_sales_price]
CometProject [cs_sales_price,ca_zip]
CometBroadcastHashJoin [cs_sales_price,cs_sold_date_sk,ca_zip,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt
index ebff95d0..4890b443 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt
@@ -1,51 +1,49 @@
== Physical Plan ==
-* HashAggregate (47)
-+- * ColumnarToRow (46)
- +- CometColumnarExchange (45)
- +- RowToColumnar (44)
- +- * HashAggregate (43)
- +- * HashAggregate (42)
- +- * HashAggregate (41)
- +- * Project (40)
- +- * BroadcastHashJoin Inner BuildRight (39)
- :- * Project (33)
- : +- * BroadcastHashJoin Inner BuildRight (32)
- : :- * Project (26)
- : : +- * BroadcastHashJoin Inner BuildRight (25)
- : : :- * SortMergeJoin LeftAnti (19)
- : : : :- * Project (13)
- : : : : +- * SortMergeJoin LeftSemi (12)
- : : : : :- * ColumnarToRow (6)
- : : : : : +- CometSort (5)
- : : : : : +- CometColumnarExchange (4)
- : : : : : +- CometProject (3)
- : : : : : +- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.catalog_sales (1)
- : : : : +- * ColumnarToRow (11)
- : : : : +- CometSort (10)
- : : : : +- CometColumnarExchange (9)
- : : : : +- CometProject (8)
- : : : : +- CometScan parquet spark_catalog.default.catalog_sales (7)
- : : : +- * ColumnarToRow (18)
- : : : +- CometSort (17)
- : : : +- CometColumnarExchange (16)
- : : : +- CometProject (15)
- : : : +- CometScan parquet spark_catalog.default.catalog_returns (14)
- : : +- BroadcastExchange (24)
- : : +- * ColumnarToRow (23)
- : : +- CometProject (22)
- : : +- CometFilter (21)
- : : +- CometScan parquet spark_catalog.default.date_dim (20)
- : +- BroadcastExchange (31)
- : +- * ColumnarToRow (30)
- : +- CometProject (29)
- : +- CometFilter (28)
- : +- CometScan parquet spark_catalog.default.customer_address (27)
- +- BroadcastExchange (38)
- +- * ColumnarToRow (37)
- +- CometProject (36)
- +- CometFilter (35)
- +- CometScan parquet spark_catalog.default.call_center (34)
+* HashAggregate (45)
++- Exchange (44)
+ +- * HashAggregate (43)
+ +- * HashAggregate (42)
+ +- * HashAggregate (41)
+ +- * Project (40)
+ +- * BroadcastHashJoin Inner BuildRight (39)
+ :- * Project (33)
+ : +- * BroadcastHashJoin Inner BuildRight (32)
+ : :- * Project (26)
+ : : +- * BroadcastHashJoin Inner BuildRight (25)
+ : : :- * SortMergeJoin LeftAnti (19)
+ : : : :- * Project (13)
+ : : : : +- * SortMergeJoin LeftSemi (12)
+ : : : : :- * ColumnarToRow (6)
+ : : : : : +- CometSort (5)
+ : : : : : +- CometExchange (4)
+ : : : : : +- CometProject (3)
+ : : : : : +- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.catalog_sales (1)
+ : : : : +- * ColumnarToRow (11)
+ : : : : +- CometSort (10)
+ : : : : +- CometExchange (9)
+ : : : : +- CometProject (8)
+ : : : : +- CometScan parquet spark_catalog.default.catalog_sales (7)
+ : : : +- * ColumnarToRow (18)
+ : : : +- CometSort (17)
+ : : : +- CometExchange (16)
+ : : : +- CometProject (15)
+ : : : +- CometScan parquet spark_catalog.default.catalog_returns (14)
+ : : +- BroadcastExchange (24)
+ : : +- * ColumnarToRow (23)
+ : : +- CometProject (22)
+ : : +- CometFilter (21)
+ : : +- CometScan parquet spark_catalog.default.date_dim (20)
+ : +- BroadcastExchange (31)
+ : +- * ColumnarToRow (30)
+ : +- CometProject (29)
+ : +- CometFilter (28)
+ : +- CometScan parquet spark_catalog.default.customer_address (27)
+ +- BroadcastExchange (38)
+ +- * ColumnarToRow (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan parquet spark_catalog.default.call_center (34)
(1) Scan parquet spark_catalog.default.catalog_sales
@@ -63,9 +61,9 @@ Condition : ((isnotnull(cs_ship_date_sk#1) AND isnotnull(cs_ship_addr_sk#2)) AND
Input [8]: [cs_ship_date_sk#1, cs_ship_addr_sk#2, cs_call_center_sk#3, cs_warehouse_sk#4, cs_order_number#5, cs_ext_ship_cost#6, cs_net_profit#7, cs_sold_date_sk#8]
Arguments: [cs_ship_date_sk#1, cs_ship_addr_sk#2, cs_call_center_sk#3, cs_warehouse_sk#4, cs_order_number#5, cs_ext_ship_cost#6, cs_net_profit#7], [cs_ship_date_sk#1, cs_ship_addr_sk#2, cs_call_center_sk#3, cs_warehouse_sk#4, cs_order_number#5, cs_ext_ship_cost#6, cs_net_profit#7]
-(4) CometColumnarExchange
+(4) CometExchange
Input [7]: [cs_ship_date_sk#1, cs_ship_addr_sk#2, cs_call_center_sk#3, cs_warehouse_sk#4, cs_order_number#5, cs_ext_ship_cost#6, cs_net_profit#7]
-Arguments: hashpartitioning(cs_order_number#5, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(cs_order_number#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(5) CometSort
Input [7]: [cs_ship_date_sk#1, cs_ship_addr_sk#2, cs_call_center_sk#3, cs_warehouse_sk#4, cs_order_number#5, cs_ext_ship_cost#6, cs_net_profit#7]
@@ -84,9 +82,9 @@ ReadSchema: struct<cs_warehouse_sk:int,cs_order_number:int>
Input [3]: [cs_warehouse_sk#9, cs_order_number#10, cs_sold_date_sk#11]
Arguments: [cs_warehouse_sk#9, cs_order_number#10], [cs_warehouse_sk#9, cs_order_number#10]
-(9) CometColumnarExchange
+(9) CometExchange
Input [2]: [cs_warehouse_sk#9, cs_order_number#10]
-Arguments: hashpartitioning(cs_order_number#10, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(cs_order_number#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(10) CometSort
Input [2]: [cs_warehouse_sk#9, cs_order_number#10]
@@ -115,9 +113,9 @@ ReadSchema: struct<cr_order_number:int>
Input [2]: [cr_order_number#12, cr_returned_date_sk#13]
Arguments: [cr_order_number#12], [cr_order_number#12]
-(16) CometColumnarExchange
+(16) CometExchange
Input [1]: [cr_order_number#12]
-Arguments: hashpartitioning(cr_order_number#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(cr_order_number#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(17) CometSort
Input [1]: [cr_order_number#12]
@@ -249,17 +247,11 @@ Functions [3]: [merge_sum(UnscaledValue(cs_ext_ship_cost#6)), merge_sum(Unscaled
Aggregate Attributes [3]: [sum(UnscaledValue(cs_ext_ship_cost#6))#20, sum(UnscaledValue(cs_net_profit#7))#21, count(cs_order_number#5)#24]
Results [3]: [sum#22, sum#23, count#25]
-(44) RowToColumnar
+(44) Exchange
Input [3]: [sum#22, sum#23, count#25]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=7]
-(45) CometColumnarExchange
-Input [3]: [sum#22, sum#23, count#25]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
-
-(46) ColumnarToRow [codegen id : 9]
-Input [3]: [sum#22, sum#23, count#25]
-
-(47) HashAggregate [codegen id : 9]
+(45) HashAggregate [codegen id : 9]
Input [3]: [sum#22, sum#23, count#25]
Keys: []
Functions [3]: [sum(UnscaledValue(cs_ext_ship_cost#6)), sum(UnscaledValue(cs_net_profit#7)), count(distinct cs_order_number#5)]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/simplified.txt
index 605ce370..18ba3932 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/simplified.txt
@@ -1,70 +1,68 @@
WholeStageCodegen (9)
HashAggregate [sum,sum,count] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),order count ,total shipping cost ,total net profit ,sum,sum,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange #1
- RowToColumnar
- WholeStageCodegen (8)
- HashAggregate [cs_order_number] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),sum,sum,count,sum,sum,count]
- HashAggregate [cs_order_number] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),sum,sum,sum,sum]
- HashAggregate [cs_order_number,cs_ext_ship_cost,cs_net_profit] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),sum,sum,sum,sum]
- Project [cs_order_number,cs_ext_ship_cost,cs_net_profit]
- BroadcastHashJoin [cs_call_center_sk,cc_call_center_sk]
- Project [cs_call_center_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
- BroadcastHashJoin [cs_ship_addr_sk,ca_address_sk]
- Project [cs_ship_addr_sk,cs_call_center_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
- BroadcastHashJoin [cs_ship_date_sk,d_date_sk]
- SortMergeJoin [cs_order_number,cr_order_number]
- InputAdapter
- WholeStageCodegen (3)
- Project [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
- SortMergeJoin [cs_order_number,cs_order_number,cs_warehouse_sk,cs_warehouse_sk]
- InputAdapter
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometSort [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
- CometColumnarExchange [cs_order_number] #2
- CometProject [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
- CometFilter [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit,cs_sold_date_sk]
- CometScan parquet spark_catalog.default.catalog_sales [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit,cs_sold_date_sk]
- InputAdapter
- WholeStageCodegen (2)
- ColumnarToRow
- InputAdapter
- CometSort [cs_warehouse_sk,cs_order_number]
- CometColumnarExchange [cs_order_number] #3
- CometProject [cs_warehouse_sk,cs_order_number]
- CometScan parquet spark_catalog.default.catalog_sales [cs_warehouse_sk,cs_order_number,cs_sold_date_sk]
- InputAdapter
- WholeStageCodegen (4)
- ColumnarToRow
- InputAdapter
- CometSort [cr_order_number]
- CometColumnarExchange [cr_order_number] #4
- CometProject [cr_order_number]
- CometScan parquet spark_catalog.default.catalog_returns [cr_order_number,cr_returned_date_sk]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (5)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ InputAdapter
+ Exchange #1
+ WholeStageCodegen (8)
+ HashAggregate [cs_order_number] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),count(cs_order_number),sum,sum,count,sum,sum,count]
+ HashAggregate [cs_order_number] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),sum,sum,sum,sum]
+ HashAggregate [cs_order_number,cs_ext_ship_cost,cs_net_profit] [sum(UnscaledValue(cs_ext_ship_cost)),sum(UnscaledValue(cs_net_profit)),sum,sum,sum,sum]
+ Project [cs_order_number,cs_ext_ship_cost,cs_net_profit]
+ BroadcastHashJoin [cs_call_center_sk,cc_call_center_sk]
+ Project [cs_call_center_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
+ BroadcastHashJoin [cs_ship_addr_sk,ca_address_sk]
+ Project [cs_ship_addr_sk,cs_call_center_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
+ BroadcastHashJoin [cs_ship_date_sk,d_date_sk]
+ SortMergeJoin [cs_order_number,cr_order_number]
+ InputAdapter
+ WholeStageCodegen (3)
+ Project [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
+ SortMergeJoin [cs_order_number,cs_order_number,cs_warehouse_sk,cs_warehouse_sk]
+ InputAdapter
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometSort [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
+ CometExchange [cs_order_number] #2
+ CometProject [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit]
+ CometFilter [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit,cs_sold_date_sk]
+ CometScan parquet spark_catalog.default.catalog_sales [cs_ship_date_sk,cs_ship_addr_sk,cs_call_center_sk,cs_warehouse_sk,cs_order_number,cs_ext_ship_cost,cs_net_profit,cs_sold_date_sk]
+ InputAdapter
+ WholeStageCodegen (2)
+ ColumnarToRow
+ InputAdapter
+ CometSort [cs_warehouse_sk,cs_order_number]
+ CometExchange [cs_order_number] #3
+ CometProject [cs_warehouse_sk,cs_order_number]
+ CometScan parquet spark_catalog.default.catalog_sales [cs_warehouse_sk,cs_order_number,cs_sold_date_sk]
+ InputAdapter
+ WholeStageCodegen (4)
+ ColumnarToRow
+ InputAdapter
+ CometSort [cr_order_number]
+ CometExchange [cr_order_number] #4
+ CometProject [cr_order_number]
+ CometScan parquet spark_catalog.default.catalog_returns [cr_order_number,cr_returned_date_sk]
InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (6)
+ BroadcastExchange #5
+ WholeStageCodegen (5)
ColumnarToRow
InputAdapter
- CometProject [ca_address_sk]
- CometFilter [ca_address_sk,ca_state]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
InputAdapter
- BroadcastExchange #7
- WholeStageCodegen (7)
+ BroadcastExchange #6
+ WholeStageCodegen (6)
ColumnarToRow
InputAdapter
- CometProject [cc_call_center_sk]
- CometFilter [cc_call_center_sk,cc_county]
- CometScan parquet spark_catalog.default.call_center [cc_call_center_sk,cc_county]
+ CometProject [ca_address_sk]
+ CometFilter [ca_address_sk,ca_state]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ InputAdapter
+ BroadcastExchange #7
+ WholeStageCodegen (7)
+ ColumnarToRow
+ InputAdapter
+ CometProject [cc_call_center_sk]
+ CometFilter [cc_call_center_sk,cc_county]
+ CometScan parquet spark_catalog.default.call_center [cc_call_center_sk,cc_county]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt
index ddf62fe9..69f720e6 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (42)
+- CometTakeOrderedAndProject (41)
+- CometHashAggregate (40)
- +- CometColumnarExchange (39)
+ +- CometExchange (39)
+- CometHashAggregate (38)
+- CometProject (37)
+- CometBroadcastHashJoin (36)
@@ -226,9 +226,9 @@ Input [6]: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25, i_
Keys [3]: [i_item_id#27, i_item_desc#28, s_state#25]
Functions [9]: [partial_count(ss_quantity#5), partial_avg(ss_quantity#5), partial_stddev_samp(cast(ss_quantity#5 as double)), partial_count(sr_return_quantity#11), partial_avg(sr_return_quantity#11), partial_stddev_samp(cast(sr_return_quantity#11 as double)), partial_count(cs_quantity#16), partial_avg(cs_quantity#16), partial_stddev_samp(cast(cs_quantity#16 as double))]
-(39) CometColumnarExchange
+(39) CometExchange
Input [21]: [i_item_id#27, i_item_desc#28, s_state#25, count#29, sum#30, count#31, n#32, avg#33, m2#34, count#35, sum#36, count#37, n#38, avg#39, m2#40, count#41, sum#42, count#43, n#44, avg#45, m2#46]
-Arguments: hashpartitioning(i_item_id#27, i_item_desc#28, s_state#25, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#27, i_item_desc#28, s_state#25, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(40) CometHashAggregate
Input [21]: [i_item_id#27, i_item_desc#28, s_state#25, count#29, sum#30, count#31, n#32, avg#33, m2#34, count#35, sum#36, count#37, n#38, avg#39, m2#40, count#41, sum#42, count#43, n#44, avg#45, m2#46]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt
index e350dfe5..a3adaf17 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
CometHashAggregate [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count(ss_quantity),avg(ss_quantity [...]
- CometColumnarExchange [i_item_id,i_item_desc,s_state] #1
+ CometExchange [i_item_id,i_item_desc,s_state] #1
CometHashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,ss_quantity,sr_return_quantity,cs_quantity]
CometProject [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
CometBroadcastHashJoin [ss_item_sk,ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_sk,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt
index 6c969939..bd8c5eb5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (41)
+- CometTakeOrderedAndProject (40)
+- CometHashAggregate (39)
- +- CometColumnarExchange (38)
+ +- CometExchange (38)
+- CometHashAggregate (37)
+- CometExpand (36)
+- CometProject (35)
@@ -219,9 +219,9 @@ Input [12]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7,
Keys [5]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33]
Functions [7]: [partial_avg(cast(cs_quantity#4 as decimal(12,2))), partial_avg(cast(cs_list_price#5 as decimal(12,2))), partial_avg(cast(cs_coupon_amt#7 as decimal(12,2))), partial_avg(cast(cs_sales_price#6 as decimal(12,2))), partial_avg(cast(cs_net_profit#8 as decimal(12,2))), partial_avg(cast(c_birth_year#19 as decimal(12,2))), partial_avg(cast(cd_dep_count#14 as decimal(12,2)))]
-(38) CometColumnarExchange
+(38) CometExchange
Input [19]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, sum#34, count#35, sum#36, count#37, sum#38, count#39, sum#40, count#41, sum#42, count#43, sum#44, count#45, sum#46, count#47]
-Arguments: hashpartitioning(i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(39) CometHashAggregate
Input [19]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, sum#34, count#35, sum#36, count#37, sum#38, count#39, sum#40, count#41, sum#42, count#43, sum#44, count#45, sum#46, count#47]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt
index 8182da77..e26491c7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,ca_country,ca_state,ca_county,agg1,agg2,agg3,agg4,agg5,agg6,agg7]
CometHashAggregate [i_item_id,ca_country,ca_state,ca_county,agg1,agg2,agg3,agg4,agg5,agg6,agg7,spark_grouping_id,sum,count,sum,count,sum,count,sum,count,sum,count,sum,count,sum,count,avg(cast(cs_quantity as decimal(12,2))),avg(cast(cs_list_price as decimal(12,2))),avg(cast(cs_coupon_amt as decimal(12,2))),avg(cast(cs_sales_price as decimal(12,2))),avg(cast(cs_net_profit as decimal(12,2))),avg(cast(c_birth_year as decimal(12,2))),avg(cast(cd_dep_count as decimal(12,2)))]
- CometColumnarExchange [i_item_id,ca_country,ca_state,ca_county,spark_grouping_id] #1
+ CometExchange [i_item_id,ca_country,ca_state,ca_county,spark_grouping_id] #1
CometHashAggregate [i_item_id,ca_country,ca_state,ca_county,spark_grouping_id,sum,count,sum,count,sum,count,sum,count,sum,count,sum,count,sum,count,cs_quantity,cs_list_price,cs_coupon_amt,cs_sales_price,cs_net_profit,c_birth_year,cd_dep_count]
CometExpand [i_item_id,ca_country,ca_state,ca_county] [cs_quantity,cs_list_price,cs_sales_price,cs_coupon_amt,cs_net_profit,cd_dep_count,c_birth_year,i_item_id,ca_country,ca_state,ca_county,spark_grouping_id]
CometProject [cs_quantity,cs_list_price,cs_sales_price,cs_coupon_amt,cs_net_profit,cd_dep_count,c_birth_year,i_item_id,ca_country,ca_state,ca_county]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt
index 2b3480d8..2191317f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (34)
+- CometTakeOrderedAndProject (33)
+- CometHashAggregate (32)
- +- CometColumnarExchange (31)
+ +- CometExchange (31)
+- CometHashAggregate (30)
+- CometProject (29)
+- CometBroadcastHashJoin (28)
@@ -180,9 +180,9 @@ Input [5]: [ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i
Keys [4]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#7))]
-(31) CometColumnarExchange
+(31) CometExchange
Input [5]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, sum#21]
-Arguments: hashpartitioning(i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(32) CometHashAggregate
Input [5]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, sum#21]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt
index 8f81e32f..46b0d650 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [brand_id,brand,i_manufact_id,i_manufact,ext_price]
CometHashAggregate [brand_id,brand,i_manufact_id,i_manufact,ext_price,i_brand,i_brand_id,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_brand,i_brand_id,i_manufact_id,i_manufact] #1
+ CometExchange [i_brand,i_brand_id,i_manufact_id,i_manufact] #1
CometHashAggregate [i_brand,i_brand_id,i_manufact_id,i_manufact,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,i_brand_id,i_brand,i_manufact_id,i_manufact]
CometBroadcastHashJoin [ss_store_sk,ss_ext_sales_price,i_brand_id,i_brand,i_manufact_id,i_manufact,ca_zip,s_store_sk,s_zip]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt
index d3726126..35e00a7d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt
@@ -7,7 +7,7 @@
:- CometProject (19)
: +- CometBroadcastHashJoin (18)
: :- CometHashAggregate (13)
- : : +- CometColumnarExchange (12)
+ : : +- CometExchange (12)
: : +- CometHashAggregate (11)
: : +- CometProject (10)
: : +- CometBroadcastHashJoin (9)
@@ -89,9 +89,9 @@ Input [3]: [sales_price#4, d_week_seq#10, d_day_name#11]
Keys [1]: [d_week_seq#10]
Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (d_day_name#11 = Sunday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#11 = Monday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#11 = Tuesday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#11 = Wednesday) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#11 = Thursday ) THEN sales_price#4 END)), partial_sum(UnscaledValue( [...]
-(12) CometColumnarExchange
+(12) CometExchange
Input [8]: [d_week_seq#10, sum#12, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18]
-Arguments: hashpartitioning(d_week_seq#10, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(d_week_seq#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(13) CometHashAggregate
Input [8]: [d_week_seq#10, sum#12, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt
index c5c06a0b..ed772bad 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt
@@ -8,7 +8,7 @@ WholeStageCodegen (1)
CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1]
CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
CometHashAggregate [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = [...]
- CometColumnarExchange [d_week_seq] #2
+ CometExchange [d_week_seq] #2
CometHashAggregate [d_week_seq,sum,sum,sum,sum,sum,sum,sum,d_day_name,sales_price]
CometProject [sales_price,d_week_seq,d_day_name]
CometBroadcastHashJoin [sold_date_sk,sales_price,d_date_sk,d_week_seq,d_day_name]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt
index c6982f07..1af8f20f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt
@@ -4,9 +4,9 @@ TakeOrderedAndProject (22)
+- Window (20)
+- * ColumnarToRow (19)
+- CometSort (18)
- +- CometColumnarExchange (17)
+ +- CometExchange (17)
+- CometHashAggregate (16)
- +- CometColumnarExchange (15)
+ +- CometExchange (15)
+- CometHashAggregate (14)
+- CometProject (13)
+- CometBroadcastHashJoin (12)
@@ -92,18 +92,18 @@ Input [6]: [cs_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8,
Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#2))]
-(15) CometColumnarExchange
+(15) CometExchange
Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
-Arguments: hashpartitioning(i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(16) CometHashAggregate
Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
Functions [1]: [sum(UnscaledValue(cs_ext_sales_price#2))]
-(17) CometColumnarExchange
+(17) CometExchange
Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: hashpartitioning(i_class#9, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_class#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(18) CometSort
Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt
index 9c30689f..9a906085 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt
@@ -7,9 +7,9 @@ TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_c
ColumnarToRow
InputAdapter
CometSort [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id]
- CometColumnarExchange [i_class] #1
+ CometExchange [i_class] #1
CometHashAggregate [i_item_desc,i_category,i_class,i_current_price,itemrevenue,_w0,i_item_id,sum,sum(UnscaledValue(cs_ext_sales_price))]
- CometColumnarExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
+ CometExchange [i_item_id,i_item_desc,i_category,i_class,i_current_price] #2
CometHashAggregate [i_item_id,i_item_desc,i_category,i_class,i_current_price,sum,cs_ext_sales_price]
CometProject [cs_ext_sales_price,i_item_id,i_item_desc,i_current_price,i_class,i_category]
CometBroadcastHashJoin [cs_ext_sales_price,cs_sold_date_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt
index ff0f63b7..dce72db2 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt
@@ -3,7 +3,7 @@
+- CometTakeOrderedAndProject (23)
+- CometFilter (22)
+- CometHashAggregate (21)
- +- CometColumnarExchange (20)
+ +- CometExchange (20)
+- CometHashAggregate (19)
+- CometProject (18)
+- CometBroadcastHashJoin (17)
@@ -118,9 +118,9 @@ Input [4]: [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#9, d_date#12]
Keys [2]: [w_warehouse_name#7, i_item_id#9]
Functions [2]: [partial_sum(CASE WHEN (d_date#12 < 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END), partial_sum(CASE WHEN (d_date#12 >= 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END)]
-(20) CometColumnarExchange
+(20) CometExchange
Input [4]: [w_warehouse_name#7, i_item_id#9, sum#13, sum#14]
-Arguments: hashpartitioning(w_warehouse_name#7, i_item_id#9, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(w_warehouse_name#7, i_item_id#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(21) CometHashAggregate
Input [4]: [w_warehouse_name#7, i_item_id#9, sum#13, sum#14]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt
index 1a4f175c..6e483f60 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt
@@ -4,7 +4,7 @@ WholeStageCodegen (1)
CometTakeOrderedAndProject [w_warehouse_name,i_item_id,inv_before,inv_after]
CometFilter [w_warehouse_name,i_item_id,inv_before,inv_after]
CometHashAggregate [w_warehouse_name,i_item_id,inv_before,inv_after,sum,sum,sum(CASE WHEN (d_date < 2000-03-11) THEN inv_quantity_on_hand ELSE 0 END),sum(CASE WHEN (d_date >= 2000-03-11) THEN inv_quantity_on_hand ELSE 0 END)]
- CometColumnarExchange [w_warehouse_name,i_item_id] #1
+ CometExchange [w_warehouse_name,i_item_id] #1
CometHashAggregate [w_warehouse_name,i_item_id,sum,sum,d_date,inv_quantity_on_hand]
CometProject [inv_quantity_on_hand,w_warehouse_name,i_item_id,d_date]
CometBroadcastHashJoin [inv_quantity_on_hand,inv_date_sk,w_warehouse_name,i_item_id,d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt
index 1f7209c6..c3f3c800 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (24)
+- CometTakeOrderedAndProject (23)
+- CometHashAggregate (22)
- +- CometColumnarExchange (21)
+ +- CometExchange (21)
+- CometHashAggregate (20)
+- CometExpand (19)
+- CometProject (18)
@@ -122,9 +122,9 @@ Input [6]: [inv_quantity_on_hand#3, i_product_name#14, i_brand#15, i_class#16, i
Keys [5]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18]
Functions [1]: [partial_avg(inv_quantity_on_hand#3)]
-(21) CometColumnarExchange
+(21) CometExchange
Input [7]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, sum#19, count#20]
-Arguments: hashpartitioning(i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(22) CometHashAggregate
Input [7]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, sum#19, count#20]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt
index 05d02283..ed997b5d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_product_name,i_brand,i_class,i_category,qoh]
CometHashAggregate [i_product_name,i_brand,i_class,i_category,qoh,spark_grouping_id,sum,count,avg(inv_quantity_on_hand)]
- CometColumnarExchange [i_product_name,i_brand,i_class,i_category,spark_grouping_id] #1
+ CometExchange [i_product_name,i_brand,i_class,i_category,spark_grouping_id] #1
CometHashAggregate [i_product_name,i_brand,i_class,i_category,spark_grouping_id,sum,count,inv_quantity_on_hand]
CometExpand [i_product_name,i_brand,i_class,i_category] [inv_quantity_on_hand,i_product_name,i_brand,i_class,i_category,spark_grouping_id]
CometProject [inv_quantity_on_hand,i_product_name,i_brand,i_class,i_category]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/explain.txt
index 2af9c608..c4d6d30d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
* ColumnarToRow (67)
+- CometHashAggregate (66)
- +- CometColumnarExchange (65)
+ +- CometExchange (65)
+- CometHashAggregate (64)
+- CometUnion (63)
:- CometProject (46)
@@ -9,7 +9,7 @@
: :- CometProject (40)
: : +- CometSortMergeJoin (39)
: : :- CometSort (24)
- : : : +- CometColumnarExchange (23)
+ : : : +- CometExchange (23)
: : : +- CometProject (22)
: : : +- CometBroadcastHashJoin (21)
: : : :- CometScan parquet spark_catalog.default.catalog_sales (1)
@@ -17,7 +17,7 @@
: : : +- CometProject (19)
: : : +- CometFilter (18)
: : : +- CometHashAggregate (17)
- : : : +- CometColumnarExchange (16)
+ : : : +- CometExchange (16)
: : : +- CometHashAggregate (15)
: : : +- CometProject (14)
: : : +- CometBroadcastHashJoin (13)
@@ -36,7 +36,7 @@
: : +- CometProject (37)
: : +- CometFilter (36)
: : +- CometHashAggregate (35)
- : : +- CometColumnarExchange (34)
+ : : +- CometExchange (34)
: : +- CometHashAggregate (33)
: : +- CometProject (32)
: : +- CometBroadcastHashJoin (31)
@@ -55,7 +55,7 @@
:- CometProject (59)
: +- CometSortMergeJoin (58)
: :- CometSort (52)
- : : +- CometColumnarExchange (51)
+ : : +- CometExchange (51)
: : +- CometProject (50)
: : +- CometBroadcastHashJoin (49)
: : :- CometScan parquet spark_catalog.default.web_sales (47)
@@ -144,9 +144,9 @@ Input [3]: [d_date#11, i_item_sk#13, _groupingexpression#15]
Keys [3]: [_groupingexpression#15, i_item_sk#13, d_date#11]
Functions [1]: [partial_count(1)]
-(16) CometColumnarExchange
+(16) CometExchange
Input [4]: [_groupingexpression#15, i_item_sk#13, d_date#11, count#16]
-Arguments: hashpartitioning(_groupingexpression#15, i_item_sk#13, d_date#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(_groupingexpression#15, i_item_sk#13, d_date#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
Input [4]: [_groupingexpression#15, i_item_sk#13, d_date#11, count#16]
@@ -174,9 +174,9 @@ Arguments: [cs_item_sk#2], [item_sk#17], LeftSemi, BuildRight
Input [5]: [cs_bill_customer_sk#1, cs_item_sk#2, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
Arguments: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5], [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
-(23) CometColumnarExchange
+(23) CometExchange
Input [4]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
-Arguments: hashpartitioning(cs_bill_customer_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(cs_bill_customer_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(24) CometSort
Input [4]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
@@ -226,9 +226,9 @@ Input [3]: [ss_quantity#20, ss_sales_price#21, c_customer_sk#23]
Keys [1]: [c_customer_sk#23]
Functions [1]: [partial_sum((cast(ss_quantity#20 as decimal(10,0)) * ss_sales_price#21))]
-(34) CometColumnarExchange
+(34) CometExchange
Input [3]: [c_customer_sk#23, sum#24, isEmpty#25]
-Arguments: hashpartitioning(c_customer_sk#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(c_customer_sk#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(35) CometHashAggregate
Input [3]: [c_customer_sk#23, sum#24, isEmpty#25]
@@ -303,9 +303,9 @@ Arguments: [ws_item_sk#33], [item_sk#39], LeftSemi, BuildRight
Input [5]: [ws_item_sk#33, ws_bill_customer_sk#34, ws_quantity#35, ws_list_price#36, ws_sold_date_sk#37]
Arguments: [ws_bill_customer_sk#34, ws_quantity#35, ws_list_price#36, ws_sold_date_sk#37], [ws_bill_customer_sk#34, ws_quantity#35, ws_list_price#36, ws_sold_date_sk#37]
-(51) CometColumnarExchange
+(51) CometExchange
Input [4]: [ws_bill_customer_sk#34, ws_quantity#35, ws_list_price#36, ws_sold_date_sk#37]
-Arguments: hashpartitioning(ws_bill_customer_sk#34, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(ws_bill_customer_sk#34, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(52) CometSort
Input [4]: [ws_bill_customer_sk#34, ws_quantity#35, ws_list_price#36, ws_sold_date_sk#37]
@@ -361,9 +361,9 @@ Input [1]: [sales#32]
Keys: []
Functions [1]: [partial_sum(sales#32)]
-(65) CometColumnarExchange
+(65) CometExchange
Input [2]: [sum#48, isEmpty#49]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(66) CometHashAggregate
Input [2]: [sum#48, isEmpty#49]
@@ -438,10 +438,10 @@ Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))
Subquery:3 Hosting operator id = 36 Hosting Expression = Subquery scalar-subquery#27, [id=#28]
* ColumnarToRow (95)
+- CometHashAggregate (94)
- +- CometColumnarExchange (93)
+ +- CometExchange (93)
+- CometHashAggregate (92)
+- CometHashAggregate (91)
- +- CometColumnarExchange (90)
+ +- CometExchange (90)
+- CometHashAggregate (89)
+- CometProject (88)
+- CometBroadcastHashJoin (87)
@@ -513,9 +513,9 @@ Input [3]: [ss_quantity#52, ss_sales_price#53, c_customer_sk#56]
Keys [1]: [c_customer_sk#56]
Functions [1]: [partial_sum((cast(ss_quantity#52 as decimal(10,0)) * ss_sales_price#53))]
-(90) CometColumnarExchange
+(90) CometExchange
Input [3]: [c_customer_sk#56, sum#59, isEmpty#60]
-Arguments: hashpartitioning(c_customer_sk#56, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=8]
+Arguments: hashpartitioning(c_customer_sk#56, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=8]
(91) CometHashAggregate
Input [3]: [c_customer_sk#56, sum#59, isEmpty#60]
@@ -527,9 +527,9 @@ Input [1]: [csales#61]
Keys: []
Functions [1]: [partial_max(csales#61)]
-(93) CometColumnarExchange
+(93) CometExchange
Input [1]: [max#62]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=9]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=9]
(94) CometHashAggregate
Input [1]: [max#62]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/simplified.txt
index 18cf2b2c..7606f105 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23a/simplified.txt
@@ -2,7 +2,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [sum(sales),sum,isEmpty,sum(sales)]
- CometColumnarExchange #1
+ CometExchange #1
CometHashAggregate [sum,isEmpty,sales]
CometUnion [sales]
CometProject [cs_quantity,cs_list_price] [sales]
@@ -10,7 +10,7 @@ WholeStageCodegen (1)
CometProject [cs_quantity,cs_list_price,cs_sold_date_sk]
CometSortMergeJoin [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk,c_customer_sk]
CometSort [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
- CometColumnarExchange [cs_bill_customer_sk] #2
+ CometExchange [cs_bill_customer_sk] #2
CometProject [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
CometBroadcastHashJoin [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk,item_sk]
CometScan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
@@ -26,7 +26,7 @@ WholeStageCodegen (1)
CometProject [item_sk]
CometFilter [item_sk,cnt]
CometHashAggregate [item_sk,cnt,_groupingexpression,i_item_sk,d_date,count,count(1)]
- CometColumnarExchange [_groupingexpression,i_item_sk,d_date] #5
+ CometExchange [_groupingexpression,i_item_sk,d_date] #5
CometHashAggregate [_groupingexpression,i_item_sk,d_date,count]
CometProject [i_item_desc] [d_date,i_item_sk,_groupingexpression]
CometBroadcastHashJoin [ss_item_sk,d_date,i_item_sk,i_item_desc]
@@ -57,10 +57,10 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [tpcds_cmax,max,max(csales)]
- CometColumnarExchange #11
+ CometExchange #11
CometHashAggregate [max,csales]
CometHashAggregate [csales,c_customer_sk,sum,isEmpty,sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price))]
- CometColumnarExchange [c_customer_sk] #12
+ CometExchange [c_customer_sk] #12
CometHashAggregate [c_customer_sk,sum,isEmpty,ss_quantity,ss_sales_price]
CometProject [ss_quantity,ss_sales_price,c_customer_sk]
CometBroadcastHashJoin [ss_quantity,ss_sales_price,ss_sold_date_sk,c_customer_sk,d_date_sk]
@@ -82,7 +82,7 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometHashAggregate [c_customer_sk,ssales,sum,isEmpty,sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price))]
- CometColumnarExchange [c_customer_sk] #9
+ CometExchange [c_customer_sk] #9
CometHashAggregate [c_customer_sk,sum,isEmpty,ss_quantity,ss_sales_price]
CometProject [ss_quantity,ss_sales_price,c_customer_sk]
CometBroadcastHashJoin [ss_customer_sk,ss_quantity,ss_sales_price,c_customer_sk]
@@ -101,7 +101,7 @@ WholeStageCodegen (1)
CometProject [ws_quantity,ws_list_price,ws_sold_date_sk]
CometSortMergeJoin [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk,c_customer_sk]
CometSort [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
- CometColumnarExchange [ws_bill_customer_sk] #16
+ CometExchange [ws_bill_customer_sk] #16
CometProject [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
CometBroadcastHashJoin [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk,item_sk]
CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt
index ee11e789..4de12b9b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt
@@ -3,7 +3,7 @@
+- CometTakeOrderedAndProject (86)
+- CometUnion (85)
:- CometHashAggregate (62)
- : +- CometColumnarExchange (61)
+ : +- CometExchange (61)
: +- CometHashAggregate (60)
: +- CometProject (59)
: +- CometBroadcastHashJoin (58)
@@ -11,7 +11,7 @@
: : +- CometBroadcastHashJoin (52)
: : :- CometSortMergeJoin (40)
: : : :- CometSort (25)
- : : : : +- CometColumnarExchange (24)
+ : : : : +- CometExchange (24)
: : : : +- CometProject (23)
: : : : +- CometBroadcastHashJoin (22)
: : : : :- CometFilter (2)
@@ -20,7 +20,7 @@
: : : : +- CometProject (20)
: : : : +- CometFilter (19)
: : : : +- CometHashAggregate (18)
- : : : : +- CometColumnarExchange (17)
+ : : : : +- CometExchange (17)
: : : : +- CometHashAggregate (16)
: : : : +- CometProject (15)
: : : : +- CometBroadcastHashJoin (14)
@@ -39,7 +39,7 @@
: : : +- CometProject (38)
: : : +- CometFilter (37)
: : : +- CometHashAggregate (36)
- : : : +- CometColumnarExchange (35)
+ : : : +- CometExchange (35)
: : : +- CometHashAggregate (34)
: : : +- CometProject (33)
: : : +- CometBroadcastHashJoin (32)
@@ -52,7 +52,7 @@
: : +- CometBroadcastExchange (51)
: : +- CometSortMergeJoin (50)
: : :- CometSort (44)
- : : : +- CometColumnarExchange (43)
+ : : : +- CometExchange (43)
: : : +- CometFilter (42)
: : : +- CometScan parquet spark_catalog.default.customer (41)
: : +- CometSort (49)
@@ -65,7 +65,7 @@
: +- CometFilter (55)
: +- CometScan parquet spark_catalog.default.date_dim (54)
+- CometHashAggregate (84)
- +- CometColumnarExchange (83)
+ +- CometExchange (83)
+- CometHashAggregate (82)
+- CometProject (81)
+- CometBroadcastHashJoin (80)
@@ -73,7 +73,7 @@
: +- CometBroadcastHashJoin (77)
: :- CometSortMergeJoin (75)
: : :- CometSort (69)
- : : : +- CometColumnarExchange (68)
+ : : : +- CometExchange (68)
: : : +- CometProject (67)
: : : +- CometBroadcastHashJoin (66)
: : : :- CometFilter (64)
@@ -169,9 +169,9 @@ Input [3]: [d_date#11, i_item_sk#13, _groupingexpression#15]
Keys [3]: [_groupingexpression#15, i_item_sk#13, d_date#11]
Functions [1]: [partial_count(1)]
-(17) CometColumnarExchange
+(17) CometExchange
Input [4]: [_groupingexpression#15, i_item_sk#13, d_date#11, count#16]
-Arguments: hashpartitioning(_groupingexpression#15, i_item_sk#13, d_date#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(_groupingexpression#15, i_item_sk#13, d_date#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(18) CometHashAggregate
Input [4]: [_groupingexpression#15, i_item_sk#13, d_date#11, count#16]
@@ -199,9 +199,9 @@ Arguments: [cs_item_sk#2], [item_sk#17], LeftSemi, BuildRight
Input [5]: [cs_bill_customer_sk#1, cs_item_sk#2, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
Arguments: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5], [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
-(24) CometColumnarExchange
+(24) CometExchange
Input [4]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
-Arguments: hashpartitioning(cs_bill_customer_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(cs_bill_customer_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(25) CometSort
Input [4]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
@@ -251,9 +251,9 @@ Input [3]: [ss_quantity#20, ss_sales_price#21, c_customer_sk#23]
Keys [1]: [c_customer_sk#23]
Functions [1]: [partial_sum((cast(ss_quantity#20 as decimal(10,0)) * ss_sales_price#21))]
-(35) CometColumnarExchange
+(35) CometExchange
Input [3]: [c_customer_sk#23, sum#24, isEmpty#25]
-Arguments: hashpartitioning(c_customer_sk#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(c_customer_sk#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(36) CometHashAggregate
Input [3]: [c_customer_sk#23, sum#24, isEmpty#25]
@@ -288,9 +288,9 @@ ReadSchema: struct<c_customer_sk:int,c_first_name:string,c_last_name:string>
Input [3]: [c_customer_sk#29, c_first_name#30, c_last_name#31]
Condition : isnotnull(c_customer_sk#29)
-(43) CometColumnarExchange
+(43) CometExchange
Input [3]: [c_customer_sk#29, c_first_name#30, c_last_name#31]
-Arguments: hashpartitioning(c_customer_sk#29, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(c_customer_sk#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(44) CometSort
Input [3]: [c_customer_sk#29, c_first_name#30, c_last_name#31]
@@ -367,9 +367,9 @@ Input [4]: [cs_quantity#3, cs_list_price#4, c_first_name#30, c_last_name#31]
Keys [2]: [c_last_name#31, c_first_name#30]
Functions [1]: [partial_sum((cast(cs_quantity#3 as decimal(10,0)) * cs_list_price#4))]
-(61) CometColumnarExchange
+(61) CometExchange
Input [4]: [c_last_name#31, c_first_name#30, sum#35, isEmpty#36]
-Arguments: hashpartitioning(c_last_name#31, c_first_name#30, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(62) CometHashAggregate
Input [4]: [c_last_name#31, c_first_name#30, sum#35, isEmpty#36]
@@ -400,9 +400,9 @@ Arguments: [ws_item_sk#37], [item_sk#43], LeftSemi, BuildRight
Input [5]: [ws_item_sk#37, ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
Arguments: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41], [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-(68) CometColumnarExchange
+(68) CometExchange
Input [4]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Arguments: hashpartitioning(ws_bill_customer_sk#38, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: hashpartitioning(ws_bill_customer_sk#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
(69) CometSort
Input [4]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
@@ -462,9 +462,9 @@ Input [4]: [ws_quantity#39, ws_list_price#40, c_first_name#51, c_last_name#52]
Keys [2]: [c_last_name#52, c_first_name#51]
Functions [1]: [partial_sum((cast(ws_quantity#39 as decimal(10,0)) * ws_list_price#40))]
-(83) CometColumnarExchange
+(83) CometExchange
Input [4]: [c_last_name#52, c_first_name#51, sum#54, isEmpty#55]
-Arguments: hashpartitioning(c_last_name#52, c_first_name#51, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
+Arguments: hashpartitioning(c_last_name#52, c_first_name#51, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=7]
(84) CometHashAggregate
Input [4]: [c_last_name#52, c_first_name#51, sum#54, isEmpty#55]
@@ -547,10 +547,10 @@ Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))
Subquery:3 Hosting operator id = 37 Hosting Expression = Subquery scalar-subquery#27, [id=#28]
* ColumnarToRow (115)
+- CometHashAggregate (114)
- +- CometColumnarExchange (113)
+ +- CometExchange (113)
+- CometHashAggregate (112)
+- CometHashAggregate (111)
- +- CometColumnarExchange (110)
+ +- CometExchange (110)
+- CometHashAggregate (109)
+- CometProject (108)
+- CometBroadcastHashJoin (107)
@@ -622,9 +622,9 @@ Input [3]: [ss_quantity#59, ss_sales_price#60, c_customer_sk#63]
Keys [1]: [c_customer_sk#63]
Functions [1]: [partial_sum((cast(ss_quantity#59 as decimal(10,0)) * ss_sales_price#60))]
-(110) CometColumnarExchange
+(110) CometExchange
Input [3]: [c_customer_sk#63, sum#66, isEmpty#67]
-Arguments: hashpartitioning(c_customer_sk#63, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=10]
+Arguments: hashpartitioning(c_customer_sk#63, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=10]
(111) CometHashAggregate
Input [3]: [c_customer_sk#63, sum#66, isEmpty#67]
@@ -636,9 +636,9 @@ Input [1]: [csales#68]
Keys: []
Functions [1]: [partial_max(csales#68)]
-(113) CometColumnarExchange
+(113) CometExchange
Input [1]: [max#69]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=11]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=11]
(114) CometHashAggregate
Input [1]: [max#69]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt
index 55f17ca3..d503bb68 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt
@@ -4,7 +4,7 @@ WholeStageCodegen (1)
CometTakeOrderedAndProject [c_last_name,c_first_name,sales]
CometUnion [c_last_name,c_first_name,sales]
CometHashAggregate [c_last_name,c_first_name,sales,sum,isEmpty,sum((cast(cs_quantity as decimal(10,0)) * cs_list_price))]
- CometColumnarExchange [c_last_name,c_first_name] #1
+ CometExchange [c_last_name,c_first_name] #1
CometHashAggregate [c_last_name,c_first_name,sum,isEmpty,cs_quantity,cs_list_price]
CometProject [cs_quantity,cs_list_price,c_first_name,c_last_name]
CometBroadcastHashJoin [cs_quantity,cs_list_price,cs_sold_date_sk,c_first_name,c_last_name,d_date_sk]
@@ -12,7 +12,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk,c_customer_sk,c_first_name,c_last_name]
CometSortMergeJoin [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk,c_customer_sk]
CometSort [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
- CometColumnarExchange [cs_bill_customer_sk] #2
+ CometExchange [cs_bill_customer_sk] #2
CometProject [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
CometBroadcastHashJoin [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk,item_sk]
CometFilter [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
@@ -29,7 +29,7 @@ WholeStageCodegen (1)
CometProject [item_sk]
CometFilter [item_sk,cnt]
CometHashAggregate [item_sk,cnt,_groupingexpression,i_item_sk,d_date,count,count(1)]
- CometColumnarExchange [_groupingexpression,i_item_sk,d_date] #5
+ CometExchange [_groupingexpression,i_item_sk,d_date] #5
CometHashAggregate [_groupingexpression,i_item_sk,d_date,count]
CometProject [i_item_desc] [d_date,i_item_sk,_groupingexpression]
CometBroadcastHashJoin [ss_item_sk,d_date,i_item_sk,i_item_desc]
@@ -60,10 +60,10 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [tpcds_cmax,max,max(csales)]
- CometColumnarExchange #11
+ CometExchange #11
CometHashAggregate [max,csales]
CometHashAggregate [csales,c_customer_sk,sum,isEmpty,sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price))]
- CometColumnarExchange [c_customer_sk] #12
+ CometExchange [c_customer_sk] #12
CometHashAggregate [c_customer_sk,sum,isEmpty,ss_quantity,ss_sales_price]
CometProject [ss_quantity,ss_sales_price,c_customer_sk]
CometBroadcastHashJoin [ss_quantity,ss_sales_price,ss_sold_date_sk,c_customer_sk,d_date_sk]
@@ -85,7 +85,7 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometHashAggregate [c_customer_sk,ssales,sum,isEmpty,sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price))]
- CometColumnarExchange [c_customer_sk] #9
+ CometExchange [c_customer_sk] #9
CometHashAggregate [c_customer_sk,sum,isEmpty,ss_quantity,ss_sales_price]
CometProject [ss_quantity,ss_sales_price,c_customer_sk]
CometBroadcastHashJoin [ss_customer_sk,ss_quantity,ss_sales_price,c_customer_sk]
@@ -98,7 +98,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name] #15
CometSortMergeJoin [c_customer_sk,c_first_name,c_last_name,c_customer_sk]
CometSort [c_customer_sk,c_first_name,c_last_name]
- CometColumnarExchange [c_customer_sk] #16
+ CometExchange [c_customer_sk] #16
CometFilter [c_customer_sk,c_first_name,c_last_name]
CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
CometSort [c_customer_sk]
@@ -112,7 +112,7 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year,d_moy]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
CometHashAggregate [c_last_name,c_first_name,sales,sum,isEmpty,sum((cast(ws_quantity as decimal(10,0)) * ws_list_price))]
- CometColumnarExchange [c_last_name,c_first_name] #18
+ CometExchange [c_last_name,c_first_name] #18
CometHashAggregate [c_last_name,c_first_name,sum,isEmpty,ws_quantity,ws_list_price]
CometProject [ws_quantity,ws_list_price,c_first_name,c_last_name]
CometBroadcastHashJoin [ws_quantity,ws_list_price,ws_sold_date_sk,c_first_name,c_last_name,d_date_sk]
@@ -120,7 +120,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk,c_customer_sk,c_first_name,c_last_name]
CometSortMergeJoin [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk,c_customer_sk]
CometSort [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
- CometColumnarExchange [ws_bill_customer_sk] #19
+ CometExchange [ws_bill_customer_sk] #19
CometProject [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
CometBroadcastHashJoin [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk,item_sk]
CometFilter [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt
index 7872cef7..3094c4e0 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt
@@ -1,50 +1,46 @@
== Physical Plan ==
-* Filter (46)
-+- * HashAggregate (45)
- +- * ColumnarToRow (44)
- +- CometColumnarExchange (43)
- +- RowToColumnar (42)
- +- * HashAggregate (41)
- +- * HashAggregate (40)
- +- * ColumnarToRow (39)
- +- CometColumnarExchange (38)
- +- RowToColumnar (37)
- +- * HashAggregate (36)
- +- * Project (35)
- +- * BroadcastHashJoin Inner BuildRight (34)
- :- * ColumnarToRow (29)
- : +- CometProject (28)
- : +- CometBroadcastHashJoin (27)
- : :- CometProject (23)
- : : +- CometBroadcastHashJoin (22)
- : : :- CometProject (18)
- : : : +- CometBroadcastHashJoin (17)
- : : : :- CometProject (12)
- : : : : +- CometSortMergeJoin (11)
- : : : : :- CometSort (5)
- : : : : : +- CometColumnarExchange (4)
- : : : : : +- CometProject (3)
- : : : : : +- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
- : : : : +- CometSort (10)
- : : : : +- CometColumnarExchange (9)
- : : : : +- CometProject (8)
- : : : : +- CometFilter (7)
- : : : : +- CometScan parquet spark_catalog.default.store_returns (6)
- : : : +- CometBroadcastExchange (16)
- : : : +- CometProject (15)
- : : : +- CometFilter (14)
- : : : +- CometScan parquet spark_catalog.default.store (13)
- : : +- CometBroadcastExchange (21)
- : : +- CometFilter (20)
- : : +- CometScan parquet spark_catalog.default.item (19)
- : +- CometBroadcastExchange (26)
- : +- CometFilter (25)
- : +- CometScan parquet spark_catalog.default.customer (24)
- +- BroadcastExchange (33)
- +- * ColumnarToRow (32)
- +- CometFilter (31)
- +- CometScan parquet spark_catalog.default.customer_address (30)
+* Filter (42)
++- * HashAggregate (41)
+ +- Exchange (40)
+ +- * HashAggregate (39)
+ +- * HashAggregate (38)
+ +- Exchange (37)
+ +- * HashAggregate (36)
+ +- * Project (35)
+ +- * BroadcastHashJoin Inner BuildRight (34)
+ :- * ColumnarToRow (29)
+ : +- CometProject (28)
+ : +- CometBroadcastHashJoin (27)
+ : :- CometProject (23)
+ : : +- CometBroadcastHashJoin (22)
+ : : :- CometProject (18)
+ : : : +- CometBroadcastHashJoin (17)
+ : : : :- CometProject (12)
+ : : : : +- CometSortMergeJoin (11)
+ : : : : :- CometSort (5)
+ : : : : : +- CometExchange (4)
+ : : : : : +- CometProject (3)
+ : : : : : +- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
+ : : : : +- CometSort (10)
+ : : : : +- CometExchange (9)
+ : : : : +- CometProject (8)
+ : : : : +- CometFilter (7)
+ : : : : +- CometScan parquet spark_catalog.default.store_returns (6)
+ : : : +- CometBroadcastExchange (16)
+ : : : +- CometProject (15)
+ : : : +- CometFilter (14)
+ : : : +- CometScan parquet spark_catalog.default.store (13)
+ : : +- CometBroadcastExchange (21)
+ : : +- CometFilter (20)
+ : : +- CometScan parquet spark_catalog.default.item (19)
+ : +- CometBroadcastExchange (26)
+ : +- CometFilter (25)
+ : +- CometScan parquet spark_catalog.default.customer (24)
+ +- BroadcastExchange (33)
+ +- * ColumnarToRow (32)
+ +- CometFilter (31)
+ +- CometScan parquet spark_catalog.default.customer_address (30)
(1) Scan parquet spark_catalog.default.store_sales
@@ -62,9 +58,9 @@ Condition : (((isnotnull(ss_ticket_number#4) AND isnotnull(ss_item_sk#1)) AND is
Input [6]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5, ss_sold_date_sk#6]
Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5], [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5]
-(4) CometColumnarExchange
+(4) CometExchange
Input [5]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5]
-Arguments: hashpartitioning(ss_ticket_number#4, ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_ticket_number#4, ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(5) CometSort
Input [5]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5]
@@ -85,9 +81,9 @@ Condition : (isnotnull(sr_ticket_number#8) AND isnotnull(sr_item_sk#7))
Input [3]: [sr_item_sk#7, sr_ticket_number#8, sr_returned_date_sk#9]
Arguments: [sr_item_sk#7, sr_ticket_number#8], [sr_item_sk#7, sr_ticket_number#8]
-(9) CometColumnarExchange
+(9) CometExchange
Input [2]: [sr_item_sk#7, sr_ticket_number#8]
-Arguments: hashpartitioning(sr_ticket_number#8, sr_item_sk#7, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(sr_ticket_number#8, sr_item_sk#7, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(10) CometSort
Input [2]: [sr_item_sk#7, sr_ticket_number#8]
@@ -216,216 +212,188 @@ Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#5))]
Aggregate Attributes [1]: [sum#28]
Results [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-(37) RowToColumnar
+(37) Exchange
Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
+Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, 5), ENSURE_REQUIREMENTS, [plan_id=4]
-(38) CometColumnarExchange
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-
-(39) ColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-
-(40) HashAggregate [codegen id : 3]
+(38) HashAggregate [codegen id : 3]
Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
Keys [10]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17]
Functions [1]: [sum(UnscaledValue(ss_net_paid#5))]
Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#5))#30]
Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, MakeDecimal(sum(UnscaledValue(ss_net_paid#5))#30,17,2) AS netpaid#31]
-(41) HashAggregate [codegen id : 3]
+(39) HashAggregate [codegen id : 3]
Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, netpaid#31]
Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
Functions [1]: [partial_sum(netpaid#31)]
Aggregate Attributes [2]: [sum#32, isEmpty#33]
Results [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-(42) RowToColumnar
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-
-(43) CometColumnarExchange
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(44) ColumnarToRow [codegen id : 4]
+(40) Exchange
Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
+Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, 5), ENSURE_REQUIREMENTS, [plan_id=5]
-(45) HashAggregate [codegen id : 4]
+(41) HashAggregate [codegen id : 4]
Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
Functions [1]: [sum(netpaid#31)]
Aggregate Attributes [1]: [sum(netpaid#31)#36]
Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, sum(netpaid#31)#36 AS paid#37]
-(46) Filter [codegen id : 4]
+(42) Filter [codegen id : 4]
Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, paid#37]
Condition : (isnotnull(paid#37) AND (cast(paid#37 as decimal(33,8)) > cast(Subquery scalar-subquery#38, [id=#39] as decimal(33,8))))
===== Subqueries =====
-Subquery:1 Hosting operator id = 46 Hosting Expression = Subquery scalar-subquery#38, [id=#39]
-* HashAggregate (77)
-+- * ColumnarToRow (76)
- +- CometColumnarExchange (75)
- +- RowToColumnar (74)
- +- * HashAggregate (73)
- +- * HashAggregate (72)
- +- * ColumnarToRow (71)
- +- CometColumnarExchange (70)
- +- RowToColumnar (69)
- +- * HashAggregate (68)
- +- * Project (67)
- +- * BroadcastHashJoin Inner BuildRight (66)
- :- * ColumnarToRow (64)
- : +- CometProject (63)
- : +- CometBroadcastHashJoin (62)
- : :- CometProject (60)
- : : +- CometBroadcastHashJoin (59)
- : : :- CometProject (55)
- : : : +- CometBroadcastHashJoin (54)
- : : : :- CometProject (52)
- : : : : +- CometSortMergeJoin (51)
- : : : : :- CometSort (48)
- : : : : : +- ReusedExchange (47)
- : : : : +- CometSort (50)
- : : : : +- ReusedExchange (49)
- : : : +- ReusedExchange (53)
- : : +- CometBroadcastExchange (58)
- : : +- CometFilter (57)
- : : +- CometScan parquet spark_catalog.default.item (56)
- : +- ReusedExchange (61)
- +- ReusedExchange (65)
-
-
-(47) ReusedExchange [Reuses operator id: 4]
+Subquery:1 Hosting operator id = 42 Hosting Expression = Subquery scalar-subquery#38, [id=#39]
+* HashAggregate (69)
++- Exchange (68)
+ +- * HashAggregate (67)
+ +- * HashAggregate (66)
+ +- Exchange (65)
+ +- * HashAggregate (64)
+ +- * Project (63)
+ +- * BroadcastHashJoin Inner BuildRight (62)
+ :- * ColumnarToRow (60)
+ : +- CometProject (59)
+ : +- CometBroadcastHashJoin (58)
+ : :- CometProject (56)
+ : : +- CometBroadcastHashJoin (55)
+ : : :- CometProject (51)
+ : : : +- CometBroadcastHashJoin (50)
+ : : : :- CometProject (48)
+ : : : : +- CometSortMergeJoin (47)
+ : : : : :- CometSort (44)
+ : : : : : +- ReusedExchange (43)
+ : : : : +- CometSort (46)
+ : : : : +- ReusedExchange (45)
+ : : : +- ReusedExchange (49)
+ : : +- CometBroadcastExchange (54)
+ : : +- CometFilter (53)
+ : : +- CometScan parquet spark_catalog.default.item (52)
+ : +- ReusedExchange (57)
+ +- ReusedExchange (61)
+
+
+(43) ReusedExchange [Reuses operator id: 4]
Output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-(48) CometSort
+(44) CometSort
Input [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44], [ss_ticket_number#43 ASC NULLS FIRST, ss_item_sk#40 ASC NULLS FIRST]
-(49) ReusedExchange [Reuses operator id: 9]
+(45) ReusedExchange [Reuses operator id: 9]
Output [2]: [sr_item_sk#45, sr_ticket_number#46]
-(50) CometSort
+(46) CometSort
Input [2]: [sr_item_sk#45, sr_ticket_number#46]
Arguments: [sr_item_sk#45, sr_ticket_number#46], [sr_ticket_number#46 ASC NULLS FIRST, sr_item_sk#45 ASC NULLS FIRST]
-(51) CometSortMergeJoin
+(47) CometSortMergeJoin
Left output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
Right output [2]: [sr_item_sk#45, sr_ticket_number#46]
Arguments: [ss_ticket_number#43, ss_item_sk#40], [sr_ticket_number#46, sr_item_sk#45], Inner
-(52) CometProject
+(48) CometProject
Input [7]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44, sr_item_sk#45, sr_ticket_number#46]
Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44], [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
-(53) ReusedExchange [Reuses operator id: 16]
+(49) ReusedExchange [Reuses operator id: 16]
Output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-(54) CometBroadcastHashJoin
+(50) CometBroadcastHashJoin
Left output [4]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
Right output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
Arguments: [ss_store_sk#42], [s_store_sk#47], Inner, BuildRight
-(55) CometProject
+(51) CometProject
Input [8]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44, s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50], [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
-(56) Scan parquet spark_catalog.default.item
+(52) Scan parquet spark_catalog.default.item
Output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_current_price:decimal(7,2),i_size:string,i_color:string,i_units:string,i_manager_id:int>
-(57) CometFilter
+(53) CometFilter
Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Condition : isnotnull(i_item_sk#51)
-(58) CometBroadcastExchange
+(54) CometBroadcastExchange
Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Arguments: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-(59) CometBroadcastHashJoin
+(55) CometBroadcastHashJoin
Left output [6]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
Right output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Arguments: [ss_item_sk#40], [i_item_sk#51], Inner, BuildRight
-(60) CometProject
+(56) CometProject
Input [12]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Arguments: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56], [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-(61) ReusedExchange [Reuses operator id: 26]
+(57) ReusedExchange [Reuses operator id: 26]
Output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
-(62) CometBroadcastHashJoin
+(58) CometBroadcastHashJoin
Left output [10]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Right output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
Arguments: [ss_customer_sk#41], [c_customer_sk#57], Inner, BuildRight
-(63) CometProject
+(59) CometProject
Input [14]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
Arguments: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60], [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
-(64) ColumnarToRow [codegen id : 2]
+(60) ColumnarToRow [codegen id : 2]
Input [12]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
-(65) ReusedExchange [Reuses operator id: 33]
+(61) ReusedExchange [Reuses operator id: 33]
Output [3]: [ca_state#61, ca_zip#62, ca_country#63]
-(66) BroadcastHashJoin [codegen id : 2]
+(62) BroadcastHashJoin [codegen id : 2]
Left keys [2]: [c_birth_country#60, s_zip#50]
Right keys [2]: [upper(ca_country#63), ca_zip#62]
Join type: Inner
Join condition: None
-(67) Project [codegen id : 2]
+(63) Project [codegen id : 2]
Output [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
Input [15]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60, ca_state#61, ca_zip#62, ca_country#63]
-(68) HashAggregate [codegen id : 2]
+(64) HashAggregate [codegen id : 2]
Input [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#44))]
Aggregate Attributes [1]: [sum#64]
Results [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-(69) RowToColumnar
+(65) Exchange
Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
+Arguments: hashpartitioning(c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, 5), ENSURE_REQUIREMENTS, [plan_id=6]
-(70) CometColumnarExchange
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-Arguments: hashpartitioning(c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
-
-(71) ColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-
-(72) HashAggregate [codegen id : 3]
+(66) HashAggregate [codegen id : 3]
Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
Functions [1]: [sum(UnscaledValue(ss_net_paid#44))]
Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#44))#30]
Results [1]: [MakeDecimal(sum(UnscaledValue(ss_net_paid#44))#30,17,2) AS netpaid#66]
-(73) HashAggregate [codegen id : 3]
+(67) HashAggregate [codegen id : 3]
Input [1]: [netpaid#66]
Keys: []
Functions [1]: [partial_avg(netpaid#66)]
Aggregate Attributes [2]: [sum#67, count#68]
Results [2]: [sum#69, count#70]
-(74) RowToColumnar
-Input [2]: [sum#69, count#70]
-
-(75) CometColumnarExchange
-Input [2]: [sum#69, count#70]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
-
-(76) ColumnarToRow [codegen id : 4]
+(68) Exchange
Input [2]: [sum#69, count#70]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=7]
-(77) HashAggregate [codegen id : 4]
+(69) HashAggregate [codegen id : 4]
Input [2]: [sum#69, count#70]
Keys: []
Functions [1]: [avg(netpaid#66)]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt
index 0b53b6fc..bd14d825 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt
@@ -3,92 +3,84 @@ WholeStageCodegen (4)
Subquery #1
WholeStageCodegen (4)
HashAggregate [sum,count] [avg(netpaid),(0.05 * avg(netpaid)),sum,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange #9
- RowToColumnar
- WholeStageCodegen (3)
- HashAggregate [netpaid] [sum,count,sum,count]
- HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,sum] [sum(UnscaledValue(ss_net_paid)),netpaid,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #10
- RowToColumnar
- WholeStageCodegen (2)
- HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
- Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
- BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
- ColumnarToRow
- InputAdapter
- CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
- CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
- CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
- CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
- CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
- ReusedExchange [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid] #3
- CometSort [sr_item_sk,sr_ticket_number]
- ReusedExchange [sr_item_sk,sr_ticket_number] #4
- ReusedExchange [s_store_sk,s_store_name,s_state,s_zip] #5
- CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #11
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- ReusedExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
- InputAdapter
- ReusedExchange [ca_state,ca_zip,ca_country] #8
- HashAggregate [c_last_name,c_first_name,s_store_name,sum,isEmpty] [sum(netpaid),paid,sum,isEmpty]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [c_last_name,c_first_name,s_store_name] #1
- RowToColumnar
+ InputAdapter
+ Exchange #9
WholeStageCodegen (3)
- HashAggregate [c_last_name,c_first_name,s_store_name,netpaid] [sum,isEmpty,sum,isEmpty]
+ HashAggregate [netpaid] [sum,count,sum,count]
HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,sum] [sum(UnscaledValue(ss_net_paid)),netpaid,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #2
- RowToColumnar
- WholeStageCodegen (2)
- HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
- Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
- BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
- ColumnarToRow
- InputAdapter
- CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
- CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
- CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
- CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
- CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
- CometColumnarExchange [ss_ticket_number,ss_item_sk] #3
- CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
- CometFilter [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
- CometSort [sr_item_sk,sr_ticket_number]
- CometColumnarExchange [sr_ticket_number,sr_item_sk] #4
- CometProject [sr_item_sk,sr_ticket_number]
- CometFilter [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
- CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
- CometBroadcastExchange [s_store_sk,s_store_name,s_state,s_zip] #5
- CometProject [s_store_sk,s_store_name,s_state,s_zip]
- CometFilter [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
- CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #6
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
- CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ InputAdapter
+ Exchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #10
+ WholeStageCodegen (2)
+ HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
+ Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
+ BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
+ ColumnarToRow
+ InputAdapter
+ CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
+ CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
+ CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
+ CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
+ CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
+ ReusedExchange [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid] #3
+ CometSort [sr_item_sk,sr_ticket_number]
+ ReusedExchange [sr_item_sk,sr_ticket_number] #4
+ ReusedExchange [s_store_sk,s_store_name,s_state,s_zip] #5
+ CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #11
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ ReusedExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
+ InputAdapter
+ ReusedExchange [ca_state,ca_zip,ca_country] #8
+ HashAggregate [c_last_name,c_first_name,s_store_name,sum,isEmpty] [sum(netpaid),paid,sum,isEmpty]
+ InputAdapter
+ Exchange [c_last_name,c_first_name,s_store_name] #1
+ WholeStageCodegen (3)
+ HashAggregate [c_last_name,c_first_name,s_store_name,netpaid] [sum,isEmpty,sum,isEmpty]
+ HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,sum] [sum(UnscaledValue(ss_net_paid)),netpaid,sum]
+ InputAdapter
+ Exchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #2
+ WholeStageCodegen (2)
+ HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
+ Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
+ BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
+ ColumnarToRow
+ InputAdapter
+ CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
+ CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
+ CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
+ CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
+ CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
+ CometExchange [ss_ticket_number,ss_item_sk] #3
+ CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
+ CometFilter [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
+ CometSort [sr_item_sk,sr_ticket_number]
+ CometExchange [sr_ticket_number,sr_item_sk] #4
+ CometProject [sr_item_sk,sr_ticket_number]
+ CometFilter [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
+ CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
+ CometBroadcastExchange [s_store_sk,s_store_name,s_state,s_zip] #5
+ CometProject [s_store_sk,s_store_name,s_state,s_zip]
+ CometFilter [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
+ CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #6
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
+ CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ InputAdapter
+ BroadcastExchange #8
+ WholeStageCodegen (1)
+ ColumnarToRow
InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometFilter [ca_state,ca_zip,ca_country]
- CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
+ CometFilter [ca_state,ca_zip,ca_country]
+ CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt
index b5f3faef..846c7a14 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt
@@ -1,50 +1,46 @@
== Physical Plan ==
-* Filter (46)
-+- * HashAggregate (45)
- +- * ColumnarToRow (44)
- +- CometColumnarExchange (43)
- +- RowToColumnar (42)
- +- * HashAggregate (41)
- +- * HashAggregate (40)
- +- * ColumnarToRow (39)
- +- CometColumnarExchange (38)
- +- RowToColumnar (37)
- +- * HashAggregate (36)
- +- * Project (35)
- +- * BroadcastHashJoin Inner BuildRight (34)
- :- * ColumnarToRow (29)
- : +- CometProject (28)
- : +- CometBroadcastHashJoin (27)
- : :- CometProject (23)
- : : +- CometBroadcastHashJoin (22)
- : : :- CometProject (18)
- : : : +- CometBroadcastHashJoin (17)
- : : : :- CometProject (12)
- : : : : +- CometSortMergeJoin (11)
- : : : : :- CometSort (5)
- : : : : : +- CometColumnarExchange (4)
- : : : : : +- CometProject (3)
- : : : : : +- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
- : : : : +- CometSort (10)
- : : : : +- CometColumnarExchange (9)
- : : : : +- CometProject (8)
- : : : : +- CometFilter (7)
- : : : : +- CometScan parquet spark_catalog.default.store_returns (6)
- : : : +- CometBroadcastExchange (16)
- : : : +- CometProject (15)
- : : : +- CometFilter (14)
- : : : +- CometScan parquet spark_catalog.default.store (13)
- : : +- CometBroadcastExchange (21)
- : : +- CometFilter (20)
- : : +- CometScan parquet spark_catalog.default.item (19)
- : +- CometBroadcastExchange (26)
- : +- CometFilter (25)
- : +- CometScan parquet spark_catalog.default.customer (24)
- +- BroadcastExchange (33)
- +- * ColumnarToRow (32)
- +- CometFilter (31)
- +- CometScan parquet spark_catalog.default.customer_address (30)
+* Filter (42)
++- * HashAggregate (41)
+ +- Exchange (40)
+ +- * HashAggregate (39)
+ +- * HashAggregate (38)
+ +- Exchange (37)
+ +- * HashAggregate (36)
+ +- * Project (35)
+ +- * BroadcastHashJoin Inner BuildRight (34)
+ :- * ColumnarToRow (29)
+ : +- CometProject (28)
+ : +- CometBroadcastHashJoin (27)
+ : :- CometProject (23)
+ : : +- CometBroadcastHashJoin (22)
+ : : :- CometProject (18)
+ : : : +- CometBroadcastHashJoin (17)
+ : : : :- CometProject (12)
+ : : : : +- CometSortMergeJoin (11)
+ : : : : :- CometSort (5)
+ : : : : : +- CometExchange (4)
+ : : : : : +- CometProject (3)
+ : : : : : +- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
+ : : : : +- CometSort (10)
+ : : : : +- CometExchange (9)
+ : : : : +- CometProject (8)
+ : : : : +- CometFilter (7)
+ : : : : +- CometScan parquet spark_catalog.default.store_returns (6)
+ : : : +- CometBroadcastExchange (16)
+ : : : +- CometProject (15)
+ : : : +- CometFilter (14)
+ : : : +- CometScan parquet spark_catalog.default.store (13)
+ : : +- CometBroadcastExchange (21)
+ : : +- CometFilter (20)
+ : : +- CometScan parquet spark_catalog.default.item (19)
+ : +- CometBroadcastExchange (26)
+ : +- CometFilter (25)
+ : +- CometScan parquet spark_catalog.default.customer (24)
+ +- BroadcastExchange (33)
+ +- * ColumnarToRow (32)
+ +- CometFilter (31)
+ +- CometScan parquet spark_catalog.default.customer_address (30)
(1) Scan parquet spark_catalog.default.store_sales
@@ -62,9 +58,9 @@ Condition : (((isnotnull(ss_ticket_number#4) AND isnotnull(ss_item_sk#1)) AND is
Input [6]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5, ss_sold_date_sk#6]
Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5], [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5]
-(4) CometColumnarExchange
+(4) CometExchange
Input [5]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5]
-Arguments: hashpartitioning(ss_ticket_number#4, ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_ticket_number#4, ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(5) CometSort
Input [5]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_ticket_number#4, ss_net_paid#5]
@@ -85,9 +81,9 @@ Condition : (isnotnull(sr_ticket_number#8) AND isnotnull(sr_item_sk#7))
Input [3]: [sr_item_sk#7, sr_ticket_number#8, sr_returned_date_sk#9]
Arguments: [sr_item_sk#7, sr_ticket_number#8], [sr_item_sk#7, sr_ticket_number#8]
-(9) CometColumnarExchange
+(9) CometExchange
Input [2]: [sr_item_sk#7, sr_ticket_number#8]
-Arguments: hashpartitioning(sr_ticket_number#8, sr_item_sk#7, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(sr_ticket_number#8, sr_item_sk#7, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(10) CometSort
Input [2]: [sr_item_sk#7, sr_ticket_number#8]
@@ -216,216 +212,188 @@ Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#5))]
Aggregate Attributes [1]: [sum#28]
Results [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-(37) RowToColumnar
+(37) Exchange
Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
+Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, 5), ENSURE_REQUIREMENTS, [plan_id=4]
-(38) CometColumnarExchange
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-
-(39) ColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-
-(40) HashAggregate [codegen id : 3]
+(38) HashAggregate [codegen id : 3]
Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
Keys [10]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17]
Functions [1]: [sum(UnscaledValue(ss_net_paid#5))]
Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#5))#30]
Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, MakeDecimal(sum(UnscaledValue(ss_net_paid#5))#30,17,2) AS netpaid#31]
-(41) HashAggregate [codegen id : 3]
+(39) HashAggregate [codegen id : 3]
Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, netpaid#31]
Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
Functions [1]: [partial_sum(netpaid#31)]
Aggregate Attributes [2]: [sum#32, isEmpty#33]
Results [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-(42) RowToColumnar
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-
-(43) CometColumnarExchange
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(44) ColumnarToRow [codegen id : 4]
+(40) Exchange
Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
+Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, 5), ENSURE_REQUIREMENTS, [plan_id=5]
-(45) HashAggregate [codegen id : 4]
+(41) HashAggregate [codegen id : 4]
Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
Functions [1]: [sum(netpaid#31)]
Aggregate Attributes [1]: [sum(netpaid#31)#36]
Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, sum(netpaid#31)#36 AS paid#37]
-(46) Filter [codegen id : 4]
+(42) Filter [codegen id : 4]
Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, paid#37]
Condition : (isnotnull(paid#37) AND (cast(paid#37 as decimal(33,8)) > cast(Subquery scalar-subquery#38, [id=#39] as decimal(33,8))))
===== Subqueries =====
-Subquery:1 Hosting operator id = 46 Hosting Expression = Subquery scalar-subquery#38, [id=#39]
-* HashAggregate (77)
-+- * ColumnarToRow (76)
- +- CometColumnarExchange (75)
- +- RowToColumnar (74)
- +- * HashAggregate (73)
- +- * HashAggregate (72)
- +- * ColumnarToRow (71)
- +- CometColumnarExchange (70)
- +- RowToColumnar (69)
- +- * HashAggregate (68)
- +- * Project (67)
- +- * BroadcastHashJoin Inner BuildRight (66)
- :- * ColumnarToRow (64)
- : +- CometProject (63)
- : +- CometBroadcastHashJoin (62)
- : :- CometProject (60)
- : : +- CometBroadcastHashJoin (59)
- : : :- CometProject (55)
- : : : +- CometBroadcastHashJoin (54)
- : : : :- CometProject (52)
- : : : : +- CometSortMergeJoin (51)
- : : : : :- CometSort (48)
- : : : : : +- ReusedExchange (47)
- : : : : +- CometSort (50)
- : : : : +- ReusedExchange (49)
- : : : +- ReusedExchange (53)
- : : +- CometBroadcastExchange (58)
- : : +- CometFilter (57)
- : : +- CometScan parquet spark_catalog.default.item (56)
- : +- ReusedExchange (61)
- +- ReusedExchange (65)
-
-
-(47) ReusedExchange [Reuses operator id: 4]
+Subquery:1 Hosting operator id = 42 Hosting Expression = Subquery scalar-subquery#38, [id=#39]
+* HashAggregate (69)
++- Exchange (68)
+ +- * HashAggregate (67)
+ +- * HashAggregate (66)
+ +- Exchange (65)
+ +- * HashAggregate (64)
+ +- * Project (63)
+ +- * BroadcastHashJoin Inner BuildRight (62)
+ :- * ColumnarToRow (60)
+ : +- CometProject (59)
+ : +- CometBroadcastHashJoin (58)
+ : :- CometProject (56)
+ : : +- CometBroadcastHashJoin (55)
+ : : :- CometProject (51)
+ : : : +- CometBroadcastHashJoin (50)
+ : : : :- CometProject (48)
+ : : : : +- CometSortMergeJoin (47)
+ : : : : :- CometSort (44)
+ : : : : : +- ReusedExchange (43)
+ : : : : +- CometSort (46)
+ : : : : +- ReusedExchange (45)
+ : : : +- ReusedExchange (49)
+ : : +- CometBroadcastExchange (54)
+ : : +- CometFilter (53)
+ : : +- CometScan parquet spark_catalog.default.item (52)
+ : +- ReusedExchange (57)
+ +- ReusedExchange (61)
+
+
+(43) ReusedExchange [Reuses operator id: 4]
Output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-(48) CometSort
+(44) CometSort
Input [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44], [ss_ticket_number#43 ASC NULLS FIRST, ss_item_sk#40 ASC NULLS FIRST]
-(49) ReusedExchange [Reuses operator id: 9]
+(45) ReusedExchange [Reuses operator id: 9]
Output [2]: [sr_item_sk#45, sr_ticket_number#46]
-(50) CometSort
+(46) CometSort
Input [2]: [sr_item_sk#45, sr_ticket_number#46]
Arguments: [sr_item_sk#45, sr_ticket_number#46], [sr_ticket_number#46 ASC NULLS FIRST, sr_item_sk#45 ASC NULLS FIRST]
-(51) CometSortMergeJoin
+(47) CometSortMergeJoin
Left output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
Right output [2]: [sr_item_sk#45, sr_ticket_number#46]
Arguments: [ss_ticket_number#43, ss_item_sk#40], [sr_ticket_number#46, sr_item_sk#45], Inner
-(52) CometProject
+(48) CometProject
Input [7]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44, sr_item_sk#45, sr_ticket_number#46]
Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44], [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
-(53) ReusedExchange [Reuses operator id: 16]
+(49) ReusedExchange [Reuses operator id: 16]
Output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-(54) CometBroadcastHashJoin
+(50) CometBroadcastHashJoin
Left output [4]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
Right output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
Arguments: [ss_store_sk#42], [s_store_sk#47], Inner, BuildRight
-(55) CometProject
+(51) CometProject
Input [8]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44, s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50], [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
-(56) Scan parquet spark_catalog.default.item
+(52) Scan parquet spark_catalog.default.item
Output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_current_price:decimal(7,2),i_size:string,i_color:string,i_units:string,i_manager_id:int>
-(57) CometFilter
+(53) CometFilter
Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Condition : isnotnull(i_item_sk#51)
-(58) CometBroadcastExchange
+(54) CometBroadcastExchange
Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Arguments: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-(59) CometBroadcastHashJoin
+(55) CometBroadcastHashJoin
Left output [6]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
Right output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Arguments: [ss_item_sk#40], [i_item_sk#51], Inner, BuildRight
-(60) CometProject
+(56) CometProject
Input [12]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Arguments: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56], [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-(61) ReusedExchange [Reuses operator id: 26]
+(57) ReusedExchange [Reuses operator id: 26]
Output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
-(62) CometBroadcastHashJoin
+(58) CometBroadcastHashJoin
Left output [10]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
Right output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
Arguments: [ss_customer_sk#41], [c_customer_sk#57], Inner, BuildRight
-(63) CometProject
+(59) CometProject
Input [14]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
Arguments: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60], [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
-(64) ColumnarToRow [codegen id : 2]
+(60) ColumnarToRow [codegen id : 2]
Input [12]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
-(65) ReusedExchange [Reuses operator id: 33]
+(61) ReusedExchange [Reuses operator id: 33]
Output [3]: [ca_state#61, ca_zip#62, ca_country#63]
-(66) BroadcastHashJoin [codegen id : 2]
+(62) BroadcastHashJoin [codegen id : 2]
Left keys [2]: [c_birth_country#60, s_zip#50]
Right keys [2]: [upper(ca_country#63), ca_zip#62]
Join type: Inner
Join condition: None
-(67) Project [codegen id : 2]
+(63) Project [codegen id : 2]
Output [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
Input [15]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60, ca_state#61, ca_zip#62, ca_country#63]
-(68) HashAggregate [codegen id : 2]
+(64) HashAggregate [codegen id : 2]
Input [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#44))]
Aggregate Attributes [1]: [sum#64]
Results [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-(69) RowToColumnar
+(65) Exchange
Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
+Arguments: hashpartitioning(c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, 5), ENSURE_REQUIREMENTS, [plan_id=6]
-(70) CometColumnarExchange
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-Arguments: hashpartitioning(c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
-
-(71) ColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-
-(72) HashAggregate [codegen id : 3]
+(66) HashAggregate [codegen id : 3]
Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
Functions [1]: [sum(UnscaledValue(ss_net_paid#44))]
Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#44))#30]
Results [1]: [MakeDecimal(sum(UnscaledValue(ss_net_paid#44))#30,17,2) AS netpaid#66]
-(73) HashAggregate [codegen id : 3]
+(67) HashAggregate [codegen id : 3]
Input [1]: [netpaid#66]
Keys: []
Functions [1]: [partial_avg(netpaid#66)]
Aggregate Attributes [2]: [sum#67, count#68]
Results [2]: [sum#69, count#70]
-(74) RowToColumnar
-Input [2]: [sum#69, count#70]
-
-(75) CometColumnarExchange
-Input [2]: [sum#69, count#70]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
-
-(76) ColumnarToRow [codegen id : 4]
+(68) Exchange
Input [2]: [sum#69, count#70]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=7]
-(77) HashAggregate [codegen id : 4]
+(69) HashAggregate [codegen id : 4]
Input [2]: [sum#69, count#70]
Keys: []
Functions [1]: [avg(netpaid#66)]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt
index 0b53b6fc..bd14d825 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt
@@ -3,92 +3,84 @@ WholeStageCodegen (4)
Subquery #1
WholeStageCodegen (4)
HashAggregate [sum,count] [avg(netpaid),(0.05 * avg(netpaid)),sum,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange #9
- RowToColumnar
- WholeStageCodegen (3)
- HashAggregate [netpaid] [sum,count,sum,count]
- HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,sum] [sum(UnscaledValue(ss_net_paid)),netpaid,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #10
- RowToColumnar
- WholeStageCodegen (2)
- HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
- Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
- BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
- ColumnarToRow
- InputAdapter
- CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
- CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
- CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
- CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
- CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
- ReusedExchange [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid] #3
- CometSort [sr_item_sk,sr_ticket_number]
- ReusedExchange [sr_item_sk,sr_ticket_number] #4
- ReusedExchange [s_store_sk,s_store_name,s_state,s_zip] #5
- CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #11
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- ReusedExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
- InputAdapter
- ReusedExchange [ca_state,ca_zip,ca_country] #8
- HashAggregate [c_last_name,c_first_name,s_store_name,sum,isEmpty] [sum(netpaid),paid,sum,isEmpty]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [c_last_name,c_first_name,s_store_name] #1
- RowToColumnar
+ InputAdapter
+ Exchange #9
WholeStageCodegen (3)
- HashAggregate [c_last_name,c_first_name,s_store_name,netpaid] [sum,isEmpty,sum,isEmpty]
+ HashAggregate [netpaid] [sum,count,sum,count]
HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,sum] [sum(UnscaledValue(ss_net_paid)),netpaid,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #2
- RowToColumnar
- WholeStageCodegen (2)
- HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
- Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
- BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
- ColumnarToRow
- InputAdapter
- CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
- CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
- CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
- CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
- CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
- CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
- CometColumnarExchange [ss_ticket_number,ss_item_sk] #3
- CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
- CometFilter [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
- CometSort [sr_item_sk,sr_ticket_number]
- CometColumnarExchange [sr_ticket_number,sr_item_sk] #4
- CometProject [sr_item_sk,sr_ticket_number]
- CometFilter [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
- CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
- CometBroadcastExchange [s_store_sk,s_store_name,s_state,s_zip] #5
- CometProject [s_store_sk,s_store_name,s_state,s_zip]
- CometFilter [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
- CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #6
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
- CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ InputAdapter
+ Exchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #10
+ WholeStageCodegen (2)
+ HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
+ Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
+ BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
+ ColumnarToRow
+ InputAdapter
+ CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
+ CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
+ CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
+ CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
+ CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
+ ReusedExchange [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid] #3
+ CometSort [sr_item_sk,sr_ticket_number]
+ ReusedExchange [sr_item_sk,sr_ticket_number] #4
+ ReusedExchange [s_store_sk,s_store_name,s_state,s_zip] #5
+ CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #11
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ ReusedExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
+ InputAdapter
+ ReusedExchange [ca_state,ca_zip,ca_country] #8
+ HashAggregate [c_last_name,c_first_name,s_store_name,sum,isEmpty] [sum(netpaid),paid,sum,isEmpty]
+ InputAdapter
+ Exchange [c_last_name,c_first_name,s_store_name] #1
+ WholeStageCodegen (3)
+ HashAggregate [c_last_name,c_first_name,s_store_name,netpaid] [sum,isEmpty,sum,isEmpty]
+ HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,sum] [sum(UnscaledValue(ss_net_paid)),netpaid,sum]
+ InputAdapter
+ Exchange [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size] #2
+ WholeStageCodegen (2)
+ HashAggregate [c_last_name,c_first_name,s_store_name,ca_state,s_state,i_color,i_current_price,i_manager_id,i_units,i_size,ss_net_paid] [sum,sum]
+ Project [ss_net_paid,s_store_name,s_state,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,ca_state]
+ BroadcastHashJoin [c_birth_country,s_zip,ca_country,ca_zip]
+ ColumnarToRow
+ InputAdapter
+ CometProject [ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_first_name,c_last_name,c_birth_country]
+ CometBroadcastHashJoin [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id,c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometProject [ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip,i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [ss_item_sk,ss_customer_sk,ss_net_paid,s_store_name,s_state,s_zip]
+ CometBroadcastHashJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid,s_store_sk,s_store_name,s_state,s_zip]
+ CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_net_paid]
+ CometSortMergeJoin [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,sr_item_sk,sr_ticket_number]
+ CometSort [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
+ CometExchange [ss_ticket_number,ss_item_sk] #3
+ CometProject [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid]
+ CometFilter [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_paid,ss_sold_date_sk]
+ CometSort [sr_item_sk,sr_ticket_number]
+ CometExchange [sr_ticket_number,sr_item_sk] #4
+ CometProject [sr_item_sk,sr_ticket_number]
+ CometFilter [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
+ CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
+ CometBroadcastExchange [s_store_sk,s_store_name,s_state,s_zip] #5
+ CometProject [s_store_sk,s_store_name,s_state,s_zip]
+ CometFilter [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
+ CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #6
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
+ CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ InputAdapter
+ BroadcastExchange #8
+ WholeStageCodegen (1)
+ ColumnarToRow
InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometFilter [ca_state,ca_zip,ca_country]
- CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
+ CometFilter [ca_state,ca_zip,ca_country]
+ CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt
index f12717c3..0d47bdcf 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (42)
+- CometTakeOrderedAndProject (41)
+- CometHashAggregate (40)
- +- CometColumnarExchange (39)
+ +- CometExchange (39)
+- CometHashAggregate (38)
+- CometProject (37)
+- CometBroadcastHashJoin (36)
@@ -226,9 +226,9 @@ Input [7]: [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_
Keys [4]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28]
Functions [3]: [partial_sum(UnscaledValue(ss_net_profit#5)), partial_sum(UnscaledValue(sr_net_loss#11)), partial_sum(UnscaledValue(cs_net_profit#16))]
-(39) CometColumnarExchange
+(39) CometExchange
Input [7]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, sum#32, sum#33, sum#34]
-Arguments: hashpartitioning(i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(40) CometHashAggregate
Input [7]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, sum#32, sum#33, sum#34]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt
index c2c652e2..13cb0505 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
CometHashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum,sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit))]
- CometColumnarExchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
+ CometExchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
CometHashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum,ss_net_profit,sr_net_loss,cs_net_profit]
CometProject [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
CometBroadcastHashJoin [ss_item_sk,ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_sk,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt
index cb4f53a0..5ba7d13f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (30)
+- CometTakeOrderedAndProject (29)
+- CometHashAggregate (28)
- +- CometColumnarExchange (27)
+ +- CometExchange (27)
+- CometHashAggregate (26)
+- CometProject (25)
+- CometBroadcastHashJoin (24)
@@ -156,9 +156,9 @@ Input [5]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i
Keys [1]: [i_item_id#17]
Functions [4]: [partial_avg(cs_quantity#4), partial_avg(UnscaledValue(cs_list_price#5)), partial_avg(UnscaledValue(cs_coupon_amt#7)), partial_avg(UnscaledValue(cs_sales_price#6))]
-(27) CometColumnarExchange
+(27) CometExchange
Input [9]: [i_item_id#17, sum#21, count#22, sum#23, count#24, sum#25, count#26, sum#27, count#28]
-Arguments: hashpartitioning(i_item_id#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(28) CometHashAggregate
Input [9]: [i_item_id#17, sum#21, count#22, sum#23, count#24, sum#25, count#26, sum#27, count#28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt
index 08745014..f643ff7b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,agg1,agg2,agg3,agg4]
CometHashAggregate [i_item_id,agg1,agg2,agg3,agg4,sum,count,sum,count,sum,count,sum,count,avg(cs_quantity),avg(UnscaledValue(cs_list_price)),avg(UnscaledValue(cs_coupon_amt)),avg(UnscaledValue(cs_sales_price))]
- CometColumnarExchange [i_item_id] #1
+ CometExchange [i_item_id] #1
CometHashAggregate [i_item_id,sum,count,sum,count,sum,count,sum,count,cs_quantity,cs_list_price,cs_coupon_amt,cs_sales_price]
CometProject [cs_quantity,cs_list_price,cs_sales_price,cs_coupon_amt,i_item_id]
CometBroadcastHashJoin [cs_promo_sk,cs_quantity,cs_list_price,cs_sales_price,cs_coupon_amt,i_item_id,p_promo_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt
index 129139b0..150b31fd 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (30)
+- CometTakeOrderedAndProject (29)
+- CometHashAggregate (28)
- +- CometColumnarExchange (27)
+ +- CometExchange (27)
+- CometHashAggregate (26)
+- CometExpand (25)
+- CometProject (24)
@@ -156,9 +156,9 @@ Input [7]: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i
Keys [3]: [i_item_id#20, s_state#21, spark_grouping_id#22]
Functions [4]: [partial_avg(ss_quantity#4), partial_avg(UnscaledValue(ss_list_price#5)), partial_avg(UnscaledValue(ss_coupon_amt#7)), partial_avg(UnscaledValue(ss_sales_price#6))]
-(27) CometColumnarExchange
+(27) CometExchange
Input [11]: [i_item_id#20, s_state#21, spark_grouping_id#22, sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29, count#30]
-Arguments: hashpartitioning(i_item_id#20, s_state#21, spark_grouping_id#22, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#20, s_state#21, spark_grouping_id#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(28) CometHashAggregate
Input [11]: [i_item_id#20, s_state#21, spark_grouping_id#22, sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29, count#30]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt
index 0c2fe68c..121f1be7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,s_state,g_state,agg1,agg2,agg3,agg4]
CometHashAggregate [i_item_id,s_state,g_state,agg1,agg2,agg3,agg4,spark_grouping_id,sum,count,sum,count,sum,count,sum,count,avg(ss_quantity),avg(UnscaledValue(ss_list_price)),avg(UnscaledValue(ss_coupon_amt)),avg(UnscaledValue(ss_sales_price))]
- CometColumnarExchange [i_item_id,s_state,spark_grouping_id] #1
+ CometExchange [i_item_id,s_state,spark_grouping_id] #1
CometHashAggregate [i_item_id,s_state,spark_grouping_id,sum,count,sum,count,sum,count,sum,count,ss_quantity,ss_list_price,ss_coupon_amt,ss_sales_price]
CometExpand [i_item_id,s_state] [ss_quantity,ss_list_price,ss_sales_price,ss_coupon_amt,i_item_id,s_state,spark_grouping_id]
CometProject [ss_quantity,ss_list_price,ss_sales_price,ss_coupon_amt,i_item_id,s_state]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/explain.txt
index 8881b95a..29af7273 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/explain.txt
@@ -1,86 +1,74 @@
== Physical Plan ==
-* BroadcastNestedLoopJoin Inner BuildRight (82)
-:- * BroadcastNestedLoopJoin Inner BuildRight (68)
-: :- * BroadcastNestedLoopJoin Inner BuildRight (54)
-: : :- * BroadcastNestedLoopJoin Inner BuildRight (40)
-: : : :- * BroadcastNestedLoopJoin Inner BuildRight (26)
-: : : : :- * ColumnarToRow (12)
-: : : : : +- CometHashAggregate (11)
-: : : : : +- CometColumnarExchange (10)
-: : : : : +- RowToColumnar (9)
-: : : : : +- * HashAggregate (8)
-: : : : : +- * HashAggregate (7)
-: : : : : +- * ColumnarToRow (6)
-: : : : : +- CometColumnarExchange (5)
-: : : : : +- CometHashAggregate (4)
-: : : : : +- CometProject (3)
-: : : : : +- CometFilter (2)
-: : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
-: : : : +- BroadcastExchange (25)
-: : : : +- * ColumnarToRow (24)
-: : : : +- CometHashAggregate (23)
-: : : : +- CometColumnarExchange (22)
-: : : : +- RowToColumnar (21)
-: : : : +- * HashAggregate (20)
-: : : : +- * HashAggregate (19)
-: : : : +- * ColumnarToRow (18)
-: : : : +- CometColumnarExchange (17)
-: : : : +- CometHashAggregate (16)
-: : : : +- CometProject (15)
-: : : : +- CometFilter (14)
-: : : : +- CometScan parquet spark_catalog.default.store_sales (13)
-: : : +- BroadcastExchange (39)
-: : : +- * ColumnarToRow (38)
-: : : +- CometHashAggregate (37)
-: : : +- CometColumnarExchange (36)
-: : : +- RowToColumnar (35)
-: : : +- * HashAggregate (34)
-: : : +- * HashAggregate (33)
-: : : +- * ColumnarToRow (32)
-: : : +- CometColumnarExchange (31)
-: : : +- CometHashAggregate (30)
-: : : +- CometProject (29)
-: : : +- CometFilter (28)
-: : : +- CometScan parquet spark_catalog.default.store_sales (27)
-: : +- BroadcastExchange (53)
-: : +- * ColumnarToRow (52)
-: : +- CometHashAggregate (51)
-: : +- CometColumnarExchange (50)
-: : +- RowToColumnar (49)
-: : +- * HashAggregate (48)
-: : +- * HashAggregate (47)
-: : +- * ColumnarToRow (46)
-: : +- CometColumnarExchange (45)
-: : +- CometHashAggregate (44)
-: : +- CometProject (43)
-: : +- CometFilter (42)
-: : +- CometScan parquet spark_catalog.default.store_sales (41)
-: +- BroadcastExchange (67)
-: +- * ColumnarToRow (66)
-: +- CometHashAggregate (65)
-: +- CometColumnarExchange (64)
-: +- RowToColumnar (63)
-: +- * HashAggregate (62)
-: +- * HashAggregate (61)
-: +- * ColumnarToRow (60)
-: +- CometColumnarExchange (59)
-: +- CometHashAggregate (58)
-: +- CometProject (57)
-: +- CometFilter (56)
-: +- CometScan parquet spark_catalog.default.store_sales (55)
-+- BroadcastExchange (81)
- +- * ColumnarToRow (80)
- +- CometHashAggregate (79)
- +- CometColumnarExchange (78)
- +- RowToColumnar (77)
- +- * HashAggregate (76)
- +- * HashAggregate (75)
- +- * ColumnarToRow (74)
- +- CometColumnarExchange (73)
- +- CometHashAggregate (72)
- +- CometProject (71)
- +- CometFilter (70)
- +- CometScan parquet spark_catalog.default.store_sales (69)
+* BroadcastNestedLoopJoin Inner BuildRight (70)
+:- * BroadcastNestedLoopJoin Inner BuildRight (58)
+: :- * BroadcastNestedLoopJoin Inner BuildRight (46)
+: : :- * BroadcastNestedLoopJoin Inner BuildRight (34)
+: : : :- * BroadcastNestedLoopJoin Inner BuildRight (22)
+: : : : :- * HashAggregate (10)
+: : : : : +- Exchange (9)
+: : : : : +- * HashAggregate (8)
+: : : : : +- * HashAggregate (7)
+: : : : : +- * ColumnarToRow (6)
+: : : : : +- CometExchange (5)
+: : : : : +- CometHashAggregate (4)
+: : : : : +- CometProject (3)
+: : : : : +- CometFilter (2)
+: : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
+: : : : +- BroadcastExchange (21)
+: : : : +- * HashAggregate (20)
+: : : : +- Exchange (19)
+: : : : +- * HashAggregate (18)
+: : : : +- * HashAggregate (17)
+: : : : +- * ColumnarToRow (16)
+: : : : +- CometExchange (15)
+: : : : +- CometHashAggregate (14)
+: : : : +- CometProject (13)
+: : : : +- CometFilter (12)
+: : : : +- CometScan parquet spark_catalog.default.store_sales (11)
+: : : +- BroadcastExchange (33)
+: : : +- * HashAggregate (32)
+: : : +- Exchange (31)
+: : : +- * HashAggregate (30)
+: : : +- * HashAggregate (29)
+: : : +- * ColumnarToRow (28)
+: : : +- CometExchange (27)
+: : : +- CometHashAggregate (26)
+: : : +- CometProject (25)
+: : : +- CometFilter (24)
+: : : +- CometScan parquet spark_catalog.default.store_sales (23)
+: : +- BroadcastExchange (45)
+: : +- * HashAggregate (44)
+: : +- Exchange (43)
+: : +- * HashAggregate (42)
+: : +- * HashAggregate (41)
+: : +- * ColumnarToRow (40)
+: : +- CometExchange (39)
+: : +- CometHashAggregate (38)
+: : +- CometProject (37)
+: : +- CometFilter (36)
+: : +- CometScan parquet spark_catalog.default.store_sales (35)
+: +- BroadcastExchange (57)
+: +- * HashAggregate (56)
+: +- Exchange (55)
+: +- * HashAggregate (54)
+: +- * HashAggregate (53)
+: +- * ColumnarToRow (52)
+: +- CometExchange (51)
+: +- CometHashAggregate (50)
+: +- CometProject (49)
+: +- CometFilter (48)
+: +- CometScan parquet spark_catalog.default.store_sales (47)
++- BroadcastExchange (69)
+ +- * HashAggregate (68)
+ +- Exchange (67)
+ +- * HashAggregate (66)
+ +- * HashAggregate (65)
+ +- * ColumnarToRow (64)
+ +- CometExchange (63)
+ +- CometHashAggregate (62)
+ +- CometProject (61)
+ +- CometFilter (60)
+ +- CometScan parquet spark_catalog.default.store_sales (59)
(1) Scan parquet spark_catalog.default.store_sales
@@ -103,9 +91,9 @@ Input [1]: [ss_list_price#3]
Keys [1]: [ss_list_price#3]
Functions [2]: [partial_avg(UnscaledValue(ss_list_price#3)), partial_count(ss_list_price#3)]
-(5) CometColumnarExchange
+(5) CometExchange
Input [4]: [ss_list_price#3, sum#6, count#7, count#8]
-Arguments: hashpartitioning(ss_list_price#3, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_list_price#3, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(6) ColumnarToRow [codegen id : 1]
Input [4]: [ss_list_price#3, sum#6, count#7, count#8]
@@ -124,338 +112,314 @@ Functions [3]: [merge_avg(UnscaledValue(ss_list_price#3)), merge_count(ss_list_p
Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#3))#9, count(ss_list_price#3)#10, count(ss_list_price#3)#11]
Results [4]: [sum#6, count#7, count#8, count#12]
-(9) RowToColumnar
+(9) Exchange
Input [4]: [sum#6, count#7, count#8, count#12]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=2]
-(10) CometColumnarExchange
-Input [4]: [sum#6, count#7, count#8, count#12]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
-
-(11) CometHashAggregate
+(10) HashAggregate [codegen id : 12]
Input [4]: [sum#6, count#7, count#8, count#12]
Keys: []
Functions [3]: [avg(UnscaledValue(ss_list_price#3)), count(ss_list_price#3), count(distinct ss_list_price#3)]
+Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#3))#9, count(ss_list_price#3)#10, count(ss_list_price#3)#11]
+Results [3]: [cast((avg(UnscaledValue(ss_list_price#3))#9 / 100.0) as decimal(11,6)) AS B1_LP#13, count(ss_list_price#3)#10 AS B1_CNT#14, count(ss_list_price#3)#11 AS B1_CNTD#15]
-(12) ColumnarToRow [codegen id : 12]
-Input [3]: [B1_LP#13, B1_CNT#14, B1_CNTD#15]
-
-(13) Scan parquet spark_catalog.default.store_sales
+(11) Scan parquet spark_catalog.default.store_sales
Output [5]: [ss_quantity#16, ss_wholesale_cost#17, ss_list_price#18, ss_coupon_amt#19, ss_sold_date_sk#20]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store_sales]
PushedFilters: [IsNotNull(ss_quantity), GreaterThanOrEqual(ss_quantity,6), LessThanOrEqual(ss_quantity,10), Or(Or(And(GreaterThanOrEqual(ss_list_price,90.00),LessThanOrEqual(ss_list_price,100.00)),And(GreaterThanOrEqual(ss_coupon_amt,2323.00),LessThanOrEqual(ss_coupon_amt,3323.00))),And(GreaterThanOrEqual(ss_wholesale_cost,31.00),LessThanOrEqual(ss_wholesale_cost,51.00)))]
ReadSchema: struct<ss_quantity:int,ss_wholesale_cost:decimal(7,2),ss_list_price:decimal(7,2),ss_coupon_amt:decimal(7,2)>
-(14) CometFilter
+(12) CometFilter
Input [5]: [ss_quantity#16, ss_wholesale_cost#17, ss_list_price#18, ss_coupon_amt#19, ss_sold_date_sk#20]
Condition : (((isnotnull(ss_quantity#16) AND (ss_quantity#16 >= 6)) AND (ss_quantity#16 <= 10)) AND ((((ss_list_price#18 >= 90.00) AND (ss_list_price#18 <= 100.00)) OR ((ss_coupon_amt#19 >= 2323.00) AND (ss_coupon_amt#19 <= 3323.00))) OR ((ss_wholesale_cost#17 >= 31.00) AND (ss_wholesale_cost#17 <= 51.00))))
-(15) CometProject
+(13) CometProject
Input [5]: [ss_quantity#16, ss_wholesale_cost#17, ss_list_price#18, ss_coupon_amt#19, ss_sold_date_sk#20]
Arguments: [ss_list_price#18], [ss_list_price#18]
-(16) CometHashAggregate
+(14) CometHashAggregate
Input [1]: [ss_list_price#18]
Keys [1]: [ss_list_price#18]
Functions [2]: [partial_avg(UnscaledValue(ss_list_price#18)), partial_count(ss_list_price#18)]
-(17) CometColumnarExchange
+(15) CometExchange
Input [4]: [ss_list_price#18, sum#21, count#22, count#23]
-Arguments: hashpartitioning(ss_list_price#18, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(ss_list_price#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
-(18) ColumnarToRow [codegen id : 2]
+(16) ColumnarToRow [codegen id : 2]
Input [4]: [ss_list_price#18, sum#21, count#22, count#23]
-(19) HashAggregate [codegen id : 2]
+(17) HashAggregate [codegen id : 2]
Input [4]: [ss_list_price#18, sum#21, count#22, count#23]
Keys [1]: [ss_list_price#18]
Functions [2]: [merge_avg(UnscaledValue(ss_list_price#18)), merge_count(ss_list_price#18)]
Aggregate Attributes [2]: [avg(UnscaledValue(ss_list_price#18))#24, count(ss_list_price#18)#25]
Results [4]: [ss_list_price#18, sum#21, count#22, count#23]
-(20) HashAggregate [codegen id : 2]
+(18) HashAggregate [codegen id : 2]
Input [4]: [ss_list_price#18, sum#21, count#22, count#23]
Keys: []
Functions [3]: [merge_avg(UnscaledValue(ss_list_price#18)), merge_count(ss_list_price#18), partial_count(distinct ss_list_price#18)]
Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#18))#24, count(ss_list_price#18)#25, count(ss_list_price#18)#26]
Results [4]: [sum#21, count#22, count#23, count#27]
-(21) RowToColumnar
+(19) Exchange
Input [4]: [sum#21, count#22, count#23, count#27]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=4]
-(22) CometColumnarExchange
-Input [4]: [sum#21, count#22, count#23, count#27]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-
-(23) CometHashAggregate
+(20) HashAggregate [codegen id : 3]
Input [4]: [sum#21, count#22, count#23, count#27]
Keys: []
Functions [3]: [avg(UnscaledValue(ss_list_price#18)), count(ss_list_price#18), count(distinct ss_list_price#18)]
+Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#18))#24, count(ss_list_price#18)#25, count(ss_list_price#18)#26]
+Results [3]: [cast((avg(UnscaledValue(ss_list_price#18))#24 / 100.0) as decimal(11,6)) AS B2_LP#28, count(ss_list_price#18)#25 AS B2_CNT#29, count(ss_list_price#18)#26 AS B2_CNTD#30]
-(24) ColumnarToRow [codegen id : 3]
-Input [3]: [B2_LP#28, B2_CNT#29, B2_CNTD#30]
-
-(25) BroadcastExchange
+(21) BroadcastExchange
Input [3]: [B2_LP#28, B2_CNT#29, B2_CNTD#30]
Arguments: IdentityBroadcastMode, [plan_id=5]
-(26) BroadcastNestedLoopJoin [codegen id : 12]
+(22) BroadcastNestedLoopJoin [codegen id : 12]
Join type: Inner
Join condition: None
-(27) Scan parquet spark_catalog.default.store_sales
+(23) Scan parquet spark_catalog.default.store_sales
Output [5]: [ss_quantity#31, ss_wholesale_cost#32, ss_list_price#33, ss_coupon_amt#34, ss_sold_date_sk#35]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store_sales]
PushedFilters: [IsNotNull(ss_quantity), GreaterThanOrEqual(ss_quantity,11), LessThanOrEqual(ss_quantity,15), Or(Or(And(GreaterThanOrEqual(ss_list_price,142.00),LessThanOrEqual(ss_list_price,152.00)),And(GreaterThanOrEqual(ss_coupon_amt,12214.00),LessThanOrEqual(ss_coupon_amt,13214.00))),And(GreaterThanOrEqual(ss_wholesale_cost,79.00),LessThanOrEqual(ss_wholesale_cost,99.00)))]
ReadSchema: struct<ss_quantity:int,ss_wholesale_cost:decimal(7,2),ss_list_price:decimal(7,2),ss_coupon_amt:decimal(7,2)>
-(28) CometFilter
+(24) CometFilter
Input [5]: [ss_quantity#31, ss_wholesale_cost#32, ss_list_price#33, ss_coupon_amt#34, ss_sold_date_sk#35]
Condition : (((isnotnull(ss_quantity#31) AND (ss_quantity#31 >= 11)) AND (ss_quantity#31 <= 15)) AND ((((ss_list_price#33 >= 142.00) AND (ss_list_price#33 <= 152.00)) OR ((ss_coupon_amt#34 >= 12214.00) AND (ss_coupon_amt#34 <= 13214.00))) OR ((ss_wholesale_cost#32 >= 79.00) AND (ss_wholesale_cost#32 <= 99.00))))
-(29) CometProject
+(25) CometProject
Input [5]: [ss_quantity#31, ss_wholesale_cost#32, ss_list_price#33, ss_coupon_amt#34, ss_sold_date_sk#35]
Arguments: [ss_list_price#33], [ss_list_price#33]
-(30) CometHashAggregate
+(26) CometHashAggregate
Input [1]: [ss_list_price#33]
Keys [1]: [ss_list_price#33]
Functions [2]: [partial_avg(UnscaledValue(ss_list_price#33)), partial_count(ss_list_price#33)]
-(31) CometColumnarExchange
+(27) CometExchange
Input [4]: [ss_list_price#33, sum#36, count#37, count#38]
-Arguments: hashpartitioning(ss_list_price#33, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: hashpartitioning(ss_list_price#33, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
-(32) ColumnarToRow [codegen id : 4]
+(28) ColumnarToRow [codegen id : 4]
Input [4]: [ss_list_price#33, sum#36, count#37, count#38]
-(33) HashAggregate [codegen id : 4]
+(29) HashAggregate [codegen id : 4]
Input [4]: [ss_list_price#33, sum#36, count#37, count#38]
Keys [1]: [ss_list_price#33]
Functions [2]: [merge_avg(UnscaledValue(ss_list_price#33)), merge_count(ss_list_price#33)]
Aggregate Attributes [2]: [avg(UnscaledValue(ss_list_price#33))#39, count(ss_list_price#33)#40]
Results [4]: [ss_list_price#33, sum#36, count#37, count#38]
-(34) HashAggregate [codegen id : 4]
+(30) HashAggregate [codegen id : 4]
Input [4]: [ss_list_price#33, sum#36, count#37, count#38]
Keys: []
Functions [3]: [merge_avg(UnscaledValue(ss_list_price#33)), merge_count(ss_list_price#33), partial_count(distinct ss_list_price#33)]
Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#33))#39, count(ss_list_price#33)#40, count(ss_list_price#33)#41]
Results [4]: [sum#36, count#37, count#38, count#42]
-(35) RowToColumnar
-Input [4]: [sum#36, count#37, count#38, count#42]
-
-(36) CometColumnarExchange
+(31) Exchange
Input [4]: [sum#36, count#37, count#38, count#42]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=7]
-(37) CometHashAggregate
+(32) HashAggregate [codegen id : 5]
Input [4]: [sum#36, count#37, count#38, count#42]
Keys: []
Functions [3]: [avg(UnscaledValue(ss_list_price#33)), count(ss_list_price#33), count(distinct ss_list_price#33)]
+Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#33))#39, count(ss_list_price#33)#40, count(ss_list_price#33)#41]
+Results [3]: [cast((avg(UnscaledValue(ss_list_price#33))#39 / 100.0) as decimal(11,6)) AS B3_LP#43, count(ss_list_price#33)#40 AS B3_CNT#44, count(ss_list_price#33)#41 AS B3_CNTD#45]
-(38) ColumnarToRow [codegen id : 5]
-Input [3]: [B3_LP#43, B3_CNT#44, B3_CNTD#45]
-
-(39) BroadcastExchange
+(33) BroadcastExchange
Input [3]: [B3_LP#43, B3_CNT#44, B3_CNTD#45]
Arguments: IdentityBroadcastMode, [plan_id=8]
-(40) BroadcastNestedLoopJoin [codegen id : 12]
+(34) BroadcastNestedLoopJoin [codegen id : 12]
Join type: Inner
Join condition: None
-(41) Scan parquet spark_catalog.default.store_sales
+(35) Scan parquet spark_catalog.default.store_sales
Output [5]: [ss_quantity#46, ss_wholesale_cost#47, ss_list_price#48, ss_coupon_amt#49, ss_sold_date_sk#50]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store_sales]
PushedFilters: [IsNotNull(ss_quantity), GreaterThanOrEqual(ss_quantity,16), LessThanOrEqual(ss_quantity,20), Or(Or(And(GreaterThanOrEqual(ss_list_price,135.00),LessThanOrEqual(ss_list_price,145.00)),And(GreaterThanOrEqual(ss_coupon_amt,6071.00),LessThanOrEqual(ss_coupon_amt,7071.00))),And(GreaterThanOrEqual(ss_wholesale_cost,38.00),LessThanOrEqual(ss_wholesale_cost,58.00)))]
ReadSchema: struct<ss_quantity:int,ss_wholesale_cost:decimal(7,2),ss_list_price:decimal(7,2),ss_coupon_amt:decimal(7,2)>
-(42) CometFilter
+(36) CometFilter
Input [5]: [ss_quantity#46, ss_wholesale_cost#47, ss_list_price#48, ss_coupon_amt#49, ss_sold_date_sk#50]
Condition : (((isnotnull(ss_quantity#46) AND (ss_quantity#46 >= 16)) AND (ss_quantity#46 <= 20)) AND ((((ss_list_price#48 >= 135.00) AND (ss_list_price#48 <= 145.00)) OR ((ss_coupon_amt#49 >= 6071.00) AND (ss_coupon_amt#49 <= 7071.00))) OR ((ss_wholesale_cost#47 >= 38.00) AND (ss_wholesale_cost#47 <= 58.00))))
-(43) CometProject
+(37) CometProject
Input [5]: [ss_quantity#46, ss_wholesale_cost#47, ss_list_price#48, ss_coupon_amt#49, ss_sold_date_sk#50]
Arguments: [ss_list_price#48], [ss_list_price#48]
-(44) CometHashAggregate
+(38) CometHashAggregate
Input [1]: [ss_list_price#48]
Keys [1]: [ss_list_price#48]
Functions [2]: [partial_avg(UnscaledValue(ss_list_price#48)), partial_count(ss_list_price#48)]
-(45) CometColumnarExchange
+(39) CometExchange
Input [4]: [ss_list_price#48, sum#51, count#52, count#53]
-Arguments: hashpartitioning(ss_list_price#48, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=9]
+Arguments: hashpartitioning(ss_list_price#48, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=9]
-(46) ColumnarToRow [codegen id : 6]
+(40) ColumnarToRow [codegen id : 6]
Input [4]: [ss_list_price#48, sum#51, count#52, count#53]
-(47) HashAggregate [codegen id : 6]
+(41) HashAggregate [codegen id : 6]
Input [4]: [ss_list_price#48, sum#51, count#52, count#53]
Keys [1]: [ss_list_price#48]
Functions [2]: [merge_avg(UnscaledValue(ss_list_price#48)), merge_count(ss_list_price#48)]
Aggregate Attributes [2]: [avg(UnscaledValue(ss_list_price#48))#54, count(ss_list_price#48)#55]
Results [4]: [ss_list_price#48, sum#51, count#52, count#53]
-(48) HashAggregate [codegen id : 6]
+(42) HashAggregate [codegen id : 6]
Input [4]: [ss_list_price#48, sum#51, count#52, count#53]
Keys: []
Functions [3]: [merge_avg(UnscaledValue(ss_list_price#48)), merge_count(ss_list_price#48), partial_count(distinct ss_list_price#48)]
Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#48))#54, count(ss_list_price#48)#55, count(ss_list_price#48)#56]
Results [4]: [sum#51, count#52, count#53, count#57]
-(49) RowToColumnar
+(43) Exchange
Input [4]: [sum#51, count#52, count#53, count#57]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=10]
-(50) CometColumnarExchange
-Input [4]: [sum#51, count#52, count#53, count#57]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=10]
-
-(51) CometHashAggregate
+(44) HashAggregate [codegen id : 7]
Input [4]: [sum#51, count#52, count#53, count#57]
Keys: []
Functions [3]: [avg(UnscaledValue(ss_list_price#48)), count(ss_list_price#48), count(distinct ss_list_price#48)]
+Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#48))#54, count(ss_list_price#48)#55, count(ss_list_price#48)#56]
+Results [3]: [cast((avg(UnscaledValue(ss_list_price#48))#54 / 100.0) as decimal(11,6)) AS B4_LP#58, count(ss_list_price#48)#55 AS B4_CNT#59, count(ss_list_price#48)#56 AS B4_CNTD#60]
-(52) ColumnarToRow [codegen id : 7]
-Input [3]: [B4_LP#58, B4_CNT#59, B4_CNTD#60]
-
-(53) BroadcastExchange
+(45) BroadcastExchange
Input [3]: [B4_LP#58, B4_CNT#59, B4_CNTD#60]
Arguments: IdentityBroadcastMode, [plan_id=11]
-(54) BroadcastNestedLoopJoin [codegen id : 12]
+(46) BroadcastNestedLoopJoin [codegen id : 12]
Join type: Inner
Join condition: None
-(55) Scan parquet spark_catalog.default.store_sales
+(47) Scan parquet spark_catalog.default.store_sales
Output [5]: [ss_quantity#61, ss_wholesale_cost#62, ss_list_price#63, ss_coupon_amt#64, ss_sold_date_sk#65]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store_sales]
PushedFilters: [IsNotNull(ss_quantity), GreaterThanOrEqual(ss_quantity,21), LessThanOrEqual(ss_quantity,25), Or(Or(And(GreaterThanOrEqual(ss_list_price,122.00),LessThanOrEqual(ss_list_price,132.00)),And(GreaterThanOrEqual(ss_coupon_amt,836.00),LessThanOrEqual(ss_coupon_amt,1836.00))),And(GreaterThanOrEqual(ss_wholesale_cost,17.00),LessThanOrEqual(ss_wholesale_cost,37.00)))]
ReadSchema: struct<ss_quantity:int,ss_wholesale_cost:decimal(7,2),ss_list_price:decimal(7,2),ss_coupon_amt:decimal(7,2)>
-(56) CometFilter
+(48) CometFilter
Input [5]: [ss_quantity#61, ss_wholesale_cost#62, ss_list_price#63, ss_coupon_amt#64, ss_sold_date_sk#65]
Condition : (((isnotnull(ss_quantity#61) AND (ss_quantity#61 >= 21)) AND (ss_quantity#61 <= 25)) AND ((((ss_list_price#63 >= 122.00) AND (ss_list_price#63 <= 132.00)) OR ((ss_coupon_amt#64 >= 836.00) AND (ss_coupon_amt#64 <= 1836.00))) OR ((ss_wholesale_cost#62 >= 17.00) AND (ss_wholesale_cost#62 <= 37.00))))
-(57) CometProject
+(49) CometProject
Input [5]: [ss_quantity#61, ss_wholesale_cost#62, ss_list_price#63, ss_coupon_amt#64, ss_sold_date_sk#65]
Arguments: [ss_list_price#63], [ss_list_price#63]
-(58) CometHashAggregate
+(50) CometHashAggregate
Input [1]: [ss_list_price#63]
Keys [1]: [ss_list_price#63]
Functions [2]: [partial_avg(UnscaledValue(ss_list_price#63)), partial_count(ss_list_price#63)]
-(59) CometColumnarExchange
+(51) CometExchange
Input [4]: [ss_list_price#63, sum#66, count#67, count#68]
-Arguments: hashpartitioning(ss_list_price#63, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=12]
+Arguments: hashpartitioning(ss_list_price#63, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=12]
-(60) ColumnarToRow [codegen id : 8]
+(52) ColumnarToRow [codegen id : 8]
Input [4]: [ss_list_price#63, sum#66, count#67, count#68]
-(61) HashAggregate [codegen id : 8]
+(53) HashAggregate [codegen id : 8]
Input [4]: [ss_list_price#63, sum#66, count#67, count#68]
Keys [1]: [ss_list_price#63]
Functions [2]: [merge_avg(UnscaledValue(ss_list_price#63)), merge_count(ss_list_price#63)]
Aggregate Attributes [2]: [avg(UnscaledValue(ss_list_price#63))#69, count(ss_list_price#63)#70]
Results [4]: [ss_list_price#63, sum#66, count#67, count#68]
-(62) HashAggregate [codegen id : 8]
+(54) HashAggregate [codegen id : 8]
Input [4]: [ss_list_price#63, sum#66, count#67, count#68]
Keys: []
Functions [3]: [merge_avg(UnscaledValue(ss_list_price#63)), merge_count(ss_list_price#63), partial_count(distinct ss_list_price#63)]
Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#63))#69, count(ss_list_price#63)#70, count(ss_list_price#63)#71]
Results [4]: [sum#66, count#67, count#68, count#72]
-(63) RowToColumnar
+(55) Exchange
Input [4]: [sum#66, count#67, count#68, count#72]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=13]
-(64) CometColumnarExchange
-Input [4]: [sum#66, count#67, count#68, count#72]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=13]
-
-(65) CometHashAggregate
+(56) HashAggregate [codegen id : 9]
Input [4]: [sum#66, count#67, count#68, count#72]
Keys: []
Functions [3]: [avg(UnscaledValue(ss_list_price#63)), count(ss_list_price#63), count(distinct ss_list_price#63)]
+Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#63))#69, count(ss_list_price#63)#70, count(ss_list_price#63)#71]
+Results [3]: [cast((avg(UnscaledValue(ss_list_price#63))#69 / 100.0) as decimal(11,6)) AS B5_LP#73, count(ss_list_price#63)#70 AS B5_CNT#74, count(ss_list_price#63)#71 AS B5_CNTD#75]
-(66) ColumnarToRow [codegen id : 9]
-Input [3]: [B5_LP#73, B5_CNT#74, B5_CNTD#75]
-
-(67) BroadcastExchange
+(57) BroadcastExchange
Input [3]: [B5_LP#73, B5_CNT#74, B5_CNTD#75]
Arguments: IdentityBroadcastMode, [plan_id=14]
-(68) BroadcastNestedLoopJoin [codegen id : 12]
+(58) BroadcastNestedLoopJoin [codegen id : 12]
Join type: Inner
Join condition: None
-(69) Scan parquet spark_catalog.default.store_sales
+(59) Scan parquet spark_catalog.default.store_sales
Output [5]: [ss_quantity#76, ss_wholesale_cost#77, ss_list_price#78, ss_coupon_amt#79, ss_sold_date_sk#80]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store_sales]
PushedFilters: [IsNotNull(ss_quantity), GreaterThanOrEqual(ss_quantity,26), LessThanOrEqual(ss_quantity,30), Or(Or(And(GreaterThanOrEqual(ss_list_price,154.00),LessThanOrEqual(ss_list_price,164.00)),And(GreaterThanOrEqual(ss_coupon_amt,7326.00),LessThanOrEqual(ss_coupon_amt,8326.00))),And(GreaterThanOrEqual(ss_wholesale_cost,7.00),LessThanOrEqual(ss_wholesale_cost,27.00)))]
ReadSchema: struct<ss_quantity:int,ss_wholesale_cost:decimal(7,2),ss_list_price:decimal(7,2),ss_coupon_amt:decimal(7,2)>
-(70) CometFilter
+(60) CometFilter
Input [5]: [ss_quantity#76, ss_wholesale_cost#77, ss_list_price#78, ss_coupon_amt#79, ss_sold_date_sk#80]
Condition : (((isnotnull(ss_quantity#76) AND (ss_quantity#76 >= 26)) AND (ss_quantity#76 <= 30)) AND ((((ss_list_price#78 >= 154.00) AND (ss_list_price#78 <= 164.00)) OR ((ss_coupon_amt#79 >= 7326.00) AND (ss_coupon_amt#79 <= 8326.00))) OR ((ss_wholesale_cost#77 >= 7.00) AND (ss_wholesale_cost#77 <= 27.00))))
-(71) CometProject
+(61) CometProject
Input [5]: [ss_quantity#76, ss_wholesale_cost#77, ss_list_price#78, ss_coupon_amt#79, ss_sold_date_sk#80]
Arguments: [ss_list_price#78], [ss_list_price#78]
-(72) CometHashAggregate
+(62) CometHashAggregate
Input [1]: [ss_list_price#78]
Keys [1]: [ss_list_price#78]
Functions [2]: [partial_avg(UnscaledValue(ss_list_price#78)), partial_count(ss_list_price#78)]
-(73) CometColumnarExchange
+(63) CometExchange
Input [4]: [ss_list_price#78, sum#81, count#82, count#83]
-Arguments: hashpartitioning(ss_list_price#78, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=15]
+Arguments: hashpartitioning(ss_list_price#78, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=15]
-(74) ColumnarToRow [codegen id : 10]
+(64) ColumnarToRow [codegen id : 10]
Input [4]: [ss_list_price#78, sum#81, count#82, count#83]
-(75) HashAggregate [codegen id : 10]
+(65) HashAggregate [codegen id : 10]
Input [4]: [ss_list_price#78, sum#81, count#82, count#83]
Keys [1]: [ss_list_price#78]
Functions [2]: [merge_avg(UnscaledValue(ss_list_price#78)), merge_count(ss_list_price#78)]
Aggregate Attributes [2]: [avg(UnscaledValue(ss_list_price#78))#84, count(ss_list_price#78)#85]
Results [4]: [ss_list_price#78, sum#81, count#82, count#83]
-(76) HashAggregate [codegen id : 10]
+(66) HashAggregate [codegen id : 10]
Input [4]: [ss_list_price#78, sum#81, count#82, count#83]
Keys: []
Functions [3]: [merge_avg(UnscaledValue(ss_list_price#78)), merge_count(ss_list_price#78), partial_count(distinct ss_list_price#78)]
Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#78))#84, count(ss_list_price#78)#85, count(ss_list_price#78)#86]
Results [4]: [sum#81, count#82, count#83, count#87]
-(77) RowToColumnar
-Input [4]: [sum#81, count#82, count#83, count#87]
-
-(78) CometColumnarExchange
+(67) Exchange
Input [4]: [sum#81, count#82, count#83, count#87]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=16]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=16]
-(79) CometHashAggregate
+(68) HashAggregate [codegen id : 11]
Input [4]: [sum#81, count#82, count#83, count#87]
Keys: []
Functions [3]: [avg(UnscaledValue(ss_list_price#78)), count(ss_list_price#78), count(distinct ss_list_price#78)]
+Aggregate Attributes [3]: [avg(UnscaledValue(ss_list_price#78))#84, count(ss_list_price#78)#85, count(ss_list_price#78)#86]
+Results [3]: [cast((avg(UnscaledValue(ss_list_price#78))#84 / 100.0) as decimal(11,6)) AS B6_LP#88, count(ss_list_price#78)#85 AS B6_CNT#89, count(ss_list_price#78)#86 AS B6_CNTD#90]
-(80) ColumnarToRow [codegen id : 11]
-Input [3]: [B6_LP#88, B6_CNT#89, B6_CNTD#90]
-
-(81) BroadcastExchange
+(69) BroadcastExchange
Input [3]: [B6_LP#88, B6_CNT#89, B6_CNTD#90]
Arguments: IdentityBroadcastMode, [plan_id=17]
-(82) BroadcastNestedLoopJoin [codegen id : 12]
+(70) BroadcastNestedLoopJoin [codegen id : 12]
Join type: Inner
Join condition: None
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/simplified.txt
index 73343f39..4a547c4e 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q28/simplified.txt
@@ -4,108 +4,96 @@ WholeStageCodegen (12)
BroadcastNestedLoopJoin
BroadcastNestedLoopJoin
BroadcastNestedLoopJoin
- ColumnarToRow
+ HashAggregate [sum,count,count,count] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),B1_LP,B1_CNT,B1_CNTD,sum,count,count,count]
InputAdapter
- CometHashAggregate [B1_LP,B1_CNT,B1_CNTD,sum,count,count,count,avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price)]
- CometColumnarExchange #1
- RowToColumnar
- WholeStageCodegen (1)
- HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
- HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [ss_list_price] #2
- CometHashAggregate [ss_list_price,sum,count,count]
- CometProject [ss_list_price]
- CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
+ Exchange #1
+ WholeStageCodegen (1)
+ HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
+ HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
+ ColumnarToRow
+ InputAdapter
+ CometExchange [ss_list_price] #2
+ CometHashAggregate [ss_list_price,sum,count,count]
+ CometProject [ss_list_price]
+ CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (3)
- ColumnarToRow
+ HashAggregate [sum,count,count,count] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),B2_LP,B2_CNT,B2_CNTD,sum,count,count,count]
InputAdapter
- CometHashAggregate [B2_LP,B2_CNT,B2_CNTD,sum,count,count,count,avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price)]
- CometColumnarExchange #4
- RowToColumnar
- WholeStageCodegen (2)
- HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
- HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [ss_list_price] #5
- CometHashAggregate [ss_list_price,sum,count,count]
- CometProject [ss_list_price]
- CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (5)
- ColumnarToRow
- InputAdapter
- CometHashAggregate [B3_LP,B3_CNT,B3_CNTD,sum,count,count,count,avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price)]
- CometColumnarExchange #7
- RowToColumnar
- WholeStageCodegen (4)
- HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
- HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [ss_list_price] #8
- CometHashAggregate [ss_list_price,sum,count,count]
- CometProject [ss_list_price]
- CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- InputAdapter
- BroadcastExchange #9
- WholeStageCodegen (7)
- ColumnarToRow
- InputAdapter
- CometHashAggregate [B4_LP,B4_CNT,B4_CNTD,sum,count,count,count,avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price)]
- CometColumnarExchange #10
- RowToColumnar
- WholeStageCodegen (6)
+ Exchange #4
+ WholeStageCodegen (2)
HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
ColumnarToRow
InputAdapter
- CometColumnarExchange [ss_list_price] #11
+ CometExchange [ss_list_price] #5
CometHashAggregate [ss_list_price,sum,count,count]
CometProject [ss_list_price]
CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- InputAdapter
- BroadcastExchange #12
- WholeStageCodegen (9)
- ColumnarToRow
- InputAdapter
- CometHashAggregate [B5_LP,B5_CNT,B5_CNTD,sum,count,count,count,avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price)]
- CometColumnarExchange #13
- RowToColumnar
- WholeStageCodegen (8)
+ InputAdapter
+ BroadcastExchange #6
+ WholeStageCodegen (5)
+ HashAggregate [sum,count,count,count] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),B3_LP,B3_CNT,B3_CNTD,sum,count,count,count]
+ InputAdapter
+ Exchange #7
+ WholeStageCodegen (4)
HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
ColumnarToRow
InputAdapter
- CometColumnarExchange [ss_list_price] #14
+ CometExchange [ss_list_price] #8
CometHashAggregate [ss_list_price,sum,count,count]
CometProject [ss_list_price]
CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
- InputAdapter
- BroadcastExchange #15
- WholeStageCodegen (11)
- ColumnarToRow
- InputAdapter
- CometHashAggregate [B6_LP,B6_CNT,B6_CNTD,sum,count,count,count,avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price)]
- CometColumnarExchange #16
- RowToColumnar
- WholeStageCodegen (10)
+ InputAdapter
+ BroadcastExchange #9
+ WholeStageCodegen (7)
+ HashAggregate [sum,count,count,count] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),B4_LP,B4_CNT,B4_CNTD,sum,count,count,count]
+ InputAdapter
+ Exchange #10
+ WholeStageCodegen (6)
HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
ColumnarToRow
InputAdapter
- CometColumnarExchange [ss_list_price] #17
+ CometExchange [ss_list_price] #11
CometHashAggregate [ss_list_price,sum,count,count]
CometProject [ss_list_price]
CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
+ InputAdapter
+ BroadcastExchange #12
+ WholeStageCodegen (9)
+ HashAggregate [sum,count,count,count] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),B5_LP,B5_CNT,B5_CNTD,sum,count,count,count]
+ InputAdapter
+ Exchange #13
+ WholeStageCodegen (8)
+ HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
+ HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
+ ColumnarToRow
+ InputAdapter
+ CometExchange [ss_list_price] #14
+ CometHashAggregate [ss_list_price,sum,count,count]
+ CometProject [ss_list_price]
+ CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
+ InputAdapter
+ BroadcastExchange #15
+ WholeStageCodegen (11)
+ HashAggregate [sum,count,count,count] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),B6_LP,B6_CNT,B6_CNTD,sum,count,count,count]
+ InputAdapter
+ Exchange #16
+ WholeStageCodegen (10)
+ HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),count(ss_list_price),sum,count,count,count,sum,count,count,count]
+ HashAggregate [ss_list_price] [avg(UnscaledValue(ss_list_price)),count(ss_list_price),sum,count,count,sum,count,count]
+ ColumnarToRow
+ InputAdapter
+ CometExchange [ss_list_price] #17
+ CometHashAggregate [ss_list_price,sum,count,count]
+ CometProject [ss_list_price]
+ CometFilter [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_quantity,ss_wholesale_cost,ss_list_price,ss_coupon_amt,ss_sold_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt
index a3346549..6248ddee 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (45)
+- CometTakeOrderedAndProject (44)
+- CometHashAggregate (43)
- +- CometColumnarExchange (42)
+ +- CometExchange (42)
+- CometHashAggregate (41)
+- CometProject (40)
+- CometBroadcastHashJoin (39)
@@ -245,9 +245,9 @@ Input [7]: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28,
Keys [4]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29]
Functions [3]: [partial_sum(ss_quantity#5), partial_sum(sr_return_quantity#11), partial_sum(cs_quantity#16)]
-(42) CometColumnarExchange
+(42) CometExchange
Input [7]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, sum#33, sum#34, sum#35]
-Arguments: hashpartitioning(i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(43) CometHashAggregate
Input [7]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, sum#33, sum#34, sum#35]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt
index 3ca13e7b..9398fcdf 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
CometHashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum,sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity)]
- CometColumnarExchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
+ CometExchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
CometHashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum,ss_quantity,sr_return_quantity,cs_quantity]
CometProject [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
CometBroadcastHashJoin [ss_item_sk,ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_sk,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt
index 9079e088..9a63fffa 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (19)
+- CometTakeOrderedAndProject (18)
+- CometHashAggregate (17)
- +- CometColumnarExchange (16)
+ +- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
+- CometBroadcastHashJoin (13)
@@ -93,9 +93,9 @@ Input [4]: [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#9]
Keys [3]: [d_year#2, i_brand#9, i_brand_id#8]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
-(16) CometColumnarExchange
+(16) CometExchange
Input [4]: [d_year#2, i_brand#9, i_brand_id#8, sum#11]
-Arguments: hashpartitioning(d_year#2, i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(d_year#2, i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
Input [4]: [d_year#2, i_brand#9, i_brand_id#8, sum#11]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt
index 65c80bae..f82fd24b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [d_year,brand_id,brand,sum_agg]
CometHashAggregate [d_year,brand_id,brand,sum_agg,i_brand,i_brand_id,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [d_year,i_brand,i_brand_id] #1
+ CometExchange [d_year,i_brand,i_brand_id] #1
CometHashAggregate [d_year,i_brand,i_brand_id,sum,ss_ext_sales_price]
CometProject [d_year,ss_ext_sales_price,i_brand_id,i_brand]
CometBroadcastHashJoin [d_year,ss_item_sk,ss_ext_sales_price,i_item_sk,i_brand_id,i_brand]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt
index a14fa607..3719c24f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt
@@ -9,7 +9,7 @@
: : +- CometBroadcastHashJoin (34)
: : :- CometFilter (17)
: : : +- CometHashAggregate (16)
- : : : +- CometColumnarExchange (15)
+ : : : +- CometExchange (15)
: : : +- CometHashAggregate (14)
: : : +- CometProject (13)
: : : +- CometBroadcastHashJoin (12)
@@ -27,10 +27,10 @@
: : +- CometBroadcastExchange (33)
: : +- CometFilter (32)
: : +- CometHashAggregate (31)
- : : +- CometColumnarExchange (30)
+ : : +- CometExchange (30)
: : +- CometHashAggregate (29)
: : +- CometHashAggregate (28)
- : : +- CometColumnarExchange (27)
+ : : +- CometExchange (27)
: : +- CometHashAggregate (26)
: : +- CometProject (25)
: : +- CometBroadcastHashJoin (24)
@@ -118,9 +118,9 @@ Input [3]: [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#9]
Keys [2]: [wr_returning_customer_sk#1, ca_state#9]
Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#3))]
-(15) CometColumnarExchange
+(15) CometExchange
Input [3]: [wr_returning_customer_sk#1, ca_state#9, sum#10]
-Arguments: hashpartitioning(wr_returning_customer_sk#1, ca_state#9, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(wr_returning_customer_sk#1, ca_state#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(16) CometHashAggregate
Input [3]: [wr_returning_customer_sk#1, ca_state#9, sum#10]
@@ -172,9 +172,9 @@ Input [3]: [wr_returning_customer_sk#14, wr_return_amt#16, ca_state#21]
Keys [2]: [wr_returning_customer_sk#14, ca_state#21]
Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#16))]
-(27) CometColumnarExchange
+(27) CometExchange
Input [3]: [wr_returning_customer_sk#14, ca_state#21, sum#22]
-Arguments: hashpartitioning(wr_returning_customer_sk#14, ca_state#21, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(wr_returning_customer_sk#14, ca_state#21, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(28) CometHashAggregate
Input [3]: [wr_returning_customer_sk#14, ca_state#21, sum#22]
@@ -186,9 +186,9 @@ Input [2]: [ctr_state#23, ctr_total_return#24]
Keys [1]: [ctr_state#23]
Functions [1]: [partial_avg(ctr_total_return#24)]
-(30) CometColumnarExchange
+(30) CometExchange
Input [3]: [ctr_state#23, sum#25, count#26]
-Arguments: hashpartitioning(ctr_state#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(ctr_state#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(31) CometHashAggregate
Input [3]: [ctr_state#23, sum#25, count#26]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt
index 576a8779..824cc511 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt
@@ -10,7 +10,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [ctr_customer_sk,ctr_state,ctr_total_return,(avg(ctr_total_return) * 1.2),ctr_state]
CometFilter [ctr_customer_sk,ctr_state,ctr_total_return]
CometHashAggregate [ctr_customer_sk,ctr_state,ctr_total_return,wr_returning_customer_sk,ca_state,sum,sum(UnscaledValue(wr_return_amt))]
- CometColumnarExchange [wr_returning_customer_sk,ca_state] #1
+ CometExchange [wr_returning_customer_sk,ca_state] #1
CometHashAggregate [wr_returning_customer_sk,ca_state,sum,wr_return_amt]
CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
@@ -36,10 +36,10 @@ WholeStageCodegen (1)
CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_state] #5
CometFilter [(avg(ctr_total_return) * 1.2),ctr_state]
CometHashAggregate [(avg(ctr_total_return) * 1.2),ctr_state,sum,count,avg(ctr_total_return)]
- CometColumnarExchange [ctr_state] #6
+ CometExchange [ctr_state] #6
CometHashAggregate [ctr_state,sum,count,ctr_total_return]
CometHashAggregate [ctr_state,ctr_total_return,wr_returning_customer_sk,ca_state,sum,sum(UnscaledValue(wr_return_amt))]
- CometColumnarExchange [wr_returning_customer_sk,ca_state] #7
+ CometExchange [wr_returning_customer_sk,ca_state] #7
CometHashAggregate [wr_returning_customer_sk,ca_state,sum,wr_return_amt]
CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/explain.txt
index 7284a48b..e5bc4e7a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/explain.txt
@@ -11,7 +11,7 @@
: : : +- CometBroadcastHashJoin (45)
: : : :- CometBroadcastHashJoin (30)
: : : : :- CometHashAggregate (15)
- : : : : : +- CometColumnarExchange (14)
+ : : : : : +- CometExchange (14)
: : : : : +- CometHashAggregate (13)
: : : : : +- CometProject (12)
: : : : : +- CometBroadcastHashJoin (11)
@@ -27,7 +27,7 @@
: : : : : +- CometScan parquet spark_catalog.default.customer_address (8)
: : : : +- CometBroadcastExchange (29)
: : : : +- CometHashAggregate (28)
- : : : : +- CometColumnarExchange (27)
+ : : : : +- CometExchange (27)
: : : : +- CometHashAggregate (26)
: : : : +- CometProject (25)
: : : : +- CometBroadcastHashJoin (24)
@@ -41,7 +41,7 @@
: : : : +- ReusedExchange (23)
: : : +- CometBroadcastExchange (44)
: : : +- CometHashAggregate (43)
- : : : +- CometColumnarExchange (42)
+ : : : +- CometExchange (42)
: : : +- CometHashAggregate (41)
: : : +- CometProject (40)
: : : +- CometBroadcastHashJoin (39)
@@ -55,7 +55,7 @@
: : : +- ReusedExchange (38)
: : +- CometBroadcastExchange (58)
: : +- CometHashAggregate (57)
- : : +- CometColumnarExchange (56)
+ : : +- CometExchange (56)
: : +- CometHashAggregate (55)
: : +- CometProject (54)
: : +- CometBroadcastHashJoin (53)
@@ -67,7 +67,7 @@
: : +- ReusedExchange (52)
: +- CometBroadcastExchange (71)
: +- CometHashAggregate (70)
- : +- CometColumnarExchange (69)
+ : +- CometExchange (69)
: +- CometHashAggregate (68)
: +- CometProject (67)
: +- CometBroadcastHashJoin (66)
@@ -79,7 +79,7 @@
: +- ReusedExchange (65)
+- CometBroadcastExchange (85)
+- CometHashAggregate (84)
- +- CometColumnarExchange (83)
+ +- CometExchange (83)
+- CometHashAggregate (82)
+- CometProject (81)
+- CometBroadcastHashJoin (80)
@@ -156,9 +156,9 @@ Input [4]: [ss_ext_sales_price#2, d_year#6, d_qoy#7, ca_county#9]
Keys [3]: [ca_county#9, d_qoy#7, d_year#6]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#2))]
-(14) CometColumnarExchange
+(14) CometExchange
Input [4]: [ca_county#9, d_qoy#7, d_year#6, sum#10]
-Arguments: hashpartitioning(ca_county#9, d_qoy#7, d_year#6, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ca_county#9, d_qoy#7, d_year#6, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(15) CometHashAggregate
Input [4]: [ca_county#9, d_qoy#7, d_year#6, sum#10]
@@ -218,9 +218,9 @@ Input [4]: [ss_ext_sales_price#12, d_year#16, d_qoy#17, ca_county#19]
Keys [3]: [ca_county#19, d_qoy#17, d_year#16]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#12))]
-(27) CometColumnarExchange
+(27) CometExchange
Input [4]: [ca_county#19, d_qoy#17, d_year#16, sum#20]
-Arguments: hashpartitioning(ca_county#19, d_qoy#17, d_year#16, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(ca_county#19, d_qoy#17, d_year#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(28) CometHashAggregate
Input [4]: [ca_county#19, d_qoy#17, d_year#16, sum#20]
@@ -289,9 +289,9 @@ Input [4]: [ss_ext_sales_price#24, d_year#28, d_qoy#29, ca_county#31]
Keys [3]: [ca_county#31, d_qoy#29, d_year#28]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#24))]
-(42) CometColumnarExchange
+(42) CometExchange
Input [4]: [ca_county#31, d_qoy#29, d_year#28, sum#32]
-Arguments: hashpartitioning(ca_county#31, d_qoy#29, d_year#28, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(ca_county#31, d_qoy#29, d_year#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(43) CometHashAggregate
Input [4]: [ca_county#31, d_qoy#29, d_year#28, sum#32]
@@ -352,9 +352,9 @@ Input [4]: [ws_ext_sales_price#35, d_year#39, d_qoy#40, ca_county#42]
Keys [3]: [ca_county#42, d_qoy#40, d_year#39]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#35))]
-(56) CometColumnarExchange
+(56) CometExchange
Input [4]: [ca_county#42, d_qoy#40, d_year#39, sum#43]
-Arguments: hashpartitioning(ca_county#42, d_qoy#40, d_year#39, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(ca_county#42, d_qoy#40, d_year#39, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(57) CometHashAggregate
Input [4]: [ca_county#42, d_qoy#40, d_year#39, sum#43]
@@ -411,9 +411,9 @@ Input [4]: [ws_ext_sales_price#46, d_year#50, d_qoy#51, ca_county#53]
Keys [3]: [ca_county#53, d_qoy#51, d_year#50]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#46))]
-(69) CometColumnarExchange
+(69) CometExchange
Input [4]: [ca_county#53, d_qoy#51, d_year#50, sum#54]
-Arguments: hashpartitioning(ca_county#53, d_qoy#51, d_year#50, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(ca_county#53, d_qoy#51, d_year#50, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(70) CometHashAggregate
Input [4]: [ca_county#53, d_qoy#51, d_year#50, sum#54]
@@ -474,9 +474,9 @@ Input [4]: [ws_ext_sales_price#57, d_year#61, d_qoy#62, ca_county#64]
Keys [3]: [ca_county#64, d_qoy#62, d_year#61]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#57))]
-(83) CometColumnarExchange
+(83) CometExchange
Input [4]: [ca_county#64, d_qoy#62, d_year#61, sum#65]
-Arguments: hashpartitioning(ca_county#64, d_qoy#62, d_year#61, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: hashpartitioning(ca_county#64, d_qoy#62, d_year#61, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
(84) CometHashAggregate
Input [4]: [ca_county#64, d_qoy#62, d_year#61, sum#65]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/simplified.txt
index 6dcf5714..d6615fe1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q31/simplified.txt
@@ -12,7 +12,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [ca_county,d_year,store_sales,ca_county,store_sales,ca_county,store_sales]
CometBroadcastHashJoin [ca_county,d_year,store_sales,ca_county,store_sales]
CometHashAggregate [ca_county,d_year,store_sales,d_qoy,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [ca_county,d_qoy,d_year] #2
+ CometExchange [ca_county,d_qoy,d_year] #2
CometHashAggregate [ca_county,d_qoy,d_year,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,d_year,d_qoy,ca_county]
CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
@@ -35,7 +35,7 @@ WholeStageCodegen (1)
CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county]
CometBroadcastExchange [ca_county,store_sales] #6
CometHashAggregate [ca_county,store_sales,d_qoy,d_year,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [ca_county,d_qoy,d_year] #7
+ CometExchange [ca_county,d_qoy,d_year] #7
CometHashAggregate [ca_county,d_qoy,d_year,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,d_year,d_qoy,ca_county]
CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
@@ -56,7 +56,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk,ca_county] #5
CometBroadcastExchange [ca_county,store_sales] #10
CometHashAggregate [ca_county,store_sales,d_qoy,d_year,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [ca_county,d_qoy,d_year] #11
+ CometExchange [ca_county,d_qoy,d_year] #11
CometHashAggregate [ca_county,d_qoy,d_year,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,d_year,d_qoy,ca_county]
CometBroadcastHashJoin [ss_addr_sk,ss_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
@@ -77,7 +77,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk,ca_county] #5
CometBroadcastExchange [ca_county,web_sales] #14
CometHashAggregate [ca_county,web_sales,d_qoy,d_year,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [ca_county,d_qoy,d_year] #15
+ CometExchange [ca_county,d_qoy,d_year] #15
CometHashAggregate [ca_county,d_qoy,d_year,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,d_year,d_qoy,ca_county]
CometBroadcastHashJoin [ws_bill_addr_sk,ws_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
@@ -90,7 +90,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk,ca_county] #5
CometBroadcastExchange [ca_county,web_sales] #16
CometHashAggregate [ca_county,web_sales,d_qoy,d_year,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [ca_county,d_qoy,d_year] #17
+ CometExchange [ca_county,d_qoy,d_year] #17
CometHashAggregate [ca_county,d_qoy,d_year,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,d_year,d_qoy,ca_county]
CometBroadcastHashJoin [ws_bill_addr_sk,ws_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
@@ -103,7 +103,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk,ca_county] #5
CometBroadcastExchange [ca_county,web_sales] #18
CometHashAggregate [ca_county,web_sales,d_qoy,d_year,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [ca_county,d_qoy,d_year] #19
+ CometExchange [ca_county,d_qoy,d_year] #19
CometHashAggregate [ca_county,d_qoy,d_year,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,d_year,d_qoy,ca_county]
CometBroadcastHashJoin [ws_bill_addr_sk,ws_ext_sales_price,d_year,d_qoy,ca_address_sk,ca_county]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/explain.txt
index 701a75e2..a0b60467 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
* ColumnarToRow (30)
+- CometHashAggregate (29)
- +- CometColumnarExchange (28)
+ +- CometExchange (28)
+- CometHashAggregate (27)
+- CometProject (26)
+- CometBroadcastHashJoin (25)
@@ -18,7 +18,7 @@
: +- CometBroadcastExchange (21)
: +- CometFilter (20)
: +- CometHashAggregate (19)
- : +- CometColumnarExchange (18)
+ : +- CometExchange (18)
: +- CometHashAggregate (17)
: +- CometProject (16)
: +- CometBroadcastHashJoin (15)
@@ -116,9 +116,9 @@ Input [2]: [cs_item_sk#7, cs_ext_discount_amt#8]
Keys [1]: [cs_item_sk#7]
Functions [1]: [partial_avg(UnscaledValue(cs_ext_discount_amt#8))]
-(18) CometColumnarExchange
+(18) CometExchange
Input [3]: [cs_item_sk#7, sum#13, count#14]
-Arguments: hashpartitioning(cs_item_sk#7, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(cs_item_sk#7, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(19) CometHashAggregate
Input [3]: [cs_item_sk#7, sum#13, count#14]
@@ -159,9 +159,9 @@ Input [1]: [cs_ext_discount_amt#2]
Keys: []
Functions [1]: [partial_sum(UnscaledValue(cs_ext_discount_amt#2))]
-(28) CometColumnarExchange
+(28) CometExchange
Input [1]: [sum#17]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(29) CometHashAggregate
Input [1]: [sum#17]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/simplified.txt
index 19be8718..14b2a34a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q32/simplified.txt
@@ -2,7 +2,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [excess discount amount,sum,sum(UnscaledValue(cs_ext_discount_amt))]
- CometColumnarExchange #1
+ CometExchange #1
CometHashAggregate [sum,cs_ext_discount_amt]
CometProject [cs_ext_discount_amt]
CometBroadcastHashJoin [cs_ext_discount_amt,cs_sold_date_sk,d_date_sk]
@@ -27,7 +27,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [(1.3 * avg(cs_ext_discount_amt)),cs_item_sk] #4
CometFilter [(1.3 * avg(cs_ext_discount_amt)),cs_item_sk]
CometHashAggregate [(1.3 * avg(cs_ext_discount_amt)),cs_item_sk,sum,count,avg(UnscaledValue(cs_ext_discount_amt))]
- CometColumnarExchange [cs_item_sk] #5
+ CometExchange [cs_item_sk] #5
CometHashAggregate [cs_item_sk,sum,count,cs_ext_discount_amt]
CometProject [cs_item_sk,cs_ext_discount_amt]
CometBroadcastHashJoin [cs_item_sk,cs_ext_discount_amt,cs_sold_date_sk,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt
index 6fb8b05f..a432ecd2 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt
@@ -2,11 +2,11 @@
* ColumnarToRow (61)
+- CometTakeOrderedAndProject (60)
+- CometHashAggregate (59)
- +- CometColumnarExchange (58)
+ +- CometExchange (58)
+- CometHashAggregate (57)
+- CometUnion (56)
:- CometHashAggregate (27)
- : +- CometColumnarExchange (26)
+ : +- CometExchange (26)
: +- CometHashAggregate (25)
: +- CometProject (24)
: +- CometBroadcastHashJoin (23)
@@ -33,7 +33,7 @@
: +- CometFilter (18)
: +- CometScan parquet spark_catalog.default.item (17)
:- CometHashAggregate (41)
- : +- CometColumnarExchange (40)
+ : +- CometExchange (40)
: +- CometHashAggregate (39)
: +- CometProject (38)
: +- CometBroadcastHashJoin (37)
@@ -47,7 +47,7 @@
: : +- ReusedExchange (33)
: +- ReusedExchange (36)
+- CometHashAggregate (55)
- +- CometColumnarExchange (54)
+ +- CometExchange (54)
+- CometHashAggregate (53)
+- CometProject (52)
+- CometBroadcastHashJoin (51)
@@ -183,9 +183,9 @@ Input [2]: [ss_ext_sales_price#3, i_manufact_id#12]
Keys [1]: [i_manufact_id#12]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#3))]
-(26) CometColumnarExchange
+(26) CometExchange
Input [2]: [i_manufact_id#12, sum#15]
-Arguments: hashpartitioning(i_manufact_id#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_manufact_id#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(27) CometHashAggregate
Input [2]: [i_manufact_id#12, sum#15]
@@ -245,9 +245,9 @@ Input [2]: [cs_ext_sales_price#18, i_manufact_id#24]
Keys [1]: [i_manufact_id#24]
Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#18))]
-(40) CometColumnarExchange
+(40) CometExchange
Input [2]: [i_manufact_id#24, sum#25]
-Arguments: hashpartitioning(i_manufact_id#24, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_manufact_id#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(41) CometHashAggregate
Input [2]: [i_manufact_id#24, sum#25]
@@ -307,9 +307,9 @@ Input [2]: [ws_ext_sales_price#28, i_manufact_id#34]
Keys [1]: [i_manufact_id#34]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#28))]
-(54) CometColumnarExchange
+(54) CometExchange
Input [2]: [i_manufact_id#34, sum#35]
-Arguments: hashpartitioning(i_manufact_id#34, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(i_manufact_id#34, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(55) CometHashAggregate
Input [2]: [i_manufact_id#34, sum#35]
@@ -326,9 +326,9 @@ Input [2]: [i_manufact_id#12, total_sales#36]
Keys [1]: [i_manufact_id#12]
Functions [1]: [partial_sum(total_sales#36)]
-(58) CometColumnarExchange
+(58) CometExchange
Input [3]: [i_manufact_id#12, sum#39, isEmpty#40]
-Arguments: hashpartitioning(i_manufact_id#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(i_manufact_id#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(59) CometHashAggregate
Input [3]: [i_manufact_id#12, sum#39, isEmpty#40]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/simplified.txt
index 964f250b..9556aa78 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/simplified.txt
@@ -3,11 +3,11 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_manufact_id,total_sales]
CometHashAggregate [i_manufact_id,total_sales,sum,isEmpty,sum(total_sales)]
- CometColumnarExchange [i_manufact_id] #1
+ CometExchange [i_manufact_id] #1
CometHashAggregate [i_manufact_id,sum,isEmpty,total_sales]
CometUnion [i_manufact_id,total_sales]
CometHashAggregate [i_manufact_id,total_sales,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_manufact_id] #2
+ CometExchange [i_manufact_id] #2
CometHashAggregate [i_manufact_id,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,i_manufact_id]
CometBroadcastHashJoin [ss_item_sk,ss_ext_sales_price,i_item_sk,i_manufact_id]
@@ -42,7 +42,7 @@ WholeStageCodegen (1)
CometFilter [i_category,i_manufact_id]
CometScan parquet spark_catalog.default.item [i_category,i_manufact_id]
CometHashAggregate [i_manufact_id,total_sales,sum,sum(UnscaledValue(cs_ext_sales_price))]
- CometColumnarExchange [i_manufact_id] #8
+ CometExchange [i_manufact_id] #8
CometHashAggregate [i_manufact_id,sum,cs_ext_sales_price]
CometProject [cs_ext_sales_price,i_manufact_id]
CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,i_item_sk,i_manufact_id]
@@ -57,7 +57,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk] #5
ReusedExchange [i_item_sk,i_manufact_id] #6
CometHashAggregate [i_manufact_id,total_sales,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [i_manufact_id] #9
+ CometExchange [i_manufact_id] #9
CometHashAggregate [i_manufact_id,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,i_manufact_id]
CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,i_item_sk,i_manufact_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt
index c33735fa..dccad575 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt
@@ -6,7 +6,7 @@
+- CometBroadcastHashJoin (28)
:- CometFilter (24)
: +- CometHashAggregate (23)
- : +- CometColumnarExchange (22)
+ : +- CometExchange (22)
: +- CometHashAggregate (21)
: +- CometProject (20)
: +- CometBroadcastHashJoin (19)
@@ -134,9 +134,9 @@ Input [2]: [ss_customer_sk#1, ss_ticket_number#4]
Keys [2]: [ss_ticket_number#4, ss_customer_sk#1]
Functions [1]: [partial_count(1)]
-(22) CometColumnarExchange
+(22) CometExchange
Input [3]: [ss_ticket_number#4, ss_customer_sk#1, count#16]
-Arguments: hashpartitioning(ss_ticket_number#4, ss_customer_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_ticket_number#4, ss_customer_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(23) CometHashAggregate
Input [3]: [ss_ticket_number#4, ss_customer_sk#1, count#16]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt
index c131bd9c..779d4cbd 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt
@@ -7,7 +7,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [ss_ticket_number,ss_customer_sk,cnt,c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag]
CometFilter [ss_ticket_number,ss_customer_sk,cnt]
CometHashAggregate [ss_ticket_number,ss_customer_sk,cnt,count,count(1)]
- CometColumnarExchange [ss_ticket_number,ss_customer_sk] #2
+ CometExchange [ss_ticket_number,ss_customer_sk] #2
CometHashAggregate [ss_ticket_number,ss_customer_sk,count]
CometProject [ss_customer_sk,ss_ticket_number]
CometBroadcastHashJoin [ss_customer_sk,ss_hdemo_sk,ss_ticket_number,hd_demo_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt
index b35338d1..c06c1dd1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt
@@ -1,50 +1,48 @@
== Physical Plan ==
-TakeOrderedAndProject (46)
-+- * HashAggregate (45)
- +- * ColumnarToRow (44)
- +- CometColumnarExchange (43)
- +- RowToColumnar (42)
- +- * HashAggregate (41)
- +- * Project (40)
- +- * BroadcastHashJoin Inner BuildRight (39)
- :- * Project (34)
- : +- * BroadcastHashJoin Inner BuildRight (33)
- : :- * Project (28)
- : : +- * Filter (27)
- : : +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (26)
- : : :- * BroadcastHashJoin ExistenceJoin(exists#2) BuildRight (19)
- : : : :- * ColumnarToRow (12)
- : : : : +- CometBroadcastHashJoin (11)
- : : : : :- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.customer (1)
- : : : : +- CometBroadcastExchange (10)
- : : : : +- CometProject (9)
- : : : : +- CometBroadcastHashJoin (8)
- : : : : :- CometScan parquet spark_catalog.default.store_sales (3)
- : : : : +- CometBroadcastExchange (7)
- : : : : +- CometProject (6)
- : : : : +- CometFilter (5)
- : : : : +- CometScan parquet spark_catalog.default.date_dim (4)
- : : : +- BroadcastExchange (18)
- : : : +- * ColumnarToRow (17)
- : : : +- CometProject (16)
- : : : +- CometBroadcastHashJoin (15)
- : : : :- CometScan parquet spark_catalog.default.web_sales (13)
- : : : +- ReusedExchange (14)
- : : +- BroadcastExchange (25)
- : : +- * ColumnarToRow (24)
- : : +- CometProject (23)
- : : +- CometBroadcastHashJoin (22)
- : : :- CometScan parquet spark_catalog.default.catalog_sales (20)
- : : +- ReusedExchange (21)
- : +- BroadcastExchange (32)
- : +- * ColumnarToRow (31)
- : +- CometFilter (30)
- : +- CometScan parquet spark_catalog.default.customer_address (29)
- +- BroadcastExchange (38)
- +- * ColumnarToRow (37)
- +- CometFilter (36)
- +- CometScan parquet spark_catalog.default.customer_demographics (35)
+TakeOrderedAndProject (44)
++- * HashAggregate (43)
+ +- Exchange (42)
+ +- * HashAggregate (41)
+ +- * Project (40)
+ +- * BroadcastHashJoin Inner BuildRight (39)
+ :- * Project (34)
+ : +- * BroadcastHashJoin Inner BuildRight (33)
+ : :- * Project (28)
+ : : +- * Filter (27)
+ : : +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (26)
+ : : :- * BroadcastHashJoin ExistenceJoin(exists#2) BuildRight (19)
+ : : : :- * ColumnarToRow (12)
+ : : : : +- CometBroadcastHashJoin (11)
+ : : : : :- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.customer (1)
+ : : : : +- CometBroadcastExchange (10)
+ : : : : +- CometProject (9)
+ : : : : +- CometBroadcastHashJoin (8)
+ : : : : :- CometScan parquet spark_catalog.default.store_sales (3)
+ : : : : +- CometBroadcastExchange (7)
+ : : : : +- CometProject (6)
+ : : : : +- CometFilter (5)
+ : : : : +- CometScan parquet spark_catalog.default.date_dim (4)
+ : : : +- BroadcastExchange (18)
+ : : : +- * ColumnarToRow (17)
+ : : : +- CometProject (16)
+ : : : +- CometBroadcastHashJoin (15)
+ : : : :- CometScan parquet spark_catalog.default.web_sales (13)
+ : : : +- ReusedExchange (14)
+ : : +- BroadcastExchange (25)
+ : : +- * ColumnarToRow (24)
+ : : +- CometProject (23)
+ : : +- CometBroadcastHashJoin (22)
+ : : :- CometScan parquet spark_catalog.default.catalog_sales (20)
+ : : +- ReusedExchange (21)
+ : +- BroadcastExchange (32)
+ : +- * ColumnarToRow (31)
+ : +- CometFilter (30)
+ : +- CometScan parquet spark_catalog.default.customer_address (29)
+ +- BroadcastExchange (38)
+ +- * ColumnarToRow (37)
+ +- CometFilter (36)
+ +- CometScan parquet spark_catalog.default.customer_demographics (35)
(1) Scan parquet spark_catalog.default.customer
@@ -240,56 +238,50 @@ Functions [10]: [partial_count(1), partial_min(cd_dep_count#25), partial_max(cd_
Aggregate Attributes [13]: [count#28, min#29, max#30, sum#31, count#32, min#33, max#34, sum#35, count#36, min#37, max#38, sum#39, count#40]
Results [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
-(42) RowToColumnar
+(42) Exchange
Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
+Arguments: hashpartitioning(ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, 5), ENSURE_REQUIREMENTS, [plan_id=5]
-(43) CometColumnarExchange
-Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
-Arguments: hashpartitioning(ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(44) ColumnarToRow [codegen id : 6]
-Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
-
-(45) HashAggregate [codegen id : 6]
+(43) HashAggregate [codegen id : 6]
Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
Keys [6]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
Functions [10]: [count(1), min(cd_dep_count#25), max(cd_dep_count#25), avg(cd_dep_count#25), min(cd_dep_employed_count#26), max(cd_dep_employed_count#26), avg(cd_dep_employed_count#26), min(cd_dep_college_count#27), max(cd_dep_college_count#27), avg(cd_dep_college_count#27)]
Aggregate Attributes [10]: [count(1)#54, min(cd_dep_count#25)#55, max(cd_dep_count#25)#56, avg(cd_dep_count#25)#57, min(cd_dep_employed_count#26)#58, max(cd_dep_employed_count#26)#59, avg(cd_dep_employed_count#26)#60, min(cd_dep_college_count#27)#61, max(cd_dep_college_count#27)#62, avg(cd_dep_college_count#27)#63]
Results [18]: [ca_state#21, cd_gender#23, cd_marital_status#24, count(1)#54 AS cnt1#64, min(cd_dep_count#25)#55 AS min(cd_dep_count)#65, max(cd_dep_count#25)#56 AS max(cd_dep_count)#66, avg(cd_dep_count#25)#57 AS avg(cd_dep_count)#67, cd_dep_employed_count#26, count(1)#54 AS cnt2#68, min(cd_dep_employed_count#26)#58 AS min(cd_dep_employed_count)#69, max(cd_dep_employed_count#26)#59 AS max(cd_dep_employed_count)#70, avg(cd_dep_employed_count#26)#60 AS avg(cd_dep_employed_count)#71, cd_dep [...]
-(46) TakeOrderedAndProject
+(44) TakeOrderedAndProject
Input [18]: [ca_state#21, cd_gender#23, cd_marital_status#24, cnt1#64, min(cd_dep_count)#65, max(cd_dep_count)#66, avg(cd_dep_count)#67, cd_dep_employed_count#26, cnt2#68, min(cd_dep_employed_count)#69, max(cd_dep_employed_count)#70, avg(cd_dep_employed_count)#71, cd_dep_college_count#27, cnt3#72, min(cd_dep_college_count)#73, max(cd_dep_college_count)#74, avg(cd_dep_college_count)#75, cd_dep_count#25]
Arguments: 100, [ca_state#21 ASC NULLS FIRST, cd_gender#23 ASC NULLS FIRST, cd_marital_status#24 ASC NULLS FIRST, cd_dep_count#25 ASC NULLS FIRST, cd_dep_employed_count#26 ASC NULLS FIRST, cd_dep_college_count#27 ASC NULLS FIRST], [ca_state#21, cd_gender#23, cd_marital_status#24, cnt1#64, min(cd_dep_count)#65, max(cd_dep_count)#66, avg(cd_dep_count)#67, cd_dep_employed_count#26, cnt2#68, min(cd_dep_employed_count)#69, max(cd_dep_employed_count)#70, avg(cd_dep_employed_count)#71, cd_dep_c [...]
===== Subqueries =====
Subquery:1 Hosting operator id = 3 Hosting Expression = ss_sold_date_sk#7 IN dynamicpruning#8
-BroadcastExchange (51)
-+- * ColumnarToRow (50)
- +- CometProject (49)
- +- CometFilter (48)
- +- CometScan parquet spark_catalog.default.date_dim (47)
+BroadcastExchange (49)
++- * ColumnarToRow (48)
+ +- CometProject (47)
+ +- CometFilter (46)
+ +- CometScan parquet spark_catalog.default.date_dim (45)
-(47) Scan parquet spark_catalog.default.date_dim
+(45) Scan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#9, d_year#10, d_qoy#11]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_qoy), EqualTo(d_year,2002), LessThan(d_qoy,4), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(48) CometFilter
+(46) CometFilter
Input [3]: [d_date_sk#9, d_year#10, d_qoy#11]
Condition : ((((isnotnull(d_year#10) AND isnotnull(d_qoy#11)) AND (d_year#10 = 2002)) AND (d_qoy#11 < 4)) AND isnotnull(d_date_sk#9))
-(49) CometProject
+(47) CometProject
Input [3]: [d_date_sk#9, d_year#10, d_qoy#11]
Arguments: [d_date_sk#9], [d_date_sk#9]
-(50) ColumnarToRow [codegen id : 1]
+(48) ColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#9]
-(51) BroadcastExchange
+(49) BroadcastExchange
Input [1]: [d_date_sk#9]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt
index f9ef660d..9deb5134 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt
@@ -1,72 +1,70 @@
TakeOrderedAndProject [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count,cnt1,min(cd_dep_count),max(cd_dep_count),avg(cd_dep_count),cnt2,min(cd_dep_employed_count),max(cd_dep_employed_count),avg(cd_dep_employed_count),cnt3,min(cd_dep_college_count),max(cd_dep_college_count),avg(cd_dep_college_count)]
WholeStageCodegen (6)
HashAggregate [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count,count,min,max,sum,count,min,max,sum,count,min,max,sum,count] [count(1),min(cd_dep_count),max(cd_dep_count),avg(cd_dep_count),min(cd_dep_employed_count),max(cd_dep_employed_count),avg(cd_dep_employed_count),min(cd_dep_college_count),max(cd_dep_college_count),avg(cd_dep_college_count),cnt1,min(cd_dep_count),max(cd_dep_count),avg(cd_dep_count),cnt2,min(cd_dep_employed_count),max(c [...]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] #1
- RowToColumnar
- WholeStageCodegen (5)
- HashAggregate [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] [count,min,max,sum,count,min,max,sum,count,min,max,sum,count,count,min,max,sum,count,min,max,sum,count,min,max,sum,count]
- Project [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
- BroadcastHashJoin [c_current_cdemo_sk,cd_demo_sk]
- Project [c_current_cdemo_sk,ca_state]
- BroadcastHashJoin [c_current_addr_sk,ca_address_sk]
- Project [c_current_cdemo_sk,c_current_addr_sk]
- Filter [exists,exists]
- BroadcastHashJoin [c_customer_sk,cs_ship_customer_sk]
- BroadcastHashJoin [c_customer_sk,ws_bill_customer_sk]
+ InputAdapter
+ Exchange [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] #1
+ WholeStageCodegen (5)
+ HashAggregate [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count] [count,min,max,sum,count,min,max,sum,count,min,max,sum,count,count,min,max,sum,count,min,max,sum,count,min,max,sum,count]
+ Project [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ BroadcastHashJoin [c_current_cdemo_sk,cd_demo_sk]
+ Project [c_current_cdemo_sk,ca_state]
+ BroadcastHashJoin [c_current_addr_sk,ca_address_sk]
+ Project [c_current_cdemo_sk,c_current_addr_sk]
+ Filter [exists,exists]
+ BroadcastHashJoin [c_customer_sk,cs_ship_customer_sk]
+ BroadcastHashJoin [c_customer_sk,ws_bill_customer_sk]
+ ColumnarToRow
+ InputAdapter
+ CometBroadcastHashJoin [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk,ss_customer_sk]
+ CometFilter [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
+ CometBroadcastExchange [ss_customer_sk] #2
+ CometProject [ss_customer_sk]
+ CometBroadcastHashJoin [ss_customer_sk,ss_sold_date_sk,d_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_qoy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
+ CometBroadcastExchange [d_date_sk] #4
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_qoy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
+ InputAdapter
+ BroadcastExchange #5
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [ws_bill_customer_sk]
+ CometBroadcastHashJoin [ws_bill_customer_sk,ws_sold_date_sk,d_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #4
+ InputAdapter
+ BroadcastExchange #6
+ WholeStageCodegen (2)
ColumnarToRow
InputAdapter
- CometBroadcastHashJoin [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk,ss_customer_sk]
- CometFilter [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_cdemo_sk,c_current_addr_sk]
- CometBroadcastExchange [ss_customer_sk] #2
- CometProject [ss_customer_sk]
- CometBroadcastHashJoin [ss_customer_sk,ss_sold_date_sk,d_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- CometBroadcastExchange [d_date_sk] #4
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- InputAdapter
- BroadcastExchange #5
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [ws_bill_customer_sk]
- CometBroadcastHashJoin [ws_bill_customer_sk,ws_sold_date_sk,d_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #4
- InputAdapter
- BroadcastExchange #6
- WholeStageCodegen (2)
- ColumnarToRow
- InputAdapter
- CometProject [cs_ship_customer_sk]
- CometBroadcastHashJoin [cs_ship_customer_sk,cs_sold_date_sk,d_date_sk]
- CometScan parquet spark_catalog.default.catalog_sales [cs_ship_customer_sk,cs_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #4
- InputAdapter
- BroadcastExchange #7
- WholeStageCodegen (3)
- ColumnarToRow
- InputAdapter
- CometFilter [ca_address_sk,ca_state]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ CometProject [cs_ship_customer_sk]
+ CometBroadcastHashJoin [cs_ship_customer_sk,cs_sold_date_sk,d_date_sk]
+ CometScan parquet spark_catalog.default.catalog_sales [cs_ship_customer_sk,cs_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #4
InputAdapter
- BroadcastExchange #8
- WholeStageCodegen (4)
+ BroadcastExchange #7
+ WholeStageCodegen (3)
ColumnarToRow
InputAdapter
- CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
- CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometFilter [ca_address_sk,ca_state]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ InputAdapter
+ BroadcastExchange #8
+ WholeStageCodegen (4)
+ ColumnarToRow
+ InputAdapter
+ CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt
index 9f22e37d..68397f3a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt
@@ -4,9 +4,9 @@ TakeOrderedAndProject (29)
+- Window (27)
+- * ColumnarToRow (26)
+- CometSort (25)
- +- CometColumnarExchange (24)
+ +- CometExchange (24)
+- CometHashAggregate (23)
- +- CometColumnarExchange (22)
+ +- CometExchange (22)
+- CometHashAggregate (21)
+- CometExpand (20)
+- CometProject (19)
@@ -131,18 +131,18 @@ Input [5]: [ss_ext_sales_price#3, ss_net_profit#4, i_category#14, i_class#15, sp
Keys [3]: [i_category#14, i_class#15, spark_grouping_id#16]
Functions [2]: [partial_sum(UnscaledValue(ss_net_profit#4)), partial_sum(UnscaledValue(ss_ext_sales_price#3))]
-(22) CometColumnarExchange
+(22) CometExchange
Input [5]: [i_category#14, i_class#15, spark_grouping_id#16, sum#17, sum#18]
-Arguments: hashpartitioning(i_category#14, i_class#15, spark_grouping_id#16, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_category#14, i_class#15, spark_grouping_id#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(23) CometHashAggregate
Input [5]: [i_category#14, i_class#15, spark_grouping_id#16, sum#17, sum#18]
Keys [3]: [i_category#14, i_class#15, spark_grouping_id#16]
Functions [2]: [sum(UnscaledValue(ss_net_profit#4)), sum(UnscaledValue(ss_ext_sales_price#3))]
-(24) CometColumnarExchange
+(24) CometExchange
Input [7]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23]
-Arguments: hashpartitioning(_w1#22, _w2#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(_w1#22, _w2#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(25) CometSort
Input [7]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt
index d7beb442..421db8da 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt
@@ -7,9 +7,9 @@ TakeOrderedAndProject [lochierarchy,i_category,rank_within_parent,gross_margin,i
ColumnarToRow
InputAdapter
CometSort [gross_margin,i_category,i_class,lochierarchy,_w0,_w1,_w2]
- CometColumnarExchange [_w1,_w2] #1
+ CometExchange [_w1,_w2] #1
CometHashAggregate [gross_margin,i_category,i_class,lochierarchy,_w0,_w1,_w2,spark_grouping_id,sum,sum,sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_category,i_class,spark_grouping_id] #2
+ CometExchange [i_category,i_class,spark_grouping_id] #2
CometHashAggregate [i_category,i_class,spark_grouping_id,sum,sum,ss_net_profit,ss_ext_sales_price]
CometExpand [i_category,i_class] [ss_ext_sales_price,ss_net_profit,i_category,i_class,spark_grouping_id]
CometProject [ss_ext_sales_price,ss_net_profit,i_category,i_class]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt
index 42dab0b3..bd5ca5ef 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (26)
+- CometTakeOrderedAndProject (25)
+- CometHashAggregate (24)
- +- CometColumnarExchange (23)
+ +- CometExchange (23)
+- CometHashAggregate (22)
+- CometProject (21)
+- CometBroadcastHashJoin (20)
@@ -132,9 +132,9 @@ Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
Keys [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
Functions: []
-(23) CometColumnarExchange
+(23) CometExchange
Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
-Arguments: hashpartitioning(i_item_id#2, i_item_desc#3, i_current_price#4, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#2, i_item_desc#3, i_current_price#4, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(24) CometHashAggregate
Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt
index 39edfb51..85e86ca8 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,i_item_desc,i_current_price]
CometHashAggregate [i_item_id,i_item_desc,i_current_price]
- CometColumnarExchange [i_item_id,i_item_desc,i_current_price] #1
+ CometExchange [i_item_id,i_item_desc,i_current_price] #1
CometHashAggregate [i_item_id,i_item_desc,i_current_price]
CometProject [i_item_id,i_item_desc,i_current_price]
CometBroadcastHashJoin [i_item_sk,i_item_id,i_item_desc,i_current_price,cs_item_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt
index 091883bd..598dccaf 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
* ColumnarToRow (47)
+- CometHashAggregate (46)
- +- CometColumnarExchange (45)
+ +- CometExchange (45)
+- CometHashAggregate (44)
+- CometProject (43)
+- CometBroadcastHashJoin (42)
:- CometBroadcastHashJoin (29)
: :- CometHashAggregate (16)
- : : +- CometColumnarExchange (15)
+ : : +- CometExchange (15)
: : +- CometHashAggregate (14)
: : +- CometProject (13)
: : +- CometBroadcastHashJoin (12)
@@ -24,7 +24,7 @@
: : +- CometScan parquet spark_catalog.default.customer (9)
: +- CometBroadcastExchange (28)
: +- CometHashAggregate (27)
- : +- CometColumnarExchange (26)
+ : +- CometExchange (26)
: +- CometHashAggregate (25)
: +- CometProject (24)
: +- CometBroadcastHashJoin (23)
@@ -36,7 +36,7 @@
: +- ReusedExchange (22)
+- CometBroadcastExchange (41)
+- CometHashAggregate (40)
- +- CometColumnarExchange (39)
+ +- CometExchange (39)
+- CometHashAggregate (38)
+- CometProject (37)
+- CometBroadcastHashJoin (36)
@@ -117,9 +117,9 @@ Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
Keys [3]: [c_last_name#9, c_first_name#8, d_date#5]
Functions: []
-(15) CometColumnarExchange
+(15) CometExchange
Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
-Arguments: hashpartitioning(c_last_name#9, c_first_name#8, d_date#5, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(c_last_name#9, c_first_name#8, d_date#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(16) CometHashAggregate
Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
@@ -167,9 +167,9 @@ Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
Keys [3]: [c_last_name#17, c_first_name#16, d_date#14]
Functions: []
-(26) CometColumnarExchange
+(26) CometExchange
Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
-Arguments: hashpartitioning(c_last_name#17, c_first_name#16, d_date#14, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(c_last_name#17, c_first_name#16, d_date#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(27) CometHashAggregate
Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
@@ -226,9 +226,9 @@ Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
Keys [3]: [c_last_name#25, c_first_name#24, d_date#22]
Functions: []
-(39) CometColumnarExchange
+(39) CometExchange
Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
-Arguments: hashpartitioning(c_last_name#25, c_first_name#24, d_date#22, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(c_last_name#25, c_first_name#24, d_date#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(40) CometHashAggregate
Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
@@ -252,9 +252,9 @@ Input: []
Keys: []
Functions [1]: [partial_count(1)]
-(45) CometColumnarExchange
+(45) CometExchange
Input [1]: [count#26]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(46) CometHashAggregate
Input [1]: [count#26]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt
index 77970e8d..8272260f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt
@@ -2,13 +2,13 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [count(1),count,count(1)]
- CometColumnarExchange #1
+ CometExchange #1
CometHashAggregate [count]
CometProject
CometBroadcastHashJoin [c_last_name,c_first_name,d_date,c_last_name,c_first_name,d_date]
CometBroadcastHashJoin [c_last_name,c_first_name,d_date,c_last_name,c_first_name,d_date]
CometHashAggregate [c_last_name,c_first_name,d_date]
- CometColumnarExchange [c_last_name,c_first_name,d_date] #2
+ CometExchange [c_last_name,c_first_name,d_date] #2
CometHashAggregate [c_last_name,c_first_name,d_date]
CometProject [c_last_name,c_first_name,d_date]
CometBroadcastHashJoin [ss_customer_sk,d_date,c_customer_sk,c_first_name,c_last_name]
@@ -33,7 +33,7 @@ WholeStageCodegen (1)
CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
CometBroadcastExchange [c_last_name,c_first_name,d_date] #6
CometHashAggregate [c_last_name,c_first_name,d_date]
- CometColumnarExchange [c_last_name,c_first_name,d_date] #7
+ CometExchange [c_last_name,c_first_name,d_date] #7
CometHashAggregate [c_last_name,c_first_name,d_date]
CometProject [c_last_name,c_first_name,d_date]
CometBroadcastHashJoin [cs_bill_customer_sk,d_date,c_customer_sk,c_first_name,c_last_name]
@@ -46,7 +46,7 @@ WholeStageCodegen (1)
ReusedExchange [c_customer_sk,c_first_name,c_last_name] #5
CometBroadcastExchange [c_last_name,c_first_name,d_date] #8
CometHashAggregate [c_last_name,c_first_name,d_date]
- CometColumnarExchange [c_last_name,c_first_name,d_date] #9
+ CometExchange [c_last_name,c_first_name,d_date] #9
CometHashAggregate [c_last_name,c_first_name,d_date]
CometProject [c_last_name,c_first_name,d_date]
CometBroadcastHashJoin [ws_bill_customer_sk,d_date,c_customer_sk,c_first_name,c_last_name]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/explain.txt
index 60c02c54..58e0e664 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/explain.txt
@@ -6,7 +6,7 @@
:- CometProject (23)
: +- CometFilter (22)
: +- CometHashAggregate (21)
- : +- CometColumnarExchange (20)
+ : +- CometExchange (20)
: +- CometHashAggregate (19)
: +- CometProject (18)
: +- CometBroadcastHashJoin (17)
@@ -30,7 +30,7 @@
+- CometProject (42)
+- CometFilter (41)
+- CometHashAggregate (40)
- +- CometColumnarExchange (39)
+ +- CometExchange (39)
+- CometHashAggregate (38)
+- CometProject (37)
+- CometBroadcastHashJoin (36)
@@ -141,9 +141,9 @@ Input [5]: [inv_quantity_on_hand#3, i_item_sk#6, w_warehouse_sk#7, w_warehouse_n
Keys [4]: [w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11]
Functions [2]: [partial_stddev_samp(cast(inv_quantity_on_hand#3 as double)), partial_avg(inv_quantity_on_hand#3)]
-(20) CometColumnarExchange
+(20) CometExchange
Input [9]: [w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, n#12, avg#13, m2#14, sum#15, count#16]
-Arguments: hashpartitioning(w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(21) CometHashAggregate
Input [9]: [w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, n#12, avg#13, m2#14, sum#15, count#16]
@@ -227,9 +227,9 @@ Input [5]: [inv_quantity_on_hand#22, i_item_sk#25, w_warehouse_sk#26, w_warehous
Keys [4]: [w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30]
Functions [2]: [partial_stddev_samp(cast(inv_quantity_on_hand#22 as double)), partial_avg(inv_quantity_on_hand#22)]
-(39) CometColumnarExchange
+(39) CometExchange
Input [9]: [w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, n#31, avg#32, m2#33, sum#34, count#35]
-Arguments: hashpartitioning(w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(40) CometHashAggregate
Input [9]: [w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, n#31, avg#32, m2#33, sum#34, count#35]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/simplified.txt
index a4d1753f..b789f3a8 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39a/simplified.txt
@@ -7,7 +7,7 @@ WholeStageCodegen (1)
CometProject [stdev] [w_warehouse_sk,i_item_sk,d_moy,mean,cov]
CometFilter [w_warehouse_sk,i_item_sk,d_moy,stdev,mean]
CometHashAggregate [w_warehouse_sk,i_item_sk,d_moy,stdev,mean,w_warehouse_name,n,avg,m2,sum,count,stddev_samp(cast(inv_quantity_on_hand as double)),avg(inv_quantity_on_hand)]
- CometColumnarExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #2
+ CometExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #2
CometHashAggregate [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy,n,avg,m2,sum,count,inv_quantity_on_hand]
CometProject [inv_quantity_on_hand,i_item_sk,w_warehouse_sk,w_warehouse_name,d_moy]
CometBroadcastHashJoin [inv_quantity_on_hand,inv_date_sk,i_item_sk,w_warehouse_sk,w_warehouse_name,d_date_sk,d_moy]
@@ -39,7 +39,7 @@ WholeStageCodegen (1)
CometProject [mean,stdev] [w_warehouse_sk,i_item_sk,d_moy,mean,cov]
CometFilter [w_warehouse_sk,i_item_sk,d_moy,stdev,mean]
CometHashAggregate [w_warehouse_sk,i_item_sk,d_moy,stdev,mean,w_warehouse_name,n,avg,m2,sum,count,stddev_samp(cast(inv_quantity_on_hand as double)),avg(inv_quantity_on_hand)]
- CometColumnarExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #8
+ CometExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #8
CometHashAggregate [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy,n,avg,m2,sum,count,inv_quantity_on_hand]
CometProject [inv_quantity_on_hand,i_item_sk,w_warehouse_sk,w_warehouse_name,d_moy]
CometBroadcastHashJoin [inv_quantity_on_hand,inv_date_sk,i_item_sk,w_warehouse_sk,w_warehouse_name,d_date_sk,d_moy]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/explain.txt
index d95f2c4c..c246e93a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/explain.txt
@@ -6,7 +6,7 @@
:- CometProject (23)
: +- CometFilter (22)
: +- CometHashAggregate (21)
- : +- CometColumnarExchange (20)
+ : +- CometExchange (20)
: +- CometHashAggregate (19)
: +- CometProject (18)
: +- CometBroadcastHashJoin (17)
@@ -30,7 +30,7 @@
+- CometProject (42)
+- CometFilter (41)
+- CometHashAggregate (40)
- +- CometColumnarExchange (39)
+ +- CometExchange (39)
+- CometHashAggregate (38)
+- CometProject (37)
+- CometBroadcastHashJoin (36)
@@ -141,9 +141,9 @@ Input [5]: [inv_quantity_on_hand#3, i_item_sk#6, w_warehouse_sk#7, w_warehouse_n
Keys [4]: [w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11]
Functions [2]: [partial_stddev_samp(cast(inv_quantity_on_hand#3 as double)), partial_avg(inv_quantity_on_hand#3)]
-(20) CometColumnarExchange
+(20) CometExchange
Input [9]: [w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, n#12, avg#13, m2#14, sum#15, count#16]
-Arguments: hashpartitioning(w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(21) CometHashAggregate
Input [9]: [w_warehouse_name#8, w_warehouse_sk#7, i_item_sk#6, d_moy#11, n#12, avg#13, m2#14, sum#15, count#16]
@@ -227,9 +227,9 @@ Input [5]: [inv_quantity_on_hand#22, i_item_sk#25, w_warehouse_sk#26, w_warehous
Keys [4]: [w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30]
Functions [2]: [partial_stddev_samp(cast(inv_quantity_on_hand#22 as double)), partial_avg(inv_quantity_on_hand#22)]
-(39) CometColumnarExchange
+(39) CometExchange
Input [9]: [w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, n#31, avg#32, m2#33, sum#34, count#35]
-Arguments: hashpartitioning(w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(40) CometHashAggregate
Input [9]: [w_warehouse_name#27, w_warehouse_sk#26, i_item_sk#25, d_moy#30, n#31, avg#32, m2#33, sum#34, count#35]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/simplified.txt
index a4d1753f..b789f3a8 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q39b/simplified.txt
@@ -7,7 +7,7 @@ WholeStageCodegen (1)
CometProject [stdev] [w_warehouse_sk,i_item_sk,d_moy,mean,cov]
CometFilter [w_warehouse_sk,i_item_sk,d_moy,stdev,mean]
CometHashAggregate [w_warehouse_sk,i_item_sk,d_moy,stdev,mean,w_warehouse_name,n,avg,m2,sum,count,stddev_samp(cast(inv_quantity_on_hand as double)),avg(inv_quantity_on_hand)]
- CometColumnarExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #2
+ CometExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #2
CometHashAggregate [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy,n,avg,m2,sum,count,inv_quantity_on_hand]
CometProject [inv_quantity_on_hand,i_item_sk,w_warehouse_sk,w_warehouse_name,d_moy]
CometBroadcastHashJoin [inv_quantity_on_hand,inv_date_sk,i_item_sk,w_warehouse_sk,w_warehouse_name,d_date_sk,d_moy]
@@ -39,7 +39,7 @@ WholeStageCodegen (1)
CometProject [mean,stdev] [w_warehouse_sk,i_item_sk,d_moy,mean,cov]
CometFilter [w_warehouse_sk,i_item_sk,d_moy,stdev,mean]
CometHashAggregate [w_warehouse_sk,i_item_sk,d_moy,stdev,mean,w_warehouse_name,n,avg,m2,sum,count,stddev_samp(cast(inv_quantity_on_hand as double)),avg(inv_quantity_on_hand)]
- CometColumnarExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #8
+ CometExchange [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy] #8
CometHashAggregate [w_warehouse_name,w_warehouse_sk,i_item_sk,d_moy,n,avg,m2,sum,count,inv_quantity_on_hand]
CometProject [inv_quantity_on_hand,i_item_sk,w_warehouse_sk,w_warehouse_name,d_moy]
CometBroadcastHashJoin [inv_quantity_on_hand,inv_date_sk,i_item_sk,w_warehouse_sk,w_warehouse_name,d_date_sk,d_moy]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt
index 960cb130..b6387767 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt
@@ -12,7 +12,7 @@
: : : :- CometBroadcastHashJoin (33)
: : : : :- CometFilter (16)
: : : : : +- CometHashAggregate (15)
- : : : : : +- CometColumnarExchange (14)
+ : : : : : +- CometExchange (14)
: : : : : +- CometHashAggregate (13)
: : : : : +- CometProject (12)
: : : : : +- CometBroadcastHashJoin (11)
@@ -28,7 +28,7 @@
: : : : : +- CometScan parquet spark_catalog.default.date_dim (8)
: : : : +- CometBroadcastExchange (32)
: : : : +- CometHashAggregate (31)
- : : : : +- CometColumnarExchange (30)
+ : : : : +- CometExchange (30)
: : : : +- CometHashAggregate (29)
: : : : +- CometProject (28)
: : : : +- CometBroadcastHashJoin (27)
@@ -45,7 +45,7 @@
: : : +- CometBroadcastExchange (48)
: : : +- CometFilter (47)
: : : +- CometHashAggregate (46)
- : : : +- CometColumnarExchange (45)
+ : : : +- CometExchange (45)
: : : +- CometHashAggregate (44)
: : : +- CometProject (43)
: : : +- CometBroadcastHashJoin (42)
@@ -59,7 +59,7 @@
: : : +- ReusedExchange (41)
: : +- CometBroadcastExchange (64)
: : +- CometHashAggregate (63)
- : : +- CometColumnarExchange (62)
+ : : +- CometExchange (62)
: : +- CometHashAggregate (61)
: : +- CometProject (60)
: : +- CometBroadcastHashJoin (59)
@@ -74,7 +74,7 @@
: +- CometBroadcastExchange (81)
: +- CometFilter (80)
: +- CometHashAggregate (79)
- : +- CometColumnarExchange (78)
+ : +- CometExchange (78)
: +- CometHashAggregate (77)
: +- CometProject (76)
: +- CometBroadcastHashJoin (75)
@@ -88,7 +88,7 @@
: +- ReusedExchange (74)
+- CometBroadcastExchange (97)
+- CometHashAggregate (96)
- +- CometColumnarExchange (95)
+ +- CometExchange (95)
+- CometHashAggregate (94)
+- CometProject (93)
+- CometBroadcastHashJoin (92)
@@ -167,9 +167,9 @@ Input [12]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_fl
Keys [8]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17]
Functions [1]: [partial_sum(((((ss_ext_list_price#13 - ss_ext_wholesale_cost#12) - ss_ext_discount_amt#10) + ss_ext_sales_price#11) / 2))]
-(14) CometColumnarExchange
+(14) CometExchange
Input [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, sum#18, isEmpty#19]
-Arguments: hashpartitioning(c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(15) CometHashAggregate
Input [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, sum#18, isEmpty#19]
@@ -245,9 +245,9 @@ Input [12]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust
Keys [8]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38]
Functions [1]: [partial_sum(((((ss_ext_list_price#34 - ss_ext_wholesale_cost#33) - ss_ext_discount_amt#31) + ss_ext_sales_price#32) / 2))]
-(30) CometColumnarExchange
+(30) CometExchange
Input [10]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, sum#39, isEmpty#40]
-Arguments: hashpartitioning(c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(31) CometHashAggregate
Input [10]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, sum#39, isEmpty#40]
@@ -316,9 +316,9 @@ Input [12]: [c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust
Keys [8]: [c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust_flag#53, c_birth_country#54, c_login#55, c_email_address#56, d_year#65]
Functions [1]: [partial_sum(((((cs_ext_list_price#61 - cs_ext_wholesale_cost#60) - cs_ext_discount_amt#58) + cs_ext_sales_price#59) / 2))]
-(45) CometColumnarExchange
+(45) CometExchange
Input [10]: [c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust_flag#53, c_birth_country#54, c_login#55, c_email_address#56, d_year#65, sum#66, isEmpty#67]
-Arguments: hashpartitioning(c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust_flag#53, c_birth_country#54, c_login#55, c_email_address#56, d_year#65, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust_flag#53, c_birth_country#54, c_login#55, c_email_address#56, d_year#65, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(46) CometHashAggregate
Input [10]: [c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust_flag#53, c_birth_country#54, c_login#55, c_email_address#56, d_year#65, sum#66, isEmpty#67]
@@ -395,9 +395,9 @@ Input [12]: [c_customer_id#71, c_first_name#72, c_last_name#73, c_preferred_cust
Keys [8]: [c_customer_id#71, c_first_name#72, c_last_name#73, c_preferred_cust_flag#74, c_birth_country#75, c_login#76, c_email_address#77, d_year#86]
Functions [1]: [partial_sum(((((cs_ext_list_price#82 - cs_ext_wholesale_cost#81) - cs_ext_discount_amt#79) + cs_ext_sales_price#80) / 2))]
-(62) CometColumnarExchange
+(62) CometExchange
Input [10]: [c_customer_id#71, c_first_name#72, c_last_name#73, c_preferred_cust_flag#74, c_birth_country#75, c_login#76, c_email_address#77, d_year#86, sum#87, isEmpty#88]
-Arguments: hashpartitioning(c_customer_id#71, c_first_name#72, c_last_name#73, c_preferred_cust_flag#74, c_birth_country#75, c_login#76, c_email_address#77, d_year#86, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(c_customer_id#71, c_first_name#72, c_last_name#73, c_preferred_cust_flag#74, c_birth_country#75, c_login#76, c_email_address#77, d_year#86, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(63) CometHashAggregate
Input [10]: [c_customer_id#71, c_first_name#72, c_last_name#73, c_preferred_cust_flag#74, c_birth_country#75, c_login#76, c_email_address#77, d_year#86, sum#87, isEmpty#88]
@@ -470,9 +470,9 @@ Input [12]: [c_customer_id#92, c_first_name#93, c_last_name#94, c_preferred_cust
Keys [8]: [c_customer_id#92, c_first_name#93, c_last_name#94, c_preferred_cust_flag#95, c_birth_country#96, c_login#97, c_email_address#98, d_year#107]
Functions [1]: [partial_sum(((((ws_ext_list_price#103 - ws_ext_wholesale_cost#102) - ws_ext_discount_amt#100) + ws_ext_sales_price#101) / 2))]
-(78) CometColumnarExchange
+(78) CometExchange
Input [10]: [c_customer_id#92, c_first_name#93, c_last_name#94, c_preferred_cust_flag#95, c_birth_country#96, c_login#97, c_email_address#98, d_year#107, sum#108, isEmpty#109]
-Arguments: hashpartitioning(c_customer_id#92, c_first_name#93, c_last_name#94, c_preferred_cust_flag#95, c_birth_country#96, c_login#97, c_email_address#98, d_year#107, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(c_customer_id#92, c_first_name#93, c_last_name#94, c_preferred_cust_flag#95, c_birth_country#96, c_login#97, c_email_address#98, d_year#107, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(79) CometHashAggregate
Input [10]: [c_customer_id#92, c_first_name#93, c_last_name#94, c_preferred_cust_flag#95, c_birth_country#96, c_login#97, c_email_address#98, d_year#107, sum#108, isEmpty#109]
@@ -549,9 +549,9 @@ Input [12]: [c_customer_id#113, c_first_name#114, c_last_name#115, c_preferred_c
Keys [8]: [c_customer_id#113, c_first_name#114, c_last_name#115, c_preferred_cust_flag#116, c_birth_country#117, c_login#118, c_email_address#119, d_year#128]
Functions [1]: [partial_sum(((((ws_ext_list_price#124 - ws_ext_wholesale_cost#123) - ws_ext_discount_amt#121) + ws_ext_sales_price#122) / 2))]
-(95) CometColumnarExchange
+(95) CometExchange
Input [10]: [c_customer_id#113, c_first_name#114, c_last_name#115, c_preferred_cust_flag#116, c_birth_country#117, c_login#118, c_email_address#119, d_year#128, sum#129, isEmpty#130]
-Arguments: hashpartitioning(c_customer_id#113, c_first_name#114, c_last_name#115, c_preferred_cust_flag#116, c_birth_country#117, c_login#118, c_email_address#119, d_year#128, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: hashpartitioning(c_customer_id#113, c_first_name#114, c_last_name#115, c_preferred_cust_flag#116, c_birth_country#117, c_login#118, c_email_address#119, d_year#128, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
(96) CometHashAggregate
Input [10]: [c_customer_id#113, c_first_name#114, c_last_name#115, c_preferred_cust_flag#116, c_birth_country#117, c_login#118, c_email_address#119, d_year#128, sum#129, isEmpty#130]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/simplified.txt
index 7e6b1257..d6651fad 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/simplified.txt
@@ -13,7 +13,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [customer_id,year_total,customer_id,customer_first_name,customer_last_name,customer_preferred_cust_flag,customer_birth_country,customer_login,customer_email_address,year_total]
CometFilter [customer_id,year_total]
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,sum(((((ss_ext_list_price - ss_ext_wholesale_cost) - ss_ext_discount_amt) + ss_ext_sales_price) / 2))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #1
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #1
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,ss_ext_list_price,ss_ext_wholesale_cost,ss_ext_discount_amt,ss_ext_sales_price]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_sales_price,ss_ext_wholesale_cost,ss_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_sales_price,ss_ext_wholesale_cost,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
@@ -36,7 +36,7 @@ WholeStageCodegen (1)
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [customer_id,customer_first_name,customer_last_name,customer_preferred_cust_flag,customer_birth_country,customer_login,customer_email_address,year_total] #5
CometHashAggregate [customer_id,customer_first_name,customer_last_name,customer_preferred_cust_flag,customer_birth_country,customer_login,customer_email_address,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,sum(((((ss_ext_list_price - ss_ext_wholesale_cost) - ss_ext_discount_amt) + ss_ext_sales_price) / 2))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #6
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #6
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,ss_ext_list_price,ss_ext_wholesale_cost,ss_ext_discount_amt,ss_ext_sales_price]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_sales_price,ss_ext_wholesale_cost,ss_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_sales_price,ss_ext_wholesale_cost,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
@@ -60,7 +60,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [customer_id,year_total] #10
CometFilter [customer_id,year_total]
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,sum(((((cs_ext_list_price - cs_ext_wholesale_cost) - cs_ext_discount_amt) + cs_ext_sales_price) / 2))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #11
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,cs_ext_list_price,cs_ext_wholesale_cost,cs_ext_discount_amt,cs_ext_sales_price]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,cs_ext_discount_amt,cs_ext_sales_price,cs_ext_wholesale_cost,cs_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,cs_ext_discount_amt,cs_ext_sales_price,cs_ext_wholesale_cost,cs_ext_list_price,cs_sold_date_sk,d_date_sk,d_year]
@@ -75,7 +75,7 @@ WholeStageCodegen (1)
ReusedExchange [d_date_sk,d_year] #4
CometBroadcastExchange [customer_id,year_total] #13
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,sum(((((cs_ext_list_price - cs_ext_wholesale_cost) - cs_ext_discount_amt) + cs_ext_sales_price) / 2))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #14
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,cs_ext_list_price,cs_ext_wholesale_cost,cs_ext_discount_amt,cs_ext_sales_price]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,cs_ext_discount_amt,cs_ext_sales_price,cs_ext_wholesale_cost,cs_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,cs_ext_discount_amt,cs_ext_sales_price,cs_ext_wholesale_cost,cs_ext_list_price,cs_sold_date_sk,d_date_sk,d_year]
@@ -91,7 +91,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [customer_id,year_total] #16
CometFilter [customer_id,year_total]
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,sum(((((ws_ext_list_price - ws_ext_wholesale_cost) - ws_ext_discount_amt) + ws_ext_sales_price) / 2))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #17
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #17
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,ws_ext_list_price,ws_ext_wholesale_cost,ws_ext_discount_amt,ws_ext_sales_price]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_sales_price,ws_ext_wholesale_cost,ws_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_sales_price,ws_ext_wholesale_cost,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
@@ -106,7 +106,7 @@ WholeStageCodegen (1)
ReusedExchange [d_date_sk,d_year] #4
CometBroadcastExchange [customer_id,year_total] #19
CometHashAggregate [customer_id,year_total,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,sum(((((ws_ext_list_price - ws_ext_wholesale_cost) - ws_ext_discount_amt) + ws_ext_sales_price) / 2))]
- CometColumnarExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #20
+ CometExchange [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year] #20
CometHashAggregate [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,d_year,sum,isEmpty,ws_ext_list_price,ws_ext_wholesale_cost,ws_ext_discount_amt,ws_ext_sales_price]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_sales_price,ws_ext_wholesale_cost,ws_ext_list_price,d_year]
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_sales_price,ws_ext_wholesale_cost,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/explain.txt
index 00967faa..224e100a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (32)
+- CometTakeOrderedAndProject (31)
+- CometHashAggregate (30)
- +- CometColumnarExchange (29)
+ +- CometExchange (29)
+- CometHashAggregate (28)
+- CometProject (27)
+- CometBroadcastHashJoin (26)
@@ -13,11 +13,11 @@
: : :- CometProject (11)
: : : +- CometSortMergeJoin (10)
: : : :- CometSort (4)
- : : : : +- CometColumnarExchange (3)
+ : : : : +- CometExchange (3)
: : : : +- CometFilter (2)
: : : : +- CometScan parquet spark_catalog.default.catalog_sales (1)
: : : +- CometSort (9)
- : : : +- CometColumnarExchange (8)
+ : : : +- CometExchange (8)
: : : +- CometProject (7)
: : : +- CometFilter (6)
: : : +- CometScan parquet spark_catalog.default.catalog_returns (5)
@@ -45,9 +45,9 @@ ReadSchema: struct<cs_warehouse_sk:int,cs_item_sk:int,cs_order_number:int,cs_sal
Input [5]: [cs_warehouse_sk#1, cs_item_sk#2, cs_order_number#3, cs_sales_price#4, cs_sold_date_sk#5]
Condition : (isnotnull(cs_warehouse_sk#1) AND isnotnull(cs_item_sk#2))
-(3) CometColumnarExchange
+(3) CometExchange
Input [5]: [cs_warehouse_sk#1, cs_item_sk#2, cs_order_number#3, cs_sales_price#4, cs_sold_date_sk#5]
-Arguments: hashpartitioning(cs_order_number#3, cs_item_sk#2, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(cs_order_number#3, cs_item_sk#2, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(4) CometSort
Input [5]: [cs_warehouse_sk#1, cs_item_sk#2, cs_order_number#3, cs_sales_price#4, cs_sold_date_sk#5]
@@ -68,9 +68,9 @@ Condition : (isnotnull(cr_order_number#8) AND isnotnull(cr_item_sk#7))
Input [4]: [cr_item_sk#7, cr_order_number#8, cr_refunded_cash#9, cr_returned_date_sk#10]
Arguments: [cr_item_sk#7, cr_order_number#8, cr_refunded_cash#9], [cr_item_sk#7, cr_order_number#8, cr_refunded_cash#9]
-(8) CometColumnarExchange
+(8) CometExchange
Input [3]: [cr_item_sk#7, cr_order_number#8, cr_refunded_cash#9]
-Arguments: hashpartitioning(cr_order_number#8, cr_item_sk#7, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(cr_order_number#8, cr_item_sk#7, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(9) CometSort
Input [3]: [cr_item_sk#7, cr_order_number#8, cr_refunded_cash#9]
@@ -166,9 +166,9 @@ Input [5]: [cs_sales_price#4, cr_refunded_cash#9, w_state#12, i_item_id#14, d_da
Keys [2]: [w_state#12, i_item_id#14]
Functions [2]: [partial_sum(CASE WHEN (d_date#17 < 2000-03-11) THEN (cs_sales_price#4 - coalesce(cast(cr_refunded_cash#9 as decimal(12,2)), 0.00)) ELSE 0.00 END), partial_sum(CASE WHEN (d_date#17 >= 2000-03-11) THEN (cs_sales_price#4 - coalesce(cast(cr_refunded_cash#9 as decimal(12,2)), 0.00)) ELSE 0.00 END)]
-(29) CometColumnarExchange
+(29) CometExchange
Input [6]: [w_state#12, i_item_id#14, sum#18, isEmpty#19, sum#20, isEmpty#21]
-Arguments: hashpartitioning(w_state#12, i_item_id#14, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(w_state#12, i_item_id#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(30) CometHashAggregate
Input [6]: [w_state#12, i_item_id#14, sum#18, isEmpty#19, sum#20, isEmpty#21]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/simplified.txt
index 6e47fa59..eac61381 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q40/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [w_state,i_item_id,sales_before,sales_after]
CometHashAggregate [w_state,i_item_id,sales_before,sales_after,sum,isEmpty,sum,isEmpty,sum(CASE WHEN (d_date < 2000-03-11) THEN (cs_sales_price - coalesce(cast(cr_refunded_cash as decimal(12,2)), 0.00)) ELSE 0.00 END),sum(CASE WHEN (d_date >= 2000-03-11) THEN (cs_sales_price - coalesce(cast(cr_refunded_cash as decimal(12,2)), 0.00)) ELSE 0.00 END)]
- CometColumnarExchange [w_state,i_item_id] #1
+ CometExchange [w_state,i_item_id] #1
CometHashAggregate [w_state,i_item_id,sum,isEmpty,sum,isEmpty,d_date,cs_sales_price,cr_refunded_cash]
CometProject [cs_sales_price,cr_refunded_cash,w_state,i_item_id,d_date]
CometBroadcastHashJoin [cs_sales_price,cs_sold_date_sk,cr_refunded_cash,w_state,i_item_id,d_date_sk,d_date]
@@ -14,7 +14,7 @@ WholeStageCodegen (1)
CometProject [cs_warehouse_sk,cs_item_sk,cs_sales_price,cs_sold_date_sk,cr_refunded_cash]
CometSortMergeJoin [cs_warehouse_sk,cs_item_sk,cs_order_number,cs_sales_price,cs_sold_date_sk,cr_item_sk,cr_order_number,cr_refunded_cash]
CometSort [cs_warehouse_sk,cs_item_sk,cs_order_number,cs_sales_price,cs_sold_date_sk]
- CometColumnarExchange [cs_order_number,cs_item_sk] #2
+ CometExchange [cs_order_number,cs_item_sk] #2
CometFilter [cs_warehouse_sk,cs_item_sk,cs_order_number,cs_sales_price,cs_sold_date_sk]
CometScan parquet spark_catalog.default.catalog_sales [cs_warehouse_sk,cs_item_sk,cs_order_number,cs_sales_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
@@ -25,7 +25,7 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_date]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
CometSort [cr_item_sk,cr_order_number,cr_refunded_cash]
- CometColumnarExchange [cr_order_number,cr_item_sk] #4
+ CometExchange [cr_order_number,cr_item_sk] #4
CometProject [cr_item_sk,cr_order_number,cr_refunded_cash]
CometFilter [cr_item_sk,cr_order_number,cr_refunded_cash,cr_returned_date_sk]
CometScan parquet spark_catalog.default.catalog_returns [cr_item_sk,cr_order_number,cr_refunded_cash,cr_returned_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/explain.txt
index 5c2b28b9..c4059e5f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (19)
+- CometTakeOrderedAndProject (18)
+- CometHashAggregate (17)
- +- CometColumnarExchange (16)
+ +- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
+- CometBroadcastHashJoin (13)
@@ -13,7 +13,7 @@
+- CometProject (11)
+- CometFilter (10)
+- CometHashAggregate (9)
- +- CometColumnarExchange (8)
+ +- CometExchange (8)
+- CometHashAggregate (7)
+- CometProject (6)
+- CometFilter (5)
@@ -55,9 +55,9 @@ Input [1]: [i_manufact#5]
Keys [1]: [i_manufact#5]
Functions [1]: [partial_count(1)]
-(8) CometColumnarExchange
+(8) CometExchange
Input [2]: [i_manufact#5, count#9]
-Arguments: hashpartitioning(i_manufact#5, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_manufact#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(9) CometHashAggregate
Input [2]: [i_manufact#5, count#9]
@@ -90,9 +90,9 @@ Input [1]: [i_product_name#3]
Keys [1]: [i_product_name#3]
Functions: []
-(16) CometColumnarExchange
+(16) CometExchange
Input [1]: [i_product_name#3]
-Arguments: hashpartitioning(i_product_name#3, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_product_name#3, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(17) CometHashAggregate
Input [1]: [i_product_name#3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/simplified.txt
index 732e8679..00710349 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q41/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_product_name]
CometHashAggregate [i_product_name]
- CometColumnarExchange [i_product_name] #1
+ CometExchange [i_product_name] #1
CometHashAggregate [i_product_name]
CometProject [i_product_name]
CometBroadcastHashJoin [i_manufact,i_product_name,i_manufact]
@@ -14,7 +14,7 @@ WholeStageCodegen (1)
CometProject [i_manufact]
CometFilter [item_cnt,i_manufact]
CometHashAggregate [item_cnt,i_manufact,count,count(1)]
- CometColumnarExchange [i_manufact] #3
+ CometExchange [i_manufact] #3
CometHashAggregate [i_manufact,count]
CometProject [i_manufact]
CometFilter [i_category,i_manufact,i_size,i_color,i_units]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/explain.txt
index ae0d6f33..c2c2c8bf 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (19)
+- CometTakeOrderedAndProject (18)
+- CometHashAggregate (17)
- +- CometColumnarExchange (16)
+ +- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
+- CometBroadcastHashJoin (13)
@@ -93,9 +93,9 @@ Input [4]: [d_year#2, ss_ext_sales_price#5, i_category_id#8, i_category#9]
Keys [3]: [d_year#2, i_category_id#8, i_category#9]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
-(16) CometColumnarExchange
+(16) CometExchange
Input [4]: [d_year#2, i_category_id#8, i_category#9, sum#11]
-Arguments: hashpartitioning(d_year#2, i_category_id#8, i_category#9, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(d_year#2, i_category_id#8, i_category#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
Input [4]: [d_year#2, i_category_id#8, i_category#9, sum#11]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/simplified.txt
index bdc71f6d..40528f52 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q42/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [d_year,i_category_id,i_category,sum(ss_ext_sales_price)]
CometHashAggregate [d_year,i_category_id,i_category,sum(ss_ext_sales_price),sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [d_year,i_category_id,i_category] #1
+ CometExchange [d_year,i_category_id,i_category] #1
CometHashAggregate [d_year,i_category_id,i_category,sum,ss_ext_sales_price]
CometProject [d_year,ss_ext_sales_price,i_category_id,i_category]
CometBroadcastHashJoin [d_year,ss_item_sk,ss_ext_sales_price,i_item_sk,i_category_id,i_category]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/explain.txt
index 41a56429..d6cb50a4 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (19)
+- CometTakeOrderedAndProject (18)
+- CometHashAggregate (17)
- +- CometColumnarExchange (16)
+ +- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
+- CometBroadcastHashJoin (13)
@@ -93,9 +93,9 @@ Input [4]: [d_day_name#3, ss_sales_price#5, s_store_id#8, s_store_name#9]
Keys [2]: [s_store_name#9, s_store_id#8]
Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (d_day_name#3 = Sunday ) THEN ss_sales_price#5 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#3 = Monday ) THEN ss_sales_price#5 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#3 = Tuesday ) THEN ss_sales_price#5 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#3 = Wednesday) THEN ss_sales_price#5 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#3 = Thursday ) THEN ss_sales_price#5 END)), partial_sum(Unsc [...]
-(16) CometColumnarExchange
+(16) CometExchange
Input [9]: [s_store_name#9, s_store_id#8, sum#11, sum#12, sum#13, sum#14, sum#15, sum#16, sum#17]
-Arguments: hashpartitioning(s_store_name#9, s_store_id#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(s_store_name#9, s_store_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
Input [9]: [s_store_name#9, s_store_id#8, sum#11, sum#12, sum#13, sum#14, sum#15, sum#16, sum#17]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/simplified.txt
index 6e1b8d6b..447131b7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q43/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [s_store_name,s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales]
CometHashAggregate [s_store_name,s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHE [...]
- CometColumnarExchange [s_store_name,s_store_id] #1
+ CometExchange [s_store_name,s_store_id] #1
CometHashAggregate [s_store_name,s_store_id,sum,sum,sum,sum,sum,sum,sum,d_day_name,ss_sales_price]
CometProject [d_day_name,ss_sales_price,s_store_id,s_store_name]
CometBroadcastHashJoin [d_day_name,ss_store_sk,ss_sales_price,s_store_sk,s_store_id,s_store_name]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/explain.txt
index 9ee93656..57e6b0e8 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/explain.txt
@@ -1,50 +1,46 @@
== Physical Plan ==
-TakeOrderedAndProject (46)
-+- * Project (45)
- +- * BroadcastHashJoin Inner BuildRight (44)
- :- * Project (42)
- : +- * BroadcastHashJoin Inner BuildRight (41)
- : :- * Project (36)
- : : +- * SortMergeJoin Inner (35)
- : : :- * Sort (19)
- : : : +- * Project (18)
- : : : +- * Filter (17)
- : : : +- Window (16)
- : : : +- WindowGroupLimit (15)
- : : : +- * ColumnarToRow (14)
- : : : +- CometSort (13)
- : : : +- CometColumnarExchange (12)
- : : : +- RowToColumnar (11)
- : : : +- WindowGroupLimit (10)
- : : : +- * ColumnarToRow (9)
- : : : +- CometSort (8)
- : : : +- CometFilter (7)
- : : : +- CometHashAggregate (6)
- : : : +- CometColumnarExchange (5)
- : : : +- CometHashAggregate (4)
- : : : +- CometProject (3)
- : : : +- CometFilter (2)
- : : : +- CometScan parquet spark_catalog.default.store_sales (1)
- : : +- * Sort (34)
- : : +- * Project (33)
- : : +- * Filter (32)
- : : +- Window (31)
- : : +- WindowGroupLimit (30)
- : : +- * ColumnarToRow (29)
- : : +- CometSort (28)
- : : +- CometColumnarExchange (27)
- : : +- RowToColumnar (26)
- : : +- WindowGroupLimit (25)
- : : +- * ColumnarToRow (24)
- : : +- CometSort (23)
- : : +- CometFilter (22)
- : : +- CometHashAggregate (21)
- : : +- ReusedExchange (20)
- : +- BroadcastExchange (40)
- : +- * ColumnarToRow (39)
- : +- CometFilter (38)
- : +- CometScan parquet spark_catalog.default.item (37)
- +- ReusedExchange (43)
+TakeOrderedAndProject (42)
++- * Project (41)
+ +- * BroadcastHashJoin Inner BuildRight (40)
+ :- * Project (38)
+ : +- * BroadcastHashJoin Inner BuildRight (37)
+ : :- * Project (32)
+ : : +- * SortMergeJoin Inner (31)
+ : : :- * Sort (17)
+ : : : +- * Project (16)
+ : : : +- * Filter (15)
+ : : : +- Window (14)
+ : : : +- WindowGroupLimit (13)
+ : : : +- * Sort (12)
+ : : : +- Exchange (11)
+ : : : +- WindowGroupLimit (10)
+ : : : +- * ColumnarToRow (9)
+ : : : +- CometSort (8)
+ : : : +- CometFilter (7)
+ : : : +- CometHashAggregate (6)
+ : : : +- CometExchange (5)
+ : : : +- CometHashAggregate (4)
+ : : : +- CometProject (3)
+ : : : +- CometFilter (2)
+ : : : +- CometScan parquet spark_catalog.default.store_sales (1)
+ : : +- * Sort (30)
+ : : +- * Project (29)
+ : : +- * Filter (28)
+ : : +- Window (27)
+ : : +- WindowGroupLimit (26)
+ : : +- * Sort (25)
+ : : +- Exchange (24)
+ : : +- WindowGroupLimit (23)
+ : : +- * ColumnarToRow (22)
+ : : +- CometSort (21)
+ : : +- CometFilter (20)
+ : : +- CometHashAggregate (19)
+ : : +- ReusedExchange (18)
+ : +- BroadcastExchange (36)
+ : +- * ColumnarToRow (35)
+ : +- CometFilter (34)
+ : +- CometScan parquet spark_catalog.default.item (33)
+ +- ReusedExchange (39)
(1) Scan parquet spark_catalog.default.store_sales
@@ -67,9 +63,9 @@ Input [2]: [ss_item_sk#1, ss_net_profit#3]
Keys [1]: [ss_item_sk#1]
Functions [1]: [partial_avg(UnscaledValue(ss_net_profit#3))]
-(5) CometColumnarExchange
+(5) CometExchange
Input [3]: [ss_item_sk#1, sum#5, count#6]
-Arguments: hashpartitioning(ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(6) CometHashAggregate
Input [3]: [ss_item_sk#1, sum#5, count#6]
@@ -91,196 +87,184 @@ Input [2]: [item_sk#7, rank_col#8]
Input [2]: [item_sk#7, rank_col#8]
Arguments: [rank_col#8 ASC NULLS FIRST], rank(rank_col#8), 10, Partial
-(11) RowToColumnar
+(11) Exchange
Input [2]: [item_sk#7, rank_col#8]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=2]
-(12) CometColumnarExchange
+(12) Sort [codegen id : 2]
Input [2]: [item_sk#7, rank_col#8]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: [rank_col#8 ASC NULLS FIRST], false, 0
-(13) CometSort
-Input [2]: [item_sk#7, rank_col#8]
-Arguments: [item_sk#7, rank_col#8], [rank_col#8 ASC NULLS FIRST]
-
-(14) ColumnarToRow [codegen id : 2]
-Input [2]: [item_sk#7, rank_col#8]
-
-(15) WindowGroupLimit
+(13) WindowGroupLimit
Input [2]: [item_sk#7, rank_col#8]
Arguments: [rank_col#8 ASC NULLS FIRST], rank(rank_col#8), 10, Final
-(16) Window
+(14) Window
Input [2]: [item_sk#7, rank_col#8]
Arguments: [rank(rank_col#8) windowspecdefinition(rank_col#8 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rnk#11], [rank_col#8 ASC NULLS FIRST]
-(17) Filter [codegen id : 3]
+(15) Filter [codegen id : 3]
Input [3]: [item_sk#7, rank_col#8, rnk#11]
Condition : ((rnk#11 < 11) AND isnotnull(item_sk#7))
-(18) Project [codegen id : 3]
+(16) Project [codegen id : 3]
Output [2]: [item_sk#7, rnk#11]
Input [3]: [item_sk#7, rank_col#8, rnk#11]
-(19) Sort [codegen id : 3]
+(17) Sort [codegen id : 3]
Input [2]: [item_sk#7, rnk#11]
Arguments: [rnk#11 ASC NULLS FIRST], false, 0
-(20) ReusedExchange [Reuses operator id: 5]
+(18) ReusedExchange [Reuses operator id: 5]
Output [3]: [ss_item_sk#12, sum#13, count#14]
-(21) CometHashAggregate
+(19) CometHashAggregate
Input [3]: [ss_item_sk#12, sum#13, count#14]
Keys [1]: [ss_item_sk#12]
Functions [1]: [avg(UnscaledValue(ss_net_profit#15))]
-(22) CometFilter
+(20) CometFilter
Input [2]: [item_sk#16, rank_col#17]
Condition : (isnotnull(rank_col#17) AND (cast(rank_col#17 as decimal(13,7)) > (0.9 * ReusedSubquery Subquery scalar-subquery#9, [id=#10])))
-(23) CometSort
+(21) CometSort
Input [2]: [item_sk#16, rank_col#17]
Arguments: [item_sk#16, rank_col#17], [rank_col#17 DESC NULLS LAST]
-(24) ColumnarToRow [codegen id : 4]
+(22) ColumnarToRow [codegen id : 4]
Input [2]: [item_sk#16, rank_col#17]
-(25) WindowGroupLimit
+(23) WindowGroupLimit
Input [2]: [item_sk#16, rank_col#17]
Arguments: [rank_col#17 DESC NULLS LAST], rank(rank_col#17), 10, Partial
-(26) RowToColumnar
+(24) Exchange
Input [2]: [item_sk#16, rank_col#17]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=3]
-(27) CometColumnarExchange
-Input [2]: [item_sk#16, rank_col#17]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
-
-(28) CometSort
-Input [2]: [item_sk#16, rank_col#17]
-Arguments: [item_sk#16, rank_col#17], [rank_col#17 DESC NULLS LAST]
-
-(29) ColumnarToRow [codegen id : 5]
+(25) Sort [codegen id : 5]
Input [2]: [item_sk#16, rank_col#17]
+Arguments: [rank_col#17 DESC NULLS LAST], false, 0
-(30) WindowGroupLimit
+(26) WindowGroupLimit
Input [2]: [item_sk#16, rank_col#17]
Arguments: [rank_col#17 DESC NULLS LAST], rank(rank_col#17), 10, Final
-(31) Window
+(27) Window
Input [2]: [item_sk#16, rank_col#17]
Arguments: [rank(rank_col#17) windowspecdefinition(rank_col#17 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rnk#18], [rank_col#17 DESC NULLS LAST]
-(32) Filter [codegen id : 6]
+(28) Filter [codegen id : 6]
Input [3]: [item_sk#16, rank_col#17, rnk#18]
Condition : ((rnk#18 < 11) AND isnotnull(item_sk#16))
-(33) Project [codegen id : 6]
+(29) Project [codegen id : 6]
Output [2]: [item_sk#16, rnk#18]
Input [3]: [item_sk#16, rank_col#17, rnk#18]
-(34) Sort [codegen id : 6]
+(30) Sort [codegen id : 6]
Input [2]: [item_sk#16, rnk#18]
Arguments: [rnk#18 ASC NULLS FIRST], false, 0
-(35) SortMergeJoin [codegen id : 9]
+(31) SortMergeJoin [codegen id : 9]
Left keys [1]: [rnk#11]
Right keys [1]: [rnk#18]
Join type: Inner
Join condition: None
-(36) Project [codegen id : 9]
+(32) Project [codegen id : 9]
Output [3]: [item_sk#7, rnk#11, item_sk#16]
Input [4]: [item_sk#7, rnk#11, item_sk#16, rnk#18]
-(37) Scan parquet spark_catalog.default.item
+(33) Scan parquet spark_catalog.default.item
Output [2]: [i_item_sk#19, i_product_name#20]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_product_name:string>
-(38) CometFilter
+(34) CometFilter
Input [2]: [i_item_sk#19, i_product_name#20]
Condition : isnotnull(i_item_sk#19)
-(39) ColumnarToRow [codegen id : 7]
+(35) ColumnarToRow [codegen id : 7]
Input [2]: [i_item_sk#19, i_product_name#20]
-(40) BroadcastExchange
+(36) BroadcastExchange
Input [2]: [i_item_sk#19, i_product_name#20]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=4]
-(41) BroadcastHashJoin [codegen id : 9]
+(37) BroadcastHashJoin [codegen id : 9]
Left keys [1]: [item_sk#7]
Right keys [1]: [i_item_sk#19]
Join type: Inner
Join condition: None
-(42) Project [codegen id : 9]
+(38) Project [codegen id : 9]
Output [3]: [rnk#11, item_sk#16, i_product_name#20]
Input [5]: [item_sk#7, rnk#11, item_sk#16, i_item_sk#19, i_product_name#20]
-(43) ReusedExchange [Reuses operator id: 40]
+(39) ReusedExchange [Reuses operator id: 36]
Output [2]: [i_item_sk#21, i_product_name#22]
-(44) BroadcastHashJoin [codegen id : 9]
+(40) BroadcastHashJoin [codegen id : 9]
Left keys [1]: [item_sk#16]
Right keys [1]: [i_item_sk#21]
Join type: Inner
Join condition: None
-(45) Project [codegen id : 9]
+(41) Project [codegen id : 9]
Output [3]: [rnk#11, i_product_name#20 AS best_performing#23, i_product_name#22 AS worst_performing#24]
Input [5]: [rnk#11, item_sk#16, i_product_name#20, i_item_sk#21, i_product_name#22]
-(46) TakeOrderedAndProject
+(42) TakeOrderedAndProject
Input [3]: [rnk#11, best_performing#23, worst_performing#24]
Arguments: 100, [rnk#11 ASC NULLS FIRST], [rnk#11, best_performing#23, worst_performing#24]
===== Subqueries =====
Subquery:1 Hosting operator id = 7 Hosting Expression = Subquery scalar-subquery#9, [id=#10]
-* ColumnarToRow (53)
-+- CometHashAggregate (52)
- +- CometColumnarExchange (51)
- +- CometHashAggregate (50)
- +- CometProject (49)
- +- CometFilter (48)
- +- CometScan parquet spark_catalog.default.store_sales (47)
+* ColumnarToRow (49)
++- CometHashAggregate (48)
+ +- CometExchange (47)
+ +- CometHashAggregate (46)
+ +- CometProject (45)
+ +- CometFilter (44)
+ +- CometScan parquet spark_catalog.default.store_sales (43)
-(47) Scan parquet spark_catalog.default.store_sales
+(43) Scan parquet spark_catalog.default.store_sales
Output [4]: [ss_addr_sk#25, ss_store_sk#26, ss_net_profit#27, ss_sold_date_sk#28]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store_sales]
PushedFilters: [IsNotNull(ss_store_sk), EqualTo(ss_store_sk,4), IsNull(ss_addr_sk)]
ReadSchema: struct<ss_addr_sk:int,ss_store_sk:int,ss_net_profit:decimal(7,2)>
-(48) CometFilter
+(44) CometFilter
Input [4]: [ss_addr_sk#25, ss_store_sk#26, ss_net_profit#27, ss_sold_date_sk#28]
Condition : ((isnotnull(ss_store_sk#26) AND (ss_store_sk#26 = 4)) AND isnull(ss_addr_sk#25))
-(49) CometProject
+(45) CometProject
Input [4]: [ss_addr_sk#25, ss_store_sk#26, ss_net_profit#27, ss_sold_date_sk#28]
Arguments: [ss_store_sk#26, ss_net_profit#27], [ss_store_sk#26, ss_net_profit#27]
-(50) CometHashAggregate
+(46) CometHashAggregate
Input [2]: [ss_store_sk#26, ss_net_profit#27]
Keys [1]: [ss_store_sk#26]
Functions [1]: [partial_avg(UnscaledValue(ss_net_profit#27))]
-(51) CometColumnarExchange
+(47) CometExchange
Input [3]: [ss_store_sk#26, sum#29, count#30]
-Arguments: hashpartitioning(ss_store_sk#26, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(ss_store_sk#26, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
-(52) CometHashAggregate
+(48) CometHashAggregate
Input [3]: [ss_store_sk#26, sum#29, count#30]
Keys [1]: [ss_store_sk#26]
Functions [1]: [avg(UnscaledValue(ss_net_profit#27))]
-(53) ColumnarToRow [codegen id : 1]
+(49) ColumnarToRow [codegen id : 1]
Input [1]: [rank_col#31]
-Subquery:2 Hosting operator id = 22 Hosting Expression = ReusedSubquery Subquery scalar-subquery#9, [id=#10]
+Subquery:2 Hosting operator id = 20 Hosting Expression = ReusedSubquery Subquery scalar-subquery#9, [id=#10]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/simplified.txt
index 886ee5e0..58687dc8 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q44/simplified.txt
@@ -15,33 +15,31 @@ TakeOrderedAndProject [rnk,best_performing,worst_performing]
Window [rank_col]
WindowGroupLimit [rank_col]
WholeStageCodegen (2)
- ColumnarToRow
+ Sort [rank_col]
InputAdapter
- CometSort [item_sk,rank_col]
- CometColumnarExchange #1
- RowToColumnar
- WindowGroupLimit [rank_col]
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometSort [item_sk,rank_col]
- CometFilter [item_sk,rank_col]
- Subquery #1
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometHashAggregate [rank_col,ss_store_sk,sum,count,avg(UnscaledValue(ss_net_profit))]
- CometColumnarExchange [ss_store_sk] #3
- CometHashAggregate [ss_store_sk,sum,count,ss_net_profit]
- CometProject [ss_store_sk,ss_net_profit]
- CometFilter [ss_addr_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_addr_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
- CometHashAggregate [item_sk,rank_col,ss_item_sk,sum,count,avg(UnscaledValue(ss_net_profit))]
- CometColumnarExchange [ss_item_sk] #2
- CometHashAggregate [ss_item_sk,sum,count,ss_net_profit]
- CometProject [ss_item_sk,ss_net_profit]
- CometFilter [ss_item_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
+ Exchange #1
+ WindowGroupLimit [rank_col]
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item_sk,rank_col]
+ CometFilter [item_sk,rank_col]
+ Subquery #1
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometHashAggregate [rank_col,ss_store_sk,sum,count,avg(UnscaledValue(ss_net_profit))]
+ CometExchange [ss_store_sk] #3
+ CometHashAggregate [ss_store_sk,sum,count,ss_net_profit]
+ CometProject [ss_store_sk,ss_net_profit]
+ CometFilter [ss_addr_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_addr_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
+ CometHashAggregate [item_sk,rank_col,ss_item_sk,sum,count,avg(UnscaledValue(ss_net_profit))]
+ CometExchange [ss_item_sk] #2
+ CometHashAggregate [ss_item_sk,sum,count,ss_net_profit]
+ CometProject [ss_item_sk,ss_net_profit]
+ CometFilter [ss_item_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_store_sk,ss_net_profit,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (6)
Sort [rnk]
@@ -51,20 +49,18 @@ TakeOrderedAndProject [rnk,best_performing,worst_performing]
Window [rank_col]
WindowGroupLimit [rank_col]
WholeStageCodegen (5)
- ColumnarToRow
+ Sort [rank_col]
InputAdapter
- CometSort [item_sk,rank_col]
- CometColumnarExchange #4
- RowToColumnar
- WindowGroupLimit [rank_col]
- WholeStageCodegen (4)
- ColumnarToRow
- InputAdapter
- CometSort [item_sk,rank_col]
- CometFilter [item_sk,rank_col]
- ReusedSubquery [rank_col] #1
- CometHashAggregate [item_sk,rank_col,ss_item_sk,sum,count,avg(UnscaledValue(ss_net_profit))]
- ReusedExchange [ss_item_sk,sum,count] #2
+ Exchange #4
+ WindowGroupLimit [rank_col]
+ WholeStageCodegen (4)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item_sk,rank_col]
+ CometFilter [item_sk,rank_col]
+ ReusedSubquery [rank_col] #1
+ CometHashAggregate [item_sk,rank_col,ss_item_sk,sum,count,avg(UnscaledValue(ss_net_profit))]
+ ReusedExchange [ss_item_sk,sum,count] #2
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/explain.txt
index 04424754..f128499e 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/explain.txt
@@ -1,42 +1,40 @@
== Physical Plan ==
-TakeOrderedAndProject (38)
-+- * HashAggregate (37)
- +- * ColumnarToRow (36)
- +- CometColumnarExchange (35)
- +- RowToColumnar (34)
- +- * HashAggregate (33)
- +- * Project (32)
- +- * Filter (31)
- +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (30)
- :- * ColumnarToRow (24)
- : +- CometProject (23)
- : +- CometBroadcastHashJoin (22)
- : :- CometProject (18)
- : : +- CometBroadcastHashJoin (17)
- : : :- CometProject (12)
- : : : +- CometBroadcastHashJoin (11)
- : : : :- CometProject (7)
- : : : : +- CometBroadcastHashJoin (6)
- : : : : :- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.web_sales (1)
- : : : : +- CometBroadcastExchange (5)
- : : : : +- CometFilter (4)
- : : : : +- CometScan parquet spark_catalog.default.customer (3)
- : : : +- CometBroadcastExchange (10)
- : : : +- CometFilter (9)
- : : : +- CometScan parquet spark_catalog.default.customer_address (8)
- : : +- CometBroadcastExchange (16)
- : : +- CometProject (15)
- : : +- CometFilter (14)
- : : +- CometScan parquet spark_catalog.default.date_dim (13)
- : +- CometBroadcastExchange (21)
- : +- CometFilter (20)
- : +- CometScan parquet spark_catalog.default.item (19)
- +- BroadcastExchange (29)
- +- * ColumnarToRow (28)
- +- CometProject (27)
- +- CometFilter (26)
- +- CometScan parquet spark_catalog.default.item (25)
+TakeOrderedAndProject (36)
++- * HashAggregate (35)
+ +- Exchange (34)
+ +- * HashAggregate (33)
+ +- * Project (32)
+ +- * Filter (31)
+ +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (30)
+ :- * ColumnarToRow (24)
+ : +- CometProject (23)
+ : +- CometBroadcastHashJoin (22)
+ : :- CometProject (18)
+ : : +- CometBroadcastHashJoin (17)
+ : : :- CometProject (12)
+ : : : +- CometBroadcastHashJoin (11)
+ : : : :- CometProject (7)
+ : : : : +- CometBroadcastHashJoin (6)
+ : : : : :- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.web_sales (1)
+ : : : : +- CometBroadcastExchange (5)
+ : : : : +- CometFilter (4)
+ : : : : +- CometScan parquet spark_catalog.default.customer (3)
+ : : : +- CometBroadcastExchange (10)
+ : : : +- CometFilter (9)
+ : : : +- CometScan parquet spark_catalog.default.customer_address (8)
+ : : +- CometBroadcastExchange (16)
+ : : +- CometProject (15)
+ : : +- CometFilter (14)
+ : : +- CometScan parquet spark_catalog.default.date_dim (13)
+ : +- CometBroadcastExchange (21)
+ : +- CometFilter (20)
+ : +- CometScan parquet spark_catalog.default.item (19)
+ +- BroadcastExchange (29)
+ +- * ColumnarToRow (28)
+ +- CometProject (27)
+ +- CometFilter (26)
+ +- CometScan parquet spark_catalog.default.item (25)
(1) Scan parquet spark_catalog.default.web_sales
@@ -197,56 +195,50 @@ Functions [1]: [partial_sum(UnscaledValue(ws_sales_price#4))]
Aggregate Attributes [1]: [sum#19]
Results [3]: [ca_zip#11, ca_city#10, sum#20]
-(34) RowToColumnar
+(34) Exchange
Input [3]: [ca_zip#11, ca_city#10, sum#20]
+Arguments: hashpartitioning(ca_zip#11, ca_city#10, 5), ENSURE_REQUIREMENTS, [plan_id=2]
-(35) CometColumnarExchange
-Input [3]: [ca_zip#11, ca_city#10, sum#20]
-Arguments: hashpartitioning(ca_zip#11, ca_city#10, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
-
-(36) ColumnarToRow [codegen id : 3]
-Input [3]: [ca_zip#11, ca_city#10, sum#20]
-
-(37) HashAggregate [codegen id : 3]
+(35) HashAggregate [codegen id : 3]
Input [3]: [ca_zip#11, ca_city#10, sum#20]
Keys [2]: [ca_zip#11, ca_city#10]
Functions [1]: [sum(UnscaledValue(ws_sales_price#4))]
Aggregate Attributes [1]: [sum(UnscaledValue(ws_sales_price#4))#21]
Results [3]: [ca_zip#11, ca_city#10, MakeDecimal(sum(UnscaledValue(ws_sales_price#4))#21,17,2) AS sum(ws_sales_price)#22]
-(38) TakeOrderedAndProject
+(36) TakeOrderedAndProject
Input [3]: [ca_zip#11, ca_city#10, sum(ws_sales_price)#22]
Arguments: 100, [ca_zip#11 ASC NULLS FIRST, ca_city#10 ASC NULLS FIRST], [ca_zip#11, ca_city#10, sum(ws_sales_price)#22]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ws_sold_date_sk#5 IN dynamicpruning#6
-BroadcastExchange (43)
-+- * ColumnarToRow (42)
- +- CometProject (41)
- +- CometFilter (40)
- +- CometScan parquet spark_catalog.default.date_dim (39)
+BroadcastExchange (41)
++- * ColumnarToRow (40)
+ +- CometProject (39)
+ +- CometFilter (38)
+ +- CometScan parquet spark_catalog.default.date_dim (37)
-(39) Scan parquet spark_catalog.default.date_dim
+(37) Scan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#12, d_year#13, d_qoy#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,2), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(40) CometFilter
+(38) CometFilter
Input [3]: [d_date_sk#12, d_year#13, d_qoy#14]
Condition : ((((isnotnull(d_qoy#14) AND isnotnull(d_year#13)) AND (d_qoy#14 = 2)) AND (d_year#13 = 2001)) AND isnotnull(d_date_sk#12))
-(41) CometProject
+(39) CometProject
Input [3]: [d_date_sk#12, d_year#13, d_qoy#14]
Arguments: [d_date_sk#12], [d_date_sk#12]
-(42) ColumnarToRow [codegen id : 1]
+(40) ColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#12]
-(43) BroadcastExchange
+(41) BroadcastExchange
Input [1]: [d_date_sk#12]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/simplified.txt
index 8550b91c..f60fdb18 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q45/simplified.txt
@@ -1,53 +1,51 @@
TakeOrderedAndProject [ca_zip,ca_city,sum(ws_sales_price)]
WholeStageCodegen (3)
HashAggregate [ca_zip,ca_city,sum] [sum(UnscaledValue(ws_sales_price)),sum(ws_sales_price),sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [ca_zip,ca_city] #1
- RowToColumnar
- WholeStageCodegen (2)
- HashAggregate [ca_zip,ca_city,ws_sales_price] [sum,sum]
- Project [ws_sales_price,ca_city,ca_zip]
- Filter [ca_zip,exists]
- BroadcastHashJoin [i_item_id,i_item_id]
- ColumnarToRow
- InputAdapter
- CometProject [ws_sales_price,ca_city,ca_zip,i_item_id]
- CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ca_city,ca_zip,i_item_sk,i_item_id]
- CometProject [ws_item_sk,ws_sales_price,ca_city,ca_zip]
- CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ws_sold_date_sk,ca_city,ca_zip,d_date_sk]
- CometProject [ws_item_sk,ws_sales_price,ws_sold_date_sk,ca_city,ca_zip]
- CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ws_sold_date_sk,c_current_addr_sk,ca_address_sk,ca_city,ca_zip]
- CometProject [ws_item_sk,ws_sales_price,ws_sold_date_sk,c_current_addr_sk]
- CometBroadcastHashJoin [ws_item_sk,ws_bill_customer_sk,ws_sales_price,ws_sold_date_sk,c_customer_sk,c_current_addr_sk]
- CometFilter [ws_item_sk,ws_bill_customer_sk,ws_sales_price,ws_sold_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sales_price,ws_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #2
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- CometBroadcastExchange [c_customer_sk,c_current_addr_sk] #3
- CometFilter [c_customer_sk,c_current_addr_sk]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
- CometBroadcastExchange [ca_address_sk,ca_city,ca_zip] #4
- CometFilter [ca_address_sk,ca_city,ca_zip]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_city,ca_zip]
- CometBroadcastExchange [d_date_sk] #5
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_qoy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
- CometBroadcastExchange [i_item_sk,i_item_id] #6
- CometFilter [i_item_sk,i_item_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
- InputAdapter
- BroadcastExchange #7
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [i_item_id]
- CometFilter [i_item_sk,i_item_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
+ InputAdapter
+ Exchange [ca_zip,ca_city] #1
+ WholeStageCodegen (2)
+ HashAggregate [ca_zip,ca_city,ws_sales_price] [sum,sum]
+ Project [ws_sales_price,ca_city,ca_zip]
+ Filter [ca_zip,exists]
+ BroadcastHashJoin [i_item_id,i_item_id]
+ ColumnarToRow
+ InputAdapter
+ CometProject [ws_sales_price,ca_city,ca_zip,i_item_id]
+ CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ca_city,ca_zip,i_item_sk,i_item_id]
+ CometProject [ws_item_sk,ws_sales_price,ca_city,ca_zip]
+ CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ws_sold_date_sk,ca_city,ca_zip,d_date_sk]
+ CometProject [ws_item_sk,ws_sales_price,ws_sold_date_sk,ca_city,ca_zip]
+ CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ws_sold_date_sk,c_current_addr_sk,ca_address_sk,ca_city,ca_zip]
+ CometProject [ws_item_sk,ws_sales_price,ws_sold_date_sk,c_current_addr_sk]
+ CometBroadcastHashJoin [ws_item_sk,ws_bill_customer_sk,ws_sales_price,ws_sold_date_sk,c_customer_sk,c_current_addr_sk]
+ CometFilter [ws_item_sk,ws_bill_customer_sk,ws_sales_price,ws_sold_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sales_price,ws_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #2
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_qoy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
+ CometBroadcastExchange [c_customer_sk,c_current_addr_sk] #3
+ CometFilter [c_customer_sk,c_current_addr_sk]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
+ CometBroadcastExchange [ca_address_sk,ca_city,ca_zip] #4
+ CometFilter [ca_address_sk,ca_city,ca_zip]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_city,ca_zip]
+ CometBroadcastExchange [d_date_sk] #5
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_qoy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_qoy]
+ CometBroadcastExchange [i_item_sk,i_item_id] #6
+ CometFilter [i_item_sk,i_item_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
+ InputAdapter
+ BroadcastExchange #7
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [i_item_id]
+ CometFilter [i_item_sk,i_item_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/explain.txt
index b8193152..75334c74 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/explain.txt
@@ -6,7 +6,7 @@
:- CometProject (33)
: +- CometBroadcastHashJoin (32)
: :- CometHashAggregate (28)
- : : +- CometColumnarExchange (27)
+ : : +- CometExchange (27)
: : +- CometHashAggregate (26)
: : +- CometProject (25)
: : +- CometBroadcastHashJoin (24)
@@ -164,9 +164,9 @@ Input [6]: [ss_customer_sk#1, ss_addr_sk#3, ss_ticket_number#5, ss_coupon_amt#6,
Keys [4]: [ss_ticket_number#5, ss_customer_sk#1, ss_addr_sk#3, ca_city#19]
Functions [2]: [partial_sum(UnscaledValue(ss_coupon_amt#6)), partial_sum(UnscaledValue(ss_net_profit#7))]
-(27) CometColumnarExchange
+(27) CometExchange
Input [6]: [ss_ticket_number#5, ss_customer_sk#1, ss_addr_sk#3, ca_city#19, sum#20, sum#21]
-Arguments: hashpartitioning(ss_ticket_number#5, ss_customer_sk#1, ss_addr_sk#3, ca_city#19, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_ticket_number#5, ss_customer_sk#1, ss_addr_sk#3, ca_city#19, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(28) CometHashAggregate
Input [6]: [ss_ticket_number#5, ss_customer_sk#1, ss_addr_sk#3, ca_city#19, sum#20, sum#21]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/simplified.txt
index 1d28a8ed..e1b53bf4 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q46/simplified.txt
@@ -7,7 +7,7 @@ WholeStageCodegen (1)
CometProject [ss_ticket_number,bought_city,amt,profit,c_current_addr_sk,c_first_name,c_last_name]
CometBroadcastHashJoin [ss_ticket_number,ss_customer_sk,bought_city,amt,profit,c_customer_sk,c_current_addr_sk,c_first_name,c_last_name]
CometHashAggregate [ss_ticket_number,ss_customer_sk,bought_city,amt,profit,ss_addr_sk,ca_city,sum,sum,sum(UnscaledValue(ss_coupon_amt)),sum(UnscaledValue(ss_net_profit))]
- CometColumnarExchange [ss_ticket_number,ss_customer_sk,ss_addr_sk,ca_city] #1
+ CometExchange [ss_ticket_number,ss_customer_sk,ss_addr_sk,ca_city] #1
CometHashAggregate [ss_ticket_number,ss_customer_sk,ss_addr_sk,ca_city,sum,sum,ss_coupon_amt,ss_net_profit]
CometProject [ss_customer_sk,ss_addr_sk,ss_ticket_number,ss_coupon_amt,ss_net_profit,ca_city]
CometBroadcastHashJoin [ss_customer_sk,ss_addr_sk,ss_ticket_number,ss_coupon_amt,ss_net_profit,ca_address_sk,ca_city]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/explain.txt
index 3fe3003c..8221d32a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/explain.txt
@@ -11,9 +11,9 @@ TakeOrderedAndProject (45)
: : +- Window (24)
: : +- * ColumnarToRow (23)
: : +- CometSort (22)
- : : +- CometColumnarExchange (21)
+ : : +- CometExchange (21)
: : +- CometHashAggregate (20)
- : : +- CometColumnarExchange (19)
+ : : +- CometExchange (19)
: : +- CometHashAggregate (18)
: : +- CometProject (17)
: : +- CometBroadcastHashJoin (16)
@@ -135,18 +135,18 @@ Input [7]: [i_brand#2, i_category#3, ss_sales_price#6, d_year#10, d_moy#11, s_st
Keys [6]: [i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11]
Functions [1]: [partial_sum(UnscaledValue(ss_sales_price#6))]
-(19) CometColumnarExchange
+(19) CometExchange
Input [7]: [i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11, sum#15]
-Arguments: hashpartitioning(i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(20) CometHashAggregate
Input [7]: [i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11, sum#15]
Keys [6]: [i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11]
Functions [1]: [sum(UnscaledValue(ss_sales_price#6))]
-(21) CometColumnarExchange
+(21) CometExchange
Input [8]: [i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11, sum_sales#16, _w0#17]
-Arguments: hashpartitioning(i_category#3, i_brand#2, s_store_name#13, s_company_name#14, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_category#3, i_brand#2, s_store_name#13, s_company_name#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(22) CometSort
Input [8]: [i_category#3, i_brand#2, s_store_name#13, s_company_name#14, d_year#10, d_moy#11, sum_sales#16, _w0#17]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/simplified.txt
index 91c2cd65..2c8192f4 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q47/simplified.txt
@@ -16,9 +16,9 @@ TakeOrderedAndProject [sum_sales,avg_monthly_sales,s_store_name,i_category,i_bra
ColumnarToRow
InputAdapter
CometSort [i_category,i_brand,s_store_name,s_company_name,d_year,d_moy,sum_sales,_w0]
- CometColumnarExchange [i_category,i_brand,s_store_name,s_company_name] #1
+ CometExchange [i_category,i_brand,s_store_name,s_company_name] #1
CometHashAggregate [i_category,i_brand,s_store_name,s_company_name,d_year,d_moy,sum_sales,_w0,sum,sum(UnscaledValue(ss_sales_price))]
- CometColumnarExchange [i_category,i_brand,s_store_name,s_company_name,d_year,d_moy] #2
+ CometExchange [i_category,i_brand,s_store_name,s_company_name,d_year,d_moy] #2
CometHashAggregate [i_category,i_brand,s_store_name,s_company_name,d_year,d_moy,sum,ss_sales_price]
CometProject [i_brand,i_category,ss_sales_price,d_year,d_moy,s_store_name,s_company_name]
CometBroadcastHashJoin [i_brand,i_category,ss_store_sk,ss_sales_price,d_year,d_moy,s_store_sk,s_store_name,s_company_name]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/explain.txt
index 3393afed..40181264 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/explain.txt
@@ -1,7 +1,7 @@
== Physical Plan ==
* ColumnarToRow (28)
+- CometHashAggregate (27)
- +- CometColumnarExchange (26)
+ +- CometExchange (26)
+- CometHashAggregate (25)
+- CometProject (24)
+- CometBroadcastHashJoin (23)
@@ -150,9 +150,9 @@ Input [1]: [ss_quantity#4]
Keys: []
Functions [1]: [partial_sum(ss_quantity#4)]
-(26) CometColumnarExchange
+(26) CometExchange
Input [1]: [sum#18]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(27) CometHashAggregate
Input [1]: [sum#18]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/simplified.txt
index 8bb0cede..a46d8d60 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q48/simplified.txt
@@ -2,7 +2,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [sum(ss_quantity),sum,sum(ss_quantity)]
- CometColumnarExchange #1
+ CometExchange #1
CometHashAggregate [sum,ss_quantity]
CometProject [ss_quantity]
CometBroadcastHashJoin [ss_quantity,ss_sold_date_sk,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/explain.txt
index 4b0d5aed..f7e750d4 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/explain.txt
@@ -1,83 +1,81 @@
== Physical Plan ==
-* ColumnarToRow (79)
-+- CometTakeOrderedAndProject (78)
- +- CometHashAggregate (77)
- +- CometColumnarExchange (76)
- +- RowToColumnar (75)
- +- * HashAggregate (74)
- +- Union (73)
- :- * Project (26)
- : +- * Filter (25)
- : +- Window (24)
- : +- * Sort (23)
- : +- Window (22)
- : +- * ColumnarToRow (21)
- : +- CometSort (20)
- : +- CometColumnarExchange (19)
- : +- CometHashAggregate (18)
- : +- CometColumnarExchange (17)
- : +- CometHashAggregate (16)
- : +- CometProject (15)
- : +- CometBroadcastHashJoin (14)
- : :- CometProject (9)
- : : +- CometBroadcastHashJoin (8)
- : : :- CometBroadcastExchange (4)
- : : : +- CometProject (3)
- : : : +- CometFilter (2)
- : : : +- CometScan parquet spark_catalog.default.web_sales (1)
- : : +- CometProject (7)
- : : +- CometFilter (6)
- : : +- CometScan parquet spark_catalog.default.web_returns (5)
- : +- CometBroadcastExchange (13)
- : +- CometProject (12)
- : +- CometFilter (11)
- : +- CometScan parquet spark_catalog.default.date_dim (10)
- :- * Project (49)
- : +- * Filter (48)
- : +- Window (47)
- : +- * Sort (46)
- : +- Window (45)
- : +- * ColumnarToRow (44)
- : +- CometSort (43)
- : +- CometColumnarExchange (42)
- : +- CometHashAggregate (41)
- : +- CometColumnarExchange (40)
- : +- CometHashAggregate (39)
- : +- CometProject (38)
- : +- CometBroadcastHashJoin (37)
- : :- CometProject (35)
- : : +- CometBroadcastHashJoin (34)
- : : :- CometBroadcastExchange (30)
- : : : +- CometProject (29)
- : : : +- CometFilter (28)
- : : : +- CometScan parquet spark_catalog.default.catalog_sales (27)
- : : +- CometProject (33)
- : : +- CometFilter (32)
- : : +- CometScan parquet spark_catalog.default.catalog_returns (31)
- : +- ReusedExchange (36)
- +- * Project (72)
- +- * Filter (71)
- +- Window (70)
- +- * Sort (69)
- +- Window (68)
- +- * ColumnarToRow (67)
- +- CometSort (66)
- +- CometColumnarExchange (65)
- +- CometHashAggregate (64)
- +- CometColumnarExchange (63)
- +- CometHashAggregate (62)
- +- CometProject (61)
- +- CometBroadcastHashJoin (60)
- :- CometProject (58)
- : +- CometBroadcastHashJoin (57)
- : :- CometBroadcastExchange (53)
- : : +- CometProject (52)
- : : +- CometFilter (51)
- : : +- CometScan parquet spark_catalog.default.store_sales (50)
- : +- CometProject (56)
- : +- CometFilter (55)
- : +- CometScan parquet spark_catalog.default.store_returns (54)
- +- ReusedExchange (59)
+TakeOrderedAndProject (77)
++- * HashAggregate (76)
+ +- Exchange (75)
+ +- * HashAggregate (74)
+ +- Union (73)
+ :- * Project (26)
+ : +- * Filter (25)
+ : +- Window (24)
+ : +- * Sort (23)
+ : +- Window (22)
+ : +- * ColumnarToRow (21)
+ : +- CometSort (20)
+ : +- CometExchange (19)
+ : +- CometHashAggregate (18)
+ : +- CometExchange (17)
+ : +- CometHashAggregate (16)
+ : +- CometProject (15)
+ : +- CometBroadcastHashJoin (14)
+ : :- CometProject (9)
+ : : +- CometBroadcastHashJoin (8)
+ : : :- CometBroadcastExchange (4)
+ : : : +- CometProject (3)
+ : : : +- CometFilter (2)
+ : : : +- CometScan parquet spark_catalog.default.web_sales (1)
+ : : +- CometProject (7)
+ : : +- CometFilter (6)
+ : : +- CometScan parquet spark_catalog.default.web_returns (5)
+ : +- CometBroadcastExchange (13)
+ : +- CometProject (12)
+ : +- CometFilter (11)
+ : +- CometScan parquet spark_catalog.default.date_dim (10)
+ :- * Project (49)
+ : +- * Filter (48)
+ : +- Window (47)
+ : +- * Sort (46)
+ : +- Window (45)
+ : +- * ColumnarToRow (44)
+ : +- CometSort (43)
+ : +- CometExchange (42)
+ : +- CometHashAggregate (41)
+ : +- CometExchange (40)
+ : +- CometHashAggregate (39)
+ : +- CometProject (38)
+ : +- CometBroadcastHashJoin (37)
+ : :- CometProject (35)
+ : : +- CometBroadcastHashJoin (34)
+ : : :- CometBroadcastExchange (30)
+ : : : +- CometProject (29)
+ : : : +- CometFilter (28)
+ : : : +- CometScan parquet spark_catalog.default.catalog_sales (27)
+ : : +- CometProject (33)
+ : : +- CometFilter (32)
+ : : +- CometScan parquet spark_catalog.default.catalog_returns (31)
+ : +- ReusedExchange (36)
+ +- * Project (72)
+ +- * Filter (71)
+ +- Window (70)
+ +- * Sort (69)
+ +- Window (68)
+ +- * ColumnarToRow (67)
+ +- CometSort (66)
+ +- CometExchange (65)
+ +- CometHashAggregate (64)
+ +- CometExchange (63)
+ +- CometHashAggregate (62)
+ +- CometProject (61)
+ +- CometBroadcastHashJoin (60)
+ :- CometProject (58)
+ : +- CometBroadcastHashJoin (57)
+ : :- CometBroadcastExchange (53)
+ : : +- CometProject (52)
+ : : +- CometFilter (51)
+ : : +- CometScan parquet spark_catalog.default.store_sales (50)
+ : +- CometProject (56)
+ : +- CometFilter (55)
+ : +- CometScan parquet spark_catalog.default.store_returns (54)
+ +- ReusedExchange (59)
(1) Scan parquet spark_catalog.default.web_sales
@@ -157,18 +155,18 @@ Input [5]: [ws_item_sk#1, ws_quantity#3, ws_net_paid#4, wr_return_quantity#10, w
Keys [1]: [ws_item_sk#1]
Functions [4]: [partial_sum(coalesce(wr_return_quantity#10, 0)), partial_sum(coalesce(ws_quantity#3, 0)), partial_sum(coalesce(cast(wr_return_amt#11 as decimal(12,2)), 0.00)), partial_sum(coalesce(cast(ws_net_paid#4 as decimal(12,2)), 0.00))]
-(17) CometColumnarExchange
+(17) CometExchange
Input [7]: [ws_item_sk#1, sum#16, sum#17, sum#18, isEmpty#19, sum#20, isEmpty#21]
-Arguments: hashpartitioning(ws_item_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ws_item_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(18) CometHashAggregate
Input [7]: [ws_item_sk#1, sum#16, sum#17, sum#18, isEmpty#19, sum#20, isEmpty#21]
Keys [1]: [ws_item_sk#1]
Functions [4]: [sum(coalesce(wr_return_quantity#10, 0)), sum(coalesce(ws_quantity#3, 0)), sum(coalesce(cast(wr_return_amt#11 as decimal(12,2)), 0.00)), sum(coalesce(cast(ws_net_paid#4 as decimal(12,2)), 0.00))]
-(19) CometColumnarExchange
+(19) CometExchange
Input [3]: [item#22, return_ratio#23, currency_ratio#24]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(20) CometSort
Input [3]: [item#22, return_ratio#23, currency_ratio#24]
@@ -258,18 +256,18 @@ Input [5]: [cs_item_sk#28, cs_quantity#30, cs_net_paid#31, cr_return_quantity#37
Keys [1]: [cs_item_sk#28]
Functions [4]: [partial_sum(coalesce(cr_return_quantity#37, 0)), partial_sum(coalesce(cs_quantity#30, 0)), partial_sum(coalesce(cast(cr_return_amount#38 as decimal(12,2)), 0.00)), partial_sum(coalesce(cast(cs_net_paid#31 as decimal(12,2)), 0.00))]
-(40) CometColumnarExchange
+(40) CometExchange
Input [7]: [cs_item_sk#28, sum#41, sum#42, sum#43, isEmpty#44, sum#45, isEmpty#46]
-Arguments: hashpartitioning(cs_item_sk#28, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(cs_item_sk#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(41) CometHashAggregate
Input [7]: [cs_item_sk#28, sum#41, sum#42, sum#43, isEmpty#44, sum#45, isEmpty#46]
Keys [1]: [cs_item_sk#28]
Functions [4]: [sum(coalesce(cr_return_quantity#37, 0)), sum(coalesce(cs_quantity#30, 0)), sum(coalesce(cast(cr_return_amount#38 as decimal(12,2)), 0.00)), sum(coalesce(cast(cs_net_paid#31 as decimal(12,2)), 0.00))]
-(42) CometColumnarExchange
+(42) CometExchange
Input [3]: [item#47, return_ratio#48, currency_ratio#49]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(43) CometSort
Input [3]: [item#47, return_ratio#48, currency_ratio#49]
@@ -359,18 +357,18 @@ Input [5]: [ss_item_sk#53, ss_quantity#55, ss_net_paid#56, sr_return_quantity#62
Keys [1]: [ss_item_sk#53]
Functions [4]: [partial_sum(coalesce(sr_return_quantity#62, 0)), partial_sum(coalesce(ss_quantity#55, 0)), partial_sum(coalesce(cast(sr_return_amt#63 as decimal(12,2)), 0.00)), partial_sum(coalesce(cast(ss_net_paid#56 as decimal(12,2)), 0.00))]
-(63) CometColumnarExchange
+(63) CometExchange
Input [7]: [ss_item_sk#53, sum#66, sum#67, sum#68, isEmpty#69, sum#70, isEmpty#71]
-Arguments: hashpartitioning(ss_item_sk#53, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(ss_item_sk#53, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(64) CometHashAggregate
Input [7]: [ss_item_sk#53, sum#66, sum#67, sum#68, isEmpty#69, sum#70, isEmpty#71]
Keys [1]: [ss_item_sk#53]
Functions [4]: [sum(coalesce(sr_return_quantity#62, 0)), sum(coalesce(ss_quantity#55, 0)), sum(coalesce(cast(sr_return_amt#63 as decimal(12,2)), 0.00)), sum(coalesce(cast(ss_net_paid#56 as decimal(12,2)), 0.00))]
-(65) CometColumnarExchange
+(65) CometExchange
Input [3]: [item#72, return_ratio#73, currency_ratio#74]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
(66) CometSort
Input [3]: [item#72, return_ratio#73, currency_ratio#74]
@@ -408,54 +406,50 @@ Functions: []
Aggregate Attributes: []
Results [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
-(75) RowToColumnar
+(75) Exchange
Input [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
+Arguments: hashpartitioning(channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26, 5), ENSURE_REQUIREMENTS, [plan_id=7]
-(76) CometColumnarExchange
-Input [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
-Arguments: hashpartitioning(channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
-
-(77) CometHashAggregate
+(76) HashAggregate [codegen id : 11]
Input [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
Keys [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
Functions: []
+Aggregate Attributes: []
+Results [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
-(78) CometTakeOrderedAndProject
-Input [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[channel#27 ASC NULLS FIRST,return_rank#25 ASC NULLS FIRST,currency_rank#26 ASC NULLS FIRST], output=[channel#27,item#22,return_ratio#23,return_rank#25,currency_rank#26]), [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26], 100, [channel#27 ASC NULLS FIRST, return_rank#25 ASC NULLS FIRST, currency_rank#26 ASC NULLS FIRST], [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
-
-(79) ColumnarToRow [codegen id : 11]
+(77) TakeOrderedAndProject
Input [5]: [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
+Arguments: 100, [channel#27 ASC NULLS FIRST, return_rank#25 ASC NULLS FIRST, currency_rank#26 ASC NULLS FIRST], [channel#27, item#22, return_ratio#23, return_rank#25, currency_rank#26]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ws_sold_date_sk#6 IN dynamicpruning#7
-BroadcastExchange (84)
-+- * ColumnarToRow (83)
- +- CometProject (82)
- +- CometFilter (81)
- +- CometScan parquet spark_catalog.default.date_dim (80)
+BroadcastExchange (82)
++- * ColumnarToRow (81)
+ +- CometProject (80)
+ +- CometFilter (79)
+ +- CometScan parquet spark_catalog.default.date_dim (78)
-(80) Scan parquet spark_catalog.default.date_dim
+(78) Scan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#13, d_year#14, d_moy#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_moy), EqualTo(d_year,2001), EqualTo(d_moy,12), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(81) CometFilter
+(79) CometFilter
Input [3]: [d_date_sk#13, d_year#14, d_moy#15]
Condition : ((((isnotnull(d_year#14) AND isnotnull(d_moy#15)) AND (d_year#14 = 2001)) AND (d_moy#15 = 12)) AND isnotnull(d_date_sk#13))
-(82) CometProject
+(80) CometProject
Input [3]: [d_date_sk#13, d_year#14, d_moy#15]
Arguments: [d_date_sk#13], [d_date_sk#13]
-(83) ColumnarToRow [codegen id : 1]
+(81) ColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#13]
-(84) BroadcastExchange
+(82) BroadcastExchange
Input [1]: [d_date_sk#13]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=8]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/simplified.txt
index 0d02a9c3..d98b2b0a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q49/simplified.txt
@@ -1,111 +1,109 @@
-WholeStageCodegen (11)
- ColumnarToRow
- InputAdapter
- CometTakeOrderedAndProject [channel,item,return_ratio,return_rank,currency_rank]
- CometHashAggregate [channel,item,return_ratio,return_rank,currency_rank]
- CometColumnarExchange [channel,item,return_ratio,return_rank,currency_rank] #1
- RowToColumnar
- WholeStageCodegen (10)
- HashAggregate [channel,item,return_ratio,return_rank,currency_rank]
- InputAdapter
- Union
- WholeStageCodegen (3)
- Project [item,return_ratio,return_rank,currency_rank]
- Filter [return_rank,currency_rank]
- InputAdapter
- Window [currency_ratio]
- WholeStageCodegen (2)
- Sort [currency_ratio]
- InputAdapter
- Window [return_ratio]
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometSort [item,return_ratio,currency_ratio]
- CometColumnarExchange #2
- CometHashAggregate [item,return_ratio,currency_ratio,ws_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sum(coalesce(wr_return_quantity, 0)),sum(coalesce(ws_quantity, 0)),sum(coalesce(cast(wr_return_amt as decimal(12,2)), 0.00)),sum(coalesce(cast(ws_net_paid as decimal(12,2)), 0.00))]
- CometColumnarExchange [ws_item_sk] #3
- CometHashAggregate [ws_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,wr_return_quantity,ws_quantity,wr_return_amt,ws_net_paid]
- CometProject [ws_item_sk,ws_quantity,ws_net_paid,wr_return_quantity,wr_return_amt]
- CometBroadcastHashJoin [ws_item_sk,ws_quantity,ws_net_paid,ws_sold_date_sk,wr_return_quantity,wr_return_amt,d_date_sk]
- CometProject [ws_item_sk,ws_quantity,ws_net_paid,ws_sold_date_sk,wr_return_quantity,wr_return_amt]
- CometBroadcastHashJoin [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_sold_date_sk,wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt]
- CometBroadcastExchange [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_sold_date_sk] #4
- CometProject [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_sold_date_sk]
- CometFilter [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_net_profit,ws_sold_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_net_profit,ws_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #5
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_moy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
- CometProject [wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt]
- CometFilter [wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt,wr_returned_date_sk]
- CometScan parquet spark_catalog.default.web_returns [wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt,wr_returned_date_sk]
- CometBroadcastExchange [d_date_sk] #6
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_year,d_moy]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
- WholeStageCodegen (6)
- Project [item,return_ratio,return_rank,currency_rank]
- Filter [return_rank,currency_rank]
- InputAdapter
- Window [currency_ratio]
- WholeStageCodegen (5)
- Sort [currency_ratio]
- InputAdapter
- Window [return_ratio]
- WholeStageCodegen (4)
- ColumnarToRow
- InputAdapter
- CometSort [item,return_ratio,currency_ratio]
- CometColumnarExchange #7
- CometHashAggregate [item,return_ratio,currency_ratio,cs_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sum(coalesce(cr_return_quantity, 0)),sum(coalesce(cs_quantity, 0)),sum(coalesce(cast(cr_return_amount as decimal(12,2)), 0.00)),sum(coalesce(cast(cs_net_paid as decimal(12,2)), 0.00))]
- CometColumnarExchange [cs_item_sk] #8
- CometHashAggregate [cs_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,cr_return_quantity,cs_quantity,cr_return_amount,cs_net_paid]
- CometProject [cs_item_sk,cs_quantity,cs_net_paid,cr_return_quantity,cr_return_amount]
- CometBroadcastHashJoin [cs_item_sk,cs_quantity,cs_net_paid,cs_sold_date_sk,cr_return_quantity,cr_return_amount,d_date_sk]
- CometProject [cs_item_sk,cs_quantity,cs_net_paid,cs_sold_date_sk,cr_return_quantity,cr_return_amount]
- CometBroadcastHashJoin [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_sold_date_sk,cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount]
- CometBroadcastExchange [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_sold_date_sk] #9
- CometProject [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_sold_date_sk]
- CometFilter [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_net_profit,cs_sold_date_sk]
- CometScan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_net_profit,cs_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- CometProject [cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount]
- CometFilter [cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount,cr_returned_date_sk]
- CometScan parquet spark_catalog.default.catalog_returns [cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount,cr_returned_date_sk]
- ReusedExchange [d_date_sk] #6
- WholeStageCodegen (9)
- Project [item,return_ratio,return_rank,currency_rank]
- Filter [return_rank,currency_rank]
- InputAdapter
- Window [currency_ratio]
- WholeStageCodegen (8)
- Sort [currency_ratio]
- InputAdapter
- Window [return_ratio]
- WholeStageCodegen (7)
- ColumnarToRow
- InputAdapter
- CometSort [item,return_ratio,currency_ratio]
- CometColumnarExchange #10
- CometHashAggregate [item,return_ratio,currency_ratio,ss_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sum(coalesce(sr_return_quantity, 0)),sum(coalesce(ss_quantity, 0)),sum(coalesce(cast(sr_return_amt as decimal(12,2)), 0.00)),sum(coalesce(cast(ss_net_paid as decimal(12,2)), 0.00))]
- CometColumnarExchange [ss_item_sk] #11
- CometHashAggregate [ss_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sr_return_quantity,ss_quantity,sr_return_amt,ss_net_paid]
- CometProject [ss_item_sk,ss_quantity,ss_net_paid,sr_return_quantity,sr_return_amt]
- CometBroadcastHashJoin [ss_item_sk,ss_quantity,ss_net_paid,ss_sold_date_sk,sr_return_quantity,sr_return_amt,d_date_sk]
- CometProject [ss_item_sk,ss_quantity,ss_net_paid,ss_sold_date_sk,sr_return_quantity,sr_return_amt]
- CometBroadcastHashJoin [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_sold_date_sk,sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt]
- CometBroadcastExchange [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_sold_date_sk] #12
- CometProject [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_sold_date_sk]
- CometFilter [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_net_profit,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_net_profit,ss_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- CometProject [sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt]
- CometFilter [sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt,sr_returned_date_sk]
- CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt,sr_returned_date_sk]
- ReusedExchange [d_date_sk] #6
+TakeOrderedAndProject [channel,return_rank,currency_rank,item,return_ratio]
+ WholeStageCodegen (11)
+ HashAggregate [channel,item,return_ratio,return_rank,currency_rank]
+ InputAdapter
+ Exchange [channel,item,return_ratio,return_rank,currency_rank] #1
+ WholeStageCodegen (10)
+ HashAggregate [channel,item,return_ratio,return_rank,currency_rank]
+ InputAdapter
+ Union
+ WholeStageCodegen (3)
+ Project [item,return_ratio,return_rank,currency_rank]
+ Filter [return_rank,currency_rank]
+ InputAdapter
+ Window [currency_ratio]
+ WholeStageCodegen (2)
+ Sort [currency_ratio]
+ InputAdapter
+ Window [return_ratio]
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item,return_ratio,currency_ratio]
+ CometExchange #2
+ CometHashAggregate [item,return_ratio,currency_ratio,ws_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sum(coalesce(wr_return_quantity, 0)),sum(coalesce(ws_quantity, 0)),sum(coalesce(cast(wr_return_amt as decimal(12,2)), 0.00)),sum(coalesce(cast(ws_net_paid as decimal(12,2)), 0.00))]
+ CometExchange [ws_item_sk] #3
+ CometHashAggregate [ws_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,wr_return_quantity,ws_quantity,wr_return_amt,ws_net_paid]
+ CometProject [ws_item_sk,ws_quantity,ws_net_paid,wr_return_quantity,wr_return_amt]
+ CometBroadcastHashJoin [ws_item_sk,ws_quantity,ws_net_paid,ws_sold_date_sk,wr_return_quantity,wr_return_amt,d_date_sk]
+ CometProject [ws_item_sk,ws_quantity,ws_net_paid,ws_sold_date_sk,wr_return_quantity,wr_return_amt]
+ CometBroadcastHashJoin [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_sold_date_sk,wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt]
+ CometBroadcastExchange [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_sold_date_sk] #4
+ CometProject [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_sold_date_sk]
+ CometFilter [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_net_profit,ws_sold_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_order_number,ws_quantity,ws_net_paid,ws_net_profit,ws_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #5
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_moy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
+ CometProject [wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt]
+ CometFilter [wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt,wr_returned_date_sk]
+ CometScan parquet spark_catalog.default.web_returns [wr_item_sk,wr_order_number,wr_return_quantity,wr_return_amt,wr_returned_date_sk]
+ CometBroadcastExchange [d_date_sk] #6
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_year,d_moy]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
+ WholeStageCodegen (6)
+ Project [item,return_ratio,return_rank,currency_rank]
+ Filter [return_rank,currency_rank]
+ InputAdapter
+ Window [currency_ratio]
+ WholeStageCodegen (5)
+ Sort [currency_ratio]
+ InputAdapter
+ Window [return_ratio]
+ WholeStageCodegen (4)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item,return_ratio,currency_ratio]
+ CometExchange #7
+ CometHashAggregate [item,return_ratio,currency_ratio,cs_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sum(coalesce(cr_return_quantity, 0)),sum(coalesce(cs_quantity, 0)),sum(coalesce(cast(cr_return_amount as decimal(12,2)), 0.00)),sum(coalesce(cast(cs_net_paid as decimal(12,2)), 0.00))]
+ CometExchange [cs_item_sk] #8
+ CometHashAggregate [cs_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,cr_return_quantity,cs_quantity,cr_return_amount,cs_net_paid]
+ CometProject [cs_item_sk,cs_quantity,cs_net_paid,cr_return_quantity,cr_return_amount]
+ CometBroadcastHashJoin [cs_item_sk,cs_quantity,cs_net_paid,cs_sold_date_sk,cr_return_quantity,cr_return_amount,d_date_sk]
+ CometProject [cs_item_sk,cs_quantity,cs_net_paid,cs_sold_date_sk,cr_return_quantity,cr_return_amount]
+ CometBroadcastHashJoin [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_sold_date_sk,cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount]
+ CometBroadcastExchange [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_sold_date_sk] #9
+ CometProject [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_sold_date_sk]
+ CometFilter [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_net_profit,cs_sold_date_sk]
+ CometScan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_order_number,cs_quantity,cs_net_paid,cs_net_profit,cs_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ CometProject [cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount]
+ CometFilter [cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount,cr_returned_date_sk]
+ CometScan parquet spark_catalog.default.catalog_returns [cr_item_sk,cr_order_number,cr_return_quantity,cr_return_amount,cr_returned_date_sk]
+ ReusedExchange [d_date_sk] #6
+ WholeStageCodegen (9)
+ Project [item,return_ratio,return_rank,currency_rank]
+ Filter [return_rank,currency_rank]
+ InputAdapter
+ Window [currency_ratio]
+ WholeStageCodegen (8)
+ Sort [currency_ratio]
+ InputAdapter
+ Window [return_ratio]
+ WholeStageCodegen (7)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item,return_ratio,currency_ratio]
+ CometExchange #10
+ CometHashAggregate [item,return_ratio,currency_ratio,ss_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sum(coalesce(sr_return_quantity, 0)),sum(coalesce(ss_quantity, 0)),sum(coalesce(cast(sr_return_amt as decimal(12,2)), 0.00)),sum(coalesce(cast(ss_net_paid as decimal(12,2)), 0.00))]
+ CometExchange [ss_item_sk] #11
+ CometHashAggregate [ss_item_sk,sum,sum,sum,isEmpty,sum,isEmpty,sr_return_quantity,ss_quantity,sr_return_amt,ss_net_paid]
+ CometProject [ss_item_sk,ss_quantity,ss_net_paid,sr_return_quantity,sr_return_amt]
+ CometBroadcastHashJoin [ss_item_sk,ss_quantity,ss_net_paid,ss_sold_date_sk,sr_return_quantity,sr_return_amt,d_date_sk]
+ CometProject [ss_item_sk,ss_quantity,ss_net_paid,ss_sold_date_sk,sr_return_quantity,sr_return_amt]
+ CometBroadcastHashJoin [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_sold_date_sk,sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt]
+ CometBroadcastExchange [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_sold_date_sk] #12
+ CometProject [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_sold_date_sk]
+ CometFilter [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_net_profit,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_ticket_number,ss_quantity,ss_net_paid,ss_net_profit,ss_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ CometProject [sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt]
+ CometFilter [sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt,sr_returned_date_sk]
+ CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_return_quantity,sr_return_amt,sr_returned_date_sk]
+ ReusedExchange [d_date_sk] #6
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/explain.txt
index ed1f8fed..82b5c669 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/explain.txt
@@ -1,76 +1,74 @@
== Physical Plan ==
-TakeOrderedAndProject (72)
-+- * HashAggregate (71)
- +- * ColumnarToRow (70)
- +- CometColumnarExchange (69)
- +- RowToColumnar (68)
- +- * HashAggregate (67)
- +- * Expand (66)
- +- Union (65)
- :- * HashAggregate (22)
- : +- * ColumnarToRow (21)
- : +- CometColumnarExchange (20)
- : +- CometHashAggregate (19)
- : +- CometProject (18)
- : +- CometBroadcastHashJoin (17)
- : :- CometProject (13)
- : : +- CometBroadcastHashJoin (12)
- : : :- CometUnion (7)
- : : : :- CometProject (3)
- : : : : +- CometFilter (2)
- : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
- : : : +- CometProject (6)
- : : : +- CometFilter (5)
- : : : +- CometScan parquet spark_catalog.default.store_returns (4)
- : : +- CometBroadcastExchange (11)
- : : +- CometProject (10)
- : : +- CometFilter (9)
- : : +- CometScan parquet spark_catalog.default.date_dim (8)
- : +- CometBroadcastExchange (16)
- : +- CometFilter (15)
- : +- CometScan parquet spark_catalog.default.store (14)
- :- * HashAggregate (41)
- : +- * ColumnarToRow (40)
- : +- CometColumnarExchange (39)
- : +- CometHashAggregate (38)
- : +- CometProject (37)
- : +- CometBroadcastHashJoin (36)
- : :- CometProject (32)
- : : +- CometBroadcastHashJoin (31)
- : : :- CometUnion (29)
- : : : :- CometProject (25)
- : : : : +- CometFilter (24)
- : : : : +- CometScan parquet spark_catalog.default.catalog_sales (23)
- : : : +- CometProject (28)
- : : : +- CometFilter (27)
- : : : +- CometScan parquet spark_catalog.default.catalog_returns (26)
- : : +- ReusedExchange (30)
- : +- CometBroadcastExchange (35)
- : +- CometFilter (34)
- : +- CometScan parquet spark_catalog.default.catalog_page (33)
- +- * HashAggregate (64)
- +- * ColumnarToRow (63)
- +- CometColumnarExchange (62)
- +- CometHashAggregate (61)
- +- CometProject (60)
- +- CometBroadcastHashJoin (59)
- :- CometProject (55)
- : +- CometBroadcastHashJoin (54)
- : :- CometUnion (52)
- : : :- CometProject (44)
- : : : +- CometFilter (43)
- : : : +- CometScan parquet spark_catalog.default.web_sales (42)
- : : +- CometProject (51)
- : : +- CometBroadcastHashJoin (50)
- : : :- CometBroadcastExchange (46)
- : : : +- CometScan parquet spark_catalog.default.web_returns (45)
- : : +- CometProject (49)
- : : +- CometFilter (48)
- : : +- CometScan parquet spark_catalog.default.web_sales (47)
- : +- ReusedExchange (53)
- +- CometBroadcastExchange (58)
- +- CometFilter (57)
- +- CometScan parquet spark_catalog.default.web_site (56)
+TakeOrderedAndProject (70)
++- * HashAggregate (69)
+ +- Exchange (68)
+ +- * HashAggregate (67)
+ +- * Expand (66)
+ +- Union (65)
+ :- * HashAggregate (22)
+ : +- * ColumnarToRow (21)
+ : +- CometExchange (20)
+ : +- CometHashAggregate (19)
+ : +- CometProject (18)
+ : +- CometBroadcastHashJoin (17)
+ : :- CometProject (13)
+ : : +- CometBroadcastHashJoin (12)
+ : : :- CometUnion (7)
+ : : : :- CometProject (3)
+ : : : : +- CometFilter (2)
+ : : : : +- CometScan parquet spark_catalog.default.store_sales (1)
+ : : : +- CometProject (6)
+ : : : +- CometFilter (5)
+ : : : +- CometScan parquet spark_catalog.default.store_returns (4)
+ : : +- CometBroadcastExchange (11)
+ : : +- CometProject (10)
+ : : +- CometFilter (9)
+ : : +- CometScan parquet spark_catalog.default.date_dim (8)
+ : +- CometBroadcastExchange (16)
+ : +- CometFilter (15)
+ : +- CometScan parquet spark_catalog.default.store (14)
+ :- * HashAggregate (41)
+ : +- * ColumnarToRow (40)
+ : +- CometExchange (39)
+ : +- CometHashAggregate (38)
+ : +- CometProject (37)
+ : +- CometBroadcastHashJoin (36)
+ : :- CometProject (32)
+ : : +- CometBroadcastHashJoin (31)
+ : : :- CometUnion (29)
+ : : : :- CometProject (25)
+ : : : : +- CometFilter (24)
+ : : : : +- CometScan parquet spark_catalog.default.catalog_sales (23)
+ : : : +- CometProject (28)
+ : : : +- CometFilter (27)
+ : : : +- CometScan parquet spark_catalog.default.catalog_returns (26)
+ : : +- ReusedExchange (30)
+ : +- CometBroadcastExchange (35)
+ : +- CometFilter (34)
+ : +- CometScan parquet spark_catalog.default.catalog_page (33)
+ +- * HashAggregate (64)
+ +- * ColumnarToRow (63)
+ +- CometExchange (62)
+ +- CometHashAggregate (61)
+ +- CometProject (60)
+ +- CometBroadcastHashJoin (59)
+ :- CometProject (55)
+ : +- CometBroadcastHashJoin (54)
+ : :- CometUnion (52)
+ : : :- CometProject (44)
+ : : : +- CometFilter (43)
+ : : : +- CometScan parquet spark_catalog.default.web_sales (42)
+ : : +- CometProject (51)
+ : : +- CometBroadcastHashJoin (50)
+ : : :- CometBroadcastExchange (46)
+ : : : +- CometScan parquet spark_catalog.default.web_returns (45)
+ : : +- CometProject (49)
+ : : +- CometFilter (48)
+ : : +- CometScan parquet spark_catalog.default.web_sales (47)
+ : +- ReusedExchange (53)
+ +- CometBroadcastExchange (58)
+ +- CometFilter (57)
+ +- CometScan parquet spark_catalog.default.web_site (56)
(1) Scan parquet spark_catalog.default.store_sales
@@ -166,9 +164,9 @@ Input [5]: [sales_price#8, profit#9, return_amt#10, net_loss#11, s_store_id#25]
Keys [1]: [s_store_id#25]
Functions [4]: [partial_sum(UnscaledValue(sales_price#8)), partial_sum(UnscaledValue(return_amt#10)), partial_sum(UnscaledValue(profit#9)), partial_sum(UnscaledValue(net_loss#11))]
-(20) CometColumnarExchange
+(20) CometExchange
Input [5]: [s_store_id#25, sum#26, sum#27, sum#28, sum#29]
-Arguments: hashpartitioning(s_store_id#25, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(s_store_id#25, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(21) ColumnarToRow [codegen id : 1]
Input [5]: [s_store_id#25, sum#26, sum#27, sum#28, sum#29]
@@ -257,9 +255,9 @@ Input [5]: [sales_price#46, profit#47, return_amt#48, net_loss#49, cp_catalog_pa
Keys [1]: [cp_catalog_page_id#62]
Functions [4]: [partial_sum(UnscaledValue(sales_price#46)), partial_sum(UnscaledValue(return_amt#48)), partial_sum(UnscaledValue(profit#47)), partial_sum(UnscaledValue(net_loss#49))]
-(39) CometColumnarExchange
+(39) CometExchange
Input [5]: [cp_catalog_page_id#62, sum#63, sum#64, sum#65, sum#66]
-Arguments: hashpartitioning(cp_catalog_page_id#62, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(cp_catalog_page_id#62, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(40) ColumnarToRow [codegen id : 2]
Input [5]: [cp_catalog_page_id#62, sum#63, sum#64, sum#65, sum#66]
@@ -367,9 +365,9 @@ Input [5]: [sales_price#83, profit#84, return_amt#85, net_loss#86, web_site_id#1
Keys [1]: [web_site_id#104]
Functions [4]: [partial_sum(UnscaledValue(sales_price#83)), partial_sum(UnscaledValue(return_amt#85)), partial_sum(UnscaledValue(profit#84)), partial_sum(UnscaledValue(net_loss#86))]
-(62) CometColumnarExchange
+(62) CometExchange
Input [5]: [web_site_id#104, sum#105, sum#106, sum#107, sum#108]
-Arguments: hashpartitioning(web_site_id#104, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(web_site_id#104, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(63) ColumnarToRow [codegen id : 3]
Input [5]: [web_site_id#104, sum#105, sum#106, sum#107, sum#108]
@@ -394,56 +392,50 @@ Functions [3]: [partial_sum(sales#34), partial_sum(returns#35), partial_sum(prof
Aggregate Attributes [6]: [sum#121, isEmpty#122, sum#123, isEmpty#124, sum#125, isEmpty#126]
Results [9]: [channel#118, id#119, spark_grouping_id#120, sum#127, isEmpty#128, sum#129, isEmpty#130, sum#131, isEmpty#132]
-(68) RowToColumnar
+(68) Exchange
Input [9]: [channel#118, id#119, spark_grouping_id#120, sum#127, isEmpty#128, sum#129, isEmpty#130, sum#131, isEmpty#132]
+Arguments: hashpartitioning(channel#118, id#119, spark_grouping_id#120, 5), ENSURE_REQUIREMENTS, [plan_id=4]
-(69) CometColumnarExchange
-Input [9]: [channel#118, id#119, spark_grouping_id#120, sum#127, isEmpty#128, sum#129, isEmpty#130, sum#131, isEmpty#132]
-Arguments: hashpartitioning(channel#118, id#119, spark_grouping_id#120, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-
-(70) ColumnarToRow [codegen id : 5]
-Input [9]: [channel#118, id#119, spark_grouping_id#120, sum#127, isEmpty#128, sum#129, isEmpty#130, sum#131, isEmpty#132]
-
-(71) HashAggregate [codegen id : 5]
+(69) HashAggregate [codegen id : 5]
Input [9]: [channel#118, id#119, spark_grouping_id#120, sum#127, isEmpty#128, sum#129, isEmpty#130, sum#131, isEmpty#132]
Keys [3]: [channel#118, id#119, spark_grouping_id#120]
Functions [3]: [sum(sales#34), sum(returns#35), sum(profit#36)]
Aggregate Attributes [3]: [sum(sales#34)#133, sum(returns#35)#134, sum(profit#36)#135]
Results [5]: [channel#118, id#119, sum(sales#34)#133 AS sales#136, sum(returns#35)#134 AS returns#137, sum(profit#36)#135 AS profit#138]
-(72) TakeOrderedAndProject
+(70) TakeOrderedAndProject
Input [5]: [channel#118, id#119, sales#136, returns#137, profit#138]
Arguments: 100, [channel#118 ASC NULLS FIRST, id#119 ASC NULLS FIRST], [channel#118, id#119, sales#136, returns#137, profit#138]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (77)
-+- * ColumnarToRow (76)
- +- CometProject (75)
- +- CometFilter (74)
- +- CometScan parquet spark_catalog.default.date_dim (73)
+BroadcastExchange (75)
++- * ColumnarToRow (74)
+ +- CometProject (73)
+ +- CometFilter (72)
+ +- CometScan parquet spark_catalog.default.date_dim (71)
-(73) Scan parquet spark_catalog.default.date_dim
+(71) Scan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#22, d_date#23]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-08-23), LessThanOrEqual(d_date,2000-09-06), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(74) CometFilter
+(72) CometFilter
Input [2]: [d_date_sk#22, d_date#23]
Condition : (((isnotnull(d_date#23) AND (d_date#23 >= 2000-08-23)) AND (d_date#23 <= 2000-09-06)) AND isnotnull(d_date_sk#22))
-(75) CometProject
+(73) CometProject
Input [2]: [d_date_sk#22, d_date#23]
Arguments: [d_date_sk#22], [d_date_sk#22]
-(76) ColumnarToRow [codegen id : 1]
+(74) ColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#22]
-(77) BroadcastExchange
+(75) BroadcastExchange
Input [1]: [d_date_sk#22]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/simplified.txt
index 7b4b411a..c6f9f60a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q5/simplified.txt
@@ -1,95 +1,93 @@
TakeOrderedAndProject [channel,id,sales,returns,profit]
WholeStageCodegen (5)
HashAggregate [channel,id,spark_grouping_id,sum,isEmpty,sum,isEmpty,sum,isEmpty] [sum(sales),sum(returns),sum(profit),sales,returns,profit,sum,isEmpty,sum,isEmpty,sum,isEmpty]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [channel,id,spark_grouping_id] #1
- RowToColumnar
- WholeStageCodegen (4)
- HashAggregate [channel,id,spark_grouping_id,sales,returns,profit] [sum,isEmpty,sum,isEmpty,sum,isEmpty,sum,isEmpty,sum,isEmpty,sum,isEmpty]
- Expand [sales,returns,profit,channel,id]
- InputAdapter
- Union
- WholeStageCodegen (1)
- HashAggregate [s_store_id,sum,sum,sum,sum] [sum(UnscaledValue(sales_price)),sum(UnscaledValue(return_amt)),sum(UnscaledValue(profit)),sum(UnscaledValue(net_loss)),sales,returns,profit,channel,id,sum,sum,sum,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [s_store_id] #2
- CometHashAggregate [s_store_id,sum,sum,sum,sum,sales_price,return_amt,profit,net_loss]
- CometProject [sales_price,profit,return_amt,net_loss,s_store_id]
- CometBroadcastHashJoin [store_sk,sales_price,profit,return_amt,net_loss,s_store_sk,s_store_id]
- CometProject [store_sk,sales_price,profit,return_amt,net_loss]
- CometBroadcastHashJoin [store_sk,date_sk,sales_price,profit,return_amt,net_loss,d_date_sk]
- CometUnion [store_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometProject [ss_store_sk,ss_sold_date_sk,ss_ext_sales_price,ss_net_profit] [store_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometFilter [ss_store_sk,ss_ext_sales_price,ss_net_profit,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_store_sk,ss_ext_sales_price,ss_net_profit,ss_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #3
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
- CometProject [sr_store_sk,sr_returned_date_sk,sr_return_amt,sr_net_loss] [store_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometFilter [sr_store_sk,sr_return_amt,sr_net_loss,sr_returned_date_sk]
- CometScan parquet spark_catalog.default.store_returns [sr_store_sk,sr_return_amt,sr_net_loss,sr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- CometBroadcastExchange [d_date_sk] #4
- CometProject [d_date_sk]
- CometFilter [d_date_sk,d_date]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
- CometBroadcastExchange [s_store_sk,s_store_id] #5
- CometFilter [s_store_sk,s_store_id]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_store_id]
- WholeStageCodegen (2)
- HashAggregate [cp_catalog_page_id,sum,sum,sum,sum] [sum(UnscaledValue(sales_price)),sum(UnscaledValue(return_amt)),sum(UnscaledValue(profit)),sum(UnscaledValue(net_loss)),sales,returns,profit,channel,id,sum,sum,sum,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [cp_catalog_page_id] #6
- CometHashAggregate [cp_catalog_page_id,sum,sum,sum,sum,sales_price,return_amt,profit,net_loss]
- CometProject [sales_price,profit,return_amt,net_loss,cp_catalog_page_id]
- CometBroadcastHashJoin [page_sk,sales_price,profit,return_amt,net_loss,cp_catalog_page_sk,cp_catalog_page_id]
- CometProject [page_sk,sales_price,profit,return_amt,net_loss]
- CometBroadcastHashJoin [page_sk,date_sk,sales_price,profit,return_amt,net_loss,d_date_sk]
- CometUnion [page_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometProject [cs_catalog_page_sk,cs_sold_date_sk,cs_ext_sales_price,cs_net_profit] [page_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometFilter [cs_catalog_page_sk,cs_ext_sales_price,cs_net_profit,cs_sold_date_sk]
- CometScan parquet spark_catalog.default.catalog_sales [cs_catalog_page_sk,cs_ext_sales_price,cs_net_profit,cs_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- CometProject [cr_catalog_page_sk,cr_returned_date_sk,cr_return_amount,cr_net_loss] [page_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometFilter [cr_catalog_page_sk,cr_return_amount,cr_net_loss,cr_returned_date_sk]
- CometScan parquet spark_catalog.default.catalog_returns [cr_catalog_page_sk,cr_return_amount,cr_net_loss,cr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #4
- CometBroadcastExchange [cp_catalog_page_sk,cp_catalog_page_id] #7
- CometFilter [cp_catalog_page_sk,cp_catalog_page_id]
- CometScan parquet spark_catalog.default.catalog_page [cp_catalog_page_sk,cp_catalog_page_id]
- WholeStageCodegen (3)
- HashAggregate [web_site_id,sum,sum,sum,sum] [sum(UnscaledValue(sales_price)),sum(UnscaledValue(return_amt)),sum(UnscaledValue(profit)),sum(UnscaledValue(net_loss)),sales,returns,profit,channel,id,sum,sum,sum,sum]
- ColumnarToRow
- InputAdapter
- CometColumnarExchange [web_site_id] #8
- CometHashAggregate [web_site_id,sum,sum,sum,sum,sales_price,return_amt,profit,net_loss]
- CometProject [sales_price,profit,return_amt,net_loss,web_site_id]
- CometBroadcastHashJoin [wsr_web_site_sk,sales_price,profit,return_amt,net_loss,web_site_sk,web_site_id]
- CometProject [wsr_web_site_sk,sales_price,profit,return_amt,net_loss]
- CometBroadcastHashJoin [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss,d_date_sk]
- CometUnion [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometProject [ws_web_site_sk,ws_sold_date_sk,ws_ext_sales_price,ws_net_profit] [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometFilter [ws_web_site_sk,ws_ext_sales_price,ws_net_profit,ws_sold_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_web_site_sk,ws_ext_sales_price,ws_net_profit,ws_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- CometProject [ws_web_site_sk,wr_returned_date_sk,wr_return_amt,wr_net_loss] [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss]
- CometBroadcastHashJoin [wr_item_sk,wr_order_number,wr_return_amt,wr_net_loss,wr_returned_date_sk,ws_item_sk,ws_web_site_sk,ws_order_number]
- CometBroadcastExchange [wr_item_sk,wr_order_number,wr_return_amt,wr_net_loss,wr_returned_date_sk] #9
- CometScan parquet spark_catalog.default.web_returns [wr_item_sk,wr_order_number,wr_return_amt,wr_net_loss,wr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- CometProject [ws_item_sk,ws_web_site_sk,ws_order_number]
- CometFilter [ws_item_sk,ws_web_site_sk,ws_order_number,ws_sold_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_web_site_sk,ws_order_number,ws_sold_date_sk]
- ReusedExchange [d_date_sk] #4
- CometBroadcastExchange [web_site_sk,web_site_id] #10
- CometFilter [web_site_sk,web_site_id]
- CometScan parquet spark_catalog.default.web_site [web_site_sk,web_site_id]
+ InputAdapter
+ Exchange [channel,id,spark_grouping_id] #1
+ WholeStageCodegen (4)
+ HashAggregate [channel,id,spark_grouping_id,sales,returns,profit] [sum,isEmpty,sum,isEmpty,sum,isEmpty,sum,isEmpty,sum,isEmpty,sum,isEmpty]
+ Expand [sales,returns,profit,channel,id]
+ InputAdapter
+ Union
+ WholeStageCodegen (1)
+ HashAggregate [s_store_id,sum,sum,sum,sum] [sum(UnscaledValue(sales_price)),sum(UnscaledValue(return_amt)),sum(UnscaledValue(profit)),sum(UnscaledValue(net_loss)),sales,returns,profit,channel,id,sum,sum,sum,sum]
+ ColumnarToRow
+ InputAdapter
+ CometExchange [s_store_id] #2
+ CometHashAggregate [s_store_id,sum,sum,sum,sum,sales_price,return_amt,profit,net_loss]
+ CometProject [sales_price,profit,return_amt,net_loss,s_store_id]
+ CometBroadcastHashJoin [store_sk,sales_price,profit,return_amt,net_loss,s_store_sk,s_store_id]
+ CometProject [store_sk,sales_price,profit,return_amt,net_loss]
+ CometBroadcastHashJoin [store_sk,date_sk,sales_price,profit,return_amt,net_loss,d_date_sk]
+ CometUnion [store_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometProject [ss_store_sk,ss_sold_date_sk,ss_ext_sales_price,ss_net_profit] [store_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometFilter [ss_store_sk,ss_ext_sales_price,ss_net_profit,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_store_sk,ss_ext_sales_price,ss_net_profit,ss_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #3
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometProject [sr_store_sk,sr_returned_date_sk,sr_return_amt,sr_net_loss] [store_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometFilter [sr_store_sk,sr_return_amt,sr_net_loss,sr_returned_date_sk]
+ CometScan parquet spark_catalog.default.store_returns [sr_store_sk,sr_return_amt,sr_net_loss,sr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ CometBroadcastExchange [d_date_sk] #4
+ CometProject [d_date_sk]
+ CometFilter [d_date_sk,d_date]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
+ CometBroadcastExchange [s_store_sk,s_store_id] #5
+ CometFilter [s_store_sk,s_store_id]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_store_id]
+ WholeStageCodegen (2)
+ HashAggregate [cp_catalog_page_id,sum,sum,sum,sum] [sum(UnscaledValue(sales_price)),sum(UnscaledValue(return_amt)),sum(UnscaledValue(profit)),sum(UnscaledValue(net_loss)),sales,returns,profit,channel,id,sum,sum,sum,sum]
+ ColumnarToRow
+ InputAdapter
+ CometExchange [cp_catalog_page_id] #6
+ CometHashAggregate [cp_catalog_page_id,sum,sum,sum,sum,sales_price,return_amt,profit,net_loss]
+ CometProject [sales_price,profit,return_amt,net_loss,cp_catalog_page_id]
+ CometBroadcastHashJoin [page_sk,sales_price,profit,return_amt,net_loss,cp_catalog_page_sk,cp_catalog_page_id]
+ CometProject [page_sk,sales_price,profit,return_amt,net_loss]
+ CometBroadcastHashJoin [page_sk,date_sk,sales_price,profit,return_amt,net_loss,d_date_sk]
+ CometUnion [page_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometProject [cs_catalog_page_sk,cs_sold_date_sk,cs_ext_sales_price,cs_net_profit] [page_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometFilter [cs_catalog_page_sk,cs_ext_sales_price,cs_net_profit,cs_sold_date_sk]
+ CometScan parquet spark_catalog.default.catalog_sales [cs_catalog_page_sk,cs_ext_sales_price,cs_net_profit,cs_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ CometProject [cr_catalog_page_sk,cr_returned_date_sk,cr_return_amount,cr_net_loss] [page_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometFilter [cr_catalog_page_sk,cr_return_amount,cr_net_loss,cr_returned_date_sk]
+ CometScan parquet spark_catalog.default.catalog_returns [cr_catalog_page_sk,cr_return_amount,cr_net_loss,cr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #4
+ CometBroadcastExchange [cp_catalog_page_sk,cp_catalog_page_id] #7
+ CometFilter [cp_catalog_page_sk,cp_catalog_page_id]
+ CometScan parquet spark_catalog.default.catalog_page [cp_catalog_page_sk,cp_catalog_page_id]
+ WholeStageCodegen (3)
+ HashAggregate [web_site_id,sum,sum,sum,sum] [sum(UnscaledValue(sales_price)),sum(UnscaledValue(return_amt)),sum(UnscaledValue(profit)),sum(UnscaledValue(net_loss)),sales,returns,profit,channel,id,sum,sum,sum,sum]
+ ColumnarToRow
+ InputAdapter
+ CometExchange [web_site_id] #8
+ CometHashAggregate [web_site_id,sum,sum,sum,sum,sales_price,return_amt,profit,net_loss]
+ CometProject [sales_price,profit,return_amt,net_loss,web_site_id]
+ CometBroadcastHashJoin [wsr_web_site_sk,sales_price,profit,return_amt,net_loss,web_site_sk,web_site_id]
+ CometProject [wsr_web_site_sk,sales_price,profit,return_amt,net_loss]
+ CometBroadcastHashJoin [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss,d_date_sk]
+ CometUnion [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometProject [ws_web_site_sk,ws_sold_date_sk,ws_ext_sales_price,ws_net_profit] [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometFilter [ws_web_site_sk,ws_ext_sales_price,ws_net_profit,ws_sold_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_web_site_sk,ws_ext_sales_price,ws_net_profit,ws_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ CometProject [ws_web_site_sk,wr_returned_date_sk,wr_return_amt,wr_net_loss] [wsr_web_site_sk,date_sk,sales_price,profit,return_amt,net_loss]
+ CometBroadcastHashJoin [wr_item_sk,wr_order_number,wr_return_amt,wr_net_loss,wr_returned_date_sk,ws_item_sk,ws_web_site_sk,ws_order_number]
+ CometBroadcastExchange [wr_item_sk,wr_order_number,wr_return_amt,wr_net_loss,wr_returned_date_sk] #9
+ CometScan parquet spark_catalog.default.web_returns [wr_item_sk,wr_order_number,wr_return_amt,wr_net_loss,wr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ CometProject [ws_item_sk,ws_web_site_sk,ws_order_number]
+ CometFilter [ws_item_sk,ws_web_site_sk,ws_order_number,ws_sold_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_web_site_sk,ws_order_number,ws_sold_date_sk]
+ ReusedExchange [d_date_sk] #4
+ CometBroadcastExchange [web_site_sk,web_site_id] #10
+ CometFilter [web_site_sk,web_site_id]
+ CometScan parquet spark_catalog.default.web_site [web_site_sk,web_site_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/explain.txt
index 1e9d7743..84eef7b5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (28)
+- CometTakeOrderedAndProject (27)
+- CometHashAggregate (26)
- +- CometColumnarExchange (25)
+ +- CometExchange (25)
+- CometHashAggregate (24)
+- CometProject (23)
+- CometBroadcastHashJoin (22)
@@ -147,9 +147,9 @@ Input [12]: [ss_sold_date_sk#5, sr_returned_date_sk#9, s_store_name#12, s_compan
Keys [10]: [s_store_name#12, s_company_id#13, s_street_number#14, s_street_name#15, s_street_type#16, s_suite_number#17, s_city#18, s_county#19, s_state#20, s_zip#21]
Functions [5]: [partial_sum(CASE WHEN ((sr_returned_date_sk#9 - ss_sold_date_sk#5) <= 30) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (((sr_returned_date_sk#9 - ss_sold_date_sk#5) > 30) AND ((sr_returned_date_sk#9 - ss_sold_date_sk#5) <= 60)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (((sr_returned_date_sk#9 - ss_sold_date_sk#5) > 60) AND ((sr_returned_date_sk#9 - ss_sold_date_sk#5) <= 90)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (((sr_returned_date_sk#9 - ss_sold_date_sk#5) > 90) AND (( [...]
-(25) CometColumnarExchange
+(25) CometExchange
Input [15]: [s_store_name#12, s_company_id#13, s_street_number#14, s_street_name#15, s_street_type#16, s_suite_number#17, s_city#18, s_county#19, s_state#20, s_zip#21, sum#26, sum#27, sum#28, sum#29, sum#30]
-Arguments: hashpartitioning(s_store_name#12, s_company_id#13, s_street_number#14, s_street_name#15, s_street_type#16, s_suite_number#17, s_city#18, s_county#19, s_state#20, s_zip#21, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(s_store_name#12, s_company_id#13, s_street_number#14, s_street_name#15, s_street_type#16, s_suite_number#17, s_city#18, s_county#19, s_state#20, s_zip#21, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(26) CometHashAggregate
Input [15]: [s_store_name#12, s_company_id#13, s_street_number#14, s_street_name#15, s_street_type#16, s_suite_number#17, s_city#18, s_county#19, s_state#20, s_zip#21, sum#26, sum#27, sum#28, sum#29, sum#30]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/simplified.txt
index 075786f9..53333446 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q50/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
CometHashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum,sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date [...]
- CometColumnarExchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
+ CometExchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
CometHashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum,sr_returned_date_sk,ss_sold_date_sk]
CometProject [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
CometBroadcastHashJoin [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/explain.txt
index 3bd86214..fcd43fe3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/explain.txt
@@ -1,47 +1,44 @@
== Physical Plan ==
-* ColumnarToRow (43)
-+- CometTakeOrderedAndProject (42)
- +- CometFilter (41)
- +- CometWindowExec (40)
- +- CometSort (39)
- +- CometColumnarExchange (38)
- +- CometProject (37)
- +- CometSortMergeJoin (36)
- :- CometSort (19)
- : +- CometColumnarExchange (18)
- : +- RowToColumnar (17)
- : +- * Project (16)
- : +- Window (15)
- : +- * ColumnarToRow (14)
- : +- CometSort (13)
- : +- CometColumnarExchange (12)
- : +- CometHashAggregate (11)
- : +- CometColumnarExchange (10)
- : +- CometHashAggregate (9)
- : +- CometProject (8)
- : +- CometBroadcastHashJoin (7)
- : :- CometFilter (2)
- : : +- CometScan parquet spark_catalog.default.web_sales (1)
- : +- CometBroadcastExchange (6)
- : +- CometProject (5)
- : +- CometFilter (4)
- : +- CometScan parquet spark_catalog.default.date_dim (3)
- +- CometSort (35)
- +- CometColumnarExchange (34)
- +- RowToColumnar (33)
- +- * Project (32)
- +- Window (31)
- +- * ColumnarToRow (30)
- +- CometSort (29)
- +- CometColumnarExchange (28)
- +- CometHashAggregate (27)
- +- CometColumnarExchange (26)
- +- CometHashAggregate (25)
- +- CometProject (24)
- +- CometBroadcastHashJoin (23)
- :- CometFilter (21)
- : +- CometScan parquet spark_catalog.default.store_sales (20)
- +- ReusedExchange (22)
+TakeOrderedAndProject (40)
++- * Filter (39)
+ +- Window (38)
+ +- * Sort (37)
+ +- Exchange (36)
+ +- * Project (35)
+ +- * SortMergeJoin FullOuter (34)
+ :- * Sort (18)
+ : +- Exchange (17)
+ : +- * Project (16)
+ : +- Window (15)
+ : +- * ColumnarToRow (14)
+ : +- CometSort (13)
+ : +- CometExchange (12)
+ : +- CometHashAggregate (11)
+ : +- CometExchange (10)
+ : +- CometHashAggregate (9)
+ : +- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
+ : :- CometFilter (2)
+ : : +- CometScan parquet spark_catalog.default.web_sales (1)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan parquet spark_catalog.default.date_dim (3)
+ +- * Sort (33)
+ +- Exchange (32)
+ +- * Project (31)
+ +- Window (30)
+ +- * ColumnarToRow (29)
+ +- CometSort (28)
+ +- CometExchange (27)
+ +- CometHashAggregate (26)
+ +- CometExchange (25)
+ +- CometHashAggregate (24)
+ +- CometProject (23)
+ +- CometBroadcastHashJoin (22)
+ :- CometFilter (20)
+ : +- CometScan parquet spark_catalog.default.store_sales (19)
+ +- ReusedExchange (21)
(1) Scan parquet spark_catalog.default.web_sales
@@ -89,18 +86,18 @@ Input [3]: [ws_item_sk#1, ws_sales_price#2, d_date#6]
Keys [2]: [ws_item_sk#1, d_date#6]
Functions [1]: [partial_sum(UnscaledValue(ws_sales_price#2))]
-(10) CometColumnarExchange
+(10) CometExchange
Input [3]: [ws_item_sk#1, d_date#6, sum#8]
-Arguments: hashpartitioning(ws_item_sk#1, d_date#6, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ws_item_sk#1, d_date#6, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(11) CometHashAggregate
Input [3]: [ws_item_sk#1, d_date#6, sum#8]
Keys [2]: [ws_item_sk#1, d_date#6]
Functions [1]: [sum(UnscaledValue(ws_sales_price#2))]
-(12) CometColumnarExchange
+(12) CometExchange
Input [4]: [item_sk#9, d_date#6, _w0#10, ws_item_sk#1]
-Arguments: hashpartitioning(ws_item_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(ws_item_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(13) CometSort
Input [4]: [item_sk#9, d_date#6, _w0#10, ws_item_sk#1]
@@ -117,18 +114,15 @@ Arguments: [sum(_w0#10) windowspecdefinition(ws_item_sk#1, d_date#6 ASC NULLS FI
Output [3]: [item_sk#9, d_date#6, cume_sales#11]
Input [5]: [item_sk#9, d_date#6, _w0#10, ws_item_sk#1, cume_sales#11]
-(17) RowToColumnar
+(17) Exchange
Input [3]: [item_sk#9, d_date#6, cume_sales#11]
+Arguments: hashpartitioning(item_sk#9, d_date#6, 5), ENSURE_REQUIREMENTS, [plan_id=3]
-(18) CometColumnarExchange
+(18) Sort [codegen id : 3]
Input [3]: [item_sk#9, d_date#6, cume_sales#11]
-Arguments: hashpartitioning(item_sk#9, d_date#6, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: [item_sk#9 ASC NULLS FIRST, d_date#6 ASC NULLS FIRST], false, 0
-(19) CometSort
-Input [3]: [item_sk#9, d_date#6, cume_sales#11]
-Arguments: [item_sk#9, d_date#6, cume_sales#11], [item_sk#9 ASC NULLS FIRST, d_date#6 ASC NULLS FIRST]
-
-(20) Scan parquet spark_catalog.default.store_sales
+(19) Scan parquet spark_catalog.default.store_sales
Output [3]: [ss_item_sk#12, ss_sales_price#13, ss_sold_date_sk#14]
Batched: true
Location: InMemoryFileIndex []
@@ -136,130 +130,125 @@ PartitionFilters: [isnotnull(ss_sold_date_sk#14), dynamicpruningexpression(ss_so
PushedFilters: [IsNotNull(ss_item_sk)]
ReadSchema: struct<ss_item_sk:int,ss_sales_price:decimal(7,2)>
-(21) CometFilter
+(20) CometFilter
Input [3]: [ss_item_sk#12, ss_sales_price#13, ss_sold_date_sk#14]
Condition : isnotnull(ss_item_sk#12)
-(22) ReusedExchange [Reuses operator id: 6]
+(21) ReusedExchange [Reuses operator id: 6]
Output [2]: [d_date_sk#16, d_date#17]
-(23) CometBroadcastHashJoin
+(22) CometBroadcastHashJoin
Left output [3]: [ss_item_sk#12, ss_sales_price#13, ss_sold_date_sk#14]
Right output [2]: [d_date_sk#16, d_date#17]
Arguments: [ss_sold_date_sk#14], [d_date_sk#16], Inner, BuildRight
-(24) CometProject
+(23) CometProject
Input [5]: [ss_item_sk#12, ss_sales_price#13, ss_sold_date_sk#14, d_date_sk#16, d_date#17]
Arguments: [ss_item_sk#12, ss_sales_price#13, d_date#17], [ss_item_sk#12, ss_sales_price#13, d_date#17]
-(25) CometHashAggregate
+(24) CometHashAggregate
Input [3]: [ss_item_sk#12, ss_sales_price#13, d_date#17]
Keys [2]: [ss_item_sk#12, d_date#17]
Functions [1]: [partial_sum(UnscaledValue(ss_sales_price#13))]
-(26) CometColumnarExchange
+(25) CometExchange
Input [3]: [ss_item_sk#12, d_date#17, sum#18]
-Arguments: hashpartitioning(ss_item_sk#12, d_date#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(ss_item_sk#12, d_date#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
-(27) CometHashAggregate
+(26) CometHashAggregate
Input [3]: [ss_item_sk#12, d_date#17, sum#18]
Keys [2]: [ss_item_sk#12, d_date#17]
Functions [1]: [sum(UnscaledValue(ss_sales_price#13))]
-(28) CometColumnarExchange
+(27) CometExchange
Input [4]: [item_sk#19, d_date#17, _w0#20, ss_item_sk#12]
-Arguments: hashpartitioning(ss_item_sk#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(ss_item_sk#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
-(29) CometSort
+(28) CometSort
Input [4]: [item_sk#19, d_date#17, _w0#20, ss_item_sk#12]
Arguments: [item_sk#19, d_date#17, _w0#20, ss_item_sk#12], [ss_item_sk#12 ASC NULLS FIRST, d_date#17 ASC NULLS FIRST]
-(30) ColumnarToRow [codegen id : 3]
+(29) ColumnarToRow [codegen id : 4]
Input [4]: [item_sk#19, d_date#17, _w0#20, ss_item_sk#12]
-(31) Window
+(30) Window
Input [4]: [item_sk#19, d_date#17, _w0#20, ss_item_sk#12]
Arguments: [sum(_w0#20) windowspecdefinition(ss_item_sk#12, d_date#17 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS cume_sales#21], [ss_item_sk#12], [d_date#17 ASC NULLS FIRST]
-(32) Project [codegen id : 4]
+(31) Project [codegen id : 5]
Output [3]: [item_sk#19, d_date#17, cume_sales#21]
Input [5]: [item_sk#19, d_date#17, _w0#20, ss_item_sk#12, cume_sales#21]
-(33) RowToColumnar
-Input [3]: [item_sk#19, d_date#17, cume_sales#21]
-
-(34) CometColumnarExchange
+(32) Exchange
Input [3]: [item_sk#19, d_date#17, cume_sales#21]
-Arguments: hashpartitioning(item_sk#19, d_date#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: hashpartitioning(item_sk#19, d_date#17, 5), ENSURE_REQUIREMENTS, [plan_id=6]
-(35) CometSort
+(33) Sort [codegen id : 6]
Input [3]: [item_sk#19, d_date#17, cume_sales#21]
-Arguments: [item_sk#19, d_date#17, cume_sales#21], [item_sk#19 ASC NULLS FIRST, d_date#17 ASC NULLS FIRST]
+Arguments: [item_sk#19 ASC NULLS FIRST, d_date#17 ASC NULLS FIRST], false, 0
-(36) CometSortMergeJoin
-Left output [3]: [item_sk#9, d_date#6, cume_sales#11]
-Right output [3]: [item_sk#19, d_date#17, cume_sales#21]
-Arguments: [item_sk#9, d_date#6], [item_sk#19, d_date#17], FullOuter
+(34) SortMergeJoin [codegen id : 7]
+Left keys [2]: [item_sk#9, d_date#6]
+Right keys [2]: [item_sk#19, d_date#17]
+Join type: FullOuter
+Join condition: None
-(37) CometProject
+(35) Project [codegen id : 7]
+Output [4]: [CASE WHEN isnotnull(item_sk#9) THEN item_sk#9 ELSE item_sk#19 END AS item_sk#22, CASE WHEN isnotnull(d_date#6) THEN d_date#6 ELSE d_date#17 END AS d_date#23, cume_sales#11 AS web_sales#24, cume_sales#21 AS store_sales#25]
Input [6]: [item_sk#9, d_date#6, cume_sales#11, item_sk#19, d_date#17, cume_sales#21]
-Arguments: [item_sk#22, d_date#23, web_sales#24, store_sales#25], [CASE WHEN isnotnull(item_sk#9) THEN item_sk#9 ELSE item_sk#19 END AS item_sk#22, CASE WHEN isnotnull(d_date#6) THEN d_date#6 ELSE d_date#17 END AS d_date#23, cume_sales#11 AS web_sales#24, cume_sales#21 AS store_sales#25]
-(38) CometColumnarExchange
+(36) Exchange
Input [4]: [item_sk#22, d_date#23, web_sales#24, store_sales#25]
-Arguments: hashpartitioning(item_sk#22, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
+Arguments: hashpartitioning(item_sk#22, 5), ENSURE_REQUIREMENTS, [plan_id=7]
-(39) CometSort
+(37) Sort [codegen id : 8]
Input [4]: [item_sk#22, d_date#23, web_sales#24, store_sales#25]
-Arguments: [item_sk#22, d_date#23, web_sales#24, store_sales#25], [item_sk#22 ASC NULLS FIRST, d_date#23 ASC NULLS FIRST]
+Arguments: [item_sk#22 ASC NULLS FIRST, d_date#23 ASC NULLS FIRST], false, 0
-(40) CometWindowExec
+(38) Window
Input [4]: [item_sk#22, d_date#23, web_sales#24, store_sales#25]
-Arguments: [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27], [max(web_sales#24) windowspecdefinition(item_sk#22, d_date#23 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS web_cumulative#26, max(store_sales#25) windowspecdefinition(item_sk#22, d_date#23 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS store_cumulative#27], [item_sk#22], [d_date#23 ASC NULLS FIRST]
+Arguments: [max(web_sales#24) windowspecdefinition(item_sk#22, d_date#23 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS web_cumulative#26, max(store_sales#25) windowspecdefinition(item_sk#22, d_date#23 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS store_cumulative#27], [item_sk#22], [d_date#23 ASC NULLS FIRST]
-(41) CometFilter
+(39) Filter [codegen id : 9]
Input [6]: [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27]
Condition : ((isnotnull(web_cumulative#26) AND isnotnull(store_cumulative#27)) AND (web_cumulative#26 > store_cumulative#27))
-(42) CometTakeOrderedAndProject
-Input [6]: [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[item_sk#22 ASC NULLS FIRST,d_date#23 ASC NULLS FIRST], output=[item_sk#22,d_date#23,web_sales#24,store_sales#25,web_cumulative#26,store_cumulative#27]), [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27], 100, [item_sk#22 ASC NULLS FIRST, d_date#23 ASC NULLS FIRST], [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27]
-
-(43) ColumnarToRow [codegen id : 5]
+(40) TakeOrderedAndProject
Input [6]: [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27]
+Arguments: 100, [item_sk#22 ASC NULLS FIRST, d_date#23 ASC NULLS FIRST], [item_sk#22, d_date#23, web_sales#24, store_sales#25, web_cumulative#26, store_cumulative#27]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ws_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (48)
-+- * ColumnarToRow (47)
- +- CometProject (46)
- +- CometFilter (45)
- +- CometScan parquet spark_catalog.default.date_dim (44)
+BroadcastExchange (45)
++- * ColumnarToRow (44)
+ +- CometProject (43)
+ +- CometFilter (42)
+ +- CometScan parquet spark_catalog.default.date_dim (41)
-(44) Scan parquet spark_catalog.default.date_dim
+(41) Scan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#5, d_date#6, d_month_seq#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_month_seq), GreaterThanOrEqual(d_month_seq,1200), LessThanOrEqual(d_month_seq,1211), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date,d_month_seq:int>
-(45) CometFilter
+(42) CometFilter
Input [3]: [d_date_sk#5, d_date#6, d_month_seq#7]
Condition : (((isnotnull(d_month_seq#7) AND (d_month_seq#7 >= 1200)) AND (d_month_seq#7 <= 1211)) AND isnotnull(d_date_sk#5))
-(46) CometProject
+(43) CometProject
Input [3]: [d_date_sk#5, d_date#6, d_month_seq#7]
Arguments: [d_date_sk#5, d_date#6], [d_date_sk#5, d_date#6]
-(47) ColumnarToRow [codegen id : 1]
+(44) ColumnarToRow [codegen id : 1]
Input [2]: [d_date_sk#5, d_date#6]
-(48) BroadcastExchange
+(45) BroadcastExchange
Input [2]: [d_date_sk#5, d_date#6]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=8]
-Subquery:2 Hosting operator id = 20 Hosting Expression = ss_sold_date_sk#14 IN dynamicpruning#4
+Subquery:2 Hosting operator id = 19 Hosting Expression = ss_sold_date_sk#14 IN dynamicpruning#4
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/simplified.txt
index 0e5a8e3d..f972c31c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q51/simplified.txt
@@ -1,62 +1,68 @@
-WholeStageCodegen (5)
- ColumnarToRow
- InputAdapter
- CometTakeOrderedAndProject [item_sk,d_date,web_sales,store_sales,web_cumulative,store_cumulative]
- CometFilter [item_sk,d_date,web_sales,store_sales,web_cumulative,store_cumulative]
- CometWindowExec [item_sk,d_date,web_sales,store_sales,web_cumulative,store_cumulative]
- CometSort [item_sk,d_date,web_sales,store_sales]
- CometColumnarExchange [item_sk] #1
- CometProject [item_sk,item_sk,d_date,d_date,cume_sales,cume_sales] [item_sk,d_date,web_sales,store_sales]
- CometSortMergeJoin [item_sk,d_date,cume_sales,item_sk,d_date,cume_sales]
- CometSort [item_sk,d_date,cume_sales]
- CometColumnarExchange [item_sk,d_date] #2
- RowToColumnar
- WholeStageCodegen (2)
- Project [item_sk,d_date,cume_sales]
+TakeOrderedAndProject [item_sk,d_date,web_sales,store_sales,web_cumulative,store_cumulative]
+ WholeStageCodegen (9)
+ Filter [web_cumulative,store_cumulative]
+ InputAdapter
+ Window [web_sales,item_sk,d_date,store_sales]
+ WholeStageCodegen (8)
+ Sort [item_sk,d_date]
+ InputAdapter
+ Exchange [item_sk] #1
+ WholeStageCodegen (7)
+ Project [item_sk,item_sk,d_date,d_date,cume_sales,cume_sales]
+ SortMergeJoin [item_sk,d_date,item_sk,d_date]
+ InputAdapter
+ WholeStageCodegen (3)
+ Sort [item_sk,d_date]
InputAdapter
- Window [_w0,ws_item_sk,d_date]
- WholeStageCodegen (1)
- ColumnarToRow
+ Exchange [item_sk,d_date] #2
+ WholeStageCodegen (2)
+ Project [item_sk,d_date,cume_sales]
InputAdapter
- CometSort [item_sk,d_date,_w0,ws_item_sk]
- CometColumnarExchange [ws_item_sk] #3
- CometHashAggregate [item_sk,d_date,_w0,ws_item_sk,sum,sum(UnscaledValue(ws_sales_price))]
- CometColumnarExchange [ws_item_sk,d_date] #4
- CometHashAggregate [ws_item_sk,d_date,sum,ws_sales_price]
- CometProject [ws_item_sk,ws_sales_price,d_date]
- CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ws_sold_date_sk,d_date_sk,d_date]
- CometFilter [ws_item_sk,ws_sales_price,ws_sold_date_sk]
- CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sales_price,ws_sold_date_sk]
- SubqueryBroadcast [d_date_sk] #1
- BroadcastExchange #5
- WholeStageCodegen (1)
- ColumnarToRow
- InputAdapter
- CometProject [d_date_sk,d_date]
- CometFilter [d_date_sk,d_date,d_month_seq]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_month_seq]
- CometBroadcastExchange [d_date_sk,d_date] #6
- CometProject [d_date_sk,d_date]
- CometFilter [d_date_sk,d_date,d_month_seq]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_month_seq]
- CometSort [item_sk,d_date,cume_sales]
- CometColumnarExchange [item_sk,d_date] #7
- RowToColumnar
- WholeStageCodegen (4)
- Project [item_sk,d_date,cume_sales]
+ Window [_w0,ws_item_sk,d_date]
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item_sk,d_date,_w0,ws_item_sk]
+ CometExchange [ws_item_sk] #3
+ CometHashAggregate [item_sk,d_date,_w0,ws_item_sk,sum,sum(UnscaledValue(ws_sales_price))]
+ CometExchange [ws_item_sk,d_date] #4
+ CometHashAggregate [ws_item_sk,d_date,sum,ws_sales_price]
+ CometProject [ws_item_sk,ws_sales_price,d_date]
+ CometBroadcastHashJoin [ws_item_sk,ws_sales_price,ws_sold_date_sk,d_date_sk,d_date]
+ CometFilter [ws_item_sk,ws_sales_price,ws_sold_date_sk]
+ CometScan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sales_price,ws_sold_date_sk]
+ SubqueryBroadcast [d_date_sk] #1
+ BroadcastExchange #5
+ WholeStageCodegen (1)
+ ColumnarToRow
+ InputAdapter
+ CometProject [d_date_sk,d_date]
+ CometFilter [d_date_sk,d_date,d_month_seq]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_month_seq]
+ CometBroadcastExchange [d_date_sk,d_date] #6
+ CometProject [d_date_sk,d_date]
+ CometFilter [d_date_sk,d_date,d_month_seq]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_month_seq]
+ InputAdapter
+ WholeStageCodegen (6)
+ Sort [item_sk,d_date]
InputAdapter
- Window [_w0,ss_item_sk,d_date]
- WholeStageCodegen (3)
- ColumnarToRow
+ Exchange [item_sk,d_date] #7
+ WholeStageCodegen (5)
+ Project [item_sk,d_date,cume_sales]
InputAdapter
- CometSort [item_sk,d_date,_w0,ss_item_sk]
- CometColumnarExchange [ss_item_sk] #8
- CometHashAggregate [item_sk,d_date,_w0,ss_item_sk,sum,sum(UnscaledValue(ss_sales_price))]
- CometColumnarExchange [ss_item_sk,d_date] #9
- CometHashAggregate [ss_item_sk,d_date,sum,ss_sales_price]
- CometProject [ss_item_sk,ss_sales_price,d_date]
- CometBroadcastHashJoin [ss_item_sk,ss_sales_price,ss_sold_date_sk,d_date_sk,d_date]
- CometFilter [ss_item_sk,ss_sales_price,ss_sold_date_sk]
- CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sales_price,ss_sold_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk,d_date] #6
+ Window [_w0,ss_item_sk,d_date]
+ WholeStageCodegen (4)
+ ColumnarToRow
+ InputAdapter
+ CometSort [item_sk,d_date,_w0,ss_item_sk]
+ CometExchange [ss_item_sk] #8
+ CometHashAggregate [item_sk,d_date,_w0,ss_item_sk,sum,sum(UnscaledValue(ss_sales_price))]
+ CometExchange [ss_item_sk,d_date] #9
+ CometHashAggregate [ss_item_sk,d_date,sum,ss_sales_price]
+ CometProject [ss_item_sk,ss_sales_price,d_date]
+ CometBroadcastHashJoin [ss_item_sk,ss_sales_price,ss_sold_date_sk,d_date_sk,d_date]
+ CometFilter [ss_item_sk,ss_sales_price,ss_sold_date_sk]
+ CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sales_price,ss_sold_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk,d_date] #6
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/explain.txt
index f8beff74..eee36d22 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (19)
+- CometTakeOrderedAndProject (18)
+- CometHashAggregate (17)
- +- CometColumnarExchange (16)
+ +- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
+- CometBroadcastHashJoin (13)
@@ -93,9 +93,9 @@ Input [4]: [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#9]
Keys [3]: [d_year#2, i_brand#9, i_brand_id#8]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
-(16) CometColumnarExchange
+(16) CometExchange
Input [4]: [d_year#2, i_brand#9, i_brand_id#8, sum#11]
-Arguments: hashpartitioning(d_year#2, i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(d_year#2, i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
Input [4]: [d_year#2, i_brand#9, i_brand_id#8, sum#11]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/simplified.txt
index 99f3baf9..af5223b6 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q52/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [d_year,brand_id,brand,ext_price]
CometHashAggregate [d_year,brand_id,brand,ext_price,i_brand,i_brand_id,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [d_year,i_brand,i_brand_id] #1
+ CometExchange [d_year,i_brand,i_brand_id] #1
CometHashAggregate [d_year,i_brand,i_brand_id,sum,ss_ext_sales_price]
CometProject [d_year,ss_ext_sales_price,i_brand_id,i_brand]
CometBroadcastHashJoin [d_year,ss_item_sk,ss_ext_sales_price,i_item_sk,i_brand_id,i_brand]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/explain.txt
index 71c84c04..8156e275 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/explain.txt
@@ -5,9 +5,9 @@ TakeOrderedAndProject (29)
+- Window (26)
+- * ColumnarToRow (25)
+- CometSort (24)
- +- CometColumnarExchange (23)
+ +- CometExchange (23)
+- CometHashAggregate (22)
- +- CometColumnarExchange (21)
+ +- CometExchange (21)
+- CometHashAggregate (20)
+- CometProject (19)
+- CometBroadcastHashJoin (18)
@@ -127,18 +127,18 @@ Input [3]: [i_manufact_id#5, ss_sales_price#12, d_qoy#17]
Keys [2]: [i_manufact_id#5, d_qoy#17]
Functions [1]: [partial_sum(UnscaledValue(ss_sales_price#12))]
-(21) CometColumnarExchange
+(21) CometExchange
Input [3]: [i_manufact_id#5, d_qoy#17, sum#19]
-Arguments: hashpartitioning(i_manufact_id#5, d_qoy#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_manufact_id#5, d_qoy#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(22) CometHashAggregate
Input [3]: [i_manufact_id#5, d_qoy#17, sum#19]
Keys [2]: [i_manufact_id#5, d_qoy#17]
Functions [1]: [sum(UnscaledValue(ss_sales_price#12))]
-(23) CometColumnarExchange
+(23) CometExchange
Input [3]: [i_manufact_id#5, sum_sales#20, _w0#21]
-Arguments: hashpartitioning(i_manufact_id#5, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_manufact_id#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(24) CometSort
Input [3]: [i_manufact_id#5, sum_sales#20, _w0#21]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/simplified.txt
index a0d09de8..363b84b3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q53/simplified.txt
@@ -8,9 +8,9 @@ TakeOrderedAndProject [avg_quarterly_sales,sum_sales,i_manufact_id]
ColumnarToRow
InputAdapter
CometSort [i_manufact_id,sum_sales,_w0]
- CometColumnarExchange [i_manufact_id] #1
+ CometExchange [i_manufact_id] #1
CometHashAggregate [i_manufact_id,sum_sales,_w0,d_qoy,sum,sum(UnscaledValue(ss_sales_price))]
- CometColumnarExchange [i_manufact_id,d_qoy] #2
+ CometExchange [i_manufact_id,d_qoy] #2
CometHashAggregate [i_manufact_id,d_qoy,sum,ss_sales_price]
CometProject [i_manufact_id,ss_sales_price,d_qoy]
CometBroadcastHashJoin [i_manufact_id,ss_store_sk,ss_sales_price,d_qoy,s_store_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/explain.txt
index cf153155..d5bd8e38 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/explain.txt
@@ -2,10 +2,10 @@
* ColumnarToRow (56)
+- CometTakeOrderedAndProject (55)
+- CometHashAggregate (54)
- +- CometColumnarExchange (53)
+ +- CometExchange (53)
+- CometHashAggregate (52)
+- CometHashAggregate (51)
- +- CometColumnarExchange (50)
+ +- CometExchange (50)
+- CometHashAggregate (49)
+- CometProject (48)
+- CometBroadcastHashJoin (47)
@@ -16,7 +16,7 @@
: : :- CometProject (32)
: : : +- CometBroadcastHashJoin (31)
: : : :- CometHashAggregate (27)
- : : : : +- CometColumnarExchange (26)
+ : : : : +- CometExchange (26)
: : : : +- CometHashAggregate (25)
: : : : +- CometProject (24)
: : : : +- CometBroadcastHashJoin (23)
@@ -178,9 +178,9 @@ Input [2]: [c_customer_sk#20, c_current_addr_sk#21]
Keys [2]: [c_customer_sk#20, c_current_addr_sk#21]
Functions: []
-(26) CometColumnarExchange
+(26) CometExchange
Input [2]: [c_customer_sk#20, c_current_addr_sk#21]
-Arguments: hashpartitioning(c_customer_sk#20, c_current_addr_sk#21, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(c_customer_sk#20, c_current_addr_sk#21, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(27) CometHashAggregate
Input [2]: [c_customer_sk#20, c_current_addr_sk#21]
@@ -293,9 +293,9 @@ Input [2]: [c_customer_sk#20, ss_ext_sales_price#23]
Keys [1]: [c_customer_sk#20]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#23))]
-(50) CometColumnarExchange
+(50) CometExchange
Input [2]: [c_customer_sk#20, sum#37]
-Arguments: hashpartitioning(c_customer_sk#20, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(c_customer_sk#20, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(51) CometHashAggregate
Input [2]: [c_customer_sk#20, sum#37]
@@ -307,9 +307,9 @@ Input [1]: [segment#38]
Keys [1]: [segment#38]
Functions [1]: [partial_count(1)]
-(53) CometColumnarExchange
+(53) CometExchange
Input [2]: [segment#38, count#39]
-Arguments: hashpartitioning(segment#38, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(segment#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(54) CometHashAggregate
Input [2]: [segment#38, count#39]
@@ -390,7 +390,7 @@ Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))
Subquery:4 Hosting operator id = 63 Hosting Expression = Subquery scalar-subquery#33, [id=#34]
* ColumnarToRow (73)
+- CometHashAggregate (72)
- +- CometColumnarExchange (71)
+ +- CometExchange (71)
+- CometHashAggregate (70)
+- CometProject (69)
+- CometFilter (68)
@@ -417,9 +417,9 @@ Input [1]: [(d_month_seq + 1)#45]
Keys [1]: [(d_month_seq + 1)#45]
Functions: []
-(71) CometColumnarExchange
+(71) CometExchange
Input [1]: [(d_month_seq + 1)#45]
-Arguments: hashpartitioning((d_month_seq + 1)#45, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+Arguments: hashpartitioning((d_month_seq + 1)#45, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
(72) CometHashAggregate
Input [1]: [(d_month_seq + 1)#45]
@@ -432,7 +432,7 @@ Input [1]: [(d_month_seq + 1)#45]
Subquery:5 Hosting operator id = 63 Hosting Expression = Subquery scalar-subquery#35, [id=#36]
* ColumnarToRow (80)
+- CometHashAggregate (79)
- +- CometColumnarExchange (78)
+ +- CometExchange (78)
+- CometHashAggregate (77)
+- CometProject (76)
+- CometFilter (75)
@@ -459,9 +459,9 @@ Input [1]: [(d_month_seq + 3)#49]
Keys [1]: [(d_month_seq + 3)#49]
Functions: []
-(78) CometColumnarExchange
+(78) CometExchange
Input [1]: [(d_month_seq + 3)#49]
-Arguments: hashpartitioning((d_month_seq + 3)#49, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
+Arguments: hashpartitioning((d_month_seq + 3)#49, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=7]
(79) CometHashAggregate
Input [1]: [(d_month_seq + 3)#49]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/simplified.txt
index d3a89a6c..12901866 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q54/simplified.txt
@@ -3,10 +3,10 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [segment,num_customers,segment_base]
CometHashAggregate [segment,num_customers,segment_base,count,count(1)]
- CometColumnarExchange [segment] #1
+ CometExchange [segment] #1
CometHashAggregate [segment,count]
CometHashAggregate [segment,c_customer_sk,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [c_customer_sk] #2
+ CometExchange [c_customer_sk] #2
CometHashAggregate [c_customer_sk,sum,ss_ext_sales_price]
CometProject [c_customer_sk,ss_ext_sales_price]
CometBroadcastHashJoin [c_customer_sk,ss_ext_sales_price,ss_sold_date_sk,d_date_sk]
@@ -17,7 +17,7 @@ WholeStageCodegen (1)
CometProject [c_customer_sk,c_current_addr_sk,ss_ext_sales_price,ss_sold_date_sk]
CometBroadcastHashJoin [c_customer_sk,c_current_addr_sk,ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
CometHashAggregate [c_customer_sk,c_current_addr_sk]
- CometColumnarExchange [c_customer_sk,c_current_addr_sk] #3
+ CometExchange [c_customer_sk,c_current_addr_sk] #3
CometHashAggregate [c_customer_sk,c_current_addr_sk]
CometProject [c_customer_sk,c_current_addr_sk]
CometBroadcastHashJoin [customer_sk,c_customer_sk,c_current_addr_sk]
@@ -67,7 +67,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [(d_month_seq + 1)]
- CometColumnarExchange [(d_month_seq + 1)] #10
+ CometExchange [(d_month_seq + 1)] #10
CometHashAggregate [(d_month_seq + 1)]
CometProject [d_month_seq] [(d_month_seq + 1)]
CometFilter [d_month_seq,d_year,d_moy]
@@ -77,7 +77,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [(d_month_seq + 3)]
- CometColumnarExchange [(d_month_seq + 3)] #11
+ CometExchange [(d_month_seq + 3)] #11
CometHashAggregate [(d_month_seq + 3)]
CometProject [d_month_seq] [(d_month_seq + 3)]
CometFilter [d_month_seq,d_year,d_moy]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/explain.txt
index ee250ed9..6eff9f52 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (19)
+- CometTakeOrderedAndProject (18)
+- CometHashAggregate (17)
- +- CometColumnarExchange (16)
+ +- CometExchange (16)
+- CometHashAggregate (15)
+- CometProject (14)
+- CometBroadcastHashJoin (13)
@@ -93,9 +93,9 @@ Input [3]: [ss_ext_sales_price#5, i_brand_id#8, i_brand#9]
Keys [2]: [i_brand#9, i_brand_id#8]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
-(16) CometColumnarExchange
+(16) CometExchange
Input [3]: [i_brand#9, i_brand_id#8, sum#11]
-Arguments: hashpartitioning(i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
Input [3]: [i_brand#9, i_brand_id#8, sum#11]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/simplified.txt
index 45200509..999c8a6c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q55/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [brand_id,brand,ext_price]
CometHashAggregate [brand_id,brand,ext_price,i_brand,i_brand_id,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_brand,i_brand_id] #1
+ CometExchange [i_brand,i_brand_id] #1
CometHashAggregate [i_brand,i_brand_id,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,i_brand_id,i_brand]
CometBroadcastHashJoin [ss_item_sk,ss_ext_sales_price,i_item_sk,i_brand_id,i_brand]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/explain.txt
index 8061fb91..d9af6324 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/explain.txt
@@ -2,11 +2,11 @@
* ColumnarToRow (61)
+- CometTakeOrderedAndProject (60)
+- CometHashAggregate (59)
- +- CometColumnarExchange (58)
+ +- CometExchange (58)
+- CometHashAggregate (57)
+- CometUnion (56)
:- CometHashAggregate (27)
- : +- CometColumnarExchange (26)
+ : +- CometExchange (26)
: +- CometHashAggregate (25)
: +- CometProject (24)
: +- CometBroadcastHashJoin (23)
@@ -33,7 +33,7 @@
: +- CometFilter (18)
: +- CometScan parquet spark_catalog.default.item (17)
:- CometHashAggregate (41)
- : +- CometColumnarExchange (40)
+ : +- CometExchange (40)
: +- CometHashAggregate (39)
: +- CometProject (38)
: +- CometBroadcastHashJoin (37)
@@ -47,7 +47,7 @@
: : +- ReusedExchange (33)
: +- ReusedExchange (36)
+- CometHashAggregate (55)
- +- CometColumnarExchange (54)
+ +- CometExchange (54)
+- CometHashAggregate (53)
+- CometProject (52)
+- CometBroadcastHashJoin (51)
@@ -183,9 +183,9 @@ Input [2]: [ss_ext_sales_price#3, i_item_id#12]
Keys [1]: [i_item_id#12]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#3))]
-(26) CometColumnarExchange
+(26) CometExchange
Input [2]: [i_item_id#12, sum#15]
-Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(27) CometHashAggregate
Input [2]: [i_item_id#12, sum#15]
@@ -245,9 +245,9 @@ Input [2]: [cs_ext_sales_price#18, i_item_id#24]
Keys [1]: [i_item_id#24]
Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#18))]
-(40) CometColumnarExchange
+(40) CometExchange
Input [2]: [i_item_id#24, sum#25]
-Arguments: hashpartitioning(i_item_id#24, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_item_id#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(41) CometHashAggregate
Input [2]: [i_item_id#24, sum#25]
@@ -307,9 +307,9 @@ Input [2]: [ws_ext_sales_price#28, i_item_id#34]
Keys [1]: [i_item_id#34]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#28))]
-(54) CometColumnarExchange
+(54) CometExchange
Input [2]: [i_item_id#34, sum#35]
-Arguments: hashpartitioning(i_item_id#34, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(i_item_id#34, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(55) CometHashAggregate
Input [2]: [i_item_id#34, sum#35]
@@ -326,9 +326,9 @@ Input [2]: [i_item_id#12, total_sales#36]
Keys [1]: [i_item_id#12]
Functions [1]: [partial_sum(total_sales#36)]
-(58) CometColumnarExchange
+(58) CometExchange
Input [3]: [i_item_id#12, sum#39, isEmpty#40]
-Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(59) CometHashAggregate
Input [3]: [i_item_id#12, sum#39, isEmpty#40]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/simplified.txt
index 4bb76eec..2603a278 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q56/simplified.txt
@@ -3,11 +3,11 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,total_sales]
CometHashAggregate [i_item_id,total_sales,sum,isEmpty,sum(total_sales)]
- CometColumnarExchange [i_item_id] #1
+ CometExchange [i_item_id] #1
CometHashAggregate [i_item_id,sum,isEmpty,total_sales]
CometUnion [i_item_id,total_sales]
CometHashAggregate [i_item_id,total_sales,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_item_id] #2
+ CometExchange [i_item_id] #2
CometHashAggregate [i_item_id,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,i_item_id]
CometBroadcastHashJoin [ss_item_sk,ss_ext_sales_price,i_item_sk,i_item_id]
@@ -42,7 +42,7 @@ WholeStageCodegen (1)
CometFilter [i_item_id,i_color]
CometScan parquet spark_catalog.default.item [i_item_id,i_color]
CometHashAggregate [i_item_id,total_sales,sum,sum(UnscaledValue(cs_ext_sales_price))]
- CometColumnarExchange [i_item_id] #8
+ CometExchange [i_item_id] #8
CometHashAggregate [i_item_id,sum,cs_ext_sales_price]
CometProject [cs_ext_sales_price,i_item_id]
CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,i_item_sk,i_item_id]
@@ -57,7 +57,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk] #5
ReusedExchange [i_item_sk,i_item_id] #6
CometHashAggregate [i_item_id,total_sales,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [i_item_id] #9
+ CometExchange [i_item_id] #9
CometHashAggregate [i_item_id,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,i_item_id]
CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,i_item_sk,i_item_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/explain.txt
index b5372cff..e8a40a0f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/explain.txt
@@ -11,9 +11,9 @@ TakeOrderedAndProject (45)
: : +- Window (24)
: : +- * ColumnarToRow (23)
: : +- CometSort (22)
- : : +- CometColumnarExchange (21)
+ : : +- CometExchange (21)
: : +- CometHashAggregate (20)
- : : +- CometColumnarExchange (19)
+ : : +- CometExchange (19)
: : +- CometHashAggregate (18)
: : +- CometProject (17)
: : +- CometBroadcastHashJoin (16)
@@ -135,18 +135,18 @@ Input [6]: [i_brand#2, i_category#3, cs_sales_price#6, d_year#10, d_moy#11, cc_n
Keys [5]: [i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11]
Functions [1]: [partial_sum(UnscaledValue(cs_sales_price#6))]
-(19) CometColumnarExchange
+(19) CometExchange
Input [6]: [i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11, sum#14]
-Arguments: hashpartitioning(i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(20) CometHashAggregate
Input [6]: [i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11, sum#14]
Keys [5]: [i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11]
Functions [1]: [sum(UnscaledValue(cs_sales_price#6))]
-(21) CometColumnarExchange
+(21) CometExchange
Input [7]: [i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11, sum_sales#15, _w0#16]
-Arguments: hashpartitioning(i_category#3, i_brand#2, cc_name#13, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_category#3, i_brand#2, cc_name#13, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(22) CometSort
Input [7]: [i_category#3, i_brand#2, cc_name#13, d_year#10, d_moy#11, sum_sales#15, _w0#16]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/simplified.txt
index 454d12fa..96a3fdfa 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q57/simplified.txt
@@ -16,9 +16,9 @@ TakeOrderedAndProject [sum_sales,avg_monthly_sales,cc_name,i_category,i_brand,d_
ColumnarToRow
InputAdapter
CometSort [i_category,i_brand,cc_name,d_year,d_moy,sum_sales,_w0]
- CometColumnarExchange [i_category,i_brand,cc_name] #1
+ CometExchange [i_category,i_brand,cc_name] #1
CometHashAggregate [i_category,i_brand,cc_name,d_year,d_moy,sum_sales,_w0,sum,sum(UnscaledValue(cs_sales_price))]
- CometColumnarExchange [i_category,i_brand,cc_name,d_year,d_moy] #2
+ CometExchange [i_category,i_brand,cc_name,d_year,d_moy] #2
CometHashAggregate [i_category,i_brand,cc_name,d_year,d_moy,sum,cs_sales_price]
CometProject [i_brand,i_category,cs_sales_price,d_year,d_moy,cc_name]
CometBroadcastHashJoin [i_brand,i_category,cs_call_center_sk,cs_sales_price,d_year,d_moy,cc_call_center_sk,cc_name]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/explain.txt
index 06e3fb95..f05dbd72 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/explain.txt
@@ -7,7 +7,7 @@
: +- CometBroadcastHashJoin (33)
: :- CometFilter (19)
: : +- CometHashAggregate (18)
- : : +- CometColumnarExchange (17)
+ : : +- CometExchange (17)
: : +- CometHashAggregate (16)
: : +- CometProject (15)
: : +- CometBroadcastHashJoin (14)
@@ -27,7 +27,7 @@
: +- CometBroadcastExchange (32)
: +- CometFilter (31)
: +- CometHashAggregate (30)
- : +- CometColumnarExchange (29)
+ : +- CometExchange (29)
: +- CometHashAggregate (28)
: +- CometProject (27)
: +- CometBroadcastHashJoin (26)
@@ -40,7 +40,7 @@
+- CometBroadcastExchange (47)
+- CometFilter (46)
+- CometHashAggregate (45)
- +- CometColumnarExchange (44)
+ +- CometExchange (44)
+- CometHashAggregate (43)
+- CometProject (42)
+- CometBroadcastHashJoin (41)
@@ -129,9 +129,9 @@ Input [2]: [ss_ext_sales_price#2, i_item_id#6]
Keys [1]: [i_item_id#6]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#2))]
-(17) CometColumnarExchange
+(17) CometExchange
Input [2]: [i_item_id#6, sum#10]
-Arguments: hashpartitioning(i_item_id#6, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#6, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(18) CometHashAggregate
Input [2]: [i_item_id#6, sum#10]
@@ -183,9 +183,9 @@ Input [2]: [cs_ext_sales_price#14, i_item_id#18]
Keys [1]: [i_item_id#18]
Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#14))]
-(29) CometColumnarExchange
+(29) CometExchange
Input [2]: [i_item_id#18, sum#20]
-Arguments: hashpartitioning(i_item_id#18, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_item_id#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(30) CometHashAggregate
Input [2]: [i_item_id#18, sum#20]
@@ -250,9 +250,9 @@ Input [2]: [ws_ext_sales_price#24, i_item_id#28]
Keys [1]: [i_item_id#28]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#24))]
-(44) CometColumnarExchange
+(44) CometExchange
Input [2]: [i_item_id#28, sum#30]
-Arguments: hashpartitioning(i_item_id#28, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(i_item_id#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(45) CometHashAggregate
Input [2]: [i_item_id#28, sum#30]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/simplified.txt
index 326fdcb7..36fc8ac5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q58/simplified.txt
@@ -8,7 +8,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [item_id,ss_item_rev,item_id,cs_item_rev]
CometFilter [item_id,ss_item_rev]
CometHashAggregate [item_id,ss_item_rev,i_item_id,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_item_id] #1
+ CometExchange [i_item_id] #1
CometHashAggregate [i_item_id,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,i_item_id]
CometBroadcastHashJoin [ss_ext_sales_price,ss_sold_date_sk,i_item_id,d_date_sk]
@@ -48,7 +48,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [item_id,cs_item_rev] #6
CometFilter [item_id,cs_item_rev]
CometHashAggregate [item_id,cs_item_rev,i_item_id,sum,sum(UnscaledValue(cs_ext_sales_price))]
- CometColumnarExchange [i_item_id] #7
+ CometExchange [i_item_id] #7
CometHashAggregate [i_item_id,sum,cs_ext_sales_price]
CometProject [cs_ext_sales_price,i_item_id]
CometBroadcastHashJoin [cs_ext_sales_price,cs_sold_date_sk,i_item_id,d_date_sk]
@@ -62,7 +62,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [item_id,ws_item_rev] #8
CometFilter [item_id,ws_item_rev]
CometHashAggregate [item_id,ws_item_rev,i_item_id,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [i_item_id] #9
+ CometExchange [i_item_id] #9
CometHashAggregate [i_item_id,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,i_item_id]
CometBroadcastHashJoin [ws_ext_sales_price,ws_sold_date_sk,i_item_id,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/explain.txt
index 51afdb3c..a3f89c9d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/explain.txt
@@ -8,7 +8,7 @@
: :- CometProject (15)
: : +- CometBroadcastHashJoin (14)
: : :- CometHashAggregate (10)
- : : : +- CometColumnarExchange (9)
+ : : : +- CometExchange (9)
: : : +- CometHashAggregate (8)
: : : +- CometProject (7)
: : : +- CometBroadcastHashJoin (6)
@@ -81,9 +81,9 @@ Input [4]: [ss_store_sk#1, ss_sales_price#2, d_week_seq#5, d_day_name#6]
Keys [2]: [d_week_seq#5, ss_store_sk#1]
Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (d_day_name#6 = Sunday ) THEN ss_sales_price#2 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#6 = Monday ) THEN ss_sales_price#2 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#6 = Tuesday ) THEN ss_sales_price#2 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#6 = Wednesday) THEN ss_sales_price#2 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#6 = Thursday ) THEN ss_sales_price#2 END)), partial_sum(Unsc [...]
-(9) CometColumnarExchange
+(9) CometExchange
Input [9]: [d_week_seq#5, ss_store_sk#1, sum#7, sum#8, sum#9, sum#10, sum#11, sum#12, sum#13]
-Arguments: hashpartitioning(d_week_seq#5, ss_store_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(d_week_seq#5, ss_store_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(10) CometHashAggregate
Input [9]: [d_week_seq#5, ss_store_sk#1, sum#7, sum#8, sum#9, sum#10, sum#11, sum#12, sum#13]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/simplified.txt
index bb7a8581..140a7643 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q59/simplified.txt
@@ -9,7 +9,7 @@ WholeStageCodegen (1)
CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,s_store_id,s_store_name]
CometBroadcastHashJoin [d_week_seq,ss_store_sk,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,s_store_sk,s_store_id,s_store_name]
CometHashAggregate [d_week_seq,ss_store_sk,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN ss_sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN ss_sales_price END)),sum(UnscaledVal [...]
- CometColumnarExchange [d_week_seq,ss_store_sk] #1
+ CometExchange [d_week_seq,ss_store_sk] #1
CometHashAggregate [d_week_seq,ss_store_sk,sum,sum,sum,sum,sum,sum,sum,d_day_name,ss_sales_price]
CometProject [ss_store_sk,ss_sales_price,d_week_seq,d_day_name]
CometBroadcastHashJoin [ss_store_sk,ss_sales_price,ss_sold_date_sk,d_date_sk,d_week_seq,d_day_name]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/explain.txt
index 0341c20d..7a804466 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/explain.txt
@@ -3,7 +3,7 @@
+- CometTakeOrderedAndProject (37)
+- CometFilter (36)
+- CometHashAggregate (35)
- +- CometColumnarExchange (34)
+ +- CometExchange (34)
+- CometHashAggregate (33)
+- CometProject (32)
+- CometBroadcastHashJoin (31)
@@ -33,7 +33,7 @@
+- CometBroadcastExchange (27)
+- CometFilter (26)
+- CometHashAggregate (25)
- +- CometColumnarExchange (24)
+ +- CometExchange (24)
+- CometHashAggregate (23)
+- CometFilter (22)
+- CometScan parquet spark_catalog.default.item (21)
@@ -154,9 +154,9 @@ Input [2]: [i_current_price#16, i_category#17]
Keys [1]: [i_category#17]
Functions [1]: [partial_avg(UnscaledValue(i_current_price#16))]
-(24) CometColumnarExchange
+(24) CometExchange
Input [3]: [i_category#17, sum#18, count#19]
-Arguments: hashpartitioning(i_category#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_category#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(25) CometHashAggregate
Input [3]: [i_category#17, sum#18, count#19]
@@ -198,9 +198,9 @@ Input [1]: [ca_state#2]
Keys [1]: [ca_state#2]
Functions [1]: [partial_count(1)]
-(34) CometColumnarExchange
+(34) CometExchange
Input [2]: [ca_state#2, count#21]
-Arguments: hashpartitioning(ca_state#2, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(ca_state#2, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(35) CometHashAggregate
Input [2]: [ca_state#2, count#21]
@@ -253,7 +253,7 @@ Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))
Subquery:2 Hosting operator id = 40 Hosting Expression = Subquery scalar-subquery#11, [id=#12]
* ColumnarToRow (50)
+- CometHashAggregate (49)
- +- CometColumnarExchange (48)
+ +- CometExchange (48)
+- CometHashAggregate (47)
+- CometProject (46)
+- CometFilter (45)
@@ -280,9 +280,9 @@ Input [1]: [d_month_seq#24]
Keys [1]: [d_month_seq#24]
Functions: []
-(48) CometColumnarExchange
+(48) CometExchange
Input [1]: [d_month_seq#24]
-Arguments: hashpartitioning(d_month_seq#24, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(d_month_seq#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(49) CometHashAggregate
Input [1]: [d_month_seq#24]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/simplified.txt
index 0e983b96..05566cbc 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q6/simplified.txt
@@ -4,7 +4,7 @@ WholeStageCodegen (1)
CometTakeOrderedAndProject [state,cnt]
CometFilter [state,cnt]
CometHashAggregate [state,cnt,ca_state,count,count(1)]
- CometColumnarExchange [ca_state] #1
+ CometExchange [ca_state] #1
CometHashAggregate [ca_state,count]
CometProject [ca_state]
CometBroadcastHashJoin [ca_state,ss_item_sk,i_item_sk]
@@ -34,7 +34,7 @@ WholeStageCodegen (1)
ColumnarToRow
InputAdapter
CometHashAggregate [d_month_seq]
- CometColumnarExchange [d_month_seq] #5
+ CometExchange [d_month_seq] #5
CometHashAggregate [d_month_seq]
CometProject [d_month_seq]
CometFilter [d_month_seq,d_year,d_moy]
@@ -53,7 +53,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [avg(i_current_price),i_category] #8
CometFilter [avg(i_current_price),i_category]
CometHashAggregate [avg(i_current_price),i_category,sum,count,avg(UnscaledValue(i_current_price))]
- CometColumnarExchange [i_category] #9
+ CometExchange [i_category] #9
CometHashAggregate [i_category,sum,count,i_current_price]
CometFilter [i_current_price,i_category]
CometScan parquet spark_catalog.default.item [i_current_price,i_category]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/explain.txt
index 0aeb4e2b..3b510192 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/explain.txt
@@ -2,11 +2,11 @@
* ColumnarToRow (61)
+- CometTakeOrderedAndProject (60)
+- CometHashAggregate (59)
- +- CometColumnarExchange (58)
+ +- CometExchange (58)
+- CometHashAggregate (57)
+- CometUnion (56)
:- CometHashAggregate (27)
- : +- CometColumnarExchange (26)
+ : +- CometExchange (26)
: +- CometHashAggregate (25)
: +- CometProject (24)
: +- CometBroadcastHashJoin (23)
@@ -33,7 +33,7 @@
: +- CometFilter (18)
: +- CometScan parquet spark_catalog.default.item (17)
:- CometHashAggregate (41)
- : +- CometColumnarExchange (40)
+ : +- CometExchange (40)
: +- CometHashAggregate (39)
: +- CometProject (38)
: +- CometBroadcastHashJoin (37)
@@ -47,7 +47,7 @@
: : +- ReusedExchange (33)
: +- ReusedExchange (36)
+- CometHashAggregate (55)
- +- CometColumnarExchange (54)
+ +- CometExchange (54)
+- CometHashAggregate (53)
+- CometProject (52)
+- CometBroadcastHashJoin (51)
@@ -183,9 +183,9 @@ Input [2]: [ss_ext_sales_price#3, i_item_id#12]
Keys [1]: [i_item_id#12]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#3))]
-(26) CometColumnarExchange
+(26) CometExchange
Input [2]: [i_item_id#12, sum#15]
-Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(27) CometHashAggregate
Input [2]: [i_item_id#12, sum#15]
@@ -245,9 +245,9 @@ Input [2]: [cs_ext_sales_price#18, i_item_id#24]
Keys [1]: [i_item_id#24]
Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#18))]
-(40) CometColumnarExchange
+(40) CometExchange
Input [2]: [i_item_id#24, sum#25]
-Arguments: hashpartitioning(i_item_id#24, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_item_id#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(41) CometHashAggregate
Input [2]: [i_item_id#24, sum#25]
@@ -307,9 +307,9 @@ Input [2]: [ws_ext_sales_price#28, i_item_id#34]
Keys [1]: [i_item_id#34]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#28))]
-(54) CometColumnarExchange
+(54) CometExchange
Input [2]: [i_item_id#34, sum#35]
-Arguments: hashpartitioning(i_item_id#34, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(i_item_id#34, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(55) CometHashAggregate
Input [2]: [i_item_id#34, sum#35]
@@ -326,9 +326,9 @@ Input [2]: [i_item_id#12, total_sales#36]
Keys [1]: [i_item_id#12]
Functions [1]: [partial_sum(total_sales#36)]
-(58) CometColumnarExchange
+(58) CometExchange
Input [3]: [i_item_id#12, sum#39, isEmpty#40]
-Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(i_item_id#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(59) CometHashAggregate
Input [3]: [i_item_id#12, sum#39, isEmpty#40]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/simplified.txt
index 871f477b..4dd8b3b5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q60/simplified.txt
@@ -3,11 +3,11 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [i_item_id,total_sales]
CometHashAggregate [i_item_id,total_sales,sum,isEmpty,sum(total_sales)]
- CometColumnarExchange [i_item_id] #1
+ CometExchange [i_item_id] #1
CometHashAggregate [i_item_id,sum,isEmpty,total_sales]
CometUnion [i_item_id,total_sales]
CometHashAggregate [i_item_id,total_sales,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange [i_item_id] #2
+ CometExchange [i_item_id] #2
CometHashAggregate [i_item_id,sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price,i_item_id]
CometBroadcastHashJoin [ss_item_sk,ss_ext_sales_price,i_item_sk,i_item_id]
@@ -42,7 +42,7 @@ WholeStageCodegen (1)
CometFilter [i_item_id,i_category]
CometScan parquet spark_catalog.default.item [i_item_id,i_category]
CometHashAggregate [i_item_id,total_sales,sum,sum(UnscaledValue(cs_ext_sales_price))]
- CometColumnarExchange [i_item_id] #8
+ CometExchange [i_item_id] #8
CometHashAggregate [i_item_id,sum,cs_ext_sales_price]
CometProject [cs_ext_sales_price,i_item_id]
CometBroadcastHashJoin [cs_item_sk,cs_ext_sales_price,i_item_sk,i_item_id]
@@ -57,7 +57,7 @@ WholeStageCodegen (1)
ReusedExchange [ca_address_sk] #5
ReusedExchange [i_item_sk,i_item_id] #6
CometHashAggregate [i_item_id,total_sales,sum,sum(UnscaledValue(ws_ext_sales_price))]
- CometColumnarExchange [i_item_id] #9
+ CometExchange [i_item_id] #9
CometHashAggregate [i_item_id,sum,ws_ext_sales_price]
CometProject [ws_ext_sales_price,i_item_id]
CometBroadcastHashJoin [ws_item_sk,ws_ext_sales_price,i_item_sk,i_item_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/explain.txt
index 3bbacfbb..8fcd9439 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/explain.txt
@@ -3,7 +3,7 @@
+- * BroadcastNestedLoopJoin Inner BuildRight (64)
:- * ColumnarToRow (41)
: +- CometHashAggregate (40)
- : +- CometColumnarExchange (39)
+ : +- CometExchange (39)
: +- CometHashAggregate (38)
: +- CometProject (37)
: +- CometBroadcastHashJoin (36)
@@ -45,7 +45,7 @@
+- BroadcastExchange (63)
+- * ColumnarToRow (62)
+- CometHashAggregate (61)
- +- CometColumnarExchange (60)
+ +- CometExchange (60)
+- CometHashAggregate (59)
+- CometProject (58)
+- CometBroadcastHashJoin (57)
@@ -247,9 +247,9 @@ Input [1]: [ss_ext_sales_price#5]
Keys: []
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
-(39) CometColumnarExchange
+(39) CometExchange
Input [1]: [sum#23]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(40) CometHashAggregate
Input [1]: [sum#23]
@@ -336,9 +336,9 @@ Input [1]: [ss_ext_sales_price#28]
Keys: []
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#28))]
-(60) CometColumnarExchange
+(60) CometExchange
Input [1]: [sum#37]
-Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(61) CometHashAggregate
Input [1]: [sum#37]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/simplified.txt
index b95b0d61..54861441 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q61/simplified.txt
@@ -4,7 +4,7 @@ WholeStageCodegen (2)
ColumnarToRow
InputAdapter
CometHashAggregate [promotions,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange #1
+ CometExchange #1
CometHashAggregate [sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price]
CometBroadcastHashJoin [ss_item_sk,ss_ext_sales_price,i_item_sk]
@@ -57,7 +57,7 @@ WholeStageCodegen (2)
ColumnarToRow
InputAdapter
CometHashAggregate [total,sum,sum(UnscaledValue(ss_ext_sales_price))]
- CometColumnarExchange #10
+ CometExchange #10
CometHashAggregate [sum,ss_ext_sales_price]
CometProject [ss_ext_sales_price]
CometBroadcastHashJoin [ss_item_sk,ss_ext_sales_price,i_item_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/explain.txt
index f5841110..82005f93 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/explain.txt
@@ -2,7 +2,7 @@
* ColumnarToRow (28)
+- CometTakeOrderedAndProject (27)
+- CometHashAggregate (26)
- +- CometColumnarExchange (25)
+ +- CometExchange (25)
+- CometHashAggregate (24)
+- CometProject (23)
+- CometBroadcastHashJoin (22)
@@ -145,9 +145,9 @@ Input [5]: [ws_ship_date_sk#1, ws_sold_date_sk#5, sm_type#9, web_name#11, _group
Keys [3]: [_groupingexpression#14, sm_type#9, web_name#11]
Functions [5]: [partial_sum(CASE WHEN ((ws_ship_date_sk#1 - ws_sold_date_sk#5) <= 30) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (((ws_ship_date_sk#1 - ws_sold_date_sk#5) > 30) AND ((ws_ship_date_sk#1 - ws_sold_date_sk#5) <= 60)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (((ws_ship_date_sk#1 - ws_sold_date_sk#5) > 60) AND ((ws_ship_date_sk#1 - ws_sold_date_sk#5) <= 90)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (((ws_ship_date_sk#1 - ws_sold_date_sk#5) > 90) AND ((ws_ship_date_sk#1 - ws_s [...]
-(25) CometColumnarExchange
+(25) CometExchange
Input [8]: [_groupingexpression#14, sm_type#9, web_name#11, sum#15, sum#16, sum#17, sum#18, sum#19]
-Arguments: hashpartitioning(_groupingexpression#14, sm_type#9, web_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(_groupingexpression#14, sm_type#9, web_name#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(26) CometHashAggregate
Input [8]: [_groupingexpression#14, sm_type#9, web_name#11, sum#15, sum#16, sum#17, sum#18, sum#19]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/simplified.txt
index c7bcf72f..6a9b1ced 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q62/simplified.txt
@@ -3,7 +3,7 @@ WholeStageCodegen (1)
InputAdapter
CometTakeOrderedAndProject [substr(w_warehouse_name, 1, 20),sm_type,web_name,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
CometHashAggregate [substr(w_warehouse_name, 1, 20),sm_type,web_name,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,_groupingexpression,sum,sum,sum,sum,sum,sum(CASE WHEN ((ws_ship_date_sk - ws_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((ws_ship_date_sk - ws_sold_date_sk) > 30) AND ((ws_ship_date_sk - ws_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((ws_ship_date_sk - ws_sold_date_sk) > 60) AND ((ws_ship_date_sk - ws_sold_date_sk) <= 90)) TH [...]
- CometColumnarExchange [_groupingexpression,sm_type,web_name] #1
+ CometExchange [_groupingexpression,sm_type,web_name] #1
CometHashAggregate [_groupingexpression,sm_type,web_name,sum,sum,sum,sum,sum,ws_ship_date_sk,ws_sold_date_sk]
CometProject [w_warehouse_name] [ws_ship_date_sk,ws_sold_date_sk,sm_type,web_name,_groupingexpression]
CometBroadcastHashJoin [ws_ship_date_sk,ws_sold_date_sk,w_warehouse_name,sm_type,web_name,d_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/explain.txt
index 5fd26366..4dfd50a3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/explain.txt
@@ -5,9 +5,9 @@ TakeOrderedAndProject (29)
+- Window (26)
+- * ColumnarToRow (25)
+- CometSort (24)
- +- CometColumnarExchange (23)
+ +- CometExchange (23)
+- CometHashAggregate (22)
- +- CometColumnarExchange (21)
+ +- CometExchange (21)
+- CometHashAggregate (20)
+- CometProject (19)
+- CometBroadcastHashJoin (18)
@@ -127,18 +127,18 @@ Input [3]: [i_manager_id#5, ss_sales_price#12, d_moy#17]
Keys [2]: [i_manager_id#5, d_moy#17]
Functions [1]: [partial_sum(UnscaledValue(ss_sales_price#12))]
-(21) CometColumnarExchange
+(21) CometExchange
Input [3]: [i_manager_id#5, d_moy#17, sum#19]
-Arguments: hashpartitioning(i_manager_id#5, d_moy#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(i_manager_id#5, d_moy#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(22) CometHashAggregate
Input [3]: [i_manager_id#5, d_moy#17, sum#19]
Keys [2]: [i_manager_id#5, d_moy#17]
Functions [1]: [sum(UnscaledValue(ss_sales_price#12))]
-(23) CometColumnarExchange
+(23) CometExchange
Input [3]: [i_manager_id#5, sum_sales#20, _w0#21]
-Arguments: hashpartitioning(i_manager_id#5, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(i_manager_id#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(24) CometSort
Input [3]: [i_manager_id#5, sum_sales#20, _w0#21]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/simplified.txt
index 1b538c9a..8ea16ea7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q63/simplified.txt
@@ -8,9 +8,9 @@ TakeOrderedAndProject [i_manager_id,avg_monthly_sales,sum_sales]
ColumnarToRow
InputAdapter
CometSort [i_manager_id,sum_sales,_w0]
- CometColumnarExchange [i_manager_id] #1
+ CometExchange [i_manager_id] #1
CometHashAggregate [i_manager_id,sum_sales,_w0,d_moy,sum,sum(UnscaledValue(ss_sales_price))]
- CometColumnarExchange [i_manager_id,d_moy] #2
+ CometExchange [i_manager_id,d_moy] #2
CometHashAggregate [i_manager_id,d_moy,sum,ss_sales_price]
CometProject [i_manager_id,ss_sales_price,d_moy]
CometBroadcastHashJoin [i_manager_id,ss_store_sk,ss_sales_price,d_moy,s_store_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q64/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q64/explain.txt
index b4e05a86..35bc1885 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q64/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q64/explain.txt
@@ -7,7 +7,7 @@
+- * SortMergeJoin Inner (171)
:- * ColumnarToRow (101)
: +- CometSort (100)
- : +- CometColumnarExchange (99)
+ : +- CometExchange (99)
: +- CometHashAggregate (98)
: +- CometHashAggregate (97)
: +- CometProject (96)
@@ -43,7 +43,7 @@
: : : : : : : : : : : : : : : :- CometProject (30)
: : : : : : : : : : : : : : : : +- CometSortMergeJoin (29)
: : : : : : : : : : : : : : : : :- CometSort (10)
- : : : : : : : : : : : : : : : : : +- CometColumnarExchange (9)
+ : : : : : : : : : : : : : : : : : +- CometExchange (9)
: : : : : : : : : : : : : : : : : +- CometProject (8)
: : : : : : : : : : : : : : : : : +- CometBroadcastHashJoin (7)
: : : : : : : : : : : : : : : : : :- CometBroadcastExchange (3)
@@ -56,17 +56,17 @@
: : : : : : : : : : : : : : : : +- CometProject (27)
: : : : : : : : : : : : : : : : +- CometFilter (26)
: : : : : : : : : : : : : : : : +- CometHashAggregate (25)
- : : : : : : : : : : : : : : : : +- CometColumnarExchange (24)
+ : : : : : : : : : : : : : : : : +- CometExchange (24)
: : : : : : : : : : : : : : : : +- CometHashAggregate (23)
: : : : : : : : : : : : : : : : +- CometProject (22)
: : : : : : : : : : : : : : : : +- CometSortMergeJoin (21)
: : : : : : : : : : : : : : : : :- CometSort (15)
- : : : : : : : : : : : : : : : : : +- CometColumnarExchange (14)
+ : : : : : : : : : : : : : : : : : +- CometExchange (14)
: : : : : : : : : : : : : : : : : +- CometProject (13)
: : : : : : : : : : : : : : : : : +- CometFilter (12)
: : : : : : : : : : : : : : : : : +- CometScan parquet spark_catalog.default.catalog_sales (11)
: : : : : : : : : : : : : : : : +- CometSort (20)
- : : : : : : : : : : : : : : : : +- CometColumnarExchange (19)
+ : : : : : : : : : : : : : : : : +- CometExchange (19)
: : : : : : : : : : : : : : : : +- CometProject (18)
: : : : : : : : : : : : : : : : +- CometFilter (17)
: : : : : : : : : : : : : : : : +- CometScan parquet spark_catalog.default.catalog_returns (16)
@@ -108,7 +108,7 @@
: +- CometScan parquet spark_catalog.default.item (91)
+- * ColumnarToRow (170)
+- CometSort (169)
- +- CometColumnarExchange (168)
+ +- CometExchange (168)
+- CometHashAggregate (167)
+- CometHashAggregate (166)
+- CometProject (165)
@@ -144,7 +144,7 @@
: : : : : : : : : : : : : : :- CometProject (118)
: : : : : : : : : : : : : : : +- CometSortMergeJoin (117)
: : : : : : : : : : : : : : : :- CometSort (111)
- : : : : : : : : : : : : : : : : +- CometColumnarExchange (110)
+ : : : : : : : : : : : : : : : : +- CometExchange (110)
: : : : : : : : : : : : : : : : +- CometProject (109)
: : : : : : : : : : : : : : : : +- CometBroadcastHashJoin (108)
: : : : : : : : : : : : : : : : :- CometBroadcastExchange (104)
@@ -217,9 +217,9 @@ Arguments: [ss_item_sk#1, ss_ticket_number#8], [sr_item_sk#14, sr_ticket_number#
Input [14]: [ss_item_sk#1, ss_customer_sk#2, ss_cdemo_sk#3, ss_hdemo_sk#4, ss_addr_sk#5, ss_store_sk#6, ss_promo_sk#7, ss_ticket_number#8, ss_wholesale_cost#9, ss_list_price#10, ss_coupon_amt#11, ss_sold_date_sk#12, sr_item_sk#14, sr_ticket_number#15]
Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_cdemo_sk#3, ss_hdemo_sk#4, ss_addr_sk#5, ss_store_sk#6, ss_promo_sk#7, ss_wholesale_cost#9, ss_list_price#10, ss_coupon_amt#11, ss_sold_date_sk#12], [ss_item_sk#1, ss_customer_sk#2, ss_cdemo_sk#3, ss_hdemo_sk#4, ss_addr_sk#5, ss_store_sk#6, ss_promo_sk#7, ss_wholesale_cost#9, ss_list_price#10, ss_coupon_amt#11, ss_sold_date_sk#12]
-(9) CometColumnarExchange
+(9) CometExchange
Input [11]: [ss_item_sk#1, ss_customer_sk#2, ss_cdemo_sk#3, ss_hdemo_sk#4, ss_addr_sk#5, ss_store_sk#6, ss_promo_sk#7, ss_wholesale_cost#9, ss_list_price#10, ss_coupon_amt#11, ss_sold_date_sk#12]
-Arguments: hashpartitioning(ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=1]
+Arguments: hashpartitioning(ss_item_sk#1, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(10) CometSort
Input [11]: [ss_item_sk#1, ss_customer_sk#2, ss_cdemo_sk#3, ss_hdemo_sk#4, ss_addr_sk#5, ss_store_sk#6, ss_promo_sk#7, ss_wholesale_cost#9, ss_list_price#10, ss_coupon_amt#11, ss_sold_date_sk#12]
@@ -240,9 +240,9 @@ Condition : (isnotnull(cs_item_sk#17) AND isnotnull(cs_order_number#18))
Input [4]: [cs_item_sk#17, cs_order_number#18, cs_ext_list_price#19, cs_sold_date_sk#20]
Arguments: [cs_item_sk#17, cs_order_number#18, cs_ext_list_price#19], [cs_item_sk#17, cs_order_number#18, cs_ext_list_price#19]
-(14) CometColumnarExchange
+(14) CometExchange
Input [3]: [cs_item_sk#17, cs_order_number#18, cs_ext_list_price#19]
-Arguments: hashpartitioning(cs_item_sk#17, cs_order_number#18, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+Arguments: hashpartitioning(cs_item_sk#17, cs_order_number#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
(15) CometSort
Input [3]: [cs_item_sk#17, cs_order_number#18, cs_ext_list_price#19]
@@ -263,9 +263,9 @@ Condition : (isnotnull(cr_item_sk#21) AND isnotnull(cr_order_number#22))
Input [6]: [cr_item_sk#21, cr_order_number#22, cr_refunded_cash#23, cr_reversed_charge#24, cr_store_credit#25, cr_returned_date_sk#26]
Arguments: [cr_item_sk#21, cr_order_number#22, cr_refunded_cash#23, cr_reversed_charge#24, cr_store_credit#25], [cr_item_sk#21, cr_order_number#22, cr_refunded_cash#23, cr_reversed_charge#24, cr_store_credit#25]
-(19) CometColumnarExchange
+(19) CometExchange
Input [5]: [cr_item_sk#21, cr_order_number#22, cr_refunded_cash#23, cr_reversed_charge#24, cr_store_credit#25]
-Arguments: hashpartitioning(cr_item_sk#21, cr_order_number#22, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=3]
+Arguments: hashpartitioning(cr_item_sk#21, cr_order_number#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
(20) CometSort
Input [5]: [cr_item_sk#21, cr_order_number#22, cr_refunded_cash#23, cr_reversed_charge#24, cr_store_credit#25]
@@ -285,9 +285,9 @@ Input [5]: [cs_item_sk#17, cs_ext_list_price#19, cr_refunded_cash#23, cr_reverse
Keys [1]: [cs_item_sk#17]
Functions [2]: [partial_sum(UnscaledValue(cs_ext_list_price#19)), partial_sum(((cr_refunded_cash#23 + cr_reversed_charge#24) + cr_store_credit#25))]
-(24) CometColumnarExchange
+(24) CometExchange
Input [4]: [cs_item_sk#17, sum#27, sum#28, isEmpty#29]
-Arguments: hashpartitioning(cs_item_sk#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+Arguments: hashpartitioning(cs_item_sk#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
(25) CometHashAggregate
Input [4]: [cs_item_sk#17, sum#27, sum#28, isEmpty#29]
@@ -629,9 +629,9 @@ Input [19]: [i_product_name#71, i_item_sk#68, s_store_name#35, s_zip#36, ca_stre
Keys [15]: [i_product_name#71, i_item_sk#68, s_store_name#35, s_zip#36, ca_street_number#57, ca_street_name#58, ca_city#59, ca_zip#60, ca_street_number#62, ca_street_name#63, ca_city#64, ca_zip#65, d_year#33, d_year#44, d_year#46]
Functions [4]: [count(1), sum(UnscaledValue(ss_wholesale_cost#9)), sum(UnscaledValue(ss_list_price#10)), sum(UnscaledValue(ss_coupon_amt#11))]
-(99) CometColumnarExchange
+(99) CometExchange
Input [17]: [product_name#76, item_sk#77, store_name#78, store_zip#79, b_street_number#80, b_streen_name#81, b_city#82, b_zip#83, c_street_number#84, c_street_name#85, c_city#86, c_zip#87, syear#88, cnt#89, s1#90, s2#91, s3#92]
-Arguments: hashpartitioning(item_sk#77, store_name#78, store_zip#79, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+Arguments: hashpartitioning(item_sk#77, store_name#78, store_zip#79, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
(100) CometSort
Input [17]: [product_name#76, item_sk#77, store_name#78, store_zip#79, b_street_number#80, b_streen_name#81, b_city#82, b_zip#83, c_street_number#84, c_street_name#85, c_city#86, c_zip#87, syear#88, cnt#89, s1#90, s2#91, s3#92]
... 103247 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org