You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ag...@apache.org on 2025/03/11 18:18:59 UTC
(datafusion-comet) branch main updated: chore: Stop disabling readside padding in TPC stability suite (#1491)
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 1be8c5242 chore: Stop disabling readside padding in TPC stability suite (#1491)
1be8c5242 is described below
commit 1be8c5242084bdf9c9784d85f88ceca93273b85b
Author: Andy Grove <ag...@apache.org>
AuthorDate: Tue Mar 11 12:18:54 2025 -0600
chore: Stop disabling readside padding in TPC stability suite (#1491)
---
.../approved-plans-v1_4-spark3_5/q1/explain.txt | 67 +-
.../approved-plans-v1_4-spark3_5/q1/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q10/explain.txt | 99 +-
.../q10/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q11/explain.txt | 680 ++++++------
.../q11/simplified.txt | 20 +-
.../approved-plans-v1_4-spark3_5/q12/explain.txt | 183 +--
.../q12/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q13/explain.txt | 167 +--
.../q13/simplified.txt | 7 +-
.../approved-plans-v1_4-spark3_5/q15/explain.txt | 151 +--
.../q15/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q16/explain.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q17/explain.txt | 164 +--
.../q17/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q18/explain.txt | 208 ++--
.../q18/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q19/explain.txt | 160 +--
.../q19/simplified.txt | 12 +-
.../approved-plans-v1_4-spark3_5/q2/explain.txt | 189 ++--
.../approved-plans-v1_4-spark3_5/q2/simplified.txt | 9 +-
.../approved-plans-v1_4-spark3_5/q20/explain.txt | 183 +--
.../q20/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q21/explain.txt | 68 +-
.../q21/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q22/explain.txt | 135 +--
.../q22/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q23b/explain.txt | 563 +++++-----
.../q23b/simplified.txt | 23 +-
.../approved-plans-v1_4-spark3_5/q24a/explain.txt | 532 ++++-----
.../q24a/simplified.txt | 22 +-
.../approved-plans-v1_4-spark3_5/q24b/explain.txt | 532 ++++-----
.../q24b/simplified.txt | 22 +-
.../approved-plans-v1_4-spark3_5/q25/explain.txt | 152 +--
.../q25/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q26/explain.txt | 139 +--
.../q26/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q27/explain.txt | 148 +--
.../q27/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q29/explain.txt | 172 +--
.../q29/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q3/explain.txt | 30 +-
.../approved-plans-v1_4-spark3_5/q3/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q30/explain.txt | 422 +++----
.../q30/simplified.txt | 58 +-
.../approved-plans-v1_4-spark3_5/q33/explain.txt | 3 +-
.../approved-plans-v1_4-spark3_5/q34/explain.txt | 73 +-
.../q34/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q35/explain.txt | 150 +--
.../q35/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q36/explain.txt | 177 +--
.../q36/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q37/explain.txt | 110 +-
.../q37/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q38/explain.txt | 317 +++---
.../q38/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q4/explain.txt | 986 ++++++++--------
.../approved-plans-v1_4-spark3_5/q4/simplified.txt | 30 +-
.../approved-plans-v1_4-spark3_5/q40/explain.txt | 185 ++--
.../q40/simplified.txt | 7 +-
.../approved-plans-v1_4-spark3_5/q41/explain.txt | 59 +-
.../q41/simplified.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q42/explain.txt | 30 +-
.../q42/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q43/explain.txt | 70 +-
.../q43/simplified.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q44/explain.txt | 117 +-
.../q44/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q45/explain.txt | 266 ++---
.../q45/simplified.txt | 12 +-
.../approved-plans-v1_4-spark3_5/q46/explain.txt | 89 +-
.../q46/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q47/explain.txt | 397 +++----
.../q47/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q48/explain.txt | 175 +--
.../q48/simplified.txt | 7 +-
.../approved-plans-v1_4-spark3_5/q5/explain.txt | 535 ++++-----
.../approved-plans-v1_4-spark3_5/q5/simplified.txt | 15 +-
.../approved-plans-v1_4-spark3_5/q50/explain.txt | 189 ++--
.../q50/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q52/explain.txt | 30 +-
.../q52/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q53/explain.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q54/explain.txt | 346 +++---
.../q54/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q55/explain.txt | 30 +-
.../q55/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q56/explain.txt | 383 +++----
.../q56/simplified.txt | 17 +-
.../approved-plans-v1_4-spark3_5/q57/explain.txt | 397 +++----
.../q57/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q58/explain.txt | 507 ++++-----
.../q58/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q59/explain.txt | 295 ++---
.../q59/simplified.txt | 19 +-
.../approved-plans-v1_4-spark3_5/q6/explain.txt | 417 +++----
.../approved-plans-v1_4-spark3_5/q6/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q60/explain.txt | 384 +++----
.../q60/simplified.txt | 17 +-
.../approved-plans-v1_4-spark3_5/q61/explain.txt | 6 +-
.../approved-plans-v1_4-spark3_5/q62/explain.txt | 155 +--
.../q62/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q63/explain.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q64/explain.txt | 1101 +++++++++---------
.../q64/simplified.txt | 17 +-
.../approved-plans-v1_4-spark3_5/q65/explain.txt | 195 ++--
.../q65/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q66/explain.txt | 411 +++----
.../q66/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q67/explain.txt | 202 ++--
.../q67/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q68/explain.txt | 89 +-
.../q68/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q69/explain.txt | 101 +-
.../q69/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q7/explain.txt | 139 +--
.../approved-plans-v1_4-spark3_5/q7/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q70/explain.txt | 282 ++---
.../q70/simplified.txt | 62 +-
.../approved-plans-v1_4-spark3_5/q71/explain.txt | 164 +--
.../q71/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q72/explain.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q73/explain.txt | 71 +-
.../q73/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q74/explain.txt | 670 +++++------
.../q74/simplified.txt | 20 +-
.../approved-plans-v1_4-spark3_5/q75/explain.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q76/explain.txt | 249 +++--
.../q76/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q79/explain.txt | 65 +-
.../q79/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q8/explain.txt | 280 ++---
.../approved-plans-v1_4-spark3_5/q8/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q80/explain.txt | 749 +++++++------
.../q80/simplified.txt | 15 +-
.../approved-plans-v1_4-spark3_5/q81/explain.txt | 419 +++----
.../q81/simplified.txt | 63 +-
.../approved-plans-v1_4-spark3_5/q82/explain.txt | 110 +-
.../q82/simplified.txt | 2 +-
.../approved-plans-v1_4-spark3_5/q83/explain.txt | 485 ++++----
.../q83/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q84/explain.txt | 247 +++--
.../q84/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q85/explain.txt | 283 ++---
.../q85/simplified.txt | 17 +-
.../approved-plans-v1_4-spark3_5/q86/explain.txt | 127 ++-
.../q86/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q87/explain.txt | 335 +++---
.../q87/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q89/explain.txt | 259 ++---
.../q89/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q91/explain.txt | 368 +++---
.../q91/simplified.txt | 10 +-
.../approved-plans-v1_4-spark3_5/q93/explain.txt | 4 +-
.../approved-plans-v1_4-spark3_5/q94/explain.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q95/explain.txt | 8 +-
.../approved-plans-v1_4-spark3_5/q98/explain.txt | 205 ++--
.../q98/simplified.txt | 5 +-
.../approved-plans-v1_4-spark3_5/q99/explain.txt | 155 +--
.../q99/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q1/explain.txt | 65 +-
.../approved-plans-v1_4-spark4_0/q1/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q10/explain.txt | 99 +-
.../q10/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q11/explain.txt | 794 ++++++-------
.../q11/simplified.txt | 20 +-
.../approved-plans-v1_4-spark4_0/q12/explain.txt | 205 ++--
.../q12/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q13/explain.txt | 183 +--
.../q13/simplified.txt | 7 +-
.../approved-plans-v1_4-spark4_0/q15/explain.txt | 165 +--
.../q15/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q16/explain.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q17/explain.txt | 182 +--
.../q17/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q18/explain.txt | 224 ++--
.../q18/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q19/explain.txt | 174 +--
.../q19/simplified.txt | 12 +-
.../approved-plans-v1_4-spark4_0/q2/explain.txt | 239 ++--
.../approved-plans-v1_4-spark4_0/q2/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q20/explain.txt | 205 ++--
.../q20/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q21/explain.txt | 78 +-
.../q21/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q22/explain.txt | 149 +--
.../q22/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q23b/explain.txt | 593 +++++-----
.../q23b/simplified.txt | 39 +-
.../approved-plans-v1_4-spark4_0/q24a/explain.txt | 532 ++++-----
.../q24a/simplified.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q24b/explain.txt | 532 ++++-----
.../q24b/simplified.txt | 22 +-
.../approved-plans-v1_4-spark4_0/q25/explain.txt | 166 +--
.../q25/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q26/explain.txt | 155 +--
.../q26/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q27/explain.txt | 162 +--
.../q27/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q29/explain.txt | 186 ++--
.../q29/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q3/explain.txt | 40 +-
.../approved-plans-v1_4-spark4_0/q3/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q30/explain.txt | 412 +++----
.../q30/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q33/explain.txt | 3 +-
.../approved-plans-v1_4-spark4_0/q34/explain.txt | 75 +-
.../q34/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q35/explain.txt | 150 +--
.../q35/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q36/explain.txt | 197 ++--
.../q36/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q37/explain.txt | 110 +-
.../q37/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q38/explain.txt | 317 +++---
.../q38/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q4/explain.txt | 1164 +++++++++----------
.../approved-plans-v1_4-spark4_0/q4/simplified.txt | 30 +-
.../approved-plans-v1_4-spark4_0/q40/explain.txt | 199 ++--
.../q40/simplified.txt | 7 +-
.../approved-plans-v1_4-spark4_0/q41/explain.txt | 59 +-
.../q41/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q42/explain.txt | 40 +-
.../q42/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q43/explain.txt | 80 +-
.../q43/simplified.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q44/explain.txt | 153 +--
.../q44/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q45/explain.txt | 266 ++---
.../q45/simplified.txt | 12 +-
.../approved-plans-v1_4-spark4_0/q46/explain.txt | 103 +-
.../q46/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q47/explain.txt | 425 +++----
.../q47/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q48/explain.txt | 191 ++--
.../q48/simplified.txt | 7 +-
.../approved-plans-v1_4-spark4_0/q5/explain.txt | 585 +++++-----
.../approved-plans-v1_4-spark4_0/q5/simplified.txt | 15 +-
.../approved-plans-v1_4-spark4_0/q50/explain.txt | 203 ++--
.../q50/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q52/explain.txt | 40 +-
.../q52/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q53/explain.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q54/explain.txt | 384 +++----
.../q54/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q55/explain.txt | 40 +-
.../q55/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q56/explain.txt | 486 ++++----
.../q56/simplified.txt | 17 +-
.../approved-plans-v1_4-spark4_0/q57/explain.txt | 425 +++----
.../q57/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q58/explain.txt | 561 +++++-----
.../q58/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q59/explain.txt | 325 +++---
.../q59/simplified.txt | 15 +-
.../approved-plans-v1_4-spark4_0/q6/explain.txt | 443 ++++----
.../approved-plans-v1_4-spark4_0/q6/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q60/explain.txt | 486 ++++----
.../q60/simplified.txt | 17 +-
.../approved-plans-v1_4-spark4_0/q61/explain.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q62/explain.txt | 169 +--
.../q62/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q63/explain.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q64/explain.txt | 1121 ++++++++++---------
.../q64/simplified.txt | 17 +-
.../approved-plans-v1_4-spark4_0/q65/explain.txt | 233 ++--
.../q65/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q66/explain.txt | 523 ++++-----
.../q66/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q67/explain.txt | 216 ++--
.../q67/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q68/explain.txt | 103 +-
.../q68/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q69/explain.txt | 103 +-
.../q69/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q7/explain.txt | 155 +--
.../approved-plans-v1_4-spark4_0/q7/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q70/explain.txt | 296 ++---
.../q70/simplified.txt | 68 +-
.../approved-plans-v1_4-spark4_0/q71/explain.txt | 178 +--
.../q71/simplified.txt | 2 +-
.../approved-plans-v1_4-spark4_0/q72/explain.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q73/explain.txt | 73 +-
.../q73/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q74/explain.txt | 786 ++++++-------
.../q74/simplified.txt | 20 +-
.../approved-plans-v1_4-spark4_0/q75/explain.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q76/explain.txt | 289 ++---
.../q76/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q79/explain.txt | 61 +-
.../q79/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q8/explain.txt | 292 ++---
.../approved-plans-v1_4-spark4_0/q8/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q80/explain.txt | 811 +++++++-------
.../q80/simplified.txt | 15 +-
.../approved-plans-v1_4-spark4_0/q81/explain.txt | 411 +++----
.../q81/simplified.txt | 15 +-
.../approved-plans-v1_4-spark4_0/q82/explain.txt | 110 +-
.../q82/simplified.txt | 2 +-
.../q83.ansi/explain.txt | 531 ++++-----
.../q83.ansi/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q84/explain.txt | 247 +++--
.../q84/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q85/explain.txt | 299 ++---
.../q85/simplified.txt | 17 +-
.../approved-plans-v1_4-spark4_0/q86/explain.txt | 147 +--
.../q86/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q87/explain.txt | 335 +++---
.../q87/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q89/explain.txt | 281 ++---
.../q89/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q91/explain.txt | 390 +++----
.../q91/simplified.txt | 10 +-
.../approved-plans-v1_4-spark4_0/q93/explain.txt | 4 +-
.../approved-plans-v1_4-spark4_0/q94/explain.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q95/explain.txt | 8 +-
.../approved-plans-v1_4-spark4_0/q98/explain.txt | 227 ++--
.../q98/simplified.txt | 5 +-
.../approved-plans-v1_4-spark4_0/q99/explain.txt | 169 +--
.../q99/simplified.txt | 5 +-
.../approved-plans-v1_4/q1/explain.txt | 67 +-
.../approved-plans-v1_4/q1/simplified.txt | 5 +-
.../approved-plans-v1_4/q10/explain.txt | 159 +--
.../approved-plans-v1_4/q10/simplified.txt | 5 +-
.../approved-plans-v1_4/q11/explain.txt | 680 ++++++------
.../approved-plans-v1_4/q11/simplified.txt | 20 +-
.../approved-plans-v1_4/q12/explain.txt | 185 ++--
.../approved-plans-v1_4/q12/simplified.txt | 5 +-
.../approved-plans-v1_4/q13/explain.txt | 171 +--
.../approved-plans-v1_4/q13/simplified.txt | 7 +-
.../approved-plans-v1_4/q14b/explain.txt | 20 +-
.../approved-plans-v1_4/q15/explain.txt | 151 +--
.../approved-plans-v1_4/q15/simplified.txt | 5 +-
.../approved-plans-v1_4/q16/explain.txt | 4 +-
.../approved-plans-v1_4/q17/explain.txt | 168 +--
.../approved-plans-v1_4/q17/simplified.txt | 10 +-
.../approved-plans-v1_4/q18/explain.txt | 210 ++--
.../approved-plans-v1_4/q18/simplified.txt | 10 +-
.../approved-plans-v1_4/q19/explain.txt | 160 +--
.../approved-plans-v1_4/q19/simplified.txt | 12 +-
.../approved-plans-v1_4/q2/explain.txt | 209 ++--
.../approved-plans-v1_4/q2/simplified.txt | 5 +-
.../approved-plans-v1_4/q20/explain.txt | 185 ++--
.../approved-plans-v1_4/q20/simplified.txt | 5 +-
.../approved-plans-v1_4/q21/explain.txt | 68 +-
.../approved-plans-v1_4/q21/simplified.txt | 2 +-
.../approved-plans-v1_4/q22/explain.txt | 135 +--
.../approved-plans-v1_4/q22/simplified.txt | 5 +-
.../approved-plans-v1_4/q23b/explain.txt | 537 ++++-----
.../approved-plans-v1_4/q23b/simplified.txt | 23 +-
.../approved-plans-v1_4/q24a/explain.txt | 518 ++++-----
.../approved-plans-v1_4/q24a/simplified.txt | 22 +-
.../approved-plans-v1_4/q24b/explain.txt | 518 ++++-----
.../approved-plans-v1_4/q24b/simplified.txt | 22 +-
.../approved-plans-v1_4/q25/explain.txt | 152 +--
.../approved-plans-v1_4/q25/simplified.txt | 10 +-
.../approved-plans-v1_4/q26/explain.txt | 141 +--
.../approved-plans-v1_4/q26/simplified.txt | 5 +-
.../approved-plans-v1_4/q27/explain.txt | 148 +--
.../approved-plans-v1_4/q27/simplified.txt | 10 +-
.../approved-plans-v1_4/q29/explain.txt | 172 +--
.../approved-plans-v1_4/q29/simplified.txt | 10 +-
.../approved-plans-v1_4/q3/explain.txt | 30 +-
.../approved-plans-v1_4/q3/simplified.txt | 2 +-
.../approved-plans-v1_4/q30/explain.txt | 338 +++---
.../approved-plans-v1_4/q30/simplified.txt | 10 +-
.../approved-plans-v1_4/q32/explain.txt | 28 +-
.../approved-plans-v1_4/q33/explain.txt | 187 ++--
.../approved-plans-v1_4/q33/simplified.txt | 15 +-
.../approved-plans-v1_4/q34/explain.txt | 75 +-
.../approved-plans-v1_4/q34/simplified.txt | 5 +-
.../approved-plans-v1_4/q35/explain.txt | 192 ++--
.../approved-plans-v1_4/q35/simplified.txt | 10 +-
.../approved-plans-v1_4/q36/explain.txt | 177 +--
.../approved-plans-v1_4/q36/simplified.txt | 5 +-
.../approved-plans-v1_4/q37/explain.txt | 110 +-
.../approved-plans-v1_4/q37/simplified.txt | 2 +-
.../approved-plans-v1_4/q38/explain.txt | 317 +++---
.../approved-plans-v1_4/q38/simplified.txt | 5 +-
.../approved-plans-v1_4/q4/explain.txt | 986 ++++++++--------
.../approved-plans-v1_4/q4/simplified.txt | 30 +-
.../approved-plans-v1_4/q40/explain.txt | 185 ++--
.../approved-plans-v1_4/q40/simplified.txt | 7 +-
.../approved-plans-v1_4/q41/explain.txt | 63 +-
.../approved-plans-v1_4/q41/simplified.txt | 4 +-
.../approved-plans-v1_4/q42/explain.txt | 30 +-
.../approved-plans-v1_4/q42/simplified.txt | 2 +-
.../approved-plans-v1_4/q43/explain.txt | 70 +-
.../approved-plans-v1_4/q43/simplified.txt | 4 +-
.../approved-plans-v1_4/q44/explain.txt | 117 +-
.../approved-plans-v1_4/q44/simplified.txt | 5 +-
.../approved-plans-v1_4/q45/explain.txt | 266 ++---
.../approved-plans-v1_4/q45/simplified.txt | 12 +-
.../approved-plans-v1_4/q46/explain.txt | 89 +-
.../approved-plans-v1_4/q46/simplified.txt | 5 +-
.../approved-plans-v1_4/q47/explain.txt | 393 +++----
.../approved-plans-v1_4/q47/simplified.txt | 5 +-
.../approved-plans-v1_4/q48/explain.txt | 179 +--
.../approved-plans-v1_4/q48/simplified.txt | 7 +-
.../approved-plans-v1_4/q5/explain.txt | 535 ++++-----
.../approved-plans-v1_4/q5/simplified.txt | 15 +-
.../approved-plans-v1_4/q50/explain.txt | 189 ++--
.../approved-plans-v1_4/q50/simplified.txt | 5 +-
.../approved-plans-v1_4/q52/explain.txt | 30 +-
.../approved-plans-v1_4/q52/simplified.txt | 2 +-
.../approved-plans-v1_4/q53/explain.txt | 4 +-
.../approved-plans-v1_4/q54/explain.txt | 332 +++---
.../approved-plans-v1_4/q54/simplified.txt | 10 +-
.../approved-plans-v1_4/q55/explain.txt | 30 +-
.../approved-plans-v1_4/q55/simplified.txt | 2 +-
.../approved-plans-v1_4/q56/explain.txt | 384 +++----
.../approved-plans-v1_4/q56/simplified.txt | 17 +-
.../approved-plans-v1_4/q57/explain.txt | 393 +++----
.../approved-plans-v1_4/q57/simplified.txt | 5 +-
.../approved-plans-v1_4/q58/explain.txt | 609 +++++-----
.../approved-plans-v1_4/q58/simplified.txt | 37 +-
.../approved-plans-v1_4/q59/explain.txt | 293 ++---
.../approved-plans-v1_4/q59/simplified.txt | 15 +-
.../approved-plans-v1_4/q6/explain.txt | 371 ++++---
.../approved-plans-v1_4/q6/simplified.txt | 10 +-
.../approved-plans-v1_4/q60/explain.txt | 384 +++----
.../approved-plans-v1_4/q60/simplified.txt | 17 +-
.../approved-plans-v1_4/q61/explain.txt | 8 +-
.../approved-plans-v1_4/q62/explain.txt | 155 +--
.../approved-plans-v1_4/q62/simplified.txt | 5 +-
.../approved-plans-v1_4/q63/explain.txt | 4 +-
.../approved-plans-v1_4/q64/explain.txt | 1103 +++++++++---------
.../approved-plans-v1_4/q64/simplified.txt | 17 +-
.../approved-plans-v1_4/q65/explain.txt | 195 ++--
.../approved-plans-v1_4/q65/simplified.txt | 5 +-
.../approved-plans-v1_4/q66/explain.txt | 413 +++----
.../approved-plans-v1_4/q66/simplified.txt | 5 +-
.../approved-plans-v1_4/q67/explain.txt | 172 +--
.../approved-plans-v1_4/q67/simplified.txt | 10 +-
.../approved-plans-v1_4/q68/explain.txt | 89 +-
.../approved-plans-v1_4/q68/simplified.txt | 5 +-
.../approved-plans-v1_4/q69/explain.txt | 161 +--
.../approved-plans-v1_4/q69/simplified.txt | 5 +-
.../approved-plans-v1_4/q7/explain.txt | 141 +--
.../approved-plans-v1_4/q7/simplified.txt | 5 +-
.../approved-plans-v1_4/q70/explain.txt | 284 ++---
.../approved-plans-v1_4/q70/simplified.txt | 60 +-
.../approved-plans-v1_4/q71/explain.txt | 166 +--
.../approved-plans-v1_4/q71/simplified.txt | 2 +-
.../approved-plans-v1_4/q72/explain.txt | 8 +-
.../approved-plans-v1_4/q73/explain.txt | 73 +-
.../approved-plans-v1_4/q73/simplified.txt | 5 +-
.../approved-plans-v1_4/q74/explain.txt | 670 +++++------
.../approved-plans-v1_4/q74/simplified.txt | 20 +-
.../approved-plans-v1_4/q75/explain.txt | 4 +-
.../approved-plans-v1_4/q76/explain.txt | 249 +++--
.../approved-plans-v1_4/q76/simplified.txt | 5 +-
.../approved-plans-v1_4/q79/explain.txt | 65 +-
.../approved-plans-v1_4/q79/simplified.txt | 5 +-
.../approved-plans-v1_4/q8/explain.txt | 280 ++---
.../approved-plans-v1_4/q8/simplified.txt | 10 +-
.../approved-plans-v1_4/q80/explain.txt | 749 +++++++------
.../approved-plans-v1_4/q80/simplified.txt | 15 +-
.../approved-plans-v1_4/q81/explain.txt | 335 +++---
.../approved-plans-v1_4/q81/simplified.txt | 15 +-
.../approved-plans-v1_4/q82/explain.txt | 110 +-
.../approved-plans-v1_4/q82/simplified.txt | 2 +-
.../approved-plans-v1_4/q83/explain.txt | 569 ++++++----
.../approved-plans-v1_4/q83/simplified.txt | 39 +-
.../approved-plans-v1_4/q84/explain.txt | 247 +++--
.../approved-plans-v1_4/q84/simplified.txt | 5 +-
.../approved-plans-v1_4/q85/explain.txt | 285 ++---
.../approved-plans-v1_4/q85/simplified.txt | 17 +-
.../approved-plans-v1_4/q86/explain.txt | 127 ++-
.../approved-plans-v1_4/q86/simplified.txt | 5 +-
.../approved-plans-v1_4/q87/explain.txt | 335 +++---
.../approved-plans-v1_4/q87/simplified.txt | 5 +-
.../approved-plans-v1_4/q89/explain.txt | 261 ++---
.../approved-plans-v1_4/q89/simplified.txt | 5 +-
.../approved-plans-v1_4/q91/explain.txt | 370 ++++---
.../approved-plans-v1_4/q91/simplified.txt | 10 +-
.../approved-plans-v1_4/q92/explain.txt | 28 +-
.../approved-plans-v1_4/q93/explain.txt | 4 +-
.../approved-plans-v1_4/q94/explain.txt | 8 +-
.../approved-plans-v1_4/q95/explain.txt | 8 +-
.../approved-plans-v1_4/q98/explain.txt | 207 ++--
.../approved-plans-v1_4/q98/simplified.txt | 5 +-
.../approved-plans-v1_4/q99/explain.txt | 155 +--
.../approved-plans-v1_4/q99/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q10a/explain.txt | 87 +-
.../q10a/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q11/explain.txt | 670 +++++------
.../q11/simplified.txt | 20 +-
.../approved-plans-v2_7-spark3_5/q12/explain.txt | 183 +--
.../q12/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q18a/explain.txt | 1029 ++++++++---------
.../q18a/simplified.txt | 17 +-
.../approved-plans-v2_7-spark3_5/q20/explain.txt | 183 +--
.../q20/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q22/explain.txt | 133 +--
.../q22/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q22a/explain.txt | 321 +++---
.../q22a/simplified.txt | 7 +-
.../approved-plans-v2_7-spark3_5/q24/explain.txt | 552 ++++-----
.../q24/simplified.txt | 22 +-
.../approved-plans-v2_7-spark3_5/q27a/explain.txt | 474 ++++----
.../q27a/simplified.txt | 12 +-
.../approved-plans-v2_7-spark3_5/q34/explain.txt | 73 +-
.../q34/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q35/explain.txt | 150 +--
.../q35/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q35a/explain.txt | 136 +--
.../q35a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q36a/explain.txt | 275 ++---
.../q36a/simplified.txt | 7 +-
.../approved-plans-v2_7-spark3_5/q47/explain.txt | 397 +++----
.../q47/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q57/explain.txt | 397 +++----
.../q57/simplified.txt | 5 +-
.../approved-plans-v2_7-spark3_5/q5a/explain.txt | 711 ++++++------
.../q5a/simplified.txt | 15 +-
.../approved-plans-v2_7-spark3_5/q6/explain.txt | 417 +++----
.../approved-plans-v2_7-spark3_5/q6/simplified.txt | 10 +-
.../approved-plans-v2_7-spark3_5/q64/explain.txt | 1101 +++++++++---------
.../q64/simplified.txt | 17 +-
.../approved-plans-v2_7-spark3_5/q67a/explain.txt | 536 ++++-----
.../q67a/simplified.txt | 12 +-
.../approved-plans-v2_7-spark3_5/q70a/explain.txt | 456 ++++----
.../q70a/simplified.txt | 64 +-
.../approved-plans-v2_7-spark3_5/q72/explain.txt | 8 +-
.../approved-plans-v2_7-spark3_5/q74/explain.txt | 670 +++++------
.../q74/simplified.txt | 20 +-
.../approved-plans-v2_7-spark3_5/q75/explain.txt | 4 +-
.../approved-plans-v2_7-spark3_5/q80a/explain.txt | 925 ++++++++--------
.../q80a/simplified.txt | 15 +-
.../approved-plans-v2_7-spark3_5/q86a/explain.txt | 211 ++--
.../q86a/simplified.txt | 7 +-
.../approved-plans-v2_7-spark3_5/q98/explain.txt | 197 ++--
.../q98/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q10a/explain.txt | 87 +-
.../q10a/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q11/explain.txt | 786 ++++++-------
.../q11/simplified.txt | 20 +-
.../approved-plans-v2_7-spark4_0/q12/explain.txt | 205 ++--
.../q12/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q18a/explain.txt | 1171 ++++++++++----------
.../q18a/simplified.txt | 17 +-
.../approved-plans-v2_7-spark4_0/q20/explain.txt | 205 ++--
.../q20/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q22/explain.txt | 133 +--
.../q22/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q22a/explain.txt | 467 ++++----
.../q22a/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q24/explain.txt | 552 ++++-----
.../q24/simplified.txt | 22 +-
.../approved-plans-v2_7-spark4_0/q27a/explain.txt | 566 +++++-----
.../q27a/simplified.txt | 12 +-
.../approved-plans-v2_7-spark4_0/q34/explain.txt | 75 +-
.../q34/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q35/explain.txt | 150 +--
.../q35/simplified.txt | 10 +-
.../approved-plans-v2_7-spark4_0/q35a/explain.txt | 150 +--
.../q35a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark4_0/q36a/explain.txt | 359 +++---
.../q36a/simplified.txt | 7 +-
.../approved-plans-v2_7-spark4_0/q47/explain.txt | 425 +++----
.../q47/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q57/explain.txt | 425 +++----
.../q57/simplified.txt | 5 +-
.../approved-plans-v2_7-spark4_0/q5a/explain.txt | 757 ++++++-------
.../q5a/simplified.txt | 15 +-
.../approved-plans-v2_7-spark4_0/q6/explain.txt | 443 ++++----
.../approved-plans-v2_7-spark4_0/q6/simplified.txt | 10 +-
.../approved-plans-v2_7-spark4_0/q64/explain.txt | 1121 ++++++++++---------
.../q64/simplified.txt | 17 +-
.../approved-plans-v2_7-spark4_0/q67a/explain.txt | 856 +++++++-------
.../q67a/simplified.txt | 10 +-
.../approved-plans-v2_7-spark4_0/q70a/explain.txt | 470 ++++----
.../q70a/simplified.txt | 70 +-
.../approved-plans-v2_7-spark4_0/q72/explain.txt | 8 +-
.../approved-plans-v2_7-spark4_0/q74/explain.txt | 786 ++++++-------
.../q74/simplified.txt | 20 +-
.../approved-plans-v2_7-spark4_0/q75/explain.txt | 4 +-
.../approved-plans-v2_7-spark4_0/q80a/explain.txt | 983 ++++++++--------
.../q80a/simplified.txt | 15 +-
.../approved-plans-v2_7-spark4_0/q86a/explain.txt | 309 +++---
.../q86a/simplified.txt | 7 +-
.../approved-plans-v2_7-spark4_0/q98/explain.txt | 219 ++--
.../q98/simplified.txt | 5 +-
.../approved-plans-v2_7/q10a/explain.txt | 157 +--
.../approved-plans-v2_7/q10a/simplified.txt | 5 +-
.../approved-plans-v2_7/q11/explain.txt | 670 +++++------
.../approved-plans-v2_7/q11/simplified.txt | 20 +-
.../approved-plans-v2_7/q12/explain.txt | 185 ++--
.../approved-plans-v2_7/q12/simplified.txt | 5 +-
.../approved-plans-v2_7/q14/explain.txt | 20 +-
.../approved-plans-v2_7/q18a/explain.txt | 1033 ++++++++---------
.../approved-plans-v2_7/q18a/simplified.txt | 17 +-
.../approved-plans-v2_7/q20/explain.txt | 185 ++--
.../approved-plans-v2_7/q20/simplified.txt | 5 +-
.../approved-plans-v2_7/q22/explain.txt | 133 +--
.../approved-plans-v2_7/q22/simplified.txt | 5 +-
.../approved-plans-v2_7/q22a/explain.txt | 323 +++---
.../approved-plans-v2_7/q22a/simplified.txt | 7 +-
.../approved-plans-v2_7/q24/explain.txt | 542 ++++-----
.../approved-plans-v2_7/q24/simplified.txt | 22 +-
.../approved-plans-v2_7/q27a/explain.txt | 474 ++++----
.../approved-plans-v2_7/q27a/simplified.txt | 12 +-
.../approved-plans-v2_7/q34/explain.txt | 75 +-
.../approved-plans-v2_7/q34/simplified.txt | 5 +-
.../approved-plans-v2_7/q35/explain.txt | 192 ++--
.../approved-plans-v2_7/q35/simplified.txt | 10 +-
.../approved-plans-v2_7/q35a/explain.txt | 168 +--
.../approved-plans-v2_7/q35a/simplified.txt | 10 +-
.../approved-plans-v2_7/q36a/explain.txt | 283 ++---
.../approved-plans-v2_7/q36a/simplified.txt | 7 +-
.../approved-plans-v2_7/q47/explain.txt | 393 +++----
.../approved-plans-v2_7/q47/simplified.txt | 5 +-
.../approved-plans-v2_7/q57/explain.txt | 393 +++----
.../approved-plans-v2_7/q57/simplified.txt | 5 +-
.../approved-plans-v2_7/q5a/explain.txt | 711 ++++++------
.../approved-plans-v2_7/q5a/simplified.txt | 15 +-
.../approved-plans-v2_7/q6/explain.txt | 377 ++++---
.../approved-plans-v2_7/q6/simplified.txt | 10 +-
.../approved-plans-v2_7/q64/explain.txt | 1103 +++++++++---------
.../approved-plans-v2_7/q64/simplified.txt | 17 +-
.../approved-plans-v2_7/q67a/explain.txt | 518 ++++-----
.../approved-plans-v2_7/q67a/simplified.txt | 12 +-
.../approved-plans-v2_7/q70a/explain.txt | 448 ++++----
.../approved-plans-v2_7/q70a/simplified.txt | 62 +-
.../approved-plans-v2_7/q72/explain.txt | 8 +-
.../approved-plans-v2_7/q74/explain.txt | 670 +++++------
.../approved-plans-v2_7/q74/simplified.txt | 20 +-
.../approved-plans-v2_7/q75/explain.txt | 4 +-
.../approved-plans-v2_7/q80a/explain.txt | 925 ++++++++--------
.../approved-plans-v2_7/q80a/simplified.txt | 15 +-
.../approved-plans-v2_7/q86a/explain.txt | 233 ++--
.../approved-plans-v2_7/q86a/simplified.txt | 7 +-
.../approved-plans-v2_7/q98/explain.txt | 199 ++--
.../approved-plans-v2_7/q98/simplified.txt | 5 +-
.../spark/sql/comet/CometPlanStabilitySuite.scala | 1 -
636 files changed, 50052 insertions(+), 47207 deletions(-)
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt
index a4f0925a4..6b5d05df7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/explain.txt
@@ -1,8 +1,8 @@
== Physical Plan ==
-* CometColumnarToRow (40)
-+- CometTakeOrderedAndProject (39)
- +- CometProject (38)
- +- CometBroadcastHashJoin (37)
+* CometColumnarToRow (41)
++- CometTakeOrderedAndProject (40)
+ +- CometProject (39)
+ +- CometBroadcastHashJoin (38)
:- CometProject (33)
: +- CometBroadcastHashJoin (32)
: :- CometProject (27)
@@ -36,9 +36,10 @@
: +- CometProject (30)
: +- CometFilter (29)
: +- CometScan parquet spark_catalog.default.store (28)
- +- CometBroadcastExchange (36)
- +- CometFilter (35)
- +- CometScan parquet spark_catalog.default.customer (34)
+ +- CometBroadcastExchange (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan parquet spark_catalog.default.customer (34)
(1) CometScan parquet spark_catalog.default.store_returns
@@ -172,12 +173,12 @@ Arguments: [ctr_customer_sk#9, ctr_store_sk#10], [ctr_customer_sk#9, ctr_store_s
Output [2]: [s_store_sk#24, s_state#25]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
-PushedFilters: [IsNotNull(s_state), IsNotNull(s_store_sk)]
+PushedFilters: [IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_state:string>
(29) CometFilter
Input [2]: [s_store_sk#24, s_state#25]
-Condition : ((isnotnull(s_state#25) AND (rpad(s_state#25, 2, ) = TN)) AND isnotnull(s_store_sk#24))
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#25, 2, true, false, true) = TN) AND isnotnull(s_store_sk#24))
(30) CometProject
Input [2]: [s_store_sk#24, s_state#25]
@@ -207,55 +208,59 @@ ReadSchema: struct<c_customer_sk:int,c_customer_id:string>
Input [2]: [c_customer_sk#26, c_customer_id#27]
Condition : isnotnull(c_customer_sk#26)
-(36) CometBroadcastExchange
+(36) CometProject
Input [2]: [c_customer_sk#26, c_customer_id#27]
-Arguments: [c_customer_sk#26, c_customer_id#27]
+Arguments: [c_customer_sk#26, c_customer_id#28], [c_customer_sk#26, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#27, 16, true, false, true) AS c_customer_id#28]
-(37) CometBroadcastHashJoin
+(37) CometBroadcastExchange
+Input [2]: [c_customer_sk#26, c_customer_id#28]
+Arguments: [c_customer_sk#26, c_customer_id#28]
+
+(38) CometBroadcastHashJoin
Left output [1]: [ctr_customer_sk#9]
-Right output [2]: [c_customer_sk#26, c_customer_id#27]
+Right output [2]: [c_customer_sk#26, c_customer_id#28]
Arguments: [ctr_customer_sk#9], [c_customer_sk#26], Inner, BuildRight
-(38) CometProject
-Input [3]: [ctr_customer_sk#9, c_customer_sk#26, c_customer_id#27]
-Arguments: [c_customer_id#27], [c_customer_id#27]
+(39) CometProject
+Input [3]: [ctr_customer_sk#9, c_customer_sk#26, c_customer_id#28]
+Arguments: [c_customer_id#28], [c_customer_id#28]
-(39) CometTakeOrderedAndProject
-Input [1]: [c_customer_id#27]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#27 ASC NULLS FIRST], output=[c_customer_id#27]), [c_customer_id#27], 100, [c_customer_id#27 ASC NULLS FIRST], [c_customer_id#27]
+(40) CometTakeOrderedAndProject
+Input [1]: [c_customer_id#28]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#28 ASC NULLS FIRST], output=[c_customer_id#28]), [c_customer_id#28], 100, [c_customer_id#28 ASC NULLS FIRST], [c_customer_id#28]
-(40) CometColumnarToRow [codegen id : 1]
-Input [1]: [c_customer_id#27]
+(41) CometColumnarToRow [codegen id : 1]
+Input [1]: [c_customer_id#28]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = sr_returned_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (45)
-+- * CometColumnarToRow (44)
- +- CometProject (43)
- +- CometFilter (42)
- +- CometScan parquet spark_catalog.default.date_dim (41)
+BroadcastExchange (46)
++- * CometColumnarToRow (45)
+ +- CometProject (44)
+ +- CometFilter (43)
+ +- CometScan parquet spark_catalog.default.date_dim (42)
-(41) CometScan parquet spark_catalog.default.date_dim
+(42) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_year#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(42) CometFilter
+(43) CometFilter
Input [2]: [d_date_sk#6, d_year#7]
Condition : ((isnotnull(d_year#7) AND (d_year#7 = 2000)) AND isnotnull(d_date_sk#6))
-(43) CometProject
+(44) CometProject
Input [2]: [d_date_sk#6, d_year#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(44) CometColumnarToRow [codegen id : 1]
+(45) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(45) BroadcastExchange
+(46) BroadcastExchange
Input [1]: [d_date_sk#6]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt
index 3ca476a1f..c8df9e635 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q1/simplified.txt
@@ -47,5 +47,6 @@ WholeStageCodegen (1)
CometFilter [s_store_sk,s_state]
CometScan parquet spark_catalog.default.store [s_store_sk,s_state]
CometBroadcastExchange [c_customer_sk,c_customer_id] #8
- CometFilter [c_customer_sk,c_customer_id]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id]
+ CometProject [c_customer_id] [c_customer_sk,c_customer_id]
+ CometFilter [c_customer_sk,c_customer_id]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt
index 3c2df9693..2036dea19 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/explain.txt
@@ -1,11 +1,11 @@
== Physical Plan ==
-TakeOrderedAndProject (46)
-+- * HashAggregate (45)
- +- * CometColumnarToRow (44)
- +- CometColumnarExchange (43)
- +- * HashAggregate (42)
- +- * Project (41)
- +- * BroadcastHashJoin Inner BuildRight (40)
+TakeOrderedAndProject (47)
++- * HashAggregate (46)
+ +- * CometColumnarToRow (45)
+ +- CometColumnarExchange (44)
+ +- * HashAggregate (43)
+ +- * Project (42)
+ +- * BroadcastHashJoin Inner BuildRight (41)
:- * Project (35)
: +- * BroadcastHashJoin Inner BuildRight (34)
: :- * Project (28)
@@ -41,10 +41,11 @@ TakeOrderedAndProject (46)
: +- CometProject (31)
: +- CometFilter (30)
: +- CometScan parquet spark_catalog.default.customer_address (29)
- +- BroadcastExchange (39)
- +- * CometColumnarToRow (38)
- +- CometFilter (37)
- +- CometScan parquet spark_catalog.default.customer_demographics (36)
+ +- BroadcastExchange (40)
+ +- * CometColumnarToRow (39)
+ +- CometProject (38)
+ +- CometFilter (37)
+ +- CometScan parquet spark_catalog.default.customer_demographics (36)
(1) CometScan parquet spark_catalog.default.customer
@@ -220,77 +221,81 @@ ReadSchema: struct<cd_demo_sk:int,cd_gender:string,cd_marital_status:string,cd_e
Input [9]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
Condition : isnotnull(cd_demo_sk#22)
-(38) CometColumnarToRow [codegen id : 4]
+(38) CometProject
Input [9]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+Arguments: [cd_demo_sk#22, cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30], [cd_demo_sk#22, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_gender#23, 1, true, false, true) AS cd_gender#31, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_ma [...]
-(39) BroadcastExchange
-Input [9]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=4]
+(39) CometColumnarToRow [codegen id : 4]
+Input [9]: [cd_demo_sk#22, cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+
+(40) BroadcastExchange
+Input [9]: [cd_demo_sk#22, cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-(40) BroadcastHashJoin [codegen id : 5]
+(41) BroadcastHashJoin [codegen id : 5]
Left keys [1]: [c_current_cdemo_sk#4]
Right keys [1]: [cd_demo_sk#22]
Join type: Inner
Join condition: None
-(41) Project [codegen id : 5]
-Output [8]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
-Input [10]: [c_current_cdemo_sk#4, cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+(42) Project [codegen id : 5]
+Output [8]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+Input [10]: [c_current_cdemo_sk#4, cd_demo_sk#22, cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
-(42) HashAggregate [codegen id : 5]
-Input [8]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
-Keys [8]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+(43) HashAggregate [codegen id : 5]
+Input [8]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+Keys [8]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
Functions [1]: [partial_count(1)]
-Aggregate Attributes [1]: [count#31]
-Results [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
+Aggregate Attributes [1]: [count#35]
+Results [9]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#36]
-(43) CometColumnarExchange
-Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
-Arguments: hashpartitioning(cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+(44) CometColumnarExchange
+Input [9]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#36]
+Arguments: hashpartitioning(cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-(44) CometColumnarToRow [codegen id : 6]
-Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
+(45) CometColumnarToRow [codegen id : 6]
+Input [9]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#36]
-(45) HashAggregate [codegen id : 6]
-Input [9]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#32]
-Keys [8]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cd_purchase_estimate#26, cd_credit_rating#27, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
+(46) HashAggregate [codegen id : 6]
+Input [9]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30, count#36]
+Keys [8]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cd_purchase_estimate#26, cd_credit_rating#34, cd_dep_count#28, cd_dep_employed_count#29, cd_dep_college_count#30]
Functions [1]: [count(1)]
-Aggregate Attributes [1]: [count(1)#33]
-Results [14]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, count(1)#33 AS cnt1#34, cd_purchase_estimate#26, count(1)#33 AS cnt2#35, cd_credit_rating#27, count(1)#33 AS cnt3#36, cd_dep_count#28, count(1)#33 AS cnt4#37, cd_dep_employed_count#29, count(1)#33 AS cnt5#38, cd_dep_college_count#30, count(1)#33 AS cnt6#39]
+Aggregate Attributes [1]: [count(1)#37]
+Results [14]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, count(1)#37 AS cnt1#38, cd_purchase_estimate#26, count(1)#37 AS cnt2#39, cd_credit_rating#34, count(1)#37 AS cnt3#40, cd_dep_count#28, count(1)#37 AS cnt4#41, cd_dep_employed_count#29, count(1)#37 AS cnt5#42, cd_dep_college_count#30, count(1)#37 AS cnt6#43]
-(46) TakeOrderedAndProject
-Input [14]: [cd_gender#23, cd_marital_status#24, cd_education_status#25, cnt1#34, cd_purchase_estimate#26, cnt2#35, cd_credit_rating#27, cnt3#36, cd_dep_count#28, cnt4#37, cd_dep_employed_count#29, cnt5#38, cd_dep_college_count#30, cnt6#39]
-Arguments: 100, [cd_gender#23 ASC NULLS FIRST, cd_marital_status#24 ASC NULLS FIRST, cd_education_status#25 ASC NULLS FIRST, cd_purchase_estimate#26 ASC NULLS FIRST, cd_credit_rating#27 ASC NULLS FIRST, cd_dep_count#28 ASC NULLS FIRST, cd_dep_employed_count#29 ASC NULLS FIRST, cd_dep_college_count#30 ASC NULLS FIRST], [cd_gender#23, cd_marital_status#24, cd_education_status#25, cnt1#34, cd_purchase_estimate#26, cnt2#35, cd_credit_rating#27, cnt3#36, cd_dep_count#28, cnt4#37, cd_dep_emplo [...]
+(47) TakeOrderedAndProject
+Input [14]: [cd_gender#31, cd_marital_status#32, cd_education_status#33, cnt1#38, cd_purchase_estimate#26, cnt2#39, cd_credit_rating#34, cnt3#40, cd_dep_count#28, cnt4#41, cd_dep_employed_count#29, cnt5#42, cd_dep_college_count#30, cnt6#43]
+Arguments: 100, [cd_gender#31 ASC NULLS FIRST, cd_marital_status#32 ASC NULLS FIRST, cd_education_status#33 ASC NULLS FIRST, cd_purchase_estimate#26 ASC NULLS FIRST, cd_credit_rating#34 ASC NULLS FIRST, cd_dep_count#28 ASC NULLS FIRST, cd_dep_employed_count#29 ASC NULLS FIRST, cd_dep_college_count#30 ASC NULLS FIRST], [cd_gender#31, cd_marital_status#32, cd_education_status#33, cnt1#38, cd_purchase_estimate#26, cnt2#39, cd_credit_rating#34, cnt3#40, cd_dep_count#28, cnt4#41, cd_dep_emplo [...]
===== Subqueries =====
Subquery:1 Hosting operator id = 3 Hosting Expression = ss_sold_date_sk#7 IN dynamicpruning#8
-BroadcastExchange (51)
-+- * CometColumnarToRow (50)
- +- CometProject (49)
- +- CometFilter (48)
- +- CometScan parquet spark_catalog.default.date_dim (47)
+BroadcastExchange (52)
++- * CometColumnarToRow (51)
+ +- CometProject (50)
+ +- CometFilter (49)
+ +- CometScan parquet spark_catalog.default.date_dim (48)
-(47) CometScan parquet spark_catalog.default.date_dim
+(48) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#9, d_year#10, d_moy#11]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_moy), EqualTo(d_year,2002), GreaterThanOrEqual(d_moy,1), LessThanOrEqual(d_moy,4), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(48) CometFilter
+(49) CometFilter
Input [3]: [d_date_sk#9, d_year#10, d_moy#11]
Condition : (((((isnotnull(d_year#10) AND isnotnull(d_moy#11)) AND (d_year#10 = 2002)) AND (d_moy#11 >= 1)) AND (d_moy#11 <= 4)) AND isnotnull(d_date_sk#9))
-(49) CometProject
+(50) CometProject
Input [3]: [d_date_sk#9, d_year#10, d_moy#11]
Arguments: [d_date_sk#9], [d_date_sk#9]
-(50) CometColumnarToRow [codegen id : 1]
+(51) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#9]
-(51) BroadcastExchange
+(52) BroadcastExchange
Input [1]: [d_date_sk#9]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt
index bc0ed4910..3678c1c7f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q10/simplified.txt
@@ -68,5 +68,6 @@ TakeOrderedAndProject [cd_gender,cd_marital_status,cd_education_status,cd_purcha
WholeStageCodegen (4)
CometColumnarToRow
InputAdapter
- CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
- CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometProject [cd_gender,cd_marital_status,cd_education_status,cd_credit_rating] [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_education_status,cd_purchase_estimate,cd_credit_rating,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt
index 736e858c8..f2cee5ac3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/explain.txt
@@ -1,455 +1,475 @@
== Physical Plan ==
-* CometColumnarToRow (69)
-+- CometTakeOrderedAndProject (68)
- +- CometProject (67)
- +- CometBroadcastHashJoin (66)
- :- CometProject (51)
- : +- CometBroadcastHashJoin (50)
- : :- CometProject (34)
- : : +- CometBroadcastHashJoin (33)
- : : :- CometFilter (16)
- : : : +- CometHashAggregate (15)
- : : : +- CometExchange (14)
- : : : +- CometHashAggregate (13)
- : : : +- CometProject (12)
- : : : +- CometBroadcastHashJoin (11)
- : : : :- CometProject (7)
- : : : : +- CometBroadcastHashJoin (6)
- : : : : :- CometFilter (2)
- : : : : : +- CometScan parquet spark_catalog.default.customer (1)
- : : : : +- CometBroadcastExchange (5)
- : : : : +- CometFilter (4)
- : : : : +- CometScan parquet spark_catalog.default.store_sales (3)
- : : : +- CometBroadcastExchange (10)
- : : : +- CometFilter (9)
- : : : +- CometScan parquet spark_catalog.default.date_dim (8)
- : : +- CometBroadcastExchange (32)
- : : +- CometHashAggregate (31)
- : : +- CometExchange (30)
- : : +- CometHashAggregate (29)
- : : +- CometProject (28)
- : : +- CometBroadcastHashJoin (27)
- : : :- CometProject (23)
- : : : +- CometBroadcastHashJoin (22)
- : : : :- CometFilter (18)
- : : : : +- CometScan parquet spark_catalog.default.customer (17)
- : : : +- CometBroadcastExchange (21)
- : : : +- CometFilter (20)
- : : : +- CometScan parquet spark_catalog.default.store_sales (19)
- : : +- CometBroadcastExchange (26)
- : : +- CometFilter (25)
- : : +- CometScan parquet spark_catalog.default.date_dim (24)
- : +- CometBroadcastExchange (49)
- : +- CometFilter (48)
- : +- CometHashAggregate (47)
- : +- CometExchange (46)
- : +- CometHashAggregate (45)
- : +- CometProject (44)
- : +- CometBroadcastHashJoin (43)
- : :- CometProject (41)
- : : +- CometBroadcastHashJoin (40)
- : : :- CometFilter (36)
- : : : +- CometScan parquet spark_catalog.default.customer (35)
- : : +- CometBroadcastExchange (39)
- : : +- CometFilter (38)
- : : +- CometScan parquet spark_catalog.default.web_sales (37)
- : +- ReusedExchange (42)
- +- CometBroadcastExchange (65)
- +- CometHashAggregate (64)
- +- CometExchange (63)
- +- CometHashAggregate (62)
- +- CometProject (61)
- +- CometBroadcastHashJoin (60)
- :- CometProject (58)
- : +- CometBroadcastHashJoin (57)
- : :- CometFilter (53)
- : : +- CometScan parquet spark_catalog.default.customer (52)
- : +- CometBroadcastExchange (56)
- : +- CometFilter (55)
- : +- CometScan parquet spark_catalog.default.web_sales (54)
- +- ReusedExchange (59)
+* CometColumnarToRow (73)
++- CometTakeOrderedAndProject (72)
+ +- CometProject (71)
+ +- CometBroadcastHashJoin (70)
+ :- CometProject (54)
+ : +- CometBroadcastHashJoin (53)
+ : :- CometProject (36)
+ : : +- CometBroadcastHashJoin (35)
+ : : :- CometFilter (17)
+ : : : +- CometHashAggregate (16)
+ : : : +- CometExchange (15)
+ : : : +- CometHashAggregate (14)
+ : : : +- CometProject (13)
+ : : : +- CometBroadcastHashJoin (12)
+ : : : :- CometProject (8)
+ : : : : +- CometBroadcastHashJoin (7)
+ : : : : :- CometProject (3)
+ : : : : : +- CometFilter (2)
+ : : : : : +- CometScan parquet spark_catalog.default.customer (1)
+ : : : : +- CometBroadcastExchange (6)
+ : : : : +- CometFilter (5)
+ : : : : +- CometScan parquet spark_catalog.default.store_sales (4)
+ : : : +- CometBroadcastExchange (11)
+ : : : +- CometFilter (10)
+ : : : +- CometScan parquet spark_catalog.default.date_dim (9)
+ : : +- CometBroadcastExchange (34)
+ : : +- CometHashAggregate (33)
+ : : +- CometExchange (32)
+ : : +- CometHashAggregate (31)
+ : : +- CometProject (30)
+ : : +- CometBroadcastHashJoin (29)
+ : : :- CometProject (25)
+ : : : +- CometBroadcastHashJoin (24)
+ : : : :- CometProject (20)
+ : : : : +- CometFilter (19)
+ : : : : +- CometScan parquet spark_catalog.default.customer (18)
+ : : : +- CometBroadcastExchange (23)
+ : : : +- CometFilter (22)
+ : : : +- CometScan parquet spark_catalog.default.store_sales (21)
+ : : +- CometBroadcastExchange (28)
+ : : +- CometFilter (27)
+ : : +- CometScan parquet spark_catalog.default.date_dim (26)
+ : +- CometBroadcastExchange (52)
+ : +- CometFilter (51)
+ : +- CometHashAggregate (50)
+ : +- CometExchange (49)
+ : +- CometHashAggregate (48)
+ : +- CometProject (47)
+ : +- CometBroadcastHashJoin (46)
+ : :- CometProject (44)
+ : : +- CometBroadcastHashJoin (43)
+ : : :- CometProject (39)
+ : : : +- CometFilter (38)
+ : : : +- CometScan parquet spark_catalog.default.customer (37)
+ : : +- CometBroadcastExchange (42)
+ : : +- CometFilter (41)
+ : : +- CometScan parquet spark_catalog.default.web_sales (40)
+ : +- ReusedExchange (45)
+ +- CometBroadcastExchange (69)
+ +- CometHashAggregate (68)
+ +- CometExchange (67)
+ +- CometHashAggregate (66)
+ +- CometProject (65)
+ +- CometBroadcastHashJoin (64)
+ :- CometProject (62)
+ : +- CometBroadcastHashJoin (61)
+ : :- CometProject (57)
+ : : +- CometFilter (56)
+ : : +- CometScan parquet spark_catalog.default.customer (55)
+ : +- CometBroadcastExchange (60)
+ : +- CometFilter (59)
+ : +- CometScan parquet spark_catalog.default.web_sales (58)
+ +- ReusedExchange (63)
(1) CometScan parquet spark_catalog.default.customer
Output [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
+PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
(2) CometFilter
Input [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
-Condition : (isnotnull(c_customer_sk#1) AND isnotnull(c_customer_id#2))
+Condition : (isnotnull(c_customer_sk#1) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#2, 16, true, false, true)))
-(3) CometScan parquet spark_catalog.default.store_sales
-Output [4]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
+(3) CometProject
+Input [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
+Arguments: [c_customer_sk#1, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14], [c_customer_sk#1, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#2, 16, true, false, true) AS c_customer_id#9, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#3, 20, true, false, true) AS [...]
+
+(4) CometScan parquet spark_catalog.default.store_sales
+Output [4]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#12), dynamicpruningexpression(ss_sold_date_sk#12 IN dynamicpruning#13)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#18), dynamicpruningexpression(ss_sold_date_sk#18 IN dynamicpruning#19)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_ext_discount_amt:decimal(7,2),ss_ext_list_price:decimal(7,2)>
-(4) CometFilter
-Input [4]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
-Condition : isnotnull(ss_customer_sk#9)
+(5) CometFilter
+Input [4]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
+Condition : isnotnull(ss_customer_sk#15)
-(5) CometBroadcastExchange
-Input [4]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
-Arguments: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
+(6) CometBroadcastExchange
+Input [4]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
+Arguments: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
-(6) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
-Right output [4]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
-Arguments: [c_customer_sk#1], [ss_customer_sk#9], Inner, BuildRight
+(7) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#1, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14]
+Right output [4]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
+Arguments: [c_customer_sk#1], [ss_customer_sk#15], Inner, BuildRight
-(7) CometProject
-Input [12]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
-Arguments: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12], [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
+(8) CometProject
+Input [12]: [c_customer_sk#1, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
-(8) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#14, d_year#15]
+(9) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(9) CometFilter
-Input [2]: [d_date_sk#14, d_year#15]
-Condition : ((isnotnull(d_year#15) AND (d_year#15 = 2001)) AND isnotnull(d_date_sk#14))
+(10) CometFilter
+Input [2]: [d_date_sk#20, d_year#21]
+Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_date_sk#20))
-(10) CometBroadcastExchange
-Input [2]: [d_date_sk#14, d_year#15]
-Arguments: [d_date_sk#14, d_year#15]
+(11) CometBroadcastExchange
+Input [2]: [d_date_sk#20, d_year#21]
+Arguments: [d_date_sk#20, d_year#21]
-(11) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12]
-Right output [2]: [d_date_sk#14, d_year#15]
-Arguments: [ss_sold_date_sk#12], [d_date_sk#14], Inner, BuildRight
+(12) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18]
+Right output [2]: [d_date_sk#20, d_year#21]
+Arguments: [ss_sold_date_sk#18], [d_date_sk#20], Inner, BuildRight
-(12) CometProject
-Input [12]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, ss_sold_date_sk#12, d_date_sk#14, d_year#15]
-Arguments: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, d_year#15], [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, d_year#15]
+(13) CometProject
+Input [12]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, ss_sold_date_sk#18, d_date_sk#20, d_year#21]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, d_year#21], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, d_year#21]
-(13) CometHashAggregate
-Input [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_list_price#11, d_year#15]
-Keys [8]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
-Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#11 - ss_ext_discount_amt#10)))]
+(14) CometHashAggregate
+Input [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_list_price#17, d_year#21]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14]
+Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))]
-(14) CometExchange
-Input [9]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, sum#16]
-Arguments: hashpartitioning(c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(15) CometExchange
+Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
+Arguments: hashpartitioning(c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(15) CometHashAggregate
-Input [9]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, sum#16]
-Keys [8]: [c_customer_id#2, c_first_name#3, c_last_name#4, d_year#15, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
-Functions [1]: [sum(UnscaledValue((ss_ext_list_price#11 - ss_ext_discount_amt#10)))]
+(16) CometHashAggregate
+Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, sum#22]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#21, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14]
+Functions [1]: [sum(UnscaledValue((ss_ext_list_price#17 - ss_ext_discount_amt#16)))]
-(16) CometFilter
-Input [2]: [customer_id#17, year_total#18]
-Condition : (isnotnull(year_total#18) AND (year_total#18 > 0.00))
+(17) CometFilter
+Input [2]: [customer_id#23, year_total#24]
+Condition : (isnotnull(year_total#24) AND (year_total#24 > 0.00))
-(17) CometScan parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#19, c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26]
+(18) CometScan parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
+PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(18) CometFilter
-Input [8]: [c_customer_sk#19, c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26]
-Condition : (isnotnull(c_customer_sk#19) AND isnotnull(c_customer_id#20))
+(19) CometFilter
+Input [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
+Condition : (isnotnull(c_customer_sk#25) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#26, 16, true, false, true)))
-(19) CometScan parquet spark_catalog.default.store_sales
-Output [4]: [ss_customer_sk#27, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
+(20) CometProject
+Input [8]: [c_customer_sk#25, c_customer_id#26, c_first_name#27, c_last_name#28, c_preferred_cust_flag#29, c_birth_country#30, c_login#31, c_email_address#32]
+Arguments: [c_customer_sk#25, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14], [c_customer_sk#25, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#26, 16, true, false, true) AS c_customer_id#9, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#27, 20, true, false, tru [...]
+
+(21) CometScan parquet spark_catalog.default.store_sales
+Output [4]: [ss_customer_sk#33, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#30), dynamicpruningexpression(ss_sold_date_sk#30 IN dynamicpruning#31)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#36), dynamicpruningexpression(ss_sold_date_sk#36 IN dynamicpruning#37)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_ext_discount_amt:decimal(7,2),ss_ext_list_price:decimal(7,2)>
-(20) CometFilter
-Input [4]: [ss_customer_sk#27, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
-Condition : isnotnull(ss_customer_sk#27)
+(22) CometFilter
+Input [4]: [ss_customer_sk#33, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
+Condition : isnotnull(ss_customer_sk#33)
-(21) CometBroadcastExchange
-Input [4]: [ss_customer_sk#27, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
-Arguments: [ss_customer_sk#27, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
+(23) CometBroadcastExchange
+Input [4]: [ss_customer_sk#33, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
+Arguments: [ss_customer_sk#33, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
-(22) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#19, c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26]
-Right output [4]: [ss_customer_sk#27, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
-Arguments: [c_customer_sk#19], [ss_customer_sk#27], Inner, BuildRight
+(24) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#25, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14]
+Right output [4]: [ss_customer_sk#33, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
+Arguments: [c_customer_sk#25], [ss_customer_sk#33], Inner, BuildRight
-(23) CometProject
-Input [12]: [c_customer_sk#19, c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_customer_sk#27, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
-Arguments: [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30], [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
+(25) CometProject
+Input [12]: [c_customer_sk#25, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_customer_sk#33, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
-(24) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#32, d_year#33]
+(26) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#38, d_year#39]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(25) CometFilter
-Input [2]: [d_date_sk#32, d_year#33]
-Condition : ((isnotnull(d_year#33) AND (d_year#33 = 2002)) AND isnotnull(d_date_sk#32))
-
-(26) CometBroadcastExchange
-Input [2]: [d_date_sk#32, d_year#33]
-Arguments: [d_date_sk#32, d_year#33]
+(27) CometFilter
+Input [2]: [d_date_sk#38, d_year#39]
+Condition : ((isnotnull(d_year#39) AND (d_year#39 = 2002)) AND isnotnull(d_date_sk#38))
-(27) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30]
-Right output [2]: [d_date_sk#32, d_year#33]
-Arguments: [ss_sold_date_sk#30], [d_date_sk#32], Inner, BuildRight
+(28) CometBroadcastExchange
+Input [2]: [d_date_sk#38, d_year#39]
+Arguments: [d_date_sk#38, d_year#39]
-(28) CometProject
-Input [12]: [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, ss_sold_date_sk#30, d_date_sk#32, d_year#33]
-Arguments: [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, d_year#33], [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, d_year#33]
+(29) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36]
+Right output [2]: [d_date_sk#38, d_year#39]
+Arguments: [ss_sold_date_sk#36], [d_date_sk#38], Inner, BuildRight
-(29) CometHashAggregate
-Input [10]: [c_customer_id#20, c_first_name#21, c_last_name#22, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, ss_ext_discount_amt#28, ss_ext_list_price#29, d_year#33]
-Keys [8]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26]
-Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#29 - ss_ext_discount_amt#28)))]
-
-(30) CometExchange
-Input [9]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, sum#34]
-Arguments: hashpartitioning(c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+(30) CometProject
+Input [12]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, ss_sold_date_sk#36, d_date_sk#38, d_year#39]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, d_year#39], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, d_year#39]
(31) CometHashAggregate
-Input [9]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26, sum#34]
-Keys [8]: [c_customer_id#20, c_first_name#21, c_last_name#22, d_year#33, c_preferred_cust_flag#23, c_birth_country#24, c_login#25, c_email_address#26]
-Functions [1]: [sum(UnscaledValue((ss_ext_list_price#29 - ss_ext_discount_amt#28)))]
-
-(32) CometBroadcastExchange
-Input [3]: [customer_id#35, customer_preferred_cust_flag#36, year_total#37]
-Arguments: [customer_id#35, customer_preferred_cust_flag#36, year_total#37]
-
-(33) CometBroadcastHashJoin
-Left output [2]: [customer_id#17, year_total#18]
-Right output [3]: [customer_id#35, customer_preferred_cust_flag#36, year_total#37]
-Arguments: [rpad(customer_id#17, 16, )], [rpad(customer_id#35, 16, )], Inner, BuildRight
-
-(34) CometProject
-Input [5]: [customer_id#17, year_total#18, customer_id#35, customer_preferred_cust_flag#36, year_total#37]
-Arguments: [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37], [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37]
-
-(35) CometScan parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#38, c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45]
+Input [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, ss_ext_discount_amt#34, ss_ext_list_price#35, d_year#39]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#39, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14]
+Functions [1]: [partial_sum(UnscaledValue((ss_ext_list_price#35 - ss_ext_discount_amt#34)))]
+
+(32) CometExchange
+Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#39, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, sum#40]
+Arguments: hashpartitioning(c_customer_id#9, c_first_name#10, c_last_name#11, d_year#39, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(33) CometHashAggregate
+Input [9]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#39, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14, sum#40]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, d_year#39, c_preferred_cust_flag#12, c_birth_country#30, c_login#13, c_email_address#14]
+Functions [1]: [sum(UnscaledValue((ss_ext_list_price#35 - ss_ext_discount_amt#34)))]
+
+(34) CometBroadcastExchange
+Input [3]: [customer_id#41, customer_preferred_cust_flag#42, year_total#43]
+Arguments: [customer_id#41, customer_preferred_cust_flag#42, year_total#43]
+
+(35) CometBroadcastHashJoin
+Left output [2]: [customer_id#23, year_total#24]
+Right output [3]: [customer_id#41, customer_preferred_cust_flag#42, year_total#43]
+Arguments: [customer_id#23], [customer_id#41], Inner, BuildRight
+
+(36) CometProject
+Input [5]: [customer_id#23, year_total#24, customer_id#41, customer_preferred_cust_flag#42, year_total#43]
+Arguments: [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43], [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43]
+
+(37) CometScan parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#44, c_customer_id#45, c_first_name#46, c_last_name#47, c_preferred_cust_flag#48, c_birth_country#49, c_login#50, c_email_address#51]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
+PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(36) CometFilter
-Input [8]: [c_customer_sk#38, c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45]
-Condition : (isnotnull(c_customer_sk#38) AND isnotnull(c_customer_id#39))
+(38) CometFilter
+Input [8]: [c_customer_sk#44, c_customer_id#45, c_first_name#46, c_last_name#47, c_preferred_cust_flag#48, c_birth_country#49, c_login#50, c_email_address#51]
+Condition : (isnotnull(c_customer_sk#44) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#45, 16, true, false, true)))
+
+(39) CometProject
+Input [8]: [c_customer_sk#44, c_customer_id#45, c_first_name#46, c_last_name#47, c_preferred_cust_flag#48, c_birth_country#49, c_login#50, c_email_address#51]
+Arguments: [c_customer_sk#44, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57], [c_customer_sk#44, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#45, 16, true, false, true) AS c_customer_id#52, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#46, 20, true, false, t [...]
-(37) CometScan parquet spark_catalog.default.web_sales
-Output [4]: [ws_bill_customer_sk#46, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
+(40) CometScan parquet spark_catalog.default.web_sales
+Output [4]: [ws_bill_customer_sk#58, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#49), dynamicpruningexpression(ws_sold_date_sk#49 IN dynamicpruning#50)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#61), dynamicpruningexpression(ws_sold_date_sk#61 IN dynamicpruning#62)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int,ws_ext_discount_amt:decimal(7,2),ws_ext_list_price:decimal(7,2)>
-(38) CometFilter
-Input [4]: [ws_bill_customer_sk#46, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-Condition : isnotnull(ws_bill_customer_sk#46)
-
-(39) CometBroadcastExchange
-Input [4]: [ws_bill_customer_sk#46, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-Arguments: [ws_bill_customer_sk#46, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-
-(40) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#38, c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45]
-Right output [4]: [ws_bill_customer_sk#46, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-Arguments: [c_customer_sk#38], [ws_bill_customer_sk#46], Inner, BuildRight
+(41) CometFilter
+Input [4]: [ws_bill_customer_sk#58, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
+Condition : isnotnull(ws_bill_customer_sk#58)
-(41) CometProject
-Input [12]: [c_customer_sk#38, c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_bill_customer_sk#46, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-Arguments: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49], [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-
-(42) ReusedExchange [Reuses operator id: 10]
-Output [2]: [d_date_sk#51, d_year#52]
+(42) CometBroadcastExchange
+Input [4]: [ws_bill_customer_sk#58, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
+Arguments: [ws_bill_customer_sk#58, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
(43) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49]
-Right output [2]: [d_date_sk#51, d_year#52]
-Arguments: [ws_sold_date_sk#49], [d_date_sk#51], Inner, BuildRight
+Left output [8]: [c_customer_sk#44, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57]
+Right output [4]: [ws_bill_customer_sk#58, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
+Arguments: [c_customer_sk#44], [ws_bill_customer_sk#58], Inner, BuildRight
(44) CometProject
-Input [12]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, ws_sold_date_sk#49, d_date_sk#51, d_year#52]
-Arguments: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, d_year#52], [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, d_year#52]
-
-(45) CometHashAggregate
-Input [10]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, ws_ext_discount_amt#47, ws_ext_list_price#48, d_year#52]
-Keys [8]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52]
-Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#48 - ws_ext_discount_amt#47)))]
-
-(46) CometExchange
-Input [9]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, sum#53]
-Arguments: hashpartitioning(c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
-
-(47) CometHashAggregate
-Input [9]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52, sum#53]
-Keys [8]: [c_customer_id#39, c_first_name#40, c_last_name#41, c_preferred_cust_flag#42, c_birth_country#43, c_login#44, c_email_address#45, d_year#52]
-Functions [1]: [sum(UnscaledValue((ws_ext_list_price#48 - ws_ext_discount_amt#47)))]
-
-(48) CometFilter
-Input [2]: [customer_id#54, year_total#55]
-Condition : (isnotnull(year_total#55) AND (year_total#55 > 0.00))
-
-(49) CometBroadcastExchange
-Input [2]: [customer_id#54, year_total#55]
-Arguments: [customer_id#54, year_total#55]
-
-(50) CometBroadcastHashJoin
-Left output [4]: [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37]
-Right output [2]: [customer_id#54, year_total#55]
-Arguments: [rpad(customer_id#17, 16, )], [rpad(customer_id#54, 16, )], Inner, BuildRight
-
-(51) CometProject
-Input [6]: [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37, customer_id#54, year_total#55]
-Arguments: [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37, year_total#55], [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37, year_total#55]
-
-(52) CometScan parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#56, c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63]
+Input [12]: [c_customer_sk#44, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_bill_customer_sk#58, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
+Arguments: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61], [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
+
+(45) ReusedExchange [Reuses operator id: 11]
+Output [2]: [d_date_sk#63, d_year#64]
+
+(46) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61]
+Right output [2]: [d_date_sk#63, d_year#64]
+Arguments: [ws_sold_date_sk#61], [d_date_sk#63], Inner, BuildRight
+
+(47) CometProject
+Input [12]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, ws_sold_date_sk#61, d_date_sk#63, d_year#64]
+Arguments: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, d_year#64], [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, d_year#64]
+
+(48) CometHashAggregate
+Input [10]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, ws_ext_discount_amt#59, ws_ext_list_price#60, d_year#64]
+Keys [8]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, d_year#64]
+Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#60 - ws_ext_discount_amt#59)))]
+
+(49) CometExchange
+Input [9]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, d_year#64, sum#65]
+Arguments: hashpartitioning(c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, d_year#64, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+
+(50) CometHashAggregate
+Input [9]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, d_year#64, sum#65]
+Keys [8]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#49, c_login#56, c_email_address#57, d_year#64]
+Functions [1]: [sum(UnscaledValue((ws_ext_list_price#60 - ws_ext_discount_amt#59)))]
+
+(51) CometFilter
+Input [2]: [customer_id#66, year_total#67]
+Condition : (isnotnull(year_total#67) AND (year_total#67 > 0.00))
+
+(52) CometBroadcastExchange
+Input [2]: [customer_id#66, year_total#67]
+Arguments: [customer_id#66, year_total#67]
+
+(53) CometBroadcastHashJoin
+Left output [4]: [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43]
+Right output [2]: [customer_id#66, year_total#67]
+Arguments: [customer_id#23], [customer_id#66], Inner, BuildRight
+
+(54) CometProject
+Input [6]: [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43, customer_id#66, year_total#67]
+Arguments: [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43, year_total#67], [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43, year_total#67]
+
+(55) CometScan parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#68, c_customer_id#69, c_first_name#70, c_last_name#71, c_preferred_cust_flag#72, c_birth_country#73, c_login#74, c_email_address#75]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
+PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(53) CometFilter
-Input [8]: [c_customer_sk#56, c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63]
-Condition : (isnotnull(c_customer_sk#56) AND isnotnull(c_customer_id#57))
+(56) CometFilter
+Input [8]: [c_customer_sk#68, c_customer_id#69, c_first_name#70, c_last_name#71, c_preferred_cust_flag#72, c_birth_country#73, c_login#74, c_email_address#75]
+Condition : (isnotnull(c_customer_sk#68) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#69, 16, true, false, true)))
+
+(57) CometProject
+Input [8]: [c_customer_sk#68, c_customer_id#69, c_first_name#70, c_last_name#71, c_preferred_cust_flag#72, c_birth_country#73, c_login#74, c_email_address#75]
+Arguments: [c_customer_sk#68, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57], [c_customer_sk#68, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#69, 16, true, false, true) AS c_customer_id#52, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#70, 20, true, false, t [...]
-(54) CometScan parquet spark_catalog.default.web_sales
-Output [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+(58) CometScan parquet spark_catalog.default.web_sales
+Output [4]: [ws_bill_customer_sk#76, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#67), dynamicpruningexpression(ws_sold_date_sk#67 IN dynamicpruning#68)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#79), dynamicpruningexpression(ws_sold_date_sk#79 IN dynamicpruning#80)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int,ws_ext_discount_amt:decimal(7,2),ws_ext_list_price:decimal(7,2)>
-(55) CometFilter
-Input [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-Condition : isnotnull(ws_bill_customer_sk#64)
+(59) CometFilter
+Input [4]: [ws_bill_customer_sk#76, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
+Condition : isnotnull(ws_bill_customer_sk#76)
-(56) CometBroadcastExchange
-Input [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-Arguments: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+(60) CometBroadcastExchange
+Input [4]: [ws_bill_customer_sk#76, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
+Arguments: [ws_bill_customer_sk#76, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
-(57) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#56, c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63]
-Right output [4]: [ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-Arguments: [c_customer_sk#56], [ws_bill_customer_sk#64], Inner, BuildRight
+(61) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#68, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57]
+Right output [4]: [ws_bill_customer_sk#76, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
+Arguments: [c_customer_sk#68], [ws_bill_customer_sk#76], Inner, BuildRight
-(58) CometProject
-Input [12]: [c_customer_sk#56, c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_bill_customer_sk#64, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-Arguments: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67], [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
+(62) CometProject
+Input [12]: [c_customer_sk#68, c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_bill_customer_sk#76, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
+Arguments: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79], [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
-(59) ReusedExchange [Reuses operator id: 26]
-Output [2]: [d_date_sk#69, d_year#70]
+(63) ReusedExchange [Reuses operator id: 28]
+Output [2]: [d_date_sk#81, d_year#82]
-(60) CometBroadcastHashJoin
-Left output [10]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67]
-Right output [2]: [d_date_sk#69, d_year#70]
-Arguments: [ws_sold_date_sk#67], [d_date_sk#69], Inner, BuildRight
+(64) CometBroadcastHashJoin
+Left output [10]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79]
+Right output [2]: [d_date_sk#81, d_year#82]
+Arguments: [ws_sold_date_sk#79], [d_date_sk#81], Inner, BuildRight
-(61) CometProject
-Input [12]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, ws_sold_date_sk#67, d_date_sk#69, d_year#70]
-Arguments: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70], [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70]
+(65) CometProject
+Input [12]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, ws_sold_date_sk#79, d_date_sk#81, d_year#82]
+Arguments: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, d_year#82], [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, d_year#82]
-(62) CometHashAggregate
-Input [10]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, ws_ext_discount_amt#65, ws_ext_list_price#66, d_year#70]
-Keys [8]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70]
-Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
+(66) CometHashAggregate
+Input [10]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, ws_ext_discount_amt#77, ws_ext_list_price#78, d_year#82]
+Keys [8]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, d_year#82]
+Functions [1]: [partial_sum(UnscaledValue((ws_ext_list_price#78 - ws_ext_discount_amt#77)))]
-(63) CometExchange
-Input [9]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, sum#71]
-Arguments: hashpartitioning(c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
+(67) CometExchange
+Input [9]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, d_year#82, sum#83]
+Arguments: hashpartitioning(c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, d_year#82, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
-(64) CometHashAggregate
-Input [9]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70, sum#71]
-Keys [8]: [c_customer_id#57, c_first_name#58, c_last_name#59, c_preferred_cust_flag#60, c_birth_country#61, c_login#62, c_email_address#63, d_year#70]
-Functions [1]: [sum(UnscaledValue((ws_ext_list_price#66 - ws_ext_discount_amt#65)))]
+(68) CometHashAggregate
+Input [9]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, d_year#82, sum#83]
+Keys [8]: [c_customer_id#52, c_first_name#53, c_last_name#54, c_preferred_cust_flag#55, c_birth_country#73, c_login#56, c_email_address#57, d_year#82]
+Functions [1]: [sum(UnscaledValue((ws_ext_list_price#78 - ws_ext_discount_amt#77)))]
-(65) CometBroadcastExchange
-Input [2]: [customer_id#72, year_total#73]
-Arguments: [customer_id#72, year_total#73]
+(69) CometBroadcastExchange
+Input [2]: [customer_id#84, year_total#85]
+Arguments: [customer_id#84, year_total#85]
-(66) CometBroadcastHashJoin
-Left output [5]: [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37, year_total#55]
-Right output [2]: [customer_id#72, year_total#73]
-Arguments: [rpad(customer_id#17, 16, )], [rpad(customer_id#72, 16, )], Inner, (CASE WHEN (year_total#55 > 0.00) THEN (year_total#73 / year_total#55) END > CASE WHEN (year_total#18 > 0.00) THEN (year_total#37 / year_total#18) END), BuildRight
+(70) CometBroadcastHashJoin
+Left output [5]: [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43, year_total#67]
+Right output [2]: [customer_id#84, year_total#85]
+Arguments: [customer_id#23], [customer_id#84], Inner, (CASE WHEN (year_total#67 > 0.00) THEN (year_total#85 / year_total#67) END > CASE WHEN (year_total#24 > 0.00) THEN (year_total#43 / year_total#24) END), BuildRight
-(67) CometProject
-Input [7]: [customer_id#17, year_total#18, customer_preferred_cust_flag#36, year_total#37, year_total#55, customer_id#72, year_total#73]
-Arguments: [customer_preferred_cust_flag#36], [customer_preferred_cust_flag#36]
+(71) CometProject
+Input [7]: [customer_id#23, year_total#24, customer_preferred_cust_flag#42, year_total#43, year_total#67, customer_id#84, year_total#85]
+Arguments: [customer_preferred_cust_flag#42], [customer_preferred_cust_flag#42]
-(68) CometTakeOrderedAndProject
-Input [1]: [customer_preferred_cust_flag#36]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[customer_preferred_cust_flag#36 ASC NULLS FIRST], output=[customer_preferred_cust_flag#36]), [customer_preferred_cust_flag#36], 100, [customer_preferred_cust_flag#36 ASC NULLS FIRST], [customer_preferred_cust_flag#36]
+(72) CometTakeOrderedAndProject
+Input [1]: [customer_preferred_cust_flag#42]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[customer_preferred_cust_flag#42 ASC NULLS FIRST], output=[customer_preferred_cust_flag#42]), [customer_preferred_cust_flag#42], 100, [customer_preferred_cust_flag#42 ASC NULLS FIRST], [customer_preferred_cust_flag#42]
-(69) CometColumnarToRow [codegen id : 1]
-Input [1]: [customer_preferred_cust_flag#36]
+(73) CometColumnarToRow [codegen id : 1]
+Input [1]: [customer_preferred_cust_flag#42]
===== Subqueries =====
-Subquery:1 Hosting operator id = 3 Hosting Expression = ss_sold_date_sk#12 IN dynamicpruning#13
-BroadcastExchange (73)
-+- * CometColumnarToRow (72)
- +- CometFilter (71)
- +- CometScan parquet spark_catalog.default.date_dim (70)
+Subquery:1 Hosting operator id = 4 Hosting Expression = ss_sold_date_sk#18 IN dynamicpruning#19
+BroadcastExchange (77)
++- * CometColumnarToRow (76)
+ +- CometFilter (75)
+ +- CometScan parquet spark_catalog.default.date_dim (74)
-(70) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#14, d_year#15]
+(74) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(71) CometFilter
-Input [2]: [d_date_sk#14, d_year#15]
-Condition : ((isnotnull(d_year#15) AND (d_year#15 = 2001)) AND isnotnull(d_date_sk#14))
+(75) CometFilter
+Input [2]: [d_date_sk#20, d_year#21]
+Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_date_sk#20))
-(72) CometColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#14, d_year#15]
+(76) CometColumnarToRow [codegen id : 1]
+Input [2]: [d_date_sk#20, d_year#21]
-(73) BroadcastExchange
-Input [2]: [d_date_sk#14, d_year#15]
+(77) BroadcastExchange
+Input [2]: [d_date_sk#20, d_year#21]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=5]
-Subquery:2 Hosting operator id = 19 Hosting Expression = ss_sold_date_sk#30 IN dynamicpruning#31
-BroadcastExchange (77)
-+- * CometColumnarToRow (76)
- +- CometFilter (75)
- +- CometScan parquet spark_catalog.default.date_dim (74)
+Subquery:2 Hosting operator id = 21 Hosting Expression = ss_sold_date_sk#36 IN dynamicpruning#37
+BroadcastExchange (81)
++- * CometColumnarToRow (80)
+ +- CometFilter (79)
+ +- CometScan parquet spark_catalog.default.date_dim (78)
-(74) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#32, d_year#33]
+(78) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#38, d_year#39]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(75) CometFilter
-Input [2]: [d_date_sk#32, d_year#33]
-Condition : ((isnotnull(d_year#33) AND (d_year#33 = 2002)) AND isnotnull(d_date_sk#32))
+(79) CometFilter
+Input [2]: [d_date_sk#38, d_year#39]
+Condition : ((isnotnull(d_year#39) AND (d_year#39 = 2002)) AND isnotnull(d_date_sk#38))
-(76) CometColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#32, d_year#33]
+(80) CometColumnarToRow [codegen id : 1]
+Input [2]: [d_date_sk#38, d_year#39]
-(77) BroadcastExchange
-Input [2]: [d_date_sk#32, d_year#33]
+(81) BroadcastExchange
+Input [2]: [d_date_sk#38, d_year#39]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=6]
-Subquery:3 Hosting operator id = 37 Hosting Expression = ws_sold_date_sk#49 IN dynamicpruning#13
+Subquery:3 Hosting operator id = 40 Hosting Expression = ws_sold_date_sk#61 IN dynamicpruning#19
-Subquery:4 Hosting operator id = 54 Hosting Expression = ws_sold_date_sk#67 IN dynamicpruning#31
+Subquery:4 Hosting operator id = 58 Hosting Expression = ws_sold_date_sk#79 IN dynamicpruning#37
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt
index 8896bc14b..2a70a18b4 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q11/simplified.txt
@@ -16,8 +16,9 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
CometBroadcastExchange [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk] #2
CometFilter [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
@@ -39,8 +40,9 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk,d_date_sk,d_year]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
CometBroadcastExchange [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk] #7
CometFilter [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_discount_amt,ss_ext_list_price,ss_sold_date_sk]
@@ -63,8 +65,9 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #12
CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
CometScan parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
@@ -78,8 +81,9 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk,d_date_sk,d_year]
CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
CometBroadcastHashJoin [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address,ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
- CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometProject [c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometFilter [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_country,c_login,c_email_address]
CometBroadcastExchange [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk] #15
CometFilter [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
CometScan parquet spark_catalog.default.web_sales [ws_bill_customer_sk,ws_ext_discount_amt,ws_ext_list_price,ws_sold_date_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt
index e83d23aab..65679eb2b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/explain.txt
@@ -1,26 +1,27 @@
== Physical Plan ==
-TakeOrderedAndProject (22)
-+- * Project (21)
- +- Window (20)
- +- * CometColumnarToRow (19)
- +- CometSort (18)
- +- CometExchange (17)
- +- CometHashAggregate (16)
- +- CometExchange (15)
- +- CometHashAggregate (14)
- +- CometProject (13)
- +- CometBroadcastHashJoin (12)
- :- CometProject (7)
- : +- CometBroadcastHashJoin (6)
+TakeOrderedAndProject (23)
++- * Project (22)
+ +- Window (21)
+ +- * CometColumnarToRow (20)
+ +- CometSort (19)
+ +- CometExchange (18)
+ +- CometHashAggregate (17)
+ +- CometExchange (16)
+ +- CometHashAggregate (15)
+ +- CometProject (14)
+ +- CometBroadcastHashJoin (13)
+ :- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
: :- CometFilter (2)
: : +- CometScan parquet spark_catalog.default.web_sales (1)
- : +- CometBroadcastExchange (5)
- : +- CometFilter (4)
- : +- CometScan parquet spark_catalog.default.item (3)
- +- CometBroadcastExchange (11)
- +- CometProject (10)
- +- CometFilter (9)
- +- CometScan parquet spark_catalog.default.date_dim (8)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan parquet spark_catalog.default.item (3)
+ +- CometBroadcastExchange (12)
+ +- CometProject (11)
+ +- CometFilter (10)
+ +- CometScan parquet spark_catalog.default.date_dim (9)
(1) CometScan parquet spark_catalog.default.web_sales
@@ -44,116 +45,120 @@ ReadSchema: struct<i_item_sk:int,i_item_id:string,i_item_desc:string,i_current_p
(4) CometFilter
Input [6]: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Condition : (rpad(i_category#10, 50, ) IN (Sports ,Books ,Home ) AND isnotnull(i_item_sk#5))
+Condition : (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_category#10, 50, true, false, true) IN (Sports ,Books ,Home ) AND isnotnull(i_item_sk#5))
-(5) CometBroadcastExchange
+(5) CometProject
Input [6]: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Arguments: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+Arguments: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13], [i_item_sk#5, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#6, 16, true, false, true) AS i_item_id#11, i_item_desc#7, i_current_price#8, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_class#9, 50, true, false, true) AS i_class#12, staticinvoke(class org.ap [...]
-(6) CometBroadcastHashJoin
+(6) CometBroadcastExchange
+Input [6]: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Arguments: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+
+(7) CometBroadcastHashJoin
Left output [3]: [ws_item_sk#1, ws_ext_sales_price#2, ws_sold_date_sk#3]
-Right output [6]: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+Right output [6]: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
Arguments: [ws_item_sk#1], [i_item_sk#5], Inner, BuildRight
-(7) CometProject
-Input [9]: [ws_item_sk#1, ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Arguments: [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10], [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+(8) CometProject
+Input [9]: [ws_item_sk#1, ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Arguments: [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13], [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
-(8) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#11, d_date#12]
+(9) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(9) CometFilter
-Input [2]: [d_date_sk#11, d_date#12]
-Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 1999-02-22)) AND (d_date#12 <= 1999-03-24)) AND isnotnull(d_date_sk#11))
+(10) CometFilter
+Input [2]: [d_date_sk#14, d_date#15]
+Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(10) CometProject
-Input [2]: [d_date_sk#11, d_date#12]
-Arguments: [d_date_sk#11], [d_date_sk#11]
+(11) CometProject
+Input [2]: [d_date_sk#14, d_date#15]
+Arguments: [d_date_sk#14], [d_date_sk#14]
-(11) CometBroadcastExchange
-Input [1]: [d_date_sk#11]
-Arguments: [d_date_sk#11]
+(12) CometBroadcastExchange
+Input [1]: [d_date_sk#14]
+Arguments: [d_date_sk#14]
-(12) CometBroadcastHashJoin
-Left output [7]: [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Right output [1]: [d_date_sk#11]
-Arguments: [ws_sold_date_sk#3], [d_date_sk#11], Inner, BuildRight
+(13) CometBroadcastHashJoin
+Left output [7]: [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Right output [1]: [d_date_sk#14]
+Arguments: [ws_sold_date_sk#3], [d_date_sk#14], Inner, BuildRight
-(13) CometProject
-Input [8]: [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10, d_date_sk#11]
-Arguments: [ws_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10], [ws_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+(14) CometProject
+Input [8]: [ws_ext_sales_price#2, ws_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13, d_date_sk#14]
+Arguments: [ws_ext_sales_price#2, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13], [ws_ext_sales_price#2, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
-(14) CometHashAggregate
-Input [6]: [ws_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
+(15) CometHashAggregate
+Input [6]: [ws_ext_sales_price#2, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [partial_sum(UnscaledValue(ws_ext_sales_price#2))]
-(15) CometExchange
-Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
-Arguments: hashpartitioning(i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(16) CometExchange
+Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
+Arguments: hashpartitioning(i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(16) CometHashAggregate
-Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
-Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
+(17) CometHashAggregate
+Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
+Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [sum(UnscaledValue(ws_ext_sales_price#2))]
-(17) CometExchange
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: hashpartitioning(i_class#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+(18) CometExchange
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(18) CometSort
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6], [i_class#9 ASC NULLS FIRST]
+(19) CometSort
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11], [i_class#12 ASC NULLS FIRST]
-(19) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
+(20) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
-(20) Window
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: [sum(_w0#15) windowspecdefinition(i_class#9, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#16], [i_class#9]
+(21) Window
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [sum(_w0#18) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#19], [i_class#12]
-(21) Project [codegen id : 2]
-Output [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, ((_w0#15 * 100) / _we0#16) AS revenueratio#17, i_item_id#6]
-Input [8]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6, _we0#16]
+(22) Project [codegen id : 2]
+Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, ((_w0#18 * 100) / _we0#19) AS revenueratio#20, i_item_id#11]
+Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11, _we0#19]
-(22) TakeOrderedAndProject
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, revenueratio#17, i_item_id#6]
-Arguments: 100, [i_category#10 ASC NULLS FIRST, i_class#9 ASC NULLS FIRST, i_item_id#6 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#17 ASC NULLS FIRST], [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, revenueratio#17]
+(23) TakeOrderedAndProject
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20, i_item_id#11]
+Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#20 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ws_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (27)
-+- * CometColumnarToRow (26)
- +- CometProject (25)
- +- CometFilter (24)
- +- CometScan parquet spark_catalog.default.date_dim (23)
+BroadcastExchange (28)
++- * CometColumnarToRow (27)
+ +- CometProject (26)
+ +- CometFilter (25)
+ +- CometScan parquet spark_catalog.default.date_dim (24)
-(23) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#11, d_date#12]
+(24) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(24) CometFilter
-Input [2]: [d_date_sk#11, d_date#12]
-Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 1999-02-22)) AND (d_date#12 <= 1999-03-24)) AND isnotnull(d_date_sk#11))
+(25) CometFilter
+Input [2]: [d_date_sk#14, d_date#15]
+Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(25) CometProject
-Input [2]: [d_date_sk#11, d_date#12]
-Arguments: [d_date_sk#11], [d_date_sk#11]
+(26) CometProject
+Input [2]: [d_date_sk#14, d_date#15]
+Arguments: [d_date_sk#14], [d_date_sk#14]
-(26) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#11]
+(27) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#14]
-(27) BroadcastExchange
-Input [1]: [d_date_sk#11]
+(28) BroadcastExchange
+Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt
index c4c49fe33..2ced47772 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q12/simplified.txt
@@ -26,8 +26,9 @@ TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_c
CometFilter [d_date_sk,d_date]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
- CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
CometBroadcastExchange [d_date_sk] #5
CometProject [d_date_sk]
CometFilter [d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt
index a9b0d0e78..d6d427c98 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/explain.txt
@@ -1,12 +1,12 @@
== Physical Plan ==
-* CometColumnarToRow (33)
-+- CometHashAggregate (32)
- +- CometExchange (31)
- +- CometHashAggregate (30)
- +- CometProject (29)
- +- CometBroadcastHashJoin (28)
- :- CometProject (24)
- : +- CometBroadcastHashJoin (23)
+* CometColumnarToRow (34)
++- CometHashAggregate (33)
+ +- CometExchange (32)
+ +- CometHashAggregate (31)
+ +- CometProject (30)
+ +- CometBroadcastHashJoin (29)
+ :- CometProject (25)
+ : +- CometBroadcastHashJoin (24)
: :- CometProject (19)
: : +- CometBroadcastHashJoin (18)
: : :- CometProject (13)
@@ -26,12 +26,13 @@
: : +- CometProject (16)
: : +- CometFilter (15)
: : +- CometScan parquet spark_catalog.default.date_dim (14)
- : +- CometBroadcastExchange (22)
- : +- CometFilter (21)
- : +- CometScan parquet spark_catalog.default.customer_demographics (20)
- +- CometBroadcastExchange (27)
- +- CometFilter (26)
- +- CometScan parquet spark_catalog.default.household_demographics (25)
+ : +- CometBroadcastExchange (23)
+ : +- CometProject (22)
+ : +- CometFilter (21)
+ : +- CometScan parquet spark_catalog.default.customer_demographics (20)
+ +- CometBroadcastExchange (28)
+ +- CometFilter (27)
+ +- CometScan parquet spark_catalog.default.household_demographics (26)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -79,148 +80,152 @@ ReadSchema: struct<ca_address_sk:int,ca_state:string,ca_country:string>
(9) CometFilter
Input [3]: [ca_address_sk#13, ca_state#14, ca_country#15]
-Condition : (((isnotnull(ca_country#15) AND (ca_country#15 = United States)) AND isnotnull(ca_address_sk#13)) AND ((rpad(ca_state#14, 2, ) IN (TX,OH) OR rpad(ca_state#14, 2, ) IN (OR,NM,KY)) OR rpad(ca_state#14, 2, ) IN (VA,TX,MS)))
+Condition : (((isnotnull(ca_country#15) AND (ca_country#15 = United States)) AND isnotnull(ca_address_sk#13)) AND ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#14, 2, true, false, true) IN (TX,OH) OR staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#14, 2, true, false, true) IN (OR,NM,KY)) OR staticinvoke(class org.apache.spark.sql.catalyst.util.Char [...]
(10) CometProject
Input [3]: [ca_address_sk#13, ca_state#14, ca_country#15]
-Arguments: [ca_address_sk#13, ca_state#14], [ca_address_sk#13, ca_state#14]
+Arguments: [ca_address_sk#13, ca_state#16], [ca_address_sk#13, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#14, 2, true, false, true) AS ca_state#16]
(11) CometBroadcastExchange
-Input [2]: [ca_address_sk#13, ca_state#14]
-Arguments: [ca_address_sk#13, ca_state#14]
+Input [2]: [ca_address_sk#13, ca_state#16]
+Arguments: [ca_address_sk#13, ca_state#16]
(12) CometBroadcastHashJoin
Left output [9]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_addr_sk#3, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_net_profit#9, ss_sold_date_sk#10]
-Right output [2]: [ca_address_sk#13, ca_state#14]
-Arguments: [ss_addr_sk#3], [ca_address_sk#13], Inner, ((((rpad(ca_state#14, 2, ) IN (TX,OH) AND (ss_net_profit#9 >= 100.00)) AND (ss_net_profit#9 <= 200.00)) OR ((rpad(ca_state#14, 2, ) IN (OR,NM,KY) AND (ss_net_profit#9 >= 150.00)) AND (ss_net_profit#9 <= 300.00))) OR ((rpad(ca_state#14, 2, ) IN (VA,TX,MS) AND (ss_net_profit#9 >= 50.00)) AND (ss_net_profit#9 <= 250.00))), BuildRight
+Right output [2]: [ca_address_sk#13, ca_state#16]
+Arguments: [ss_addr_sk#3], [ca_address_sk#13], Inner, ((((ca_state#16 IN (TX,OH) AND (ss_net_profit#9 >= 100.00)) AND (ss_net_profit#9 <= 200.00)) OR ((ca_state#16 IN (OR,NM,KY) AND (ss_net_profit#9 >= 150.00)) AND (ss_net_profit#9 <= 300.00))) OR ((ca_state#16 IN (VA,TX,MS) AND (ss_net_profit#9 >= 50.00)) AND (ss_net_profit#9 <= 250.00))), BuildRight
(13) CometProject
-Input [11]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_addr_sk#3, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_net_profit#9, ss_sold_date_sk#10, ca_address_sk#13, ca_state#14]
+Input [11]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_addr_sk#3, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_net_profit#9, ss_sold_date_sk#10, ca_address_sk#13, ca_state#16]
Arguments: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_sold_date_sk#10], [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_sold_date_sk#10]
(14) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#16, d_year#17]
+Output [2]: [d_date_sk#17, d_year#18]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
(15) CometFilter
-Input [2]: [d_date_sk#16, d_year#17]
-Condition : ((isnotnull(d_year#17) AND (d_year#17 = 2001)) AND isnotnull(d_date_sk#16))
+Input [2]: [d_date_sk#17, d_year#18]
+Condition : ((isnotnull(d_year#18) AND (d_year#18 = 2001)) AND isnotnull(d_date_sk#17))
(16) CometProject
-Input [2]: [d_date_sk#16, d_year#17]
-Arguments: [d_date_sk#16], [d_date_sk#16]
+Input [2]: [d_date_sk#17, d_year#18]
+Arguments: [d_date_sk#17], [d_date_sk#17]
(17) CometBroadcastExchange
-Input [1]: [d_date_sk#16]
-Arguments: [d_date_sk#16]
+Input [1]: [d_date_sk#17]
+Arguments: [d_date_sk#17]
(18) CometBroadcastHashJoin
Left output [7]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_sold_date_sk#10]
-Right output [1]: [d_date_sk#16]
-Arguments: [ss_sold_date_sk#10], [d_date_sk#16], Inner, BuildRight
+Right output [1]: [d_date_sk#17]
+Arguments: [ss_sold_date_sk#10], [d_date_sk#17], Inner, BuildRight
(19) CometProject
-Input [8]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_sold_date_sk#10, d_date_sk#16]
+Input [8]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, ss_sold_date_sk#10, d_date_sk#17]
Arguments: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8], [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8]
(20) CometScan parquet spark_catalog.default.customer_demographics
-Output [3]: [cd_demo_sk#18, cd_marital_status#19, cd_education_status#20]
+Output [3]: [cd_demo_sk#19, cd_marital_status#20, cd_education_status#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_demographics]
PushedFilters: [IsNotNull(cd_demo_sk)]
ReadSchema: struct<cd_demo_sk:int,cd_marital_status:string,cd_education_status:string>
(21) CometFilter
-Input [3]: [cd_demo_sk#18, cd_marital_status#19, cd_education_status#20]
-Condition : (isnotnull(cd_demo_sk#18) AND ((((rpad(cd_marital_status#19, 1, ) = M) AND (rpad(cd_education_status#20, 20, ) = Advanced Degree )) OR ((rpad(cd_marital_status#19, 1, ) = S) AND (rpad(cd_education_status#20, 20, ) = College ))) OR ((rpad(cd_marital_status#19, 1, ) = W) AND (rpad(cd_education_status#20, 20, ) = 2 yr Degree ))))
+Input [3]: [cd_demo_sk#19, cd_marital_status#20, cd_education_status#21]
+Condition : (isnotnull(cd_demo_sk#19) AND ((((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_marital_status#20, 1, true, false, true) = M) AND (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_education_status#21, 20, true, false, true) = Advanced Degree )) OR ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSide [...]
-(22) CometBroadcastExchange
-Input [3]: [cd_demo_sk#18, cd_marital_status#19, cd_education_status#20]
-Arguments: [cd_demo_sk#18, cd_marital_status#19, cd_education_status#20]
+(22) CometProject
+Input [3]: [cd_demo_sk#19, cd_marital_status#20, cd_education_status#21]
+Arguments: [cd_demo_sk#19, cd_marital_status#22, cd_education_status#23], [cd_demo_sk#19, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_marital_status#20, 1, true, false, true) AS cd_marital_status#22, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_education_status#21, 20, true, false, true) AS cd_education_status#23]
-(23) CometBroadcastHashJoin
+(23) CometBroadcastExchange
+Input [3]: [cd_demo_sk#19, cd_marital_status#22, cd_education_status#23]
+Arguments: [cd_demo_sk#19, cd_marital_status#22, cd_education_status#23]
+
+(24) CometBroadcastHashJoin
Left output [6]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8]
-Right output [3]: [cd_demo_sk#18, cd_marital_status#19, cd_education_status#20]
-Arguments: [ss_cdemo_sk#1], [cd_demo_sk#18], Inner, ((((((rpad(cd_marital_status#19, 1, ) = M) AND (rpad(cd_education_status#20, 20, ) = Advanced Degree )) AND (ss_sales_price#6 >= 100.00)) AND (ss_sales_price#6 <= 150.00)) OR ((((rpad(cd_marital_status#19, 1, ) = S) AND (rpad(cd_education_status#20, 20, ) = College )) AND (ss_sales_price#6 >= 50.00)) AND (ss_sales_price#6 <= 100.00))) OR ((((rpad(cd_marital_status#19, 1, ) = W) AND (rpad(cd_education_status#20, 20, [...]
+Right output [3]: [cd_demo_sk#19, cd_marital_status#22, cd_education_status#23]
+Arguments: [ss_cdemo_sk#1], [cd_demo_sk#19], Inner, ((((((cd_marital_status#22 = M) AND (cd_education_status#23 = Advanced Degree )) AND (ss_sales_price#6 >= 100.00)) AND (ss_sales_price#6 <= 150.00)) OR ((((cd_marital_status#22 = S) AND (cd_education_status#23 = College )) AND (ss_sales_price#6 >= 50.00)) AND (ss_sales_price#6 <= 100.00))) OR ((((cd_marital_status#22 = W) AND (cd_education_status#23 = 2 yr Degree )) AND (ss_sales_price#6 >= 150.00)) AND (ss_sales [...]
-(24) CometProject
-Input [9]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_demo_sk#18, cd_marital_status#19, cd_education_status#20]
-Arguments: [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#19, cd_education_status#20], [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#19, cd_education_status#20]
+(25) CometProject
+Input [9]: [ss_cdemo_sk#1, ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_demo_sk#19, cd_marital_status#22, cd_education_status#23]
+Arguments: [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#22, cd_education_status#23], [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#22, cd_education_status#23]
-(25) CometScan parquet spark_catalog.default.household_demographics
-Output [2]: [hd_demo_sk#21, hd_dep_count#22]
+(26) CometScan parquet spark_catalog.default.household_demographics
+Output [2]: [hd_demo_sk#24, hd_dep_count#25]
Batched: true
Location [not included in comparison]/{warehouse_dir}/household_demographics]
PushedFilters: [IsNotNull(hd_demo_sk), Or(EqualTo(hd_dep_count,3),EqualTo(hd_dep_count,1))]
ReadSchema: struct<hd_demo_sk:int,hd_dep_count:int>
-(26) CometFilter
-Input [2]: [hd_demo_sk#21, hd_dep_count#22]
-Condition : (isnotnull(hd_demo_sk#21) AND ((hd_dep_count#22 = 3) OR (hd_dep_count#22 = 1)))
+(27) CometFilter
+Input [2]: [hd_demo_sk#24, hd_dep_count#25]
+Condition : (isnotnull(hd_demo_sk#24) AND ((hd_dep_count#25 = 3) OR (hd_dep_count#25 = 1)))
-(27) CometBroadcastExchange
-Input [2]: [hd_demo_sk#21, hd_dep_count#22]
-Arguments: [hd_demo_sk#21, hd_dep_count#22]
+(28) CometBroadcastExchange
+Input [2]: [hd_demo_sk#24, hd_dep_count#25]
+Arguments: [hd_demo_sk#24, hd_dep_count#25]
-(28) CometBroadcastHashJoin
-Left output [7]: [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#19, cd_education_status#20]
-Right output [2]: [hd_demo_sk#21, hd_dep_count#22]
-Arguments: [ss_hdemo_sk#2], [hd_demo_sk#21], Inner, (((((((rpad(cd_marital_status#19, 1, ) = M) AND (rpad(cd_education_status#20, 20, ) = Advanced Degree )) AND (ss_sales_price#6 >= 100.00)) AND (ss_sales_price#6 <= 150.00)) AND (hd_dep_count#22 = 3)) OR (((((rpad(cd_marital_status#19, 1, ) = S) AND (rpad(cd_education_status#20, 20, ) = College )) AND (ss_sales_price#6 >= 50.00)) AND (ss_sales_price#6 <= 100.00)) AND (hd_dep_count#22 = 1))) OR (((((rpad(cd_marital_sta [...]
+(29) CometBroadcastHashJoin
+Left output [7]: [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#22, cd_education_status#23]
+Right output [2]: [hd_demo_sk#24, hd_dep_count#25]
+Arguments: [ss_hdemo_sk#2], [hd_demo_sk#24], Inner, (((((((cd_marital_status#22 = M) AND (cd_education_status#23 = Advanced Degree )) AND (ss_sales_price#6 >= 100.00)) AND (ss_sales_price#6 <= 150.00)) AND (hd_dep_count#25 = 3)) OR (((((cd_marital_status#22 = S) AND (cd_education_status#23 = College )) AND (ss_sales_price#6 >= 50.00)) AND (ss_sales_price#6 <= 100.00)) AND (hd_dep_count#25 = 1))) OR (((((cd_marital_status#22 = W) AND (cd_education_status#23 = 2 yr Degree [...]
-(29) CometProject
-Input [9]: [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#19, cd_education_status#20, hd_demo_sk#21, hd_dep_count#22]
+(30) CometProject
+Input [9]: [ss_hdemo_sk#2, ss_quantity#5, ss_sales_price#6, ss_ext_sales_price#7, ss_ext_wholesale_cost#8, cd_marital_status#22, cd_education_status#23, hd_demo_sk#24, hd_dep_count#25]
Arguments: [ss_quantity#5, ss_ext_sales_price#7, ss_ext_wholesale_cost#8], [ss_quantity#5, ss_ext_sales_price#7, ss_ext_wholesale_cost#8]
-(30) CometHashAggregate
+(31) CometHashAggregate
Input [3]: [ss_quantity#5, ss_ext_sales_price#7, ss_ext_wholesale_cost#8]
Keys: []
Functions [4]: [partial_avg(ss_quantity#5), partial_avg(UnscaledValue(ss_ext_sales_price#7)), partial_avg(UnscaledValue(ss_ext_wholesale_cost#8)), partial_sum(UnscaledValue(ss_ext_wholesale_cost#8))]
-(31) CometExchange
-Input [7]: [sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29]
+(32) CometExchange
+Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(32) CometHashAggregate
-Input [7]: [sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29]
+(33) CometHashAggregate
+Input [7]: [sum#26, count#27, sum#28, count#29, sum#30, count#31, sum#32]
Keys: []
Functions [4]: [avg(ss_quantity#5), avg(UnscaledValue(ss_ext_sales_price#7)), avg(UnscaledValue(ss_ext_wholesale_cost#8)), sum(UnscaledValue(ss_ext_wholesale_cost#8))]
-(33) CometColumnarToRow [codegen id : 1]
-Input [4]: [avg(ss_quantity)#30, avg(ss_ext_sales_price)#31, avg(ss_ext_wholesale_cost)#32, sum(ss_ext_wholesale_cost)#33]
+(34) CometColumnarToRow [codegen id : 1]
+Input [4]: [avg(ss_quantity)#33, avg(ss_ext_sales_price)#34, avg(ss_ext_wholesale_cost)#35, sum(ss_ext_wholesale_cost)#36]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#10 IN dynamicpruning#11
-BroadcastExchange (38)
-+- * CometColumnarToRow (37)
- +- CometProject (36)
- +- CometFilter (35)
- +- CometScan parquet spark_catalog.default.date_dim (34)
+BroadcastExchange (39)
++- * CometColumnarToRow (38)
+ +- CometProject (37)
+ +- CometFilter (36)
+ +- CometScan parquet spark_catalog.default.date_dim (35)
-(34) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#16, d_year#17]
+(35) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#17, d_year#18]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(35) CometFilter
-Input [2]: [d_date_sk#16, d_year#17]
-Condition : ((isnotnull(d_year#17) AND (d_year#17 = 2001)) AND isnotnull(d_date_sk#16))
+(36) CometFilter
+Input [2]: [d_date_sk#17, d_year#18]
+Condition : ((isnotnull(d_year#18) AND (d_year#18 = 2001)) AND isnotnull(d_date_sk#17))
-(36) CometProject
-Input [2]: [d_date_sk#16, d_year#17]
-Arguments: [d_date_sk#16], [d_date_sk#16]
+(37) CometProject
+Input [2]: [d_date_sk#17, d_year#18]
+Arguments: [d_date_sk#17], [d_date_sk#17]
-(37) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#16]
+(38) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#17]
-(38) BroadcastExchange
-Input [1]: [d_date_sk#16]
+(39) BroadcastExchange
+Input [1]: [d_date_sk#17]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt
index c4d3b744d..dd7f62f6c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q13/simplified.txt
@@ -28,7 +28,7 @@ WholeStageCodegen (1)
CometFilter [s_store_sk]
CometScan parquet spark_catalog.default.store [s_store_sk]
CometBroadcastExchange [ca_address_sk,ca_state] #4
- CometProject [ca_address_sk,ca_state]
+ CometProject [ca_state] [ca_address_sk,ca_state]
CometFilter [ca_address_sk,ca_state,ca_country]
CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state,ca_country]
CometBroadcastExchange [d_date_sk] #5
@@ -36,8 +36,9 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [cd_demo_sk,cd_marital_status,cd_education_status] #6
- CometFilter [cd_demo_sk,cd_marital_status,cd_education_status]
- CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_marital_status,cd_education_status]
+ CometProject [cd_marital_status,cd_education_status] [cd_demo_sk,cd_marital_status,cd_education_status]
+ CometFilter [cd_demo_sk,cd_marital_status,cd_education_status]
+ CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_marital_status,cd_education_status]
CometBroadcastExchange [hd_demo_sk,hd_dep_count] #7
CometFilter [hd_demo_sk,hd_dep_count]
CometScan parquet spark_catalog.default.household_demographics [hd_demo_sk,hd_dep_count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt
index 7def27001..a56066b83 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-* CometColumnarToRow (23)
-+- CometTakeOrderedAndProject (22)
- +- CometHashAggregate (21)
- +- CometExchange (20)
- +- CometHashAggregate (19)
- +- CometProject (18)
- +- CometBroadcastHashJoin (17)
- :- CometProject (12)
- : +- CometBroadcastHashJoin (11)
+* CometColumnarToRow (24)
++- CometTakeOrderedAndProject (23)
+ +- CometHashAggregate (22)
+ +- CometExchange (21)
+ +- CometHashAggregate (20)
+ +- CometProject (19)
+ +- CometBroadcastHashJoin (18)
+ :- CometProject (13)
+ : +- CometBroadcastHashJoin (12)
: :- CometProject (7)
: : +- CometBroadcastHashJoin (6)
: : :- CometFilter (2)
@@ -15,13 +15,14 @@
: : +- CometBroadcastExchange (5)
: : +- CometFilter (4)
: : +- CometScan parquet spark_catalog.default.customer (3)
- : +- CometBroadcastExchange (10)
- : +- CometFilter (9)
- : +- CometScan parquet spark_catalog.default.customer_address (8)
- +- CometBroadcastExchange (16)
- +- CometProject (15)
- +- CometFilter (14)
- +- CometScan parquet spark_catalog.default.date_dim (13)
+ : +- CometBroadcastExchange (11)
+ : +- CometProject (10)
+ : +- CometFilter (9)
+ : +- CometScan parquet spark_catalog.default.customer_address (8)
+ +- CometBroadcastExchange (17)
+ +- CometProject (16)
+ +- CometFilter (15)
+ +- CometScan parquet spark_catalog.default.date_dim (14)
(1) CometScan parquet spark_catalog.default.catalog_sales
@@ -71,98 +72,102 @@ ReadSchema: struct<ca_address_sk:int,ca_state:string,ca_zip:string>
Input [3]: [ca_address_sk#7, ca_state#8, ca_zip#9]
Condition : isnotnull(ca_address_sk#7)
-(10) CometBroadcastExchange
+(10) CometProject
Input [3]: [ca_address_sk#7, ca_state#8, ca_zip#9]
-Arguments: [ca_address_sk#7, ca_state#8, ca_zip#9]
+Arguments: [ca_address_sk#7, ca_state#10, ca_zip#11], [ca_address_sk#7, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#8, 2, true, false, true) AS ca_state#10, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#9, 10, true, false, true) AS ca_zip#11]
-(11) CometBroadcastHashJoin
+(11) CometBroadcastExchange
+Input [3]: [ca_address_sk#7, ca_state#10, ca_zip#11]
+Arguments: [ca_address_sk#7, ca_state#10, ca_zip#11]
+
+(12) CometBroadcastHashJoin
Left output [3]: [cs_sales_price#2, cs_sold_date_sk#3, c_current_addr_sk#6]
-Right output [3]: [ca_address_sk#7, ca_state#8, ca_zip#9]
-Arguments: [c_current_addr_sk#6], [ca_address_sk#7], Inner, ((substr(ca_zip#9, 1, 5) IN (85669,86197,88274,83405,86475,85392,85460,80348,81792) OR rpad(ca_state#8, 2, ) IN (CA,WA,GA)) OR (cs_sales_price#2 > 500.00)), BuildRight
+Right output [3]: [ca_address_sk#7, ca_state#10, ca_zip#11]
+Arguments: [c_current_addr_sk#6], [ca_address_sk#7], Inner, ((substr(ca_zip#11, 1, 5) IN (85669,86197,88274,83405,86475,85392,85460,80348,81792) OR ca_state#10 IN (CA,WA,GA)) OR (cs_sales_price#2 > 500.00)), BuildRight
-(12) CometProject
-Input [6]: [cs_sales_price#2, cs_sold_date_sk#3, c_current_addr_sk#6, ca_address_sk#7, ca_state#8, ca_zip#9]
-Arguments: [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#9], [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#9]
+(13) CometProject
+Input [6]: [cs_sales_price#2, cs_sold_date_sk#3, c_current_addr_sk#6, ca_address_sk#7, ca_state#10, ca_zip#11]
+Arguments: [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#11], [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#11]
-(13) CometScan parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#10, d_year#11, d_qoy#12]
+(14) CometScan parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#12, d_year#13, d_qoy#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,2), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(14) CometFilter
-Input [3]: [d_date_sk#10, d_year#11, d_qoy#12]
-Condition : ((((isnotnull(d_qoy#12) AND isnotnull(d_year#11)) AND (d_qoy#12 = 2)) AND (d_year#11 = 2001)) AND isnotnull(d_date_sk#10))
+(15) CometFilter
+Input [3]: [d_date_sk#12, d_year#13, d_qoy#14]
+Condition : ((((isnotnull(d_qoy#14) AND isnotnull(d_year#13)) AND (d_qoy#14 = 2)) AND (d_year#13 = 2001)) AND isnotnull(d_date_sk#12))
-(15) CometProject
-Input [3]: [d_date_sk#10, d_year#11, d_qoy#12]
-Arguments: [d_date_sk#10], [d_date_sk#10]
+(16) CometProject
+Input [3]: [d_date_sk#12, d_year#13, d_qoy#14]
+Arguments: [d_date_sk#12], [d_date_sk#12]
-(16) CometBroadcastExchange
-Input [1]: [d_date_sk#10]
-Arguments: [d_date_sk#10]
+(17) CometBroadcastExchange
+Input [1]: [d_date_sk#12]
+Arguments: [d_date_sk#12]
-(17) CometBroadcastHashJoin
-Left output [3]: [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#9]
-Right output [1]: [d_date_sk#10]
-Arguments: [cs_sold_date_sk#3], [d_date_sk#10], Inner, BuildRight
+(18) CometBroadcastHashJoin
+Left output [3]: [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#11]
+Right output [1]: [d_date_sk#12]
+Arguments: [cs_sold_date_sk#3], [d_date_sk#12], Inner, BuildRight
-(18) CometProject
-Input [4]: [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#9, d_date_sk#10]
-Arguments: [cs_sales_price#2, ca_zip#9], [cs_sales_price#2, ca_zip#9]
+(19) CometProject
+Input [4]: [cs_sales_price#2, cs_sold_date_sk#3, ca_zip#11, d_date_sk#12]
+Arguments: [cs_sales_price#2, ca_zip#11], [cs_sales_price#2, ca_zip#11]
-(19) CometHashAggregate
-Input [2]: [cs_sales_price#2, ca_zip#9]
-Keys [1]: [ca_zip#9]
+(20) CometHashAggregate
+Input [2]: [cs_sales_price#2, ca_zip#11]
+Keys [1]: [ca_zip#11]
Functions [1]: [partial_sum(UnscaledValue(cs_sales_price#2))]
-(20) CometExchange
-Input [2]: [ca_zip#9, sum#13]
-Arguments: hashpartitioning(ca_zip#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(21) CometExchange
+Input [2]: [ca_zip#11, sum#15]
+Arguments: hashpartitioning(ca_zip#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(21) CometHashAggregate
-Input [2]: [ca_zip#9, sum#13]
-Keys [1]: [ca_zip#9]
+(22) CometHashAggregate
+Input [2]: [ca_zip#11, sum#15]
+Keys [1]: [ca_zip#11]
Functions [1]: [sum(UnscaledValue(cs_sales_price#2))]
-(22) CometTakeOrderedAndProject
-Input [2]: [ca_zip#9, sum(cs_sales_price)#14]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[ca_zip#9 ASC NULLS FIRST], output=[ca_zip#9,sum(cs_sales_price)#14]), [ca_zip#9, sum(cs_sales_price)#14], 100, [ca_zip#9 ASC NULLS FIRST], [ca_zip#9, sum(cs_sales_price)#14]
+(23) CometTakeOrderedAndProject
+Input [2]: [ca_zip#11, sum(cs_sales_price)#16]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ca_zip#11 ASC NULLS FIRST], output=[ca_zip#11,sum(cs_sales_price)#16]), [ca_zip#11, sum(cs_sales_price)#16], 100, [ca_zip#11 ASC NULLS FIRST], [ca_zip#11, sum(cs_sales_price)#16]
-(23) CometColumnarToRow [codegen id : 1]
-Input [2]: [ca_zip#9, sum(cs_sales_price)#14]
+(24) CometColumnarToRow [codegen id : 1]
+Input [2]: [ca_zip#11, sum(cs_sales_price)#16]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (28)
-+- * CometColumnarToRow (27)
- +- CometProject (26)
- +- CometFilter (25)
- +- CometScan parquet spark_catalog.default.date_dim (24)
+BroadcastExchange (29)
++- * CometColumnarToRow (28)
+ +- CometProject (27)
+ +- CometFilter (26)
+ +- CometScan parquet spark_catalog.default.date_dim (25)
-(24) CometScan parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#10, d_year#11, d_qoy#12]
+(25) CometScan parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#12, d_year#13, d_qoy#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_qoy), IsNotNull(d_year), EqualTo(d_qoy,2), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(25) CometFilter
-Input [3]: [d_date_sk#10, d_year#11, d_qoy#12]
-Condition : ((((isnotnull(d_qoy#12) AND isnotnull(d_year#11)) AND (d_qoy#12 = 2)) AND (d_year#11 = 2001)) AND isnotnull(d_date_sk#10))
+(26) CometFilter
+Input [3]: [d_date_sk#12, d_year#13, d_qoy#14]
+Condition : ((((isnotnull(d_qoy#14) AND isnotnull(d_year#13)) AND (d_qoy#14 = 2)) AND (d_year#13 = 2001)) AND isnotnull(d_date_sk#12))
-(26) CometProject
-Input [3]: [d_date_sk#10, d_year#11, d_qoy#12]
-Arguments: [d_date_sk#10], [d_date_sk#10]
+(27) CometProject
+Input [3]: [d_date_sk#12, d_year#13, d_qoy#14]
+Arguments: [d_date_sk#12], [d_date_sk#12]
-(27) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#10]
+(28) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#12]
-(28) BroadcastExchange
-Input [1]: [d_date_sk#10]
+(29) BroadcastExchange
+Input [1]: [d_date_sk#12]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt
index 425f20e10..b4021b7f2 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q15/simplified.txt
@@ -25,8 +25,9 @@ WholeStageCodegen (1)
CometFilter [c_customer_sk,c_current_addr_sk]
CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
CometBroadcastExchange [ca_address_sk,ca_state,ca_zip] #4
- CometFilter [ca_address_sk,ca_state,ca_zip]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state,ca_zip]
+ CometProject [ca_state,ca_zip] [ca_address_sk,ca_state,ca_zip]
+ CometFilter [ca_address_sk,ca_state,ca_zip]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state,ca_zip]
CometBroadcastExchange [d_date_sk] #5
CometProject [d_date_sk]
CometFilter [d_date_sk,d_year,d_qoy]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt
index 658998567..79da5676b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q16/explain.txt
@@ -147,12 +147,12 @@ Arguments: [cs_ship_addr_sk#2, cs_call_center_sk#3, cs_order_number#5, cs_ext_sh
Output [2]: [ca_address_sk#16, ca_state#17]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
-PushedFilters: [IsNotNull(ca_state), IsNotNull(ca_address_sk)]
+PushedFilters: [IsNotNull(ca_address_sk)]
ReadSchema: struct<ca_address_sk:int,ca_state:string>
(24) CometFilter
Input [2]: [ca_address_sk#16, ca_state#17]
-Condition : ((isnotnull(ca_state#17) AND (rpad(ca_state#17, 2, ) = GA)) AND isnotnull(ca_address_sk#16))
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#17, 2, true, false, true) = GA) AND isnotnull(ca_address_sk#16))
(25) CometProject
Input [2]: [ca_address_sk#16, ca_state#17]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt
index da2f5536b..ab58eb37e 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-* CometColumnarToRow (42)
-+- CometTakeOrderedAndProject (41)
- +- CometHashAggregate (40)
- +- CometExchange (39)
- +- CometHashAggregate (38)
- +- CometProject (37)
- +- CometBroadcastHashJoin (36)
- :- CometProject (32)
- : +- CometBroadcastHashJoin (31)
+* CometColumnarToRow (44)
++- CometTakeOrderedAndProject (43)
+ +- CometHashAggregate (42)
+ +- CometExchange (41)
+ +- CometHashAggregate (40)
+ +- CometProject (39)
+ +- CometBroadcastHashJoin (38)
+ :- CometProject (33)
+ : +- CometBroadcastHashJoin (32)
: :- CometProject (27)
: : +- CometBroadcastHashJoin (26)
: : :- CometProject (24)
@@ -35,12 +35,14 @@
: : : +- CometFilter (20)
: : : +- CometScan parquet spark_catalog.default.date_dim (19)
: : +- ReusedExchange (25)
- : +- CometBroadcastExchange (30)
- : +- CometFilter (29)
- : +- CometScan parquet spark_catalog.default.store (28)
- +- CometBroadcastExchange (35)
- +- CometFilter (34)
- +- CometScan parquet spark_catalog.default.item (33)
+ : +- CometBroadcastExchange (31)
+ : +- CometProject (30)
+ : +- CometFilter (29)
+ : +- CometScan parquet spark_catalog.default.store (28)
+ +- CometBroadcastExchange (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan parquet spark_catalog.default.item (34)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -109,12 +111,12 @@ Arguments: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, ss_sold_date_sk#6, sr_re
Output [2]: [d_date_sk#19, d_quarter_name#20]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
-PushedFilters: [IsNotNull(d_quarter_name), IsNotNull(d_date_sk)]
+PushedFilters: [IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_quarter_name:string>
(14) CometFilter
Input [2]: [d_date_sk#19, d_quarter_name#20]
-Condition : ((isnotnull(d_quarter_name#20) AND (rpad(d_quarter_name#20, 6, ) = 2001Q1)) AND isnotnull(d_date_sk#19))
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, d_quarter_name#20, 6, true, false, true) = 2001Q1) AND isnotnull(d_date_sk#19))
(15) CometProject
Input [2]: [d_date_sk#19, d_quarter_name#20]
@@ -142,7 +144,7 @@ ReadSchema: struct<d_date_sk:int,d_quarter_name:string>
(20) CometFilter
Input [2]: [d_date_sk#21, d_quarter_name#22]
-Condition : (rpad(d_quarter_name#22, 6, ) IN (2001Q1,2001Q2,2001Q3) AND isnotnull(d_date_sk#21))
+Condition : (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, d_quarter_name#22, 6, true, false, true) IN (2001Q1,2001Q2,2001Q3) AND isnotnull(d_date_sk#21))
(21) CometProject
Input [2]: [d_date_sk#21, d_quarter_name#22]
@@ -184,123 +186,131 @@ ReadSchema: struct<s_store_sk:int,s_state:string>
Input [2]: [s_store_sk#24, s_state#25]
Condition : isnotnull(s_store_sk#24)
-(30) CometBroadcastExchange
+(30) CometProject
Input [2]: [s_store_sk#24, s_state#25]
-Arguments: [s_store_sk#24, s_state#25]
+Arguments: [s_store_sk#24, s_state#26], [s_store_sk#24, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#25, 2, true, false, true) AS s_state#26]
+
+(31) CometBroadcastExchange
+Input [2]: [s_store_sk#24, s_state#26]
+Arguments: [s_store_sk#24, s_state#26]
-(31) CometBroadcastHashJoin
+(32) CometBroadcastHashJoin
Left output [5]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, sr_return_quantity#11, cs_quantity#16]
-Right output [2]: [s_store_sk#24, s_state#25]
+Right output [2]: [s_store_sk#24, s_state#26]
Arguments: [ss_store_sk#3], [s_store_sk#24], Inner, BuildRight
-(32) CometProject
-Input [7]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_sk#24, s_state#25]
-Arguments: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25], [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25]
+(33) CometProject
+Input [7]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_sk#24, s_state#26]
+Arguments: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26], [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26]
-(33) CometScan parquet spark_catalog.default.item
-Output [3]: [i_item_sk#26, i_item_id#27, i_item_desc#28]
+(34) CometScan parquet spark_catalog.default.item
+Output [3]: [i_item_sk#27, i_item_id#28, i_item_desc#29]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_item_id:string,i_item_desc:string>
-(34) CometFilter
-Input [3]: [i_item_sk#26, i_item_id#27, i_item_desc#28]
-Condition : isnotnull(i_item_sk#26)
+(35) CometFilter
+Input [3]: [i_item_sk#27, i_item_id#28, i_item_desc#29]
+Condition : isnotnull(i_item_sk#27)
+
+(36) CometProject
+Input [3]: [i_item_sk#27, i_item_id#28, i_item_desc#29]
+Arguments: [i_item_sk#27, i_item_id#30, i_item_desc#29], [i_item_sk#27, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#28, 16, true, false, true) AS i_item_id#30, i_item_desc#29]
-(35) CometBroadcastExchange
-Input [3]: [i_item_sk#26, i_item_id#27, i_item_desc#28]
-Arguments: [i_item_sk#26, i_item_id#27, i_item_desc#28]
+(37) CometBroadcastExchange
+Input [3]: [i_item_sk#27, i_item_id#30, i_item_desc#29]
+Arguments: [i_item_sk#27, i_item_id#30, i_item_desc#29]
-(36) CometBroadcastHashJoin
-Left output [5]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25]
-Right output [3]: [i_item_sk#26, i_item_id#27, i_item_desc#28]
-Arguments: [ss_item_sk#1], [i_item_sk#26], Inner, BuildRight
+(38) CometBroadcastHashJoin
+Left output [5]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26]
+Right output [3]: [i_item_sk#27, i_item_id#30, i_item_desc#29]
+Arguments: [ss_item_sk#1], [i_item_sk#27], Inner, BuildRight
-(37) CometProject
-Input [8]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25, i_item_sk#26, i_item_id#27, i_item_desc#28]
-Arguments: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25, i_item_id#27, i_item_desc#28], [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25, i_item_id#27, i_item_desc#28]
+(39) CometProject
+Input [8]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26, i_item_sk#27, i_item_id#30, i_item_desc#29]
+Arguments: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26, i_item_id#30, i_item_desc#29], [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26, i_item_id#30, i_item_desc#29]
-(38) CometHashAggregate
-Input [6]: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#25, i_item_id#27, i_item_desc#28]
-Keys [3]: [i_item_id#27, i_item_desc#28, s_state#25]
+(40) CometHashAggregate
+Input [6]: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_state#26, i_item_id#30, i_item_desc#29]
+Keys [3]: [i_item_id#30, i_item_desc#29, s_state#26]
Functions [9]: [partial_count(ss_quantity#5), partial_avg(ss_quantity#5), partial_stddev_samp(cast(ss_quantity#5 as double)), partial_count(sr_return_quantity#11), partial_avg(sr_return_quantity#11), partial_stddev_samp(cast(sr_return_quantity#11 as double)), partial_count(cs_quantity#16), partial_avg(cs_quantity#16), partial_stddev_samp(cast(cs_quantity#16 as double))]
-(39) CometExchange
-Input [21]: [i_item_id#27, i_item_desc#28, s_state#25, count#29, sum#30, count#31, n#32, avg#33, m2#34, count#35, sum#36, count#37, n#38, avg#39, m2#40, count#41, sum#42, count#43, n#44, avg#45, m2#46]
-Arguments: hashpartitioning(i_item_id#27, i_item_desc#28, s_state#25, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(41) CometExchange
+Input [21]: [i_item_id#30, i_item_desc#29, s_state#26, count#31, sum#32, count#33, n#34, avg#35, m2#36, count#37, sum#38, count#39, n#40, avg#41, m2#42, count#43, sum#44, count#45, n#46, avg#47, m2#48]
+Arguments: hashpartitioning(i_item_id#30, i_item_desc#29, s_state#26, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(40) CometHashAggregate
-Input [21]: [i_item_id#27, i_item_desc#28, s_state#25, count#29, sum#30, count#31, n#32, avg#33, m2#34, count#35, sum#36, count#37, n#38, avg#39, m2#40, count#41, sum#42, count#43, n#44, avg#45, m2#46]
-Keys [3]: [i_item_id#27, i_item_desc#28, s_state#25]
+(42) CometHashAggregate
+Input [21]: [i_item_id#30, i_item_desc#29, s_state#26, count#31, sum#32, count#33, n#34, avg#35, m2#36, count#37, sum#38, count#39, n#40, avg#41, m2#42, count#43, sum#44, count#45, n#46, avg#47, m2#48]
+Keys [3]: [i_item_id#30, i_item_desc#29, s_state#26]
Functions [9]: [count(ss_quantity#5), avg(ss_quantity#5), stddev_samp(cast(ss_quantity#5 as double)), count(sr_return_quantity#11), avg(sr_return_quantity#11), stddev_samp(cast(sr_return_quantity#11 as double)), count(cs_quantity#16), avg(cs_quantity#16), stddev_samp(cast(cs_quantity#16 as double))]
-(41) CometTakeOrderedAndProject
-Input [15]: [i_item_id#27, i_item_desc#28, s_state#25, store_sales_quantitycount#47, store_sales_quantityave#48, store_sales_quantitystdev#49, store_sales_quantitycov#50, as_store_returns_quantitycount#51, as_store_returns_quantityave#52, as_store_returns_quantitystdev#53, store_returns_quantitycov#54, catalog_sales_quantitycount#55, catalog_sales_quantityave#56, catalog_sales_quantitystdev#57, catalog_sales_quantitycov#58]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#27 ASC NULLS FIRST,i_item_desc#28 ASC NULLS FIRST,s_state#25 ASC NULLS FIRST], output=[i_item_id#27,i_item_desc#28,s_state#25,store_sales_quantitycount#47,store_sales_quantityave#48,store_sales_quantitystdev#49,store_sales_quantitycov#50,as_store_returns_quantitycount#51,as_store_returns_quantityave#52,as_store_returns_quantitystdev#53,store_returns_quantitycov#54,catalog_sales_quantitycount#55,catalog_sales_quantityave#56,ca [...]
+(43) CometTakeOrderedAndProject
+Input [15]: [i_item_id#30, i_item_desc#29, s_state#26, store_sales_quantitycount#49, store_sales_quantityave#50, store_sales_quantitystdev#51, store_sales_quantitycov#52, as_store_returns_quantitycount#53, as_store_returns_quantityave#54, as_store_returns_quantitystdev#55, store_returns_quantitycov#56, catalog_sales_quantitycount#57, catalog_sales_quantityave#58, catalog_sales_quantitystdev#59, catalog_sales_quantitycov#60]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#30 ASC NULLS FIRST,i_item_desc#29 ASC NULLS FIRST,s_state#26 ASC NULLS FIRST], output=[i_item_id#30,i_item_desc#29,s_state#26,store_sales_quantitycount#49,store_sales_quantityave#50,store_sales_quantitystdev#51,store_sales_quantitycov#52,as_store_returns_quantitycount#53,as_store_returns_quantityave#54,as_store_returns_quantitystdev#55,store_returns_quantitycov#56,catalog_sales_quantitycount#57,catalog_sales_quantityave#58,ca [...]
-(42) CometColumnarToRow [codegen id : 1]
-Input [15]: [i_item_id#27, i_item_desc#28, s_state#25, store_sales_quantitycount#47, store_sales_quantityave#48, store_sales_quantitystdev#49, store_sales_quantitycov#50, as_store_returns_quantitycount#51, as_store_returns_quantityave#52, as_store_returns_quantitystdev#53, store_returns_quantitycov#54, catalog_sales_quantitycount#55, catalog_sales_quantityave#56, catalog_sales_quantitystdev#57, catalog_sales_quantitycov#58]
+(44) CometColumnarToRow [codegen id : 1]
+Input [15]: [i_item_id#30, i_item_desc#29, s_state#26, store_sales_quantitycount#49, store_sales_quantityave#50, store_sales_quantitystdev#51, store_sales_quantitycov#52, as_store_returns_quantitycount#53, as_store_returns_quantityave#54, as_store_returns_quantitystdev#55, store_returns_quantitycov#56, catalog_sales_quantitycount#57, catalog_sales_quantityave#58, catalog_sales_quantitystdev#59, catalog_sales_quantitycov#60]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#6 IN dynamicpruning#7
-BroadcastExchange (47)
-+- * CometColumnarToRow (46)
- +- CometProject (45)
- +- CometFilter (44)
- +- CometScan parquet spark_catalog.default.date_dim (43)
+BroadcastExchange (49)
++- * CometColumnarToRow (48)
+ +- CometProject (47)
+ +- CometFilter (46)
+ +- CometScan parquet spark_catalog.default.date_dim (45)
-(43) CometScan parquet spark_catalog.default.date_dim
+(45) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#19, d_quarter_name#20]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
-PushedFilters: [IsNotNull(d_quarter_name), IsNotNull(d_date_sk)]
+PushedFilters: [IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_quarter_name:string>
-(44) CometFilter
+(46) CometFilter
Input [2]: [d_date_sk#19, d_quarter_name#20]
-Condition : ((isnotnull(d_quarter_name#20) AND (rpad(d_quarter_name#20, 6, ) = 2001Q1)) AND isnotnull(d_date_sk#19))
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, d_quarter_name#20, 6, true, false, true) = 2001Q1) AND isnotnull(d_date_sk#19))
-(45) CometProject
+(47) CometProject
Input [2]: [d_date_sk#19, d_quarter_name#20]
Arguments: [d_date_sk#19], [d_date_sk#19]
-(46) CometColumnarToRow [codegen id : 1]
+(48) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#19]
-(47) BroadcastExchange
+(49) BroadcastExchange
Input [1]: [d_date_sk#19]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
Subquery:2 Hosting operator id = 3 Hosting Expression = sr_returned_date_sk#12 IN dynamicpruning#13
-BroadcastExchange (52)
-+- * CometColumnarToRow (51)
- +- CometProject (50)
- +- CometFilter (49)
- +- CometScan parquet spark_catalog.default.date_dim (48)
+BroadcastExchange (54)
++- * CometColumnarToRow (53)
+ +- CometProject (52)
+ +- CometFilter (51)
+ +- CometScan parquet spark_catalog.default.date_dim (50)
-(48) CometScan parquet spark_catalog.default.date_dim
+(50) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#21, d_quarter_name#22]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_quarter_name:string>
-(49) CometFilter
+(51) CometFilter
Input [2]: [d_date_sk#21, d_quarter_name#22]
-Condition : (rpad(d_quarter_name#22, 6, ) IN (2001Q1,2001Q2,2001Q3) AND isnotnull(d_date_sk#21))
+Condition : (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, d_quarter_name#22, 6, true, false, true) IN (2001Q1,2001Q2,2001Q3) AND isnotnull(d_date_sk#21))
-(50) CometProject
+(52) CometProject
Input [2]: [d_date_sk#21, d_quarter_name#22]
Arguments: [d_date_sk#21], [d_date_sk#21]
-(51) CometColumnarToRow [codegen id : 1]
+(53) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#21]
-(52) BroadcastExchange
+(54) BroadcastExchange
Input [1]: [d_date_sk#21]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt
index 97000b91d..43e08a2dc 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q17/simplified.txt
@@ -54,8 +54,10 @@ WholeStageCodegen (1)
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
ReusedExchange [d_date_sk] #7
CometBroadcastExchange [s_store_sk,s_state] #8
- CometFilter [s_store_sk,s_state]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_state]
+ CometProject [s_state] [s_store_sk,s_state]
+ CometFilter [s_store_sk,s_state]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_state]
CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc] #9
- CometFilter [i_item_sk,i_item_id,i_item_desc]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
+ CometProject [i_item_id] [i_item_sk,i_item_id,i_item_desc]
+ CometFilter [i_item_sk,i_item_id,i_item_desc]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt
index 58810282d..29e12f97b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/explain.txt
@@ -1,16 +1,16 @@
== Physical Plan ==
-* CometColumnarToRow (41)
-+- CometTakeOrderedAndProject (40)
- +- CometHashAggregate (39)
- +- CometExchange (38)
- +- CometHashAggregate (37)
- +- CometExpand (36)
- +- CometProject (35)
- +- CometBroadcastHashJoin (34)
- :- CometProject (30)
- : +- CometBroadcastHashJoin (29)
- : :- CometProject (24)
- : : +- CometBroadcastHashJoin (23)
+* CometColumnarToRow (43)
++- CometTakeOrderedAndProject (42)
+ +- CometHashAggregate (41)
+ +- CometExchange (40)
+ +- CometHashAggregate (39)
+ +- CometExpand (38)
+ +- CometProject (37)
+ +- CometBroadcastHashJoin (36)
+ :- CometProject (31)
+ : +- CometBroadcastHashJoin (30)
+ : :- CometProject (25)
+ : : +- CometBroadcastHashJoin (24)
: : :- CometProject (19)
: : : +- CometBroadcastHashJoin (18)
: : : :- CometProject (14)
@@ -30,16 +30,18 @@
: : : +- CometBroadcastExchange (17)
: : : +- CometFilter (16)
: : : +- CometScan parquet spark_catalog.default.customer_demographics (15)
- : : +- CometBroadcastExchange (22)
- : : +- CometFilter (21)
- : : +- CometScan parquet spark_catalog.default.customer_address (20)
- : +- CometBroadcastExchange (28)
- : +- CometProject (27)
- : +- CometFilter (26)
- : +- CometScan parquet spark_catalog.default.date_dim (25)
- +- CometBroadcastExchange (33)
- +- CometFilter (32)
- +- CometScan parquet spark_catalog.default.item (31)
+ : : +- CometBroadcastExchange (23)
+ : : +- CometProject (22)
+ : : +- CometFilter (21)
+ : : +- CometScan parquet spark_catalog.default.customer_address (20)
+ : +- CometBroadcastExchange (29)
+ : +- CometProject (28)
+ : +- CometFilter (27)
+ : +- CometScan parquet spark_catalog.default.date_dim (26)
+ +- CometBroadcastExchange (35)
+ +- CometProject (34)
+ +- CometFilter (33)
+ +- CometScan parquet spark_catalog.default.item (32)
(1) CometScan parquet spark_catalog.default.catalog_sales
@@ -58,12 +60,12 @@ Condition : ((isnotnull(cs_bill_cdemo_sk#2) AND isnotnull(cs_bill_customer_sk#1)
Output [4]: [cd_demo_sk#11, cd_gender#12, cd_education_status#13, cd_dep_count#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_demographics]
-PushedFilters: [IsNotNull(cd_gender), IsNotNull(cd_education_status), IsNotNull(cd_demo_sk)]
+PushedFilters: [IsNotNull(cd_demo_sk)]
ReadSchema: struct<cd_demo_sk:int,cd_gender:string,cd_education_status:string,cd_dep_count:int>
(4) CometFilter
Input [4]: [cd_demo_sk#11, cd_gender#12, cd_education_status#13, cd_dep_count#14]
-Condition : ((((isnotnull(cd_gender#12) AND isnotnull(cd_education_status#13)) AND (rpad(cd_gender#12, 1, ) = F)) AND (rpad(cd_education_status#13, 20, ) = Unknown )) AND isnotnull(cd_demo_sk#11))
+Condition : (((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_gender#12, 1, true, false, true) = F) AND (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_education_status#13, 20, true, false, true) = Unknown )) AND isnotnull(cd_demo_sk#11))
(5) CometProject
Input [4]: [cd_demo_sk#11, cd_gender#12, cd_education_status#13, cd_dep_count#14]
@@ -143,128 +145,136 @@ ReadSchema: struct<ca_address_sk:int,ca_county:string,ca_state:string,ca_country
(21) CometFilter
Input [4]: [ca_address_sk#21, ca_county#22, ca_state#23, ca_country#24]
-Condition : (rpad(ca_state#23, 2, ) IN (MS,IN,ND,OK,NM,VA) AND isnotnull(ca_address_sk#21))
+Condition : (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#23, 2, true, false, true) IN (MS,IN,ND,OK,NM,VA) AND isnotnull(ca_address_sk#21))
-(22) CometBroadcastExchange
+(22) CometProject
Input [4]: [ca_address_sk#21, ca_county#22, ca_state#23, ca_country#24]
-Arguments: [ca_address_sk#21, ca_county#22, ca_state#23, ca_country#24]
+Arguments: [ca_address_sk#21, ca_county#22, ca_state#25, ca_country#24], [ca_address_sk#21, ca_county#22, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#23, 2, true, false, true) AS ca_state#25, ca_country#24]
+
+(23) CometBroadcastExchange
+Input [4]: [ca_address_sk#21, ca_county#22, ca_state#25, ca_country#24]
+Arguments: [ca_address_sk#21, ca_county#22, ca_state#25, ca_country#24]
-(23) CometBroadcastHashJoin
+(24) CometBroadcastHashJoin
Left output [10]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_current_addr_sk#17, c_birth_year#19]
-Right output [4]: [ca_address_sk#21, ca_county#22, ca_state#23, ca_country#24]
+Right output [4]: [ca_address_sk#21, ca_county#22, ca_state#25, ca_country#24]
Arguments: [c_current_addr_sk#17], [ca_address_sk#21], Inner, BuildRight
-(24) CometProject
-Input [14]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_current_addr_sk#17, c_birth_year#19, ca_address_sk#21, ca_county#22, ca_state#23, ca_country#24]
-Arguments: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24], [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24]
+(25) CometProject
+Input [14]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_current_addr_sk#17, c_birth_year#19, ca_address_sk#21, ca_county#22, ca_state#25, ca_country#24]
+Arguments: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24], [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24]
-(25) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#25, d_year#26]
+(26) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#26, d_year#27]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,1998), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(26) CometFilter
-Input [2]: [d_date_sk#25, d_year#26]
-Condition : ((isnotnull(d_year#26) AND (d_year#26 = 1998)) AND isnotnull(d_date_sk#25))
+(27) CometFilter
+Input [2]: [d_date_sk#26, d_year#27]
+Condition : ((isnotnull(d_year#27) AND (d_year#27 = 1998)) AND isnotnull(d_date_sk#26))
-(27) CometProject
-Input [2]: [d_date_sk#25, d_year#26]
-Arguments: [d_date_sk#25], [d_date_sk#25]
+(28) CometProject
+Input [2]: [d_date_sk#26, d_year#27]
+Arguments: [d_date_sk#26], [d_date_sk#26]
-(28) CometBroadcastExchange
-Input [1]: [d_date_sk#25]
-Arguments: [d_date_sk#25]
+(29) CometBroadcastExchange
+Input [1]: [d_date_sk#26]
+Arguments: [d_date_sk#26]
-(29) CometBroadcastHashJoin
-Left output [12]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24]
-Right output [1]: [d_date_sk#25]
-Arguments: [cs_sold_date_sk#9], [d_date_sk#25], Inner, BuildRight
+(30) CometBroadcastHashJoin
+Left output [12]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24]
+Right output [1]: [d_date_sk#26]
+Arguments: [cs_sold_date_sk#9], [d_date_sk#26], Inner, BuildRight
-(30) CometProject
-Input [13]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24, d_date_sk#25]
-Arguments: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24], [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24]
+(31) CometProject
+Input [13]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cs_sold_date_sk#9, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24, d_date_sk#26]
+Arguments: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24], [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24]
-(31) CometScan parquet spark_catalog.default.item
-Output [2]: [i_item_sk#27, i_item_id#28]
+(32) CometScan parquet spark_catalog.default.item
+Output [2]: [i_item_sk#28, i_item_id#29]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_item_id:string>
-(32) CometFilter
-Input [2]: [i_item_sk#27, i_item_id#28]
-Condition : isnotnull(i_item_sk#27)
+(33) CometFilter
+Input [2]: [i_item_sk#28, i_item_id#29]
+Condition : isnotnull(i_item_sk#28)
+
+(34) CometProject
+Input [2]: [i_item_sk#28, i_item_id#29]
+Arguments: [i_item_sk#28, i_item_id#30], [i_item_sk#28, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#29, 16, true, false, true) AS i_item_id#30]
-(33) CometBroadcastExchange
-Input [2]: [i_item_sk#27, i_item_id#28]
-Arguments: [i_item_sk#27, i_item_id#28]
+(35) CometBroadcastExchange
+Input [2]: [i_item_sk#28, i_item_id#30]
+Arguments: [i_item_sk#28, i_item_id#30]
-(34) CometBroadcastHashJoin
-Left output [11]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24]
-Right output [2]: [i_item_sk#27, i_item_id#28]
-Arguments: [cs_item_sk#3], [i_item_sk#27], Inner, BuildRight
+(36) CometBroadcastHashJoin
+Left output [11]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24]
+Right output [2]: [i_item_sk#28, i_item_id#30]
+Arguments: [cs_item_sk#3], [i_item_sk#28], Inner, BuildRight
-(35) CometProject
-Input [13]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#23, ca_country#24, i_item_sk#27, i_item_id#28]
-Arguments: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#28, ca_country#24, ca_state#23, ca_county#22], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#28, ca_country#24, ca_state#23, ca_county#22]
+(37) CometProject
+Input [13]: [cs_item_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, ca_county#22, ca_state#25, ca_country#24, i_item_sk#28, i_item_id#30]
+Arguments: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#30, ca_country#24, ca_state#25, ca_county#22], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#30, ca_country#24, ca_state#25, ca_county#22]
-(36) CometExpand
-Input [11]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#28, ca_country#24, ca_state#23, ca_county#22]
-Arguments: [[cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#28, ca_country#24, ca_state#23, ca_county#22, 0], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#28, ca_country#24, ca_state#23, null, 1], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#2 [...]
+(38) CometExpand
+Input [11]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#30, ca_country#24, ca_state#25, ca_county#22]
+Arguments: [[cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#30, ca_country#24, ca_state#25, ca_county#22, 0], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#30, ca_country#24, ca_state#25, null, 1], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#3 [...]
-(37) CometHashAggregate
-Input [12]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33]
-Keys [5]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33]
+(39) CometHashAggregate
+Input [12]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, cs_net_profit#8, cd_dep_count#14, c_birth_year#19, i_item_id#31, ca_country#32, ca_state#33, ca_county#34, spark_grouping_id#35]
+Keys [5]: [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, spark_grouping_id#35]
Functions [7]: [partial_avg(cast(cs_quantity#4 as decimal(12,2))), partial_avg(cast(cs_list_price#5 as decimal(12,2))), partial_avg(cast(cs_coupon_amt#7 as decimal(12,2))), partial_avg(cast(cs_sales_price#6 as decimal(12,2))), partial_avg(cast(cs_net_profit#8 as decimal(12,2))), partial_avg(cast(c_birth_year#19 as decimal(12,2))), partial_avg(cast(cd_dep_count#14 as decimal(12,2)))]
-(38) CometExchange
-Input [19]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, sum#34, count#35, sum#36, count#37, sum#38, count#39, sum#40, count#41, sum#42, count#43, sum#44, count#45, sum#46, count#47]
-Arguments: hashpartitioning(i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(40) CometExchange
+Input [19]: [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, spark_grouping_id#35, sum#36, count#37, sum#38, count#39, sum#40, count#41, sum#42, count#43, sum#44, count#45, sum#46, count#47, sum#48, count#49]
+Arguments: hashpartitioning(i_item_id#31, ca_country#32, ca_state#33, ca_county#34, spark_grouping_id#35, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(39) CometHashAggregate
-Input [19]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33, sum#34, count#35, sum#36, count#37, sum#38, count#39, sum#40, count#41, sum#42, count#43, sum#44, count#45, sum#46, count#47]
-Keys [5]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, spark_grouping_id#33]
+(41) CometHashAggregate
+Input [19]: [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, spark_grouping_id#35, sum#36, count#37, sum#38, count#39, sum#40, count#41, sum#42, count#43, sum#44, count#45, sum#46, count#47, sum#48, count#49]
+Keys [5]: [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, spark_grouping_id#35]
Functions [7]: [avg(cast(cs_quantity#4 as decimal(12,2))), avg(cast(cs_list_price#5 as decimal(12,2))), avg(cast(cs_coupon_amt#7 as decimal(12,2))), avg(cast(cs_sales_price#6 as decimal(12,2))), avg(cast(cs_net_profit#8 as decimal(12,2))), avg(cast(c_birth_year#19 as decimal(12,2))), avg(cast(cd_dep_count#14 as decimal(12,2)))]
-(40) CometTakeOrderedAndProject
-Input [11]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, agg1#48, agg2#49, agg3#50, agg4#51, agg5#52, agg6#53, agg7#54]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[ca_country#30 ASC NULLS FIRST,ca_state#31 ASC NULLS FIRST,ca_county#32 ASC NULLS FIRST,i_item_id#29 ASC NULLS FIRST], output=[i_item_id#29,ca_country#30,ca_state#31,ca_county#32,agg1#48,agg2#49,agg3#50,agg4#51,agg5#52,agg6#53,agg7#54]), [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, agg1#48, agg2#49, agg3#50, agg4#51, agg5#52, agg6#53, agg7#54], 100, [ca_country#30 ASC NULLS FIRST, ca_state#31 ASC NULLS FIRST, ca_county#32 AS [...]
+(42) CometTakeOrderedAndProject
+Input [11]: [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, agg1#50, agg2#51, agg3#52, agg4#53, agg5#54, agg6#55, agg7#56]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ca_country#32 ASC NULLS FIRST,ca_state#33 ASC NULLS FIRST,ca_county#34 ASC NULLS FIRST,i_item_id#31 ASC NULLS FIRST], output=[i_item_id#31,ca_country#32,ca_state#33,ca_county#34,agg1#50,agg2#51,agg3#52,agg4#53,agg5#54,agg6#55,agg7#56]), [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, agg1#50, agg2#51, agg3#52, agg4#53, agg5#54, agg6#55, agg7#56], 100, [ca_country#32 ASC NULLS FIRST, ca_state#33 ASC NULLS FIRST, ca_county#34 AS [...]
-(41) CometColumnarToRow [codegen id : 1]
-Input [11]: [i_item_id#29, ca_country#30, ca_state#31, ca_county#32, agg1#48, agg2#49, agg3#50, agg4#51, agg5#52, agg6#53, agg7#54]
+(43) CometColumnarToRow [codegen id : 1]
+Input [11]: [i_item_id#31, ca_country#32, ca_state#33, ca_county#34, agg1#50, agg2#51, agg3#52, agg4#53, agg5#54, agg6#55, agg7#56]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#9 IN dynamicpruning#10
-BroadcastExchange (46)
-+- * CometColumnarToRow (45)
- +- CometProject (44)
- +- CometFilter (43)
- +- CometScan parquet spark_catalog.default.date_dim (42)
+BroadcastExchange (48)
++- * CometColumnarToRow (47)
+ +- CometProject (46)
+ +- CometFilter (45)
+ +- CometScan parquet spark_catalog.default.date_dim (44)
-(42) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#25, d_year#26]
+(44) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#26, d_year#27]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,1998), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(43) CometFilter
-Input [2]: [d_date_sk#25, d_year#26]
-Condition : ((isnotnull(d_year#26) AND (d_year#26 = 1998)) AND isnotnull(d_date_sk#25))
+(45) CometFilter
+Input [2]: [d_date_sk#26, d_year#27]
+Condition : ((isnotnull(d_year#27) AND (d_year#27 = 1998)) AND isnotnull(d_date_sk#26))
-(44) CometProject
-Input [2]: [d_date_sk#25, d_year#26]
-Arguments: [d_date_sk#25], [d_date_sk#25]
+(46) CometProject
+Input [2]: [d_date_sk#26, d_year#27]
+Arguments: [d_date_sk#26], [d_date_sk#26]
-(45) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#25]
+(47) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#26]
-(46) BroadcastExchange
-Input [1]: [d_date_sk#25]
+(48) BroadcastExchange
+Input [1]: [d_date_sk#26]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt
index eaf02960f..c07cc28c7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q18/simplified.txt
@@ -40,12 +40,14 @@ WholeStageCodegen (1)
CometFilter [cd_demo_sk]
CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk]
CometBroadcastExchange [ca_address_sk,ca_county,ca_state,ca_country] #6
- CometFilter [ca_address_sk,ca_county,ca_state,ca_country]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state,ca_country]
+ CometProject [ca_state] [ca_address_sk,ca_county,ca_state,ca_country]
+ CometFilter [ca_address_sk,ca_county,ca_state,ca_country]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state,ca_country]
CometBroadcastExchange [d_date_sk] #7
CometProject [d_date_sk]
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [i_item_sk,i_item_id] #8
- CometFilter [i_item_sk,i_item_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
+ CometProject [i_item_id] [i_item_sk,i_item_id]
+ CometFilter [i_item_sk,i_item_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt
index c1bf0f14a..11c0339bb 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-* CometColumnarToRow (34)
-+- CometTakeOrderedAndProject (33)
- +- CometHashAggregate (32)
- +- CometExchange (31)
- +- CometHashAggregate (30)
- +- CometProject (29)
- +- CometBroadcastHashJoin (28)
- :- CometProject (24)
- : +- CometBroadcastHashJoin (23)
+* CometColumnarToRow (36)
++- CometTakeOrderedAndProject (35)
+ +- CometHashAggregate (34)
+ +- CometExchange (33)
+ +- CometHashAggregate (32)
+ +- CometProject (31)
+ +- CometBroadcastHashJoin (30)
+ :- CometProject (25)
+ : +- CometBroadcastHashJoin (24)
: :- CometProject (19)
: : +- CometBroadcastHashJoin (18)
: : :- CometProject (14)
@@ -27,12 +27,14 @@
: : +- CometBroadcastExchange (17)
: : +- CometFilter (16)
: : +- CometScan parquet spark_catalog.default.customer (15)
- : +- CometBroadcastExchange (22)
- : +- CometFilter (21)
- : +- CometScan parquet spark_catalog.default.customer_address (20)
- +- CometBroadcastExchange (27)
- +- CometFilter (26)
- +- CometScan parquet spark_catalog.default.store (25)
+ : +- CometBroadcastExchange (23)
+ : +- CometProject (22)
+ : +- CometFilter (21)
+ : +- CometScan parquet spark_catalog.default.customer_address (20)
+ +- CometBroadcastExchange (29)
+ +- CometProject (28)
+ +- CometFilter (27)
+ +- CometScan parquet spark_catalog.default.store (26)
(1) CometScan parquet spark_catalog.default.date_dim
@@ -88,111 +90,119 @@ Condition : ((isnotnull(i_manager_id#14) AND (i_manager_id#14 = 8)) AND isnotnul
(11) CometProject
Input [6]: [i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, i_manager_id#14]
-Arguments: [i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13], [i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
+Arguments: [i_item_sk#9, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16], [i_item_sk#9, i_brand_id#10, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_brand#11, 50, true, false, true) AS i_brand#15, i_manufact_id#12, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_manufact#13, 50, true, false, true) AS i_manufact#16]
(12) CometBroadcastExchange
-Input [5]: [i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
-Arguments: [i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
+Input [5]: [i_item_sk#9, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
+Arguments: [i_item_sk#9, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
(13) CometBroadcastHashJoin
Left output [4]: [ss_item_sk#4, ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7]
-Right output [5]: [i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
+Right output [5]: [i_item_sk#9, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
Arguments: [ss_item_sk#4], [i_item_sk#9], Inner, BuildRight
(14) CometProject
-Input [9]: [ss_item_sk#4, ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_item_sk#9, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
-Arguments: [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13], [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
+Input [9]: [ss_item_sk#4, ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_item_sk#9, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
+Arguments: [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16], [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
(15) CometScan parquet spark_catalog.default.customer
-Output [2]: [c_customer_sk#15, c_current_addr_sk#16]
+Output [2]: [c_customer_sk#17, c_current_addr_sk#18]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_current_addr_sk)]
ReadSchema: struct<c_customer_sk:int,c_current_addr_sk:int>
(16) CometFilter
-Input [2]: [c_customer_sk#15, c_current_addr_sk#16]
-Condition : (isnotnull(c_customer_sk#15) AND isnotnull(c_current_addr_sk#16))
+Input [2]: [c_customer_sk#17, c_current_addr_sk#18]
+Condition : (isnotnull(c_customer_sk#17) AND isnotnull(c_current_addr_sk#18))
(17) CometBroadcastExchange
-Input [2]: [c_customer_sk#15, c_current_addr_sk#16]
-Arguments: [c_customer_sk#15, c_current_addr_sk#16]
+Input [2]: [c_customer_sk#17, c_current_addr_sk#18]
+Arguments: [c_customer_sk#17, c_current_addr_sk#18]
(18) CometBroadcastHashJoin
-Left output [7]: [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
-Right output [2]: [c_customer_sk#15, c_current_addr_sk#16]
-Arguments: [ss_customer_sk#5], [c_customer_sk#15], Inner, BuildRight
+Left output [7]: [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
+Right output [2]: [c_customer_sk#17, c_current_addr_sk#18]
+Arguments: [ss_customer_sk#5], [c_customer_sk#17], Inner, BuildRight
(19) CometProject
-Input [9]: [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, c_customer_sk#15, c_current_addr_sk#16]
-Arguments: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, c_current_addr_sk#16], [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, c_current_addr_sk#16]
+Input [9]: [ss_customer_sk#5, ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, c_customer_sk#17, c_current_addr_sk#18]
+Arguments: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, c_current_addr_sk#18], [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, c_current_addr_sk#18]
(20) CometScan parquet spark_catalog.default.customer_address
-Output [2]: [ca_address_sk#17, ca_zip#18]
+Output [2]: [ca_address_sk#19, ca_zip#20]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
-PushedFilters: [IsNotNull(ca_address_sk), IsNotNull(ca_zip)]
+PushedFilters: [IsNotNull(ca_address_sk)]
ReadSchema: struct<ca_address_sk:int,ca_zip:string>
(21) CometFilter
-Input [2]: [ca_address_sk#17, ca_zip#18]
-Condition : (isnotnull(ca_address_sk#17) AND isnotnull(ca_zip#18))
+Input [2]: [ca_address_sk#19, ca_zip#20]
+Condition : (isnotnull(ca_address_sk#19) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#20, 10, true, false, true)))
-(22) CometBroadcastExchange
-Input [2]: [ca_address_sk#17, ca_zip#18]
-Arguments: [ca_address_sk#17, ca_zip#18]
+(22) CometProject
+Input [2]: [ca_address_sk#19, ca_zip#20]
+Arguments: [ca_address_sk#19, ca_zip#21], [ca_address_sk#19, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#20, 10, true, false, true) AS ca_zip#21]
-(23) CometBroadcastHashJoin
-Left output [7]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, c_current_addr_sk#16]
-Right output [2]: [ca_address_sk#17, ca_zip#18]
-Arguments: [c_current_addr_sk#16], [ca_address_sk#17], Inner, BuildRight
+(23) CometBroadcastExchange
+Input [2]: [ca_address_sk#19, ca_zip#21]
+Arguments: [ca_address_sk#19, ca_zip#21]
-(24) CometProject
-Input [9]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, c_current_addr_sk#16, ca_address_sk#17, ca_zip#18]
-Arguments: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, ca_zip#18], [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, ca_zip#18]
+(24) CometBroadcastHashJoin
+Left output [7]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, c_current_addr_sk#18]
+Right output [2]: [ca_address_sk#19, ca_zip#21]
+Arguments: [c_current_addr_sk#18], [ca_address_sk#19], Inner, BuildRight
-(25) CometScan parquet spark_catalog.default.store
-Output [2]: [s_store_sk#19, s_zip#20]
+(25) CometProject
+Input [9]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, c_current_addr_sk#18, ca_address_sk#19, ca_zip#21]
+Arguments: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, ca_zip#21], [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, ca_zip#21]
+
+(26) CometScan parquet spark_catalog.default.store
+Output [2]: [s_store_sk#22, s_zip#23]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
-PushedFilters: [IsNotNull(s_zip), IsNotNull(s_store_sk)]
+PushedFilters: [IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_zip:string>
-(26) CometFilter
-Input [2]: [s_store_sk#19, s_zip#20]
-Condition : (isnotnull(s_zip#20) AND isnotnull(s_store_sk#19))
+(27) CometFilter
+Input [2]: [s_store_sk#22, s_zip#23]
+Condition : (isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_zip#23, 10, true, false, true)) AND isnotnull(s_store_sk#22))
+
+(28) CometProject
+Input [2]: [s_store_sk#22, s_zip#23]
+Arguments: [s_store_sk#22, s_zip#24], [s_store_sk#22, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_zip#23, 10, true, false, true) AS s_zip#24]
-(27) CometBroadcastExchange
-Input [2]: [s_store_sk#19, s_zip#20]
-Arguments: [s_store_sk#19, s_zip#20]
+(29) CometBroadcastExchange
+Input [2]: [s_store_sk#22, s_zip#24]
+Arguments: [s_store_sk#22, s_zip#24]
-(28) CometBroadcastHashJoin
-Left output [7]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, ca_zip#18]
-Right output [2]: [s_store_sk#19, s_zip#20]
-Arguments: [ss_store_sk#6], [s_store_sk#19], Inner, NOT (substr(ca_zip#18, 1, 5) = substr(s_zip#20, 1, 5)), BuildRight
+(30) CometBroadcastHashJoin
+Left output [7]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, ca_zip#21]
+Right output [2]: [s_store_sk#22, s_zip#24]
+Arguments: [ss_store_sk#6], [s_store_sk#22], Inner, NOT (substr(ca_zip#21, 1, 5) = substr(s_zip#24, 1, 5)), BuildRight
-(29) CometProject
-Input [9]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13, ca_zip#18, s_store_sk#19, s_zip#20]
-Arguments: [ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13], [ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
+(31) CometProject
+Input [9]: [ss_store_sk#6, ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16, ca_zip#21, s_store_sk#22, s_zip#24]
+Arguments: [ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16], [ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
-(30) CometHashAggregate
-Input [5]: [ss_ext_sales_price#7, i_brand_id#10, i_brand#11, i_manufact_id#12, i_manufact#13]
-Keys [4]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13]
+(32) CometHashAggregate
+Input [5]: [ss_ext_sales_price#7, i_brand_id#10, i_brand#15, i_manufact_id#12, i_manufact#16]
+Keys [4]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#7))]
-(31) CometExchange
-Input [5]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, sum#21]
-Arguments: hashpartitioning(i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(33) CometExchange
+Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
+Arguments: hashpartitioning(i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(32) CometHashAggregate
-Input [5]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13, sum#21]
-Keys [4]: [i_brand#11, i_brand_id#10, i_manufact_id#12, i_manufact#13]
+(34) CometHashAggregate
+Input [5]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16, sum#25]
+Keys [4]: [i_brand#15, i_brand_id#10, i_manufact_id#12, i_manufact#16]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#7))]
-(33) CometTakeOrderedAndProject
-Input [5]: [brand_id#22, brand#23, i_manufact_id#12, i_manufact#13, ext_price#24]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[ext_price#24 DESC NULLS LAST,brand#23 ASC NULLS FIRST,brand_id#22 ASC NULLS FIRST,i_manufact_id#12 ASC NULLS FIRST,i_manufact#13 ASC NULLS FIRST], output=[brand_id#22,brand#23,i_manufact_id#12,i_manufact#13,ext_price#24]), [brand_id#22, brand#23, i_manufact_id#12, i_manufact#13, ext_price#24], 100, [ext_price#24 DESC NULLS LAST, brand#23 ASC NULLS FIRST, brand_id#22 ASC NULLS FIRST, i_manufact_id#12 ASC NULLS FIRST, i_manufact#13 ASC N [...]
+(35) CometTakeOrderedAndProject
+Input [5]: [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[ext_price#28 DESC NULLS LAST,brand#27 ASC NULLS FIRST,brand_id#26 ASC NULLS FIRST,i_manufact_id#12 ASC NULLS FIRST,i_manufact#16 ASC NULLS FIRST], output=[brand_id#26,brand#27,i_manufact_id#12,i_manufact#16,ext_price#28]), [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28], 100, [ext_price#28 DESC NULLS LAST, brand#27 ASC NULLS FIRST, brand_id#26 ASC NULLS FIRST, i_manufact_id#12 ASC NULLS FIRST, i_manufact#16 ASC N [...]
-(34) CometColumnarToRow [codegen id : 1]
-Input [5]: [brand_id#22, brand#23, i_manufact_id#12, i_manufact#13, ext_price#24]
+(36) CometColumnarToRow [codegen id : 1]
+Input [5]: [brand_id#26, brand#27, i_manufact_id#12, i_manufact#16, ext_price#28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt
index bfbf7fe94..7561e8d86 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q19/simplified.txt
@@ -22,15 +22,17 @@ WholeStageCodegen (1)
CometFilter [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ext_sales_price,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ext_sales_price,ss_sold_date_sk]
CometBroadcastExchange [i_item_sk,i_brand_id,i_brand,i_manufact_id,i_manufact] #3
- CometProject [i_item_sk,i_brand_id,i_brand,i_manufact_id,i_manufact]
+ CometProject [i_brand,i_manufact] [i_item_sk,i_brand_id,i_brand,i_manufact_id,i_manufact]
CometFilter [i_item_sk,i_brand_id,i_brand,i_manufact_id,i_manufact,i_manager_id]
CometScan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_brand,i_manufact_id,i_manufact,i_manager_id]
CometBroadcastExchange [c_customer_sk,c_current_addr_sk] #4
CometFilter [c_customer_sk,c_current_addr_sk]
CometScan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
CometBroadcastExchange [ca_address_sk,ca_zip] #5
- CometFilter [ca_address_sk,ca_zip]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_zip]
+ CometProject [ca_zip] [ca_address_sk,ca_zip]
+ CometFilter [ca_address_sk,ca_zip]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_zip]
CometBroadcastExchange [s_store_sk,s_zip] #6
- CometFilter [s_store_sk,s_zip]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_zip]
+ CometProject [s_zip] [s_store_sk,s_zip]
+ CometFilter [s_store_sk,s_zip]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_zip]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt
index 8e81d0b6a..5ff3a2a9b 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/explain.txt
@@ -1,37 +1,38 @@
== Physical Plan ==
-* CometColumnarToRow (33)
-+- CometSort (32)
- +- CometColumnarExchange (31)
- +- CometProject (30)
- +- CometBroadcastHashJoin (29)
- :- CometProject (19)
- : +- CometBroadcastHashJoin (18)
- : :- CometHashAggregate (13)
- : : +- CometExchange (12)
- : : +- CometHashAggregate (11)
- : : +- CometProject (10)
- : : +- CometBroadcastHashJoin (9)
+* CometColumnarToRow (34)
++- CometSort (33)
+ +- CometColumnarExchange (32)
+ +- CometProject (31)
+ +- CometBroadcastHashJoin (30)
+ :- CometProject (20)
+ : +- CometBroadcastHashJoin (19)
+ : :- CometHashAggregate (14)
+ : : +- CometExchange (13)
+ : : +- CometHashAggregate (12)
+ : : +- CometProject (11)
+ : : +- CometBroadcastHashJoin (10)
: : :- CometUnion (5)
: : : :- CometProject (2)
: : : : +- CometScan parquet spark_catalog.default.web_sales (1)
: : : +- CometProject (4)
: : : +- CometScan parquet spark_catalog.default.catalog_sales (3)
- : : +- CometBroadcastExchange (8)
- : : +- CometFilter (7)
- : : +- CometScan parquet spark_catalog.default.date_dim (6)
- : +- CometBroadcastExchange (17)
- : +- CometProject (16)
- : +- CometFilter (15)
- : +- CometScan parquet spark_catalog.default.date_dim (14)
- +- CometBroadcastExchange (28)
- +- CometProject (27)
- +- CometBroadcastHashJoin (26)
- :- CometHashAggregate (21)
- : +- ReusedExchange (20)
- +- CometBroadcastExchange (25)
- +- CometProject (24)
- +- CometFilter (23)
- +- CometScan parquet spark_catalog.default.date_dim (22)
+ : : +- CometBroadcastExchange (9)
+ : : +- CometProject (8)
+ : : +- CometFilter (7)
+ : : +- CometScan parquet spark_catalog.default.date_dim (6)
+ : +- CometBroadcastExchange (18)
+ : +- CometProject (17)
+ : +- CometFilter (16)
+ : +- CometScan parquet spark_catalog.default.date_dim (15)
+ +- CometBroadcastExchange (29)
+ +- CometProject (28)
+ +- CometBroadcastHashJoin (27)
+ :- CometHashAggregate (22)
+ : +- ReusedExchange (21)
+ +- CometBroadcastExchange (26)
+ +- CometProject (25)
+ +- CometFilter (24)
+ +- CometScan parquet spark_catalog.default.date_dim (23)
(1) CometScan parquet spark_catalog.default.web_sales
@@ -71,118 +72,122 @@ ReadSchema: struct<d_date_sk:int,d_week_seq:int,d_day_name:string>
Input [3]: [d_date_sk#9, d_week_seq#10, d_day_name#11]
Condition : (isnotnull(d_date_sk#9) AND isnotnull(d_week_seq#10))
-(8) CometBroadcastExchange
+(8) CometProject
Input [3]: [d_date_sk#9, d_week_seq#10, d_day_name#11]
-Arguments: [d_date_sk#9, d_week_seq#10, d_day_name#11]
+Arguments: [d_date_sk#9, d_week_seq#10, d_day_name#12], [d_date_sk#9, d_week_seq#10, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, d_day_name#11, 9, true, false, true) AS d_day_name#12]
-(9) CometBroadcastHashJoin
+(9) CometBroadcastExchange
+Input [3]: [d_date_sk#9, d_week_seq#10, d_day_name#12]
+Arguments: [d_date_sk#9, d_week_seq#10, d_day_name#12]
+
+(10) CometBroadcastHashJoin
Left output [2]: [sold_date_sk#3, sales_price#4]
-Right output [3]: [d_date_sk#9, d_week_seq#10, d_day_name#11]
+Right output [3]: [d_date_sk#9, d_week_seq#10, d_day_name#12]
Arguments: [sold_date_sk#3], [d_date_sk#9], Inner, BuildRight
-(10) CometProject
-Input [5]: [sold_date_sk#3, sales_price#4, d_date_sk#9, d_week_seq#10, d_day_name#11]
-Arguments: [sales_price#4, d_week_seq#10, d_day_name#11], [sales_price#4, d_week_seq#10, d_day_name#11]
+(11) CometProject
+Input [5]: [sold_date_sk#3, sales_price#4, d_date_sk#9, d_week_seq#10, d_day_name#12]
+Arguments: [sales_price#4, d_week_seq#10, d_day_name#12], [sales_price#4, d_week_seq#10, d_day_name#12]
-(11) CometHashAggregate
-Input [3]: [sales_price#4, d_week_seq#10, d_day_name#11]
+(12) CometHashAggregate
+Input [3]: [sales_price#4, d_week_seq#10, d_day_name#12]
Keys [1]: [d_week_seq#10]
-Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Sunday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Monday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Tuesday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Wednesday) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Thu [...]
+Functions [7]: [partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END)), partial_sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END)), partial_sum(UnscaledValue( [...]
-(12) CometExchange
-Input [8]: [d_week_seq#10, sum#12, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18]
+(13) CometExchange
+Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
Arguments: hashpartitioning(d_week_seq#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(13) CometHashAggregate
-Input [8]: [d_week_seq#10, sum#12, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18]
+(14) CometHashAggregate
+Input [8]: [d_week_seq#10, sum#13, sum#14, sum#15, sum#16, sum#17, sum#18, sum#19]
Keys [1]: [d_week_seq#10]
-Functions [7]: [sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Sunday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Monday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Tuesday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Wednesday) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#11, 9, ) = Thursday ) THEN sales_price#4 END)), sum(Un [...]
+Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Friday ) THEN sales [...]
-(14) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_week_seq#19, d_year#20]
+(15) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_week_seq#20, d_year#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_week_seq)]
ReadSchema: struct<d_week_seq:int,d_year:int>
-(15) CometFilter
-Input [2]: [d_week_seq#19, d_year#20]
-Condition : ((isnotnull(d_year#20) AND (d_year#20 = 2001)) AND isnotnull(d_week_seq#19))
+(16) CometFilter
+Input [2]: [d_week_seq#20, d_year#21]
+Condition : ((isnotnull(d_year#21) AND (d_year#21 = 2001)) AND isnotnull(d_week_seq#20))
-(16) CometProject
-Input [2]: [d_week_seq#19, d_year#20]
-Arguments: [d_week_seq#19], [d_week_seq#19]
+(17) CometProject
+Input [2]: [d_week_seq#20, d_year#21]
+Arguments: [d_week_seq#20], [d_week_seq#20]
-(17) CometBroadcastExchange
-Input [1]: [d_week_seq#19]
-Arguments: [d_week_seq#19]
+(18) CometBroadcastExchange
+Input [1]: [d_week_seq#20]
+Arguments: [d_week_seq#20]
-(18) CometBroadcastHashJoin
-Left output [8]: [d_week_seq#10, sun_sales#21, mon_sales#22, tue_sales#23, wed_sales#24, thu_sales#25, fri_sales#26, sat_sales#27]
-Right output [1]: [d_week_seq#19]
-Arguments: [d_week_seq#10], [d_week_seq#19], Inner, BuildRight
+(19) CometBroadcastHashJoin
+Left output [8]: [d_week_seq#10, sun_sales#22, mon_sales#23, tue_sales#24, wed_sales#25, thu_sales#26, fri_sales#27, sat_sales#28]
+Right output [1]: [d_week_seq#20]
+Arguments: [d_week_seq#10], [d_week_seq#20], Inner, BuildRight
-(19) CometProject
-Input [9]: [d_week_seq#10, sun_sales#21, mon_sales#22, tue_sales#23, wed_sales#24, thu_sales#25, fri_sales#26, sat_sales#27, d_week_seq#19]
-Arguments: [d_week_seq1#28, sun_sales1#29, mon_sales1#30, tue_sales1#31, wed_sales1#32, thu_sales1#33, fri_sales1#34, sat_sales1#35], [d_week_seq#10 AS d_week_seq1#28, sun_sales#21 AS sun_sales1#29, mon_sales#22 AS mon_sales1#30, tue_sales#23 AS tue_sales1#31, wed_sales#24 AS wed_sales1#32, thu_sales#25 AS thu_sales1#33, fri_sales#26 AS fri_sales1#34, sat_sales#27 AS sat_sales1#35]
+(20) CometProject
+Input [9]: [d_week_seq#10, sun_sales#22, mon_sales#23, tue_sales#24, wed_sales#25, thu_sales#26, fri_sales#27, sat_sales#28, d_week_seq#20]
+Arguments: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36], [d_week_seq#10 AS d_week_seq1#29, sun_sales#22 AS sun_sales1#30, mon_sales#23 AS mon_sales1#31, tue_sales#24 AS tue_sales1#32, wed_sales#25 AS wed_sales1#33, thu_sales#26 AS thu_sales1#34, fri_sales#27 AS fri_sales1#35, sat_sales#28 AS sat_sales1#36]
-(20) ReusedExchange [Reuses operator id: 12]
-Output [8]: [d_week_seq#36, sum#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43]
+(21) ReusedExchange [Reuses operator id: 13]
+Output [8]: [d_week_seq#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43, sum#44]
-(21) CometHashAggregate
-Input [8]: [d_week_seq#36, sum#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43]
-Keys [1]: [d_week_seq#36]
-Functions [7]: [sum(UnscaledValue(CASE WHEN (rpad(d_day_name#44, 9, ) = Sunday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#44, 9, ) = Monday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#44, 9, ) = Tuesday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#44, 9, ) = Wednesday) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (rpad(d_day_name#44, 9, ) = Thursday ) THEN sales_price#4 END)), sum(Un [...]
+(22) CometHashAggregate
+Input [8]: [d_week_seq#37, sum#38, sum#39, sum#40, sum#41, sum#42, sum#43, sum#44]
+Keys [1]: [d_week_seq#37]
+Functions [7]: [sum(UnscaledValue(CASE WHEN (d_day_name#12 = Sunday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Monday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Tuesday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Wednesday) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Thursday ) THEN sales_price#4 END)), sum(UnscaledValue(CASE WHEN (d_day_name#12 = Friday ) THEN sales [...]
-(22) CometScan parquet spark_catalog.default.date_dim
+(23) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_week_seq#45, d_year#46]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_week_seq)]
ReadSchema: struct<d_week_seq:int,d_year:int>
-(23) CometFilter
+(24) CometFilter
Input [2]: [d_week_seq#45, d_year#46]
Condition : ((isnotnull(d_year#46) AND (d_year#46 = 2002)) AND isnotnull(d_week_seq#45))
-(24) CometProject
+(25) CometProject
Input [2]: [d_week_seq#45, d_year#46]
Arguments: [d_week_seq#45], [d_week_seq#45]
-(25) CometBroadcastExchange
+(26) CometBroadcastExchange
Input [1]: [d_week_seq#45]
Arguments: [d_week_seq#45]
-(26) CometBroadcastHashJoin
-Left output [8]: [d_week_seq#36, sun_sales#47, mon_sales#48, tue_sales#49, wed_sales#50, thu_sales#51, fri_sales#52, sat_sales#53]
+(27) CometBroadcastHashJoin
+Left output [8]: [d_week_seq#37, sun_sales#47, mon_sales#48, tue_sales#49, wed_sales#50, thu_sales#51, fri_sales#52, sat_sales#53]
Right output [1]: [d_week_seq#45]
-Arguments: [d_week_seq#36], [d_week_seq#45], Inner, BuildRight
+Arguments: [d_week_seq#37], [d_week_seq#45], Inner, BuildRight
-(27) CometProject
-Input [9]: [d_week_seq#36, sun_sales#47, mon_sales#48, tue_sales#49, wed_sales#50, thu_sales#51, fri_sales#52, sat_sales#53, d_week_seq#45]
-Arguments: [d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61], [d_week_seq#36 AS d_week_seq2#54, sun_sales#47 AS sun_sales2#55, mon_sales#48 AS mon_sales2#56, tue_sales#49 AS tue_sales2#57, wed_sales#50 AS wed_sales2#58, thu_sales#51 AS thu_sales2#59, fri_sales#52 AS fri_sales2#60, sat_sales#53 AS sat_sales2#61]
+(28) CometProject
+Input [9]: [d_week_seq#37, sun_sales#47, mon_sales#48, tue_sales#49, wed_sales#50, thu_sales#51, fri_sales#52, sat_sales#53, d_week_seq#45]
+Arguments: [d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61], [d_week_seq#37 AS d_week_seq2#54, sun_sales#47 AS sun_sales2#55, mon_sales#48 AS mon_sales2#56, tue_sales#49 AS tue_sales2#57, wed_sales#50 AS wed_sales2#58, thu_sales#51 AS thu_sales2#59, fri_sales#52 AS fri_sales2#60, sat_sales#53 AS sat_sales2#61]
-(28) CometBroadcastExchange
+(29) CometBroadcastExchange
Input [8]: [d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61]
Arguments: [d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61]
-(29) CometBroadcastHashJoin
-Left output [8]: [d_week_seq1#28, sun_sales1#29, mon_sales1#30, tue_sales1#31, wed_sales1#32, thu_sales1#33, fri_sales1#34, sat_sales1#35]
+(30) CometBroadcastHashJoin
+Left output [8]: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36]
Right output [8]: [d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61]
-Arguments: [d_week_seq1#28], [(d_week_seq2#54 - 53)], Inner, BuildRight
+Arguments: [d_week_seq1#29], [(d_week_seq2#54 - 53)], Inner, BuildRight
-(30) CometProject
-Input [16]: [d_week_seq1#28, sun_sales1#29, mon_sales1#30, tue_sales1#31, wed_sales1#32, thu_sales1#33, fri_sales1#34, sat_sales1#35, d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61]
-Arguments: [d_week_seq1#28, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68], [d_week_seq1#28, round((sun_sales1#29 / sun_sales2#55), 2) AS round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1#30 / mon_sales2#56), 2) AS round((mon_sales1 / mon_sales2), 2)# [...]
+(31) CometProject
+Input [16]: [d_week_seq1#29, sun_sales1#30, mon_sales1#31, tue_sales1#32, wed_sales1#33, thu_sales1#34, fri_sales1#35, sat_sales1#36, d_week_seq2#54, sun_sales2#55, mon_sales2#56, tue_sales2#57, wed_sales2#58, thu_sales2#59, fri_sales2#60, sat_sales2#61]
+Arguments: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68], [d_week_seq1#29, round((sun_sales1#30 / sun_sales2#55), 2) AS round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1#31 / mon_sales2#56), 2) AS round((mon_sales1 / mon_sales2), 2)# [...]
-(31) CometColumnarExchange
-Input [8]: [d_week_seq1#28, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68]
-Arguments: rangepartitioning(d_week_seq1#28 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+(32) CometColumnarExchange
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68]
+Arguments: rangepartitioning(d_week_seq1#29 ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
-(32) CometSort
-Input [8]: [d_week_seq1#28, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68]
-Arguments: [d_week_seq1#28, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68], [d_week_seq1#28 ASC NULLS FIRST]
+(33) CometSort
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68]
+Arguments: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68], [d_week_seq1#29 ASC NULLS FIRST]
-(33) CometColumnarToRow [codegen id : 1]
-Input [8]: [d_week_seq1#28, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68]
+(34) CometColumnarToRow [codegen id : 1]
+Input [8]: [d_week_seq1#29, round((sun_sales1 / sun_sales2), 2)#62, round((mon_sales1 / mon_sales2), 2)#63, round((tue_sales1 / tue_sales2), 2)#64, round((wed_sales1 / wed_sales2), 2)#65, round((thu_sales1 / thu_sales2), 2)#66, round((fri_sales1 / fri_sales2), 2)#67, round((sat_sales1 / sat_sales2), 2)#68]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt
index 2b6061153..8e6713bf2 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q2/simplified.txt
@@ -7,7 +7,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1,d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2]
CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq1,sun_sales1,mon_sales1,tue_sales1,wed_sales1,thu_sales1,fri_sales1,sat_sales1]
CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
- CometHashAggregate [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Wednesday) THEN sales_price [...]
+ CometHashAggregate [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = [...]
CometExchange [d_week_seq] #2
CometHashAggregate [d_week_seq,sum,sum,sum,sum,sum,sum,sum,d_day_name,sales_price]
CometProject [sales_price,d_week_seq,d_day_name]
@@ -18,8 +18,9 @@ WholeStageCodegen (1)
CometProject [cs_sold_date_sk,cs_ext_sales_price] [sold_date_sk,sales_price]
CometScan parquet spark_catalog.default.catalog_sales [cs_ext_sales_price,cs_sold_date_sk]
CometBroadcastExchange [d_date_sk,d_week_seq,d_day_name] #3
- CometFilter [d_date_sk,d_week_seq,d_day_name]
- CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq,d_day_name]
+ CometProject [d_day_name] [d_date_sk,d_week_seq,d_day_name]
+ CometFilter [d_date_sk,d_week_seq,d_day_name]
+ CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq,d_day_name]
CometBroadcastExchange [d_week_seq] #4
CometProject [d_week_seq]
CometFilter [d_week_seq,d_year]
@@ -27,7 +28,7 @@ WholeStageCodegen (1)
CometBroadcastExchange [d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2] #5
CometProject [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales] [d_week_seq2,sun_sales2,mon_sales2,tue_sales2,wed_sales2,thu_sales2,fri_sales2,sat_sales2]
CometBroadcastHashJoin [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,d_week_seq]
- CometHashAggregate [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (rpad(d_day_name, 9, ) = Wednesday) THEN sales_pri [...]
+ CometHashAggregate [d_week_seq,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales,sum,sum,sum,sum,sum,sum,sum,sum(UnscaledValue(CASE WHEN (d_day_name = Sunday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Monday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Tuesday ) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name = Wednesday) THEN sales_price END)),sum(UnscaledValue(CASE WHEN (d_day_name [...]
ReusedExchange [d_week_seq,sum,sum,sum,sum,sum,sum,sum] #2
CometBroadcastExchange [d_week_seq] #6
CometProject [d_week_seq]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt
index 02c92eeb6..74999264a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/explain.txt
@@ -1,26 +1,27 @@
== Physical Plan ==
-TakeOrderedAndProject (22)
-+- * Project (21)
- +- Window (20)
- +- * CometColumnarToRow (19)
- +- CometSort (18)
- +- CometExchange (17)
- +- CometHashAggregate (16)
- +- CometExchange (15)
- +- CometHashAggregate (14)
- +- CometProject (13)
- +- CometBroadcastHashJoin (12)
- :- CometProject (7)
- : +- CometBroadcastHashJoin (6)
+TakeOrderedAndProject (23)
++- * Project (22)
+ +- Window (21)
+ +- * CometColumnarToRow (20)
+ +- CometSort (19)
+ +- CometExchange (18)
+ +- CometHashAggregate (17)
+ +- CometExchange (16)
+ +- CometHashAggregate (15)
+ +- CometProject (14)
+ +- CometBroadcastHashJoin (13)
+ :- CometProject (8)
+ : +- CometBroadcastHashJoin (7)
: :- CometFilter (2)
: : +- CometScan parquet spark_catalog.default.catalog_sales (1)
- : +- CometBroadcastExchange (5)
- : +- CometFilter (4)
- : +- CometScan parquet spark_catalog.default.item (3)
- +- CometBroadcastExchange (11)
- +- CometProject (10)
- +- CometFilter (9)
- +- CometScan parquet spark_catalog.default.date_dim (8)
+ : +- CometBroadcastExchange (6)
+ : +- CometProject (5)
+ : +- CometFilter (4)
+ : +- CometScan parquet spark_catalog.default.item (3)
+ +- CometBroadcastExchange (12)
+ +- CometProject (11)
+ +- CometFilter (10)
+ +- CometScan parquet spark_catalog.default.date_dim (9)
(1) CometScan parquet spark_catalog.default.catalog_sales
@@ -44,116 +45,120 @@ ReadSchema: struct<i_item_sk:int,i_item_id:string,i_item_desc:string,i_current_p
(4) CometFilter
Input [6]: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Condition : (rpad(i_category#10, 50, ) IN (Sports ,Books ,Home ) AND isnotnull(i_item_sk#5))
+Condition : (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_category#10, 50, true, false, true) IN (Sports ,Books ,Home ) AND isnotnull(i_item_sk#5))
-(5) CometBroadcastExchange
+(5) CometProject
Input [6]: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Arguments: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+Arguments: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13], [i_item_sk#5, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#6, 16, true, false, true) AS i_item_id#11, i_item_desc#7, i_current_price#8, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_class#9, 50, true, false, true) AS i_class#12, staticinvoke(class org.ap [...]
-(6) CometBroadcastHashJoin
+(6) CometBroadcastExchange
+Input [6]: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Arguments: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+
+(7) CometBroadcastHashJoin
Left output [3]: [cs_item_sk#1, cs_ext_sales_price#2, cs_sold_date_sk#3]
-Right output [6]: [i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+Right output [6]: [i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
Arguments: [cs_item_sk#1], [i_item_sk#5], Inner, BuildRight
-(7) CometProject
-Input [9]: [cs_item_sk#1, cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_sk#5, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Arguments: [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10], [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+(8) CometProject
+Input [9]: [cs_item_sk#1, cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_sk#5, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Arguments: [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13], [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
-(8) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#11, d_date#12]
+(9) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(9) CometFilter
-Input [2]: [d_date_sk#11, d_date#12]
-Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 1999-02-22)) AND (d_date#12 <= 1999-03-24)) AND isnotnull(d_date_sk#11))
+(10) CometFilter
+Input [2]: [d_date_sk#14, d_date#15]
+Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(10) CometProject
-Input [2]: [d_date_sk#11, d_date#12]
-Arguments: [d_date_sk#11], [d_date_sk#11]
+(11) CometProject
+Input [2]: [d_date_sk#14, d_date#15]
+Arguments: [d_date_sk#14], [d_date_sk#14]
-(11) CometBroadcastExchange
-Input [1]: [d_date_sk#11]
-Arguments: [d_date_sk#11]
+(12) CometBroadcastExchange
+Input [1]: [d_date_sk#14]
+Arguments: [d_date_sk#14]
-(12) CometBroadcastHashJoin
-Left output [7]: [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Right output [1]: [d_date_sk#11]
-Arguments: [cs_sold_date_sk#3], [d_date_sk#11], Inner, BuildRight
+(13) CometBroadcastHashJoin
+Left output [7]: [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Right output [1]: [d_date_sk#14]
+Arguments: [cs_sold_date_sk#3], [d_date_sk#14], Inner, BuildRight
-(13) CometProject
-Input [8]: [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10, d_date_sk#11]
-Arguments: [cs_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10], [cs_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
+(14) CometProject
+Input [8]: [cs_ext_sales_price#2, cs_sold_date_sk#3, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13, d_date_sk#14]
+Arguments: [cs_ext_sales_price#2, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13], [cs_ext_sales_price#2, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
-(14) CometHashAggregate
-Input [6]: [cs_ext_sales_price#2, i_item_id#6, i_item_desc#7, i_current_price#8, i_class#9, i_category#10]
-Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
+(15) CometHashAggregate
+Input [6]: [cs_ext_sales_price#2, i_item_id#11, i_item_desc#7, i_current_price#8, i_class#12, i_category#13]
+Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [partial_sum(UnscaledValue(cs_ext_sales_price#2))]
-(15) CometExchange
-Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
-Arguments: hashpartitioning(i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(16) CometExchange
+Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
+Arguments: hashpartitioning(i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(16) CometHashAggregate
-Input [6]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8, sum#13]
-Keys [5]: [i_item_id#6, i_item_desc#7, i_category#10, i_class#9, i_current_price#8]
+(17) CometHashAggregate
+Input [6]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8, sum#16]
+Keys [5]: [i_item_id#11, i_item_desc#7, i_category#13, i_class#12, i_current_price#8]
Functions [1]: [sum(UnscaledValue(cs_ext_sales_price#2))]
-(17) CometExchange
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: hashpartitioning(i_class#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+(18) CometExchange
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: hashpartitioning(i_class#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(18) CometSort
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6], [i_class#9 ASC NULLS FIRST]
+(19) CometSort
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11], [i_class#12 ASC NULLS FIRST]
-(19) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
+(20) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
-(20) Window
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6]
-Arguments: [sum(_w0#15) windowspecdefinition(i_class#9, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#16], [i_class#9]
+(21) Window
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11]
+Arguments: [sum(_w0#18) windowspecdefinition(i_class#12, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#19], [i_class#12]
-(21) Project [codegen id : 2]
-Output [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, ((_w0#15 * 100) / _we0#16) AS revenueratio#17, i_item_id#6]
-Input [8]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, _w0#15, i_item_id#6, _we0#16]
+(22) Project [codegen id : 2]
+Output [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, ((_w0#18 * 100) / _we0#19) AS revenueratio#20, i_item_id#11]
+Input [8]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, _w0#18, i_item_id#11, _we0#19]
-(22) TakeOrderedAndProject
-Input [7]: [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, revenueratio#17, i_item_id#6]
-Arguments: 100, [i_category#10 ASC NULLS FIRST, i_class#9 ASC NULLS FIRST, i_item_id#6 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#17 ASC NULLS FIRST], [i_item_desc#7, i_category#10, i_class#9, i_current_price#8, itemrevenue#14, revenueratio#17]
+(23) TakeOrderedAndProject
+Input [7]: [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20, i_item_id#11]
+Arguments: 100, [i_category#13 ASC NULLS FIRST, i_class#12 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST, i_item_desc#7 ASC NULLS FIRST, revenueratio#20 ASC NULLS FIRST], [i_item_desc#7, i_category#13, i_class#12, i_current_price#8, itemrevenue#17, revenueratio#20]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#3 IN dynamicpruning#4
-BroadcastExchange (27)
-+- * CometColumnarToRow (26)
- +- CometProject (25)
- +- CometFilter (24)
- +- CometScan parquet spark_catalog.default.date_dim (23)
+BroadcastExchange (28)
++- * CometColumnarToRow (27)
+ +- CometProject (26)
+ +- CometFilter (25)
+ +- CometScan parquet spark_catalog.default.date_dim (24)
-(23) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#11, d_date#12]
+(24) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#14, d_date#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,1999-02-22), LessThanOrEqual(d_date,1999-03-24), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
-(24) CometFilter
-Input [2]: [d_date_sk#11, d_date#12]
-Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 1999-02-22)) AND (d_date#12 <= 1999-03-24)) AND isnotnull(d_date_sk#11))
+(25) CometFilter
+Input [2]: [d_date_sk#14, d_date#15]
+Condition : (((isnotnull(d_date#15) AND (d_date#15 >= 1999-02-22)) AND (d_date#15 <= 1999-03-24)) AND isnotnull(d_date_sk#14))
-(25) CometProject
-Input [2]: [d_date_sk#11, d_date#12]
-Arguments: [d_date_sk#11], [d_date_sk#11]
+(26) CometProject
+Input [2]: [d_date_sk#14, d_date#15]
+Arguments: [d_date_sk#14], [d_date_sk#14]
-(26) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#11]
+(27) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#14]
-(27) BroadcastExchange
-Input [1]: [d_date_sk#11]
+(28) BroadcastExchange
+Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt
index 147d91b58..31da5e681 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q20/simplified.txt
@@ -26,8 +26,9 @@ TakeOrderedAndProject [i_category,i_class,i_item_id,i_item_desc,revenueratio,i_c
CometFilter [d_date_sk,d_date]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date]
CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category] #4
- CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometProject [i_item_id,i_class,i_category] [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_class,i_category]
CometBroadcastExchange [d_date_sk] #5
CometProject [d_date_sk]
CometFilter [d_date_sk,d_date]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt
index 0340ef91c..f575a19a9 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/explain.txt
@@ -74,69 +74,69 @@ Condition : (((isnotnull(i_current_price#10) AND (i_current_price#10 >= 0.99)) A
(10) CometProject
Input [3]: [i_item_sk#8, i_item_id#9, i_current_price#10]
-Arguments: [i_item_sk#8, i_item_id#9], [i_item_sk#8, i_item_id#9]
+Arguments: [i_item_sk#8, i_item_id#11], [i_item_sk#8, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#9, 16, true, false, true) AS i_item_id#11]
(11) CometBroadcastExchange
-Input [2]: [i_item_sk#8, i_item_id#9]
-Arguments: [i_item_sk#8, i_item_id#9]
+Input [2]: [i_item_sk#8, i_item_id#11]
+Arguments: [i_item_sk#8, i_item_id#11]
(12) CometBroadcastHashJoin
Left output [4]: [inv_item_sk#1, inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7]
-Right output [2]: [i_item_sk#8, i_item_id#9]
+Right output [2]: [i_item_sk#8, i_item_id#11]
Arguments: [inv_item_sk#1], [i_item_sk#8], Inner, BuildRight
(13) CometProject
-Input [6]: [inv_item_sk#1, inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_sk#8, i_item_id#9]
-Arguments: [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#9], [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#9]
+Input [6]: [inv_item_sk#1, inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_sk#8, i_item_id#11]
+Arguments: [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#11], [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#11]
(14) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#11, d_date#12]
+Output [2]: [d_date_sk#12, d_date#13]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-02-10), LessThanOrEqual(d_date,2000-04-10), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
(15) CometFilter
-Input [2]: [d_date_sk#11, d_date#12]
-Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 2000-02-10)) AND (d_date#12 <= 2000-04-10)) AND isnotnull(d_date_sk#11))
+Input [2]: [d_date_sk#12, d_date#13]
+Condition : (((isnotnull(d_date#13) AND (d_date#13 >= 2000-02-10)) AND (d_date#13 <= 2000-04-10)) AND isnotnull(d_date_sk#12))
(16) CometBroadcastExchange
-Input [2]: [d_date_sk#11, d_date#12]
-Arguments: [d_date_sk#11, d_date#12]
+Input [2]: [d_date_sk#12, d_date#13]
+Arguments: [d_date_sk#12, d_date#13]
(17) CometBroadcastHashJoin
-Left output [4]: [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#9]
-Right output [2]: [d_date_sk#11, d_date#12]
-Arguments: [inv_date_sk#4], [d_date_sk#11], Inner, BuildRight
+Left output [4]: [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#11]
+Right output [2]: [d_date_sk#12, d_date#13]
+Arguments: [inv_date_sk#4], [d_date_sk#12], Inner, BuildRight
(18) CometProject
-Input [6]: [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#9, d_date_sk#11, d_date#12]
-Arguments: [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#9, d_date#12], [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#9, d_date#12]
+Input [6]: [inv_quantity_on_hand#3, inv_date_sk#4, w_warehouse_name#7, i_item_id#11, d_date_sk#12, d_date#13]
+Arguments: [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#11, d_date#13], [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#11, d_date#13]
(19) CometHashAggregate
-Input [4]: [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#9, d_date#12]
-Keys [2]: [w_warehouse_name#7, i_item_id#9]
-Functions [2]: [partial_sum(CASE WHEN (d_date#12 < 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END), partial_sum(CASE WHEN (d_date#12 >= 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END)]
+Input [4]: [inv_quantity_on_hand#3, w_warehouse_name#7, i_item_id#11, d_date#13]
+Keys [2]: [w_warehouse_name#7, i_item_id#11]
+Functions [2]: [partial_sum(CASE WHEN (d_date#13 < 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END), partial_sum(CASE WHEN (d_date#13 >= 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END)]
(20) CometExchange
-Input [4]: [w_warehouse_name#7, i_item_id#9, sum#13, sum#14]
-Arguments: hashpartitioning(w_warehouse_name#7, i_item_id#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+Input [4]: [w_warehouse_name#7, i_item_id#11, sum#14, sum#15]
+Arguments: hashpartitioning(w_warehouse_name#7, i_item_id#11, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(21) CometHashAggregate
-Input [4]: [w_warehouse_name#7, i_item_id#9, sum#13, sum#14]
-Keys [2]: [w_warehouse_name#7, i_item_id#9]
-Functions [2]: [sum(CASE WHEN (d_date#12 < 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END), sum(CASE WHEN (d_date#12 >= 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END)]
+Input [4]: [w_warehouse_name#7, i_item_id#11, sum#14, sum#15]
+Keys [2]: [w_warehouse_name#7, i_item_id#11]
+Functions [2]: [sum(CASE WHEN (d_date#13 < 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END), sum(CASE WHEN (d_date#13 >= 2000-03-11) THEN inv_quantity_on_hand#3 ELSE 0 END)]
(22) CometFilter
-Input [4]: [w_warehouse_name#7, i_item_id#9, inv_before#15, inv_after#16]
-Condition : (CASE WHEN (inv_before#15 > 0) THEN (knownfloatingpointnormalized(normalizenanandzero((cast(inv_after#16 as double) / knownfloatingpointnormalized(normalizenanandzero(cast(inv_before#15 as double)))))) >= 0.666667) END AND CASE WHEN (inv_before#15 > 0) THEN (knownfloatingpointnormalized(normalizenanandzero((cast(inv_after#16 as double) / knownfloatingpointnormalized(normalizenanandzero(cast(inv_before#15 as double)))))) <= 1.5) END)
+Input [4]: [w_warehouse_name#7, i_item_id#11, inv_before#16, inv_after#17]
+Condition : (CASE WHEN (inv_before#16 > 0) THEN (knownfloatingpointnormalized(normalizenanandzero((cast(inv_after#17 as double) / knownfloatingpointnormalized(normalizenanandzero(cast(inv_before#16 as double)))))) >= 0.666667) END AND CASE WHEN (inv_before#16 > 0) THEN (knownfloatingpointnormalized(normalizenanandzero((cast(inv_after#17 as double) / knownfloatingpointnormalized(normalizenanandzero(cast(inv_before#16 as double)))))) <= 1.5) END)
(23) CometTakeOrderedAndProject
-Input [4]: [w_warehouse_name#7, i_item_id#9, inv_before#15, inv_after#16]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[w_warehouse_name#7 ASC NULLS FIRST,i_item_id#9 ASC NULLS FIRST], output=[w_warehouse_name#7,i_item_id#9,inv_before#15,inv_after#16]), [w_warehouse_name#7, i_item_id#9, inv_before#15, inv_after#16], 100, [w_warehouse_name#7 ASC NULLS FIRST, i_item_id#9 ASC NULLS FIRST], [w_warehouse_name#7, i_item_id#9, inv_before#15, inv_after#16]
+Input [4]: [w_warehouse_name#7, i_item_id#11, inv_before#16, inv_after#17]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[w_warehouse_name#7 ASC NULLS FIRST,i_item_id#11 ASC NULLS FIRST], output=[w_warehouse_name#7,i_item_id#11,inv_before#16,inv_after#17]), [w_warehouse_name#7, i_item_id#11, inv_before#16, inv_after#17], 100, [w_warehouse_name#7 ASC NULLS FIRST, i_item_id#11 ASC NULLS FIRST], [w_warehouse_name#7, i_item_id#11, inv_before#16, inv_after#17]
(24) CometColumnarToRow [codegen id : 1]
-Input [4]: [w_warehouse_name#7, i_item_id#9, inv_before#15, inv_after#16]
+Input [4]: [w_warehouse_name#7, i_item_id#11, inv_before#16, inv_after#17]
===== Subqueries =====
@@ -148,21 +148,21 @@ BroadcastExchange (28)
(25) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#11, d_date#12]
+Output [2]: [d_date_sk#12, d_date#13]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-02-10), LessThanOrEqual(d_date,2000-04-10), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
(26) CometFilter
-Input [2]: [d_date_sk#11, d_date#12]
-Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 2000-02-10)) AND (d_date#12 <= 2000-04-10)) AND isnotnull(d_date_sk#11))
+Input [2]: [d_date_sk#12, d_date#13]
+Condition : (((isnotnull(d_date#13) AND (d_date#13 >= 2000-02-10)) AND (d_date#13 <= 2000-04-10)) AND isnotnull(d_date_sk#12))
(27) CometColumnarToRow [codegen id : 1]
-Input [2]: [d_date_sk#11, d_date#12]
+Input [2]: [d_date_sk#12, d_date#13]
(28) BroadcastExchange
-Input [2]: [d_date_sk#11, d_date#12]
+Input [2]: [d_date_sk#12, d_date#13]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt
index 6ff97b53a..887a2b3d5 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q21/simplified.txt
@@ -25,7 +25,7 @@ WholeStageCodegen (1)
CometFilter [w_warehouse_sk,w_warehouse_name]
CometScan parquet spark_catalog.default.warehouse [w_warehouse_sk,w_warehouse_name]
CometBroadcastExchange [i_item_sk,i_item_id] #4
- CometProject [i_item_sk,i_item_id]
+ CometProject [i_item_id] [i_item_sk,i_item_id]
CometFilter [i_item_sk,i_item_id,i_current_price]
CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_current_price]
CometBroadcastExchange [d_date_sk,d_date] #5
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt
index 759689b1b..2b0bf616f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/explain.txt
@@ -1,14 +1,14 @@
== Physical Plan ==
-* CometColumnarToRow (24)
-+- CometTakeOrderedAndProject (23)
- +- CometHashAggregate (22)
- +- CometExchange (21)
- +- CometHashAggregate (20)
- +- CometExpand (19)
- +- CometProject (18)
- +- CometBroadcastHashJoin (17)
- :- CometProject (13)
- : +- CometBroadcastHashJoin (12)
+* CometColumnarToRow (25)
++- CometTakeOrderedAndProject (24)
+ +- CometHashAggregate (23)
+ +- CometExchange (22)
+ +- CometHashAggregate (21)
+ +- CometExpand (20)
+ +- CometProject (19)
+ +- CometBroadcastHashJoin (18)
+ :- CometProject (14)
+ : +- CometBroadcastHashJoin (13)
: :- CometProject (8)
: : +- CometBroadcastHashJoin (7)
: : :- CometFilter (2)
@@ -17,12 +17,13 @@
: : +- CometProject (5)
: : +- CometFilter (4)
: : +- CometScan parquet spark_catalog.default.date_dim (3)
- : +- CometBroadcastExchange (11)
- : +- CometFilter (10)
- : +- CometScan parquet spark_catalog.default.item (9)
- +- CometBroadcastExchange (16)
- +- CometFilter (15)
- +- CometScan parquet spark_catalog.default.warehouse (14)
+ : +- CometBroadcastExchange (12)
+ : +- CometProject (11)
+ : +- CometFilter (10)
+ : +- CometScan parquet spark_catalog.default.item (9)
+ +- CometBroadcastExchange (17)
+ +- CometFilter (16)
+ +- CometScan parquet spark_catalog.default.warehouse (15)
(1) CometScan parquet spark_catalog.default.inventory
@@ -76,97 +77,101 @@ ReadSchema: struct<i_item_sk:int,i_brand:string,i_class:string,i_category:string
Input [5]: [i_item_sk#8, i_brand#9, i_class#10, i_category#11, i_product_name#12]
Condition : isnotnull(i_item_sk#8)
-(11) CometBroadcastExchange
+(11) CometProject
Input [5]: [i_item_sk#8, i_brand#9, i_class#10, i_category#11, i_product_name#12]
-Arguments: [i_item_sk#8, i_brand#9, i_class#10, i_category#11, i_product_name#12]
+Arguments: [i_item_sk#8, i_brand#13, i_class#14, i_category#15, i_product_name#16], [i_item_sk#8, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_brand#9, 50, true, false, true) AS i_brand#13, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_class#10, 50, true, false, true) AS i_class#14, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, [...]
-(12) CometBroadcastHashJoin
+(12) CometBroadcastExchange
+Input [5]: [i_item_sk#8, i_brand#13, i_class#14, i_category#15, i_product_name#16]
+Arguments: [i_item_sk#8, i_brand#13, i_class#14, i_category#15, i_product_name#16]
+
+(13) CometBroadcastHashJoin
Left output [3]: [inv_item_sk#1, inv_warehouse_sk#2, inv_quantity_on_hand#3]
-Right output [5]: [i_item_sk#8, i_brand#9, i_class#10, i_category#11, i_product_name#12]
+Right output [5]: [i_item_sk#8, i_brand#13, i_class#14, i_category#15, i_product_name#16]
Arguments: [inv_item_sk#1], [i_item_sk#8], Inner, BuildRight
-(13) CometProject
-Input [8]: [inv_item_sk#1, inv_warehouse_sk#2, inv_quantity_on_hand#3, i_item_sk#8, i_brand#9, i_class#10, i_category#11, i_product_name#12]
-Arguments: [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#9, i_class#10, i_category#11, i_product_name#12], [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#9, i_class#10, i_category#11, i_product_name#12]
+(14) CometProject
+Input [8]: [inv_item_sk#1, inv_warehouse_sk#2, inv_quantity_on_hand#3, i_item_sk#8, i_brand#13, i_class#14, i_category#15, i_product_name#16]
+Arguments: [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#13, i_class#14, i_category#15, i_product_name#16], [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#13, i_class#14, i_category#15, i_product_name#16]
-(14) CometScan parquet spark_catalog.default.warehouse
-Output [1]: [w_warehouse_sk#13]
+(15) CometScan parquet spark_catalog.default.warehouse
+Output [1]: [w_warehouse_sk#17]
Batched: true
Location [not included in comparison]/{warehouse_dir}/warehouse]
PushedFilters: [IsNotNull(w_warehouse_sk)]
ReadSchema: struct<w_warehouse_sk:int>
-(15) CometFilter
-Input [1]: [w_warehouse_sk#13]
-Condition : isnotnull(w_warehouse_sk#13)
+(16) CometFilter
+Input [1]: [w_warehouse_sk#17]
+Condition : isnotnull(w_warehouse_sk#17)
-(16) CometBroadcastExchange
-Input [1]: [w_warehouse_sk#13]
-Arguments: [w_warehouse_sk#13]
+(17) CometBroadcastExchange
+Input [1]: [w_warehouse_sk#17]
+Arguments: [w_warehouse_sk#17]
-(17) CometBroadcastHashJoin
-Left output [6]: [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#9, i_class#10, i_category#11, i_product_name#12]
-Right output [1]: [w_warehouse_sk#13]
-Arguments: [inv_warehouse_sk#2], [w_warehouse_sk#13], Inner, BuildRight
+(18) CometBroadcastHashJoin
+Left output [6]: [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#13, i_class#14, i_category#15, i_product_name#16]
+Right output [1]: [w_warehouse_sk#17]
+Arguments: [inv_warehouse_sk#2], [w_warehouse_sk#17], Inner, BuildRight
-(18) CometProject
-Input [7]: [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#9, i_class#10, i_category#11, i_product_name#12, w_warehouse_sk#13]
-Arguments: [inv_quantity_on_hand#3, i_product_name#12, i_brand#9, i_class#10, i_category#11], [inv_quantity_on_hand#3, i_product_name#12, i_brand#9, i_class#10, i_category#11]
+(19) CometProject
+Input [7]: [inv_warehouse_sk#2, inv_quantity_on_hand#3, i_brand#13, i_class#14, i_category#15, i_product_name#16, w_warehouse_sk#17]
+Arguments: [inv_quantity_on_hand#3, i_product_name#16, i_brand#13, i_class#14, i_category#15], [inv_quantity_on_hand#3, i_product_name#16, i_brand#13, i_class#14, i_category#15]
-(19) CometExpand
-Input [5]: [inv_quantity_on_hand#3, i_product_name#12, i_brand#9, i_class#10, i_category#11]
-Arguments: [[inv_quantity_on_hand#3, i_product_name#12, i_brand#9, i_class#10, i_category#11, 0], [inv_quantity_on_hand#3, i_product_name#12, i_brand#9, i_class#10, null, 1], [inv_quantity_on_hand#3, i_product_name#12, i_brand#9, null, null, 3], [inv_quantity_on_hand#3, i_product_name#12, null, null, null, 7], [inv_quantity_on_hand#3, null, null, null, null, 15]], [inv_quantity_on_hand#3, i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18]
+(20) CometExpand
+Input [5]: [inv_quantity_on_hand#3, i_product_name#16, i_brand#13, i_class#14, i_category#15]
+Arguments: [[inv_quantity_on_hand#3, i_product_name#16, i_brand#13, i_class#14, i_category#15, 0], [inv_quantity_on_hand#3, i_product_name#16, i_brand#13, i_class#14, null, 1], [inv_quantity_on_hand#3, i_product_name#16, i_brand#13, null, null, 3], [inv_quantity_on_hand#3, i_product_name#16, null, null, null, 7], [inv_quantity_on_hand#3, null, null, null, null, 15]], [inv_quantity_on_hand#3, i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22]
-(20) CometHashAggregate
-Input [6]: [inv_quantity_on_hand#3, i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18]
-Keys [5]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18]
+(21) CometHashAggregate
+Input [6]: [inv_quantity_on_hand#3, i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22]
+Keys [5]: [i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22]
Functions [1]: [partial_avg(inv_quantity_on_hand#3)]
-(21) CometExchange
-Input [7]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, sum#19, count#20]
-Arguments: hashpartitioning(i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(22) CometExchange
+Input [7]: [i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22, sum#23, count#24]
+Arguments: hashpartitioning(i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(22) CometHashAggregate
-Input [7]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18, sum#19, count#20]
-Keys [5]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, spark_grouping_id#18]
+(23) CometHashAggregate
+Input [7]: [i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22, sum#23, count#24]
+Keys [5]: [i_product_name#18, i_brand#19, i_class#20, i_category#21, spark_grouping_id#22]
Functions [1]: [avg(inv_quantity_on_hand#3)]
-(23) CometTakeOrderedAndProject
-Input [5]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, qoh#21]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[qoh#21 ASC NULLS FIRST,i_product_name#14 ASC NULLS FIRST,i_brand#15 ASC NULLS FIRST,i_class#16 ASC NULLS FIRST,i_category#17 ASC NULLS FIRST], output=[i_product_name#14,i_brand#15,i_class#16,i_category#17,qoh#21]), [i_product_name#14, i_brand#15, i_class#16, i_category#17, qoh#21], 100, [qoh#21 ASC NULLS FIRST, i_product_name#14 ASC NULLS FIRST, i_brand#15 ASC NULLS FIRST, i_class#16 ASC NULLS FIRST, i_category#17 ASC NULLS FIRST], [i_ [...]
+(24) CometTakeOrderedAndProject
+Input [5]: [i_product_name#18, i_brand#19, i_class#20, i_category#21, qoh#25]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[qoh#25 ASC NULLS FIRST,i_product_name#18 ASC NULLS FIRST,i_brand#19 ASC NULLS FIRST,i_class#20 ASC NULLS FIRST,i_category#21 ASC NULLS FIRST], output=[i_product_name#18,i_brand#19,i_class#20,i_category#21,qoh#25]), [i_product_name#18, i_brand#19, i_class#20, i_category#21, qoh#25], 100, [qoh#25 ASC NULLS FIRST, i_product_name#18 ASC NULLS FIRST, i_brand#19 ASC NULLS FIRST, i_class#20 ASC NULLS FIRST, i_category#21 ASC NULLS FIRST], [i_ [...]
-(24) CometColumnarToRow [codegen id : 1]
-Input [5]: [i_product_name#14, i_brand#15, i_class#16, i_category#17, qoh#21]
+(25) CometColumnarToRow [codegen id : 1]
+Input [5]: [i_product_name#18, i_brand#19, i_class#20, i_category#21, qoh#25]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = inv_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (29)
-+- * CometColumnarToRow (28)
- +- CometProject (27)
- +- CometFilter (26)
- +- CometScan parquet spark_catalog.default.date_dim (25)
+BroadcastExchange (30)
++- * CometColumnarToRow (29)
+ +- CometProject (28)
+ +- CometFilter (27)
+ +- CometScan parquet spark_catalog.default.date_dim (26)
-(25) CometScan parquet spark_catalog.default.date_dim
+(26) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_month_seq#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_month_seq), GreaterThanOrEqual(d_month_seq,1200), LessThanOrEqual(d_month_seq,1211), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_month_seq:int>
-(26) CometFilter
+(27) CometFilter
Input [2]: [d_date_sk#6, d_month_seq#7]
Condition : (((isnotnull(d_month_seq#7) AND (d_month_seq#7 >= 1200)) AND (d_month_seq#7 <= 1211)) AND isnotnull(d_date_sk#6))
-(27) CometProject
+(28) CometProject
Input [2]: [d_date_sk#6, d_month_seq#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(28) CometColumnarToRow [codegen id : 1]
+(29) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(29) BroadcastExchange
+(30) BroadcastExchange
Input [1]: [d_date_sk#6]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt
index f9d4ce909..eefe4d665 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q22/simplified.txt
@@ -27,8 +27,9 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_month_seq]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
CometBroadcastExchange [i_item_sk,i_brand,i_class,i_category,i_product_name] #4
- CometFilter [i_item_sk,i_brand,i_class,i_category,i_product_name]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_brand,i_class,i_category,i_product_name]
+ CometProject [i_brand,i_class,i_category,i_product_name] [i_item_sk,i_brand,i_class,i_category,i_product_name]
+ CometFilter [i_item_sk,i_brand,i_class,i_category,i_product_name]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_brand,i_class,i_category,i_product_name]
CometBroadcastExchange [w_warehouse_sk] #5
CometFilter [w_warehouse_sk]
CometScan parquet spark_catalog.default.warehouse [w_warehouse_sk]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt
index 8ed54d7af..994066e37 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/explain.txt
@@ -1,14 +1,14 @@
== Physical Plan ==
-* CometColumnarToRow (87)
-+- CometTakeOrderedAndProject (86)
- +- CometUnion (85)
- :- CometHashAggregate (62)
- : +- CometExchange (61)
- : +- CometHashAggregate (60)
- : +- CometProject (59)
- : +- CometBroadcastHashJoin (58)
- : :- CometProject (53)
- : : +- CometBroadcastHashJoin (52)
+* CometColumnarToRow (88)
++- CometTakeOrderedAndProject (87)
+ +- CometUnion (86)
+ :- CometHashAggregate (63)
+ : +- CometExchange (62)
+ : +- CometHashAggregate (61)
+ : +- CometProject (60)
+ : +- CometBroadcastHashJoin (59)
+ : :- CometProject (54)
+ : : +- CometBroadcastHashJoin (53)
: : :- CometSortMergeJoin (40)
: : : :- CometSort (25)
: : : : +- CometExchange (24)
@@ -49,43 +49,44 @@
: : : +- CometBroadcastExchange (31)
: : : +- CometFilter (30)
: : : +- CometScan parquet spark_catalog.default.customer (29)
- : : +- CometBroadcastExchange (51)
- : : +- CometSortMergeJoin (50)
- : : :- CometSort (44)
- : : : +- CometExchange (43)
- : : : +- CometFilter (42)
- : : : +- CometScan parquet spark_catalog.default.customer (41)
- : : +- CometSort (49)
- : : +- CometProject (48)
- : : +- CometFilter (47)
- : : +- CometHashAggregate (46)
- : : +- ReusedExchange (45)
- : +- CometBroadcastExchange (57)
- : +- CometProject (56)
- : +- CometFilter (55)
- : +- CometScan parquet spark_catalog.default.date_dim (54)
- +- CometHashAggregate (84)
- +- CometExchange (83)
- +- CometHashAggregate (82)
- +- CometProject (81)
- +- CometBroadcastHashJoin (80)
- :- CometProject (78)
- : +- CometBroadcastHashJoin (77)
- : :- CometSortMergeJoin (75)
- : : :- CometSort (69)
- : : : +- CometExchange (68)
- : : : +- CometProject (67)
- : : : +- CometBroadcastHashJoin (66)
- : : : :- CometFilter (64)
- : : : : +- CometScan parquet spark_catalog.default.web_sales (63)
- : : : +- ReusedExchange (65)
- : : +- CometSort (74)
- : : +- CometProject (73)
- : : +- CometFilter (72)
- : : +- CometHashAggregate (71)
- : : +- ReusedExchange (70)
- : +- ReusedExchange (76)
- +- ReusedExchange (79)
+ : : +- CometBroadcastExchange (52)
+ : : +- CometProject (51)
+ : : +- CometSortMergeJoin (50)
+ : : :- CometSort (44)
+ : : : +- CometExchange (43)
+ : : : +- CometFilter (42)
+ : : : +- CometScan parquet spark_catalog.default.customer (41)
+ : : +- CometSort (49)
+ : : +- CometProject (48)
+ : : +- CometFilter (47)
+ : : +- CometHashAggregate (46)
+ : : +- ReusedExchange (45)
+ : +- CometBroadcastExchange (58)
+ : +- CometProject (57)
+ : +- CometFilter (56)
+ : +- CometScan parquet spark_catalog.default.date_dim (55)
+ +- CometHashAggregate (85)
+ +- CometExchange (84)
+ +- CometHashAggregate (83)
+ +- CometProject (82)
+ +- CometBroadcastHashJoin (81)
+ :- CometProject (79)
+ : +- CometBroadcastHashJoin (78)
+ : :- CometSortMergeJoin (76)
+ : : :- CometSort (70)
+ : : : +- CometExchange (69)
+ : : : +- CometProject (68)
+ : : : +- CometBroadcastHashJoin (67)
+ : : : :- CometFilter (65)
+ : : : : +- CometScan parquet spark_catalog.default.web_sales (64)
+ : : : +- ReusedExchange (66)
+ : : +- CometSort (75)
+ : : +- CometProject (74)
+ : : +- CometFilter (73)
+ : : +- CometHashAggregate (72)
+ : : +- ReusedExchange (71)
+ : +- ReusedExchange (77)
+ +- ReusedExchange (80)
(1) CometScan parquet spark_catalog.default.catalog_sales
@@ -321,367 +322,371 @@ Left output [3]: [c_customer_sk#29, c_first_name#30, c_last_name#31]
Right output [1]: [c_customer_sk#23]
Arguments: [c_customer_sk#29], [c_customer_sk#23], LeftSemi
-(51) CometBroadcastExchange
+(51) CometProject
Input [3]: [c_customer_sk#29, c_first_name#30, c_last_name#31]
-Arguments: [c_customer_sk#29, c_first_name#30, c_last_name#31]
+Arguments: [c_customer_sk#29, c_first_name#32, c_last_name#33], [c_customer_sk#29, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#30, 20, true, false, true) AS c_first_name#32, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_last_name#31, 30, true, false, true) AS c_last_name#33]
-(52) CometBroadcastHashJoin
+(52) CometBroadcastExchange
+Input [3]: [c_customer_sk#29, c_first_name#32, c_last_name#33]
+Arguments: [c_customer_sk#29, c_first_name#32, c_last_name#33]
+
+(53) CometBroadcastHashJoin
Left output [4]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5]
-Right output [3]: [c_customer_sk#29, c_first_name#30, c_last_name#31]
+Right output [3]: [c_customer_sk#29, c_first_name#32, c_last_name#33]
Arguments: [cs_bill_customer_sk#1], [c_customer_sk#29], Inner, BuildRight
-(53) CometProject
-Input [7]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_customer_sk#29, c_first_name#30, c_last_name#31]
-Arguments: [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#30, c_last_name#31], [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#30, c_last_name#31]
+(54) CometProject
+Input [7]: [cs_bill_customer_sk#1, cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_customer_sk#29, c_first_name#32, c_last_name#33]
+Arguments: [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#32, c_last_name#33], [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#32, c_last_name#33]
-(54) CometScan parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#32, d_year#33, d_moy#34]
+(55) CometScan parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#34, d_year#35, d_moy#36]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_moy), EqualTo(d_year,2000), EqualTo(d_moy,2), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(55) CometFilter
-Input [3]: [d_date_sk#32, d_year#33, d_moy#34]
-Condition : ((((isnotnull(d_year#33) AND isnotnull(d_moy#34)) AND (d_year#33 = 2000)) AND (d_moy#34 = 2)) AND isnotnull(d_date_sk#32))
+(56) CometFilter
+Input [3]: [d_date_sk#34, d_year#35, d_moy#36]
+Condition : ((((isnotnull(d_year#35) AND isnotnull(d_moy#36)) AND (d_year#35 = 2000)) AND (d_moy#36 = 2)) AND isnotnull(d_date_sk#34))
-(56) CometProject
-Input [3]: [d_date_sk#32, d_year#33, d_moy#34]
-Arguments: [d_date_sk#32], [d_date_sk#32]
+(57) CometProject
+Input [3]: [d_date_sk#34, d_year#35, d_moy#36]
+Arguments: [d_date_sk#34], [d_date_sk#34]
-(57) CometBroadcastExchange
-Input [1]: [d_date_sk#32]
-Arguments: [d_date_sk#32]
+(58) CometBroadcastExchange
+Input [1]: [d_date_sk#34]
+Arguments: [d_date_sk#34]
-(58) CometBroadcastHashJoin
-Left output [5]: [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#30, c_last_name#31]
-Right output [1]: [d_date_sk#32]
-Arguments: [cs_sold_date_sk#5], [d_date_sk#32], Inner, BuildRight
+(59) CometBroadcastHashJoin
+Left output [5]: [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#32, c_last_name#33]
+Right output [1]: [d_date_sk#34]
+Arguments: [cs_sold_date_sk#5], [d_date_sk#34], Inner, BuildRight
-(59) CometProject
-Input [6]: [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#30, c_last_name#31, d_date_sk#32]
-Arguments: [cs_quantity#3, cs_list_price#4, c_first_name#30, c_last_name#31], [cs_quantity#3, cs_list_price#4, c_first_name#30, c_last_name#31]
+(60) CometProject
+Input [6]: [cs_quantity#3, cs_list_price#4, cs_sold_date_sk#5, c_first_name#32, c_last_name#33, d_date_sk#34]
+Arguments: [cs_quantity#3, cs_list_price#4, c_first_name#32, c_last_name#33], [cs_quantity#3, cs_list_price#4, c_first_name#32, c_last_name#33]
-(60) CometHashAggregate
-Input [4]: [cs_quantity#3, cs_list_price#4, c_first_name#30, c_last_name#31]
-Keys [2]: [c_last_name#31, c_first_name#30]
+(61) CometHashAggregate
+Input [4]: [cs_quantity#3, cs_list_price#4, c_first_name#32, c_last_name#33]
+Keys [2]: [c_last_name#33, c_first_name#32]
Functions [1]: [partial_sum((cast(cs_quantity#3 as decimal(10,0)) * cs_list_price#4))]
-(61) CometExchange
-Input [4]: [c_last_name#31, c_first_name#30, sum#35, isEmpty#36]
-Arguments: hashpartitioning(c_last_name#31, c_first_name#30, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
+(62) CometExchange
+Input [4]: [c_last_name#33, c_first_name#32, sum#37, isEmpty#38]
+Arguments: hashpartitioning(c_last_name#33, c_first_name#32, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=5]
-(62) CometHashAggregate
-Input [4]: [c_last_name#31, c_first_name#30, sum#35, isEmpty#36]
-Keys [2]: [c_last_name#31, c_first_name#30]
+(63) CometHashAggregate
+Input [4]: [c_last_name#33, c_first_name#32, sum#37, isEmpty#38]
+Keys [2]: [c_last_name#33, c_first_name#32]
Functions [1]: [sum((cast(cs_quantity#3 as decimal(10,0)) * cs_list_price#4))]
-(63) CometScan parquet spark_catalog.default.web_sales
-Output [5]: [ws_item_sk#37, ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
+(64) CometScan parquet spark_catalog.default.web_sales
+Output [5]: [ws_item_sk#39, ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#41), dynamicpruningexpression(ws_sold_date_sk#41 IN dynamicpruning#42)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#43), dynamicpruningexpression(ws_sold_date_sk#43 IN dynamicpruning#44)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_item_sk:int,ws_bill_customer_sk:int,ws_quantity:int,ws_list_price:decimal(7,2)>
-(64) CometFilter
-Input [5]: [ws_item_sk#37, ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Condition : isnotnull(ws_bill_customer_sk#38)
+(65) CometFilter
+Input [5]: [ws_item_sk#39, ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Condition : isnotnull(ws_bill_customer_sk#40)
-(65) ReusedExchange [Reuses operator id: 21]
-Output [1]: [item_sk#43]
+(66) ReusedExchange [Reuses operator id: 21]
+Output [1]: [item_sk#45]
-(66) CometBroadcastHashJoin
-Left output [5]: [ws_item_sk#37, ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Right output [1]: [item_sk#43]
-Arguments: [ws_item_sk#37], [item_sk#43], LeftSemi, BuildRight
+(67) CometBroadcastHashJoin
+Left output [5]: [ws_item_sk#39, ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Right output [1]: [item_sk#45]
+Arguments: [ws_item_sk#39], [item_sk#45], LeftSemi, BuildRight
-(67) CometProject
-Input [5]: [ws_item_sk#37, ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Arguments: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41], [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
+(68) CometProject
+Input [5]: [ws_item_sk#39, ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Arguments: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43], [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
-(68) CometExchange
-Input [4]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Arguments: hashpartitioning(ws_bill_customer_sk#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
+(69) CometExchange
+Input [4]: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Arguments: hashpartitioning(ws_bill_customer_sk#40, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=6]
-(69) CometSort
-Input [4]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Arguments: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41], [ws_bill_customer_sk#38 ASC NULLS FIRST]
+(70) CometSort
+Input [4]: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Arguments: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43], [ws_bill_customer_sk#40 ASC NULLS FIRST]
-(70) ReusedExchange [Reuses operator id: 35]
-Output [3]: [c_customer_sk#44, sum#45, isEmpty#46]
+(71) ReusedExchange [Reuses operator id: 35]
+Output [3]: [c_customer_sk#46, sum#47, isEmpty#48]
-(71) CometHashAggregate
-Input [3]: [c_customer_sk#44, sum#45, isEmpty#46]
-Keys [1]: [c_customer_sk#44]
-Functions [1]: [sum((cast(ss_quantity#47 as decimal(10,0)) * ss_sales_price#48))]
+(72) CometHashAggregate
+Input [3]: [c_customer_sk#46, sum#47, isEmpty#48]
+Keys [1]: [c_customer_sk#46]
+Functions [1]: [sum((cast(ss_quantity#49 as decimal(10,0)) * ss_sales_price#50))]
-(72) CometFilter
-Input [2]: [c_customer_sk#44, ssales#49]
-Condition : (isnotnull(ssales#49) AND (cast(ssales#49 as decimal(38,8)) > (0.500000 * ReusedSubquery Subquery scalar-subquery#27, [id=#28])))
+(73) CometFilter
+Input [2]: [c_customer_sk#46, ssales#51]
+Condition : (isnotnull(ssales#51) AND (cast(ssales#51 as decimal(38,8)) > (0.500000 * ReusedSubquery Subquery scalar-subquery#27, [id=#28])))
-(73) CometProject
-Input [2]: [c_customer_sk#44, ssales#49]
-Arguments: [c_customer_sk#44], [c_customer_sk#44]
+(74) CometProject
+Input [2]: [c_customer_sk#46, ssales#51]
+Arguments: [c_customer_sk#46], [c_customer_sk#46]
-(74) CometSort
-Input [1]: [c_customer_sk#44]
-Arguments: [c_customer_sk#44], [c_customer_sk#44 ASC NULLS FIRST]
+(75) CometSort
+Input [1]: [c_customer_sk#46]
+Arguments: [c_customer_sk#46], [c_customer_sk#46 ASC NULLS FIRST]
-(75) CometSortMergeJoin
-Left output [4]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Right output [1]: [c_customer_sk#44]
-Arguments: [ws_bill_customer_sk#38], [c_customer_sk#44], LeftSemi
+(76) CometSortMergeJoin
+Left output [4]: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Right output [1]: [c_customer_sk#46]
+Arguments: [ws_bill_customer_sk#40], [c_customer_sk#46], LeftSemi
-(76) ReusedExchange [Reuses operator id: 51]
-Output [3]: [c_customer_sk#50, c_first_name#51, c_last_name#52]
+(77) ReusedExchange [Reuses operator id: 52]
+Output [3]: [c_customer_sk#52, c_first_name#53, c_last_name#54]
-(77) CometBroadcastHashJoin
-Left output [4]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41]
-Right output [3]: [c_customer_sk#50, c_first_name#51, c_last_name#52]
-Arguments: [ws_bill_customer_sk#38], [c_customer_sk#50], Inner, BuildRight
+(78) CometBroadcastHashJoin
+Left output [4]: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43]
+Right output [3]: [c_customer_sk#52, c_first_name#53, c_last_name#54]
+Arguments: [ws_bill_customer_sk#40], [c_customer_sk#52], Inner, BuildRight
-(78) CometProject
-Input [7]: [ws_bill_customer_sk#38, ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41, c_customer_sk#50, c_first_name#51, c_last_name#52]
-Arguments: [ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41, c_first_name#51, c_last_name#52], [ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41, c_first_name#51, c_last_name#52]
+(79) CometProject
+Input [7]: [ws_bill_customer_sk#40, ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43, c_customer_sk#52, c_first_name#53, c_last_name#54]
+Arguments: [ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43, c_first_name#53, c_last_name#54], [ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43, c_first_name#53, c_last_name#54]
-(79) ReusedExchange [Reuses operator id: 57]
-Output [1]: [d_date_sk#53]
+(80) ReusedExchange [Reuses operator id: 58]
+Output [1]: [d_date_sk#55]
-(80) CometBroadcastHashJoin
-Left output [5]: [ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41, c_first_name#51, c_last_name#52]
-Right output [1]: [d_date_sk#53]
-Arguments: [ws_sold_date_sk#41], [d_date_sk#53], Inner, BuildRight
+(81) CometBroadcastHashJoin
+Left output [5]: [ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43, c_first_name#53, c_last_name#54]
+Right output [1]: [d_date_sk#55]
+Arguments: [ws_sold_date_sk#43], [d_date_sk#55], Inner, BuildRight
-(81) CometProject
-Input [6]: [ws_quantity#39, ws_list_price#40, ws_sold_date_sk#41, c_first_name#51, c_last_name#52, d_date_sk#53]
-Arguments: [ws_quantity#39, ws_list_price#40, c_first_name#51, c_last_name#52], [ws_quantity#39, ws_list_price#40, c_first_name#51, c_last_name#52]
+(82) CometProject
+Input [6]: [ws_quantity#41, ws_list_price#42, ws_sold_date_sk#43, c_first_name#53, c_last_name#54, d_date_sk#55]
+Arguments: [ws_quantity#41, ws_list_price#42, c_first_name#53, c_last_name#54], [ws_quantity#41, ws_list_price#42, c_first_name#53, c_last_name#54]
-(82) CometHashAggregate
-Input [4]: [ws_quantity#39, ws_list_price#40, c_first_name#51, c_last_name#52]
-Keys [2]: [c_last_name#52, c_first_name#51]
-Functions [1]: [partial_sum((cast(ws_quantity#39 as decimal(10,0)) * ws_list_price#40))]
+(83) CometHashAggregate
+Input [4]: [ws_quantity#41, ws_list_price#42, c_first_name#53, c_last_name#54]
+Keys [2]: [c_last_name#54, c_first_name#53]
+Functions [1]: [partial_sum((cast(ws_quantity#41 as decimal(10,0)) * ws_list_price#42))]
-(83) CometExchange
-Input [4]: [c_last_name#52, c_first_name#51, sum#54, isEmpty#55]
-Arguments: hashpartitioning(c_last_name#52, c_first_name#51, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=7]
+(84) CometExchange
+Input [4]: [c_last_name#54, c_first_name#53, sum#56, isEmpty#57]
+Arguments: hashpartitioning(c_last_name#54, c_first_name#53, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=7]
-(84) CometHashAggregate
-Input [4]: [c_last_name#52, c_first_name#51, sum#54, isEmpty#55]
-Keys [2]: [c_last_name#52, c_first_name#51]
-Functions [1]: [sum((cast(ws_quantity#39 as decimal(10,0)) * ws_list_price#40))]
+(85) CometHashAggregate
+Input [4]: [c_last_name#54, c_first_name#53, sum#56, isEmpty#57]
+Keys [2]: [c_last_name#54, c_first_name#53]
+Functions [1]: [sum((cast(ws_quantity#41 as decimal(10,0)) * ws_list_price#42))]
-(85) CometUnion
-Child 0 Input [3]: [c_last_name#31, c_first_name#30, sales#56]
-Child 1 Input [3]: [c_last_name#52, c_first_name#51, sales#57]
+(86) CometUnion
+Child 0 Input [3]: [c_last_name#33, c_first_name#32, sales#58]
+Child 1 Input [3]: [c_last_name#54, c_first_name#53, sales#59]
-(86) CometTakeOrderedAndProject
-Input [3]: [c_last_name#31, c_first_name#30, sales#56]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_last_name#31 ASC NULLS FIRST,c_first_name#30 ASC NULLS FIRST,sales#56 ASC NULLS FIRST], output=[c_last_name#31,c_first_name#30,sales#56]), [c_last_name#31, c_first_name#30, sales#56], 100, [c_last_name#31 ASC NULLS FIRST, c_first_name#30 ASC NULLS FIRST, sales#56 ASC NULLS FIRST], [c_last_name#31, c_first_name#30, sales#56]
+(87) CometTakeOrderedAndProject
+Input [3]: [c_last_name#33, c_first_name#32, sales#58]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_last_name#33 ASC NULLS FIRST,c_first_name#32 ASC NULLS FIRST,sales#58 ASC NULLS FIRST], output=[c_last_name#33,c_first_name#32,sales#58]), [c_last_name#33, c_first_name#32, sales#58], 100, [c_last_name#33 ASC NULLS FIRST, c_first_name#32 ASC NULLS FIRST, sales#58 ASC NULLS FIRST], [c_last_name#33, c_first_name#32, sales#58]
-(87) CometColumnarToRow [codegen id : 1]
-Input [3]: [c_last_name#31, c_first_name#30, sales#56]
+(88) CometColumnarToRow [codegen id : 1]
+Input [3]: [c_last_name#33, c_first_name#32, sales#58]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#5 IN dynamicpruning#6
-BroadcastExchange (92)
-+- * CometColumnarToRow (91)
- +- CometProject (90)
- +- CometFilter (89)
- +- CometScan parquet spark_catalog.default.date_dim (88)
+BroadcastExchange (93)
++- * CometColumnarToRow (92)
+ +- CometProject (91)
+ +- CometFilter (90)
+ +- CometScan parquet spark_catalog.default.date_dim (89)
-(88) CometScan parquet spark_catalog.default.date_dim
-Output [3]: [d_date_sk#32, d_year#33, d_moy#34]
+(89) CometScan parquet spark_catalog.default.date_dim
+Output [3]: [d_date_sk#34, d_year#35, d_moy#36]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_moy), EqualTo(d_year,2000), EqualTo(d_moy,2), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(89) CometFilter
-Input [3]: [d_date_sk#32, d_year#33, d_moy#34]
-Condition : ((((isnotnull(d_year#33) AND isnotnull(d_moy#34)) AND (d_year#33 = 2000)) AND (d_moy#34 = 2)) AND isnotnull(d_date_sk#32))
+(90) CometFilter
+Input [3]: [d_date_sk#34, d_year#35, d_moy#36]
+Condition : ((((isnotnull(d_year#35) AND isnotnull(d_moy#36)) AND (d_year#35 = 2000)) AND (d_moy#36 = 2)) AND isnotnull(d_date_sk#34))
-(90) CometProject
-Input [3]: [d_date_sk#32, d_year#33, d_moy#34]
-Arguments: [d_date_sk#32], [d_date_sk#32]
+(91) CometProject
+Input [3]: [d_date_sk#34, d_year#35, d_moy#36]
+Arguments: [d_date_sk#34], [d_date_sk#34]
-(91) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#32]
+(92) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#34]
-(92) BroadcastExchange
-Input [1]: [d_date_sk#32]
+(93) BroadcastExchange
+Input [1]: [d_date_sk#34]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=8]
Subquery:2 Hosting operator id = 3 Hosting Expression = ss_sold_date_sk#8 IN dynamicpruning#9
-BroadcastExchange (97)
-+- * CometColumnarToRow (96)
- +- CometProject (95)
- +- CometFilter (94)
- +- CometScan parquet spark_catalog.default.date_dim (93)
+BroadcastExchange (98)
++- * CometColumnarToRow (97)
+ +- CometProject (96)
+ +- CometFilter (95)
+ +- CometScan parquet spark_catalog.default.date_dim (94)
-(93) CometScan parquet spark_catalog.default.date_dim
+(94) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#10, d_date#11, d_year#12]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [In(d_year, [2000,2001,2002,2003]), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date,d_year:int>
-(94) CometFilter
+(95) CometFilter
Input [3]: [d_date_sk#10, d_date#11, d_year#12]
Condition : (d_year#12 IN (2000,2001,2002,2003) AND isnotnull(d_date_sk#10))
-(95) CometProject
+(96) CometProject
Input [3]: [d_date_sk#10, d_date#11, d_year#12]
Arguments: [d_date_sk#10, d_date#11], [d_date_sk#10, d_date#11]
-(96) CometColumnarToRow [codegen id : 1]
+(97) CometColumnarToRow [codegen id : 1]
Input [2]: [d_date_sk#10, d_date#11]
-(97) BroadcastExchange
+(98) BroadcastExchange
Input [2]: [d_date_sk#10, d_date#11]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=9]
Subquery:3 Hosting operator id = 37 Hosting Expression = Subquery scalar-subquery#27, [id=#28]
-* CometColumnarToRow (115)
-+- CometHashAggregate (114)
- +- CometExchange (113)
- +- CometHashAggregate (112)
- +- CometHashAggregate (111)
- +- CometExchange (110)
- +- CometHashAggregate (109)
- +- CometProject (108)
- +- CometBroadcastHashJoin (107)
- :- CometProject (102)
- : +- CometBroadcastHashJoin (101)
- : :- CometFilter (99)
- : : +- CometScan parquet spark_catalog.default.store_sales (98)
- : +- ReusedExchange (100)
- +- CometBroadcastExchange (106)
- +- CometProject (105)
- +- CometFilter (104)
- +- CometScan parquet spark_catalog.default.date_dim (103)
-
-
-(98) CometScan parquet spark_catalog.default.store_sales
-Output [4]: [ss_customer_sk#58, ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61]
+* CometColumnarToRow (116)
++- CometHashAggregate (115)
+ +- CometExchange (114)
+ +- CometHashAggregate (113)
+ +- CometHashAggregate (112)
+ +- CometExchange (111)
+ +- CometHashAggregate (110)
+ +- CometProject (109)
+ +- CometBroadcastHashJoin (108)
+ :- CometProject (103)
+ : +- CometBroadcastHashJoin (102)
+ : :- CometFilter (100)
+ : : +- CometScan parquet spark_catalog.default.store_sales (99)
+ : +- ReusedExchange (101)
+ +- CometBroadcastExchange (107)
+ +- CometProject (106)
+ +- CometFilter (105)
+ +- CometScan parquet spark_catalog.default.date_dim (104)
+
+
+(99) CometScan parquet spark_catalog.default.store_sales
+Output [4]: [ss_customer_sk#60, ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#61), dynamicpruningexpression(ss_sold_date_sk#61 IN dynamicpruning#62)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#63), dynamicpruningexpression(ss_sold_date_sk#63 IN dynamicpruning#64)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_quantity:int,ss_sales_price:decimal(7,2)>
-(99) CometFilter
-Input [4]: [ss_customer_sk#58, ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61]
-Condition : isnotnull(ss_customer_sk#58)
+(100) CometFilter
+Input [4]: [ss_customer_sk#60, ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63]
+Condition : isnotnull(ss_customer_sk#60)
-(100) ReusedExchange [Reuses operator id: 31]
-Output [1]: [c_customer_sk#63]
+(101) ReusedExchange [Reuses operator id: 31]
+Output [1]: [c_customer_sk#65]
-(101) CometBroadcastHashJoin
-Left output [4]: [ss_customer_sk#58, ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61]
-Right output [1]: [c_customer_sk#63]
-Arguments: [ss_customer_sk#58], [c_customer_sk#63], Inner, BuildRight
+(102) CometBroadcastHashJoin
+Left output [4]: [ss_customer_sk#60, ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63]
+Right output [1]: [c_customer_sk#65]
+Arguments: [ss_customer_sk#60], [c_customer_sk#65], Inner, BuildRight
-(102) CometProject
-Input [5]: [ss_customer_sk#58, ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61, c_customer_sk#63]
-Arguments: [ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61, c_customer_sk#63], [ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61, c_customer_sk#63]
+(103) CometProject
+Input [5]: [ss_customer_sk#60, ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63, c_customer_sk#65]
+Arguments: [ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63, c_customer_sk#65], [ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63, c_customer_sk#65]
-(103) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#64, d_year#65]
+(104) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#66, d_year#67]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [In(d_year, [2000,2001,2002,2003]), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(104) CometFilter
-Input [2]: [d_date_sk#64, d_year#65]
-Condition : (d_year#65 IN (2000,2001,2002,2003) AND isnotnull(d_date_sk#64))
-
-(105) CometProject
-Input [2]: [d_date_sk#64, d_year#65]
-Arguments: [d_date_sk#64], [d_date_sk#64]
+(105) CometFilter
+Input [2]: [d_date_sk#66, d_year#67]
+Condition : (d_year#67 IN (2000,2001,2002,2003) AND isnotnull(d_date_sk#66))
-(106) CometBroadcastExchange
-Input [1]: [d_date_sk#64]
-Arguments: [d_date_sk#64]
+(106) CometProject
+Input [2]: [d_date_sk#66, d_year#67]
+Arguments: [d_date_sk#66], [d_date_sk#66]
-(107) CometBroadcastHashJoin
-Left output [4]: [ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61, c_customer_sk#63]
-Right output [1]: [d_date_sk#64]
-Arguments: [ss_sold_date_sk#61], [d_date_sk#64], Inner, BuildRight
+(107) CometBroadcastExchange
+Input [1]: [d_date_sk#66]
+Arguments: [d_date_sk#66]
-(108) CometProject
-Input [5]: [ss_quantity#59, ss_sales_price#60, ss_sold_date_sk#61, c_customer_sk#63, d_date_sk#64]
-Arguments: [ss_quantity#59, ss_sales_price#60, c_customer_sk#63], [ss_quantity#59, ss_sales_price#60, c_customer_sk#63]
+(108) CometBroadcastHashJoin
+Left output [4]: [ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63, c_customer_sk#65]
+Right output [1]: [d_date_sk#66]
+Arguments: [ss_sold_date_sk#63], [d_date_sk#66], Inner, BuildRight
-(109) CometHashAggregate
-Input [3]: [ss_quantity#59, ss_sales_price#60, c_customer_sk#63]
-Keys [1]: [c_customer_sk#63]
-Functions [1]: [partial_sum((cast(ss_quantity#59 as decimal(10,0)) * ss_sales_price#60))]
+(109) CometProject
+Input [5]: [ss_quantity#61, ss_sales_price#62, ss_sold_date_sk#63, c_customer_sk#65, d_date_sk#66]
+Arguments: [ss_quantity#61, ss_sales_price#62, c_customer_sk#65], [ss_quantity#61, ss_sales_price#62, c_customer_sk#65]
-(110) CometExchange
-Input [3]: [c_customer_sk#63, sum#66, isEmpty#67]
-Arguments: hashpartitioning(c_customer_sk#63, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=10]
+(110) CometHashAggregate
+Input [3]: [ss_quantity#61, ss_sales_price#62, c_customer_sk#65]
+Keys [1]: [c_customer_sk#65]
+Functions [1]: [partial_sum((cast(ss_quantity#61 as decimal(10,0)) * ss_sales_price#62))]
-(111) CometHashAggregate
-Input [3]: [c_customer_sk#63, sum#66, isEmpty#67]
-Keys [1]: [c_customer_sk#63]
-Functions [1]: [sum((cast(ss_quantity#59 as decimal(10,0)) * ss_sales_price#60))]
+(111) CometExchange
+Input [3]: [c_customer_sk#65, sum#68, isEmpty#69]
+Arguments: hashpartitioning(c_customer_sk#65, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=10]
(112) CometHashAggregate
-Input [1]: [csales#68]
+Input [3]: [c_customer_sk#65, sum#68, isEmpty#69]
+Keys [1]: [c_customer_sk#65]
+Functions [1]: [sum((cast(ss_quantity#61 as decimal(10,0)) * ss_sales_price#62))]
+
+(113) CometHashAggregate
+Input [1]: [csales#70]
Keys: []
-Functions [1]: [partial_max(csales#68)]
+Functions [1]: [partial_max(csales#70)]
-(113) CometExchange
-Input [1]: [max#69]
+(114) CometExchange
+Input [1]: [max#71]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=11]
-(114) CometHashAggregate
-Input [1]: [max#69]
+(115) CometHashAggregate
+Input [1]: [max#71]
Keys: []
-Functions [1]: [max(csales#68)]
+Functions [1]: [max(csales#70)]
-(115) CometColumnarToRow [codegen id : 1]
-Input [1]: [tpcds_cmax#70]
+(116) CometColumnarToRow [codegen id : 1]
+Input [1]: [tpcds_cmax#72]
-Subquery:4 Hosting operator id = 98 Hosting Expression = ss_sold_date_sk#61 IN dynamicpruning#62
-BroadcastExchange (120)
-+- * CometColumnarToRow (119)
- +- CometProject (118)
- +- CometFilter (117)
- +- CometScan parquet spark_catalog.default.date_dim (116)
+Subquery:4 Hosting operator id = 99 Hosting Expression = ss_sold_date_sk#63 IN dynamicpruning#64
+BroadcastExchange (121)
++- * CometColumnarToRow (120)
+ +- CometProject (119)
+ +- CometFilter (118)
+ +- CometScan parquet spark_catalog.default.date_dim (117)
-(116) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#64, d_year#65]
+(117) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#66, d_year#67]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [In(d_year, [2000,2001,2002,2003]), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(117) CometFilter
-Input [2]: [d_date_sk#64, d_year#65]
-Condition : (d_year#65 IN (2000,2001,2002,2003) AND isnotnull(d_date_sk#64))
+(118) CometFilter
+Input [2]: [d_date_sk#66, d_year#67]
+Condition : (d_year#67 IN (2000,2001,2002,2003) AND isnotnull(d_date_sk#66))
-(118) CometProject
-Input [2]: [d_date_sk#64, d_year#65]
-Arguments: [d_date_sk#64], [d_date_sk#64]
+(119) CometProject
+Input [2]: [d_date_sk#66, d_year#67]
+Arguments: [d_date_sk#66], [d_date_sk#66]
-(119) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#64]
+(120) CometColumnarToRow [codegen id : 1]
+Input [1]: [d_date_sk#66]
-(120) BroadcastExchange
-Input [1]: [d_date_sk#64]
+(121) BroadcastExchange
+Input [1]: [d_date_sk#66]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=12]
Subquery:5 Hosting operator id = 47 Hosting Expression = ReusedSubquery Subquery scalar-subquery#27, [id=#28]
-Subquery:6 Hosting operator id = 63 Hosting Expression = ws_sold_date_sk#41 IN dynamicpruning#6
+Subquery:6 Hosting operator id = 64 Hosting Expression = ws_sold_date_sk#43 IN dynamicpruning#6
-Subquery:7 Hosting operator id = 72 Hosting Expression = ReusedSubquery Subquery scalar-subquery#27, [id=#28]
+Subquery:7 Hosting operator id = 73 Hosting Expression = ReusedSubquery Subquery scalar-subquery#27, [id=#28]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt
index 9493bffd2..feed92036 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q23b/simplified.txt
@@ -96,17 +96,18 @@ WholeStageCodegen (1)
CometFilter [c_customer_sk]
CometScan parquet spark_catalog.default.customer [c_customer_sk]
CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name] #15
- CometSortMergeJoin [c_customer_sk,c_first_name,c_last_name,c_customer_sk]
- CometSort [c_customer_sk,c_first_name,c_last_name]
- CometExchange [c_customer_sk] #16
- CometFilter [c_customer_sk,c_first_name,c_last_name]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
- CometSort [c_customer_sk]
- CometProject [c_customer_sk]
- CometFilter [c_customer_sk,ssales]
- ReusedSubquery [tpcds_cmax] #3
- CometHashAggregate [c_customer_sk,ssales,sum,isEmpty,sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price))]
- ReusedExchange [c_customer_sk,sum,isEmpty] #9
+ CometProject [c_first_name,c_last_name] [c_customer_sk,c_first_name,c_last_name]
+ CometSortMergeJoin [c_customer_sk,c_first_name,c_last_name,c_customer_sk]
+ CometSort [c_customer_sk,c_first_name,c_last_name]
+ CometExchange [c_customer_sk] #16
+ CometFilter [c_customer_sk,c_first_name,c_last_name]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
+ CometSort [c_customer_sk]
+ CometProject [c_customer_sk]
+ CometFilter [c_customer_sk,ssales]
+ ReusedSubquery [tpcds_cmax] #3
+ CometHashAggregate [c_customer_sk,ssales,sum,isEmpty,sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price))]
+ ReusedExchange [c_customer_sk,sum,isEmpty] #9
CometBroadcastExchange [d_date_sk] #17
CometProject [d_date_sk]
CometFilter [d_date_sk,d_year,d_moy]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt
index 684266874..f056d3083 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/explain.txt
@@ -1,20 +1,20 @@
== Physical Plan ==
-* Filter (44)
-+- * HashAggregate (43)
- +- * CometColumnarToRow (42)
- +- CometColumnarExchange (41)
- +- * HashAggregate (40)
- +- * HashAggregate (39)
- +- * CometColumnarToRow (38)
- +- CometColumnarExchange (37)
- +- * HashAggregate (36)
- +- * Project (35)
- +- * BroadcastHashJoin Inner BuildRight (34)
- :- * CometColumnarToRow (29)
- : +- CometProject (28)
- : +- CometBroadcastHashJoin (27)
- : :- CometProject (23)
- : : +- CometBroadcastHashJoin (22)
+* Filter (47)
++- * HashAggregate (46)
+ +- * CometColumnarToRow (45)
+ +- CometColumnarExchange (44)
+ +- * HashAggregate (43)
+ +- * HashAggregate (42)
+ +- * CometColumnarToRow (41)
+ +- CometColumnarExchange (40)
+ +- * HashAggregate (39)
+ +- * Project (38)
+ +- * BroadcastHashJoin Inner BuildRight (37)
+ :- * CometColumnarToRow (31)
+ : +- CometProject (30)
+ : +- CometBroadcastHashJoin (29)
+ : :- CometProject (24)
+ : : +- CometBroadcastHashJoin (23)
: : :- CometProject (18)
: : : +- CometBroadcastHashJoin (17)
: : : :- CometProject (12)
@@ -33,16 +33,19 @@
: : : +- CometProject (15)
: : : +- CometFilter (14)
: : : +- CometScan parquet spark_catalog.default.store (13)
- : : +- CometBroadcastExchange (21)
- : : +- CometFilter (20)
- : : +- CometScan parquet spark_catalog.default.item (19)
- : +- CometBroadcastExchange (26)
- : +- CometFilter (25)
- : +- CometScan parquet spark_catalog.default.customer (24)
- +- BroadcastExchange (33)
- +- * CometColumnarToRow (32)
- +- CometFilter (31)
- +- CometScan parquet spark_catalog.default.customer_address (30)
+ : : +- CometBroadcastExchange (22)
+ : : +- CometProject (21)
+ : : +- CometFilter (20)
+ : : +- CometScan parquet spark_catalog.default.item (19)
+ : +- CometBroadcastExchange (28)
+ : +- CometProject (27)
+ : +- CometFilter (26)
+ : +- CometScan parquet spark_catalog.default.customer (25)
+ +- BroadcastExchange (36)
+ +- * CometColumnarToRow (35)
+ +- CometProject (34)
+ +- CometFilter (33)
+ +- CometScan parquet spark_catalog.default.customer_address (32)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -104,316 +107,333 @@ Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5], [ss_i
Output [5]: [s_store_sk#10, s_store_name#11, s_market_id#12, s_state#13, s_zip#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
-PushedFilters: [IsNotNull(s_market_id), EqualTo(s_market_id,8), IsNotNull(s_store_sk), IsNotNull(s_zip)]
+PushedFilters: [IsNotNull(s_market_id), EqualTo(s_market_id,8), IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_store_name:string,s_market_id:int,s_state:string,s_zip:string>
(14) CometFilter
Input [5]: [s_store_sk#10, s_store_name#11, s_market_id#12, s_state#13, s_zip#14]
-Condition : (((isnotnull(s_market_id#12) AND (s_market_id#12 = 8)) AND isnotnull(s_store_sk#10)) AND isnotnull(s_zip#14))
+Condition : (((isnotnull(s_market_id#12) AND (s_market_id#12 = 8)) AND isnotnull(s_store_sk#10)) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_zip#14, 10, true, false, true)))
(15) CometProject
Input [5]: [s_store_sk#10, s_store_name#11, s_market_id#12, s_state#13, s_zip#14]
-Arguments: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14], [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
+Arguments: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16], [s_store_sk#10, s_store_name#11, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#13, 2, true, false, true) AS s_state#15, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_zip#14, 10, true, false, true) AS s_zip#16]
(16) CometBroadcastExchange
-Input [4]: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
-Arguments: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
+Input [4]: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
+Arguments: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
(17) CometBroadcastHashJoin
Left output [4]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5]
-Right output [4]: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
+Right output [4]: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
Arguments: [ss_store_sk#3], [s_store_sk#10], Inner, BuildRight
(18) CometProject
-Input [8]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5, s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
-Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14], [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14]
+Input [8]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5, s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
+Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16], [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16]
(19) CometScan parquet spark_catalog.default.item
-Output [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
+Output [6]: [i_item_sk#17, i_current_price#18, i_size#19, i_color#20, i_units#21, i_manager_id#22]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
-PushedFilters: [IsNotNull(i_color), IsNotNull(i_item_sk)]
+PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_current_price:decimal(7,2),i_size:string,i_color:string,i_units:string,i_manager_id:int>
(20) CometFilter
-Input [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Condition : ((isnotnull(i_color#18) AND (rpad(i_color#18, 20, ) = pale )) AND isnotnull(i_item_sk#15))
+Input [6]: [i_item_sk#17, i_current_price#18, i_size#19, i_color#20, i_units#21, i_manager_id#22]
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_color#20, 20, true, false, true) = pale ) AND isnotnull(i_item_sk#17))
+
+(21) CometProject
+Input [6]: [i_item_sk#17, i_current_price#18, i_size#19, i_color#20, i_units#21, i_manager_id#22]
+Arguments: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22], [i_item_sk#17, i_current_price#18, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_size#19, 20, true, false, true) AS i_size#23, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_color#20, 20, true, false, true) AS i_color#24, staticinvoke(class org.apache.spark.sql.cata [...]
-(21) CometBroadcastExchange
-Input [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Arguments: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
+(22) CometBroadcastExchange
+Input [6]: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Arguments: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
-(22) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14]
-Right output [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Arguments: [ss_item_sk#1], [i_item_sk#15], Inner, BuildRight
+(23) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16]
+Right output [6]: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Arguments: [ss_item_sk#1], [i_item_sk#17], Inner, BuildRight
-(23) CometProject
-Input [12]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Arguments: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20], [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
+(24) CometProject
+Input [12]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Arguments: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22], [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
-(24) CometScan parquet spark_catalog.default.customer
-Output [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
+(25) CometScan parquet spark_catalog.default.customer
+Output [4]: [c_customer_sk#26, c_first_name#27, c_last_name#28, c_birth_country#29]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_birth_country)]
ReadSchema: struct<c_customer_sk:int,c_first_name:string,c_last_name:string,c_birth_country:string>
-(25) CometFilter
-Input [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Condition : (isnotnull(c_customer_sk#21) AND isnotnull(c_birth_country#24))
+(26) CometFilter
+Input [4]: [c_customer_sk#26, c_first_name#27, c_last_name#28, c_birth_country#29]
+Condition : (isnotnull(c_customer_sk#26) AND isnotnull(c_birth_country#29))
-(26) CometBroadcastExchange
-Input [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Arguments: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
+(27) CometProject
+Input [4]: [c_customer_sk#26, c_first_name#27, c_last_name#28, c_birth_country#29]
+Arguments: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29], [c_customer_sk#26, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#27, 20, true, false, true) AS c_first_name#30, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_last_name#28, 30, true, false, true) AS c_last_name#31, c_birth_country#29]
-(27) CometBroadcastHashJoin
-Left output [10]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Right output [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Arguments: [ss_customer_sk#2], [c_customer_sk#21], Inner, BuildRight
+(28) CometBroadcastExchange
+Input [4]: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
+Arguments: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
-(28) CometProject
-Input [14]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Arguments: [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24], [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24]
+(29) CometBroadcastHashJoin
+Left output [10]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Right output [4]: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
+Arguments: [ss_customer_sk#2], [c_customer_sk#26], Inner, BuildRight
-(29) CometColumnarToRow [codegen id : 2]
-Input [12]: [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24]
+(30) CometProject
+Input [14]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
+Arguments: [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29], [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29]
-(30) CometScan parquet spark_catalog.default.customer_address
-Output [3]: [ca_state#25, ca_zip#26, ca_country#27]
+(31) CometColumnarToRow [codegen id : 2]
+Input [12]: [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29]
+
+(32) CometScan parquet spark_catalog.default.customer_address
+Output [3]: [ca_state#32, ca_zip#33, ca_country#34]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
-PushedFilters: [IsNotNull(ca_country), IsNotNull(ca_zip)]
+PushedFilters: [IsNotNull(ca_country)]
ReadSchema: struct<ca_state:string,ca_zip:string,ca_country:string>
-(31) CometFilter
-Input [3]: [ca_state#25, ca_zip#26, ca_country#27]
-Condition : (isnotnull(ca_country#27) AND isnotnull(ca_zip#26))
+(33) CometFilter
+Input [3]: [ca_state#32, ca_zip#33, ca_country#34]
+Condition : (isnotnull(ca_country#34) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#33, 10, true, false, true)))
+
+(34) CometProject
+Input [3]: [ca_state#32, ca_zip#33, ca_country#34]
+Arguments: [ca_state#35, ca_zip#36, ca_country#34], [staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#32, 2, true, false, true) AS ca_state#35, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#33, 10, true, false, true) AS ca_zip#36, ca_country#34]
-(32) CometColumnarToRow [codegen id : 1]
-Input [3]: [ca_state#25, ca_zip#26, ca_country#27]
+(35) CometColumnarToRow [codegen id : 1]
+Input [3]: [ca_state#35, ca_zip#36, ca_country#34]
-(33) BroadcastExchange
-Input [3]: [ca_state#25, ca_zip#26, ca_country#27]
-Arguments: HashedRelationBroadcastMode(List(upper(input[2, string, false]), rpad(input[1, string, false], 10, )),false), [plan_id=3]
+(36) BroadcastExchange
+Input [3]: [ca_state#35, ca_zip#36, ca_country#34]
+Arguments: HashedRelationBroadcastMode(List(upper(input[2, string, true]), input[1, string, true]),false), [plan_id=3]
-(34) BroadcastHashJoin [codegen id : 2]
-Left keys [2]: [c_birth_country#24, rpad(s_zip#14, 10, )]
-Right keys [2]: [upper(ca_country#27), rpad(ca_zip#26, 10, )]
+(37) BroadcastHashJoin [codegen id : 2]
+Left keys [2]: [c_birth_country#29, s_zip#16]
+Right keys [2]: [upper(ca_country#34), ca_zip#36]
Join type: Inner
Join condition: None
-(35) Project [codegen id : 2]
-Output [11]: [ss_net_paid#5, s_store_name#11, s_state#13, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, ca_state#25]
-Input [15]: [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24, ca_state#25, ca_zip#26, ca_country#27]
+(38) Project [codegen id : 2]
+Output [11]: [ss_net_paid#5, s_store_name#11, s_state#15, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, ca_state#35]
+Input [15]: [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29, ca_state#35, ca_zip#36, ca_country#34]
-(36) HashAggregate [codegen id : 2]
-Input [11]: [ss_net_paid#5, s_store_name#11, s_state#13, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, ca_state#25]
-Keys [10]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17]
+(39) HashAggregate [codegen id : 2]
+Input [11]: [ss_net_paid#5, s_store_name#11, s_state#15, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, ca_state#35]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23]
Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#5))]
-Aggregate Attributes [1]: [sum#28]
-Results [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
+Aggregate Attributes [1]: [sum#37]
+Results [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
-(37) CometColumnarExchange
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+(40) CometColumnarExchange
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-(38) CometColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
+(41) CometColumnarToRow [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
-(39) HashAggregate [codegen id : 3]
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-Keys [10]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17]
+(42) HashAggregate [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23]
Functions [1]: [sum(UnscaledValue(ss_net_paid#5))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#5))#30]
-Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, MakeDecimal(sum(UnscaledValue(ss_net_paid#5))#30,17,2) AS netpaid#31]
-
-(40) HashAggregate [codegen id : 3]
-Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, netpaid#31]
-Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
-Functions [1]: [partial_sum(netpaid#31)]
-Aggregate Attributes [2]: [sum#32, isEmpty#33]
-Results [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-
-(41) CometColumnarExchange
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(42) CometColumnarToRow [codegen id : 4]
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-
-(43) HashAggregate [codegen id : 4]
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
-Functions [1]: [sum(netpaid#31)]
-Aggregate Attributes [1]: [sum(netpaid#31)#36]
-Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, sum(netpaid#31)#36 AS paid#37]
-
-(44) Filter [codegen id : 4]
-Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, paid#37]
-Condition : (isnotnull(paid#37) AND (cast(paid#37 as decimal(33,8)) > cast(Subquery scalar-subquery#38, [id=#39] as decimal(33,8))))
+Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#5))#39]
+Results [4]: [c_last_name#31, c_first_name#30, s_store_name#11, MakeDecimal(sum(UnscaledValue(ss_net_paid#5))#39,17,2) AS netpaid#40]
+
+(43) HashAggregate [codegen id : 3]
+Input [4]: [c_last_name#31, c_first_name#30, s_store_name#11, netpaid#40]
+Keys [3]: [c_last_name#31, c_first_name#30, s_store_name#11]
+Functions [1]: [partial_sum(netpaid#40)]
+Aggregate Attributes [2]: [sum#41, isEmpty#42]
+Results [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+
+(44) CometColumnarExchange
+Input [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, s_store_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+
+(45) CometColumnarToRow [codegen id : 4]
+Input [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+
+(46) HashAggregate [codegen id : 4]
+Input [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+Keys [3]: [c_last_name#31, c_first_name#30, s_store_name#11]
+Functions [1]: [sum(netpaid#40)]
+Aggregate Attributes [1]: [sum(netpaid#40)#45]
+Results [4]: [c_last_name#31, c_first_name#30, s_store_name#11, sum(netpaid#40)#45 AS paid#46]
+
+(47) Filter [codegen id : 4]
+Input [4]: [c_last_name#31, c_first_name#30, s_store_name#11, paid#46]
+Condition : (isnotnull(paid#46) AND (cast(paid#46 as decimal(33,8)) > cast(Subquery scalar-subquery#47, [id=#48] as decimal(33,8))))
===== Subqueries =====
-Subquery:1 Hosting operator id = 44 Hosting Expression = Subquery scalar-subquery#38, [id=#39]
-* HashAggregate (73)
-+- * CometColumnarToRow (72)
- +- CometColumnarExchange (71)
- +- * HashAggregate (70)
- +- * HashAggregate (69)
- +- * CometColumnarToRow (68)
- +- CometColumnarExchange (67)
- +- * HashAggregate (66)
- +- * Project (65)
- +- * BroadcastHashJoin Inner BuildRight (64)
- :- * CometColumnarToRow (62)
- : +- CometProject (61)
- : +- CometBroadcastHashJoin (60)
- : :- CometProject (58)
- : : +- CometBroadcastHashJoin (57)
- : : :- CometProject (53)
- : : : +- CometBroadcastHashJoin (52)
- : : : :- CometProject (50)
- : : : : +- CometSortMergeJoin (49)
- : : : : :- CometSort (46)
- : : : : : +- ReusedExchange (45)
- : : : : +- CometSort (48)
- : : : : +- ReusedExchange (47)
- : : : +- ReusedExchange (51)
- : : +- CometBroadcastExchange (56)
- : : +- CometFilter (55)
- : : +- CometScan parquet spark_catalog.default.item (54)
- : +- ReusedExchange (59)
- +- ReusedExchange (63)
-
-
-(45) ReusedExchange [Reuses operator id: 4]
-Output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-
-(46) CometSort
-Input [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44], [ss_ticket_number#43 ASC NULLS FIRST, ss_item_sk#40 ASC NULLS FIRST]
-
-(47) ReusedExchange [Reuses operator id: 9]
-Output [2]: [sr_item_sk#45, sr_ticket_number#46]
-
-(48) CometSort
-Input [2]: [sr_item_sk#45, sr_ticket_number#46]
-Arguments: [sr_item_sk#45, sr_ticket_number#46], [sr_ticket_number#46 ASC NULLS FIRST, sr_item_sk#45 ASC NULLS FIRST]
-
-(49) CometSortMergeJoin
-Left output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-Right output [2]: [sr_item_sk#45, sr_ticket_number#46]
-Arguments: [ss_ticket_number#43, ss_item_sk#40], [sr_ticket_number#46, sr_item_sk#45], Inner
-
-(50) CometProject
-Input [7]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44, sr_item_sk#45, sr_ticket_number#46]
-Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44], [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
-
-(51) ReusedExchange [Reuses operator id: 16]
-Output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-
-(52) CometBroadcastHashJoin
-Left output [4]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
-Right output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-Arguments: [ss_store_sk#42], [s_store_sk#47], Inner, BuildRight
+Subquery:1 Hosting operator id = 47 Hosting Expression = Subquery scalar-subquery#47, [id=#48]
+* HashAggregate (77)
++- * CometColumnarToRow (76)
+ +- CometColumnarExchange (75)
+ +- * HashAggregate (74)
+ +- * HashAggregate (73)
+ +- * CometColumnarToRow (72)
+ +- CometColumnarExchange (71)
+ +- * HashAggregate (70)
+ +- * Project (69)
+ +- * BroadcastHashJoin Inner BuildRight (68)
+ :- * CometColumnarToRow (66)
+ : +- CometProject (65)
+ : +- CometBroadcastHashJoin (64)
+ : :- CometProject (62)
+ : : +- CometBroadcastHashJoin (61)
+ : : :- CometProject (56)
+ : : : +- CometBroadcastHashJoin (55)
+ : : : :- CometProject (53)
+ : : : : +- CometSortMergeJoin (52)
+ : : : : :- CometSort (49)
+ : : : : : +- ReusedExchange (48)
+ : : : : +- CometSort (51)
+ : : : : +- ReusedExchange (50)
+ : : : +- ReusedExchange (54)
+ : : +- CometBroadcastExchange (60)
+ : : +- CometProject (59)
+ : : +- CometFilter (58)
+ : : +- CometScan parquet spark_catalog.default.item (57)
+ : +- ReusedExchange (63)
+ +- ReusedExchange (67)
+
+
+(48) ReusedExchange [Reuses operator id: 4]
+Output [5]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53]
+
+(49) CometSort
+Input [5]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53]
+Arguments: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53], [ss_ticket_number#52 ASC NULLS FIRST, ss_item_sk#49 ASC NULLS FIRST]
+
+(50) ReusedExchange [Reuses operator id: 9]
+Output [2]: [sr_item_sk#54, sr_ticket_number#55]
+
+(51) CometSort
+Input [2]: [sr_item_sk#54, sr_ticket_number#55]
+Arguments: [sr_item_sk#54, sr_ticket_number#55], [sr_ticket_number#55 ASC NULLS FIRST, sr_item_sk#54 ASC NULLS FIRST]
+
+(52) CometSortMergeJoin
+Left output [5]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53]
+Right output [2]: [sr_item_sk#54, sr_ticket_number#55]
+Arguments: [ss_ticket_number#52, ss_item_sk#49], [sr_ticket_number#55, sr_item_sk#54], Inner
(53) CometProject
-Input [8]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44, s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50], [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
+Input [7]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53, sr_item_sk#54, sr_ticket_number#55]
+Arguments: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53], [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53]
-(54) CometScan parquet spark_catalog.default.item
-Output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
+(54) ReusedExchange [Reuses operator id: 16]
+Output [4]: [s_store_sk#56, s_store_name#57, s_state#15, s_zip#16]
+
+(55) CometBroadcastHashJoin
+Left output [4]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53]
+Right output [4]: [s_store_sk#56, s_store_name#57, s_state#15, s_zip#16]
+Arguments: [ss_store_sk#51], [s_store_sk#56], Inner, BuildRight
+
+(56) CometProject
+Input [8]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53, s_store_sk#56, s_store_name#57, s_state#15, s_zip#16]
+Arguments: [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16], [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16]
+
+(57) CometScan parquet spark_catalog.default.item
+Output [6]: [i_item_sk#58, i_current_price#59, i_size#60, i_color#61, i_units#62, i_manager_id#63]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_current_price:decimal(7,2),i_size:string,i_color:string,i_units:string,i_manager_id:int>
-(55) CometFilter
-Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Condition : isnotnull(i_item_sk#51)
+(58) CometFilter
+Input [6]: [i_item_sk#58, i_current_price#59, i_size#60, i_color#61, i_units#62, i_manager_id#63]
+Condition : isnotnull(i_item_sk#58)
+
+(59) CometProject
+Input [6]: [i_item_sk#58, i_current_price#59, i_size#60, i_color#61, i_units#62, i_manager_id#63]
+Arguments: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63], [i_item_sk#58, i_current_price#59, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_size#60, 20, true, false, true) AS i_size#23, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_color#61, 20, true, false, true) AS i_color#24, staticinvoke(class org.apache.spark.sql.cata [...]
-(56) CometBroadcastExchange
-Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Arguments: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
+(60) CometBroadcastExchange
+Input [6]: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Arguments: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
-(57) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
-Right output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Arguments: [ss_item_sk#40], [i_item_sk#51], Inner, BuildRight
+(61) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16]
+Right output [6]: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Arguments: [ss_item_sk#49], [i_item_sk#58], Inner, BuildRight
-(58) CometProject
-Input [12]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Arguments: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56], [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
+(62) CometProject
+Input [12]: [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Arguments: [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63], [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
-(59) ReusedExchange [Reuses operator id: 26]
-Output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
+(63) ReusedExchange [Reuses operator id: 28]
+Output [4]: [c_customer_sk#64, c_first_name#30, c_last_name#31, c_birth_country#65]
-(60) CometBroadcastHashJoin
-Left output [10]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Right output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
-Arguments: [ss_customer_sk#41], [c_customer_sk#57], Inner, BuildRight
+(64) CometBroadcastHashJoin
+Left output [10]: [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Right output [4]: [c_customer_sk#64, c_first_name#30, c_last_name#31, c_birth_country#65]
+Arguments: [ss_customer_sk#50], [c_customer_sk#64], Inner, BuildRight
-(61) CometProject
-Input [14]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
-Arguments: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60], [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
+(65) CometProject
+Input [14]: [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_customer_sk#64, c_first_name#30, c_last_name#31, c_birth_country#65]
+Arguments: [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65], [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65]
-(62) CometColumnarToRow [codegen id : 2]
-Input [12]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
+(66) CometColumnarToRow [codegen id : 2]
+Input [12]: [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65]
-(63) ReusedExchange [Reuses operator id: 33]
-Output [3]: [ca_state#61, ca_zip#62, ca_country#63]
+(67) ReusedExchange [Reuses operator id: 36]
+Output [3]: [ca_state#35, ca_zip#36, ca_country#66]
-(64) BroadcastHashJoin [codegen id : 2]
-Left keys [2]: [c_birth_country#60, rpad(s_zip#50, 10, )]
-Right keys [2]: [upper(ca_country#63), rpad(ca_zip#62, 10, )]
+(68) BroadcastHashJoin [codegen id : 2]
+Left keys [2]: [c_birth_country#65, s_zip#16]
+Right keys [2]: [upper(ca_country#66), ca_zip#36]
Join type: Inner
Join condition: None
-(65) Project [codegen id : 2]
-Output [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
-Input [15]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60, ca_state#61, ca_zip#62, ca_country#63]
-
-(66) HashAggregate [codegen id : 2]
-Input [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
-Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
-Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#44))]
-Aggregate Attributes [1]: [sum#64]
-Results [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-
-(67) CometColumnarExchange
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-Arguments: hashpartitioning(c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
-
-(68) CometColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-
-(69) HashAggregate [codegen id : 3]
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
-Functions [1]: [sum(UnscaledValue(ss_net_paid#44))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#44))#30]
-Results [1]: [MakeDecimal(sum(UnscaledValue(ss_net_paid#44))#30,17,2) AS netpaid#66]
-
-(70) HashAggregate [codegen id : 3]
-Input [1]: [netpaid#66]
-Keys: []
-Functions [1]: [partial_avg(netpaid#66)]
-Aggregate Attributes [2]: [sum#67, count#68]
-Results [2]: [sum#69, count#70]
+(69) Project [codegen id : 2]
+Output [11]: [ss_net_paid#53, s_store_name#57, s_state#15, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, ca_state#35]
+Input [15]: [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65, ca_state#35, ca_zip#36, ca_country#66]
+
+(70) HashAggregate [codegen id : 2]
+Input [11]: [ss_net_paid#53, s_store_name#57, s_state#15, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, ca_state#35]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23]
+Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#53))]
+Aggregate Attributes [1]: [sum#67]
+Results [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
(71) CometColumnarExchange
-Input [2]: [sum#69, count#70]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+
+(72) CometColumnarToRow [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
+
+(73) HashAggregate [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23]
+Functions [1]: [sum(UnscaledValue(ss_net_paid#53))]
+Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#53))#39]
+Results [1]: [MakeDecimal(sum(UnscaledValue(ss_net_paid#53))#39,17,2) AS netpaid#69]
+
+(74) HashAggregate [codegen id : 3]
+Input [1]: [netpaid#69]
+Keys: []
+Functions [1]: [partial_avg(netpaid#69)]
+Aggregate Attributes [2]: [sum#70, count#71]
+Results [2]: [sum#72, count#73]
+
+(75) CometColumnarExchange
+Input [2]: [sum#72, count#73]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
-(72) CometColumnarToRow [codegen id : 4]
-Input [2]: [sum#69, count#70]
+(76) CometColumnarToRow [codegen id : 4]
+Input [2]: [sum#72, count#73]
-(73) HashAggregate [codegen id : 4]
-Input [2]: [sum#69, count#70]
+(77) HashAggregate [codegen id : 4]
+Input [2]: [sum#72, count#73]
Keys: []
-Functions [1]: [avg(netpaid#66)]
-Aggregate Attributes [1]: [avg(netpaid#66)#71]
-Results [1]: [(0.05 * avg(netpaid#66)#71) AS (0.05 * avg(netpaid))#72]
+Functions [1]: [avg(netpaid#69)]
+Aggregate Attributes [1]: [avg(netpaid#69)#74]
+Results [1]: [(0.05 * avg(netpaid#69)#74) AS (0.05 * avg(netpaid))#75]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt
index df6657f73..fb62bdf3c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24a/simplified.txt
@@ -32,8 +32,9 @@ WholeStageCodegen (4)
ReusedExchange [sr_item_sk,sr_ticket_number] #4
ReusedExchange [s_store_sk,s_store_name,s_state,s_zip] #5
CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #11
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [i_size,i_color,i_units] [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
ReusedExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
InputAdapter
ReusedExchange [ca_state,ca_zip,ca_country] #8
@@ -72,19 +73,22 @@ WholeStageCodegen (4)
CometFilter [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
CometBroadcastExchange [s_store_sk,s_store_name,s_state,s_zip] #5
- CometProject [s_store_sk,s_store_name,s_state,s_zip]
+ CometProject [s_state,s_zip] [s_store_sk,s_store_name,s_state,s_zip]
CometFilter [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
CometScan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #6
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [i_size,i_color,i_units] [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
- CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometProject [c_first_name,c_last_name] [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
InputAdapter
BroadcastExchange #8
WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
- CometFilter [ca_state,ca_zip,ca_country]
- CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
+ CometProject [ca_state,ca_zip] [ca_state,ca_zip,ca_country]
+ CometFilter [ca_state,ca_zip,ca_country]
+ CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt
index 864a38428..a04d66938 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/explain.txt
@@ -1,20 +1,20 @@
== Physical Plan ==
-* Filter (44)
-+- * HashAggregate (43)
- +- * CometColumnarToRow (42)
- +- CometColumnarExchange (41)
- +- * HashAggregate (40)
- +- * HashAggregate (39)
- +- * CometColumnarToRow (38)
- +- CometColumnarExchange (37)
- +- * HashAggregate (36)
- +- * Project (35)
- +- * BroadcastHashJoin Inner BuildRight (34)
- :- * CometColumnarToRow (29)
- : +- CometProject (28)
- : +- CometBroadcastHashJoin (27)
- : :- CometProject (23)
- : : +- CometBroadcastHashJoin (22)
+* Filter (47)
++- * HashAggregate (46)
+ +- * CometColumnarToRow (45)
+ +- CometColumnarExchange (44)
+ +- * HashAggregate (43)
+ +- * HashAggregate (42)
+ +- * CometColumnarToRow (41)
+ +- CometColumnarExchange (40)
+ +- * HashAggregate (39)
+ +- * Project (38)
+ +- * BroadcastHashJoin Inner BuildRight (37)
+ :- * CometColumnarToRow (31)
+ : +- CometProject (30)
+ : +- CometBroadcastHashJoin (29)
+ : :- CometProject (24)
+ : : +- CometBroadcastHashJoin (23)
: : :- CometProject (18)
: : : +- CometBroadcastHashJoin (17)
: : : :- CometProject (12)
@@ -33,16 +33,19 @@
: : : +- CometProject (15)
: : : +- CometFilter (14)
: : : +- CometScan parquet spark_catalog.default.store (13)
- : : +- CometBroadcastExchange (21)
- : : +- CometFilter (20)
- : : +- CometScan parquet spark_catalog.default.item (19)
- : +- CometBroadcastExchange (26)
- : +- CometFilter (25)
- : +- CometScan parquet spark_catalog.default.customer (24)
- +- BroadcastExchange (33)
- +- * CometColumnarToRow (32)
- +- CometFilter (31)
- +- CometScan parquet spark_catalog.default.customer_address (30)
+ : : +- CometBroadcastExchange (22)
+ : : +- CometProject (21)
+ : : +- CometFilter (20)
+ : : +- CometScan parquet spark_catalog.default.item (19)
+ : +- CometBroadcastExchange (28)
+ : +- CometProject (27)
+ : +- CometFilter (26)
+ : +- CometScan parquet spark_catalog.default.customer (25)
+ +- BroadcastExchange (36)
+ +- * CometColumnarToRow (35)
+ +- CometProject (34)
+ +- CometFilter (33)
+ +- CometScan parquet spark_catalog.default.customer_address (32)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -104,316 +107,333 @@ Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5], [ss_i
Output [5]: [s_store_sk#10, s_store_name#11, s_market_id#12, s_state#13, s_zip#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
-PushedFilters: [IsNotNull(s_market_id), EqualTo(s_market_id,8), IsNotNull(s_store_sk), IsNotNull(s_zip)]
+PushedFilters: [IsNotNull(s_market_id), EqualTo(s_market_id,8), IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_store_name:string,s_market_id:int,s_state:string,s_zip:string>
(14) CometFilter
Input [5]: [s_store_sk#10, s_store_name#11, s_market_id#12, s_state#13, s_zip#14]
-Condition : (((isnotnull(s_market_id#12) AND (s_market_id#12 = 8)) AND isnotnull(s_store_sk#10)) AND isnotnull(s_zip#14))
+Condition : (((isnotnull(s_market_id#12) AND (s_market_id#12 = 8)) AND isnotnull(s_store_sk#10)) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_zip#14, 10, true, false, true)))
(15) CometProject
Input [5]: [s_store_sk#10, s_store_name#11, s_market_id#12, s_state#13, s_zip#14]
-Arguments: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14], [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
+Arguments: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16], [s_store_sk#10, s_store_name#11, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#13, 2, true, false, true) AS s_state#15, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_zip#14, 10, true, false, true) AS s_zip#16]
(16) CometBroadcastExchange
-Input [4]: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
-Arguments: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
+Input [4]: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
+Arguments: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
(17) CometBroadcastHashJoin
Left output [4]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5]
-Right output [4]: [s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
+Right output [4]: [s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
Arguments: [ss_store_sk#3], [s_store_sk#10], Inner, BuildRight
(18) CometProject
-Input [8]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5, s_store_sk#10, s_store_name#11, s_state#13, s_zip#14]
-Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14], [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14]
+Input [8]: [ss_item_sk#1, ss_customer_sk#2, ss_store_sk#3, ss_net_paid#5, s_store_sk#10, s_store_name#11, s_state#15, s_zip#16]
+Arguments: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16], [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16]
(19) CometScan parquet spark_catalog.default.item
-Output [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
+Output [6]: [i_item_sk#17, i_current_price#18, i_size#19, i_color#20, i_units#21, i_manager_id#22]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
-PushedFilters: [IsNotNull(i_color), IsNotNull(i_item_sk)]
+PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_current_price:decimal(7,2),i_size:string,i_color:string,i_units:string,i_manager_id:int>
(20) CometFilter
-Input [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Condition : ((isnotnull(i_color#18) AND (rpad(i_color#18, 20, ) = chiffon )) AND isnotnull(i_item_sk#15))
+Input [6]: [i_item_sk#17, i_current_price#18, i_size#19, i_color#20, i_units#21, i_manager_id#22]
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_color#20, 20, true, false, true) = chiffon ) AND isnotnull(i_item_sk#17))
+
+(21) CometProject
+Input [6]: [i_item_sk#17, i_current_price#18, i_size#19, i_color#20, i_units#21, i_manager_id#22]
+Arguments: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22], [i_item_sk#17, i_current_price#18, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_size#19, 20, true, false, true) AS i_size#23, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_color#20, 20, true, false, true) AS i_color#24, staticinvoke(class org.apache.spark.sql.cata [...]
-(21) CometBroadcastExchange
-Input [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Arguments: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
+(22) CometBroadcastExchange
+Input [6]: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Arguments: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
-(22) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14]
-Right output [6]: [i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Arguments: [ss_item_sk#1], [i_item_sk#15], Inner, BuildRight
+(23) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16]
+Right output [6]: [i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Arguments: [ss_item_sk#1], [i_item_sk#17], Inner, BuildRight
-(23) CometProject
-Input [12]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_item_sk#15, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Arguments: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20], [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
+(24) CometProject
+Input [12]: [ss_item_sk#1, ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_item_sk#17, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Arguments: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22], [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
-(24) CometScan parquet spark_catalog.default.customer
-Output [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
+(25) CometScan parquet spark_catalog.default.customer
+Output [4]: [c_customer_sk#26, c_first_name#27, c_last_name#28, c_birth_country#29]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_birth_country)]
ReadSchema: struct<c_customer_sk:int,c_first_name:string,c_last_name:string,c_birth_country:string>
-(25) CometFilter
-Input [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Condition : (isnotnull(c_customer_sk#21) AND isnotnull(c_birth_country#24))
+(26) CometFilter
+Input [4]: [c_customer_sk#26, c_first_name#27, c_last_name#28, c_birth_country#29]
+Condition : (isnotnull(c_customer_sk#26) AND isnotnull(c_birth_country#29))
-(26) CometBroadcastExchange
-Input [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Arguments: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
+(27) CometProject
+Input [4]: [c_customer_sk#26, c_first_name#27, c_last_name#28, c_birth_country#29]
+Arguments: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29], [c_customer_sk#26, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#27, 20, true, false, true) AS c_first_name#30, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_last_name#28, 30, true, false, true) AS c_last_name#31, c_birth_country#29]
-(27) CometBroadcastHashJoin
-Left output [10]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20]
-Right output [4]: [c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Arguments: [ss_customer_sk#2], [c_customer_sk#21], Inner, BuildRight
+(28) CometBroadcastExchange
+Input [4]: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
+Arguments: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
-(28) CometProject
-Input [14]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_customer_sk#21, c_first_name#22, c_last_name#23, c_birth_country#24]
-Arguments: [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24], [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24]
+(29) CometBroadcastHashJoin
+Left output [10]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22]
+Right output [4]: [c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
+Arguments: [ss_customer_sk#2], [c_customer_sk#26], Inner, BuildRight
-(29) CometColumnarToRow [codegen id : 2]
-Input [12]: [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24]
+(30) CometProject
+Input [14]: [ss_customer_sk#2, ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_customer_sk#26, c_first_name#30, c_last_name#31, c_birth_country#29]
+Arguments: [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29], [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29]
-(30) CometScan parquet spark_catalog.default.customer_address
-Output [3]: [ca_state#25, ca_zip#26, ca_country#27]
+(31) CometColumnarToRow [codegen id : 2]
+Input [12]: [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29]
+
+(32) CometScan parquet spark_catalog.default.customer_address
+Output [3]: [ca_state#32, ca_zip#33, ca_country#34]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
-PushedFilters: [IsNotNull(ca_country), IsNotNull(ca_zip)]
+PushedFilters: [IsNotNull(ca_country)]
ReadSchema: struct<ca_state:string,ca_zip:string,ca_country:string>
-(31) CometFilter
-Input [3]: [ca_state#25, ca_zip#26, ca_country#27]
-Condition : (isnotnull(ca_country#27) AND isnotnull(ca_zip#26))
+(33) CometFilter
+Input [3]: [ca_state#32, ca_zip#33, ca_country#34]
+Condition : (isnotnull(ca_country#34) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#33, 10, true, false, true)))
+
+(34) CometProject
+Input [3]: [ca_state#32, ca_zip#33, ca_country#34]
+Arguments: [ca_state#35, ca_zip#36, ca_country#34], [staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#32, 2, true, false, true) AS ca_state#35, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_zip#33, 10, true, false, true) AS ca_zip#36, ca_country#34]
-(32) CometColumnarToRow [codegen id : 1]
-Input [3]: [ca_state#25, ca_zip#26, ca_country#27]
+(35) CometColumnarToRow [codegen id : 1]
+Input [3]: [ca_state#35, ca_zip#36, ca_country#34]
-(33) BroadcastExchange
-Input [3]: [ca_state#25, ca_zip#26, ca_country#27]
-Arguments: HashedRelationBroadcastMode(List(upper(input[2, string, false]), rpad(input[1, string, false], 10, )),false), [plan_id=3]
+(36) BroadcastExchange
+Input [3]: [ca_state#35, ca_zip#36, ca_country#34]
+Arguments: HashedRelationBroadcastMode(List(upper(input[2, string, true]), input[1, string, true]),false), [plan_id=3]
-(34) BroadcastHashJoin [codegen id : 2]
-Left keys [2]: [c_birth_country#24, rpad(s_zip#14, 10, )]
-Right keys [2]: [upper(ca_country#27), rpad(ca_zip#26, 10, )]
+(37) BroadcastHashJoin [codegen id : 2]
+Left keys [2]: [c_birth_country#29, s_zip#16]
+Right keys [2]: [upper(ca_country#34), ca_zip#36]
Join type: Inner
Join condition: None
-(35) Project [codegen id : 2]
-Output [11]: [ss_net_paid#5, s_store_name#11, s_state#13, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, ca_state#25]
-Input [15]: [ss_net_paid#5, s_store_name#11, s_state#13, s_zip#14, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, c_birth_country#24, ca_state#25, ca_zip#26, ca_country#27]
+(38) Project [codegen id : 2]
+Output [11]: [ss_net_paid#5, s_store_name#11, s_state#15, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, ca_state#35]
+Input [15]: [ss_net_paid#5, s_store_name#11, s_state#15, s_zip#16, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, c_birth_country#29, ca_state#35, ca_zip#36, ca_country#34]
-(36) HashAggregate [codegen id : 2]
-Input [11]: [ss_net_paid#5, s_store_name#11, s_state#13, i_current_price#16, i_size#17, i_color#18, i_units#19, i_manager_id#20, c_first_name#22, c_last_name#23, ca_state#25]
-Keys [10]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17]
+(39) HashAggregate [codegen id : 2]
+Input [11]: [ss_net_paid#5, s_store_name#11, s_state#15, i_current_price#18, i_size#23, i_color#24, i_units#25, i_manager_id#22, c_first_name#30, c_last_name#31, ca_state#35]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23]
Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#5))]
-Aggregate Attributes [1]: [sum#28]
-Results [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
+Aggregate Attributes [1]: [sum#37]
+Results [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
-(37) CometColumnarExchange
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
+(40) CometColumnarExchange
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=4]
-(38) CometColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
+(41) CometColumnarToRow [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
-(39) HashAggregate [codegen id : 3]
-Input [11]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17, sum#29]
-Keys [10]: [c_last_name#23, c_first_name#22, s_store_name#11, ca_state#25, s_state#13, i_color#18, i_current_price#16, i_manager_id#20, i_units#19, i_size#17]
+(42) HashAggregate [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23, sum#38]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#11, ca_state#35, s_state#15, i_color#24, i_current_price#18, i_manager_id#22, i_units#25, i_size#23]
Functions [1]: [sum(UnscaledValue(ss_net_paid#5))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#5))#30]
-Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, MakeDecimal(sum(UnscaledValue(ss_net_paid#5))#30,17,2) AS netpaid#31]
-
-(40) HashAggregate [codegen id : 3]
-Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, netpaid#31]
-Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
-Functions [1]: [partial_sum(netpaid#31)]
-Aggregate Attributes [2]: [sum#32, isEmpty#33]
-Results [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-
-(41) CometColumnarExchange
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-Arguments: hashpartitioning(c_last_name#23, c_first_name#22, s_store_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-
-(42) CometColumnarToRow [codegen id : 4]
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-
-(43) HashAggregate [codegen id : 4]
-Input [5]: [c_last_name#23, c_first_name#22, s_store_name#11, sum#34, isEmpty#35]
-Keys [3]: [c_last_name#23, c_first_name#22, s_store_name#11]
-Functions [1]: [sum(netpaid#31)]
-Aggregate Attributes [1]: [sum(netpaid#31)#36]
-Results [4]: [c_last_name#23, c_first_name#22, s_store_name#11, sum(netpaid#31)#36 AS paid#37]
-
-(44) Filter [codegen id : 4]
-Input [4]: [c_last_name#23, c_first_name#22, s_store_name#11, paid#37]
-Condition : (isnotnull(paid#37) AND (cast(paid#37 as decimal(33,8)) > cast(Subquery scalar-subquery#38, [id=#39] as decimal(33,8))))
+Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#5))#39]
+Results [4]: [c_last_name#31, c_first_name#30, s_store_name#11, MakeDecimal(sum(UnscaledValue(ss_net_paid#5))#39,17,2) AS netpaid#40]
+
+(43) HashAggregate [codegen id : 3]
+Input [4]: [c_last_name#31, c_first_name#30, s_store_name#11, netpaid#40]
+Keys [3]: [c_last_name#31, c_first_name#30, s_store_name#11]
+Functions [1]: [partial_sum(netpaid#40)]
+Aggregate Attributes [2]: [sum#41, isEmpty#42]
+Results [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+
+(44) CometColumnarExchange
+Input [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, s_store_name#11, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+
+(45) CometColumnarToRow [codegen id : 4]
+Input [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+
+(46) HashAggregate [codegen id : 4]
+Input [5]: [c_last_name#31, c_first_name#30, s_store_name#11, sum#43, isEmpty#44]
+Keys [3]: [c_last_name#31, c_first_name#30, s_store_name#11]
+Functions [1]: [sum(netpaid#40)]
+Aggregate Attributes [1]: [sum(netpaid#40)#45]
+Results [4]: [c_last_name#31, c_first_name#30, s_store_name#11, sum(netpaid#40)#45 AS paid#46]
+
+(47) Filter [codegen id : 4]
+Input [4]: [c_last_name#31, c_first_name#30, s_store_name#11, paid#46]
+Condition : (isnotnull(paid#46) AND (cast(paid#46 as decimal(33,8)) > cast(Subquery scalar-subquery#47, [id=#48] as decimal(33,8))))
===== Subqueries =====
-Subquery:1 Hosting operator id = 44 Hosting Expression = Subquery scalar-subquery#38, [id=#39]
-* HashAggregate (73)
-+- * CometColumnarToRow (72)
- +- CometColumnarExchange (71)
- +- * HashAggregate (70)
- +- * HashAggregate (69)
- +- * CometColumnarToRow (68)
- +- CometColumnarExchange (67)
- +- * HashAggregate (66)
- +- * Project (65)
- +- * BroadcastHashJoin Inner BuildRight (64)
- :- * CometColumnarToRow (62)
- : +- CometProject (61)
- : +- CometBroadcastHashJoin (60)
- : :- CometProject (58)
- : : +- CometBroadcastHashJoin (57)
- : : :- CometProject (53)
- : : : +- CometBroadcastHashJoin (52)
- : : : :- CometProject (50)
- : : : : +- CometSortMergeJoin (49)
- : : : : :- CometSort (46)
- : : : : : +- ReusedExchange (45)
- : : : : +- CometSort (48)
- : : : : +- ReusedExchange (47)
- : : : +- ReusedExchange (51)
- : : +- CometBroadcastExchange (56)
- : : +- CometFilter (55)
- : : +- CometScan parquet spark_catalog.default.item (54)
- : +- ReusedExchange (59)
- +- ReusedExchange (63)
-
-
-(45) ReusedExchange [Reuses operator id: 4]
-Output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-
-(46) CometSort
-Input [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44], [ss_ticket_number#43 ASC NULLS FIRST, ss_item_sk#40 ASC NULLS FIRST]
-
-(47) ReusedExchange [Reuses operator id: 9]
-Output [2]: [sr_item_sk#45, sr_ticket_number#46]
-
-(48) CometSort
-Input [2]: [sr_item_sk#45, sr_ticket_number#46]
-Arguments: [sr_item_sk#45, sr_ticket_number#46], [sr_ticket_number#46 ASC NULLS FIRST, sr_item_sk#45 ASC NULLS FIRST]
-
-(49) CometSortMergeJoin
-Left output [5]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44]
-Right output [2]: [sr_item_sk#45, sr_ticket_number#46]
-Arguments: [ss_ticket_number#43, ss_item_sk#40], [sr_ticket_number#46, sr_item_sk#45], Inner
-
-(50) CometProject
-Input [7]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_ticket_number#43, ss_net_paid#44, sr_item_sk#45, sr_ticket_number#46]
-Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44], [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
-
-(51) ReusedExchange [Reuses operator id: 16]
-Output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-
-(52) CometBroadcastHashJoin
-Left output [4]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44]
-Right output [4]: [s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-Arguments: [ss_store_sk#42], [s_store_sk#47], Inner, BuildRight
+Subquery:1 Hosting operator id = 47 Hosting Expression = Subquery scalar-subquery#47, [id=#48]
+* HashAggregate (77)
++- * CometColumnarToRow (76)
+ +- CometColumnarExchange (75)
+ +- * HashAggregate (74)
+ +- * HashAggregate (73)
+ +- * CometColumnarToRow (72)
+ +- CometColumnarExchange (71)
+ +- * HashAggregate (70)
+ +- * Project (69)
+ +- * BroadcastHashJoin Inner BuildRight (68)
+ :- * CometColumnarToRow (66)
+ : +- CometProject (65)
+ : +- CometBroadcastHashJoin (64)
+ : :- CometProject (62)
+ : : +- CometBroadcastHashJoin (61)
+ : : :- CometProject (56)
+ : : : +- CometBroadcastHashJoin (55)
+ : : : :- CometProject (53)
+ : : : : +- CometSortMergeJoin (52)
+ : : : : :- CometSort (49)
+ : : : : : +- ReusedExchange (48)
+ : : : : +- CometSort (51)
+ : : : : +- ReusedExchange (50)
+ : : : +- ReusedExchange (54)
+ : : +- CometBroadcastExchange (60)
+ : : +- CometProject (59)
+ : : +- CometFilter (58)
+ : : +- CometScan parquet spark_catalog.default.item (57)
+ : +- ReusedExchange (63)
+ +- ReusedExchange (67)
+
+
+(48) ReusedExchange [Reuses operator id: 4]
+Output [5]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53]
+
+(49) CometSort
+Input [5]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53]
+Arguments: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53], [ss_ticket_number#52 ASC NULLS FIRST, ss_item_sk#49 ASC NULLS FIRST]
+
+(50) ReusedExchange [Reuses operator id: 9]
+Output [2]: [sr_item_sk#54, sr_ticket_number#55]
+
+(51) CometSort
+Input [2]: [sr_item_sk#54, sr_ticket_number#55]
+Arguments: [sr_item_sk#54, sr_ticket_number#55], [sr_ticket_number#55 ASC NULLS FIRST, sr_item_sk#54 ASC NULLS FIRST]
+
+(52) CometSortMergeJoin
+Left output [5]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53]
+Right output [2]: [sr_item_sk#54, sr_ticket_number#55]
+Arguments: [ss_ticket_number#52, ss_item_sk#49], [sr_ticket_number#55, sr_item_sk#54], Inner
(53) CometProject
-Input [8]: [ss_item_sk#40, ss_customer_sk#41, ss_store_sk#42, ss_net_paid#44, s_store_sk#47, s_store_name#48, s_state#49, s_zip#50]
-Arguments: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50], [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
+Input [7]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_ticket_number#52, ss_net_paid#53, sr_item_sk#54, sr_ticket_number#55]
+Arguments: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53], [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53]
-(54) CometScan parquet spark_catalog.default.item
-Output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
+(54) ReusedExchange [Reuses operator id: 16]
+Output [4]: [s_store_sk#56, s_store_name#57, s_state#15, s_zip#16]
+
+(55) CometBroadcastHashJoin
+Left output [4]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53]
+Right output [4]: [s_store_sk#56, s_store_name#57, s_state#15, s_zip#16]
+Arguments: [ss_store_sk#51], [s_store_sk#56], Inner, BuildRight
+
+(56) CometProject
+Input [8]: [ss_item_sk#49, ss_customer_sk#50, ss_store_sk#51, ss_net_paid#53, s_store_sk#56, s_store_name#57, s_state#15, s_zip#16]
+Arguments: [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16], [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16]
+
+(57) CometScan parquet spark_catalog.default.item
+Output [6]: [i_item_sk#58, i_current_price#59, i_size#60, i_color#61, i_units#62, i_manager_id#63]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_current_price:decimal(7,2),i_size:string,i_color:string,i_units:string,i_manager_id:int>
-(55) CometFilter
-Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Condition : isnotnull(i_item_sk#51)
+(58) CometFilter
+Input [6]: [i_item_sk#58, i_current_price#59, i_size#60, i_color#61, i_units#62, i_manager_id#63]
+Condition : isnotnull(i_item_sk#58)
+
+(59) CometProject
+Input [6]: [i_item_sk#58, i_current_price#59, i_size#60, i_color#61, i_units#62, i_manager_id#63]
+Arguments: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63], [i_item_sk#58, i_current_price#59, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_size#60, 20, true, false, true) AS i_size#23, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_color#61, 20, true, false, true) AS i_color#24, staticinvoke(class org.apache.spark.sql.cata [...]
-(56) CometBroadcastExchange
-Input [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Arguments: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
+(60) CometBroadcastExchange
+Input [6]: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Arguments: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
-(57) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50]
-Right output [6]: [i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Arguments: [ss_item_sk#40], [i_item_sk#51], Inner, BuildRight
+(61) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16]
+Right output [6]: [i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Arguments: [ss_item_sk#49], [i_item_sk#58], Inner, BuildRight
-(58) CometProject
-Input [12]: [ss_item_sk#40, ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_item_sk#51, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Arguments: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56], [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
+(62) CometProject
+Input [12]: [ss_item_sk#49, ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_item_sk#58, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Arguments: [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63], [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
-(59) ReusedExchange [Reuses operator id: 26]
-Output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
+(63) ReusedExchange [Reuses operator id: 28]
+Output [4]: [c_customer_sk#64, c_first_name#30, c_last_name#31, c_birth_country#65]
-(60) CometBroadcastHashJoin
-Left output [10]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56]
-Right output [4]: [c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
-Arguments: [ss_customer_sk#41], [c_customer_sk#57], Inner, BuildRight
+(64) CometBroadcastHashJoin
+Left output [10]: [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63]
+Right output [4]: [c_customer_sk#64, c_first_name#30, c_last_name#31, c_birth_country#65]
+Arguments: [ss_customer_sk#50], [c_customer_sk#64], Inner, BuildRight
-(61) CometProject
-Input [14]: [ss_customer_sk#41, ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_customer_sk#57, c_first_name#58, c_last_name#59, c_birth_country#60]
-Arguments: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60], [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
+(65) CometProject
+Input [14]: [ss_customer_sk#50, ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_customer_sk#64, c_first_name#30, c_last_name#31, c_birth_country#65]
+Arguments: [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65], [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65]
-(62) CometColumnarToRow [codegen id : 2]
-Input [12]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60]
+(66) CometColumnarToRow [codegen id : 2]
+Input [12]: [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65]
-(63) ReusedExchange [Reuses operator id: 33]
-Output [3]: [ca_state#61, ca_zip#62, ca_country#63]
+(67) ReusedExchange [Reuses operator id: 36]
+Output [3]: [ca_state#35, ca_zip#36, ca_country#66]
-(64) BroadcastHashJoin [codegen id : 2]
-Left keys [2]: [c_birth_country#60, rpad(s_zip#50, 10, )]
-Right keys [2]: [upper(ca_country#63), rpad(ca_zip#62, 10, )]
+(68) BroadcastHashJoin [codegen id : 2]
+Left keys [2]: [c_birth_country#65, s_zip#16]
+Right keys [2]: [upper(ca_country#66), ca_zip#36]
Join type: Inner
Join condition: None
-(65) Project [codegen id : 2]
-Output [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
-Input [15]: [ss_net_paid#44, s_store_name#48, s_state#49, s_zip#50, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, c_birth_country#60, ca_state#61, ca_zip#62, ca_country#63]
-
-(66) HashAggregate [codegen id : 2]
-Input [11]: [ss_net_paid#44, s_store_name#48, s_state#49, i_current_price#52, i_size#53, i_color#54, i_units#55, i_manager_id#56, c_first_name#58, c_last_name#59, ca_state#61]
-Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
-Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#44))]
-Aggregate Attributes [1]: [sum#64]
-Results [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-
-(67) CometColumnarExchange
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-Arguments: hashpartitioning(c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
-
-(68) CometColumnarToRow [codegen id : 3]
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-
-(69) HashAggregate [codegen id : 3]
-Input [11]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53, sum#65]
-Keys [10]: [c_last_name#59, c_first_name#58, s_store_name#48, ca_state#61, s_state#49, i_color#54, i_current_price#52, i_manager_id#56, i_units#55, i_size#53]
-Functions [1]: [sum(UnscaledValue(ss_net_paid#44))]
-Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#44))#30]
-Results [1]: [MakeDecimal(sum(UnscaledValue(ss_net_paid#44))#30,17,2) AS netpaid#66]
-
-(70) HashAggregate [codegen id : 3]
-Input [1]: [netpaid#66]
-Keys: []
-Functions [1]: [partial_avg(netpaid#66)]
-Aggregate Attributes [2]: [sum#67, count#68]
-Results [2]: [sum#69, count#70]
+(69) Project [codegen id : 2]
+Output [11]: [ss_net_paid#53, s_store_name#57, s_state#15, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, ca_state#35]
+Input [15]: [ss_net_paid#53, s_store_name#57, s_state#15, s_zip#16, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, c_birth_country#65, ca_state#35, ca_zip#36, ca_country#66]
+
+(70) HashAggregate [codegen id : 2]
+Input [11]: [ss_net_paid#53, s_store_name#57, s_state#15, i_current_price#59, i_size#23, i_color#24, i_units#25, i_manager_id#63, c_first_name#30, c_last_name#31, ca_state#35]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23]
+Functions [1]: [partial_sum(UnscaledValue(ss_net_paid#53))]
+Aggregate Attributes [1]: [sum#67]
+Results [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
(71) CometColumnarExchange
-Input [2]: [sum#69, count#70]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
+Arguments: hashpartitioning(c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=6]
+
+(72) CometColumnarToRow [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
+
+(73) HashAggregate [codegen id : 3]
+Input [11]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23, sum#68]
+Keys [10]: [c_last_name#31, c_first_name#30, s_store_name#57, ca_state#35, s_state#15, i_color#24, i_current_price#59, i_manager_id#63, i_units#25, i_size#23]
+Functions [1]: [sum(UnscaledValue(ss_net_paid#53))]
+Aggregate Attributes [1]: [sum(UnscaledValue(ss_net_paid#53))#39]
+Results [1]: [MakeDecimal(sum(UnscaledValue(ss_net_paid#53))#39,17,2) AS netpaid#69]
+
+(74) HashAggregate [codegen id : 3]
+Input [1]: [netpaid#69]
+Keys: []
+Functions [1]: [partial_avg(netpaid#69)]
+Aggregate Attributes [2]: [sum#70, count#71]
+Results [2]: [sum#72, count#73]
+
+(75) CometColumnarExchange
+Input [2]: [sum#72, count#73]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=7]
-(72) CometColumnarToRow [codegen id : 4]
-Input [2]: [sum#69, count#70]
+(76) CometColumnarToRow [codegen id : 4]
+Input [2]: [sum#72, count#73]
-(73) HashAggregate [codegen id : 4]
-Input [2]: [sum#69, count#70]
+(77) HashAggregate [codegen id : 4]
+Input [2]: [sum#72, count#73]
Keys: []
-Functions [1]: [avg(netpaid#66)]
-Aggregate Attributes [1]: [avg(netpaid#66)#71]
-Results [1]: [(0.05 * avg(netpaid#66)#71) AS (0.05 * avg(netpaid))#72]
+Functions [1]: [avg(netpaid#69)]
+Aggregate Attributes [1]: [avg(netpaid#69)#74]
+Results [1]: [(0.05 * avg(netpaid#69)#74) AS (0.05 * avg(netpaid))#75]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt
index df6657f73..fb62bdf3c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q24b/simplified.txt
@@ -32,8 +32,9 @@ WholeStageCodegen (4)
ReusedExchange [sr_item_sk,sr_ticket_number] #4
ReusedExchange [s_store_sk,s_store_name,s_state,s_zip] #5
CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #11
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [i_size,i_color,i_units] [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
ReusedExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
InputAdapter
ReusedExchange [ca_state,ca_zip,ca_country] #8
@@ -72,19 +73,22 @@ WholeStageCodegen (4)
CometFilter [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
CometScan parquet spark_catalog.default.store_returns [sr_item_sk,sr_ticket_number,sr_returned_date_sk]
CometBroadcastExchange [s_store_sk,s_store_name,s_state,s_zip] #5
- CometProject [s_store_sk,s_store_name,s_state,s_zip]
+ CometProject [s_state,s_zip] [s_store_sk,s_store_name,s_state,s_zip]
CometFilter [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
CometScan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_market_id,s_state,s_zip]
CometBroadcastExchange [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id] #6
- CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometProject [i_size,i_color,i_units] [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometFilter [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_current_price,i_size,i_color,i_units,i_manager_id]
CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name,c_birth_country] #7
- CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometProject [c_first_name,c_last_name] [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometFilter [c_customer_sk,c_first_name,c_last_name,c_birth_country]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name,c_birth_country]
InputAdapter
BroadcastExchange #8
WholeStageCodegen (1)
CometColumnarToRow
InputAdapter
- CometFilter [ca_state,ca_zip,ca_country]
- CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
+ CometProject [ca_state,ca_zip] [ca_state,ca_zip,ca_country]
+ CometFilter [ca_state,ca_zip,ca_country]
+ CometScan parquet spark_catalog.default.customer_address [ca_state,ca_zip,ca_country]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt
index 62c54c9f5..e63dc7818 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-* CometColumnarToRow (42)
-+- CometTakeOrderedAndProject (41)
- +- CometHashAggregate (40)
- +- CometExchange (39)
- +- CometHashAggregate (38)
- +- CometProject (37)
- +- CometBroadcastHashJoin (36)
- :- CometProject (32)
- : +- CometBroadcastHashJoin (31)
+* CometColumnarToRow (44)
++- CometTakeOrderedAndProject (43)
+ +- CometHashAggregate (42)
+ +- CometExchange (41)
+ +- CometHashAggregate (40)
+ +- CometProject (39)
+ +- CometBroadcastHashJoin (38)
+ :- CometProject (33)
+ : +- CometBroadcastHashJoin (32)
: :- CometProject (27)
: : +- CometBroadcastHashJoin (26)
: : :- CometProject (24)
@@ -35,12 +35,14 @@
: : : +- CometFilter (20)
: : : +- CometScan parquet spark_catalog.default.date_dim (19)
: : +- ReusedExchange (25)
- : +- CometBroadcastExchange (30)
- : +- CometFilter (29)
- : +- CometScan parquet spark_catalog.default.store (28)
- +- CometBroadcastExchange (35)
- +- CometFilter (34)
- +- CometScan parquet spark_catalog.default.item (33)
+ : +- CometBroadcastExchange (31)
+ : +- CometProject (30)
+ : +- CometFilter (29)
+ : +- CometScan parquet spark_catalog.default.store (28)
+ +- CometBroadcastExchange (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan parquet spark_catalog.default.item (34)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -184,123 +186,131 @@ ReadSchema: struct<s_store_sk:int,s_store_id:string,s_store_name:string>
Input [3]: [s_store_sk#26, s_store_id#27, s_store_name#28]
Condition : isnotnull(s_store_sk#26)
-(30) CometBroadcastExchange
+(30) CometProject
Input [3]: [s_store_sk#26, s_store_id#27, s_store_name#28]
-Arguments: [s_store_sk#26, s_store_id#27, s_store_name#28]
+Arguments: [s_store_sk#26, s_store_id#29, s_store_name#28], [s_store_sk#26, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_store_id#27, 16, true, false, true) AS s_store_id#29, s_store_name#28]
-(31) CometBroadcastHashJoin
+(31) CometBroadcastExchange
+Input [3]: [s_store_sk#26, s_store_id#29, s_store_name#28]
+Arguments: [s_store_sk#26, s_store_id#29, s_store_name#28]
+
+(32) CometBroadcastHashJoin
Left output [5]: [ss_item_sk#1, ss_store_sk#3, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16]
-Right output [3]: [s_store_sk#26, s_store_id#27, s_store_name#28]
+Right output [3]: [s_store_sk#26, s_store_id#29, s_store_name#28]
Arguments: [ss_store_sk#3], [s_store_sk#26], Inner, BuildRight
-(32) CometProject
-Input [8]: [ss_item_sk#1, ss_store_sk#3, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_sk#26, s_store_id#27, s_store_name#28]
-Arguments: [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28], [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28]
+(33) CometProject
+Input [8]: [ss_item_sk#1, ss_store_sk#3, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_sk#26, s_store_id#29, s_store_name#28]
+Arguments: [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28], [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28]
-(33) CometScan parquet spark_catalog.default.item
-Output [3]: [i_item_sk#29, i_item_id#30, i_item_desc#31]
+(34) CometScan parquet spark_catalog.default.item
+Output [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_item_id:string,i_item_desc:string>
-(34) CometFilter
-Input [3]: [i_item_sk#29, i_item_id#30, i_item_desc#31]
-Condition : isnotnull(i_item_sk#29)
+(35) CometFilter
+Input [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
+Condition : isnotnull(i_item_sk#30)
+
+(36) CometProject
+Input [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
+Arguments: [i_item_sk#30, i_item_id#33, i_item_desc#32], [i_item_sk#30, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#31, 16, true, false, true) AS i_item_id#33, i_item_desc#32]
-(35) CometBroadcastExchange
-Input [3]: [i_item_sk#29, i_item_id#30, i_item_desc#31]
-Arguments: [i_item_sk#29, i_item_id#30, i_item_desc#31]
+(37) CometBroadcastExchange
+Input [3]: [i_item_sk#30, i_item_id#33, i_item_desc#32]
+Arguments: [i_item_sk#30, i_item_id#33, i_item_desc#32]
-(36) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28]
-Right output [3]: [i_item_sk#29, i_item_id#30, i_item_desc#31]
-Arguments: [ss_item_sk#1], [i_item_sk#29], Inner, BuildRight
+(38) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28]
+Right output [3]: [i_item_sk#30, i_item_id#33, i_item_desc#32]
+Arguments: [ss_item_sk#1], [i_item_sk#30], Inner, BuildRight
-(37) CometProject
-Input [9]: [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28, i_item_sk#29, i_item_id#30, i_item_desc#31]
-Arguments: [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28, i_item_id#30, i_item_desc#31], [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28, i_item_id#30, i_item_desc#31]
+(39) CometProject
+Input [9]: [ss_item_sk#1, ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28, i_item_sk#30, i_item_id#33, i_item_desc#32]
+Arguments: [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28, i_item_id#33, i_item_desc#32], [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28, i_item_id#33, i_item_desc#32]
-(38) CometHashAggregate
-Input [7]: [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#27, s_store_name#28, i_item_id#30, i_item_desc#31]
-Keys [4]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28]
+(40) CometHashAggregate
+Input [7]: [ss_net_profit#5, sr_net_loss#11, cs_net_profit#16, s_store_id#29, s_store_name#28, i_item_id#33, i_item_desc#32]
+Keys [4]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28]
Functions [3]: [partial_sum(UnscaledValue(ss_net_profit#5)), partial_sum(UnscaledValue(sr_net_loss#11)), partial_sum(UnscaledValue(cs_net_profit#16))]
-(39) CometExchange
-Input [7]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, sum#32, sum#33, sum#34]
-Arguments: hashpartitioning(i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(41) CometExchange
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
+Arguments: hashpartitioning(i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(40) CometHashAggregate
-Input [7]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, sum#32, sum#33, sum#34]
-Keys [4]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28]
+(42) CometHashAggregate
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, sum#34, sum#35, sum#36]
+Keys [4]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28]
Functions [3]: [sum(UnscaledValue(ss_net_profit#5)), sum(UnscaledValue(sr_net_loss#11)), sum(UnscaledValue(cs_net_profit#16))]
-(41) CometTakeOrderedAndProject
-Input [7]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, store_sales_profit#35, store_returns_loss#36, catalog_sales_profit#37]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#30 ASC NULLS FIRST,i_item_desc#31 ASC NULLS FIRST,s_store_id#27 ASC NULLS FIRST,s_store_name#28 ASC NULLS FIRST], output=[i_item_id#30,i_item_desc#31,s_store_id#27,s_store_name#28,store_sales_profit#35,store_returns_loss#36,catalog_sales_profit#37]), [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, store_sales_profit#35, store_returns_loss#36, catalog_sales_profit#37], 100, [i_item_id#30 ASC NULLS FIRST, i_item_ [...]
+(43) CometTakeOrderedAndProject
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#33 ASC NULLS FIRST,i_item_desc#32 ASC NULLS FIRST,s_store_id#29 ASC NULLS FIRST,s_store_name#28 ASC NULLS FIRST], output=[i_item_id#33,i_item_desc#32,s_store_id#29,s_store_name#28,store_sales_profit#37,store_returns_loss#38,catalog_sales_profit#39]), [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39], 100, [i_item_id#33 ASC NULLS FIRST, i_item_ [...]
-(42) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_id#30, i_item_desc#31, s_store_id#27, s_store_name#28, store_sales_profit#35, store_returns_loss#36, catalog_sales_profit#37]
+(44) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_id#33, i_item_desc#32, s_store_id#29, s_store_name#28, store_sales_profit#37, store_returns_loss#38, catalog_sales_profit#39]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#6 IN dynamicpruning#7
-BroadcastExchange (47)
-+- * CometColumnarToRow (46)
- +- CometProject (45)
- +- CometFilter (44)
- +- CometScan parquet spark_catalog.default.date_dim (43)
+BroadcastExchange (49)
++- * CometColumnarToRow (48)
+ +- CometProject (47)
+ +- CometFilter (46)
+ +- CometScan parquet spark_catalog.default.date_dim (45)
-(43) CometScan parquet spark_catalog.default.date_dim
+(45) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#19, d_year#20, d_moy#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_moy), IsNotNull(d_year), EqualTo(d_moy,4), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(44) CometFilter
+(46) CometFilter
Input [3]: [d_date_sk#19, d_year#20, d_moy#21]
Condition : ((((isnotnull(d_moy#21) AND isnotnull(d_year#20)) AND (d_moy#21 = 4)) AND (d_year#20 = 2001)) AND isnotnull(d_date_sk#19))
-(45) CometProject
+(47) CometProject
Input [3]: [d_date_sk#19, d_year#20, d_moy#21]
Arguments: [d_date_sk#19], [d_date_sk#19]
-(46) CometColumnarToRow [codegen id : 1]
+(48) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#19]
-(47) BroadcastExchange
+(49) BroadcastExchange
Input [1]: [d_date_sk#19]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
Subquery:2 Hosting operator id = 3 Hosting Expression = sr_returned_date_sk#12 IN dynamicpruning#13
-BroadcastExchange (52)
-+- * CometColumnarToRow (51)
- +- CometProject (50)
- +- CometFilter (49)
- +- CometScan parquet spark_catalog.default.date_dim (48)
+BroadcastExchange (54)
++- * CometColumnarToRow (53)
+ +- CometProject (52)
+ +- CometFilter (51)
+ +- CometScan parquet spark_catalog.default.date_dim (50)
-(48) CometScan parquet spark_catalog.default.date_dim
+(50) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#22, d_year#23, d_moy#24]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_moy), IsNotNull(d_year), GreaterThanOrEqual(d_moy,4), LessThanOrEqual(d_moy,10), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(49) CometFilter
+(51) CometFilter
Input [3]: [d_date_sk#22, d_year#23, d_moy#24]
Condition : (((((isnotnull(d_moy#24) AND isnotnull(d_year#23)) AND (d_moy#24 >= 4)) AND (d_moy#24 <= 10)) AND (d_year#23 = 2001)) AND isnotnull(d_date_sk#22))
-(50) CometProject
+(52) CometProject
Input [3]: [d_date_sk#22, d_year#23, d_moy#24]
Arguments: [d_date_sk#22], [d_date_sk#22]
-(51) CometColumnarToRow [codegen id : 1]
+(53) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#22]
-(52) BroadcastExchange
+(54) BroadcastExchange
Input [1]: [d_date_sk#22]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt
index e64e8bc84..b82d177de 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q25/simplified.txt
@@ -54,8 +54,10 @@ WholeStageCodegen (1)
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
ReusedExchange [d_date_sk] #7
CometBroadcastExchange [s_store_sk,s_store_id,s_store_name] #8
- CometFilter [s_store_sk,s_store_id,s_store_name]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
+ CometProject [s_store_id] [s_store_sk,s_store_id,s_store_name]
+ CometFilter [s_store_sk,s_store_id,s_store_name]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc] #9
- CometFilter [i_item_sk,i_item_id,i_item_desc]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
+ CometProject [i_item_id] [i_item_sk,i_item_id,i_item_desc]
+ CometFilter [i_item_sk,i_item_id,i_item_desc]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt
index 2c6cde0c2..a065e1234 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-* CometColumnarToRow (30)
-+- CometTakeOrderedAndProject (29)
- +- CometHashAggregate (28)
- +- CometExchange (27)
- +- CometHashAggregate (26)
- +- CometProject (25)
- +- CometBroadcastHashJoin (24)
- :- CometProject (19)
- : +- CometBroadcastHashJoin (18)
+* CometColumnarToRow (31)
++- CometTakeOrderedAndProject (30)
+ +- CometHashAggregate (29)
+ +- CometExchange (28)
+ +- CometHashAggregate (27)
+ +- CometProject (26)
+ +- CometBroadcastHashJoin (25)
+ :- CometProject (20)
+ : +- CometBroadcastHashJoin (19)
: :- CometProject (14)
: : +- CometBroadcastHashJoin (13)
: : :- CometProject (8)
@@ -22,13 +22,14 @@
: : +- CometProject (11)
: : +- CometFilter (10)
: : +- CometScan parquet spark_catalog.default.date_dim (9)
- : +- CometBroadcastExchange (17)
- : +- CometFilter (16)
- : +- CometScan parquet spark_catalog.default.item (15)
- +- CometBroadcastExchange (23)
- +- CometProject (22)
- +- CometFilter (21)
- +- CometScan parquet spark_catalog.default.promotion (20)
+ : +- CometBroadcastExchange (18)
+ : +- CometProject (17)
+ : +- CometFilter (16)
+ : +- CometScan parquet spark_catalog.default.item (15)
+ +- CometBroadcastExchange (24)
+ +- CometProject (23)
+ +- CometFilter (22)
+ +- CometScan parquet spark_catalog.default.promotion (21)
(1) CometScan parquet spark_catalog.default.catalog_sales
@@ -47,12 +48,12 @@ Condition : ((isnotnull(cs_bill_cdemo_sk#1) AND isnotnull(cs_item_sk#2)) AND isn
Output [4]: [cd_demo_sk#10, cd_gender#11, cd_marital_status#12, cd_education_status#13]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_demographics]
-PushedFilters: [IsNotNull(cd_gender), IsNotNull(cd_marital_status), IsNotNull(cd_education_status), IsNotNull(cd_demo_sk)]
+PushedFilters: [IsNotNull(cd_demo_sk)]
ReadSchema: struct<cd_demo_sk:int,cd_gender:string,cd_marital_status:string,cd_education_status:string>
(4) CometFilter
Input [4]: [cd_demo_sk#10, cd_gender#11, cd_marital_status#12, cd_education_status#13]
-Condition : ((((((isnotnull(cd_gender#11) AND isnotnull(cd_marital_status#12)) AND isnotnull(cd_education_status#13)) AND (rpad(cd_gender#11, 1, ) = M)) AND (rpad(cd_marital_status#12, 1, ) = S)) AND (rpad(cd_education_status#13, 20, ) = College )) AND isnotnull(cd_demo_sk#10))
+Condition : ((((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_gender#11, 1, true, false, true) = M) AND (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_marital_status#12, 1, true, false, true) = S)) AND (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_education_status#13, 20, true, false, true) = Co [...]
(5) CometProject
Input [4]: [cd_demo_sk#10, cd_gender#11, cd_marital_status#12, cd_education_status#13]
@@ -110,97 +111,101 @@ ReadSchema: struct<i_item_sk:int,i_item_id:string>
Input [2]: [i_item_sk#16, i_item_id#17]
Condition : isnotnull(i_item_sk#16)
-(17) CometBroadcastExchange
+(17) CometProject
Input [2]: [i_item_sk#16, i_item_id#17]
-Arguments: [i_item_sk#16, i_item_id#17]
+Arguments: [i_item_sk#16, i_item_id#18], [i_item_sk#16, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#17, 16, true, false, true) AS i_item_id#18]
-(18) CometBroadcastHashJoin
+(18) CometBroadcastExchange
+Input [2]: [i_item_sk#16, i_item_id#18]
+Arguments: [i_item_sk#16, i_item_id#18]
+
+(19) CometBroadcastHashJoin
Left output [6]: [cs_item_sk#2, cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7]
-Right output [2]: [i_item_sk#16, i_item_id#17]
+Right output [2]: [i_item_sk#16, i_item_id#18]
Arguments: [cs_item_sk#2], [i_item_sk#16], Inner, BuildRight
-(19) CometProject
-Input [8]: [cs_item_sk#2, cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_sk#16, i_item_id#17]
-Arguments: [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17], [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17]
+(20) CometProject
+Input [8]: [cs_item_sk#2, cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_sk#16, i_item_id#18]
+Arguments: [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18], [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18]
-(20) CometScan parquet spark_catalog.default.promotion
-Output [3]: [p_promo_sk#18, p_channel_email#19, p_channel_event#20]
+(21) CometScan parquet spark_catalog.default.promotion
+Output [3]: [p_promo_sk#19, p_channel_email#20, p_channel_event#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/promotion]
PushedFilters: [IsNotNull(p_promo_sk)]
ReadSchema: struct<p_promo_sk:int,p_channel_email:string,p_channel_event:string>
-(21) CometFilter
-Input [3]: [p_promo_sk#18, p_channel_email#19, p_channel_event#20]
-Condition : (((rpad(p_channel_email#19, 1, ) = N) OR (rpad(p_channel_event#20, 1, ) = N)) AND isnotnull(p_promo_sk#18))
+(22) CometFilter
+Input [3]: [p_promo_sk#19, p_channel_email#20, p_channel_event#21]
+Condition : (((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, p_channel_email#20, 1, true, false, true) = N) OR (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, p_channel_event#21, 1, true, false, true) = N)) AND isnotnull(p_promo_sk#19))
-(22) CometProject
-Input [3]: [p_promo_sk#18, p_channel_email#19, p_channel_event#20]
-Arguments: [p_promo_sk#18], [p_promo_sk#18]
+(23) CometProject
+Input [3]: [p_promo_sk#19, p_channel_email#20, p_channel_event#21]
+Arguments: [p_promo_sk#19], [p_promo_sk#19]
-(23) CometBroadcastExchange
-Input [1]: [p_promo_sk#18]
-Arguments: [p_promo_sk#18]
+(24) CometBroadcastExchange
+Input [1]: [p_promo_sk#19]
+Arguments: [p_promo_sk#19]
-(24) CometBroadcastHashJoin
-Left output [6]: [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17]
-Right output [1]: [p_promo_sk#18]
-Arguments: [cs_promo_sk#3], [p_promo_sk#18], Inner, BuildRight
+(25) CometBroadcastHashJoin
+Left output [6]: [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18]
+Right output [1]: [p_promo_sk#19]
+Arguments: [cs_promo_sk#3], [p_promo_sk#19], Inner, BuildRight
-(25) CometProject
-Input [7]: [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17, p_promo_sk#18]
-Arguments: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17]
+(26) CometProject
+Input [7]: [cs_promo_sk#3, cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18, p_promo_sk#19]
+Arguments: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18], [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18]
-(26) CometHashAggregate
-Input [5]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#17]
-Keys [1]: [i_item_id#17]
+(27) CometHashAggregate
+Input [5]: [cs_quantity#4, cs_list_price#5, cs_sales_price#6, cs_coupon_amt#7, i_item_id#18]
+Keys [1]: [i_item_id#18]
Functions [4]: [partial_avg(cs_quantity#4), partial_avg(UnscaledValue(cs_list_price#5)), partial_avg(UnscaledValue(cs_coupon_amt#7)), partial_avg(UnscaledValue(cs_sales_price#6))]
-(27) CometExchange
-Input [9]: [i_item_id#17, sum#21, count#22, sum#23, count#24, sum#25, count#26, sum#27, count#28]
-Arguments: hashpartitioning(i_item_id#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(28) CometExchange
+Input [9]: [i_item_id#18, sum#22, count#23, sum#24, count#25, sum#26, count#27, sum#28, count#29]
+Arguments: hashpartitioning(i_item_id#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(28) CometHashAggregate
-Input [9]: [i_item_id#17, sum#21, count#22, sum#23, count#24, sum#25, count#26, sum#27, count#28]
-Keys [1]: [i_item_id#17]
+(29) CometHashAggregate
+Input [9]: [i_item_id#18, sum#22, count#23, sum#24, count#25, sum#26, count#27, sum#28, count#29]
+Keys [1]: [i_item_id#18]
Functions [4]: [avg(cs_quantity#4), avg(UnscaledValue(cs_list_price#5)), avg(UnscaledValue(cs_coupon_amt#7)), avg(UnscaledValue(cs_sales_price#6))]
-(29) CometTakeOrderedAndProject
-Input [5]: [i_item_id#17, agg1#29, agg2#30, agg3#31, agg4#32]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#17 ASC NULLS FIRST], output=[i_item_id#17,agg1#29,agg2#30,agg3#31,agg4#32]), [i_item_id#17, agg1#29, agg2#30, agg3#31, agg4#32], 100, [i_item_id#17 ASC NULLS FIRST], [i_item_id#17, agg1#29, agg2#30, agg3#31, agg4#32]
+(30) CometTakeOrderedAndProject
+Input [5]: [i_item_id#18, agg1#30, agg2#31, agg3#32, agg4#33]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#18 ASC NULLS FIRST], output=[i_item_id#18,agg1#30,agg2#31,agg3#32,agg4#33]), [i_item_id#18, agg1#30, agg2#31, agg3#32, agg4#33], 100, [i_item_id#18 ASC NULLS FIRST], [i_item_id#18, agg1#30, agg2#31, agg3#32, agg4#33]
-(30) CometColumnarToRow [codegen id : 1]
-Input [5]: [i_item_id#17, agg1#29, agg2#30, agg3#31, agg4#32]
+(31) CometColumnarToRow [codegen id : 1]
+Input [5]: [i_item_id#18, agg1#30, agg2#31, agg3#32, agg4#33]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = cs_sold_date_sk#8 IN dynamicpruning#9
-BroadcastExchange (35)
-+- * CometColumnarToRow (34)
- +- CometProject (33)
- +- CometFilter (32)
- +- CometScan parquet spark_catalog.default.date_dim (31)
+BroadcastExchange (36)
++- * CometColumnarToRow (35)
+ +- CometProject (34)
+ +- CometFilter (33)
+ +- CometScan parquet spark_catalog.default.date_dim (32)
-(31) CometScan parquet spark_catalog.default.date_dim
+(32) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#14, d_year#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2000), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(32) CometFilter
+(33) CometFilter
Input [2]: [d_date_sk#14, d_year#15]
Condition : ((isnotnull(d_year#15) AND (d_year#15 = 2000)) AND isnotnull(d_date_sk#14))
-(33) CometProject
+(34) CometProject
Input [2]: [d_date_sk#14, d_year#15]
Arguments: [d_date_sk#14], [d_date_sk#14]
-(34) CometColumnarToRow [codegen id : 1]
+(35) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#14]
-(35) BroadcastExchange
+(36) BroadcastExchange
Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt
index 16f8d0848..0564bab69 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q26/simplified.txt
@@ -32,8 +32,9 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [i_item_sk,i_item_id] #5
- CometFilter [i_item_sk,i_item_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
+ CometProject [i_item_id] [i_item_sk,i_item_id]
+ CometFilter [i_item_sk,i_item_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
CometBroadcastExchange [p_promo_sk] #6
CometProject [p_promo_sk]
CometFilter [p_promo_sk,p_channel_email,p_channel_event]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt
index b0f1de0f1..65443cbc1 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/explain.txt
@@ -1,14 +1,14 @@
== Physical Plan ==
-* CometColumnarToRow (30)
-+- CometTakeOrderedAndProject (29)
- +- CometHashAggregate (28)
- +- CometExchange (27)
- +- CometHashAggregate (26)
- +- CometExpand (25)
- +- CometProject (24)
- +- CometBroadcastHashJoin (23)
- :- CometProject (19)
- : +- CometBroadcastHashJoin (18)
+* CometColumnarToRow (32)
++- CometTakeOrderedAndProject (31)
+ +- CometHashAggregate (30)
+ +- CometExchange (29)
+ +- CometHashAggregate (28)
+ +- CometExpand (27)
+ +- CometProject (26)
+ +- CometBroadcastHashJoin (25)
+ :- CometProject (20)
+ : +- CometBroadcastHashJoin (19)
: :- CometProject (14)
: : +- CometBroadcastHashJoin (13)
: : :- CometProject (8)
@@ -23,12 +23,14 @@
: : +- CometProject (11)
: : +- CometFilter (10)
: : +- CometScan parquet spark_catalog.default.date_dim (9)
- : +- CometBroadcastExchange (17)
- : +- CometFilter (16)
- : +- CometScan parquet spark_catalog.default.store (15)
- +- CometBroadcastExchange (22)
- +- CometFilter (21)
- +- CometScan parquet spark_catalog.default.item (20)
+ : +- CometBroadcastExchange (18)
+ : +- CometProject (17)
+ : +- CometFilter (16)
+ : +- CometScan parquet spark_catalog.default.store (15)
+ +- CometBroadcastExchange (24)
+ +- CometProject (23)
+ +- CometFilter (22)
+ +- CometScan parquet spark_catalog.default.item (21)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -47,12 +49,12 @@ Condition : ((isnotnull(ss_cdemo_sk#2) AND isnotnull(ss_store_sk#3)) AND isnotnu
Output [4]: [cd_demo_sk#10, cd_gender#11, cd_marital_status#12, cd_education_status#13]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_demographics]
-PushedFilters: [IsNotNull(cd_gender), IsNotNull(cd_marital_status), IsNotNull(cd_education_status), IsNotNull(cd_demo_sk)]
+PushedFilters: [IsNotNull(cd_demo_sk)]
ReadSchema: struct<cd_demo_sk:int,cd_gender:string,cd_marital_status:string,cd_education_status:string>
(4) CometFilter
Input [4]: [cd_demo_sk#10, cd_gender#11, cd_marital_status#12, cd_education_status#13]
-Condition : ((((((isnotnull(cd_gender#11) AND isnotnull(cd_marital_status#12)) AND isnotnull(cd_education_status#13)) AND (rpad(cd_gender#11, 1, ) = M)) AND (rpad(cd_marital_status#12, 1, ) = S)) AND (rpad(cd_education_status#13, 20, ) = College )) AND isnotnull(cd_demo_sk#10))
+Condition : ((((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_gender#11, 1, true, false, true) = M) AND (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_marital_status#12, 1, true, false, true) = S)) AND (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_education_status#13, 20, true, false, true) = Co [...]
(5) CometProject
Input [4]: [cd_demo_sk#10, cd_gender#11, cd_marital_status#12, cd_education_status#13]
@@ -103,104 +105,112 @@ Arguments: [ss_item_sk#1, ss_store_sk#3, ss_quantity#4, ss_list_price#5, ss_sale
Output [2]: [s_store_sk#16, s_state#17]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
-PushedFilters: [IsNotNull(s_state), IsNotNull(s_store_sk)]
+PushedFilters: [IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_state:string>
(16) CometFilter
Input [2]: [s_store_sk#16, s_state#17]
-Condition : ((isnotnull(s_state#17) AND (rpad(s_state#17, 2, ) = TN)) AND isnotnull(s_store_sk#16))
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#17, 2, true, false, true) = TN) AND isnotnull(s_store_sk#16))
-(17) CometBroadcastExchange
+(17) CometProject
Input [2]: [s_store_sk#16, s_state#17]
-Arguments: [s_store_sk#16, s_state#17]
+Arguments: [s_store_sk#16, s_state#18], [s_store_sk#16, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#17, 2, true, false, true) AS s_state#18]
-(18) CometBroadcastHashJoin
+(18) CometBroadcastExchange
+Input [2]: [s_store_sk#16, s_state#18]
+Arguments: [s_store_sk#16, s_state#18]
+
+(19) CometBroadcastHashJoin
Left output [6]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7]
-Right output [2]: [s_store_sk#16, s_state#17]
+Right output [2]: [s_store_sk#16, s_state#18]
Arguments: [ss_store_sk#3], [s_store_sk#16], Inner, BuildRight
-(19) CometProject
-Input [8]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_store_sk#16, s_state#17]
-Arguments: [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#17], [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#17]
+(20) CometProject
+Input [8]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_store_sk#16, s_state#18]
+Arguments: [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#18], [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#18]
-(20) CometScan parquet spark_catalog.default.item
-Output [2]: [i_item_sk#18, i_item_id#19]
+(21) CometScan parquet spark_catalog.default.item
+Output [2]: [i_item_sk#19, i_item_id#20]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_item_id:string>
-(21) CometFilter
-Input [2]: [i_item_sk#18, i_item_id#19]
-Condition : isnotnull(i_item_sk#18)
+(22) CometFilter
+Input [2]: [i_item_sk#19, i_item_id#20]
+Condition : isnotnull(i_item_sk#19)
+
+(23) CometProject
+Input [2]: [i_item_sk#19, i_item_id#20]
+Arguments: [i_item_sk#19, i_item_id#21], [i_item_sk#19, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#20, 16, true, false, true) AS i_item_id#21]
-(22) CometBroadcastExchange
-Input [2]: [i_item_sk#18, i_item_id#19]
-Arguments: [i_item_sk#18, i_item_id#19]
+(24) CometBroadcastExchange
+Input [2]: [i_item_sk#19, i_item_id#21]
+Arguments: [i_item_sk#19, i_item_id#21]
-(23) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#17]
-Right output [2]: [i_item_sk#18, i_item_id#19]
-Arguments: [ss_item_sk#1], [i_item_sk#18], Inner, BuildRight
+(25) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#18]
+Right output [2]: [i_item_sk#19, i_item_id#21]
+Arguments: [ss_item_sk#1], [i_item_sk#19], Inner, BuildRight
-(24) CometProject
-Input [8]: [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#17, i_item_sk#18, i_item_id#19]
-Arguments: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#19, s_state#17], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#19, s_state#17]
+(26) CometProject
+Input [8]: [ss_item_sk#1, ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, s_state#18, i_item_sk#19, i_item_id#21]
+Arguments: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#21, s_state#18], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#21, s_state#18]
-(25) CometExpand
-Input [6]: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#19, s_state#17]
-Arguments: [[ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#19, s_state#17, 0], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#19, null, 1], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, null, null, 3]], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#20, s_state#21, spark_grouping_id#22]
+(27) CometExpand
+Input [6]: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#21, s_state#18]
+Arguments: [[ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#21, s_state#18, 0], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#21, null, 1], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, null, null, 3]], [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#22, s_state#23, spark_grouping_id#24]
-(26) CometHashAggregate
-Input [7]: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#20, s_state#21, spark_grouping_id#22]
-Keys [3]: [i_item_id#20, s_state#21, spark_grouping_id#22]
+(28) CometHashAggregate
+Input [7]: [ss_quantity#4, ss_list_price#5, ss_sales_price#6, ss_coupon_amt#7, i_item_id#22, s_state#23, spark_grouping_id#24]
+Keys [3]: [i_item_id#22, s_state#23, spark_grouping_id#24]
Functions [4]: [partial_avg(ss_quantity#4), partial_avg(UnscaledValue(ss_list_price#5)), partial_avg(UnscaledValue(ss_coupon_amt#7)), partial_avg(UnscaledValue(ss_sales_price#6))]
-(27) CometExchange
-Input [11]: [i_item_id#20, s_state#21, spark_grouping_id#22, sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29, count#30]
-Arguments: hashpartitioning(i_item_id#20, s_state#21, spark_grouping_id#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(29) CometExchange
+Input [11]: [i_item_id#22, s_state#23, spark_grouping_id#24, sum#25, count#26, sum#27, count#28, sum#29, count#30, sum#31, count#32]
+Arguments: hashpartitioning(i_item_id#22, s_state#23, spark_grouping_id#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(28) CometHashAggregate
-Input [11]: [i_item_id#20, s_state#21, spark_grouping_id#22, sum#23, count#24, sum#25, count#26, sum#27, count#28, sum#29, count#30]
-Keys [3]: [i_item_id#20, s_state#21, spark_grouping_id#22]
+(30) CometHashAggregate
+Input [11]: [i_item_id#22, s_state#23, spark_grouping_id#24, sum#25, count#26, sum#27, count#28, sum#29, count#30, sum#31, count#32]
+Keys [3]: [i_item_id#22, s_state#23, spark_grouping_id#24]
Functions [4]: [avg(ss_quantity#4), avg(UnscaledValue(ss_list_price#5)), avg(UnscaledValue(ss_coupon_amt#7)), avg(UnscaledValue(ss_sales_price#6))]
-(29) CometTakeOrderedAndProject
-Input [7]: [i_item_id#20, s_state#21, g_state#31, agg1#32, agg2#33, agg3#34, agg4#35]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#20 ASC NULLS FIRST,s_state#21 ASC NULLS FIRST], output=[i_item_id#20,s_state#21,g_state#31,agg1#32,agg2#33,agg3#34,agg4#35]), [i_item_id#20, s_state#21, g_state#31, agg1#32, agg2#33, agg3#34, agg4#35], 100, [i_item_id#20 ASC NULLS FIRST, s_state#21 ASC NULLS FIRST], [i_item_id#20, s_state#21, g_state#31, agg1#32, agg2#33, agg3#34, agg4#35]
+(31) CometTakeOrderedAndProject
+Input [7]: [i_item_id#22, s_state#23, g_state#33, agg1#34, agg2#35, agg3#36, agg4#37]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#22 ASC NULLS FIRST,s_state#23 ASC NULLS FIRST], output=[i_item_id#22,s_state#23,g_state#33,agg1#34,agg2#35,agg3#36,agg4#37]), [i_item_id#22, s_state#23, g_state#33, agg1#34, agg2#35, agg3#36, agg4#37], 100, [i_item_id#22 ASC NULLS FIRST, s_state#23 ASC NULLS FIRST], [i_item_id#22, s_state#23, g_state#33, agg1#34, agg2#35, agg3#36, agg4#37]
-(30) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_id#20, s_state#21, g_state#31, agg1#32, agg2#33, agg3#34, agg4#35]
+(32) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_id#22, s_state#23, g_state#33, agg1#34, agg2#35, agg3#36, agg4#37]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#8 IN dynamicpruning#9
-BroadcastExchange (35)
-+- * CometColumnarToRow (34)
- +- CometProject (33)
- +- CometFilter (32)
- +- CometScan parquet spark_catalog.default.date_dim (31)
+BroadcastExchange (37)
++- * CometColumnarToRow (36)
+ +- CometProject (35)
+ +- CometFilter (34)
+ +- CometScan parquet spark_catalog.default.date_dim (33)
-(31) CometScan parquet spark_catalog.default.date_dim
+(33) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#14, d_year#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(32) CometFilter
+(34) CometFilter
Input [2]: [d_date_sk#14, d_year#15]
Condition : ((isnotnull(d_year#15) AND (d_year#15 = 2002)) AND isnotnull(d_date_sk#14))
-(33) CometProject
+(35) CometProject
Input [2]: [d_date_sk#14, d_year#15]
Arguments: [d_date_sk#14], [d_date_sk#14]
-(34) CometColumnarToRow [codegen id : 1]
+(36) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#14]
-(35) BroadcastExchange
+(37) BroadcastExchange
Input [1]: [d_date_sk#14]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt
index e65b02d4d..a95181dd4 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q27/simplified.txt
@@ -33,8 +33,10 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [s_store_sk,s_state] #5
- CometFilter [s_store_sk,s_state]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_state]
+ CometProject [s_state] [s_store_sk,s_state]
+ CometFilter [s_store_sk,s_state]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_state]
CometBroadcastExchange [i_item_sk,i_item_id] #6
- CometFilter [i_item_sk,i_item_id]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
+ CometProject [i_item_id] [i_item_sk,i_item_id]
+ CometFilter [i_item_sk,i_item_id]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt
index b02bf3a4c..16d5abf00 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-* CometColumnarToRow (45)
-+- CometTakeOrderedAndProject (44)
- +- CometHashAggregate (43)
- +- CometExchange (42)
- +- CometHashAggregate (41)
- +- CometProject (40)
- +- CometBroadcastHashJoin (39)
- :- CometProject (35)
- : +- CometBroadcastHashJoin (34)
+* CometColumnarToRow (47)
++- CometTakeOrderedAndProject (46)
+ +- CometHashAggregate (45)
+ +- CometExchange (44)
+ +- CometHashAggregate (43)
+ +- CometProject (42)
+ +- CometBroadcastHashJoin (41)
+ :- CometProject (36)
+ : +- CometBroadcastHashJoin (35)
: :- CometProject (30)
: : +- CometBroadcastHashJoin (29)
: : :- CometProject (24)
@@ -38,12 +38,14 @@
: : +- CometProject (27)
: : +- CometFilter (26)
: : +- CometScan parquet spark_catalog.default.date_dim (25)
- : +- CometBroadcastExchange (33)
- : +- CometFilter (32)
- : +- CometScan parquet spark_catalog.default.store (31)
- +- CometBroadcastExchange (38)
- +- CometFilter (37)
- +- CometScan parquet spark_catalog.default.item (36)
+ : +- CometBroadcastExchange (34)
+ : +- CometProject (33)
+ : +- CometFilter (32)
+ : +- CometScan parquet spark_catalog.default.store (31)
+ +- CometBroadcastExchange (40)
+ +- CometProject (39)
+ +- CometFilter (38)
+ +- CometScan parquet spark_catalog.default.item (37)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -203,153 +205,161 @@ ReadSchema: struct<s_store_sk:int,s_store_id:string,s_store_name:string>
Input [3]: [s_store_sk#27, s_store_id#28, s_store_name#29]
Condition : isnotnull(s_store_sk#27)
-(33) CometBroadcastExchange
+(33) CometProject
Input [3]: [s_store_sk#27, s_store_id#28, s_store_name#29]
-Arguments: [s_store_sk#27, s_store_id#28, s_store_name#29]
+Arguments: [s_store_sk#27, s_store_id#30, s_store_name#29], [s_store_sk#27, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_store_id#28, 16, true, false, true) AS s_store_id#30, s_store_name#29]
-(34) CometBroadcastHashJoin
+(34) CometBroadcastExchange
+Input [3]: [s_store_sk#27, s_store_id#30, s_store_name#29]
+Arguments: [s_store_sk#27, s_store_id#30, s_store_name#29]
+
+(35) CometBroadcastHashJoin
Left output [5]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, sr_return_quantity#11, cs_quantity#16]
-Right output [3]: [s_store_sk#27, s_store_id#28, s_store_name#29]
+Right output [3]: [s_store_sk#27, s_store_id#30, s_store_name#29]
Arguments: [ss_store_sk#3], [s_store_sk#27], Inner, BuildRight
-(35) CometProject
-Input [8]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_sk#27, s_store_id#28, s_store_name#29]
-Arguments: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29], [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29]
+(36) CometProject
+Input [8]: [ss_item_sk#1, ss_store_sk#3, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_sk#27, s_store_id#30, s_store_name#29]
+Arguments: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29], [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29]
-(36) CometScan parquet spark_catalog.default.item
-Output [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
+(37) CometScan parquet spark_catalog.default.item
+Output [3]: [i_item_sk#31, i_item_id#32, i_item_desc#33]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
PushedFilters: [IsNotNull(i_item_sk)]
ReadSchema: struct<i_item_sk:int,i_item_id:string,i_item_desc:string>
-(37) CometFilter
-Input [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
-Condition : isnotnull(i_item_sk#30)
+(38) CometFilter
+Input [3]: [i_item_sk#31, i_item_id#32, i_item_desc#33]
+Condition : isnotnull(i_item_sk#31)
+
+(39) CometProject
+Input [3]: [i_item_sk#31, i_item_id#32, i_item_desc#33]
+Arguments: [i_item_sk#31, i_item_id#34, i_item_desc#33], [i_item_sk#31, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#32, 16, true, false, true) AS i_item_id#34, i_item_desc#33]
-(38) CometBroadcastExchange
-Input [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
-Arguments: [i_item_sk#30, i_item_id#31, i_item_desc#32]
+(40) CometBroadcastExchange
+Input [3]: [i_item_sk#31, i_item_id#34, i_item_desc#33]
+Arguments: [i_item_sk#31, i_item_id#34, i_item_desc#33]
-(39) CometBroadcastHashJoin
-Left output [6]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29]
-Right output [3]: [i_item_sk#30, i_item_id#31, i_item_desc#32]
-Arguments: [ss_item_sk#1], [i_item_sk#30], Inner, BuildRight
+(41) CometBroadcastHashJoin
+Left output [6]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29]
+Right output [3]: [i_item_sk#31, i_item_id#34, i_item_desc#33]
+Arguments: [ss_item_sk#1], [i_item_sk#31], Inner, BuildRight
-(40) CometProject
-Input [9]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29, i_item_sk#30, i_item_id#31, i_item_desc#32]
-Arguments: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29, i_item_id#31, i_item_desc#32], [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29, i_item_id#31, i_item_desc#32]
+(42) CometProject
+Input [9]: [ss_item_sk#1, ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29, i_item_sk#31, i_item_id#34, i_item_desc#33]
+Arguments: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29, i_item_id#34, i_item_desc#33], [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29, i_item_id#34, i_item_desc#33]
-(41) CometHashAggregate
-Input [7]: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#28, s_store_name#29, i_item_id#31, i_item_desc#32]
-Keys [4]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29]
+(43) CometHashAggregate
+Input [7]: [ss_quantity#5, sr_return_quantity#11, cs_quantity#16, s_store_id#30, s_store_name#29, i_item_id#34, i_item_desc#33]
+Keys [4]: [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29]
Functions [3]: [partial_sum(ss_quantity#5), partial_sum(sr_return_quantity#11), partial_sum(cs_quantity#16)]
-(42) CometExchange
-Input [7]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, sum#33, sum#34, sum#35]
-Arguments: hashpartitioning(i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(44) CometExchange
+Input [7]: [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29, sum#35, sum#36, sum#37]
+Arguments: hashpartitioning(i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(43) CometHashAggregate
-Input [7]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, sum#33, sum#34, sum#35]
-Keys [4]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29]
+(45) CometHashAggregate
+Input [7]: [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29, sum#35, sum#36, sum#37]
+Keys [4]: [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29]
Functions [3]: [sum(ss_quantity#5), sum(sr_return_quantity#11), sum(cs_quantity#16)]
-(44) CometTakeOrderedAndProject
-Input [7]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, store_sales_quantity#36, store_returns_quantity#37, catalog_sales_quantity#38]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#31 ASC NULLS FIRST,i_item_desc#32 ASC NULLS FIRST,s_store_id#28 ASC NULLS FIRST,s_store_name#29 ASC NULLS FIRST], output=[i_item_id#31,i_item_desc#32,s_store_id#28,s_store_name#29,store_sales_quantity#36,store_returns_quantity#37,catalog_sales_quantity#38]), [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, store_sales_quantity#36, store_returns_quantity#37, catalog_sales_quantity#38], 100, [i_item_id#31 ASC NULL [...]
+(46) CometTakeOrderedAndProject
+Input [7]: [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29, store_sales_quantity#38, store_returns_quantity#39, catalog_sales_quantity#40]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#34 ASC NULLS FIRST,i_item_desc#33 ASC NULLS FIRST,s_store_id#30 ASC NULLS FIRST,s_store_name#29 ASC NULLS FIRST], output=[i_item_id#34,i_item_desc#33,s_store_id#30,s_store_name#29,store_sales_quantity#38,store_returns_quantity#39,catalog_sales_quantity#40]), [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29, store_sales_quantity#38, store_returns_quantity#39, catalog_sales_quantity#40], 100, [i_item_id#34 ASC NULL [...]
-(45) CometColumnarToRow [codegen id : 1]
-Input [7]: [i_item_id#31, i_item_desc#32, s_store_id#28, s_store_name#29, store_sales_quantity#36, store_returns_quantity#37, catalog_sales_quantity#38]
+(47) CometColumnarToRow [codegen id : 1]
+Input [7]: [i_item_id#34, i_item_desc#33, s_store_id#30, s_store_name#29, store_sales_quantity#38, store_returns_quantity#39, catalog_sales_quantity#40]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#6 IN dynamicpruning#7
-BroadcastExchange (50)
-+- * CometColumnarToRow (49)
- +- CometProject (48)
- +- CometFilter (47)
- +- CometScan parquet spark_catalog.default.date_dim (46)
+BroadcastExchange (52)
++- * CometColumnarToRow (51)
+ +- CometProject (50)
+ +- CometFilter (49)
+ +- CometScan parquet spark_catalog.default.date_dim (48)
-(46) CometScan parquet spark_catalog.default.date_dim
+(48) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#19, d_year#20, d_moy#21]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_moy), IsNotNull(d_year), EqualTo(d_moy,9), EqualTo(d_year,1999), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(47) CometFilter
+(49) CometFilter
Input [3]: [d_date_sk#19, d_year#20, d_moy#21]
Condition : ((((isnotnull(d_moy#21) AND isnotnull(d_year#20)) AND (d_moy#21 = 9)) AND (d_year#20 = 1999)) AND isnotnull(d_date_sk#19))
-(48) CometProject
+(50) CometProject
Input [3]: [d_date_sk#19, d_year#20, d_moy#21]
Arguments: [d_date_sk#19], [d_date_sk#19]
-(49) CometColumnarToRow [codegen id : 1]
+(51) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#19]
-(50) BroadcastExchange
+(52) BroadcastExchange
Input [1]: [d_date_sk#19]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
Subquery:2 Hosting operator id = 3 Hosting Expression = sr_returned_date_sk#12 IN dynamicpruning#13
-BroadcastExchange (55)
-+- * CometColumnarToRow (54)
- +- CometProject (53)
- +- CometFilter (52)
- +- CometScan parquet spark_catalog.default.date_dim (51)
+BroadcastExchange (57)
++- * CometColumnarToRow (56)
+ +- CometProject (55)
+ +- CometFilter (54)
+ +- CometScan parquet spark_catalog.default.date_dim (53)
-(51) CometScan parquet spark_catalog.default.date_dim
+(53) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#22, d_year#23, d_moy#24]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_moy), IsNotNull(d_year), GreaterThanOrEqual(d_moy,9), LessThanOrEqual(d_moy,12), EqualTo(d_year,1999), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_moy:int>
-(52) CometFilter
+(54) CometFilter
Input [3]: [d_date_sk#22, d_year#23, d_moy#24]
Condition : (((((isnotnull(d_moy#24) AND isnotnull(d_year#23)) AND (d_moy#24 >= 9)) AND (d_moy#24 <= 12)) AND (d_year#23 = 1999)) AND isnotnull(d_date_sk#22))
-(53) CometProject
+(55) CometProject
Input [3]: [d_date_sk#22, d_year#23, d_moy#24]
Arguments: [d_date_sk#22], [d_date_sk#22]
-(54) CometColumnarToRow [codegen id : 1]
+(56) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#22]
-(55) BroadcastExchange
+(57) BroadcastExchange
Input [1]: [d_date_sk#22]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
Subquery:3 Hosting operator id = 8 Hosting Expression = cs_sold_date_sk#17 IN dynamicpruning#18
-BroadcastExchange (60)
-+- * CometColumnarToRow (59)
- +- CometProject (58)
- +- CometFilter (57)
- +- CometScan parquet spark_catalog.default.date_dim (56)
+BroadcastExchange (62)
++- * CometColumnarToRow (61)
+ +- CometProject (60)
+ +- CometFilter (59)
+ +- CometScan parquet spark_catalog.default.date_dim (58)
-(56) CometScan parquet spark_catalog.default.date_dim
+(58) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#25, d_year#26]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [In(d_year, [1999,2000,2001]), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(57) CometFilter
+(59) CometFilter
Input [2]: [d_date_sk#25, d_year#26]
Condition : (d_year#26 IN (1999,2000,2001) AND isnotnull(d_date_sk#25))
-(58) CometProject
+(60) CometProject
Input [2]: [d_date_sk#25, d_year#26]
Arguments: [d_date_sk#25], [d_date_sk#25]
-(59) CometColumnarToRow [codegen id : 1]
+(61) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#25]
-(60) BroadcastExchange
+(62) BroadcastExchange
Input [1]: [d_date_sk#25]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt
index 9340a4e5f..411614bc7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q29/simplified.txt
@@ -64,8 +64,10 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [s_store_sk,s_store_id,s_store_name] #10
- CometFilter [s_store_sk,s_store_id,s_store_name]
- CometScan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
+ CometProject [s_store_id] [s_store_sk,s_store_id,s_store_name]
+ CometFilter [s_store_sk,s_store_id,s_store_name]
+ CometScan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
CometBroadcastExchange [i_item_sk,i_item_id,i_item_desc] #11
- CometFilter [i_item_sk,i_item_id,i_item_desc]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
+ CometProject [i_item_id] [i_item_sk,i_item_id,i_item_desc]
+ CometFilter [i_item_sk,i_item_id,i_item_desc]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt
index e80a2eaa7..56f0aa3fc 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/explain.txt
@@ -73,39 +73,39 @@ Condition : ((isnotnull(i_manufact_id#10) AND (i_manufact_id#10 = 128)) AND isno
(11) CometProject
Input [4]: [i_item_sk#7, i_brand_id#8, i_brand#9, i_manufact_id#10]
-Arguments: [i_item_sk#7, i_brand_id#8, i_brand#9], [i_item_sk#7, i_brand_id#8, i_brand#9]
+Arguments: [i_item_sk#7, i_brand_id#8, i_brand#11], [i_item_sk#7, i_brand_id#8, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_brand#9, 50, true, false, true) AS i_brand#11]
(12) CometBroadcastExchange
-Input [3]: [i_item_sk#7, i_brand_id#8, i_brand#9]
-Arguments: [i_item_sk#7, i_brand_id#8, i_brand#9]
+Input [3]: [i_item_sk#7, i_brand_id#8, i_brand#11]
+Arguments: [i_item_sk#7, i_brand_id#8, i_brand#11]
(13) CometBroadcastHashJoin
Left output [3]: [d_year#2, ss_item_sk#4, ss_ext_sales_price#5]
-Right output [3]: [i_item_sk#7, i_brand_id#8, i_brand#9]
+Right output [3]: [i_item_sk#7, i_brand_id#8, i_brand#11]
Arguments: [ss_item_sk#4], [i_item_sk#7], Inner, BuildRight
(14) CometProject
-Input [6]: [d_year#2, ss_item_sk#4, ss_ext_sales_price#5, i_item_sk#7, i_brand_id#8, i_brand#9]
-Arguments: [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#9], [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#9]
+Input [6]: [d_year#2, ss_item_sk#4, ss_ext_sales_price#5, i_item_sk#7, i_brand_id#8, i_brand#11]
+Arguments: [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#11], [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#11]
(15) CometHashAggregate
-Input [4]: [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#9]
-Keys [3]: [d_year#2, i_brand#9, i_brand_id#8]
+Input [4]: [d_year#2, ss_ext_sales_price#5, i_brand_id#8, i_brand#11]
+Keys [3]: [d_year#2, i_brand#11, i_brand_id#8]
Functions [1]: [partial_sum(UnscaledValue(ss_ext_sales_price#5))]
(16) CometExchange
-Input [4]: [d_year#2, i_brand#9, i_brand_id#8, sum#11]
-Arguments: hashpartitioning(d_year#2, i_brand#9, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
+Arguments: hashpartitioning(d_year#2, i_brand#11, i_brand_id#8, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(17) CometHashAggregate
-Input [4]: [d_year#2, i_brand#9, i_brand_id#8, sum#11]
-Keys [3]: [d_year#2, i_brand#9, i_brand_id#8]
+Input [4]: [d_year#2, i_brand#11, i_brand_id#8, sum#12]
+Keys [3]: [d_year#2, i_brand#11, i_brand_id#8]
Functions [1]: [sum(UnscaledValue(ss_ext_sales_price#5))]
(18) CometTakeOrderedAndProject
-Input [4]: [d_year#2, brand_id#12, brand#13, sum_agg#14]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[d_year#2 ASC NULLS FIRST,sum_agg#14 DESC NULLS LAST,brand_id#12 ASC NULLS FIRST], output=[d_year#2,brand_id#12,brand#13,sum_agg#14]), [d_year#2, brand_id#12, brand#13, sum_agg#14], 100, [d_year#2 ASC NULLS FIRST, sum_agg#14 DESC NULLS LAST, brand_id#12 ASC NULLS FIRST], [d_year#2, brand_id#12, brand#13, sum_agg#14]
+Input [4]: [d_year#2, brand_id#13, brand#14, sum_agg#15]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[d_year#2 ASC NULLS FIRST,sum_agg#15 DESC NULLS LAST,brand_id#13 ASC NULLS FIRST], output=[d_year#2,brand_id#13,brand#14,sum_agg#15]), [d_year#2, brand_id#13, brand#14, sum_agg#15], 100, [d_year#2 ASC NULLS FIRST, sum_agg#15 DESC NULLS LAST, brand_id#13 ASC NULLS FIRST], [d_year#2, brand_id#13, brand#14, sum_agg#15]
(19) CometColumnarToRow [codegen id : 1]
-Input [4]: [d_year#2, brand_id#12, brand#13, sum_agg#14]
+Input [4]: [d_year#2, brand_id#13, brand#14, sum_agg#15]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt
index 7bc02651e..928901e20 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q3/simplified.txt
@@ -16,6 +16,6 @@ WholeStageCodegen (1)
CometFilter [ss_item_sk,ss_ext_sales_price,ss_sold_date_sk]
CometScan parquet spark_catalog.default.store_sales [ss_item_sk,ss_ext_sales_price,ss_sold_date_sk]
CometBroadcastExchange [i_item_sk,i_brand_id,i_brand] #3
- CometProject [i_item_sk,i_brand_id,i_brand]
+ CometProject [i_brand] [i_item_sk,i_brand_id,i_brand]
CometFilter [i_item_sk,i_brand_id,i_brand,i_manufact_id]
CometScan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_brand,i_manufact_id]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt
index a16c487c2..2094cfc0c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/explain.txt
@@ -1,18 +1,18 @@
== Physical Plan ==
-* CometColumnarToRow (63)
-+- CometTakeOrderedAndProject (62)
- +- CometProject (61)
- +- CometBroadcastHashJoin (60)
- :- CometProject (55)
- : +- CometBroadcastHashJoin (54)
- : :- CometProject (50)
- : : +- CometBroadcastHashJoin (49)
- : : :- CometFilter (17)
- : : : +- CometHashAggregate (16)
- : : : +- CometExchange (15)
- : : : +- CometHashAggregate (14)
- : : : +- CometProject (13)
- : : : +- CometBroadcastHashJoin (12)
+* CometColumnarToRow (50)
++- CometTakeOrderedAndProject (49)
+ +- CometProject (48)
+ +- CometBroadcastHashJoin (47)
+ :- CometProject (42)
+ : +- CometBroadcastHashJoin (41)
+ : :- CometProject (36)
+ : : +- CometBroadcastHashJoin (35)
+ : : :- CometFilter (18)
+ : : : +- CometHashAggregate (17)
+ : : : +- CometExchange (16)
+ : : : +- CometHashAggregate (15)
+ : : : +- CometProject (14)
+ : : : +- CometBroadcastHashJoin (13)
: : : :- CometProject (8)
: : : : +- CometBroadcastHashJoin (7)
: : : : :- CometFilter (2)
@@ -21,47 +21,34 @@
: : : : +- CometProject (5)
: : : : +- CometFilter (4)
: : : : +- CometScan parquet spark_catalog.default.date_dim (3)
- : : : +- CometBroadcastExchange (11)
- : : : +- CometFilter (10)
- : : : +- CometScan parquet spark_catalog.default.customer_address (9)
- : : +- CometBroadcastExchange (48)
- : : +- CometFilter (47)
- : : +- CometHashAggregate (46)
- : : +- CometHashAggregate (45)
- : : +- CometProject (44)
- : : +- CometBroadcastHashJoin (43)
- : : :- CometHashAggregate (30)
- : : : +- CometExchange (29)
- : : : +- CometHashAggregate (28)
- : : : +- CometProject (27)
- : : : +- CometBroadcastHashJoin (26)
- : : : :- CometProject (22)
- : : : : +- CometBroadcastHashJoin (21)
- : : : : :- CometFilter (19)
- : : : : : +- CometScan parquet spark_catalog.default.web_returns (18)
- : : : : +- ReusedExchange (20)
- : : : +- CometBroadcastExchange (25)
- : : : +- CometFilter (24)
- : : : +- CometScan parquet spark_catalog.default.customer_address (23)
- : : +- CometBroadcastExchange (42)
- : : +- CometHashAggregate (41)
- : : +- CometExchange (40)
- : : +- CometHashAggregate (39)
- : : +- CometProject (38)
- : : +- CometBroadcastHashJoin (37)
- : : :- CometProject (35)
- : : : +- CometBroadcastHashJoin (34)
- : : : :- CometFilter (32)
- : : : : +- CometScan parquet spark_catalog.default.web_returns (31)
- : : : +- ReusedExchange (33)
- : : +- ReusedExchange (36)
- : +- CometBroadcastExchange (53)
- : +- CometFilter (52)
- : +- CometScan parquet spark_catalog.default.customer (51)
- +- CometBroadcastExchange (59)
- +- CometProject (58)
- +- CometFilter (57)
- +- CometScan parquet spark_catalog.default.customer_address (56)
+ : : : +- CometBroadcastExchange (12)
+ : : : +- CometProject (11)
+ : : : +- CometFilter (10)
+ : : : +- CometScan parquet spark_catalog.default.customer_address (9)
+ : : +- CometBroadcastExchange (34)
+ : : +- CometFilter (33)
+ : : +- CometHashAggregate (32)
+ : : +- CometExchange (31)
+ : : +- CometHashAggregate (30)
+ : : +- CometHashAggregate (29)
+ : : +- CometExchange (28)
+ : : +- CometHashAggregate (27)
+ : : +- CometProject (26)
+ : : +- CometBroadcastHashJoin (25)
+ : : :- CometProject (23)
+ : : : +- CometBroadcastHashJoin (22)
+ : : : :- CometFilter (20)
+ : : : : +- CometScan parquet spark_catalog.default.web_returns (19)
+ : : : +- ReusedExchange (21)
+ : : +- ReusedExchange (24)
+ : +- CometBroadcastExchange (40)
+ : +- CometProject (39)
+ : +- CometFilter (38)
+ : +- CometScan parquet spark_catalog.default.customer (37)
+ +- CometBroadcastExchange (46)
+ +- CometProject (45)
+ +- CometFilter (44)
+ +- CometScan parquet spark_catalog.default.customer_address (43)
(1) CometScan parquet spark_catalog.default.web_returns
@@ -113,102 +100,44 @@ ReadSchema: struct<ca_address_sk:int,ca_state:string>
(10) CometFilter
Input [2]: [ca_address_sk#8, ca_state#9]
-Condition : isnotnull(ca_address_sk#8)
+Condition : (isnotnull(ca_address_sk#8) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#9, 2, true, false, true)))
-(11) CometBroadcastExchange
+(11) CometProject
Input [2]: [ca_address_sk#8, ca_state#9]
-Arguments: [ca_address_sk#8, ca_state#9]
+Arguments: [ca_address_sk#8, ca_state#10], [ca_address_sk#8, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#9, 2, true, false, true) AS ca_state#10]
-(12) CometBroadcastHashJoin
+(12) CometBroadcastExchange
+Input [2]: [ca_address_sk#8, ca_state#10]
+Arguments: [ca_address_sk#8, ca_state#10]
+
+(13) CometBroadcastHashJoin
Left output [3]: [wr_returning_customer_sk#1, wr_returning_addr_sk#2, wr_return_amt#3]
-Right output [2]: [ca_address_sk#8, ca_state#9]
+Right output [2]: [ca_address_sk#8, ca_state#10]
Arguments: [wr_returning_addr_sk#2], [ca_address_sk#8], Inner, BuildRight
-(13) CometProject
-Input [5]: [wr_returning_customer_sk#1, wr_returning_addr_sk#2, wr_return_amt#3, ca_address_sk#8, ca_state#9]
-Arguments: [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#9], [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#9]
+(14) CometProject
+Input [5]: [wr_returning_customer_sk#1, wr_returning_addr_sk#2, wr_return_amt#3, ca_address_sk#8, ca_state#10]
+Arguments: [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#10], [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#10]
-(14) CometHashAggregate
-Input [3]: [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#9]
-Keys [2]: [wr_returning_customer_sk#1, ca_state#9]
+(15) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#1, wr_return_amt#3, ca_state#10]
+Keys [2]: [wr_returning_customer_sk#1, ca_state#10]
Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#3))]
-(15) CometExchange
-Input [3]: [wr_returning_customer_sk#1, ca_state#9, sum#10]
-Arguments: hashpartitioning(wr_returning_customer_sk#1, ca_state#9, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(16) CometExchange
+Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
+Arguments: hashpartitioning(wr_returning_customer_sk#1, ca_state#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(16) CometHashAggregate
-Input [3]: [wr_returning_customer_sk#1, ca_state#9, sum#10]
-Keys [2]: [wr_returning_customer_sk#1, ca_state#9]
+(17) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#1, ca_state#10, sum#11]
+Keys [2]: [wr_returning_customer_sk#1, ca_state#10]
Functions [1]: [sum(UnscaledValue(wr_return_amt#3))]
-(17) CometFilter
-Input [3]: [ctr_customer_sk#11, ctr_state#12, ctr_total_return#13]
-Condition : isnotnull(ctr_total_return#13)
-
-(18) CometScan parquet spark_catalog.default.web_returns
-Output [2]: [wr_returning_addr_sk#2, wr_returned_date_sk#4]
-Batched: true
-Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(wr_returned_date_sk#4), dynamicpruningexpression(wr_returned_date_sk#4 IN dynamicpruning#14)]
-PushedFilters: [IsNotNull(wr_returning_addr_sk)]
-ReadSchema: struct<wr_returning_addr_sk:int>
-
-(19) CometFilter
-Input [2]: [wr_returning_addr_sk#2, wr_returned_date_sk#4]
-Condition : isnotnull(wr_returning_addr_sk#2)
-
-(20) ReusedExchange [Reuses operator id: 6]
-Output [1]: [d_date_sk#6]
-
-(21) CometBroadcastHashJoin
-Left output [2]: [wr_returning_addr_sk#2, wr_returned_date_sk#4]
-Right output [1]: [d_date_sk#6]
-Arguments: [wr_returned_date_sk#4], [d_date_sk#6], Inner, BuildRight
-
-(22) CometProject
-Input [3]: [wr_returning_addr_sk#2, wr_returned_date_sk#4, d_date_sk#6]
-Arguments: [wr_returning_addr_sk#2], [wr_returning_addr_sk#2]
-
-(23) CometScan parquet spark_catalog.default.customer_address
-Output [2]: [ca_address_sk#8, ca_state#9]
-Batched: true
-Location [not included in comparison]/{warehouse_dir}/customer_address]
-PushedFilters: [IsNotNull(ca_state), IsNotNull(ca_address_sk)]
-ReadSchema: struct<ca_address_sk:int,ca_state:string>
-
-(24) CometFilter
-Input [2]: [ca_address_sk#8, ca_state#9]
-Condition : (isnotnull(ca_state#9) AND isnotnull(ca_address_sk#8))
-
-(25) CometBroadcastExchange
-Input [2]: [ca_address_sk#8, ca_state#9]
-Arguments: [ca_address_sk#8, ca_state#9]
-
-(26) CometBroadcastHashJoin
-Left output [1]: [wr_returning_addr_sk#2]
-Right output [2]: [ca_address_sk#8, ca_state#9]
-Arguments: [wr_returning_addr_sk#2], [ca_address_sk#8], Inner, BuildRight
-
-(27) CometProject
-Input [3]: [wr_returning_addr_sk#2, ca_address_sk#8, ca_state#9]
-Arguments: [ctr_state#12], [ca_state#9 AS ctr_state#12]
-
-(28) CometHashAggregate
-Input [1]: [ctr_state#12]
-Keys [1]: [ctr_state#12]
-Functions: []
-
-(29) CometExchange
-Input [1]: [ctr_state#12]
-Arguments: hashpartitioning(ctr_state#12, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-
-(30) CometHashAggregate
-Input [1]: [ctr_state#12]
-Keys [1]: [ctr_state#12]
-Functions: []
+(18) CometFilter
+Input [3]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14]
+Condition : isnotnull(ctr_total_return#14)
-(31) CometScan parquet spark_catalog.default.web_returns
+(19) CometScan parquet spark_catalog.default.web_returns
Output [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
Batched: true
Location: InMemoryFileIndex []
@@ -216,181 +145,174 @@ PartitionFilters: [isnotnull(wr_returned_date_sk#18), dynamicpruningexpression(w
PushedFilters: [IsNotNull(wr_returning_addr_sk)]
ReadSchema: struct<wr_returning_customer_sk:int,wr_returning_addr_sk:int,wr_return_amt:decimal(7,2)>
-(32) CometFilter
+(20) CometFilter
Input [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
Condition : isnotnull(wr_returning_addr_sk#16)
-(33) ReusedExchange [Reuses operator id: 6]
+(21) ReusedExchange [Reuses operator id: 6]
Output [1]: [d_date_sk#20]
-(34) CometBroadcastHashJoin
+(22) CometBroadcastHashJoin
Left output [4]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18]
Right output [1]: [d_date_sk#20]
Arguments: [wr_returned_date_sk#18], [d_date_sk#20], Inner, BuildRight
-(35) CometProject
+(23) CometProject
Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, wr_returned_date_sk#18, d_date_sk#20]
Arguments: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17], [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17]
-(36) ReusedExchange [Reuses operator id: 25]
-Output [2]: [ca_address_sk#21, ca_state#22]
+(24) ReusedExchange [Reuses operator id: 12]
+Output [2]: [ca_address_sk#21, ca_state#10]
-(37) CometBroadcastHashJoin
+(25) CometBroadcastHashJoin
Left output [3]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17]
-Right output [2]: [ca_address_sk#21, ca_state#22]
+Right output [2]: [ca_address_sk#21, ca_state#10]
Arguments: [wr_returning_addr_sk#16], [ca_address_sk#21], Inner, BuildRight
-(38) CometProject
-Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, ca_address_sk#21, ca_state#22]
-Arguments: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22], [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22]
+(26) CometProject
+Input [5]: [wr_returning_customer_sk#15, wr_returning_addr_sk#16, wr_return_amt#17, ca_address_sk#21, ca_state#10]
+Arguments: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#10], [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#10]
-(39) CometHashAggregate
-Input [3]: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#22]
-Keys [2]: [wr_returning_customer_sk#15, ca_state#22]
+(27) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#15, wr_return_amt#17, ca_state#10]
+Keys [2]: [wr_returning_customer_sk#15, ca_state#10]
Functions [1]: [partial_sum(UnscaledValue(wr_return_amt#17))]
-(40) CometExchange
-Input [3]: [wr_returning_customer_sk#15, ca_state#22, sum#23]
-Arguments: hashpartitioning(wr_returning_customer_sk#15, ca_state#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+(28) CometExchange
+Input [3]: [wr_returning_customer_sk#15, ca_state#10, sum#22]
+Arguments: hashpartitioning(wr_returning_customer_sk#15, ca_state#10, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(41) CometHashAggregate
-Input [3]: [wr_returning_customer_sk#15, ca_state#22, sum#23]
-Keys [2]: [wr_returning_customer_sk#15, ca_state#22]
+(29) CometHashAggregate
+Input [3]: [wr_returning_customer_sk#15, ca_state#10, sum#22]
+Keys [2]: [wr_returning_customer_sk#15, ca_state#10]
Functions [1]: [sum(UnscaledValue(wr_return_amt#17))]
-(42) CometBroadcastExchange
-Input [2]: [ctr_state#24, ctr_total_return#25]
-Arguments: [ctr_state#24, ctr_total_return#25]
-
-(43) CometBroadcastHashJoin
-Left output [1]: [ctr_state#26]
-Right output [2]: [ctr_state#24, ctr_total_return#25]
-Arguments: [rpad(ctr_state#26, 2, )], [rpad(ctr_state#24, 2, )], Inner, BuildRight
-
-(44) CometProject
-Input [3]: [ctr_state#26, ctr_state#24, ctr_total_return#25]
-Arguments: [ctr_state#26, ctr_total_return#25], [ctr_state#26, ctr_total_return#25]
-
-(45) CometHashAggregate
-Input [2]: [ctr_state#26, ctr_total_return#25]
-Keys [1]: [ctr_state#26]
-Functions [1]: [partial_avg(ctr_total_return#25)]
-
-(46) CometHashAggregate
-Input [3]: [ctr_state#26, sum#27, count#28]
-Keys [1]: [ctr_state#26]
-Functions [1]: [avg(ctr_total_return#25)]
-
-(47) CometFilter
-Input [2]: [(avg(ctr_total_return) * 1.2)#29, ctr_state#26]
-Condition : isnotnull((avg(ctr_total_return) * 1.2)#29)
-
-(48) CometBroadcastExchange
-Input [2]: [(avg(ctr_total_return) * 1.2)#29, ctr_state#26]
-Arguments: [(avg(ctr_total_return) * 1.2)#29, ctr_state#26]
-
-(49) CometBroadcastHashJoin
-Left output [3]: [ctr_customer_sk#11, ctr_state#12, ctr_total_return#13]
-Right output [2]: [(avg(ctr_total_return) * 1.2)#29, ctr_state#26]
-Arguments: [coalesce(ctr_state#12, ), isnull(ctr_state#12)], [coalesce(ctr_state#26, ), isnull(ctr_state#26)], Inner, (cast(ctr_total_return#13 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#29), BuildRight
-
-(50) CometProject
-Input [5]: [ctr_customer_sk#11, ctr_state#12, ctr_total_return#13, (avg(ctr_total_return) * 1.2)#29, ctr_state#26]
-Arguments: [ctr_customer_sk#11, ctr_total_return#13], [ctr_customer_sk#11, ctr_total_return#13]
-
-(51) CometScan parquet spark_catalog.default.customer
-Output [14]: [c_customer_sk#30, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
+(30) CometHashAggregate
+Input [2]: [ctr_state#23, ctr_total_return#24]
+Keys [1]: [ctr_state#23]
+Functions [1]: [partial_avg(ctr_total_return#24)]
+
+(31) CometExchange
+Input [3]: [ctr_state#23, sum#25, count#26]
+Arguments: hashpartitioning(ctr_state#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+
+(32) CometHashAggregate
+Input [3]: [ctr_state#23, sum#25, count#26]
+Keys [1]: [ctr_state#23]
+Functions [1]: [avg(ctr_total_return#24)]
+
+(33) CometFilter
+Input [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_state#23]
+Condition : isnotnull((avg(ctr_total_return) * 1.2)#27)
+
+(34) CometBroadcastExchange
+Input [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_state#23]
+Arguments: [(avg(ctr_total_return) * 1.2)#27, ctr_state#23]
+
+(35) CometBroadcastHashJoin
+Left output [3]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14]
+Right output [2]: [(avg(ctr_total_return) * 1.2)#27, ctr_state#23]
+Arguments: [ctr_state#13], [ctr_state#23], Inner, (cast(ctr_total_return#14 as decimal(24,7)) > (avg(ctr_total_return) * 1.2)#27), BuildRight
+
+(36) CometProject
+Input [5]: [ctr_customer_sk#12, ctr_state#13, ctr_total_return#14, (avg(ctr_total_return) * 1.2)#27, ctr_state#23]
+Arguments: [ctr_customer_sk#12, ctr_total_return#14], [ctr_customer_sk#12, ctr_total_return#14]
+
+(37) CometScan parquet spark_catalog.default.customer
+Output [14]: [c_customer_sk#28, c_customer_id#29, c_current_addr_sk#30, c_salutation#31, c_first_name#32, c_last_name#33, c_preferred_cust_flag#34, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#39, c_email_address#40, c_last_review_date#41]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_current_addr_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_current_addr_sk:int,c_salutation:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_day:int,c_birth_month:int,c_birth_year:int,c_birth_country:string,c_login:string,c_email_address:string,c_last_review_date:int>
-(52) CometFilter
-Input [14]: [c_customer_sk#30, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
-Condition : (isnotnull(c_customer_sk#30) AND isnotnull(c_current_addr_sk#32))
+(38) CometFilter
+Input [14]: [c_customer_sk#28, c_customer_id#29, c_current_addr_sk#30, c_salutation#31, c_first_name#32, c_last_name#33, c_preferred_cust_flag#34, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#39, c_email_address#40, c_last_review_date#41]
+Condition : (isnotnull(c_customer_sk#28) AND isnotnull(c_current_addr_sk#30))
+
+(39) CometProject
+Input [14]: [c_customer_sk#28, c_customer_id#29, c_current_addr_sk#30, c_salutation#31, c_first_name#32, c_last_name#33, c_preferred_cust_flag#34, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#39, c_email_address#40, c_last_review_date#41]
+Arguments: [c_customer_sk#28, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41], [c_customer_sk#28, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#29, 16, true, false, true) AS c_customer_id#42, c_current_addr_sk#30, staticin [...]
-(53) CometBroadcastExchange
-Input [14]: [c_customer_sk#30, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
-Arguments: [c_customer_sk#30, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
+(40) CometBroadcastExchange
+Input [14]: [c_customer_sk#28, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41]
+Arguments: [c_customer_sk#28, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41]
-(54) CometBroadcastHashJoin
-Left output [2]: [ctr_customer_sk#11, ctr_total_return#13]
-Right output [14]: [c_customer_sk#30, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
-Arguments: [ctr_customer_sk#11], [c_customer_sk#30], Inner, BuildRight
+(41) CometBroadcastHashJoin
+Left output [2]: [ctr_customer_sk#12, ctr_total_return#14]
+Right output [14]: [c_customer_sk#28, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41]
+Arguments: [ctr_customer_sk#12], [c_customer_sk#28], Inner, BuildRight
-(55) CometProject
-Input [16]: [ctr_customer_sk#11, ctr_total_return#13, c_customer_sk#30, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
-Arguments: [ctr_total_return#13, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43], [ctr_total_return#13, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login [...]
+(42) CometProject
+Input [16]: [ctr_customer_sk#12, ctr_total_return#14, c_customer_sk#28, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41]
+Arguments: [ctr_total_return#14, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41], [ctr_total_return#14, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login [...]
-(56) CometScan parquet spark_catalog.default.customer_address
-Output [2]: [ca_address_sk#44, ca_state#45]
+(43) CometScan parquet spark_catalog.default.customer_address
+Output [2]: [ca_address_sk#49, ca_state#50]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_address]
-PushedFilters: [IsNotNull(ca_state), IsNotNull(ca_address_sk)]
+PushedFilters: [IsNotNull(ca_address_sk)]
ReadSchema: struct<ca_address_sk:int,ca_state:string>
-(57) CometFilter
-Input [2]: [ca_address_sk#44, ca_state#45]
-Condition : ((isnotnull(ca_state#45) AND (rpad(ca_state#45, 2, ) = GA)) AND isnotnull(ca_address_sk#44))
+(44) CometFilter
+Input [2]: [ca_address_sk#49, ca_state#50]
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#50, 2, true, false, true) = GA) AND isnotnull(ca_address_sk#49))
-(58) CometProject
-Input [2]: [ca_address_sk#44, ca_state#45]
-Arguments: [ca_address_sk#44], [ca_address_sk#44]
+(45) CometProject
+Input [2]: [ca_address_sk#49, ca_state#50]
+Arguments: [ca_address_sk#49], [ca_address_sk#49]
-(59) CometBroadcastExchange
-Input [1]: [ca_address_sk#44]
-Arguments: [ca_address_sk#44]
+(46) CometBroadcastExchange
+Input [1]: [ca_address_sk#49]
+Arguments: [ca_address_sk#49]
-(60) CometBroadcastHashJoin
-Left output [14]: [ctr_total_return#13, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43]
-Right output [1]: [ca_address_sk#44]
-Arguments: [c_current_addr_sk#32], [ca_address_sk#44], Inner, BuildRight
+(47) CometBroadcastHashJoin
+Left output [14]: [ctr_total_return#14, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41]
+Right output [1]: [ca_address_sk#49]
+Arguments: [c_current_addr_sk#30], [ca_address_sk#49], Inner, BuildRight
-(61) CometProject
-Input [15]: [ctr_total_return#13, c_customer_id#31, c_current_addr_sk#32, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43, ca_address_sk#44]
-Arguments: [c_customer_id#31, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43, ctr_total_return#13], [c_customer_id#31, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43, ctr_total_return#13]
+(48) CometProject
+Input [15]: [ctr_total_return#14, c_customer_id#42, c_current_addr_sk#30, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41, ca_address_sk#49]
+Arguments: [c_customer_id#42, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41, ctr_total_return#14], [c_customer_id#42, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41, ctr_total_return#14]
-(62) CometTakeOrderedAndProject
-Input [13]: [c_customer_id#31, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43, ctr_total_return#13]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#31 ASC NULLS FIRST,c_salutation#33 ASC NULLS FIRST,c_first_name#34 ASC NULLS FIRST,c_last_name#35 ASC NULLS FIRST,c_preferred_cust_flag#36 ASC NULLS FIRST,c_birth_day#37 ASC NULLS FIRST,c_birth_month#38 ASC NULLS FIRST,c_birth_year#39 ASC NULLS FIRST,c_birth_country#40 ASC NULLS FIRST,c_login#41 ASC NULLS FIRST,c_email_address#42 ASC NULLS FIRST,c_last_review_date#43 ASC NULLS FIRST,ctr_total_return#13 ASC NULLS FIRST], o [...]
+(49) CometTakeOrderedAndProject
+Input [13]: [c_customer_id#42, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41, ctr_total_return#14]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[c_customer_id#42 ASC NULLS FIRST,c_salutation#43 ASC NULLS FIRST,c_first_name#44 ASC NULLS FIRST,c_last_name#45 ASC NULLS FIRST,c_preferred_cust_flag#46 ASC NULLS FIRST,c_birth_day#35 ASC NULLS FIRST,c_birth_month#36 ASC NULLS FIRST,c_birth_year#37 ASC NULLS FIRST,c_birth_country#38 ASC NULLS FIRST,c_login#47 ASC NULLS FIRST,c_email_address#48 ASC NULLS FIRST,c_last_review_date#41 ASC NULLS FIRST,ctr_total_return#14 ASC NULLS FIRST], o [...]
-(63) CometColumnarToRow [codegen id : 1]
-Input [13]: [c_customer_id#31, c_salutation#33, c_first_name#34, c_last_name#35, c_preferred_cust_flag#36, c_birth_day#37, c_birth_month#38, c_birth_year#39, c_birth_country#40, c_login#41, c_email_address#42, c_last_review_date#43, ctr_total_return#13]
+(50) CometColumnarToRow [codegen id : 1]
+Input [13]: [c_customer_id#42, c_salutation#43, c_first_name#44, c_last_name#45, c_preferred_cust_flag#46, c_birth_day#35, c_birth_month#36, c_birth_year#37, c_birth_country#38, c_login#47, c_email_address#48, c_last_review_date#41, ctr_total_return#14]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = wr_returned_date_sk#4 IN dynamicpruning#5
-BroadcastExchange (68)
-+- * CometColumnarToRow (67)
- +- CometProject (66)
- +- CometFilter (65)
- +- CometScan parquet spark_catalog.default.date_dim (64)
+BroadcastExchange (55)
++- * CometColumnarToRow (54)
+ +- CometProject (53)
+ +- CometFilter (52)
+ +- CometScan parquet spark_catalog.default.date_dim (51)
-(64) CometScan parquet spark_catalog.default.date_dim
+(51) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#6, d_year#7]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(65) CometFilter
+(52) CometFilter
Input [2]: [d_date_sk#6, d_year#7]
Condition : ((isnotnull(d_year#7) AND (d_year#7 = 2002)) AND isnotnull(d_date_sk#6))
-(66) CometProject
+(53) CometProject
Input [2]: [d_date_sk#6, d_year#7]
Arguments: [d_date_sk#6], [d_date_sk#6]
-(67) CometColumnarToRow [codegen id : 1]
+(54) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#6]
-(68) BroadcastExchange
+(55) BroadcastExchange
Input [1]: [d_date_sk#6]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-Subquery:2 Hosting operator id = 18 Hosting Expression = wr_returned_date_sk#4 IN dynamicpruning#5
-
-Subquery:3 Hosting operator id = 31 Hosting Expression = wr_returned_date_sk#18 IN dynamicpruning#5
+Subquery:2 Hosting operator id = 19 Hosting Expression = wr_returned_date_sk#18 IN dynamicpruning#5
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt
index 082503ba1..7f2e44862 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q30/simplified.txt
@@ -31,45 +31,31 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [ca_address_sk,ca_state] #4
- CometFilter [ca_address_sk,ca_state]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ CometProject [ca_state] [ca_address_sk,ca_state]
+ CometFilter [ca_address_sk,ca_state]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
CometBroadcastExchange [(avg(ctr_total_return) * 1.2),ctr_state] #5
CometFilter [(avg(ctr_total_return) * 1.2),ctr_state]
CometHashAggregate [(avg(ctr_total_return) * 1.2),ctr_state,sum,count,avg(ctr_total_return)]
- CometHashAggregate [ctr_state,sum,count,ctr_total_return]
- CometProject [ctr_state,ctr_total_return]
- CometBroadcastHashJoin [ctr_state,ctr_state,ctr_total_return]
- CometHashAggregate [ctr_state,ctr_state]
- CometExchange [ctr_state] #6
- CometHashAggregate [ctr_state]
- CometProject [ca_state] [ctr_state]
- CometBroadcastHashJoin [wr_returning_addr_sk,ca_address_sk,ca_state]
- CometProject [wr_returning_addr_sk]
- CometBroadcastHashJoin [wr_returning_addr_sk,wr_returned_date_sk,d_date_sk]
- CometFilter [wr_returning_addr_sk,wr_returned_date_sk]
- CometScan parquet spark_catalog.default.web_returns [wr_returning_addr_sk,wr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #3
- CometBroadcastExchange [ca_address_sk,ca_state] #7
- CometFilter [ca_address_sk,ca_state]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
- CometBroadcastExchange [ctr_state,ctr_total_return] #8
- CometHashAggregate [ctr_state,ctr_total_return,wr_returning_customer_sk,ca_state,sum,sum(UnscaledValue(wr_return_amt))]
- CometExchange [wr_returning_customer_sk,ca_state] #9
- CometHashAggregate [wr_returning_customer_sk,ca_state,sum,wr_return_amt]
- CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
- CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
- CometProject [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt]
- CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk,d_date_sk]
- CometFilter [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
- CometScan parquet spark_catalog.default.web_returns [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
- ReusedSubquery [d_date_sk] #1
- ReusedExchange [d_date_sk] #3
- ReusedExchange [ca_address_sk,ca_state] #7
- CometBroadcastExchange [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date] #10
- CometFilter [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
- CometBroadcastExchange [ca_address_sk] #11
+ CometExchange [ctr_state] #6
+ CometHashAggregate [ctr_state,sum,count,ctr_total_return]
+ CometHashAggregate [ctr_state,ctr_total_return,wr_returning_customer_sk,ca_state,sum,sum(UnscaledValue(wr_return_amt))]
+ CometExchange [wr_returning_customer_sk,ca_state] #7
+ CometHashAggregate [wr_returning_customer_sk,ca_state,sum,wr_return_amt]
+ CometProject [wr_returning_customer_sk,wr_return_amt,ca_state]
+ CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,ca_address_sk,ca_state]
+ CometProject [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt]
+ CometBroadcastHashJoin [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk,d_date_sk]
+ CometFilter [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
+ CometScan parquet spark_catalog.default.web_returns [wr_returning_customer_sk,wr_returning_addr_sk,wr_return_amt,wr_returned_date_sk]
+ ReusedSubquery [d_date_sk] #1
+ ReusedExchange [d_date_sk] #3
+ ReusedExchange [ca_address_sk,ca_state] #4
+ CometBroadcastExchange [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date] #8
+ CometProject [c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_login,c_email_address] [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometFilter [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_customer_id,c_current_addr_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address,c_last_review_date]
+ CometBroadcastExchange [ca_address_sk] #9
CometProject [ca_address_sk]
CometFilter [ca_address_sk,ca_state]
CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt
index 42aa6d520..ba336095e 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q33/explain.txt
@@ -145,12 +145,11 @@ Condition : isnotnull(i_item_sk#11)
Output [2]: [i_category#13, i_manufact_id#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/item]
-PushedFilters: [IsNotNull(i_category)]
ReadSchema: struct<i_category:string,i_manufact_id:int>
(18) CometFilter
Input [2]: [i_category#13, i_manufact_id#14]
-Condition : (isnotnull(i_category#13) AND (rpad(i_category#13, 50, ) = Electronics ))
+Condition : (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_category#13, 50, true, false, true) = Electronics )
(19) CometProject
Input [2]: [i_category#13, i_manufact_id#14]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt
index b2e2f58e4..13934b6c6 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/explain.txt
@@ -1,9 +1,9 @@
== Physical Plan ==
-* CometColumnarToRow (32)
-+- CometSort (31)
- +- CometColumnarExchange (30)
- +- CometProject (29)
- +- CometBroadcastHashJoin (28)
+* CometColumnarToRow (33)
++- CometSort (32)
+ +- CometColumnarExchange (31)
+ +- CometProject (30)
+ +- CometBroadcastHashJoin (29)
:- CometFilter (24)
: +- CometHashAggregate (23)
: +- CometExchange (22)
@@ -28,9 +28,10 @@
: +- CometProject (17)
: +- CometFilter (16)
: +- CometScan parquet spark_catalog.default.household_demographics (15)
- +- CometBroadcastExchange (27)
- +- CometFilter (26)
- +- CometScan parquet spark_catalog.default.customer (25)
+ +- CometBroadcastExchange (28)
+ +- CometProject (27)
+ +- CometFilter (26)
+ +- CometScan parquet spark_catalog.default.customer (25)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -110,7 +111,7 @@ ReadSchema: struct<hd_demo_sk:int,hd_buy_potential:string,hd_dep_count:int,hd_ve
(16) CometFilter
Input [4]: [hd_demo_sk#12, hd_buy_potential#13, hd_dep_count#14, hd_vehicle_count#15]
-Condition : ((((isnotnull(hd_vehicle_count#15) AND ((rpad(hd_buy_potential#13, 15, ) = >10000 ) OR (rpad(hd_buy_potential#13, 15, ) = unknown ))) AND (hd_vehicle_count#15 > 0)) AND CASE WHEN (hd_vehicle_count#15 > 0) THEN (knownfloatingpointnormalized(normalizenanandzero((cast(hd_dep_count#14 as double) / knownfloatingpointnormalized(normalizenanandzero(cast(hd_vehicle_count#15 as double)))))) > 1.2) END) AND isnotnull(hd_demo_sk#12))
+Condition : ((((isnotnull(hd_vehicle_count#15) AND ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, hd_buy_potential#13, 15, true, false, true) = >10000 ) OR (staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, hd_buy_potential#13, 15, true, false, true) = unknown ))) AND (hd_vehicle_count#15 > 0)) AND CASE WHEN (hd_vehicle_count#15 > 0) THEN (knownfloatingp [...]
(17) CometProject
Input [4]: [hd_demo_sk#12, hd_buy_potential#13, hd_dep_count#14, hd_vehicle_count#15]
@@ -158,59 +159,63 @@ ReadSchema: struct<c_customer_sk:int,c_salutation:string,c_first_name:string,c_l
Input [5]: [c_customer_sk#18, c_salutation#19, c_first_name#20, c_last_name#21, c_preferred_cust_flag#22]
Condition : isnotnull(c_customer_sk#18)
-(27) CometBroadcastExchange
+(27) CometProject
Input [5]: [c_customer_sk#18, c_salutation#19, c_first_name#20, c_last_name#21, c_preferred_cust_flag#22]
-Arguments: [c_customer_sk#18, c_salutation#19, c_first_name#20, c_last_name#21, c_preferred_cust_flag#22]
+Arguments: [c_customer_sk#18, c_salutation#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26], [c_customer_sk#18, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_salutation#19, 10, true, false, true) AS c_salutation#23, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#20, 20, true, false, true) AS c_first_name#24, staticinvoke(class org.apache. [...]
-(28) CometBroadcastHashJoin
+(28) CometBroadcastExchange
+Input [5]: [c_customer_sk#18, c_salutation#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26]
+Arguments: [c_customer_sk#18, c_salutation#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26]
+
+(29) CometBroadcastHashJoin
Left output [3]: [ss_ticket_number#4, ss_customer_sk#1, cnt#17]
-Right output [5]: [c_customer_sk#18, c_salutation#19, c_first_name#20, c_last_name#21, c_preferred_cust_flag#22]
+Right output [5]: [c_customer_sk#18, c_salutation#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26]
Arguments: [ss_customer_sk#1], [c_customer_sk#18], Inner, BuildRight
-(29) CometProject
-Input [8]: [ss_ticket_number#4, ss_customer_sk#1, cnt#17, c_customer_sk#18, c_salutation#19, c_first_name#20, c_last_name#21, c_preferred_cust_flag#22]
-Arguments: [c_last_name#21, c_first_name#20, c_salutation#19, c_preferred_cust_flag#22, ss_ticket_number#4, cnt#17], [c_last_name#21, c_first_name#20, c_salutation#19, c_preferred_cust_flag#22, ss_ticket_number#4, cnt#17]
+(30) CometProject
+Input [8]: [ss_ticket_number#4, ss_customer_sk#1, cnt#17, c_customer_sk#18, c_salutation#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26]
+Arguments: [c_last_name#25, c_first_name#24, c_salutation#23, c_preferred_cust_flag#26, ss_ticket_number#4, cnt#17], [c_last_name#25, c_first_name#24, c_salutation#23, c_preferred_cust_flag#26, ss_ticket_number#4, cnt#17]
-(30) CometColumnarExchange
-Input [6]: [c_last_name#21, c_first_name#20, c_salutation#19, c_preferred_cust_flag#22, ss_ticket_number#4, cnt#17]
-Arguments: rangepartitioning(c_last_name#21 ASC NULLS FIRST, c_first_name#20 ASC NULLS FIRST, c_salutation#19 ASC NULLS FIRST, c_preferred_cust_flag#22 DESC NULLS LAST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
+(31) CometColumnarExchange
+Input [6]: [c_last_name#25, c_first_name#24, c_salutation#23, c_preferred_cust_flag#26, ss_ticket_number#4, cnt#17]
+Arguments: rangepartitioning(c_last_name#25 ASC NULLS FIRST, c_first_name#24 ASC NULLS FIRST, c_salutation#23 ASC NULLS FIRST, c_preferred_cust_flag#26 DESC NULLS LAST, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=2]
-(31) CometSort
-Input [6]: [c_last_name#21, c_first_name#20, c_salutation#19, c_preferred_cust_flag#22, ss_ticket_number#4, cnt#17]
-Arguments: [c_last_name#21, c_first_name#20, c_salutation#19, c_preferred_cust_flag#22, ss_ticket_number#4, cnt#17], [c_last_name#21 ASC NULLS FIRST, c_first_name#20 ASC NULLS FIRST, c_salutation#19 ASC NULLS FIRST, c_preferred_cust_flag#22 DESC NULLS LAST]
+(32) CometSort
+Input [6]: [c_last_name#25, c_first_name#24, c_salutation#23, c_preferred_cust_flag#26, ss_ticket_number#4, cnt#17]
+Arguments: [c_last_name#25, c_first_name#24, c_salutation#23, c_preferred_cust_flag#26, ss_ticket_number#4, cnt#17], [c_last_name#25 ASC NULLS FIRST, c_first_name#24 ASC NULLS FIRST, c_salutation#23 ASC NULLS FIRST, c_preferred_cust_flag#26 DESC NULLS LAST]
-(32) CometColumnarToRow [codegen id : 1]
-Input [6]: [c_last_name#21, c_first_name#20, c_salutation#19, c_preferred_cust_flag#22, ss_ticket_number#4, cnt#17]
+(33) CometColumnarToRow [codegen id : 1]
+Input [6]: [c_last_name#25, c_first_name#24, c_salutation#23, c_preferred_cust_flag#26, ss_ticket_number#4, cnt#17]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#5 IN dynamicpruning#6
-BroadcastExchange (37)
-+- * CometColumnarToRow (36)
- +- CometProject (35)
- +- CometFilter (34)
- +- CometScan parquet spark_catalog.default.date_dim (33)
+BroadcastExchange (38)
++- * CometColumnarToRow (37)
+ +- CometProject (36)
+ +- CometFilter (35)
+ +- CometScan parquet spark_catalog.default.date_dim (34)
-(33) CometScan parquet spark_catalog.default.date_dim
+(34) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#7, d_year#8, d_dom#9]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [Or(And(GreaterThanOrEqual(d_dom,1),LessThanOrEqual(d_dom,3)),And(GreaterThanOrEqual(d_dom,25),LessThanOrEqual(d_dom,28))), In(d_year, [1999,2000,2001]), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_dom:int>
-(34) CometFilter
+(35) CometFilter
Input [3]: [d_date_sk#7, d_year#8, d_dom#9]
Condition : (((((d_dom#9 >= 1) AND (d_dom#9 <= 3)) OR ((d_dom#9 >= 25) AND (d_dom#9 <= 28))) AND d_year#8 IN (1999,2000,2001)) AND isnotnull(d_date_sk#7))
-(35) CometProject
+(36) CometProject
Input [3]: [d_date_sk#7, d_year#8, d_dom#9]
Arguments: [d_date_sk#7], [d_date_sk#7]
-(36) CometColumnarToRow [codegen id : 1]
+(37) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#7]
-(37) BroadcastExchange
+(38) BroadcastExchange
Input [1]: [d_date_sk#7]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt
index 014786e7f..7f1cb740c 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q34/simplified.txt
@@ -38,5 +38,6 @@ WholeStageCodegen (1)
CometFilter [hd_demo_sk,hd_buy_potential,hd_dep_count,hd_vehicle_count]
CometScan parquet spark_catalog.default.household_demographics [hd_demo_sk,hd_buy_potential,hd_dep_count,hd_vehicle_count]
CometBroadcastExchange [c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag] #7
- CometFilter [c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag]
+ CometProject [c_salutation,c_first_name,c_last_name,c_preferred_cust_flag] [c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag]
+ CometFilter [c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt
index 74ff12aef..0587acad0 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/explain.txt
@@ -1,13 +1,13 @@
== Physical Plan ==
-TakeOrderedAndProject (45)
-+- * HashAggregate (44)
- +- * CometColumnarToRow (43)
- +- CometColumnarExchange (42)
- +- * HashAggregate (41)
- +- * Project (40)
- +- * BroadcastHashJoin Inner BuildRight (39)
- :- * Project (34)
- : +- * BroadcastHashJoin Inner BuildRight (33)
+TakeOrderedAndProject (47)
++- * HashAggregate (46)
+ +- * CometColumnarToRow (45)
+ +- CometColumnarExchange (44)
+ +- * HashAggregate (43)
+ +- * Project (42)
+ +- * BroadcastHashJoin Inner BuildRight (41)
+ :- * Project (35)
+ : +- * BroadcastHashJoin Inner BuildRight (34)
: :- * Project (28)
: : +- * Filter (27)
: : +- * BroadcastHashJoin ExistenceJoin(exists#1) BuildRight (26)
@@ -36,14 +36,16 @@ TakeOrderedAndProject (45)
: : +- CometBroadcastHashJoin (22)
: : :- CometScan parquet spark_catalog.default.catalog_sales (20)
: : +- ReusedExchange (21)
- : +- BroadcastExchange (32)
- : +- * CometColumnarToRow (31)
- : +- CometFilter (30)
- : +- CometScan parquet spark_catalog.default.customer_address (29)
- +- BroadcastExchange (38)
- +- * CometColumnarToRow (37)
- +- CometFilter (36)
- +- CometScan parquet spark_catalog.default.customer_demographics (35)
+ : +- BroadcastExchange (33)
+ : +- * CometColumnarToRow (32)
+ : +- CometProject (31)
+ : +- CometFilter (30)
+ : +- CometScan parquet spark_catalog.default.customer_address (29)
+ +- BroadcastExchange (40)
+ +- * CometColumnarToRow (39)
+ +- CometProject (38)
+ +- CometFilter (37)
+ +- CometScan parquet spark_catalog.default.customer_demographics (36)
(1) CometScan parquet spark_catalog.default.customer
@@ -187,105 +189,113 @@ ReadSchema: struct<ca_address_sk:int,ca_state:string>
Input [2]: [ca_address_sk#20, ca_state#21]
Condition : isnotnull(ca_address_sk#20)
-(31) CometColumnarToRow [codegen id : 3]
+(31) CometProject
Input [2]: [ca_address_sk#20, ca_state#21]
+Arguments: [ca_address_sk#20, ca_state#22], [ca_address_sk#20, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, ca_state#21, 2, true, false, true) AS ca_state#22]
-(32) BroadcastExchange
-Input [2]: [ca_address_sk#20, ca_state#21]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=3]
+(32) CometColumnarToRow [codegen id : 3]
+Input [2]: [ca_address_sk#20, ca_state#22]
+
+(33) BroadcastExchange
+Input [2]: [ca_address_sk#20, ca_state#22]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
-(33) BroadcastHashJoin [codegen id : 5]
+(34) BroadcastHashJoin [codegen id : 5]
Left keys [1]: [c_current_addr_sk#5]
Right keys [1]: [ca_address_sk#20]
Join type: Inner
Join condition: None
-(34) Project [codegen id : 5]
-Output [2]: [c_current_cdemo_sk#4, ca_state#21]
-Input [4]: [c_current_cdemo_sk#4, c_current_addr_sk#5, ca_address_sk#20, ca_state#21]
+(35) Project [codegen id : 5]
+Output [2]: [c_current_cdemo_sk#4, ca_state#22]
+Input [4]: [c_current_cdemo_sk#4, c_current_addr_sk#5, ca_address_sk#20, ca_state#22]
-(35) CometScan parquet spark_catalog.default.customer_demographics
-Output [6]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
+(36) CometScan parquet spark_catalog.default.customer_demographics
+Output [6]: [cd_demo_sk#23, cd_gender#24, cd_marital_status#25, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer_demographics]
PushedFilters: [IsNotNull(cd_demo_sk)]
ReadSchema: struct<cd_demo_sk:int,cd_gender:string,cd_marital_status:string,cd_dep_count:int,cd_dep_employed_count:int,cd_dep_college_count:int>
-(36) CometFilter
-Input [6]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
-Condition : isnotnull(cd_demo_sk#22)
+(37) CometFilter
+Input [6]: [cd_demo_sk#23, cd_gender#24, cd_marital_status#25, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Condition : isnotnull(cd_demo_sk#23)
+
+(38) CometProject
+Input [6]: [cd_demo_sk#23, cd_gender#24, cd_marital_status#25, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Arguments: [cd_demo_sk#23, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28], [cd_demo_sk#23, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_gender#24, 1, true, false, true) AS cd_gender#29, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, cd_marital_status#25, 1, true, false, true) AS cd_marital_status#30, cd_dep [...]
-(37) CometColumnarToRow [codegen id : 4]
-Input [6]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
+(39) CometColumnarToRow [codegen id : 4]
+Input [6]: [cd_demo_sk#23, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
-(38) BroadcastExchange
-Input [6]: [cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
-Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [plan_id=4]
+(40) BroadcastExchange
+Input [6]: [cd_demo_sk#23, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=4]
-(39) BroadcastHashJoin [codegen id : 5]
+(41) BroadcastHashJoin [codegen id : 5]
Left keys [1]: [c_current_cdemo_sk#4]
-Right keys [1]: [cd_demo_sk#22]
+Right keys [1]: [cd_demo_sk#23]
Join type: Inner
Join condition: None
-(40) Project [codegen id : 5]
-Output [6]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
-Input [8]: [c_current_cdemo_sk#4, ca_state#21, cd_demo_sk#22, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
+(42) Project [codegen id : 5]
+Output [6]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Input [8]: [c_current_cdemo_sk#4, ca_state#22, cd_demo_sk#23, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
-(41) HashAggregate [codegen id : 5]
-Input [6]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
-Keys [6]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
-Functions [10]: [partial_count(1), partial_min(cd_dep_count#25), partial_max(cd_dep_count#25), partial_avg(cd_dep_count#25), partial_min(cd_dep_employed_count#26), partial_max(cd_dep_employed_count#26), partial_avg(cd_dep_employed_count#26), partial_min(cd_dep_college_count#27), partial_max(cd_dep_college_count#27), partial_avg(cd_dep_college_count#27)]
-Aggregate Attributes [13]: [count#28, min#29, max#30, sum#31, count#32, min#33, max#34, sum#35, count#36, min#37, max#38, sum#39, count#40]
-Results [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
+(43) HashAggregate [codegen id : 5]
+Input [6]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Keys [6]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Functions [10]: [partial_count(1), partial_min(cd_dep_count#26), partial_max(cd_dep_count#26), partial_avg(cd_dep_count#26), partial_min(cd_dep_employed_count#27), partial_max(cd_dep_employed_count#27), partial_avg(cd_dep_employed_count#27), partial_min(cd_dep_college_count#28), partial_max(cd_dep_college_count#28), partial_avg(cd_dep_college_count#28)]
+Aggregate Attributes [13]: [count#31, min#32, max#33, sum#34, count#35, min#36, max#37, sum#38, count#39, min#40, max#41, sum#42, count#43]
+Results [19]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28, count#44, min#45, max#46, sum#47, count#48, min#49, max#50, sum#51, count#52, min#53, max#54, sum#55, count#56]
-(42) CometColumnarExchange
-Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
-Arguments: hashpartitioning(ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
+(44) CometColumnarExchange
+Input [19]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28, count#44, min#45, max#46, sum#47, count#48, min#49, max#50, sum#51, count#52, min#53, max#54, sum#55, count#56]
+Arguments: hashpartitioning(ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28, 5), ENSURE_REQUIREMENTS, CometColumnarShuffle, [plan_id=5]
-(43) CometColumnarToRow [codegen id : 6]
-Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
+(45) CometColumnarToRow [codegen id : 6]
+Input [19]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28, count#44, min#45, max#46, sum#47, count#48, min#49, max#50, sum#51, count#52, min#53, max#54, sum#55, count#56]
-(44) HashAggregate [codegen id : 6]
-Input [19]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27, count#41, min#42, max#43, sum#44, count#45, min#46, max#47, sum#48, count#49, min#50, max#51, sum#52, count#53]
-Keys [6]: [ca_state#21, cd_gender#23, cd_marital_status#24, cd_dep_count#25, cd_dep_employed_count#26, cd_dep_college_count#27]
-Functions [10]: [count(1), min(cd_dep_count#25), max(cd_dep_count#25), avg(cd_dep_count#25), min(cd_dep_employed_count#26), max(cd_dep_employed_count#26), avg(cd_dep_employed_count#26), min(cd_dep_college_count#27), max(cd_dep_college_count#27), avg(cd_dep_college_count#27)]
-Aggregate Attributes [10]: [count(1)#54, min(cd_dep_count#25)#55, max(cd_dep_count#25)#56, avg(cd_dep_count#25)#57, min(cd_dep_employed_count#26)#58, max(cd_dep_employed_count#26)#59, avg(cd_dep_employed_count#26)#60, min(cd_dep_college_count#27)#61, max(cd_dep_college_count#27)#62, avg(cd_dep_college_count#27)#63]
-Results [18]: [ca_state#21, cd_gender#23, cd_marital_status#24, count(1)#54 AS cnt1#64, min(cd_dep_count#25)#55 AS min(cd_dep_count)#65, max(cd_dep_count#25)#56 AS max(cd_dep_count)#66, avg(cd_dep_count#25)#57 AS avg(cd_dep_count)#67, cd_dep_employed_count#26, count(1)#54 AS cnt2#68, min(cd_dep_employed_count#26)#58 AS min(cd_dep_employed_count)#69, max(cd_dep_employed_count#26)#59 AS max(cd_dep_employed_count)#70, avg(cd_dep_employed_count#26)#60 AS avg(cd_dep_employed_count)#71, cd_dep [...]
+(46) HashAggregate [codegen id : 6]
+Input [19]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28, count#44, min#45, max#46, sum#47, count#48, min#49, max#50, sum#51, count#52, min#53, max#54, sum#55, count#56]
+Keys [6]: [ca_state#22, cd_gender#29, cd_marital_status#30, cd_dep_count#26, cd_dep_employed_count#27, cd_dep_college_count#28]
+Functions [10]: [count(1), min(cd_dep_count#26), max(cd_dep_count#26), avg(cd_dep_count#26), min(cd_dep_employed_count#27), max(cd_dep_employed_count#27), avg(cd_dep_employed_count#27), min(cd_dep_college_count#28), max(cd_dep_college_count#28), avg(cd_dep_college_count#28)]
+Aggregate Attributes [10]: [count(1)#57, min(cd_dep_count#26)#58, max(cd_dep_count#26)#59, avg(cd_dep_count#26)#60, min(cd_dep_employed_count#27)#61, max(cd_dep_employed_count#27)#62, avg(cd_dep_employed_count#27)#63, min(cd_dep_college_count#28)#64, max(cd_dep_college_count#28)#65, avg(cd_dep_college_count#28)#66]
+Results [18]: [ca_state#22, cd_gender#29, cd_marital_status#30, count(1)#57 AS cnt1#67, min(cd_dep_count#26)#58 AS min(cd_dep_count)#68, max(cd_dep_count#26)#59 AS max(cd_dep_count)#69, avg(cd_dep_count#26)#60 AS avg(cd_dep_count)#70, cd_dep_employed_count#27, count(1)#57 AS cnt2#71, min(cd_dep_employed_count#27)#61 AS min(cd_dep_employed_count)#72, max(cd_dep_employed_count#27)#62 AS max(cd_dep_employed_count)#73, avg(cd_dep_employed_count#27)#63 AS avg(cd_dep_employed_count)#74, cd_dep [...]
-(45) TakeOrderedAndProject
-Input [18]: [ca_state#21, cd_gender#23, cd_marital_status#24, cnt1#64, min(cd_dep_count)#65, max(cd_dep_count)#66, avg(cd_dep_count)#67, cd_dep_employed_count#26, cnt2#68, min(cd_dep_employed_count)#69, max(cd_dep_employed_count)#70, avg(cd_dep_employed_count)#71, cd_dep_college_count#27, cnt3#72, min(cd_dep_college_count)#73, max(cd_dep_college_count)#74, avg(cd_dep_college_count)#75, cd_dep_count#25]
-Arguments: 100, [ca_state#21 ASC NULLS FIRST, cd_gender#23 ASC NULLS FIRST, cd_marital_status#24 ASC NULLS FIRST, cd_dep_count#25 ASC NULLS FIRST, cd_dep_employed_count#26 ASC NULLS FIRST, cd_dep_college_count#27 ASC NULLS FIRST], [ca_state#21, cd_gender#23, cd_marital_status#24, cnt1#64, min(cd_dep_count)#65, max(cd_dep_count)#66, avg(cd_dep_count)#67, cd_dep_employed_count#26, cnt2#68, min(cd_dep_employed_count)#69, max(cd_dep_employed_count)#70, avg(cd_dep_employed_count)#71, cd_dep_c [...]
+(47) TakeOrderedAndProject
+Input [18]: [ca_state#22, cd_gender#29, cd_marital_status#30, cnt1#67, min(cd_dep_count)#68, max(cd_dep_count)#69, avg(cd_dep_count)#70, cd_dep_employed_count#27, cnt2#71, min(cd_dep_employed_count)#72, max(cd_dep_employed_count)#73, avg(cd_dep_employed_count)#74, cd_dep_college_count#28, cnt3#75, min(cd_dep_college_count)#76, max(cd_dep_college_count)#77, avg(cd_dep_college_count)#78, cd_dep_count#26]
+Arguments: 100, [ca_state#22 ASC NULLS FIRST, cd_gender#29 ASC NULLS FIRST, cd_marital_status#30 ASC NULLS FIRST, cd_dep_count#26 ASC NULLS FIRST, cd_dep_employed_count#27 ASC NULLS FIRST, cd_dep_college_count#28 ASC NULLS FIRST], [ca_state#22, cd_gender#29, cd_marital_status#30, cnt1#67, min(cd_dep_count)#68, max(cd_dep_count)#69, avg(cd_dep_count)#70, cd_dep_employed_count#27, cnt2#71, min(cd_dep_employed_count)#72, max(cd_dep_employed_count)#73, avg(cd_dep_employed_count)#74, cd_dep_c [...]
===== Subqueries =====
Subquery:1 Hosting operator id = 3 Hosting Expression = ss_sold_date_sk#7 IN dynamicpruning#8
-BroadcastExchange (50)
-+- * CometColumnarToRow (49)
- +- CometProject (48)
- +- CometFilter (47)
- +- CometScan parquet spark_catalog.default.date_dim (46)
+BroadcastExchange (52)
++- * CometColumnarToRow (51)
+ +- CometProject (50)
+ +- CometFilter (49)
+ +- CometScan parquet spark_catalog.default.date_dim (48)
-(46) CometScan parquet spark_catalog.default.date_dim
+(48) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#9, d_year#10, d_qoy#11]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), IsNotNull(d_qoy), EqualTo(d_year,2002), LessThan(d_qoy,4), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int,d_qoy:int>
-(47) CometFilter
+(49) CometFilter
Input [3]: [d_date_sk#9, d_year#10, d_qoy#11]
Condition : ((((isnotnull(d_year#10) AND isnotnull(d_qoy#11)) AND (d_year#10 = 2002)) AND (d_qoy#11 < 4)) AND isnotnull(d_date_sk#9))
-(48) CometProject
+(50) CometProject
Input [3]: [d_date_sk#9, d_year#10, d_qoy#11]
Arguments: [d_date_sk#9], [d_date_sk#9]
-(49) CometColumnarToRow [codegen id : 1]
+(51) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#9]
-(50) BroadcastExchange
+(52) BroadcastExchange
Input [1]: [d_date_sk#9]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=6]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt
index 8d4a8dcb9..c497f409a 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q35/simplified.txt
@@ -60,12 +60,14 @@ TakeOrderedAndProject [ca_state,cd_gender,cd_marital_status,cd_dep_count,cd_dep_
WholeStageCodegen (3)
CometColumnarToRow
InputAdapter
- CometFilter [ca_address_sk,ca_state]
- CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
+ CometProject [ca_state] [ca_address_sk,ca_state]
+ CometFilter [ca_address_sk,ca_state]
+ CometScan parquet spark_catalog.default.customer_address [ca_address_sk,ca_state]
InputAdapter
BroadcastExchange #8
WholeStageCodegen (4)
CometColumnarToRow
InputAdapter
- CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
- CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometProject [cd_gender,cd_marital_status] [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometFilter [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
+ CometScan parquet spark_catalog.default.customer_demographics [cd_demo_sk,cd_gender,cd_marital_status,cd_dep_count,cd_dep_employed_count,cd_dep_college_count]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt
index 297a9145a..af7dffb69 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/explain.txt
@@ -1,18 +1,18 @@
== Physical Plan ==
-TakeOrderedAndProject (29)
-+- * Project (28)
- +- Window (27)
- +- * CometColumnarToRow (26)
- +- CometSort (25)
- +- CometExchange (24)
- +- CometHashAggregate (23)
- +- CometExchange (22)
- +- CometHashAggregate (21)
- +- CometExpand (20)
- +- CometProject (19)
- +- CometBroadcastHashJoin (18)
- :- CometProject (13)
- : +- CometBroadcastHashJoin (12)
+TakeOrderedAndProject (30)
++- * Project (29)
+ +- Window (28)
+ +- * CometColumnarToRow (27)
+ +- CometSort (26)
+ +- CometExchange (25)
+ +- CometHashAggregate (24)
+ +- CometExchange (23)
+ +- CometHashAggregate (22)
+ +- CometExpand (21)
+ +- CometProject (20)
+ +- CometBroadcastHashJoin (19)
+ :- CometProject (14)
+ : +- CometBroadcastHashJoin (13)
: :- CometProject (8)
: : +- CometBroadcastHashJoin (7)
: : :- CometFilter (2)
@@ -21,13 +21,14 @@ TakeOrderedAndProject (29)
: : +- CometProject (5)
: : +- CometFilter (4)
: : +- CometScan parquet spark_catalog.default.date_dim (3)
- : +- CometBroadcastExchange (11)
- : +- CometFilter (10)
- : +- CometScan parquet spark_catalog.default.item (9)
- +- CometBroadcastExchange (17)
- +- CometProject (16)
- +- CometFilter (15)
- +- CometScan parquet spark_catalog.default.store (14)
+ : +- CometBroadcastExchange (12)
+ : +- CometProject (11)
+ : +- CometFilter (10)
+ : +- CometScan parquet spark_catalog.default.item (9)
+ +- CometBroadcastExchange (18)
+ +- CometProject (17)
+ +- CometFilter (16)
+ +- CometScan parquet spark_catalog.default.store (15)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -81,117 +82,121 @@ ReadSchema: struct<i_item_sk:int,i_class:string,i_category:string>
Input [3]: [i_item_sk#9, i_class#10, i_category#11]
Condition : isnotnull(i_item_sk#9)
-(11) CometBroadcastExchange
+(11) CometProject
Input [3]: [i_item_sk#9, i_class#10, i_category#11]
-Arguments: [i_item_sk#9, i_class#10, i_category#11]
+Arguments: [i_item_sk#9, i_class#12, i_category#13], [i_item_sk#9, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_class#10, 50, true, false, true) AS i_class#12, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_category#11, 50, true, false, true) AS i_category#13]
-(12) CometBroadcastHashJoin
+(12) CometBroadcastExchange
+Input [3]: [i_item_sk#9, i_class#12, i_category#13]
+Arguments: [i_item_sk#9, i_class#12, i_category#13]
+
+(13) CometBroadcastHashJoin
Left output [4]: [ss_item_sk#1, ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4]
-Right output [3]: [i_item_sk#9, i_class#10, i_category#11]
+Right output [3]: [i_item_sk#9, i_class#12, i_category#13]
Arguments: [ss_item_sk#1], [i_item_sk#9], Inner, BuildRight
-(13) CometProject
-Input [7]: [ss_item_sk#1, ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_item_sk#9, i_class#10, i_category#11]
-Arguments: [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#10, i_category#11], [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#10, i_category#11]
+(14) CometProject
+Input [7]: [ss_item_sk#1, ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_item_sk#9, i_class#12, i_category#13]
+Arguments: [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#12, i_category#13], [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#12, i_category#13]
-(14) CometScan parquet spark_catalog.default.store
-Output [2]: [s_store_sk#12, s_state#13]
+(15) CometScan parquet spark_catalog.default.store
+Output [2]: [s_store_sk#14, s_state#15]
Batched: true
Location [not included in comparison]/{warehouse_dir}/store]
-PushedFilters: [IsNotNull(s_state), IsNotNull(s_store_sk)]
+PushedFilters: [IsNotNull(s_store_sk)]
ReadSchema: struct<s_store_sk:int,s_state:string>
-(15) CometFilter
-Input [2]: [s_store_sk#12, s_state#13]
-Condition : ((isnotnull(s_state#13) AND (rpad(s_state#13, 2, ) = TN)) AND isnotnull(s_store_sk#12))
+(16) CometFilter
+Input [2]: [s_store_sk#14, s_state#15]
+Condition : ((staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, s_state#15, 2, true, false, true) = TN) AND isnotnull(s_store_sk#14))
-(16) CometProject
-Input [2]: [s_store_sk#12, s_state#13]
-Arguments: [s_store_sk#12], [s_store_sk#12]
+(17) CometProject
+Input [2]: [s_store_sk#14, s_state#15]
+Arguments: [s_store_sk#14], [s_store_sk#14]
-(17) CometBroadcastExchange
-Input [1]: [s_store_sk#12]
-Arguments: [s_store_sk#12]
+(18) CometBroadcastExchange
+Input [1]: [s_store_sk#14]
+Arguments: [s_store_sk#14]
-(18) CometBroadcastHashJoin
-Left output [5]: [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#10, i_category#11]
-Right output [1]: [s_store_sk#12]
-Arguments: [ss_store_sk#2], [s_store_sk#12], Inner, BuildRight
+(19) CometBroadcastHashJoin
+Left output [5]: [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#12, i_category#13]
+Right output [1]: [s_store_sk#14]
+Arguments: [ss_store_sk#2], [s_store_sk#14], Inner, BuildRight
-(19) CometProject
-Input [6]: [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#10, i_category#11, s_store_sk#12]
-Arguments: [ss_ext_sales_price#3, ss_net_profit#4, i_category#11, i_class#10], [ss_ext_sales_price#3, ss_net_profit#4, i_category#11, i_class#10]
+(20) CometProject
+Input [6]: [ss_store_sk#2, ss_ext_sales_price#3, ss_net_profit#4, i_class#12, i_category#13, s_store_sk#14]
+Arguments: [ss_ext_sales_price#3, ss_net_profit#4, i_category#13, i_class#12], [ss_ext_sales_price#3, ss_net_profit#4, i_category#13, i_class#12]
-(20) CometExpand
-Input [4]: [ss_ext_sales_price#3, ss_net_profit#4, i_category#11, i_class#10]
-Arguments: [[ss_ext_sales_price#3, ss_net_profit#4, i_category#11, i_class#10, 0], [ss_ext_sales_price#3, ss_net_profit#4, i_category#11, null, 1], [ss_ext_sales_price#3, ss_net_profit#4, null, null, 3]], [ss_ext_sales_price#3, ss_net_profit#4, i_category#14, i_class#15, spark_grouping_id#16]
+(21) CometExpand
+Input [4]: [ss_ext_sales_price#3, ss_net_profit#4, i_category#13, i_class#12]
+Arguments: [[ss_ext_sales_price#3, ss_net_profit#4, i_category#13, i_class#12, 0], [ss_ext_sales_price#3, ss_net_profit#4, i_category#13, null, 1], [ss_ext_sales_price#3, ss_net_profit#4, null, null, 3]], [ss_ext_sales_price#3, ss_net_profit#4, i_category#16, i_class#17, spark_grouping_id#18]
-(21) CometHashAggregate
-Input [5]: [ss_ext_sales_price#3, ss_net_profit#4, i_category#14, i_class#15, spark_grouping_id#16]
-Keys [3]: [i_category#14, i_class#15, spark_grouping_id#16]
+(22) CometHashAggregate
+Input [5]: [ss_ext_sales_price#3, ss_net_profit#4, i_category#16, i_class#17, spark_grouping_id#18]
+Keys [3]: [i_category#16, i_class#17, spark_grouping_id#18]
Functions [2]: [partial_sum(UnscaledValue(ss_net_profit#4)), partial_sum(UnscaledValue(ss_ext_sales_price#3))]
-(22) CometExchange
-Input [5]: [i_category#14, i_class#15, spark_grouping_id#16, sum#17, sum#18]
-Arguments: hashpartitioning(i_category#14, i_class#15, spark_grouping_id#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(23) CometExchange
+Input [5]: [i_category#16, i_class#17, spark_grouping_id#18, sum#19, sum#20]
+Arguments: hashpartitioning(i_category#16, i_class#17, spark_grouping_id#18, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(23) CometHashAggregate
-Input [5]: [i_category#14, i_class#15, spark_grouping_id#16, sum#17, sum#18]
-Keys [3]: [i_category#14, i_class#15, spark_grouping_id#16]
+(24) CometHashAggregate
+Input [5]: [i_category#16, i_class#17, spark_grouping_id#18, sum#19, sum#20]
+Keys [3]: [i_category#16, i_class#17, spark_grouping_id#18]
Functions [2]: [sum(UnscaledValue(ss_net_profit#4)), sum(UnscaledValue(ss_ext_sales_price#3))]
-(24) CometExchange
-Input [7]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23]
-Arguments: hashpartitioning(_w1#22, _w2#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+(25) CometExchange
+Input [7]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, _w0#23, _w1#24, _w2#25]
+Arguments: hashpartitioning(_w1#24, _w2#25, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(25) CometSort
-Input [7]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23]
-Arguments: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23], [_w1#22 ASC NULLS FIRST, _w2#23 ASC NULLS FIRST, _w0#21 ASC NULLS FIRST]
+(26) CometSort
+Input [7]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, _w0#23, _w1#24, _w2#25]
+Arguments: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, _w0#23, _w1#24, _w2#25], [_w1#24 ASC NULLS FIRST, _w2#25 ASC NULLS FIRST, _w0#23 ASC NULLS FIRST]
-(26) CometColumnarToRow [codegen id : 1]
-Input [7]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23]
+(27) CometColumnarToRow [codegen id : 1]
+Input [7]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, _w0#23, _w1#24, _w2#25]
-(27) Window
-Input [7]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23]
-Arguments: [rank(_w0#21) windowspecdefinition(_w1#22, _w2#23, _w0#21 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank_within_parent#24], [_w1#22, _w2#23], [_w0#21 ASC NULLS FIRST]
+(28) Window
+Input [7]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, _w0#23, _w1#24, _w2#25]
+Arguments: [rank(_w0#23) windowspecdefinition(_w1#24, _w2#25, _w0#23 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank_within_parent#26], [_w1#24, _w2#25], [_w0#23 ASC NULLS FIRST]
-(28) Project [codegen id : 2]
-Output [5]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, rank_within_parent#24]
-Input [8]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, _w0#21, _w1#22, _w2#23, rank_within_parent#24]
+(29) Project [codegen id : 2]
+Output [5]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, rank_within_parent#26]
+Input [8]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, _w0#23, _w1#24, _w2#25, rank_within_parent#26]
-(29) TakeOrderedAndProject
-Input [5]: [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, rank_within_parent#24]
-Arguments: 100, [lochierarchy#20 DESC NULLS LAST, CASE WHEN (lochierarchy#20 = 0) THEN i_category#14 END ASC NULLS FIRST, rank_within_parent#24 ASC NULLS FIRST], [gross_margin#19, i_category#14, i_class#15, lochierarchy#20, rank_within_parent#24]
+(30) TakeOrderedAndProject
+Input [5]: [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, rank_within_parent#26]
+Arguments: 100, [lochierarchy#22 DESC NULLS LAST, CASE WHEN (lochierarchy#22 = 0) THEN i_category#16 END ASC NULLS FIRST, rank_within_parent#26 ASC NULLS FIRST], [gross_margin#21, i_category#16, i_class#17, lochierarchy#22, rank_within_parent#26]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#5 IN dynamicpruning#6
-BroadcastExchange (34)
-+- * CometColumnarToRow (33)
- +- CometProject (32)
- +- CometFilter (31)
- +- CometScan parquet spark_catalog.default.date_dim (30)
+BroadcastExchange (35)
++- * CometColumnarToRow (34)
+ +- CometProject (33)
+ +- CometFilter (32)
+ +- CometScan parquet spark_catalog.default.date_dim (31)
-(30) CometScan parquet spark_catalog.default.date_dim
+(31) CometScan parquet spark_catalog.default.date_dim
Output [2]: [d_date_sk#7, d_year#8]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(31) CometFilter
+(32) CometFilter
Input [2]: [d_date_sk#7, d_year#8]
Condition : ((isnotnull(d_year#8) AND (d_year#8 = 2001)) AND isnotnull(d_date_sk#7))
-(32) CometProject
+(33) CometProject
Input [2]: [d_date_sk#7, d_year#8]
Arguments: [d_date_sk#7], [d_date_sk#7]
-(33) CometColumnarToRow [codegen id : 1]
+(34) CometColumnarToRow [codegen id : 1]
Input [1]: [d_date_sk#7]
-(34) BroadcastExchange
+(35) BroadcastExchange
Input [1]: [d_date_sk#7]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=3]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt
index 5b8623c2b..a44d822d7 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q36/simplified.txt
@@ -33,8 +33,9 @@ TakeOrderedAndProject [lochierarchy,i_category,rank_within_parent,gross_margin,i
CometFilter [d_date_sk,d_year]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
CometBroadcastExchange [i_item_sk,i_class,i_category] #5
- CometFilter [i_item_sk,i_class,i_category]
- CometScan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
+ CometProject [i_class,i_category] [i_item_sk,i_class,i_category]
+ CometFilter [i_item_sk,i_class,i_category]
+ CometScan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
CometBroadcastExchange [s_store_sk] #6
CometProject [s_store_sk]
CometFilter [s_store_sk,s_state]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt
index 3f78c57bb..3471befec 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/explain.txt
@@ -40,117 +40,117 @@ Condition : ((((isnotnull(i_current_price#4) AND (i_current_price#4 >= 68.00)) A
(3) CometProject
Input [5]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, i_manufact_id#5]
-Arguments: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4], [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4]
+Arguments: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4], [i_item_sk#1, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, i_item_id#2, 16, true, false, true) AS i_item_id#6, i_item_desc#3, i_current_price#4]
(4) CometScan parquet spark_catalog.default.inventory
-Output [3]: [inv_item_sk#6, inv_quantity_on_hand#7, inv_date_sk#8]
+Output [3]: [inv_item_sk#7, inv_quantity_on_hand#8, inv_date_sk#9]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(inv_date_sk#8), dynamicpruningexpression(inv_date_sk#8 IN dynamicpruning#9)]
+PartitionFilters: [isnotnull(inv_date_sk#9), dynamicpruningexpression(inv_date_sk#9 IN dynamicpruning#10)]
PushedFilters: [IsNotNull(inv_quantity_on_hand), GreaterThanOrEqual(inv_quantity_on_hand,100), LessThanOrEqual(inv_quantity_on_hand,500), IsNotNull(inv_item_sk)]
ReadSchema: struct<inv_item_sk:int,inv_quantity_on_hand:int>
(5) CometFilter
-Input [3]: [inv_item_sk#6, inv_quantity_on_hand#7, inv_date_sk#8]
-Condition : (((isnotnull(inv_quantity_on_hand#7) AND (inv_quantity_on_hand#7 >= 100)) AND (inv_quantity_on_hand#7 <= 500)) AND isnotnull(inv_item_sk#6))
+Input [3]: [inv_item_sk#7, inv_quantity_on_hand#8, inv_date_sk#9]
+Condition : (((isnotnull(inv_quantity_on_hand#8) AND (inv_quantity_on_hand#8 >= 100)) AND (inv_quantity_on_hand#8 <= 500)) AND isnotnull(inv_item_sk#7))
(6) CometProject
-Input [3]: [inv_item_sk#6, inv_quantity_on_hand#7, inv_date_sk#8]
-Arguments: [inv_item_sk#6, inv_date_sk#8], [inv_item_sk#6, inv_date_sk#8]
+Input [3]: [inv_item_sk#7, inv_quantity_on_hand#8, inv_date_sk#9]
+Arguments: [inv_item_sk#7, inv_date_sk#9], [inv_item_sk#7, inv_date_sk#9]
(7) CometBroadcastExchange
-Input [2]: [inv_item_sk#6, inv_date_sk#8]
-Arguments: [inv_item_sk#6, inv_date_sk#8]
+Input [2]: [inv_item_sk#7, inv_date_sk#9]
+Arguments: [inv_item_sk#7, inv_date_sk#9]
(8) CometBroadcastHashJoin
-Left output [4]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4]
-Right output [2]: [inv_item_sk#6, inv_date_sk#8]
-Arguments: [i_item_sk#1], [inv_item_sk#6], Inner, BuildRight
+Left output [4]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4]
+Right output [2]: [inv_item_sk#7, inv_date_sk#9]
+Arguments: [i_item_sk#1], [inv_item_sk#7], Inner, BuildRight
(9) CometProject
-Input [6]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, inv_item_sk#6, inv_date_sk#8]
-Arguments: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, inv_date_sk#8], [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, inv_date_sk#8]
+Input [6]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4, inv_item_sk#7, inv_date_sk#9]
+Arguments: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4, inv_date_sk#9], [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4, inv_date_sk#9]
(10) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#10, d_date#11]
+Output [2]: [d_date_sk#11, d_date#12]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-02-01), LessThanOrEqual(d_date,2000-04-01), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
(11) CometFilter
-Input [2]: [d_date_sk#10, d_date#11]
-Condition : (((isnotnull(d_date#11) AND (d_date#11 >= 2000-02-01)) AND (d_date#11 <= 2000-04-01)) AND isnotnull(d_date_sk#10))
+Input [2]: [d_date_sk#11, d_date#12]
+Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 2000-02-01)) AND (d_date#12 <= 2000-04-01)) AND isnotnull(d_date_sk#11))
(12) CometProject
-Input [2]: [d_date_sk#10, d_date#11]
-Arguments: [d_date_sk#10], [d_date_sk#10]
+Input [2]: [d_date_sk#11, d_date#12]
+Arguments: [d_date_sk#11], [d_date_sk#11]
(13) CometBroadcastExchange
-Input [1]: [d_date_sk#10]
-Arguments: [d_date_sk#10]
+Input [1]: [d_date_sk#11]
+Arguments: [d_date_sk#11]
(14) CometBroadcastHashJoin
-Left output [5]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, inv_date_sk#8]
-Right output [1]: [d_date_sk#10]
-Arguments: [inv_date_sk#8], [d_date_sk#10], Inner, BuildRight
+Left output [5]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4, inv_date_sk#9]
+Right output [1]: [d_date_sk#11]
+Arguments: [inv_date_sk#9], [d_date_sk#11], Inner, BuildRight
(15) CometProject
-Input [6]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, inv_date_sk#8, d_date_sk#10]
-Arguments: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4], [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [6]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4, inv_date_sk#9, d_date_sk#11]
+Arguments: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4], [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4]
(16) CometBroadcastExchange
-Input [4]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4]
-Arguments: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [4]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4]
+Arguments: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4]
(17) CometScan parquet spark_catalog.default.catalog_sales
-Output [2]: [cs_item_sk#12, cs_sold_date_sk#13]
+Output [2]: [cs_item_sk#13, cs_sold_date_sk#14]
Batched: true
Location [not included in comparison]/{warehouse_dir}/catalog_sales]
PushedFilters: [IsNotNull(cs_item_sk)]
ReadSchema: struct<cs_item_sk:int>
(18) CometFilter
-Input [2]: [cs_item_sk#12, cs_sold_date_sk#13]
-Condition : isnotnull(cs_item_sk#12)
+Input [2]: [cs_item_sk#13, cs_sold_date_sk#14]
+Condition : isnotnull(cs_item_sk#13)
(19) CometProject
-Input [2]: [cs_item_sk#12, cs_sold_date_sk#13]
-Arguments: [cs_item_sk#12], [cs_item_sk#12]
+Input [2]: [cs_item_sk#13, cs_sold_date_sk#14]
+Arguments: [cs_item_sk#13], [cs_item_sk#13]
(20) CometBroadcastHashJoin
-Left output [4]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4]
-Right output [1]: [cs_item_sk#12]
-Arguments: [i_item_sk#1], [cs_item_sk#12], Inner, BuildLeft
+Left output [4]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4]
+Right output [1]: [cs_item_sk#13]
+Arguments: [i_item_sk#1], [cs_item_sk#13], Inner, BuildLeft
(21) CometProject
-Input [5]: [i_item_sk#1, i_item_id#2, i_item_desc#3, i_current_price#4, cs_item_sk#12]
-Arguments: [i_item_id#2, i_item_desc#3, i_current_price#4], [i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [5]: [i_item_sk#1, i_item_id#6, i_item_desc#3, i_current_price#4, cs_item_sk#13]
+Arguments: [i_item_id#6, i_item_desc#3, i_current_price#4], [i_item_id#6, i_item_desc#3, i_current_price#4]
(22) CometHashAggregate
-Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
-Keys [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
+Keys [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
Functions: []
(23) CometExchange
-Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
-Arguments: hashpartitioning(i_item_id#2, i_item_desc#3, i_current_price#4, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+Input [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
+Arguments: hashpartitioning(i_item_id#6, i_item_desc#3, i_current_price#4, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
(24) CometHashAggregate
-Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
-Keys [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
+Keys [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
Functions: []
(25) CometTakeOrderedAndProject
-Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
-Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#2 ASC NULLS FIRST], output=[i_item_id#2,i_item_desc#3,i_current_price#4]), [i_item_id#2, i_item_desc#3, i_current_price#4], 100, [i_item_id#2 ASC NULLS FIRST], [i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
+Arguments: TakeOrderedAndProject(limit=100, orderBy=[i_item_id#6 ASC NULLS FIRST], output=[i_item_id#6,i_item_desc#3,i_current_price#4]), [i_item_id#6, i_item_desc#3, i_current_price#4], 100, [i_item_id#6 ASC NULLS FIRST], [i_item_id#6, i_item_desc#3, i_current_price#4]
(26) CometColumnarToRow [codegen id : 1]
-Input [3]: [i_item_id#2, i_item_desc#3, i_current_price#4]
+Input [3]: [i_item_id#6, i_item_desc#3, i_current_price#4]
===== Subqueries =====
-Subquery:1 Hosting operator id = 4 Hosting Expression = inv_date_sk#8 IN dynamicpruning#9
+Subquery:1 Hosting operator id = 4 Hosting Expression = inv_date_sk#9 IN dynamicpruning#10
BroadcastExchange (31)
+- * CometColumnarToRow (30)
+- CometProject (29)
@@ -159,25 +159,25 @@ BroadcastExchange (31)
(27) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#10, d_date#11]
+Output [2]: [d_date_sk#11, d_date#12]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_date), GreaterThanOrEqual(d_date,2000-02-01), LessThanOrEqual(d_date,2000-04-01), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date>
(28) CometFilter
-Input [2]: [d_date_sk#10, d_date#11]
-Condition : (((isnotnull(d_date#11) AND (d_date#11 >= 2000-02-01)) AND (d_date#11 <= 2000-04-01)) AND isnotnull(d_date_sk#10))
+Input [2]: [d_date_sk#11, d_date#12]
+Condition : (((isnotnull(d_date#12) AND (d_date#12 >= 2000-02-01)) AND (d_date#12 <= 2000-04-01)) AND isnotnull(d_date_sk#11))
(29) CometProject
-Input [2]: [d_date_sk#10, d_date#11]
-Arguments: [d_date_sk#10], [d_date_sk#10]
+Input [2]: [d_date_sk#11, d_date#12]
+Arguments: [d_date_sk#11], [d_date_sk#11]
(30) CometColumnarToRow [codegen id : 1]
-Input [1]: [d_date_sk#10]
+Input [1]: [d_date_sk#11]
(31) BroadcastExchange
-Input [1]: [d_date_sk#10]
+Input [1]: [d_date_sk#11]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=2]
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt
index 13fcba859..b49b964bb 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q37/simplified.txt
@@ -12,7 +12,7 @@ WholeStageCodegen (1)
CometBroadcastHashJoin [i_item_sk,i_item_id,i_item_desc,i_current_price,inv_date_sk,d_date_sk]
CometProject [i_item_sk,i_item_id,i_item_desc,i_current_price,inv_date_sk]
CometBroadcastHashJoin [i_item_sk,i_item_id,i_item_desc,i_current_price,inv_item_sk,inv_date_sk]
- CometProject [i_item_sk,i_item_id,i_item_desc,i_current_price]
+ CometProject [i_item_id] [i_item_sk,i_item_id,i_item_desc,i_current_price]
CometFilter [i_item_sk,i_item_id,i_item_desc,i_current_price,i_manufact_id]
CometScan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc,i_current_price,i_manufact_id]
CometBroadcastExchange [inv_item_sk,inv_date_sk] #3
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt
index 8139d06c0..a6a7d781f 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/explain.txt
@@ -1,16 +1,16 @@
== Physical Plan ==
-* CometColumnarToRow (47)
-+- CometHashAggregate (46)
- +- CometExchange (45)
- +- CometHashAggregate (44)
- +- CometProject (43)
- +- CometBroadcastHashJoin (42)
- :- CometBroadcastHashJoin (29)
- : :- CometHashAggregate (16)
- : : +- CometExchange (15)
- : : +- CometHashAggregate (14)
- : : +- CometProject (13)
- : : +- CometBroadcastHashJoin (12)
+* CometColumnarToRow (48)
++- CometHashAggregate (47)
+ +- CometExchange (46)
+ +- CometHashAggregate (45)
+ +- CometProject (44)
+ +- CometBroadcastHashJoin (43)
+ :- CometBroadcastHashJoin (30)
+ : :- CometHashAggregate (17)
+ : : +- CometExchange (16)
+ : : +- CometHashAggregate (15)
+ : : +- CometProject (14)
+ : : +- CometBroadcastHashJoin (13)
: : :- CometProject (8)
: : : +- CometBroadcastHashJoin (7)
: : : :- CometFilter (2)
@@ -19,33 +19,34 @@
: : : +- CometProject (5)
: : : +- CometFilter (4)
: : : +- CometScan parquet spark_catalog.default.date_dim (3)
- : : +- CometBroadcastExchange (11)
- : : +- CometFilter (10)
- : : +- CometScan parquet spark_catalog.default.customer (9)
- : +- CometBroadcastExchange (28)
- : +- CometHashAggregate (27)
- : +- CometExchange (26)
- : +- CometHashAggregate (25)
- : +- CometProject (24)
- : +- CometBroadcastHashJoin (23)
- : :- CometProject (21)
- : : +- CometBroadcastHashJoin (20)
- : : :- CometFilter (18)
- : : : +- CometScan parquet spark_catalog.default.catalog_sales (17)
- : : +- ReusedExchange (19)
- : +- ReusedExchange (22)
- +- CometBroadcastExchange (41)
- +- CometHashAggregate (40)
- +- CometExchange (39)
- +- CometHashAggregate (38)
- +- CometProject (37)
- +- CometBroadcastHashJoin (36)
- :- CometProject (34)
- : +- CometBroadcastHashJoin (33)
- : :- CometFilter (31)
- : : +- CometScan parquet spark_catalog.default.web_sales (30)
- : +- ReusedExchange (32)
- +- ReusedExchange (35)
+ : : +- CometBroadcastExchange (12)
+ : : +- CometProject (11)
+ : : +- CometFilter (10)
+ : : +- CometScan parquet spark_catalog.default.customer (9)
+ : +- CometBroadcastExchange (29)
+ : +- CometHashAggregate (28)
+ : +- CometExchange (27)
+ : +- CometHashAggregate (26)
+ : +- CometProject (25)
+ : +- CometBroadcastHashJoin (24)
+ : :- CometProject (22)
+ : : +- CometBroadcastHashJoin (21)
+ : : :- CometFilter (19)
+ : : : +- CometScan parquet spark_catalog.default.catalog_sales (18)
+ : : +- ReusedExchange (20)
+ : +- ReusedExchange (23)
+ +- CometBroadcastExchange (42)
+ +- CometHashAggregate (41)
+ +- CometExchange (40)
+ +- CometHashAggregate (39)
+ +- CometProject (38)
+ +- CometBroadcastHashJoin (37)
+ :- CometProject (35)
+ : +- CometBroadcastHashJoin (34)
+ : :- CometFilter (32)
+ : : +- CometScan parquet spark_catalog.default.web_sales (31)
+ : +- ReusedExchange (33)
+ +- ReusedExchange (36)
(1) CometScan parquet spark_catalog.default.store_sales
@@ -99,205 +100,209 @@ ReadSchema: struct<c_customer_sk:int,c_first_name:string,c_last_name:string>
Input [3]: [c_customer_sk#7, c_first_name#8, c_last_name#9]
Condition : isnotnull(c_customer_sk#7)
-(11) CometBroadcastExchange
+(11) CometProject
Input [3]: [c_customer_sk#7, c_first_name#8, c_last_name#9]
-Arguments: [c_customer_sk#7, c_first_name#8, c_last_name#9]
+Arguments: [c_customer_sk#7, c_first_name#10, c_last_name#11], [c_customer_sk#7, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#8, 20, true, false, true) AS c_first_name#10, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_last_name#9, 30, true, false, true) AS c_last_name#11]
-(12) CometBroadcastHashJoin
+(12) CometBroadcastExchange
+Input [3]: [c_customer_sk#7, c_first_name#10, c_last_name#11]
+Arguments: [c_customer_sk#7, c_first_name#10, c_last_name#11]
+
+(13) CometBroadcastHashJoin
Left output [2]: [ss_customer_sk#1, d_date#5]
-Right output [3]: [c_customer_sk#7, c_first_name#8, c_last_name#9]
+Right output [3]: [c_customer_sk#7, c_first_name#10, c_last_name#11]
Arguments: [ss_customer_sk#1], [c_customer_sk#7], Inner, BuildRight
-(13) CometProject
-Input [5]: [ss_customer_sk#1, d_date#5, c_customer_sk#7, c_first_name#8, c_last_name#9]
-Arguments: [c_last_name#9, c_first_name#8, d_date#5], [c_last_name#9, c_first_name#8, d_date#5]
+(14) CometProject
+Input [5]: [ss_customer_sk#1, d_date#5, c_customer_sk#7, c_first_name#10, c_last_name#11]
+Arguments: [c_last_name#11, c_first_name#10, d_date#5], [c_last_name#11, c_first_name#10, d_date#5]
-(14) CometHashAggregate
-Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
-Keys [3]: [c_last_name#9, c_first_name#8, d_date#5]
+(15) CometHashAggregate
+Input [3]: [c_last_name#11, c_first_name#10, d_date#5]
+Keys [3]: [c_last_name#11, c_first_name#10, d_date#5]
Functions: []
-(15) CometExchange
-Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
-Arguments: hashpartitioning(c_last_name#9, c_first_name#8, d_date#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(16) CometExchange
+Input [3]: [c_last_name#11, c_first_name#10, d_date#5]
+Arguments: hashpartitioning(c_last_name#11, c_first_name#10, d_date#5, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(16) CometHashAggregate
-Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
-Keys [3]: [c_last_name#9, c_first_name#8, d_date#5]
+(17) CometHashAggregate
+Input [3]: [c_last_name#11, c_first_name#10, d_date#5]
+Keys [3]: [c_last_name#11, c_first_name#10, d_date#5]
Functions: []
-(17) CometScan parquet spark_catalog.default.catalog_sales
-Output [2]: [cs_bill_customer_sk#10, cs_sold_date_sk#11]
+(18) CometScan parquet spark_catalog.default.catalog_sales
+Output [2]: [cs_bill_customer_sk#12, cs_sold_date_sk#13]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(cs_sold_date_sk#11), dynamicpruningexpression(cs_sold_date_sk#11 IN dynamicpruning#12)]
+PartitionFilters: [isnotnull(cs_sold_date_sk#13), dynamicpruningexpression(cs_sold_date_sk#13 IN dynamicpruning#14)]
PushedFilters: [IsNotNull(cs_bill_customer_sk)]
ReadSchema: struct<cs_bill_customer_sk:int>
-(18) CometFilter
-Input [2]: [cs_bill_customer_sk#10, cs_sold_date_sk#11]
-Condition : isnotnull(cs_bill_customer_sk#10)
+(19) CometFilter
+Input [2]: [cs_bill_customer_sk#12, cs_sold_date_sk#13]
+Condition : isnotnull(cs_bill_customer_sk#12)
-(19) ReusedExchange [Reuses operator id: 6]
-Output [2]: [d_date_sk#13, d_date#14]
+(20) ReusedExchange [Reuses operator id: 6]
+Output [2]: [d_date_sk#15, d_date#16]
-(20) CometBroadcastHashJoin
-Left output [2]: [cs_bill_customer_sk#10, cs_sold_date_sk#11]
-Right output [2]: [d_date_sk#13, d_date#14]
-Arguments: [cs_sold_date_sk#11], [d_date_sk#13], Inner, BuildRight
+(21) CometBroadcastHashJoin
+Left output [2]: [cs_bill_customer_sk#12, cs_sold_date_sk#13]
+Right output [2]: [d_date_sk#15, d_date#16]
+Arguments: [cs_sold_date_sk#13], [d_date_sk#15], Inner, BuildRight
-(21) CometProject
-Input [4]: [cs_bill_customer_sk#10, cs_sold_date_sk#11, d_date_sk#13, d_date#14]
-Arguments: [cs_bill_customer_sk#10, d_date#14], [cs_bill_customer_sk#10, d_date#14]
+(22) CometProject
+Input [4]: [cs_bill_customer_sk#12, cs_sold_date_sk#13, d_date_sk#15, d_date#16]
+Arguments: [cs_bill_customer_sk#12, d_date#16], [cs_bill_customer_sk#12, d_date#16]
-(22) ReusedExchange [Reuses operator id: 11]
-Output [3]: [c_customer_sk#15, c_first_name#16, c_last_name#17]
+(23) ReusedExchange [Reuses operator id: 12]
+Output [3]: [c_customer_sk#17, c_first_name#18, c_last_name#19]
-(23) CometBroadcastHashJoin
-Left output [2]: [cs_bill_customer_sk#10, d_date#14]
-Right output [3]: [c_customer_sk#15, c_first_name#16, c_last_name#17]
-Arguments: [cs_bill_customer_sk#10], [c_customer_sk#15], Inner, BuildRight
+(24) CometBroadcastHashJoin
+Left output [2]: [cs_bill_customer_sk#12, d_date#16]
+Right output [3]: [c_customer_sk#17, c_first_name#18, c_last_name#19]
+Arguments: [cs_bill_customer_sk#12], [c_customer_sk#17], Inner, BuildRight
-(24) CometProject
-Input [5]: [cs_bill_customer_sk#10, d_date#14, c_customer_sk#15, c_first_name#16, c_last_name#17]
-Arguments: [c_last_name#17, c_first_name#16, d_date#14], [c_last_name#17, c_first_name#16, d_date#14]
+(25) CometProject
+Input [5]: [cs_bill_customer_sk#12, d_date#16, c_customer_sk#17, c_first_name#18, c_last_name#19]
+Arguments: [c_last_name#19, c_first_name#18, d_date#16], [c_last_name#19, c_first_name#18, d_date#16]
-(25) CometHashAggregate
-Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
-Keys [3]: [c_last_name#17, c_first_name#16, d_date#14]
+(26) CometHashAggregate
+Input [3]: [c_last_name#19, c_first_name#18, d_date#16]
+Keys [3]: [c_last_name#19, c_first_name#18, d_date#16]
Functions: []
-(26) CometExchange
-Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
-Arguments: hashpartitioning(c_last_name#17, c_first_name#16, d_date#14, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+(27) CometExchange
+Input [3]: [c_last_name#19, c_first_name#18, d_date#16]
+Arguments: hashpartitioning(c_last_name#19, c_first_name#18, d_date#16, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
-(27) CometHashAggregate
-Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
-Keys [3]: [c_last_name#17, c_first_name#16, d_date#14]
+(28) CometHashAggregate
+Input [3]: [c_last_name#19, c_first_name#18, d_date#16]
+Keys [3]: [c_last_name#19, c_first_name#18, d_date#16]
Functions: []
-(28) CometBroadcastExchange
-Input [3]: [c_last_name#17, c_first_name#16, d_date#14]
-Arguments: [c_last_name#17, c_first_name#16, d_date#14]
+(29) CometBroadcastExchange
+Input [3]: [c_last_name#19, c_first_name#18, d_date#16]
+Arguments: [c_last_name#19, c_first_name#18, d_date#16]
-(29) CometBroadcastHashJoin
-Left output [3]: [c_last_name#9, c_first_name#8, d_date#5]
-Right output [3]: [c_last_name#17, c_first_name#16, d_date#14]
-Arguments: [coalesce(c_last_name#9, ), isnull(c_last_name#9), coalesce(c_first_name#8, ), isnull(c_first_name#8), coalesce(d_date#5, 1970-01-01), isnull(d_date#5)], [coalesce(c_last_name#17, ), isnull(c_last_name#17), coalesce(c_first_name#16, ), isnull(c_first_name#16), coalesce(d_date#14, 1970-01-01), isnull(d_date#14)], LeftSemi, BuildRight
+(30) CometBroadcastHashJoin
+Left output [3]: [c_last_name#11, c_first_name#10, d_date#5]
+Right output [3]: [c_last_name#19, c_first_name#18, d_date#16]
+Arguments: [coalesce(c_last_name#11, ), isnull(c_last_name#11), coalesce(c_first_name#10, ), isnull(c_first_name#10), coalesce(d_date#5, 1970-01-01), isnull(d_date#5)], [coalesce(c_last_name#19, ), isnull(c_last_name#19), coalesce(c_first_name#18, ), isnull(c_first_name#18), coalesce(d_date#16, 1970-01-01), isnull(d_date#16)], LeftSemi, BuildRight
-(30) CometScan parquet spark_catalog.default.web_sales
-Output [2]: [ws_bill_customer_sk#18, ws_sold_date_sk#19]
+(31) CometScan parquet spark_catalog.default.web_sales
+Output [2]: [ws_bill_customer_sk#20, ws_sold_date_sk#21]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ws_sold_date_sk#19), dynamicpruningexpression(ws_sold_date_sk#19 IN dynamicpruning#20)]
+PartitionFilters: [isnotnull(ws_sold_date_sk#21), dynamicpruningexpression(ws_sold_date_sk#21 IN dynamicpruning#22)]
PushedFilters: [IsNotNull(ws_bill_customer_sk)]
ReadSchema: struct<ws_bill_customer_sk:int>
-(31) CometFilter
-Input [2]: [ws_bill_customer_sk#18, ws_sold_date_sk#19]
-Condition : isnotnull(ws_bill_customer_sk#18)
+(32) CometFilter
+Input [2]: [ws_bill_customer_sk#20, ws_sold_date_sk#21]
+Condition : isnotnull(ws_bill_customer_sk#20)
-(32) ReusedExchange [Reuses operator id: 6]
-Output [2]: [d_date_sk#21, d_date#22]
+(33) ReusedExchange [Reuses operator id: 6]
+Output [2]: [d_date_sk#23, d_date#24]
-(33) CometBroadcastHashJoin
-Left output [2]: [ws_bill_customer_sk#18, ws_sold_date_sk#19]
-Right output [2]: [d_date_sk#21, d_date#22]
-Arguments: [ws_sold_date_sk#19], [d_date_sk#21], Inner, BuildRight
+(34) CometBroadcastHashJoin
+Left output [2]: [ws_bill_customer_sk#20, ws_sold_date_sk#21]
+Right output [2]: [d_date_sk#23, d_date#24]
+Arguments: [ws_sold_date_sk#21], [d_date_sk#23], Inner, BuildRight
-(34) CometProject
-Input [4]: [ws_bill_customer_sk#18, ws_sold_date_sk#19, d_date_sk#21, d_date#22]
-Arguments: [ws_bill_customer_sk#18, d_date#22], [ws_bill_customer_sk#18, d_date#22]
+(35) CometProject
+Input [4]: [ws_bill_customer_sk#20, ws_sold_date_sk#21, d_date_sk#23, d_date#24]
+Arguments: [ws_bill_customer_sk#20, d_date#24], [ws_bill_customer_sk#20, d_date#24]
-(35) ReusedExchange [Reuses operator id: 11]
-Output [3]: [c_customer_sk#23, c_first_name#24, c_last_name#25]
+(36) ReusedExchange [Reuses operator id: 12]
+Output [3]: [c_customer_sk#25, c_first_name#26, c_last_name#27]
-(36) CometBroadcastHashJoin
-Left output [2]: [ws_bill_customer_sk#18, d_date#22]
-Right output [3]: [c_customer_sk#23, c_first_name#24, c_last_name#25]
-Arguments: [ws_bill_customer_sk#18], [c_customer_sk#23], Inner, BuildRight
+(37) CometBroadcastHashJoin
+Left output [2]: [ws_bill_customer_sk#20, d_date#24]
+Right output [3]: [c_customer_sk#25, c_first_name#26, c_last_name#27]
+Arguments: [ws_bill_customer_sk#20], [c_customer_sk#25], Inner, BuildRight
-(37) CometProject
-Input [5]: [ws_bill_customer_sk#18, d_date#22, c_customer_sk#23, c_first_name#24, c_last_name#25]
-Arguments: [c_last_name#25, c_first_name#24, d_date#22], [c_last_name#25, c_first_name#24, d_date#22]
+(38) CometProject
+Input [5]: [ws_bill_customer_sk#20, d_date#24, c_customer_sk#25, c_first_name#26, c_last_name#27]
+Arguments: [c_last_name#27, c_first_name#26, d_date#24], [c_last_name#27, c_first_name#26, d_date#24]
-(38) CometHashAggregate
-Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
-Keys [3]: [c_last_name#25, c_first_name#24, d_date#22]
+(39) CometHashAggregate
+Input [3]: [c_last_name#27, c_first_name#26, d_date#24]
+Keys [3]: [c_last_name#27, c_first_name#26, d_date#24]
Functions: []
-(39) CometExchange
-Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
-Arguments: hashpartitioning(c_last_name#25, c_first_name#24, d_date#22, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
+(40) CometExchange
+Input [3]: [c_last_name#27, c_first_name#26, d_date#24]
+Arguments: hashpartitioning(c_last_name#27, c_first_name#26, d_date#24, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=3]
-(40) CometHashAggregate
-Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
-Keys [3]: [c_last_name#25, c_first_name#24, d_date#22]
+(41) CometHashAggregate
+Input [3]: [c_last_name#27, c_first_name#26, d_date#24]
+Keys [3]: [c_last_name#27, c_first_name#26, d_date#24]
Functions: []
-(41) CometBroadcastExchange
-Input [3]: [c_last_name#25, c_first_name#24, d_date#22]
-Arguments: [c_last_name#25, c_first_name#24, d_date#22]
+(42) CometBroadcastExchange
+Input [3]: [c_last_name#27, c_first_name#26, d_date#24]
+Arguments: [c_last_name#27, c_first_name#26, d_date#24]
-(42) CometBroadcastHashJoin
-Left output [3]: [c_last_name#9, c_first_name#8, d_date#5]
-Right output [3]: [c_last_name#25, c_first_name#24, d_date#22]
-Arguments: [coalesce(c_last_name#9, ), isnull(c_last_name#9), coalesce(c_first_name#8, ), isnull(c_first_name#8), coalesce(d_date#5, 1970-01-01), isnull(d_date#5)], [coalesce(c_last_name#25, ), isnull(c_last_name#25), coalesce(c_first_name#24, ), isnull(c_first_name#24), coalesce(d_date#22, 1970-01-01), isnull(d_date#22)], LeftSemi, BuildRight
+(43) CometBroadcastHashJoin
+Left output [3]: [c_last_name#11, c_first_name#10, d_date#5]
+Right output [3]: [c_last_name#27, c_first_name#26, d_date#24]
+Arguments: [coalesce(c_last_name#11, ), isnull(c_last_name#11), coalesce(c_first_name#10, ), isnull(c_first_name#10), coalesce(d_date#5, 1970-01-01), isnull(d_date#5)], [coalesce(c_last_name#27, ), isnull(c_last_name#27), coalesce(c_first_name#26, ), isnull(c_first_name#26), coalesce(d_date#24, 1970-01-01), isnull(d_date#24)], LeftSemi, BuildRight
-(43) CometProject
-Input [3]: [c_last_name#9, c_first_name#8, d_date#5]
+(44) CometProject
+Input [3]: [c_last_name#11, c_first_name#10, d_date#5]
-(44) CometHashAggregate
+(45) CometHashAggregate
Input: []
Keys: []
Functions [1]: [partial_count(1)]
-(45) CometExchange
-Input [1]: [count#26]
+(46) CometExchange
+Input [1]: [count#28]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=4]
-(46) CometHashAggregate
-Input [1]: [count#26]
+(47) CometHashAggregate
+Input [1]: [count#28]
Keys: []
Functions [1]: [count(1)]
-(47) CometColumnarToRow [codegen id : 1]
-Input [1]: [count(1)#27]
+(48) CometColumnarToRow [codegen id : 1]
+Input [1]: [count(1)#29]
===== Subqueries =====
Subquery:1 Hosting operator id = 1 Hosting Expression = ss_sold_date_sk#2 IN dynamicpruning#3
-BroadcastExchange (52)
-+- * CometColumnarToRow (51)
- +- CometProject (50)
- +- CometFilter (49)
- +- CometScan parquet spark_catalog.default.date_dim (48)
+BroadcastExchange (53)
++- * CometColumnarToRow (52)
+ +- CometProject (51)
+ +- CometFilter (50)
+ +- CometScan parquet spark_catalog.default.date_dim (49)
-(48) CometScan parquet spark_catalog.default.date_dim
+(49) CometScan parquet spark_catalog.default.date_dim
Output [3]: [d_date_sk#4, d_date#5, d_month_seq#6]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_month_seq), GreaterThanOrEqual(d_month_seq,1200), LessThanOrEqual(d_month_seq,1211), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_date:date,d_month_seq:int>
-(49) CometFilter
+(50) CometFilter
Input [3]: [d_date_sk#4, d_date#5, d_month_seq#6]
Condition : (((isnotnull(d_month_seq#6) AND (d_month_seq#6 >= 1200)) AND (d_month_seq#6 <= 1211)) AND isnotnull(d_date_sk#4))
-(50) CometProject
+(51) CometProject
Input [3]: [d_date_sk#4, d_date#5, d_month_seq#6]
Arguments: [d_date_sk#4, d_date#5], [d_date_sk#4, d_date#5]
-(51) CometColumnarToRow [codegen id : 1]
+(52) CometColumnarToRow [codegen id : 1]
Input [2]: [d_date_sk#4, d_date#5]
-(52) BroadcastExchange
+(53) BroadcastExchange
Input [2]: [d_date_sk#4, d_date#5]
Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [plan_id=5]
-Subquery:2 Hosting operator id = 17 Hosting Expression = cs_sold_date_sk#11 IN dynamicpruning#3
+Subquery:2 Hosting operator id = 18 Hosting Expression = cs_sold_date_sk#13 IN dynamicpruning#3
-Subquery:3 Hosting operator id = 30 Hosting Expression = ws_sold_date_sk#19 IN dynamicpruning#3
+Subquery:3 Hosting operator id = 31 Hosting Expression = ws_sold_date_sk#21 IN dynamicpruning#3
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt
index 5ce896e6e..fe3c247c3 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q38/simplified.txt
@@ -29,8 +29,9 @@ WholeStageCodegen (1)
CometFilter [d_date_sk,d_date,d_month_seq]
CometScan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_month_seq]
CometBroadcastExchange [c_customer_sk,c_first_name,c_last_name] #5
- CometFilter [c_customer_sk,c_first_name,c_last_name]
- CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
+ CometProject [c_first_name,c_last_name] [c_customer_sk,c_first_name,c_last_name]
+ CometFilter [c_customer_sk,c_first_name,c_last_name]
+ CometScan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
CometBroadcastExchange [c_last_name,c_first_name,d_date] #6
CometHashAggregate [c_last_name,c_first_name,d_date]
CometExchange [c_last_name,c_first_name,d_date] #7
diff --git a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt
index dcf3eae4b..b0b57f84d 100644
--- a/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt
+++ b/spark/src/test/resources/tpcds-plan-stability/approved-plans-v1_4-spark3_5/q4/explain.txt
@@ -1,641 +1,671 @@
== Physical Plan ==
-* CometColumnarToRow (101)
-+- CometTakeOrderedAndProject (100)
- +- CometProject (99)
- +- CometBroadcastHashJoin (98)
- :- CometProject (83)
- : +- CometBroadcastHashJoin (82)
- : :- CometProject (66)
- : : +- CometBroadcastHashJoin (65)
- : : :- CometProject (50)
- : : : +- CometBroadcastHashJoin (49)
- : : : :- CometBroadcastHashJoin (33)
- : : : : :- CometFilter (16)
- : : : : : +- CometHashAggregate (15)
- : : : : : +- CometExchange (14)
- : : : : : +- CometHashAggregate (13)
- : : : : : +- CometProject (12)
- : : : : : +- CometBroadcastHashJoin (11)
- : : : : : :- CometProject (7)
- : : : : : : +- CometBroadcastHashJoin (6)
- : : : : : : :- CometFilter (2)
- : : : : : : : +- CometScan parquet spark_catalog.default.customer (1)
- : : : : : : +- CometBroadcastExchange (5)
- : : : : : : +- CometFilter (4)
- : : : : : : +- CometScan parquet spark_catalog.default.store_sales (3)
- : : : : : +- CometBroadcastExchange (10)
- : : : : : +- CometFilter (9)
- : : : : : +- CometScan parquet spark_catalog.default.date_dim (8)
- : : : : +- CometBroadcastExchange (32)
- : : : : +- CometHashAggregate (31)
- : : : : +- CometExchange (30)
- : : : : +- CometHashAggregate (29)
- : : : : +- CometProject (28)
- : : : : +- CometBroadcastHashJoin (27)
- : : : : :- CometProject (23)
- : : : : : +- CometBroadcastHashJoin (22)
- : : : : : :- CometFilter (18)
- : : : : : : +- CometScan parquet spark_catalog.default.customer (17)
- : : : : : +- CometBroadcastExchange (21)
- : : : : : +- CometFilter (20)
- : : : : : +- CometScan parquet spark_catalog.default.store_sales (19)
- : : : : +- CometBroadcastExchange (26)
- : : : : +- CometFilter (25)
- : : : : +- CometScan parquet spark_catalog.default.date_dim (24)
- : : : +- CometBroadcastExchange (48)
- : : : +- CometFilter (47)
- : : : +- CometHashAggregate (46)
- : : : +- CometExchange (45)
- : : : +- CometHashAggregate (44)
- : : : +- CometProject (43)
- : : : +- CometBroadcastHashJoin (42)
- : : : :- CometProject (40)
- : : : : +- CometBroadcastHashJoin (39)
- : : : : :- CometFilter (35)
- : : : : : +- CometScan parquet spark_catalog.default.customer (34)
- : : : : +- CometBroadcastExchange (38)
- : : : : +- CometFilter (37)
- : : : : +- CometScan parquet spark_catalog.default.catalog_sales (36)
- : : : +- ReusedExchange (41)
- : : +- CometBroadcastExchange (64)
- : : +- CometHashAggregate (63)
- : : +- CometExchange (62)
- : : +- CometHashAggregate (61)
- : : +- CometProject (60)
- : : +- CometBroadcastHashJoin (59)
- : : :- CometProject (57)
- : : : +- CometBroadcastHashJoin (56)
- : : : :- CometFilter (52)
- : : : : +- CometScan parquet spark_catalog.default.customer (51)
- : : : +- CometBroadcastExchange (55)
- : : : +- CometFilter (54)
- : : : +- CometScan parquet spark_catalog.default.catalog_sales (53)
- : : +- ReusedExchange (58)
- : +- CometBroadcastExchange (81)
- : +- CometFilter (80)
- : +- CometHashAggregate (79)
- : +- CometExchange (78)
- : +- CometHashAggregate (77)
- : +- CometProject (76)
- : +- CometBroadcastHashJoin (75)
- : :- CometProject (73)
- : : +- CometBroadcastHashJoin (72)
- : : :- CometFilter (68)
- : : : +- CometScan parquet spark_catalog.default.customer (67)
- : : +- CometBroadcastExchange (71)
- : : +- CometFilter (70)
- : : +- CometScan parquet spark_catalog.default.web_sales (69)
- : +- ReusedExchange (74)
- +- CometBroadcastExchange (97)
- +- CometHashAggregate (96)
- +- CometExchange (95)
- +- CometHashAggregate (94)
- +- CometProject (93)
- +- CometBroadcastHashJoin (92)
- :- CometProject (90)
- : +- CometBroadcastHashJoin (89)
- : :- CometFilter (85)
- : : +- CometScan parquet spark_catalog.default.customer (84)
- : +- CometBroadcastExchange (88)
- : +- CometFilter (87)
- : +- CometScan parquet spark_catalog.default.web_sales (86)
- +- ReusedExchange (91)
+* CometColumnarToRow (107)
++- CometTakeOrderedAndProject (106)
+ +- CometProject (105)
+ +- CometBroadcastHashJoin (104)
+ :- CometProject (88)
+ : +- CometBroadcastHashJoin (87)
+ : :- CometProject (70)
+ : : +- CometBroadcastHashJoin (69)
+ : : :- CometProject (53)
+ : : : +- CometBroadcastHashJoin (52)
+ : : : :- CometBroadcastHashJoin (35)
+ : : : : :- CometFilter (17)
+ : : : : : +- CometHashAggregate (16)
+ : : : : : +- CometExchange (15)
+ : : : : : +- CometHashAggregate (14)
+ : : : : : +- CometProject (13)
+ : : : : : +- CometBroadcastHashJoin (12)
+ : : : : : :- CometProject (8)
+ : : : : : : +- CometBroadcastHashJoin (7)
+ : : : : : : :- CometProject (3)
+ : : : : : : : +- CometFilter (2)
+ : : : : : : : +- CometScan parquet spark_catalog.default.customer (1)
+ : : : : : : +- CometBroadcastExchange (6)
+ : : : : : : +- CometFilter (5)
+ : : : : : : +- CometScan parquet spark_catalog.default.store_sales (4)
+ : : : : : +- CometBroadcastExchange (11)
+ : : : : : +- CometFilter (10)
+ : : : : : +- CometScan parquet spark_catalog.default.date_dim (9)
+ : : : : +- CometBroadcastExchange (34)
+ : : : : +- CometHashAggregate (33)
+ : : : : +- CometExchange (32)
+ : : : : +- CometHashAggregate (31)
+ : : : : +- CometProject (30)
+ : : : : +- CometBroadcastHashJoin (29)
+ : : : : :- CometProject (25)
+ : : : : : +- CometBroadcastHashJoin (24)
+ : : : : : :- CometProject (20)
+ : : : : : : +- CometFilter (19)
+ : : : : : : +- CometScan parquet spark_catalog.default.customer (18)
+ : : : : : +- CometBroadcastExchange (23)
+ : : : : : +- CometFilter (22)
+ : : : : : +- CometScan parquet spark_catalog.default.store_sales (21)
+ : : : : +- CometBroadcastExchange (28)
+ : : : : +- CometFilter (27)
+ : : : : +- CometScan parquet spark_catalog.default.date_dim (26)
+ : : : +- CometBroadcastExchange (51)
+ : : : +- CometFilter (50)
+ : : : +- CometHashAggregate (49)
+ : : : +- CometExchange (48)
+ : : : +- CometHashAggregate (47)
+ : : : +- CometProject (46)
+ : : : +- CometBroadcastHashJoin (45)
+ : : : :- CometProject (43)
+ : : : : +- CometBroadcastHashJoin (42)
+ : : : : :- CometProject (38)
+ : : : : : +- CometFilter (37)
+ : : : : : +- CometScan parquet spark_catalog.default.customer (36)
+ : : : : +- CometBroadcastExchange (41)
+ : : : : +- CometFilter (40)
+ : : : : +- CometScan parquet spark_catalog.default.catalog_sales (39)
+ : : : +- ReusedExchange (44)
+ : : +- CometBroadcastExchange (68)
+ : : +- CometHashAggregate (67)
+ : : +- CometExchange (66)
+ : : +- CometHashAggregate (65)
+ : : +- CometProject (64)
+ : : +- CometBroadcastHashJoin (63)
+ : : :- CometProject (61)
+ : : : +- CometBroadcastHashJoin (60)
+ : : : :- CometProject (56)
+ : : : : +- CometFilter (55)
+ : : : : +- CometScan parquet spark_catalog.default.customer (54)
+ : : : +- CometBroadcastExchange (59)
+ : : : +- CometFilter (58)
+ : : : +- CometScan parquet spark_catalog.default.catalog_sales (57)
+ : : +- ReusedExchange (62)
+ : +- CometBroadcastExchange (86)
+ : +- CometFilter (85)
+ : +- CometHashAggregate (84)
+ : +- CometExchange (83)
+ : +- CometHashAggregate (82)
+ : +- CometProject (81)
+ : +- CometBroadcastHashJoin (80)
+ : :- CometProject (78)
+ : : +- CometBroadcastHashJoin (77)
+ : : :- CometProject (73)
+ : : : +- CometFilter (72)
+ : : : +- CometScan parquet spark_catalog.default.customer (71)
+ : : +- CometBroadcastExchange (76)
+ : : +- CometFilter (75)
+ : : +- CometScan parquet spark_catalog.default.web_sales (74)
+ : +- ReusedExchange (79)
+ +- CometBroadcastExchange (103)
+ +- CometHashAggregate (102)
+ +- CometExchange (101)
+ +- CometHashAggregate (100)
+ +- CometProject (99)
+ +- CometBroadcastHashJoin (98)
+ :- CometProject (96)
+ : +- CometBroadcastHashJoin (95)
+ : :- CometProject (91)
+ : : +- CometFilter (90)
+ : : +- CometScan parquet spark_catalog.default.customer (89)
+ : +- CometBroadcastExchange (94)
+ : +- CometFilter (93)
+ : +- CometScan parquet spark_catalog.default.web_sales (92)
+ +- ReusedExchange (97)
(1) CometScan parquet spark_catalog.default.customer
Output [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
+PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
(2) CometFilter
Input [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
-Condition : (isnotnull(c_customer_sk#1) AND isnotnull(c_customer_id#2))
+Condition : (isnotnull(c_customer_sk#1) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#2, 16, true, false, true)))
-(3) CometScan parquet spark_catalog.default.store_sales
-Output [6]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
+(3) CometProject
+Input [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
+Arguments: [c_customer_sk#1, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14], [c_customer_sk#1, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#2, 16, true, false, true) AS c_customer_id#9, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#3, 20, true, false, true) AS [...]
+
+(4) CometScan parquet spark_catalog.default.store_sales
+Output [6]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#14), dynamicpruningexpression(ss_sold_date_sk#14 IN dynamicpruning#15)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#20), dynamicpruningexpression(ss_sold_date_sk#20 IN dynamicpruning#21)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_ext_discount_amt:decimal(7,2),ss_ext_sales_price:decimal(7,2),ss_ext_wholesale_cost:decimal(7,2),ss_ext_list_price:decimal(7,2)>
-(4) CometFilter
-Input [6]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
-Condition : isnotnull(ss_customer_sk#9)
+(5) CometFilter
+Input [6]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
+Condition : isnotnull(ss_customer_sk#15)
-(5) CometBroadcastExchange
-Input [6]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
-Arguments: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
+(6) CometBroadcastExchange
+Input [6]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
+Arguments: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
-(6) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8]
-Right output [6]: [ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
-Arguments: [c_customer_sk#1], [ss_customer_sk#9], Inner, BuildRight
+(7) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#1, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14]
+Right output [6]: [ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
+Arguments: [c_customer_sk#1], [ss_customer_sk#15], Inner, BuildRight
-(7) CometProject
-Input [14]: [c_customer_sk#1, c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_customer_sk#9, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
-Arguments: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14], [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
+(8) CometProject
+Input [14]: [c_customer_sk#1, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_customer_sk#15, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
-(8) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#16, d_year#17]
+(9) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#22, d_year#23]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2001), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(9) CometFilter
-Input [2]: [d_date_sk#16, d_year#17]
-Condition : ((isnotnull(d_year#17) AND (d_year#17 = 2001)) AND isnotnull(d_date_sk#16))
+(10) CometFilter
+Input [2]: [d_date_sk#22, d_year#23]
+Condition : ((isnotnull(d_year#23) AND (d_year#23 = 2001)) AND isnotnull(d_date_sk#22))
-(10) CometBroadcastExchange
-Input [2]: [d_date_sk#16, d_year#17]
-Arguments: [d_date_sk#16, d_year#17]
+(11) CometBroadcastExchange
+Input [2]: [d_date_sk#22, d_year#23]
+Arguments: [d_date_sk#22, d_year#23]
-(11) CometBroadcastHashJoin
-Left output [12]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14]
-Right output [2]: [d_date_sk#16, d_year#17]
-Arguments: [ss_sold_date_sk#14], [d_date_sk#16], Inner, BuildRight
+(12) CometBroadcastHashJoin
+Left output [12]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20]
+Right output [2]: [d_date_sk#22, d_year#23]
+Arguments: [ss_sold_date_sk#20], [d_date_sk#22], Inner, BuildRight
-(12) CometProject
-Input [14]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, ss_sold_date_sk#14, d_date_sk#16, d_year#17]
-Arguments: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, d_year#17], [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, d_year#17]
+(13) CometProject
+Input [14]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, ss_sold_date_sk#20, d_date_sk#22, d_year#23]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, d_year#23], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, d_year#23]
-(13) CometHashAggregate
-Input [12]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, ss_ext_discount_amt#10, ss_ext_sales_price#11, ss_ext_wholesale_cost#12, ss_ext_list_price#13, d_year#17]
-Keys [8]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17]
-Functions [1]: [partial_sum(((((ss_ext_list_price#13 - ss_ext_wholesale_cost#12) - ss_ext_discount_amt#10) + ss_ext_sales_price#11) / 2))]
+(14) CometHashAggregate
+Input [12]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, ss_ext_discount_amt#16, ss_ext_sales_price#17, ss_ext_wholesale_cost#18, ss_ext_list_price#19, d_year#23]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, d_year#23]
+Functions [1]: [partial_sum(((((ss_ext_list_price#19 - ss_ext_wholesale_cost#18) - ss_ext_discount_amt#16) + ss_ext_sales_price#17) / 2))]
-(14) CometExchange
-Input [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, sum#18, isEmpty#19]
-Arguments: hashpartitioning(c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
+(15) CometExchange
+Input [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, d_year#23, sum#24, isEmpty#25]
+Arguments: hashpartitioning(c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, d_year#23, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=1]
-(15) CometHashAggregate
-Input [10]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17, sum#18, isEmpty#19]
-Keys [8]: [c_customer_id#2, c_first_name#3, c_last_name#4, c_preferred_cust_flag#5, c_birth_country#6, c_login#7, c_email_address#8, d_year#17]
-Functions [1]: [sum(((((ss_ext_list_price#13 - ss_ext_wholesale_cost#12) - ss_ext_discount_amt#10) + ss_ext_sales_price#11) / 2))]
+(16) CometHashAggregate
+Input [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, d_year#23, sum#24, isEmpty#25]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#6, c_login#13, c_email_address#14, d_year#23]
+Functions [1]: [sum(((((ss_ext_list_price#19 - ss_ext_wholesale_cost#18) - ss_ext_discount_amt#16) + ss_ext_sales_price#17) / 2))]
-(16) CometFilter
-Input [2]: [customer_id#20, year_total#21]
-Condition : (isnotnull(year_total#21) AND (year_total#21 > 0.000000))
+(17) CometFilter
+Input [2]: [customer_id#26, year_total#27]
+Condition : (isnotnull(year_total#27) AND (year_total#27 > 0.000000))
-(17) CometScan parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#22, c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29]
+(18) CometScan parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#28, c_customer_id#29, c_first_name#30, c_last_name#31, c_preferred_cust_flag#32, c_birth_country#33, c_login#34, c_email_address#35]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
+PushedFilters: [IsNotNull(c_customer_sk)]
ReadSchema: struct<c_customer_sk:int,c_customer_id:string,c_first_name:string,c_last_name:string,c_preferred_cust_flag:string,c_birth_country:string,c_login:string,c_email_address:string>
-(18) CometFilter
-Input [8]: [c_customer_sk#22, c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29]
-Condition : (isnotnull(c_customer_sk#22) AND isnotnull(c_customer_id#23))
+(19) CometFilter
+Input [8]: [c_customer_sk#28, c_customer_id#29, c_first_name#30, c_last_name#31, c_preferred_cust_flag#32, c_birth_country#33, c_login#34, c_email_address#35]
+Condition : (isnotnull(c_customer_sk#28) AND isnotnull(staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#29, 16, true, false, true)))
+
+(20) CometProject
+Input [8]: [c_customer_sk#28, c_customer_id#29, c_first_name#30, c_last_name#31, c_preferred_cust_flag#32, c_birth_country#33, c_login#34, c_email_address#35]
+Arguments: [c_customer_sk#28, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14], [c_customer_sk#28, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_customer_id#29, 16, true, false, true) AS c_customer_id#9, staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, c_first_name#30, 20, true, false, tru [...]
-(19) CometScan parquet spark_catalog.default.store_sales
-Output [6]: [ss_customer_sk#30, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
+(21) CometScan parquet spark_catalog.default.store_sales
+Output [6]: [ss_customer_sk#36, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
Batched: true
Location: InMemoryFileIndex []
-PartitionFilters: [isnotnull(ss_sold_date_sk#35), dynamicpruningexpression(ss_sold_date_sk#35 IN dynamicpruning#36)]
+PartitionFilters: [isnotnull(ss_sold_date_sk#41), dynamicpruningexpression(ss_sold_date_sk#41 IN dynamicpruning#42)]
PushedFilters: [IsNotNull(ss_customer_sk)]
ReadSchema: struct<ss_customer_sk:int,ss_ext_discount_amt:decimal(7,2),ss_ext_sales_price:decimal(7,2),ss_ext_wholesale_cost:decimal(7,2),ss_ext_list_price:decimal(7,2)>
-(20) CometFilter
-Input [6]: [ss_customer_sk#30, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
-Condition : isnotnull(ss_customer_sk#30)
+(22) CometFilter
+Input [6]: [ss_customer_sk#36, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
+Condition : isnotnull(ss_customer_sk#36)
-(21) CometBroadcastExchange
-Input [6]: [ss_customer_sk#30, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
-Arguments: [ss_customer_sk#30, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
+(23) CometBroadcastExchange
+Input [6]: [ss_customer_sk#36, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
+Arguments: [ss_customer_sk#36, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
-(22) CometBroadcastHashJoin
-Left output [8]: [c_customer_sk#22, c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29]
-Right output [6]: [ss_customer_sk#30, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
-Arguments: [c_customer_sk#22], [ss_customer_sk#30], Inner, BuildRight
+(24) CometBroadcastHashJoin
+Left output [8]: [c_customer_sk#28, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14]
+Right output [6]: [ss_customer_sk#36, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
+Arguments: [c_customer_sk#28], [ss_customer_sk#36], Inner, BuildRight
-(23) CometProject
-Input [14]: [c_customer_sk#22, c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_customer_sk#30, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
-Arguments: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35], [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_da [...]
+(25) CometProject
+Input [14]: [c_customer_sk#28, c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_customer_sk#36, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
-(24) CometScan parquet spark_catalog.default.date_dim
-Output [2]: [d_date_sk#37, d_year#38]
+(26) CometScan parquet spark_catalog.default.date_dim
+Output [2]: [d_date_sk#43, d_year#44]
Batched: true
Location [not included in comparison]/{warehouse_dir}/date_dim]
PushedFilters: [IsNotNull(d_year), EqualTo(d_year,2002), IsNotNull(d_date_sk)]
ReadSchema: struct<d_date_sk:int,d_year:int>
-(25) CometFilter
-Input [2]: [d_date_sk#37, d_year#38]
-Condition : ((isnotnull(d_year#38) AND (d_year#38 = 2002)) AND isnotnull(d_date_sk#37))
-
-(26) CometBroadcastExchange
-Input [2]: [d_date_sk#37, d_year#38]
-Arguments: [d_date_sk#37, d_year#38]
-
-(27) CometBroadcastHashJoin
-Left output [12]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35]
-Right output [2]: [d_date_sk#37, d_year#38]
-Arguments: [ss_sold_date_sk#35], [d_date_sk#37], Inner, BuildRight
+(27) CometFilter
+Input [2]: [d_date_sk#43, d_year#44]
+Condition : ((isnotnull(d_year#44) AND (d_year#44 = 2002)) AND isnotnull(d_date_sk#43))
-(28) CometProject
-Input [14]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, ss_sold_date_sk#35, d_date_sk#37, d_year#38]
-Arguments: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, d_year#38], [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, d_year#38]
+(28) CometBroadcastExchange
+Input [2]: [d_date_sk#43, d_year#44]
+Arguments: [d_date_sk#43, d_year#44]
-(29) CometHashAggregate
-Input [12]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, ss_ext_discount_amt#31, ss_ext_sales_price#32, ss_ext_wholesale_cost#33, ss_ext_list_price#34, d_year#38]
-Keys [8]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38]
-Functions [1]: [partial_sum(((((ss_ext_list_price#34 - ss_ext_wholesale_cost#33) - ss_ext_discount_amt#31) + ss_ext_sales_price#32) / 2))]
+(29) CometBroadcastHashJoin
+Left output [12]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41]
+Right output [2]: [d_date_sk#43, d_year#44]
+Arguments: [ss_sold_date_sk#41], [d_date_sk#43], Inner, BuildRight
-(30) CometExchange
-Input [10]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, sum#39, isEmpty#40]
-Arguments: hashpartitioning(c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+(30) CometProject
+Input [14]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, ss_sold_date_sk#41, d_date_sk#43, d_year#44]
+Arguments: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, d_year#44], [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, d_year#44]
(31) CometHashAggregate
-Input [10]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38, sum#39, isEmpty#40]
-Keys [8]: [c_customer_id#23, c_first_name#24, c_last_name#25, c_preferred_cust_flag#26, c_birth_country#27, c_login#28, c_email_address#29, d_year#38]
-Functions [1]: [sum(((((ss_ext_list_price#34 - ss_ext_wholesale_cost#33) - ss_ext_discount_amt#31) + ss_ext_sales_price#32) / 2))]
-
-(32) CometBroadcastExchange
-Input [8]: [customer_id#41, customer_first_name#42, customer_last_name#43, customer_preferred_cust_flag#44, customer_birth_country#45, customer_login#46, customer_email_address#47, year_total#48]
-Arguments: [customer_id#41, customer_first_name#42, customer_last_name#43, customer_preferred_cust_flag#44, customer_birth_country#45, customer_login#46, customer_email_address#47, year_total#48]
-
-(33) CometBroadcastHashJoin
-Left output [2]: [customer_id#20, year_total#21]
-Right output [8]: [customer_id#41, customer_first_name#42, customer_last_name#43, customer_preferred_cust_flag#44, customer_birth_country#45, customer_login#46, customer_email_address#47, year_total#48]
-Arguments: [rpad(customer_id#20, 16, )], [rpad(customer_id#41, 16, )], Inner, BuildRight
-
-(34) CometScan parquet spark_catalog.default.customer
-Output [8]: [c_customer_sk#49, c_customer_id#50, c_first_name#51, c_last_name#52, c_preferred_cust_flag#53, c_birth_country#54, c_login#55, c_email_address#56]
+Input [12]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, ss_ext_discount_amt#37, ss_ext_sales_price#38, ss_ext_wholesale_cost#39, ss_ext_list_price#40, d_year#44]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, d_year#44]
+Functions [1]: [partial_sum(((((ss_ext_list_price#40 - ss_ext_wholesale_cost#39) - ss_ext_discount_amt#37) + ss_ext_sales_price#38) / 2))]
+
+(32) CometExchange
+Input [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, d_year#44, sum#45, isEmpty#46]
+Arguments: hashpartitioning(c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, d_year#44, 5), ENSURE_REQUIREMENTS, CometNativeShuffle, [plan_id=2]
+
+(33) CometHashAggregate
+Input [10]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, d_year#44, sum#45, isEmpty#46]
+Keys [8]: [c_customer_id#9, c_first_name#10, c_last_name#11, c_preferred_cust_flag#12, c_birth_country#33, c_login#13, c_email_address#14, d_year#44]
+Functions [1]: [sum(((((ss_ext_list_price#40 - ss_ext_wholesale_cost#39) - ss_ext_discount_amt#37) + ss_ext_sales_price#38) / 2))]
+
+(34) CometBroadcastExchange
+Input [8]: [customer_id#47, customer_first_name#48, customer_last_name#49, customer_preferred_cust_flag#50, customer_birth_country#51, customer_login#52, customer_email_address#53, year_total#54]
+Arguments: [customer_id#47, customer_first_name#48, customer_last_name#49, customer_preferred_cust_flag#50, customer_birth_country#51, customer_login#52, customer_email_address#53, year_total#54]
+
+(35) CometBroadcastHashJoin
+Left output [2]: [customer_id#26, year_total#27]
+Right output [8]: [customer_id#47, customer_first_name#48, customer_last_name#49, customer_preferred_cust_flag#50, customer_birth_country#51, customer_login#52, customer_email_address#53, year_total#54]
+Arguments: [customer_id#26], [customer_id#47], Inner, BuildRight
+
+(36) CometScan parquet spark_catalog.default.customer
+Output [8]: [c_customer_sk#55, c_customer_id#56, c_first_name#57, c_last_name#58, c_preferred_cust_flag#59, c_birth_country#60, c_login#61, c_email_address#62]
Batched: true
Location [not included in comparison]/{warehouse_dir}/customer]
-PushedFilters: [IsNotNull(c_customer_sk), IsNotNull(c_customer_id)]
... 120799 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org