You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by ho...@apache.org on 2024/06/18 07:36:35 UTC
(incubator-gluten) branch main updated: [VL] Set s.g.s.c.b.v.coalesceBatchesBeforeShuffle=true by default (#6056)
This is an automated email from the ASF dual-hosted git repository.
hongze pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new ffdc64aff [VL] Set s.g.s.c.b.v.coalesceBatchesBeforeShuffle=true by default (#6056)
ffdc64aff is described below
commit ffdc64aff3641ba0cfaf00348e740b5802b0b8fb
Author: Hongze Zhang <ho...@intel.com>
AuthorDate: Tue Jun 18 15:36:30 2024 +0800
[VL] Set s.g.s.c.b.v.coalesceBatchesBeforeShuffle=true by default (#6056)
---
.../extension/FlushableHashAggregateRule.scala | 106 ++-
.../tpch-approved-plan/v1-bhj-ras/spark32/1.txt | 100 +--
.../tpch-approved-plan/v1-bhj-ras/spark32/10.txt | 187 +++---
.../tpch-approved-plan/v1-bhj-ras/spark32/11.txt | 176 ++---
.../tpch-approved-plan/v1-bhj-ras/spark32/12.txt | 134 ++--
.../tpch-approved-plan/v1-bhj-ras/spark32/13.txt | 177 ++---
.../tpch-approved-plan/v1-bhj-ras/spark32/14.txt | 103 +--
.../tpch-approved-plan/v1-bhj-ras/spark32/15.txt | 146 +++--
.../tpch-approved-plan/v1-bhj-ras/spark32/16.txt | 193 +++---
.../tpch-approved-plan/v1-bhj-ras/spark32/17.txt | 125 ++--
.../tpch-approved-plan/v1-bhj-ras/spark32/18.txt | 304 ++++-----
.../tpch-approved-plan/v1-bhj-ras/spark32/19.txt | 99 +--
.../tpch-approved-plan/v1-bhj-ras/spark32/20.txt | 403 ++++++------
.../tpch-approved-plan/v1-bhj-ras/spark32/21.txt | 245 +++----
.../tpch-approved-plan/v1-bhj-ras/spark32/22.txt | 122 ++--
.../tpch-approved-plan/v1-bhj-ras/spark32/3.txt | 153 ++---
.../tpch-approved-plan/v1-bhj-ras/spark32/4.txt | 140 ++--
.../tpch-approved-plan/v1-bhj-ras/spark32/5.txt | 276 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark32/6.txt | 65 +-
.../tpch-approved-plan/v1-bhj-ras/spark32/7.txt | 260 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark32/8.txt | 348 +++++-----
.../tpch-approved-plan/v1-bhj-ras/spark32/9.txt | 270 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark33/1.txt | 100 +--
.../tpch-approved-plan/v1-bhj-ras/spark33/10.txt | 187 +++---
.../tpch-approved-plan/v1-bhj-ras/spark33/11.txt | 341 +++++-----
.../tpch-approved-plan/v1-bhj-ras/spark33/12.txt | 134 ++--
.../tpch-approved-plan/v1-bhj-ras/spark33/13.txt | 177 ++---
.../tpch-approved-plan/v1-bhj-ras/spark33/14.txt | 103 +--
.../tpch-approved-plan/v1-bhj-ras/spark33/15.txt | 239 +++----
.../tpch-approved-plan/v1-bhj-ras/spark33/16.txt | 193 +++---
.../tpch-approved-plan/v1-bhj-ras/spark33/17.txt | 125 ++--
.../tpch-approved-plan/v1-bhj-ras/spark33/18.txt | 304 ++++-----
.../tpch-approved-plan/v1-bhj-ras/spark33/19.txt | 99 +--
.../tpch-approved-plan/v1-bhj-ras/spark33/20.txt | 395 ++++++------
.../tpch-approved-plan/v1-bhj-ras/spark33/21.txt | 241 +++----
.../tpch-approved-plan/v1-bhj-ras/spark33/22.txt | 197 +++---
.../tpch-approved-plan/v1-bhj-ras/spark33/3.txt | 153 ++---
.../tpch-approved-plan/v1-bhj-ras/spark33/4.txt | 140 ++--
.../tpch-approved-plan/v1-bhj-ras/spark33/5.txt | 276 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark33/6.txt | 65 +-
.../tpch-approved-plan/v1-bhj-ras/spark33/7.txt | 260 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark33/8.txt | 348 +++++-----
.../tpch-approved-plan/v1-bhj-ras/spark33/9.txt | 270 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark34/1.txt | 100 +--
.../tpch-approved-plan/v1-bhj-ras/spark34/10.txt | 187 +++---
.../tpch-approved-plan/v1-bhj-ras/spark34/11.txt | 341 +++++-----
.../tpch-approved-plan/v1-bhj-ras/spark34/12.txt | 134 ++--
.../tpch-approved-plan/v1-bhj-ras/spark34/13.txt | 177 ++---
.../tpch-approved-plan/v1-bhj-ras/spark34/14.txt | 103 +--
.../tpch-approved-plan/v1-bhj-ras/spark34/15.txt | 239 +++----
.../tpch-approved-plan/v1-bhj-ras/spark34/16.txt | 193 +++---
.../tpch-approved-plan/v1-bhj-ras/spark34/17.txt | 125 ++--
.../tpch-approved-plan/v1-bhj-ras/spark34/18.txt | 304 ++++-----
.../tpch-approved-plan/v1-bhj-ras/spark34/19.txt | 99 +--
.../tpch-approved-plan/v1-bhj-ras/spark34/20.txt | 312 ++++-----
.../tpch-approved-plan/v1-bhj-ras/spark34/21.txt | 241 +++----
.../tpch-approved-plan/v1-bhj-ras/spark34/22.txt | 197 +++---
.../tpch-approved-plan/v1-bhj-ras/spark34/3.txt | 153 ++---
.../tpch-approved-plan/v1-bhj-ras/spark34/4.txt | 140 ++--
.../tpch-approved-plan/v1-bhj-ras/spark34/5.txt | 276 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark34/6.txt | 65 +-
.../tpch-approved-plan/v1-bhj-ras/spark34/7.txt | 260 ++++----
.../tpch-approved-plan/v1-bhj-ras/spark34/8.txt | 348 +++++-----
.../tpch-approved-plan/v1-bhj-ras/spark34/9.txt | 270 ++++----
.../tpch-approved-plan/v1-bhj/spark32/1.txt | 100 +--
.../tpch-approved-plan/v1-bhj/spark32/10.txt | 187 +++---
.../tpch-approved-plan/v1-bhj/spark32/11.txt | 176 ++---
.../tpch-approved-plan/v1-bhj/spark32/12.txt | 134 ++--
.../tpch-approved-plan/v1-bhj/spark32/13.txt | 177 ++---
.../tpch-approved-plan/v1-bhj/spark32/14.txt | 103 +--
.../tpch-approved-plan/v1-bhj/spark32/15.txt | 146 +++--
.../tpch-approved-plan/v1-bhj/spark32/16.txt | 193 +++---
.../tpch-approved-plan/v1-bhj/spark32/17.txt | 125 ++--
.../tpch-approved-plan/v1-bhj/spark32/18.txt | 304 ++++-----
.../tpch-approved-plan/v1-bhj/spark32/19.txt | 99 +--
.../tpch-approved-plan/v1-bhj/spark32/20.txt | 403 ++++++------
.../tpch-approved-plan/v1-bhj/spark32/21.txt | 245 +++----
.../tpch-approved-plan/v1-bhj/spark32/22.txt | 122 ++--
.../tpch-approved-plan/v1-bhj/spark32/3.txt | 153 ++---
.../tpch-approved-plan/v1-bhj/spark32/4.txt | 140 ++--
.../tpch-approved-plan/v1-bhj/spark32/5.txt | 276 ++++----
.../tpch-approved-plan/v1-bhj/spark32/6.txt | 65 +-
.../tpch-approved-plan/v1-bhj/spark32/7.txt | 260 ++++----
.../tpch-approved-plan/v1-bhj/spark32/8.txt | 348 +++++-----
.../tpch-approved-plan/v1-bhj/spark32/9.txt | 270 ++++----
.../tpch-approved-plan/v1-bhj/spark33/1.txt | 100 +--
.../tpch-approved-plan/v1-bhj/spark33/10.txt | 187 +++---
.../tpch-approved-plan/v1-bhj/spark33/11.txt | 341 +++++-----
.../tpch-approved-plan/v1-bhj/spark33/12.txt | 134 ++--
.../tpch-approved-plan/v1-bhj/spark33/13.txt | 177 ++---
.../tpch-approved-plan/v1-bhj/spark33/14.txt | 103 +--
.../tpch-approved-plan/v1-bhj/spark33/15.txt | 239 +++----
.../tpch-approved-plan/v1-bhj/spark33/16.txt | 193 +++---
.../tpch-approved-plan/v1-bhj/spark33/17.txt | 125 ++--
.../tpch-approved-plan/v1-bhj/spark33/18.txt | 304 ++++-----
.../tpch-approved-plan/v1-bhj/spark33/19.txt | 99 +--
.../tpch-approved-plan/v1-bhj/spark33/20.txt | 395 ++++++------
.../tpch-approved-plan/v1-bhj/spark33/21.txt | 241 +++----
.../tpch-approved-plan/v1-bhj/spark33/22.txt | 230 ++++---
.../tpch-approved-plan/v1-bhj/spark33/3.txt | 153 ++---
.../tpch-approved-plan/v1-bhj/spark33/4.txt | 140 ++--
.../tpch-approved-plan/v1-bhj/spark33/5.txt | 276 ++++----
.../tpch-approved-plan/v1-bhj/spark33/6.txt | 65 +-
.../tpch-approved-plan/v1-bhj/spark33/7.txt | 260 ++++----
.../tpch-approved-plan/v1-bhj/spark33/8.txt | 348 +++++-----
.../tpch-approved-plan/v1-bhj/spark33/9.txt | 270 ++++----
.../tpch-approved-plan/v1-bhj/spark34/1.txt | 100 +--
.../tpch-approved-plan/v1-bhj/spark34/10.txt | 187 +++---
.../tpch-approved-plan/v1-bhj/spark34/11.txt | 341 +++++-----
.../tpch-approved-plan/v1-bhj/spark34/12.txt | 134 ++--
.../tpch-approved-plan/v1-bhj/spark34/13.txt | 177 ++---
.../tpch-approved-plan/v1-bhj/spark34/14.txt | 103 +--
.../tpch-approved-plan/v1-bhj/spark34/15.txt | 239 +++----
.../tpch-approved-plan/v1-bhj/spark34/16.txt | 193 +++---
.../tpch-approved-plan/v1-bhj/spark34/17.txt | 125 ++--
.../tpch-approved-plan/v1-bhj/spark34/18.txt | 304 ++++-----
.../tpch-approved-plan/v1-bhj/spark34/19.txt | 99 +--
.../tpch-approved-plan/v1-bhj/spark34/20.txt | 312 ++++-----
.../tpch-approved-plan/v1-bhj/spark34/21.txt | 241 +++----
.../tpch-approved-plan/v1-bhj/spark34/22.txt | 230 ++++---
.../tpch-approved-plan/v1-bhj/spark34/3.txt | 153 ++---
.../tpch-approved-plan/v1-bhj/spark34/4.txt | 140 ++--
.../tpch-approved-plan/v1-bhj/spark34/5.txt | 276 ++++----
.../tpch-approved-plan/v1-bhj/spark34/6.txt | 65 +-
.../tpch-approved-plan/v1-bhj/spark34/7.txt | 260 ++++----
.../tpch-approved-plan/v1-bhj/spark34/8.txt | 348 +++++-----
.../tpch-approved-plan/v1-bhj/spark34/9.txt | 270 ++++----
.../tpch-approved-plan/v1-ras/spark32/1.txt | 100 +--
.../tpch-approved-plan/v1-ras/spark32/10.txt | 345 +++++-----
.../tpch-approved-plan/v1-ras/spark32/11.txt | 284 ++++----
.../tpch-approved-plan/v1-ras/spark32/12.txt | 190 +++---
.../tpch-approved-plan/v1-ras/spark32/13.txt | 204 +++---
.../tpch-approved-plan/v1-ras/spark32/14.txt | 132 ++--
.../tpch-approved-plan/v1-ras/spark32/15.txt | 173 ++---
.../tpch-approved-plan/v1-ras/spark32/16.txt | 251 ++++----
.../tpch-approved-plan/v1-ras/spark32/17.txt | 221 ++++---
.../tpch-approved-plan/v1-ras/spark32/18.txt | 382 ++++++-----
.../tpch-approved-plan/v1-ras/spark32/19.txt | 128 ++--
.../tpch-approved-plan/v1-ras/spark32/20.txt | 502 ++++++++-------
.../tpch-approved-plan/v1-ras/spark32/21.txt | 476 ++++++++------
.../tpch-approved-plan/v1-ras/spark32/22.txt | 178 ++---
.../tpch-approved-plan/v1-ras/spark32/3.txt | 230 ++++---
.../tpch-approved-plan/v1-ras/spark32/4.txt | 194 +++---
.../tpch-approved-plan/v1-ras/spark32/5.txt | 538 +++++++++-------
.../tpch-approved-plan/v1-ras/spark32/6.txt | 65 +-
.../tpch-approved-plan/v1-ras/spark32/7.txt | 511 ++++++++-------
.../tpch-approved-plan/v1-ras/spark32/8.txt | 714 ++++++++++++---------
.../tpch-approved-plan/v1-ras/spark32/9.txt | 534 ++++++++-------
.../tpch-approved-plan/v1-ras/spark33/1.txt | 100 +--
.../tpch-approved-plan/v1-ras/spark33/10.txt | 345 +++++-----
.../tpch-approved-plan/v1-ras/spark33/11.txt | 478 +++++++-------
.../tpch-approved-plan/v1-ras/spark33/12.txt | 190 +++---
.../tpch-approved-plan/v1-ras/spark33/13.txt | 204 +++---
.../tpch-approved-plan/v1-ras/spark33/14.txt | 132 ++--
.../tpch-approved-plan/v1-ras/spark33/15.txt | 266 ++++----
.../tpch-approved-plan/v1-ras/spark33/16.txt | 251 ++++----
.../tpch-approved-plan/v1-ras/spark33/17.txt | 221 ++++---
.../tpch-approved-plan/v1-ras/spark33/18.txt | 382 ++++++-----
.../tpch-approved-plan/v1-ras/spark33/19.txt | 128 ++--
.../tpch-approved-plan/v1-ras/spark33/20.txt | 494 +++++++-------
.../tpch-approved-plan/v1-ras/spark33/21.txt | 472 ++++++++------
.../tpch-approved-plan/v1-ras/spark33/22.txt | 253 ++++----
.../tpch-approved-plan/v1-ras/spark33/3.txt | 230 ++++---
.../tpch-approved-plan/v1-ras/spark33/4.txt | 194 +++---
.../tpch-approved-plan/v1-ras/spark33/5.txt | 538 +++++++++-------
.../tpch-approved-plan/v1-ras/spark33/6.txt | 65 +-
.../tpch-approved-plan/v1-ras/spark33/7.txt | 511 ++++++++-------
.../tpch-approved-plan/v1-ras/spark33/8.txt | 714 ++++++++++++---------
.../tpch-approved-plan/v1-ras/spark33/9.txt | 534 ++++++++-------
.../tpch-approved-plan/v1-ras/spark34/1.txt | 100 +--
.../tpch-approved-plan/v1-ras/spark34/10.txt | 345 +++++-----
.../tpch-approved-plan/v1-ras/spark34/11.txt | 478 +++++++-------
.../tpch-approved-plan/v1-ras/spark34/12.txt | 190 +++---
.../tpch-approved-plan/v1-ras/spark34/13.txt | 204 +++---
.../tpch-approved-plan/v1-ras/spark34/14.txt | 132 ++--
.../tpch-approved-plan/v1-ras/spark34/15.txt | 266 ++++----
.../tpch-approved-plan/v1-ras/spark34/16.txt | 251 ++++----
.../tpch-approved-plan/v1-ras/spark34/17.txt | 221 ++++---
.../tpch-approved-plan/v1-ras/spark34/18.txt | 382 ++++++-----
.../tpch-approved-plan/v1-ras/spark34/19.txt | 128 ++--
.../tpch-approved-plan/v1-ras/spark34/20.txt | 494 +++++++-------
.../tpch-approved-plan/v1-ras/spark34/21.txt | 472 ++++++++------
.../tpch-approved-plan/v1-ras/spark34/22.txt | 253 ++++----
.../tpch-approved-plan/v1-ras/spark34/3.txt | 230 ++++---
.../tpch-approved-plan/v1-ras/spark34/4.txt | 194 +++---
.../tpch-approved-plan/v1-ras/spark34/5.txt | 538 +++++++++-------
.../tpch-approved-plan/v1-ras/spark34/6.txt | 65 +-
.../tpch-approved-plan/v1-ras/spark34/7.txt | 511 ++++++++-------
.../tpch-approved-plan/v1-ras/spark34/8.txt | 714 ++++++++++++---------
.../tpch-approved-plan/v1-ras/spark34/9.txt | 534 ++++++++-------
.../resources/tpch-approved-plan/v1/spark32/1.txt | 100 +--
.../resources/tpch-approved-plan/v1/spark32/10.txt | 345 +++++-----
.../resources/tpch-approved-plan/v1/spark32/11.txt | 284 ++++----
.../resources/tpch-approved-plan/v1/spark32/12.txt | 190 +++---
.../resources/tpch-approved-plan/v1/spark32/13.txt | 204 +++---
.../resources/tpch-approved-plan/v1/spark32/14.txt | 132 ++--
.../resources/tpch-approved-plan/v1/spark32/15.txt | 173 ++---
.../resources/tpch-approved-plan/v1/spark32/16.txt | 251 ++++----
.../resources/tpch-approved-plan/v1/spark32/17.txt | 221 ++++---
.../resources/tpch-approved-plan/v1/spark32/18.txt | 382 ++++++-----
.../resources/tpch-approved-plan/v1/spark32/19.txt | 128 ++--
.../resources/tpch-approved-plan/v1/spark32/20.txt | 502 ++++++++-------
.../resources/tpch-approved-plan/v1/spark32/21.txt | 476 ++++++++------
.../resources/tpch-approved-plan/v1/spark32/22.txt | 178 ++---
.../resources/tpch-approved-plan/v1/spark32/3.txt | 230 ++++---
.../resources/tpch-approved-plan/v1/spark32/4.txt | 194 +++---
.../resources/tpch-approved-plan/v1/spark32/5.txt | 538 +++++++++-------
.../resources/tpch-approved-plan/v1/spark32/6.txt | 65 +-
.../resources/tpch-approved-plan/v1/spark32/7.txt | 511 ++++++++-------
.../resources/tpch-approved-plan/v1/spark32/8.txt | 714 ++++++++++++---------
.../resources/tpch-approved-plan/v1/spark32/9.txt | 534 ++++++++-------
.../resources/tpch-approved-plan/v1/spark33/1.txt | 100 +--
.../resources/tpch-approved-plan/v1/spark33/10.txt | 345 +++++-----
.../resources/tpch-approved-plan/v1/spark33/11.txt | 478 +++++++-------
.../resources/tpch-approved-plan/v1/spark33/12.txt | 190 +++---
.../resources/tpch-approved-plan/v1/spark33/13.txt | 204 +++---
.../resources/tpch-approved-plan/v1/spark33/14.txt | 132 ++--
.../resources/tpch-approved-plan/v1/spark33/15.txt | 266 ++++----
.../resources/tpch-approved-plan/v1/spark33/16.txt | 251 ++++----
.../resources/tpch-approved-plan/v1/spark33/17.txt | 221 ++++---
.../resources/tpch-approved-plan/v1/spark33/18.txt | 382 ++++++-----
.../resources/tpch-approved-plan/v1/spark33/19.txt | 128 ++--
.../resources/tpch-approved-plan/v1/spark33/20.txt | 494 +++++++-------
.../resources/tpch-approved-plan/v1/spark33/21.txt | 472 ++++++++------
.../resources/tpch-approved-plan/v1/spark33/22.txt | 286 +++++----
.../resources/tpch-approved-plan/v1/spark33/3.txt | 230 ++++---
.../resources/tpch-approved-plan/v1/spark33/4.txt | 194 +++---
.../resources/tpch-approved-plan/v1/spark33/5.txt | 538 +++++++++-------
.../resources/tpch-approved-plan/v1/spark33/6.txt | 65 +-
.../resources/tpch-approved-plan/v1/spark33/7.txt | 511 ++++++++-------
.../resources/tpch-approved-plan/v1/spark33/8.txt | 714 ++++++++++++---------
.../resources/tpch-approved-plan/v1/spark33/9.txt | 534 ++++++++-------
.../resources/tpch-approved-plan/v1/spark34/1.txt | 100 +--
.../resources/tpch-approved-plan/v1/spark34/10.txt | 345 +++++-----
.../resources/tpch-approved-plan/v1/spark34/11.txt | 478 +++++++-------
.../resources/tpch-approved-plan/v1/spark34/12.txt | 190 +++---
.../resources/tpch-approved-plan/v1/spark34/13.txt | 204 +++---
.../resources/tpch-approved-plan/v1/spark34/14.txt | 132 ++--
.../resources/tpch-approved-plan/v1/spark34/15.txt | 266 ++++----
.../resources/tpch-approved-plan/v1/spark34/16.txt | 251 ++++----
.../resources/tpch-approved-plan/v1/spark34/17.txt | 221 ++++---
.../resources/tpch-approved-plan/v1/spark34/18.txt | 382 ++++++-----
.../resources/tpch-approved-plan/v1/spark34/19.txt | 128 ++--
.../resources/tpch-approved-plan/v1/spark34/20.txt | 494 +++++++-------
.../resources/tpch-approved-plan/v1/spark34/21.txt | 472 ++++++++------
.../resources/tpch-approved-plan/v1/spark34/22.txt | 286 +++++----
.../resources/tpch-approved-plan/v1/spark34/3.txt | 230 ++++---
.../resources/tpch-approved-plan/v1/spark34/4.txt | 194 +++---
.../resources/tpch-approved-plan/v1/spark34/5.txt | 538 +++++++++-------
.../resources/tpch-approved-plan/v1/spark34/6.txt | 65 +-
.../resources/tpch-approved-plan/v1/spark34/7.txt | 511 ++++++++-------
.../resources/tpch-approved-plan/v1/spark34/8.txt | 714 ++++++++++++---------
.../resources/tpch-approved-plan/v1/spark34/9.txt | 534 ++++++++-------
.../org/apache/spark/sql/GlutenImplicitsTest.scala | 8 +-
.../errors/GlutenQueryExecutionErrorsSuite.scala | 9 +
.../scala/org/apache/gluten/GlutenConfig.scala | 11 +-
256 files changed, 35079 insertions(+), 29973 deletions(-)
diff --git a/backends-velox/src/main/scala/org/apache/gluten/extension/FlushableHashAggregateRule.scala b/backends-velox/src/main/scala/org/apache/gluten/extension/FlushableHashAggregateRule.scala
index f850b6f45..777bf5538 100644
--- a/backends-velox/src/main/scala/org/apache/gluten/extension/FlushableHashAggregateRule.scala
+++ b/backends-velox/src/main/scala/org/apache/gluten/extension/FlushableHashAggregateRule.scala
@@ -16,7 +16,7 @@
*/
package org.apache.gluten.extension
-import org.apache.gluten.execution.{FlushableHashAggregateExecTransformer, HashAggregateExecTransformer, ProjectExecTransformer, RegularHashAggregateExecTransformer}
+import org.apache.gluten.execution._
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.catalyst.expressions.aggregate.{Partial, PartialMerge}
@@ -30,74 +30,55 @@ import org.apache.spark.sql.execution.exchange.ShuffleExchangeLike
* optimizations such as flushing and abandoning.
*/
case class FlushableHashAggregateRule(session: SparkSession) extends Rule[SparkPlan] {
+ import FlushableHashAggregateRule._
override def apply(plan: SparkPlan): SparkPlan = plan.transformUp {
- case shuffle: ShuffleExchangeLike =>
+ case s: ShuffleExchangeLike =>
// If an exchange follows a hash aggregate in which all functions are in partial mode,
// then it's safe to convert the hash aggregate to flushable hash aggregate.
- shuffle.child match {
- case HashAggPropagatedToShuffle(proj, agg) =>
- shuffle.withNewChildren(
- Seq(proj.withNewChildren(Seq(FlushableHashAggregateExecTransformer(
- agg.requiredChildDistributionExpressions,
- agg.groupingExpressions,
- agg.aggregateExpressions,
- agg.aggregateAttributes,
- agg.initialInputBufferOffset,
- agg.resultExpressions,
- agg.child
- )))))
- case HashAggWithShuffle(agg) =>
- shuffle.withNewChildren(
- Seq(FlushableHashAggregateExecTransformer(
- agg.requiredChildDistributionExpressions,
- agg.groupingExpressions,
- agg.aggregateExpressions,
- agg.aggregateAttributes,
- agg.initialInputBufferOffset,
- agg.resultExpressions,
- agg.child
- )))
- case _ =>
- shuffle
- }
+ val out = s.withNewChildren(
+ List(
+ replaceEligibleAggregates(s.child) {
+ agg =>
+ FlushableHashAggregateExecTransformer(
+ agg.requiredChildDistributionExpressions,
+ agg.groupingExpressions,
+ agg.aggregateExpressions,
+ agg.aggregateAttributes,
+ agg.initialInputBufferOffset,
+ agg.resultExpressions,
+ agg.child
+ )
+ }
+ )
+ )
+ out
}
-}
-object HashAggPropagatedToShuffle {
- def unapply(
- plan: SparkPlan): Option[(ProjectExecTransformer, RegularHashAggregateExecTransformer)] = {
- if (!plan.isInstanceOf[ProjectExecTransformer]) {
- return None
- }
- val proj = plan.asInstanceOf[ProjectExecTransformer]
- val child = proj.child
- if (!child.isInstanceOf[RegularHashAggregateExecTransformer]) {
- return None
- }
- val agg = child.asInstanceOf[RegularHashAggregateExecTransformer]
- if (!agg.aggregateExpressions.forall(p => p.mode == Partial || p.mode == PartialMerge)) {
- return None
+ private def replaceEligibleAggregates(plan: SparkPlan)(
+ func: RegularHashAggregateExecTransformer => SparkPlan): SparkPlan = {
+ def transformDown: SparkPlan => SparkPlan = {
+ case agg: RegularHashAggregateExecTransformer
+ if !agg.aggregateExpressions.forall(p => p.mode == Partial || p.mode == PartialMerge) =>
+ // Not a intermediate agg. Skip.
+ agg
+ case agg: RegularHashAggregateExecTransformer
+ if isAggInputAlreadyDistributedWithAggKeys(agg) =>
+ // Data already grouped by aggregate keys, Skip.
+ agg
+ case agg: RegularHashAggregateExecTransformer =>
+ func(agg)
+ case p if !canPropagate(p) => p
+ case other => other.withNewChildren(other.children.map(transformDown))
}
- if (FlushableHashAggregateRule.isAggInputAlreadyDistributedWithAggKeys(agg)) {
- return None
- }
- Some((proj, agg))
+
+ val out = transformDown(plan)
+ out
}
-}
-object HashAggWithShuffle {
- def unapply(plan: SparkPlan): Option[RegularHashAggregateExecTransformer] = {
- if (!plan.isInstanceOf[RegularHashAggregateExecTransformer]) {
- return None
- }
- val agg = plan.asInstanceOf[RegularHashAggregateExecTransformer]
- if (!agg.aggregateExpressions.forall(p => p.mode == Partial || p.mode == PartialMerge)) {
- return None
- }
- if (FlushableHashAggregateRule.isAggInputAlreadyDistributedWithAggKeys(agg)) {
- return None
- }
- Some(agg)
+ private def canPropagate(plan: SparkPlan): Boolean = plan match {
+ case _: ProjectExecTransformer => true
+ case _: VeloxAppendBatchesExec => true
+ case _ => false
}
}
@@ -112,7 +93,8 @@ object FlushableHashAggregateRule {
* only on a single partition among the whole cluster. Spark's planner may use this information to
* perform optimizations like doing "partial_count(a, b, c)" directly on the output data.
*/
- def isAggInputAlreadyDistributedWithAggKeys(agg: HashAggregateExecTransformer): Boolean = {
+ private def isAggInputAlreadyDistributedWithAggKeys(
+ agg: HashAggregateExecTransformer): Boolean = {
if (agg.groupingExpressions.isEmpty) {
// Empty grouping set () should not be satisfied by any partitioning patterns.
// E.g.,
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/1.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/1.txt
index 9316f6f8e..53edb933c 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/1.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/1.txt
@@ -1,29 +1,31 @@
== Physical Plan ==
-AdaptiveSparkPlan (28)
+AdaptiveSparkPlan (30)
+- == Final Plan ==
- VeloxColumnarToRowExec (19)
- +- ^ SortExecTransformer (17)
- +- ^ InputIteratorTransformer (16)
- +- ShuffleQueryStage (14)
- +- ColumnarExchange (13)
- +- ^ RegularHashAggregateExecTransformer (11)
- +- ^ InputIteratorTransformer (10)
- +- ShuffleQueryStage (8)
- +- ColumnarExchange (7)
- +- ^ ProjectExecTransformer (5)
- +- ^ FlushableHashAggregateExecTransformer (4)
- +- ^ ProjectExecTransformer (3)
- +- ^ NoopFilter (2)
- +- ^ Scan parquet (1)
+ VeloxColumnarToRowExec (21)
+ +- ^ SortExecTransformer (19)
+ +- ^ InputIteratorTransformer (18)
+ +- ShuffleQueryStage (16)
+ +- ColumnarExchange (15)
+ +- VeloxAppendBatches (14)
+ +- ^ RegularHashAggregateExecTransformer (12)
+ +- ^ InputIteratorTransformer (11)
+ +- ShuffleQueryStage (9)
+ +- ColumnarExchange (8)
+ +- VeloxAppendBatches (7)
+ +- ^ ProjectExecTransformer (5)
+ +- ^ FlushableHashAggregateExecTransformer (4)
+ +- ^ ProjectExecTransformer (3)
+ +- ^ NoopFilter (2)
+ +- ^ Scan parquet (1)
+- == Initial Plan ==
- Sort (27)
- +- Exchange (26)
- +- HashAggregate (25)
- +- Exchange (24)
- +- HashAggregate (23)
- +- Project (22)
- +- Filter (21)
- +- Scan parquet (20)
+ Sort (29)
+ +- Exchange (28)
+ +- HashAggregate (27)
+ +- Exchange (26)
+ +- HashAggregate (25)
+ +- Project (24)
+ +- Filter (23)
+ +- Scan parquet (22)
(1) Scan parquet
@@ -56,97 +58,105 @@ Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X,
Input [18]: [hash_partition_key#X, l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: false
-(7) ColumnarExchange
+(7) VeloxAppendBatches
+Input [18]: [hash_partition_key#X, l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
+Arguments: X
+
+(8) ColumnarExchange
Input [18]: [hash_partition_key#X, l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: hashpartitioning(l_returnflag#X, l_linestatus#X, 1), ENSURE_REQUIREMENTS, [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X], [plan_id=X], [id=#X]
-(8) ShuffleQueryStage
+(9) ShuffleQueryStage
Output [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: X
-(9) InputAdapter
+(10) InputAdapter
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
-(10) InputIteratorTransformer
+(11) InputIteratorTransformer
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
-(11) RegularHashAggregateExecTransformer
+(12) RegularHashAggregateExecTransformer
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Keys [2]: [l_returnflag#X, l_linestatus#X]
Functions [8]: [sum(l_quantity#X), sum(l_extendedprice#X), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true)), sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal [...]
Aggregate Attributes [8]: [sum(l_quantity#X)#X, sum(l_extendedprice#X)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_disc [...]
Results [10]: [l_returnflag#X, l_linestatus#X, sum(l_quantity#X)#X AS sum_qty#X, sum(l_extendedprice#X)#X AS sum_base_price#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS sum_disc_price#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2)) [...]
-(12) WholeStageCodegenTransformer (X)
+(13) WholeStageCodegenTransformer (X)
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: false
-(13) ColumnarExchange
+(14) VeloxAppendBatches
+Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
+Arguments: X
+
+(15) ColumnarExchange
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: rangepartitioning(l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(14) ShuffleQueryStage
+(16) ShuffleQueryStage
Output [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: X
-(15) InputAdapter
+(17) InputAdapter
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
-(16) InputIteratorTransformer
+(18) InputIteratorTransformer
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
-(17) SortExecTransformer
+(19) SortExecTransformer
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: [l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST], true, 0
-(18) WholeStageCodegenTransformer (X)
+(20) WholeStageCodegenTransformer (X)
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: false
-(19) VeloxColumnarToRowExec
+(21) VeloxColumnarToRowExec
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
-(20) Scan parquet
+(22) Scan parquet
Output [7]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), LessThanOrEqual(l_shipdate,1998-09-02)]
ReadSchema: struct<l_quantity:decimal(12,2),l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_tax:decimal(12,2),l_returnflag:string,l_linestatus:string,l_shipdate:date>
-(21) Filter
+(23) Filter
Input [7]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X, l_shipdate#X]
Condition : (isnotnull(l_shipdate#X) AND (l_shipdate#X <= 1998-09-02))
-(22) Project
+(24) Project
Output [6]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X]
Input [7]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X, l_shipdate#X]
-(23) HashAggregate
+(25) HashAggregate
Input [6]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X]
Keys [2]: [l_returnflag#X, l_linestatus#X]
Functions [8]: [partial_sum(l_quantity#X), partial_sum(l_extendedprice#X), partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true)), partial_sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precis [...]
Aggregate Attributes [15]: [sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Results [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
-(24) Exchange
+(26) Exchange
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: hashpartitioning(l_returnflag#X, l_linestatus#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(25) HashAggregate
+(27) HashAggregate
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Keys [2]: [l_returnflag#X, l_linestatus#X]
Functions [8]: [sum(l_quantity#X), sum(l_extendedprice#X), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true)), sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal [...]
Aggregate Attributes [8]: [sum(l_quantity#X)#X, sum(l_extendedprice#X)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_disc [...]
Results [10]: [l_returnflag#X, l_linestatus#X, sum(l_quantity#X)#X AS sum_qty#X, sum(l_extendedprice#X)#X AS sum_base_price#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS sum_disc_price#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2)) [...]
-(26) Exchange
+(28) Exchange
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: rangepartitioning(l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(27) Sort
+(29) Sort
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: [l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST], true, 0
-(28) AdaptiveSparkPlan
+(30) AdaptiveSparkPlan
Output [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/10.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/10.txt
index d67e373da..29f28ac26 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/10.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/10.txt
@@ -1,64 +1,65 @@
== Physical Plan ==
-AdaptiveSparkPlan (67)
+AdaptiveSparkPlan (68)
+- == Final Plan ==
- VeloxColumnarToRowExec (43)
- +- TakeOrderedAndProjectExecTransformer (42)
- +- ^ ProjectExecTransformer (40)
- +- ^ RegularHashAggregateExecTransformer (39)
- +- ^ InputIteratorTransformer (38)
- +- ShuffleQueryStage (36)
- +- ColumnarExchange (35)
- +- ^ ProjectExecTransformer (33)
- +- ^ FlushableHashAggregateExecTransformer (32)
- +- ^ ProjectExecTransformer (31)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (30)
- :- ^ ProjectExecTransformer (22)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (21)
- : :- ^ ProjectExecTransformer (12)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
- : : :- ^ NoopFilter (2)
- : : : +- ^ Scan parquet (1)
- : : +- ^ InputIteratorTransformer (10)
- : : +- BroadcastQueryStage (8)
- : : +- ColumnarBroadcastExchange (7)
- : : +- ^ ProjectExecTransformer (5)
- : : +- ^ NoopFilter (4)
- : : +- ^ Scan parquet (3)
- : +- ^ InputIteratorTransformer (20)
- : +- BroadcastQueryStage (18)
- : +- ColumnarBroadcastExchange (17)
- : +- ^ ProjectExecTransformer (15)
- : +- ^ NoopFilter (14)
- : +- ^ Scan parquet (13)
- +- ^ InputIteratorTransformer (29)
- +- BroadcastQueryStage (27)
- +- ColumnarBroadcastExchange (26)
- +- ^ NoopFilter (24)
- +- ^ Scan parquet (23)
+ VeloxColumnarToRowExec (44)
+ +- TakeOrderedAndProjectExecTransformer (43)
+ +- ^ ProjectExecTransformer (41)
+ +- ^ RegularHashAggregateExecTransformer (40)
+ +- ^ InputIteratorTransformer (39)
+ +- ShuffleQueryStage (37)
+ +- ColumnarExchange (36)
+ +- VeloxAppendBatches (35)
+ +- ^ ProjectExecTransformer (33)
+ +- ^ FlushableHashAggregateExecTransformer (32)
+ +- ^ ProjectExecTransformer (31)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (30)
+ :- ^ ProjectExecTransformer (22)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (21)
+ : :- ^ ProjectExecTransformer (12)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
+ : : :- ^ NoopFilter (2)
+ : : : +- ^ Scan parquet (1)
+ : : +- ^ InputIteratorTransformer (10)
+ : : +- BroadcastQueryStage (8)
+ : : +- ColumnarBroadcastExchange (7)
+ : : +- ^ ProjectExecTransformer (5)
+ : : +- ^ NoopFilter (4)
+ : : +- ^ Scan parquet (3)
+ : +- ^ InputIteratorTransformer (20)
+ : +- BroadcastQueryStage (18)
+ : +- ColumnarBroadcastExchange (17)
+ : +- ^ ProjectExecTransformer (15)
+ : +- ^ NoopFilter (14)
+ : +- ^ Scan parquet (13)
+ +- ^ InputIteratorTransformer (29)
+ +- BroadcastQueryStage (27)
+ +- ColumnarBroadcastExchange (26)
+ +- ^ NoopFilter (24)
+ +- ^ Scan parquet (23)
+- == Initial Plan ==
- TakeOrderedAndProject (66)
- +- HashAggregate (65)
- +- Exchange (64)
- +- HashAggregate (63)
- +- Project (62)
- +- BroadcastHashJoin Inner BuildRight (61)
- :- Project (57)
- : +- BroadcastHashJoin Inner BuildRight (56)
- : :- Project (51)
- : : +- BroadcastHashJoin Inner BuildRight (50)
- : : :- Filter (45)
- : : : +- Scan parquet (44)
- : : +- BroadcastExchange (49)
- : : +- Project (48)
- : : +- Filter (47)
- : : +- Scan parquet (46)
- : +- BroadcastExchange (55)
- : +- Project (54)
- : +- Filter (53)
- : +- Scan parquet (52)
- +- BroadcastExchange (60)
- +- Filter (59)
- +- Scan parquet (58)
+ TakeOrderedAndProject (67)
+ +- HashAggregate (66)
+ +- Exchange (65)
+ +- HashAggregate (64)
+ +- Project (63)
+ +- BroadcastHashJoin Inner BuildRight (62)
+ :- Project (58)
+ : +- BroadcastHashJoin Inner BuildRight (57)
+ : :- Project (52)
+ : : +- BroadcastHashJoin Inner BuildRight (51)
+ : : :- Filter (46)
+ : : : +- Scan parquet (45)
+ : : +- BroadcastExchange (50)
+ : : +- Project (49)
+ : : +- Filter (48)
+ : : +- Scan parquet (47)
+ : +- BroadcastExchange (56)
+ : +- Project (55)
+ : +- Filter (54)
+ : +- Scan parquet (53)
+ +- BroadcastExchange (61)
+ +- Filter (60)
+ +- Scan parquet (59)
(1) Scan parquet
@@ -209,155 +210,159 @@ Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X
Input [10]: [hash_partition_key#X, c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: false
-(35) ColumnarExchange
+(35) VeloxAppendBatches
+Input [10]: [hash_partition_key#X, c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
+Arguments: X
+
+(36) ColumnarExchange
Input [10]: [hash_partition_key#X, c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, 1), ENSURE_REQUIREMENTS, [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(36) ShuffleQueryStage
+(37) ShuffleQueryStage
Output [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: X
-(37) InputAdapter
+(38) InputAdapter
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
-(38) InputIteratorTransformer
+(39) InputIteratorTransformer
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
-(39) RegularHashAggregateExecTransformer
+(40) RegularHashAggregateExecTransformer
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Keys [7]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [8]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
-(40) ProjectExecTransformer
+(41) ProjectExecTransformer
Output [8]: [c_custkey#X, c_name#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Input [8]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
-(41) WholeStageCodegenTransformer (X)
+(42) WholeStageCodegenTransformer (X)
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: false
-(42) TakeOrderedAndProjectExecTransformer
+(43) TakeOrderedAndProjectExecTransformer
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: X, [revenue#X DESC NULLS LAST], [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X], 0
-(43) VeloxColumnarToRowExec
+(44) VeloxColumnarToRowExec
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
-(44) Scan parquet
+(45) Scan parquet
Output [7]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_custkey), IsNotNull(c_nationkey)]
ReadSchema: struct<c_custkey:bigint,c_name:string,c_address:string,c_nationkey:bigint,c_phone:string,c_acctbal:decimal(12,2),c_comment:string>
-(45) Filter
+(46) Filter
Input [7]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X]
Condition : (isnotnull(c_custkey#X) AND isnotnull(c_nationkey#X))
-(46) Scan parquet
+(47) Scan parquet
Output [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderdate), GreaterThanOrEqual(o_orderdate,1993-10-01), LessThan(o_orderdate,1994-01-01), IsNotNull(o_custkey), IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_orderdate:date>
-(47) Filter
+(48) Filter
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Condition : ((((isnotnull(o_orderdate#X) AND (o_orderdate#X >= 1993-10-01)) AND (o_orderdate#X < 1994-01-01)) AND isnotnull(o_custkey#X)) AND isnotnull(o_orderkey#X))
-(48) Project
+(49) Project
Output [2]: [o_orderkey#X, o_custkey#X]
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
-(49) BroadcastExchange
+(50) BroadcastExchange
Input [2]: [o_orderkey#X, o_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[1, bigint, true]),false), [plan_id=X]
-(50) BroadcastHashJoin
+(51) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(51) Project
+(52) Project
Output [8]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, o_orderkey#X]
Input [9]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, o_orderkey#X, o_custkey#X]
-(52) Scan parquet
+(53) Scan parquet
Output [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_returnflag#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_returnflag), EqualTo(l_returnflag,R), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_returnflag:string>
-(53) Filter
+(54) Filter
Input [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_returnflag#X]
Condition : ((isnotnull(l_returnflag#X) AND (l_returnflag#X = R)) AND isnotnull(l_orderkey#X))
-(54) Project
+(55) Project
Output [3]: [l_orderkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_returnflag#X]
-(55) BroadcastExchange
+(56) BroadcastExchange
Input [3]: [l_orderkey#X, l_extendedprice#X, l_discount#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(56) BroadcastHashJoin
+(57) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(57) Project
+(58) Project
Output [9]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X]
Input [11]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, o_orderkey#X, l_orderkey#X, l_extendedprice#X, l_discount#X]
-(58) Scan parquet
+(59) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(59) Filter
+(60) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : isnotnull(n_nationkey#X)
-(60) BroadcastExchange
+(61) BroadcastExchange
Input [2]: [n_nationkey#X, n_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(61) BroadcastHashJoin
+(62) BroadcastHashJoin
Left keys [1]: [c_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(62) Project
+(63) Project
Output [9]: [c_custkey#X, c_name#X, c_address#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X, n_name#X]
Input [11]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X, n_nationkey#X, n_name#X]
-(63) HashAggregate
+(64) HashAggregate
Input [9]: [c_custkey#X, c_name#X, c_address#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X, n_name#X]
Keys [7]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
-(64) Exchange
+(65) Exchange
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(65) HashAggregate
+(66) HashAggregate
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Keys [7]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [8]: [c_custkey#X, c_name#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
-(66) TakeOrderedAndProject
+(67) TakeOrderedAndProject
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: X, [revenue#X DESC NULLS LAST], [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
-(67) AdaptiveSparkPlan
+(68) AdaptiveSparkPlan
Output [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/11.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/11.txt
index f65747c37..b8f466eb9 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/11.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/11.txt
@@ -1,55 +1,57 @@
== Physical Plan ==
-AdaptiveSparkPlan (58)
+AdaptiveSparkPlan (60)
+- == Final Plan ==
- VeloxColumnarToRowExec (38)
- +- ^ SortExecTransformer (36)
- +- ^ InputIteratorTransformer (35)
- +- ShuffleQueryStage (33)
- +- ColumnarExchange (32)
- +- ^ FilterExecTransformer (30)
- +- ^ RegularHashAggregateExecTransformer (29)
- +- ^ InputIteratorTransformer (28)
- +- ShuffleQueryStage (26)
- +- ColumnarExchange (25)
- +- ^ ProjectExecTransformer (23)
- +- ^ FlushableHashAggregateExecTransformer (22)
- +- ^ ProjectExecTransformer (21)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
- :- ^ ProjectExecTransformer (11)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
- : :- ^ NoopFilter (2)
- : : +- ^ Scan parquet (1)
- : +- ^ InputIteratorTransformer (9)
- : +- BroadcastQueryStage (7)
- : +- ColumnarBroadcastExchange (6)
- : +- ^ NoopFilter (4)
- : +- ^ Scan parquet (3)
- +- ^ InputIteratorTransformer (19)
- +- BroadcastQueryStage (17)
- +- ColumnarBroadcastExchange (16)
- +- ^ ProjectExecTransformer (14)
- +- ^ NoopFilter (13)
- +- ^ Scan parquet (12)
+ VeloxColumnarToRowExec (40)
+ +- ^ SortExecTransformer (38)
+ +- ^ InputIteratorTransformer (37)
+ +- ShuffleQueryStage (35)
+ +- ColumnarExchange (34)
+ +- VeloxAppendBatches (33)
+ +- ^ FilterExecTransformer (31)
+ +- ^ RegularHashAggregateExecTransformer (30)
+ +- ^ InputIteratorTransformer (29)
+ +- ShuffleQueryStage (27)
+ +- ColumnarExchange (26)
+ +- VeloxAppendBatches (25)
+ +- ^ ProjectExecTransformer (23)
+ +- ^ FlushableHashAggregateExecTransformer (22)
+ +- ^ ProjectExecTransformer (21)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
+ :- ^ ProjectExecTransformer (11)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
+ : :- ^ NoopFilter (2)
+ : : +- ^ Scan parquet (1)
+ : +- ^ InputIteratorTransformer (9)
+ : +- BroadcastQueryStage (7)
+ : +- ColumnarBroadcastExchange (6)
+ : +- ^ NoopFilter (4)
+ : +- ^ Scan parquet (3)
+ +- ^ InputIteratorTransformer (19)
+ +- BroadcastQueryStage (17)
+ +- ColumnarBroadcastExchange (16)
+ +- ^ ProjectExecTransformer (14)
+ +- ^ NoopFilter (13)
+ +- ^ Scan parquet (12)
+- == Initial Plan ==
- Sort (57)
- +- Exchange (56)
- +- Filter (55)
- +- HashAggregate (54)
- +- Exchange (53)
- +- HashAggregate (52)
- +- Project (51)
- +- BroadcastHashJoin Inner BuildRight (50)
- :- Project (45)
- : +- BroadcastHashJoin Inner BuildRight (44)
- : :- Filter (40)
- : : +- Scan parquet (39)
- : +- BroadcastExchange (43)
- : +- Filter (42)
- : +- Scan parquet (41)
- +- BroadcastExchange (49)
- +- Project (48)
- +- Filter (47)
- +- Scan parquet (46)
+ Sort (59)
+ +- Exchange (58)
+ +- Filter (57)
+ +- HashAggregate (56)
+ +- Exchange (55)
+ +- HashAggregate (54)
+ +- Project (53)
+ +- BroadcastHashJoin Inner BuildRight (52)
+ :- Project (47)
+ : +- BroadcastHashJoin Inner BuildRight (46)
+ : :- Filter (42)
+ : : +- Scan parquet (41)
+ : +- BroadcastExchange (45)
+ : +- Filter (44)
+ : +- Scan parquet (43)
+ +- BroadcastExchange (51)
+ +- Project (50)
+ +- Filter (49)
+ +- Scan parquet (48)
(1) Scan parquet
@@ -158,153 +160,161 @@ Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Input [4]: [hash_partition_key#X, ps_partkey#X, sum#X, isEmpty#X]
Arguments: false
-(25) ColumnarExchange
+(25) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, ps_partkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(26) ColumnarExchange
Input [4]: [hash_partition_key#X, ps_partkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(ps_partkey#X, 1), ENSURE_REQUIREMENTS, [ps_partkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(26) ShuffleQueryStage
+(27) ShuffleQueryStage
Output [3]: [ps_partkey#X, sum#X, isEmpty#X]
Arguments: X
-(27) InputAdapter
+(28) InputAdapter
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
-(28) InputIteratorTransformer
+(29) InputIteratorTransformer
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
-(29) RegularHashAggregateExecTransformer
+(30) RegularHashAggregateExecTransformer
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Keys [1]: [ps_partkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))#X]
Results [2]: [ps_partkey#X, sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))#X AS value#X]
-(30) FilterExecTransformer
+(31) FilterExecTransformer
Input [2]: [ps_partkey#X, value#X]
Arguments: (isnotnull(value#X) AND (cast(value#X as decimal(38,6)) > Subquery subquery#X, [id=#X]))
-(31) WholeStageCodegenTransformer (X)
+(32) WholeStageCodegenTransformer (X)
Input [2]: [ps_partkey#X, value#X]
Arguments: false
-(32) ColumnarExchange
+(33) VeloxAppendBatches
+Input [2]: [ps_partkey#X, value#X]
+Arguments: X
+
+(34) ColumnarExchange
Input [2]: [ps_partkey#X, value#X]
Arguments: rangepartitioning(value#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(33) ShuffleQueryStage
+(35) ShuffleQueryStage
Output [2]: [ps_partkey#X, value#X]
Arguments: X
-(34) InputAdapter
+(36) InputAdapter
Input [2]: [ps_partkey#X, value#X]
-(35) InputIteratorTransformer
+(37) InputIteratorTransformer
Input [2]: [ps_partkey#X, value#X]
-(36) SortExecTransformer
+(38) SortExecTransformer
Input [2]: [ps_partkey#X, value#X]
Arguments: [value#X DESC NULLS LAST], true, 0
-(37) WholeStageCodegenTransformer (X)
+(39) WholeStageCodegenTransformer (X)
Input [2]: [ps_partkey#X, value#X]
Arguments: false
-(38) VeloxColumnarToRowExec
+(40) VeloxColumnarToRowExec
Input [2]: [ps_partkey#X, value#X]
-(39) Scan parquet
+(41) Scan parquet
Output [4]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_suppkey)]
ReadSchema: struct<ps_partkey:bigint,ps_suppkey:bigint,ps_availqty:int,ps_supplycost:decimal(12,2)>
-(40) Filter
+(42) Filter
Input [4]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Condition : isnotnull(ps_suppkey#X)
-(41) Scan parquet
+(43) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(42) Filter
+(44) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(43) BroadcastExchange
+(45) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(44) BroadcastHashJoin
+(46) BroadcastHashJoin
Left keys [1]: [ps_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(45) Project
+(47) Project
Output [4]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X, s_nationkey#X]
Input [6]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, ps_supplycost#X, s_suppkey#X, s_nationkey#X]
-(46) Scan parquet
+(48) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_name), EqualTo(n_name,GERMANY), IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(47) Filter
+(49) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : ((isnotnull(n_name#X) AND (n_name#X = GERMANY)) AND isnotnull(n_nationkey#X))
-(48) Project
+(50) Project
Output [1]: [n_nationkey#X]
Input [2]: [n_nationkey#X, n_name#X]
-(49) BroadcastExchange
+(51) BroadcastExchange
Input [1]: [n_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(50) BroadcastHashJoin
+(52) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(51) Project
+(53) Project
Output [3]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X]
Input [5]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X, s_nationkey#X, n_nationkey#X]
-(52) HashAggregate
+(54) HashAggregate
Input [3]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X]
Keys [1]: [ps_partkey#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [ps_partkey#X, sum#X, isEmpty#X]
-(53) Exchange
+(55) Exchange
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(ps_partkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(54) HashAggregate
+(56) HashAggregate
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Keys [1]: [ps_partkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))#X]
Results [2]: [ps_partkey#X, sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(cast(ps_availqty#X as decimal(10,0)) as decimal(12,2)))), DecimalType(23,2), true))#X AS value#X]
-(55) Filter
+(57) Filter
Input [2]: [ps_partkey#X, value#X]
Condition : (isnotnull(value#X) AND (cast(value#X as decimal(38,6)) > Subquery subquery#X, [id=#X]))
-(56) Exchange
+(58) Exchange
Input [2]: [ps_partkey#X, value#X]
Arguments: rangepartitioning(value#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(57) Sort
+(59) Sort
Input [2]: [ps_partkey#X, value#X]
Arguments: [value#X DESC NULLS LAST], true, 0
-(58) AdaptiveSparkPlan
+(60) AdaptiveSparkPlan
Output [2]: [ps_partkey#X, value#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/12.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/12.txt
index f1b7bbfe6..1e60a9391 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/12.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/12.txt
@@ -1,41 +1,43 @@
== Physical Plan ==
-AdaptiveSparkPlan (42)
+AdaptiveSparkPlan (44)
+- == Final Plan ==
- VeloxColumnarToRowExec (28)
- +- ^ SortExecTransformer (26)
- +- ^ InputIteratorTransformer (25)
- +- ShuffleQueryStage (23)
- +- ColumnarExchange (22)
- +- ^ RegularHashAggregateExecTransformer (20)
- +- ^ InputIteratorTransformer (19)
- +- ShuffleQueryStage (17)
- +- ColumnarExchange (16)
- +- ^ ProjectExecTransformer (14)
- +- ^ FlushableHashAggregateExecTransformer (13)
- +- ^ ProjectExecTransformer (12)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
- :- ^ InputIteratorTransformer (7)
- : +- BroadcastQueryStage (5)
- : +- ColumnarBroadcastExchange (4)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ ProjectExecTransformer (10)
- +- ^ NoopFilter (9)
- +- ^ Scan parquet (8)
+ VeloxColumnarToRowExec (30)
+ +- ^ SortExecTransformer (28)
+ +- ^ InputIteratorTransformer (27)
+ +- ShuffleQueryStage (25)
+ +- ColumnarExchange (24)
+ +- VeloxAppendBatches (23)
+ +- ^ RegularHashAggregateExecTransformer (21)
+ +- ^ InputIteratorTransformer (20)
+ +- ShuffleQueryStage (18)
+ +- ColumnarExchange (17)
+ +- VeloxAppendBatches (16)
+ +- ^ ProjectExecTransformer (14)
+ +- ^ FlushableHashAggregateExecTransformer (13)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
+ :- ^ InputIteratorTransformer (7)
+ : +- BroadcastQueryStage (5)
+ : +- ColumnarBroadcastExchange (4)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ ProjectExecTransformer (10)
+ +- ^ NoopFilter (9)
+ +- ^ Scan parquet (8)
+- == Initial Plan ==
- Sort (41)
- +- Exchange (40)
- +- HashAggregate (39)
- +- Exchange (38)
- +- HashAggregate (37)
- +- Project (36)
- +- BroadcastHashJoin Inner BuildLeft (35)
- :- BroadcastExchange (31)
- : +- Filter (30)
- : +- Scan parquet (29)
- +- Project (34)
- +- Filter (33)
- +- Scan parquet (32)
+ Sort (43)
+ +- Exchange (42)
+ +- HashAggregate (41)
+ +- Exchange (40)
+ +- HashAggregate (39)
+ +- Project (38)
+ +- BroadcastHashJoin Inner BuildLeft (37)
+ :- BroadcastExchange (33)
+ : +- Filter (32)
+ : +- Scan parquet (31)
+ +- Project (36)
+ +- Filter (35)
+ +- Scan parquet (34)
(1) Scan parquet
@@ -106,121 +108,129 @@ Input [3]: [l_shipmode#X, sum#X, sum#X]
Input [4]: [hash_partition_key#X, l_shipmode#X, sum#X, sum#X]
Arguments: false
-(16) ColumnarExchange
+(16) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, l_shipmode#X, sum#X, sum#X]
+Arguments: X
+
+(17) ColumnarExchange
Input [4]: [hash_partition_key#X, l_shipmode#X, sum#X, sum#X]
Arguments: hashpartitioning(l_shipmode#X, 1), ENSURE_REQUIREMENTS, [l_shipmode#X, sum#X, sum#X], [plan_id=X], [id=#X]
-(17) ShuffleQueryStage
+(18) ShuffleQueryStage
Output [3]: [l_shipmode#X, sum#X, sum#X]
Arguments: X
-(18) InputAdapter
+(19) InputAdapter
Input [3]: [l_shipmode#X, sum#X, sum#X]
-(19) InputIteratorTransformer
+(20) InputIteratorTransformer
Input [3]: [l_shipmode#X, sum#X, sum#X]
-(20) RegularHashAggregateExecTransformer
+(21) RegularHashAggregateExecTransformer
Input [3]: [l_shipmode#X, sum#X, sum#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]
-(21) WholeStageCodegenTransformer (X)
+(22) WholeStageCodegenTransformer (X)
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: false
-(22) ColumnarExchange
+(23) VeloxAppendBatches
+Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
+Arguments: X
+
+(24) ColumnarExchange
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: rangepartitioning(l_shipmode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(23) ShuffleQueryStage
+(25) ShuffleQueryStage
Output [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: X
-(24) InputAdapter
+(26) InputAdapter
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
-(25) InputIteratorTransformer
+(27) InputIteratorTransformer
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
-(26) SortExecTransformer
+(28) SortExecTransformer
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: [l_shipmode#X ASC NULLS FIRST], true, 0
-(27) WholeStageCodegenTransformer (X)
+(29) WholeStageCodegenTransformer (X)
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: false
-(28) VeloxColumnarToRowExec
+(30) VeloxColumnarToRowExec
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
-(29) Scan parquet
+(31) Scan parquet
Output [2]: [o_orderkey#X, o_orderpriority#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_orderpriority:string>
-(30) Filter
+(32) Filter
Input [2]: [o_orderkey#X, o_orderpriority#X]
Condition : isnotnull(o_orderkey#X)
-(31) BroadcastExchange
+(33) BroadcastExchange
Input [2]: [o_orderkey#X, o_orderpriority#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(32) Scan parquet
+(34) Scan parquet
Output [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_commitdate), IsNotNull(l_receiptdate), IsNotNull(l_shipdate), In(l_shipmode, [MAIL,SHIP]), GreaterThanOrEqual(l_receiptdate,1994-01-01), LessThan(l_receiptdate,1995-01-01), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_shipdate:date,l_commitdate:date,l_receiptdate:date,l_shipmode:string>
-(33) Filter
+(35) Filter
Input [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
Condition : ((((((((isnotnull(l_commitdate#X) AND isnotnull(l_receiptdate#X)) AND isnotnull(l_shipdate#X)) AND l_shipmode#X IN (MAIL,SHIP)) AND (l_commitdate#X < l_receiptdate#X)) AND (l_shipdate#X < l_commitdate#X)) AND (l_receiptdate#X >= 1994-01-01)) AND (l_receiptdate#X < 1995-01-01)) AND isnotnull(l_orderkey#X))
-(34) Project
+(36) Project
Output [2]: [l_orderkey#X, l_shipmode#X]
Input [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
-(35) BroadcastHashJoin
+(37) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(36) Project
+(38) Project
Output [2]: [o_orderpriority#X, l_shipmode#X]
Input [4]: [o_orderkey#X, o_orderpriority#X, l_orderkey#X, l_shipmode#X]
-(37) HashAggregate
+(39) HashAggregate
Input [2]: [o_orderpriority#X, l_shipmode#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [partial_sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum#X, sum#X]
Results [3]: [l_shipmode#X, sum#X, sum#X]
-(38) Exchange
+(40) Exchange
Input [3]: [l_shipmode#X, sum#X, sum#X]
Arguments: hashpartitioning(l_shipmode#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(39) HashAggregate
+(41) HashAggregate
Input [3]: [l_shipmode#X, sum#X, sum#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]
-(40) Exchange
+(42) Exchange
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: rangepartitioning(l_shipmode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(41) Sort
+(43) Sort
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: [l_shipmode#X ASC NULLS FIRST], true, 0
-(42) AdaptiveSparkPlan
+(44) AdaptiveSparkPlan
Output [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/13.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/13.txt
index 12147d4b3..c525944e1 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/13.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/13.txt
@@ -1,49 +1,52 @@
== Physical Plan ==
-AdaptiveSparkPlan (52)
+AdaptiveSparkPlan (55)
+- == Final Plan ==
- VeloxColumnarToRowExec (36)
- +- ^ SortExecTransformer (34)
- +- ^ InputIteratorTransformer (33)
- +- ShuffleQueryStage (31)
- +- ColumnarExchange (30)
- +- ^ RegularHashAggregateExecTransformer (28)
- +- ^ InputIteratorTransformer (27)
- +- ShuffleQueryStage (25)
- +- ColumnarExchange (24)
- +- ^ ProjectExecTransformer (22)
- +- ^ FlushableHashAggregateExecTransformer (21)
- +- ^ ProjectExecTransformer (20)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16)
- +- ColumnarExchange (15)
- +- ^ ProjectExecTransformer (13)
- +- ^ FlushableHashAggregateExecTransformer (12)
- +- ^ ProjectExecTransformer (11)
- +- ^ BroadcastHashJoinExecTransformer LeftOuter BuildRight (10)
- :- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (9)
- +- BroadcastQueryStage (7)
- +- ColumnarBroadcastExchange (6)
- +- ^ ProjectExecTransformer (4)
- +- ^ NoopFilter (3)
- +- ^ Scan parquet (2)
+ VeloxColumnarToRowExec (39)
+ +- ^ SortExecTransformer (37)
+ +- ^ InputIteratorTransformer (36)
+ +- ShuffleQueryStage (34)
+ +- ColumnarExchange (33)
+ +- VeloxAppendBatches (32)
+ +- ^ RegularHashAggregateExecTransformer (30)
+ +- ^ InputIteratorTransformer (29)
+ +- ShuffleQueryStage (27)
+ +- ColumnarExchange (26)
+ +- VeloxAppendBatches (25)
+ +- ^ ProjectExecTransformer (23)
+ +- ^ FlushableHashAggregateExecTransformer (22)
+ +- ^ ProjectExecTransformer (21)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ ProjectExecTransformer (13)
+ +- ^ FlushableHashAggregateExecTransformer (12)
+ +- ^ ProjectExecTransformer (11)
+ +- ^ BroadcastHashJoinExecTransformer LeftOuter BuildRight (10)
+ :- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (9)
+ +- BroadcastQueryStage (7)
+ +- ColumnarBroadcastExchange (6)
+ +- ^ ProjectExecTransformer (4)
+ +- ^ NoopFilter (3)
+ +- ^ Scan parquet (2)
+- == Initial Plan ==
- Sort (51)
- +- Exchange (50)
- +- HashAggregate (49)
- +- Exchange (48)
- +- HashAggregate (47)
- +- HashAggregate (46)
- +- Exchange (45)
- +- HashAggregate (44)
- +- Project (43)
- +- BroadcastHashJoin LeftOuter BuildRight (42)
- :- Scan parquet (37)
- +- BroadcastExchange (41)
- +- Project (40)
- +- Filter (39)
- +- Scan parquet (38)
+ Sort (54)
+ +- Exchange (53)
+ +- HashAggregate (52)
+ +- Exchange (51)
+ +- HashAggregate (50)
+ +- HashAggregate (49)
+ +- Exchange (48)
+ +- HashAggregate (47)
+ +- Project (46)
+ +- BroadcastHashJoin LeftOuter BuildRight (45)
+ :- Scan parquet (40)
+ +- BroadcastExchange (44)
+ +- Project (43)
+ +- Filter (42)
+ +- Scan parquet (41)
(1) Scan parquet
@@ -109,174 +112,186 @@ Input [2]: [c_custkey#X, count#X]
Input [3]: [hash_partition_key#X, c_custkey#X, count#X]
Arguments: false
-(15) ColumnarExchange
+(15) VeloxAppendBatches
+Input [3]: [hash_partition_key#X, c_custkey#X, count#X]
+Arguments: X
+
+(16) ColumnarExchange
Input [3]: [hash_partition_key#X, c_custkey#X, count#X]
Arguments: hashpartitioning(c_custkey#X, 1), ENSURE_REQUIREMENTS, [c_custkey#X, count#X], [plan_id=X], [id=#X]
-(16) ShuffleQueryStage
+(17) ShuffleQueryStage
Output [2]: [c_custkey#X, count#X]
Arguments: X
-(17) InputAdapter
+(18) InputAdapter
Input [2]: [c_custkey#X, count#X]
-(18) InputIteratorTransformer
+(19) InputIteratorTransformer
Input [2]: [c_custkey#X, count#X]
-(19) RegularHashAggregateExecTransformer
+(20) RegularHashAggregateExecTransformer
Input [2]: [c_custkey#X, count#X]
Keys [1]: [c_custkey#X]
Functions [1]: [count(o_orderkey#X)]
Aggregate Attributes [1]: [count(o_orderkey#X)#X]
Results [2]: [c_custkey#X, count(o_orderkey#X)#X]
-(20) ProjectExecTransformer
+(21) ProjectExecTransformer
Output [1]: [count(o_orderkey#X)#X AS c_count#X]
Input [2]: [c_custkey#X, count(o_orderkey#X)#X]
-(21) FlushableHashAggregateExecTransformer
+(22) FlushableHashAggregateExecTransformer
Input [1]: [c_count#X]
Keys [1]: [c_count#X]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#X]
Results [2]: [c_count#X, count#X]
-(22) ProjectExecTransformer
+(23) ProjectExecTransformer
Output [3]: [hash(c_count#X, 42) AS hash_partition_key#X, c_count#X, count#X]
Input [2]: [c_count#X, count#X]
-(23) WholeStageCodegenTransformer (X)
+(24) WholeStageCodegenTransformer (X)
Input [3]: [hash_partition_key#X, c_count#X, count#X]
Arguments: false
-(24) ColumnarExchange
+(25) VeloxAppendBatches
+Input [3]: [hash_partition_key#X, c_count#X, count#X]
+Arguments: X
+
+(26) ColumnarExchange
Input [3]: [hash_partition_key#X, c_count#X, count#X]
Arguments: hashpartitioning(c_count#X, 1), ENSURE_REQUIREMENTS, [c_count#X, count#X], [plan_id=X], [id=#X]
-(25) ShuffleQueryStage
+(27) ShuffleQueryStage
Output [2]: [c_count#X, count#X]
Arguments: X
-(26) InputAdapter
+(28) InputAdapter
Input [2]: [c_count#X, count#X]
-(27) InputIteratorTransformer
+(29) InputIteratorTransformer
Input [2]: [c_count#X, count#X]
-(28) RegularHashAggregateExecTransformer
+(30) RegularHashAggregateExecTransformer
Input [2]: [c_count#X, count#X]
Keys [1]: [c_count#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [c_count#X, count(1)#X AS custdist#X]
-(29) WholeStageCodegenTransformer (X)
+(31) WholeStageCodegenTransformer (X)
Input [2]: [c_count#X, custdist#X]
Arguments: false
-(30) ColumnarExchange
+(32) VeloxAppendBatches
+Input [2]: [c_count#X, custdist#X]
+Arguments: X
+
+(33) ColumnarExchange
Input [2]: [c_count#X, custdist#X]
Arguments: rangepartitioning(custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(31) ShuffleQueryStage
+(34) ShuffleQueryStage
Output [2]: [c_count#X, custdist#X]
Arguments: X
-(32) InputAdapter
+(35) InputAdapter
Input [2]: [c_count#X, custdist#X]
-(33) InputIteratorTransformer
+(36) InputIteratorTransformer
Input [2]: [c_count#X, custdist#X]
-(34) SortExecTransformer
+(37) SortExecTransformer
Input [2]: [c_count#X, custdist#X]
Arguments: [custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST], true, 0
-(35) WholeStageCodegenTransformer (X)
+(38) WholeStageCodegenTransformer (X)
Input [2]: [c_count#X, custdist#X]
Arguments: false
-(36) VeloxColumnarToRowExec
+(39) VeloxColumnarToRowExec
Input [2]: [c_count#X, custdist#X]
-(37) Scan parquet
+(40) Scan parquet
Output [1]: [c_custkey#X]
Batched: true
Location: InMemoryFileIndex [*]
ReadSchema: struct<c_custkey:bigint>
-(38) Scan parquet
+(41) Scan parquet
Output [3]: [o_orderkey#X, o_custkey#X, o_comment#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_comment), IsNotNull(o_custkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_comment:string>
-(39) Filter
+(42) Filter
Input [3]: [o_orderkey#X, o_custkey#X, o_comment#X]
Condition : ((isnotnull(o_comment#X) AND NOT o_comment#X LIKE %special%requests%) AND isnotnull(o_custkey#X))
-(40) Project
+(43) Project
Output [2]: [o_orderkey#X, o_custkey#X]
Input [3]: [o_orderkey#X, o_custkey#X, o_comment#X]
-(41) BroadcastExchange
+(44) BroadcastExchange
Input [2]: [o_orderkey#X, o_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[1, bigint, true]),false), [plan_id=X]
-(42) BroadcastHashJoin
+(45) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(43) Project
+(46) Project
Output [2]: [c_custkey#X, o_orderkey#X]
Input [3]: [c_custkey#X, o_orderkey#X, o_custkey#X]
-(44) HashAggregate
+(47) HashAggregate
Input [2]: [c_custkey#X, o_orderkey#X]
Keys [1]: [c_custkey#X]
Functions [1]: [partial_count(o_orderkey#X)]
Aggregate Attributes [1]: [count#X]
Results [2]: [c_custkey#X, count#X]
-(45) Exchange
+(48) Exchange
Input [2]: [c_custkey#X, count#X]
Arguments: hashpartitioning(c_custkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(46) HashAggregate
+(49) HashAggregate
Input [2]: [c_custkey#X, count#X]
Keys [1]: [c_custkey#X]
Functions [1]: [count(o_orderkey#X)]
Aggregate Attributes [1]: [count(o_orderkey#X)#X]
Results [1]: [count(o_orderkey#X)#X AS c_count#X]
-(47) HashAggregate
+(50) HashAggregate
Input [1]: [c_count#X]
Keys [1]: [c_count#X]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#X]
Results [2]: [c_count#X, count#X]
-(48) Exchange
+(51) Exchange
Input [2]: [c_count#X, count#X]
Arguments: hashpartitioning(c_count#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(49) HashAggregate
+(52) HashAggregate
Input [2]: [c_count#X, count#X]
Keys [1]: [c_count#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [c_count#X, count(1)#X AS custdist#X]
-(50) Exchange
+(53) Exchange
Input [2]: [c_count#X, custdist#X]
Arguments: rangepartitioning(custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(51) Sort
+(54) Sort
Input [2]: [c_count#X, custdist#X]
Arguments: [custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST], true, 0
-(52) AdaptiveSparkPlan
+(55) AdaptiveSparkPlan
Output [2]: [c_count#X, custdist#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/14.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/14.txt
index 87dc45754..fbfcf0703 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/14.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/14.txt
@@ -1,35 +1,36 @@
== Physical Plan ==
-AdaptiveSparkPlan (34)
+AdaptiveSparkPlan (35)
+- == Final Plan ==
- VeloxColumnarToRowExec (22)
- +- ^ ProjectExecTransformer (20)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16)
- +- ColumnarExchange (15)
- +- ^ FlushableHashAggregateExecTransformer (13)
- +- ^ ProjectExecTransformer (12)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
- :- ^ ProjectExecTransformer (3)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (10)
- +- BroadcastQueryStage (8)
- +- ColumnarBroadcastExchange (7)
- +- ^ NoopFilter (5)
- +- ^ Scan parquet (4)
+ VeloxColumnarToRowExec (23)
+ +- ^ ProjectExecTransformer (21)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ FlushableHashAggregateExecTransformer (13)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
+ :- ^ ProjectExecTransformer (3)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (10)
+ +- BroadcastQueryStage (8)
+ +- ColumnarBroadcastExchange (7)
+ +- ^ NoopFilter (5)
+ +- ^ Scan parquet (4)
+- == Initial Plan ==
- HashAggregate (33)
- +- Exchange (32)
- +- HashAggregate (31)
- +- Project (30)
- +- BroadcastHashJoin Inner BuildRight (29)
- :- Project (25)
- : +- Filter (24)
- : +- Scan parquet (23)
- +- BroadcastExchange (28)
- +- Filter (27)
- +- Scan parquet (26)
+ HashAggregate (34)
+ +- Exchange (33)
+ +- HashAggregate (32)
+ +- Project (31)
+ +- BroadcastHashJoin Inner BuildRight (30)
+ :- Project (26)
+ : +- Filter (25)
+ : +- Scan parquet (24)
+ +- BroadcastExchange (29)
+ +- Filter (28)
+ +- Scan parquet (27)
(1) Scan parquet
@@ -96,95 +97,99 @@ Results [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: false
-(15) ColumnarExchange
+(15) VeloxAppendBatches
+Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
+Arguments: X
+
+(16) ColumnarExchange
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(16) ShuffleQueryStage
+(17) ShuffleQueryStage
Output [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: X
-(17) InputAdapter
+(18) InputAdapter
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
-(18) InputIteratorTransformer
+(19) InputIteratorTransformer
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
-(19) RegularHashAggregateExecTransformer
+(20) RegularHashAggregateExecTransformer
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Keys: []
Functions [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), Decima [...]
Aggregate Attributes [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13, [...]
Results [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), Decima [...]
-(20) ProjectExecTransformer
+(21) ProjectExecTransformer
Output [1]: [CheckOverflow((promote_precision(CheckOverflow((100.0000 * promote_precision(sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END)#X)), DecimalType(38,6), true)) / promote_precision(cast(sum(CheckOverflow((promote_precision(cast(l_extendedpri [...]
Input [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalT [...]
-(21) WholeStageCodegenTransformer (X)
+(22) WholeStageCodegenTransformer (X)
Input [1]: [promo_revenue#X]
Arguments: false
-(22) VeloxColumnarToRowExec
+(23) VeloxColumnarToRowExec
Input [1]: [promo_revenue#X]
-(23) Scan parquet
+(24) Scan parquet
Output [4]: [l_partkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1995-09-01), LessThan(l_shipdate,1995-10-01), IsNotNull(l_partkey)]
ReadSchema: struct<l_partkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(24) Filter
+(25) Filter
Input [4]: [l_partkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : (((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1995-09-01)) AND (l_shipdate#X < 1995-10-01)) AND isnotnull(l_partkey#X))
-(25) Project
+(26) Project
Output [3]: [l_partkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_partkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(26) Scan parquet
+(27) Scan parquet
Output [2]: [p_partkey#X, p_type#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_partkey)]
ReadSchema: struct<p_partkey:bigint,p_type:string>
-(27) Filter
+(28) Filter
Input [2]: [p_partkey#X, p_type#X]
Condition : isnotnull(p_partkey#X)
-(28) BroadcastExchange
+(29) BroadcastExchange
Input [2]: [p_partkey#X, p_type#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(29) BroadcastHashJoin
+(30) BroadcastHashJoin
Left keys [1]: [l_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(30) Project
+(31) Project
Output [3]: [l_extendedprice#X, l_discount#X, p_type#X]
Input [5]: [l_partkey#X, l_extendedprice#X, l_discount#X, p_partkey#X, p_type#X]
-(31) HashAggregate
+(32) HashAggregate
Input [3]: [l_extendedprice#X, l_discount#X, p_type#X]
Keys: []
Functions [2]: [partial_sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END), partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal( [...]
Aggregate Attributes [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Results [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
-(32) Exchange
+(33) Exchange
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X]
-(33) HashAggregate
+(34) HashAggregate
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Keys: []
Functions [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), Decima [...]
Aggregate Attributes [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13, [...]
Results [1]: [CheckOverflow((promote_precision(CheckOverflow((100.0000 * promote_precision(sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) ELSE 0.0000 END)#X)), DecimalType(38,6), true)) / promote_precision(cast(sum(CheckOverflow((promote_precision(cast(l_extendedpr [...]
-(34) AdaptiveSparkPlan
+(35) AdaptiveSparkPlan
Output [1]: [promo_revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/15.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/15.txt
index 0bfcd38f0..b5073901c 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/15.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/15.txt
@@ -1,43 +1,45 @@
== Physical Plan ==
-AdaptiveSparkPlan (44)
+AdaptiveSparkPlan (46)
+- == Final Plan ==
- VeloxColumnarToRowExec (29)
- +- ^ SortExecTransformer (27)
- +- ^ InputIteratorTransformer (26)
- +- ShuffleQueryStage (24)
- +- ColumnarExchange (23)
- +- ^ ProjectExecTransformer (21)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (20)
- :- ^ InputIteratorTransformer (7)
- : +- BroadcastQueryStage (5)
- : +- ColumnarBroadcastExchange (4)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ FilterExecTransformer (19)
- +- ^ RegularHashAggregateExecTransformer (18)
- +- ^ InputIteratorTransformer (17)
- +- ShuffleQueryStage (15)
- +- ColumnarExchange (14)
- +- ^ ProjectExecTransformer (12)
- +- ^ FlushableHashAggregateExecTransformer (11)
- +- ^ ProjectExecTransformer (10)
- +- ^ NoopFilter (9)
- +- ^ Scan parquet (8)
+ VeloxColumnarToRowExec (31)
+ +- ^ SortExecTransformer (29)
+ +- ^ InputIteratorTransformer (28)
+ +- ShuffleQueryStage (26)
+ +- ColumnarExchange (25)
+ +- VeloxAppendBatches (24)
+ +- ^ ProjectExecTransformer (22)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (21)
+ :- ^ InputIteratorTransformer (7)
+ : +- BroadcastQueryStage (5)
+ : +- ColumnarBroadcastExchange (4)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ FilterExecTransformer (20)
+ +- ^ RegularHashAggregateExecTransformer (19)
+ +- ^ InputIteratorTransformer (18)
+ +- ShuffleQueryStage (16)
+ +- ColumnarExchange (15)
+ +- VeloxAppendBatches (14)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ FlushableHashAggregateExecTransformer (11)
+ +- ^ ProjectExecTransformer (10)
+ +- ^ NoopFilter (9)
+ +- ^ Scan parquet (8)
+- == Initial Plan ==
- Sort (43)
- +- Exchange (42)
- +- Project (41)
- +- BroadcastHashJoin Inner BuildLeft (40)
- :- BroadcastExchange (32)
- : +- Filter (31)
- : +- Scan parquet (30)
- +- Filter (39)
- +- HashAggregate (38)
- +- Exchange (37)
- +- HashAggregate (36)
- +- Project (35)
- +- Filter (34)
- +- Scan parquet (33)
+ Sort (45)
+ +- Exchange (44)
+ +- Project (43)
+ +- BroadcastHashJoin Inner BuildLeft (42)
+ :- BroadcastExchange (34)
+ : +- Filter (33)
+ : +- Scan parquet (32)
+ +- Filter (41)
+ +- HashAggregate (40)
+ +- Exchange (39)
+ +- HashAggregate (38)
+ +- Project (37)
+ +- Filter (36)
+ +- Scan parquet (35)
(1) Scan parquet
@@ -99,138 +101,146 @@ Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: false
-(14) ColumnarExchange
+(14) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(15) ColumnarExchange
Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_suppkey#X, 1), ENSURE_REQUIREMENTS, [l_suppkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(15) ShuffleQueryStage
+(16) ShuffleQueryStage
Output [3]: [l_suppkey#X, sum#X, isEmpty#X]
Arguments: X
-(16) InputAdapter
+(17) InputAdapter
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(17) InputIteratorTransformer
+(18) InputIteratorTransformer
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(18) RegularHashAggregateExecTransformer
+(19) RegularHashAggregateExecTransformer
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [2]: [l_suppkey#X AS supplier_no#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS total_revenue#X]
-(19) FilterExecTransformer
+(20) FilterExecTransformer
Input [2]: [supplier_no#X, total_revenue#X]
Arguments: (isnotnull(total_revenue#X) AND (total_revenue#X = Subquery subquery#X, [id=#X]))
-(20) BroadcastHashJoinExecTransformer
+(21) BroadcastHashJoinExecTransformer
Left keys [1]: [s_suppkey#X]
Right keys [1]: [supplier_no#X]
Join condition: None
-(21) ProjectExecTransformer
+(22) ProjectExecTransformer
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Input [6]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, supplier_no#X, total_revenue#X]
-(22) WholeStageCodegenTransformer (X)
+(23) WholeStageCodegenTransformer (X)
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: false
-(23) ColumnarExchange
+(24) VeloxAppendBatches
+Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
+Arguments: X
+
+(25) ColumnarExchange
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: rangepartitioning(s_suppkey#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(24) ShuffleQueryStage
+(26) ShuffleQueryStage
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: X
-(25) InputAdapter
+(27) InputAdapter
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
-(26) InputIteratorTransformer
+(28) InputIteratorTransformer
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
-(27) SortExecTransformer
+(29) SortExecTransformer
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: [s_suppkey#X ASC NULLS FIRST], true, 0
-(28) WholeStageCodegenTransformer (X)
+(30) WholeStageCodegenTransformer (X)
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: false
-(29) VeloxColumnarToRowExec
+(31) VeloxColumnarToRowExec
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
-(30) Scan parquet
+(32) Scan parquet
Output [4]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey)]
ReadSchema: struct<s_suppkey:bigint,s_name:string,s_address:string,s_phone:string>
-(31) Filter
+(33) Filter
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X]
Condition : isnotnull(s_suppkey#X)
-(32) BroadcastExchange
+(34) BroadcastExchange
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(33) Scan parquet
+(35) Scan parquet
Output [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1996-01-01), LessThan(l_shipdate,1996-04-01), IsNotNull(l_suppkey)]
ReadSchema: struct<l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(34) Filter
+(36) Filter
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : (((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1996-01-01)) AND (l_shipdate#X < 1996-04-01)) AND isnotnull(l_suppkey#X))
-(35) Project
+(37) Project
Output [3]: [l_suppkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(36) HashAggregate
+(38) HashAggregate
Input [3]: [l_suppkey#X, l_extendedprice#X, l_discount#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(37) Exchange
+(39) Exchange
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(38) HashAggregate
+(40) HashAggregate
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [2]: [l_suppkey#X AS supplier_no#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS total_revenue#X]
-(39) Filter
+(41) Filter
Input [2]: [supplier_no#X, total_revenue#X]
Condition : (isnotnull(total_revenue#X) AND (total_revenue#X = Subquery subquery#X, [id=#X]))
-(40) BroadcastHashJoin
+(42) BroadcastHashJoin
Left keys [1]: [s_suppkey#X]
Right keys [1]: [supplier_no#X]
Join condition: None
-(41) Project
+(43) Project
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Input [6]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, supplier_no#X, total_revenue#X]
-(42) Exchange
+(44) Exchange
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: rangepartitioning(s_suppkey#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(43) Sort
+(45) Sort
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: [s_suppkey#X ASC NULLS FIRST], true, 0
-(44) AdaptiveSparkPlan
+(46) AdaptiveSparkPlan
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/16.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/16.txt
index ccc548821..f3cef9275 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/16.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/16.txt
@@ -1,53 +1,56 @@
== Physical Plan ==
-AdaptiveSparkPlan (56)
+AdaptiveSparkPlan (59)
+- == Final Plan ==
- VeloxColumnarToRowExec (35)
- +- ^ SortExecTransformer (33)
- +- ^ InputIteratorTransformer (32)
- +- ShuffleQueryStage (30)
- +- ColumnarExchange (29)
- +- ^ RegularHashAggregateExecTransformer (27)
- +- ^ InputIteratorTransformer (26)
- +- ShuffleQueryStage (24)
- +- ColumnarExchange (23)
- +- ^ ProjectExecTransformer (21)
- +- ^ FlushableHashAggregateExecTransformer (20)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16)
- +- ColumnarExchange (15)
- +- ^ ProjectExecTransformer (13)
- +- ^ FlushableHashAggregateExecTransformer (12)
- +- ^ ProjectExecTransformer (11)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
- :- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (9)
- +- BroadcastQueryStage (7)
- +- ColumnarBroadcastExchange (6)
- +- ^ NoopFilter (4)
- +- ^ Scan parquet (3)
+ VeloxColumnarToRowExec (38)
+ +- ^ SortExecTransformer (36)
+ +- ^ InputIteratorTransformer (35)
+ +- ShuffleQueryStage (33)
+ +- ColumnarExchange (32)
+ +- VeloxAppendBatches (31)
+ +- ^ RegularHashAggregateExecTransformer (29)
+ +- ^ InputIteratorTransformer (28)
+ +- ShuffleQueryStage (26)
+ +- ColumnarExchange (25)
+ +- VeloxAppendBatches (24)
+ +- ^ ProjectExecTransformer (22)
+ +- ^ FlushableHashAggregateExecTransformer (21)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ ProjectExecTransformer (13)
+ +- ^ FlushableHashAggregateExecTransformer (12)
+ +- ^ ProjectExecTransformer (11)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
+ :- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (9)
+ +- BroadcastQueryStage (7)
+ +- ColumnarBroadcastExchange (6)
+ +- ^ NoopFilter (4)
+ +- ^ Scan parquet (3)
+- == Initial Plan ==
- Sort (55)
- +- Exchange (54)
- +- HashAggregate (53)
- +- Exchange (52)
- +- HashAggregate (51)
- +- HashAggregate (50)
- +- Exchange (49)
- +- HashAggregate (48)
- +- Project (47)
- +- BroadcastHashJoin Inner BuildRight (46)
- :- BroadcastHashJoin LeftAnti BuildRight (42)
- : :- Filter (37)
- : : +- Scan parquet (36)
- : +- BroadcastExchange (41)
- : +- Project (40)
- : +- Filter (39)
- : +- Scan parquet (38)
- +- BroadcastExchange (45)
- +- Filter (44)
- +- Scan parquet (43)
+ Sort (58)
+ +- Exchange (57)
+ +- HashAggregate (56)
+ +- Exchange (55)
+ +- HashAggregate (54)
+ +- HashAggregate (53)
+ +- Exchange (52)
+ +- HashAggregate (51)
+ +- Project (50)
+ +- BroadcastHashJoin Inner BuildRight (49)
+ :- BroadcastHashJoin LeftAnti BuildRight (45)
+ : :- Filter (40)
+ : : +- Scan parquet (39)
+ : +- BroadcastExchange (44)
+ : +- Project (43)
+ : +- Filter (42)
+ : +- Scan parquet (41)
+ +- BroadcastExchange (48)
+ +- Filter (47)
+ +- Scan parquet (46)
(1) Scan parquet
@@ -114,195 +117,207 @@ Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Input [5]: [hash_partition_key#X, p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Arguments: false
-(15) ColumnarExchange
+(15) VeloxAppendBatches
+Input [5]: [hash_partition_key#X, p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
+Arguments: X
+
+(16) ColumnarExchange
Input [5]: [hash_partition_key#X, p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Arguments: hashpartitioning(p_brand#X, p_type#X, p_size#X, ps_suppkey#X, 1), ENSURE_REQUIREMENTS, [p_brand#X, p_type#X, p_size#X, ps_suppkey#X], [plan_id=X], [id=#X]
-(16) ShuffleQueryStage
+(17) ShuffleQueryStage
Output [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Arguments: X
-(17) InputAdapter
+(18) InputAdapter
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
-(18) InputIteratorTransformer
+(19) InputIteratorTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
-(19) RegularHashAggregateExecTransformer
+(20) RegularHashAggregateExecTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Keys [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Functions: []
Aggregate Attributes: []
Results [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
-(20) FlushableHashAggregateExecTransformer
+(21) FlushableHashAggregateExecTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Keys [3]: [p_brand#X, p_type#X, p_size#X]
Functions [1]: [partial_count(distinct ps_suppkey#X)]
Aggregate Attributes [1]: [count(ps_suppkey#X)#X]
Results [4]: [p_brand#X, p_type#X, p_size#X, count#X]
-(21) ProjectExecTransformer
+(22) ProjectExecTransformer
Output [5]: [hash(p_brand#X, p_type#X, p_size#X, 42) AS hash_partition_key#X, p_brand#X, p_type#X, p_size#X, count#X]
Input [4]: [p_brand#X, p_type#X, p_size#X, count#X]
-(22) WholeStageCodegenTransformer (X)
+(23) WholeStageCodegenTransformer (X)
Input [5]: [hash_partition_key#X, p_brand#X, p_type#X, p_size#X, count#X]
Arguments: false
-(23) ColumnarExchange
+(24) VeloxAppendBatches
+Input [5]: [hash_partition_key#X, p_brand#X, p_type#X, p_size#X, count#X]
+Arguments: X
+
+(25) ColumnarExchange
Input [5]: [hash_partition_key#X, p_brand#X, p_type#X, p_size#X, count#X]
Arguments: hashpartitioning(p_brand#X, p_type#X, p_size#X, 1), ENSURE_REQUIREMENTS, [p_brand#X, p_type#X, p_size#X, count#X], [plan_id=X], [id=#X]
-(24) ShuffleQueryStage
+(26) ShuffleQueryStage
Output [4]: [p_brand#X, p_type#X, p_size#X, count#X]
Arguments: X
-(25) InputAdapter
+(27) InputAdapter
Input [4]: [p_brand#X, p_type#X, p_size#X, count#X]
-(26) InputIteratorTransformer
+(28) InputIteratorTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, count#X]
-(27) RegularHashAggregateExecTransformer
+(29) RegularHashAggregateExecTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, count#X]
Keys [3]: [p_brand#X, p_type#X, p_size#X]
Functions [1]: [count(distinct ps_suppkey#X)]
Aggregate Attributes [1]: [count(ps_suppkey#X)#X]
Results [4]: [p_brand#X, p_type#X, p_size#X, count(ps_suppkey#X)#X AS supplier_cnt#X]
-(28) WholeStageCodegenTransformer (X)
+(30) WholeStageCodegenTransformer (X)
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: false
-(29) ColumnarExchange
+(31) VeloxAppendBatches
+Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
+Arguments: X
+
+(32) ColumnarExchange
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: rangepartitioning(supplier_cnt#X DESC NULLS LAST, p_brand#X ASC NULLS FIRST, p_type#X ASC NULLS FIRST, p_size#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(30) ShuffleQueryStage
+(33) ShuffleQueryStage
Output [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: X
-(31) InputAdapter
+(34) InputAdapter
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
-(32) InputIteratorTransformer
+(35) InputIteratorTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
-(33) SortExecTransformer
+(36) SortExecTransformer
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: [supplier_cnt#X DESC NULLS LAST, p_brand#X ASC NULLS FIRST, p_type#X ASC NULLS FIRST, p_size#X ASC NULLS FIRST], true, 0
-(34) WholeStageCodegenTransformer (X)
+(37) WholeStageCodegenTransformer (X)
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: false
-(35) VeloxColumnarToRowExec
+(38) VeloxColumnarToRowExec
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
-(36) Scan parquet
+(39) Scan parquet
Output [2]: [ps_partkey#X, ps_suppkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_partkey)]
ReadSchema: struct<ps_partkey:bigint,ps_suppkey:bigint>
-(37) Filter
+(40) Filter
Input [2]: [ps_partkey#X, ps_suppkey#X]
Condition : isnotnull(ps_partkey#X)
-(38) Scan parquet
+(41) Scan parquet
Output [2]: [s_suppkey#X, s_comment#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_comment)]
ReadSchema: struct<s_suppkey:bigint,s_comment:string>
-(39) Filter
+(42) Filter
Input [2]: [s_suppkey#X, s_comment#X]
Condition : (isnotnull(s_comment#X) AND s_comment#X LIKE %Customer%Complaints%)
-(40) Project
+(43) Project
Output [1]: [s_suppkey#X]
Input [2]: [s_suppkey#X, s_comment#X]
-(41) BroadcastExchange
+(44) BroadcastExchange
Input [1]: [s_suppkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),true), [plan_id=X]
-(42) BroadcastHashJoin
+(45) BroadcastHashJoin
Left keys [1]: [ps_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(43) Scan parquet
+(46) Scan parquet
Output [4]: [p_partkey#X, p_brand#X, p_type#X, p_size#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_brand), IsNotNull(p_type), Not(EqualTo(p_brand,Brand#X)), Not(StringStartsWith(p_type,MEDIUM POLISHED)), In(p_size, [14,19,23,3,36,45,49,9]), IsNotNull(p_partkey)]
ReadSchema: struct<p_partkey:bigint,p_brand:string,p_type:string,p_size:int>
-(44) Filter
+(47) Filter
Input [4]: [p_partkey#X, p_brand#X, p_type#X, p_size#X]
Condition : (((((isnotnull(p_brand#X) AND isnotnull(p_type#X)) AND NOT (p_brand#X = Brand#X)) AND NOT StartsWith(p_type#X, MEDIUM POLISHED)) AND p_size#X IN (49,14,23,45,19,3,36,9)) AND isnotnull(p_partkey#X))
-(45) BroadcastExchange
+(48) BroadcastExchange
Input [4]: [p_partkey#X, p_brand#X, p_type#X, p_size#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(46) BroadcastHashJoin
+(49) BroadcastHashJoin
Left keys [1]: [ps_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(47) Project
+(50) Project
Output [4]: [ps_suppkey#X, p_brand#X, p_type#X, p_size#X]
Input [6]: [ps_partkey#X, ps_suppkey#X, p_partkey#X, p_brand#X, p_type#X, p_size#X]
-(48) HashAggregate
+(51) HashAggregate
Input [4]: [ps_suppkey#X, p_brand#X, p_type#X, p_size#X]
Keys [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Functions: []
Aggregate Attributes: []
Results [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
-(49) Exchange
+(52) Exchange
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Arguments: hashpartitioning(p_brand#X, p_type#X, p_size#X, ps_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(50) HashAggregate
+(53) HashAggregate
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Keys [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Functions: []
Aggregate Attributes: []
Results [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
-(51) HashAggregate
+(54) HashAggregate
Input [4]: [p_brand#X, p_type#X, p_size#X, ps_suppkey#X]
Keys [3]: [p_brand#X, p_type#X, p_size#X]
Functions [1]: [partial_count(distinct ps_suppkey#X)]
Aggregate Attributes [1]: [count(ps_suppkey#X)#X]
Results [4]: [p_brand#X, p_type#X, p_size#X, count#X]
-(52) Exchange
+(55) Exchange
Input [4]: [p_brand#X, p_type#X, p_size#X, count#X]
Arguments: hashpartitioning(p_brand#X, p_type#X, p_size#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(53) HashAggregate
+(56) HashAggregate
Input [4]: [p_brand#X, p_type#X, p_size#X, count#X]
Keys [3]: [p_brand#X, p_type#X, p_size#X]
Functions [1]: [count(distinct ps_suppkey#X)]
Aggregate Attributes [1]: [count(ps_suppkey#X)#X]
Results [4]: [p_brand#X, p_type#X, p_size#X, count(ps_suppkey#X)#X AS supplier_cnt#X]
-(54) Exchange
+(57) Exchange
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: rangepartitioning(supplier_cnt#X DESC NULLS LAST, p_brand#X ASC NULLS FIRST, p_type#X ASC NULLS FIRST, p_size#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(55) Sort
+(58) Sort
Input [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: [supplier_cnt#X DESC NULLS LAST, p_brand#X ASC NULLS FIRST, p_type#X ASC NULLS FIRST, p_size#X ASC NULLS FIRST], true, 0
-(56) AdaptiveSparkPlan
+(59) AdaptiveSparkPlan
Output [4]: [p_brand#X, p_type#X, p_size#X, supplier_cnt#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/17.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/17.txt
index 9d81e1d50..7c5359849 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/17.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/17.txt
@@ -1,37 +1,38 @@
== Physical Plan ==
-AdaptiveSparkPlan (35)
+AdaptiveSparkPlan (36)
+- == Final Plan ==
- VeloxColumnarToRowExec (14)
- +- ^ ProjectExecTransformer (12)
- +- ^ RegularHashAggregateExecTransformer (11)
- +- ^ InputIteratorTransformer (10)
- +- ShuffleQueryStage (8)
- +- ColumnarExchange (7)
- +- ^ FlushableHashAggregateExecTransformer (5)
- +- ^ InputIteratorTransformer (4)
- +- RowToVeloxColumnar (2)
- +- LocalTableScan (1)
+ VeloxColumnarToRowExec (15)
+ +- ^ ProjectExecTransformer (13)
+ +- ^ RegularHashAggregateExecTransformer (12)
+ +- ^ InputIteratorTransformer (11)
+ +- ShuffleQueryStage (9)
+ +- ColumnarExchange (8)
+ +- VeloxAppendBatches (7)
+ +- ^ FlushableHashAggregateExecTransformer (5)
+ +- ^ InputIteratorTransformer (4)
+ +- RowToVeloxColumnar (2)
+ +- LocalTableScan (1)
+- == Initial Plan ==
- HashAggregate (34)
- +- Exchange (33)
- +- HashAggregate (32)
- +- Project (31)
- +- BroadcastHashJoin Inner BuildRight (30)
- :- Project (22)
- : +- BroadcastHashJoin Inner BuildRight (21)
- : :- Filter (16)
- : : +- Scan parquet (15)
- : +- BroadcastExchange (20)
- : +- Project (19)
- : +- Filter (18)
- : +- Scan parquet (17)
- +- BroadcastExchange (29)
- +- Filter (28)
- +- HashAggregate (27)
- +- Exchange (26)
- +- HashAggregate (25)
- +- Filter (24)
- +- Scan parquet (23)
+ HashAggregate (35)
+ +- Exchange (34)
+ +- HashAggregate (33)
+ +- Project (32)
+ +- BroadcastHashJoin Inner BuildRight (31)
+ :- Project (23)
+ : +- BroadcastHashJoin Inner BuildRight (22)
+ : :- Filter (17)
+ : : +- Scan parquet (16)
+ : +- BroadcastExchange (21)
+ : +- Project (20)
+ : +- Filter (19)
+ : +- Scan parquet (18)
+ +- BroadcastExchange (30)
+ +- Filter (29)
+ +- HashAggregate (28)
+ +- Exchange (27)
+ +- HashAggregate (26)
+ +- Filter (25)
+ +- Scan parquet (24)
(1) LocalTableScan
@@ -58,141 +59,145 @@ Results [2]: [sum#X, isEmpty#X]
Input [2]: [sum#X, isEmpty#X]
Arguments: false
-(7) ColumnarExchange
+(7) VeloxAppendBatches
+Input [2]: [sum#X, isEmpty#X]
+Arguments: X
+
+(8) ColumnarExchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(8) ShuffleQueryStage
+(9) ShuffleQueryStage
Output [2]: [sum#X, isEmpty#X]
Arguments: X
-(9) InputAdapter
+(10) InputAdapter
Input [2]: [sum#X, isEmpty#X]
-(10) InputIteratorTransformer
+(11) InputIteratorTransformer
Input [2]: [sum#X, isEmpty#X]
-(11) RegularHashAggregateExecTransformer
+(12) RegularHashAggregateExecTransformer
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(l_extendedprice#X)]
Aggregate Attributes [1]: [sum(l_extendedprice#X)#X]
Results [1]: [sum(l_extendedprice#X)#X]
-(12) ProjectExecTransformer
+(13) ProjectExecTransformer
Output [1]: [CheckOverflow((promote_precision(sum(l_extendedprice#X)#X) / 7.00), DecimalType(27,6), true) AS avg_yearly#X]
Input [1]: [sum(l_extendedprice#X)#X]
-(13) WholeStageCodegenTransformer (X)
+(14) WholeStageCodegenTransformer (X)
Input [1]: [avg_yearly#X]
Arguments: false
-(14) VeloxColumnarToRowExec
+(15) VeloxColumnarToRowExec
Input [1]: [avg_yearly#X]
-(15) Scan parquet
+(16) Scan parquet
Output [3]: [l_partkey#X, l_quantity#X, l_extendedprice#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_partkey), IsNotNull(l_quantity)]
ReadSchema: struct<l_partkey:bigint,l_quantity:decimal(12,2),l_extendedprice:decimal(12,2)>
-(16) Filter
+(17) Filter
Input [3]: [l_partkey#X, l_quantity#X, l_extendedprice#X]
Condition : (isnotnull(l_partkey#X) AND isnotnull(l_quantity#X))
-(17) Scan parquet
+(18) Scan parquet
Output [3]: [p_partkey#X, p_brand#X, p_container#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_brand), IsNotNull(p_container), EqualTo(p_brand,Brand#X), EqualTo(p_container,MED BOX), IsNotNull(p_partkey)]
ReadSchema: struct<p_partkey:bigint,p_brand:string,p_container:string>
-(18) Filter
+(19) Filter
Input [3]: [p_partkey#X, p_brand#X, p_container#X]
Condition : ((((isnotnull(p_brand#X) AND isnotnull(p_container#X)) AND (p_brand#X = Brand#X)) AND (p_container#X = MED BOX)) AND isnotnull(p_partkey#X))
-(19) Project
+(20) Project
Output [1]: [p_partkey#X]
Input [3]: [p_partkey#X, p_brand#X, p_container#X]
-(20) BroadcastExchange
+(21) BroadcastExchange
Input [1]: [p_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(21) BroadcastHashJoin
+(22) BroadcastHashJoin
Left keys [1]: [l_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(22) Project
+(23) Project
Output [3]: [l_quantity#X, l_extendedprice#X, p_partkey#X]
Input [4]: [l_partkey#X, l_quantity#X, l_extendedprice#X, p_partkey#X]
-(23) Scan parquet
+(24) Scan parquet
Output [2]: [l_partkey#X, l_quantity#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_partkey)]
ReadSchema: struct<l_partkey:bigint,l_quantity:decimal(12,2)>
-(24) Filter
+(25) Filter
Input [2]: [l_partkey#X, l_quantity#X]
Condition : isnotnull(l_partkey#X)
-(25) HashAggregate
+(26) HashAggregate
Input [2]: [l_partkey#X, l_quantity#X]
Keys [1]: [l_partkey#X]
Functions [1]: [partial_avg(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, count#X]
Results [3]: [l_partkey#X, sum#X, count#X]
-(26) Exchange
+(27) Exchange
Input [3]: [l_partkey#X, sum#X, count#X]
Arguments: hashpartitioning(l_partkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(27) HashAggregate
+(28) HashAggregate
Input [3]: [l_partkey#X, sum#X, count#X]
Keys [1]: [l_partkey#X]
Functions [1]: [avg(l_quantity#X)]
Aggregate Attributes [1]: [avg(l_quantity#X)#X]
Results [2]: [CheckOverflow((0.200000 * promote_precision(avg(l_quantity#X)#X)), DecimalType(18,7), true) AS (0.2 * avg(l_quantity))#X, l_partkey#X]
-(28) Filter
+(29) Filter
Input [2]: [(0.2 * avg(l_quantity))#X, l_partkey#X]
Condition : isnotnull((0.2 * avg(l_quantity))#X)
-(29) BroadcastExchange
+(30) BroadcastExchange
Input [2]: [(0.2 * avg(l_quantity))#X, l_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[1, bigint, true]),false), [plan_id=X]
-(30) BroadcastHashJoin
+(31) BroadcastHashJoin
Left keys [1]: [p_partkey#X]
Right keys [1]: [l_partkey#X]
Join condition: (cast(l_quantity#X as decimal(18,7)) < (0.2 * avg(l_quantity))#X)
-(31) Project
+(32) Project
Output [1]: [l_extendedprice#X]
Input [5]: [l_quantity#X, l_extendedprice#X, p_partkey#X, (0.2 * avg(l_quantity))#X, l_partkey#X]
-(32) HashAggregate
+(33) HashAggregate
Input [1]: [l_extendedprice#X]
Keys: []
Functions [1]: [partial_sum(l_extendedprice#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [2]: [sum#X, isEmpty#X]
-(33) Exchange
+(34) Exchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X]
-(34) HashAggregate
+(35) HashAggregate
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(l_extendedprice#X)]
Aggregate Attributes [1]: [sum(l_extendedprice#X)#X]
Results [1]: [CheckOverflow((promote_precision(sum(l_extendedprice#X)#X) / 7.00), DecimalType(27,6), true) AS avg_yearly#X]
-(35) AdaptiveSparkPlan
+(36) AdaptiveSparkPlan
Output [1]: [avg_yearly#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/18.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/18.txt
index e582fd47c..a0a6e26a6 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/18.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/18.txt
@@ -1,80 +1,82 @@
== Physical Plan ==
-AdaptiveSparkPlan (86)
+AdaptiveSparkPlan (88)
+- == Final Plan ==
- VeloxColumnarToRowExec (53)
- +- TakeOrderedAndProjectExecTransformer (52)
- +- ^ RegularHashAggregateExecTransformer (50)
- +- ^ InputIteratorTransformer (49)
- +- ShuffleQueryStage (47)
- +- ColumnarExchange (46)
- +- ^ ProjectExecTransformer (44)
- +- ^ FlushableHashAggregateExecTransformer (43)
- +- ^ ProjectExecTransformer (42)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (41)
- :- ^ ProjectExecTransformer (28)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (27)
- : :- ^ InputIteratorTransformer (7)
- : : +- BroadcastQueryStage (5)
- : : +- ColumnarBroadcastExchange (4)
- : : +- ^ NoopFilter (2)
- : : +- ^ Scan parquet (1)
- : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (26)
- : :- ^ NoopFilter (9)
- : : +- ^ Scan parquet (8)
- : +- ^ InputIteratorTransformer (25)
- : +- BroadcastQueryStage (23)
- : +- ColumnarBroadcastExchange (22)
- : +- ^ ProjectExecTransformer (20)
- : +- ^ FilterExecTransformer (19)
- : +- ^ RegularHashAggregateExecTransformer (18)
- : +- ^ InputIteratorTransformer (17)
- : +- ShuffleQueryStage (15)
- : +- ColumnarExchange (14)
- : +- ^ ProjectExecTransformer (12)
- : +- ^ FlushableHashAggregateExecTransformer (11)
- : +- ^ Scan parquet (10)
- +- ^ InputIteratorTransformer (40)
- +- BroadcastQueryStage (38)
- +- ColumnarBroadcastExchange (37)
- +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (35)
- :- ^ NoopFilter (30)
- : +- ^ Scan parquet (29)
- +- ^ InputIteratorTransformer (34)
- +- BroadcastQueryStage (32)
- +- ReusedExchange (31)
+ VeloxColumnarToRowExec (55)
+ +- TakeOrderedAndProjectExecTransformer (54)
+ +- ^ RegularHashAggregateExecTransformer (52)
+ +- ^ InputIteratorTransformer (51)
+ +- ShuffleQueryStage (49)
+ +- ColumnarExchange (48)
+ +- VeloxAppendBatches (47)
+ +- ^ ProjectExecTransformer (45)
+ +- ^ FlushableHashAggregateExecTransformer (44)
+ +- ^ ProjectExecTransformer (43)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (42)
+ :- ^ ProjectExecTransformer (29)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (28)
+ : :- ^ InputIteratorTransformer (7)
+ : : +- BroadcastQueryStage (5)
+ : : +- ColumnarBroadcastExchange (4)
+ : : +- ^ NoopFilter (2)
+ : : +- ^ Scan parquet (1)
+ : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (27)
+ : :- ^ NoopFilter (9)
+ : : +- ^ Scan parquet (8)
+ : +- ^ InputIteratorTransformer (26)
+ : +- BroadcastQueryStage (24)
+ : +- ColumnarBroadcastExchange (23)
+ : +- ^ ProjectExecTransformer (21)
+ : +- ^ FilterExecTransformer (20)
+ : +- ^ RegularHashAggregateExecTransformer (19)
+ : +- ^ InputIteratorTransformer (18)
+ : +- ShuffleQueryStage (16)
+ : +- ColumnarExchange (15)
+ : +- VeloxAppendBatches (14)
+ : +- ^ ProjectExecTransformer (12)
+ : +- ^ FlushableHashAggregateExecTransformer (11)
+ : +- ^ Scan parquet (10)
+ +- ^ InputIteratorTransformer (41)
+ +- BroadcastQueryStage (39)
+ +- ColumnarBroadcastExchange (38)
+ +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (36)
+ :- ^ NoopFilter (31)
+ : +- ^ Scan parquet (30)
+ +- ^ InputIteratorTransformer (35)
+ +- BroadcastQueryStage (33)
+ +- ReusedExchange (32)
+- == Initial Plan ==
- TakeOrderedAndProject (85)
- +- HashAggregate (84)
- +- Exchange (83)
- +- HashAggregate (82)
- +- Project (81)
- +- BroadcastHashJoin Inner BuildRight (80)
- :- Project (68)
- : +- BroadcastHashJoin Inner BuildLeft (67)
- : :- BroadcastExchange (56)
- : : +- Filter (55)
- : : +- Scan parquet (54)
- : +- BroadcastHashJoin LeftSemi BuildRight (66)
- : :- Filter (58)
- : : +- Scan parquet (57)
- : +- BroadcastExchange (65)
- : +- Project (64)
- : +- Filter (63)
- : +- HashAggregate (62)
- : +- Exchange (61)
- : +- HashAggregate (60)
- : +- Scan parquet (59)
- +- BroadcastExchange (79)
- +- BroadcastHashJoin LeftSemi BuildRight (78)
- :- Filter (70)
- : +- Scan parquet (69)
- +- BroadcastExchange (77)
- +- Project (76)
- +- Filter (75)
- +- HashAggregate (74)
- +- Exchange (73)
- +- HashAggregate (72)
- +- Scan parquet (71)
+ TakeOrderedAndProject (87)
+ +- HashAggregate (86)
+ +- Exchange (85)
+ +- HashAggregate (84)
+ +- Project (83)
+ +- BroadcastHashJoin Inner BuildRight (82)
+ :- Project (70)
+ : +- BroadcastHashJoin Inner BuildLeft (69)
+ : :- BroadcastExchange (58)
+ : : +- Filter (57)
+ : : +- Scan parquet (56)
+ : +- BroadcastHashJoin LeftSemi BuildRight (68)
+ : :- Filter (60)
+ : : +- Scan parquet (59)
+ : +- BroadcastExchange (67)
+ : +- Project (66)
+ : +- Filter (65)
+ : +- HashAggregate (64)
+ : +- Exchange (63)
+ : +- HashAggregate (62)
+ : +- Scan parquet (61)
+ +- BroadcastExchange (81)
+ +- BroadcastHashJoin LeftSemi BuildRight (80)
+ :- Filter (72)
+ : +- Scan parquet (71)
+ +- BroadcastExchange (79)
+ +- Project (78)
+ +- Filter (77)
+ +- HashAggregate (76)
+ +- Exchange (75)
+ +- HashAggregate (74)
+ +- Scan parquet (73)
(1) Scan parquet
@@ -138,333 +140,341 @@ Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
Input [4]: [hash_partition_key#X, l_orderkey#X, sum#X, isEmpty#X]
Arguments: false
-(14) ColumnarExchange
+(14) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, l_orderkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(15) ColumnarExchange
Input [4]: [hash_partition_key#X, l_orderkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_orderkey#X, 1), ENSURE_REQUIREMENTS, [l_orderkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(15) ShuffleQueryStage
+(16) ShuffleQueryStage
Output [3]: [l_orderkey#X, sum#X, isEmpty#X]
Arguments: X
-(16) InputAdapter
+(17) InputAdapter
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
-(17) InputIteratorTransformer
+(18) InputIteratorTransformer
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
-(18) RegularHashAggregateExecTransformer
+(19) RegularHashAggregateExecTransformer
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
Keys [1]: [l_orderkey#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [2]: [l_orderkey#X, sum(l_quantity#X)#X AS sum(l_quantity#X)#X]
-(19) FilterExecTransformer
+(20) FilterExecTransformer
Input [2]: [l_orderkey#X, sum(l_quantity#X)#X]
Arguments: (isnotnull(sum(l_quantity#X)#X) AND (sum(l_quantity#X)#X > 300.00))
-(20) ProjectExecTransformer
+(21) ProjectExecTransformer
Output [1]: [l_orderkey#X]
Input [2]: [l_orderkey#X, sum(l_quantity#X)#X]
-(21) WholeStageCodegenTransformer (X)
+(22) WholeStageCodegenTransformer (X)
Input [1]: [l_orderkey#X]
Arguments: false
-(22) ColumnarBroadcastExchange
+(23) ColumnarBroadcastExchange
Input [1]: [l_orderkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(23) BroadcastQueryStage
+(24) BroadcastQueryStage
Output [1]: [l_orderkey#X]
Arguments: X
-(24) InputAdapter
+(25) InputAdapter
Input [1]: [l_orderkey#X]
-(25) InputIteratorTransformer
+(26) InputIteratorTransformer
Input [1]: [l_orderkey#X]
-(26) BroadcastHashJoinExecTransformer
+(27) BroadcastHashJoinExecTransformer
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(27) BroadcastHashJoinExecTransformer
+(28) BroadcastHashJoinExecTransformer
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(28) ProjectExecTransformer
+(29) ProjectExecTransformer
Output [5]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X]
Input [6]: [c_custkey#X, c_name#X, o_orderkey#X, o_custkey#X, o_totalprice#X, o_orderdate#X]
-(29) Scan parquet
+(30) Scan parquet
Output [2]: [l_orderkey#X, l_quantity#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_quantity:decimal(12,2)>
-(30) NoopFilter
+(31) NoopFilter
Input [2]: [l_orderkey#X, l_quantity#X]
Arguments: [l_orderkey#X, l_quantity#X]
-(31) ReusedExchange [Reuses operator id: 22]
+(32) ReusedExchange [Reuses operator id: 23]
Output [1]: [l_orderkey#X]
-(32) BroadcastQueryStage
+(33) BroadcastQueryStage
Output [1]: [l_orderkey#X]
Arguments: X
-(33) InputAdapter
+(34) InputAdapter
Input [1]: [l_orderkey#X]
-(34) InputIteratorTransformer
+(35) InputIteratorTransformer
Input [1]: [l_orderkey#X]
-(35) BroadcastHashJoinExecTransformer
+(36) BroadcastHashJoinExecTransformer
Left keys [1]: [l_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(36) WholeStageCodegenTransformer (X)
+(37) WholeStageCodegenTransformer (X)
Input [2]: [l_orderkey#X, l_quantity#X]
Arguments: false
-(37) ColumnarBroadcastExchange
+(38) ColumnarBroadcastExchange
Input [2]: [l_orderkey#X, l_quantity#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(38) BroadcastQueryStage
+(39) BroadcastQueryStage
Output [2]: [l_orderkey#X, l_quantity#X]
Arguments: X
-(39) InputAdapter
+(40) InputAdapter
Input [2]: [l_orderkey#X, l_quantity#X]
-(40) InputIteratorTransformer
+(41) InputIteratorTransformer
Input [2]: [l_orderkey#X, l_quantity#X]
-(41) BroadcastHashJoinExecTransformer
+(42) BroadcastHashJoinExecTransformer
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(42) ProjectExecTransformer
+(43) ProjectExecTransformer
Output [6]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X, l_quantity#X]
Input [7]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X, l_orderkey#X, l_quantity#X]
-(43) FlushableHashAggregateExecTransformer
+(44) FlushableHashAggregateExecTransformer
Input [6]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X, l_quantity#X]
Keys [5]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X]
Functions [1]: [partial_sum(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
-(44) ProjectExecTransformer
+(45) ProjectExecTransformer
Output [8]: [hash(c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, 42) AS hash_partition_key#X, c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Input [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
-(45) WholeStageCodegenTransformer (X)
+(46) WholeStageCodegenTransformer (X)
Input [8]: [hash_partition_key#X, c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Arguments: false
-(46) ColumnarExchange
+(47) VeloxAppendBatches
+Input [8]: [hash_partition_key#X, c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
+Arguments: X
+
+(48) ColumnarExchange
Input [8]: [hash_partition_key#X, c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, 1), ENSURE_REQUIREMENTS, [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(47) ShuffleQueryStage
+(49) ShuffleQueryStage
Output [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Arguments: X
-(48) InputAdapter
+(50) InputAdapter
Input [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
-(49) InputIteratorTransformer
+(51) InputIteratorTransformer
Input [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
-(50) RegularHashAggregateExecTransformer
+(52) RegularHashAggregateExecTransformer
Input [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Keys [5]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity#X)#X AS sum(l_quantity)#X]
-(51) WholeStageCodegenTransformer (X)
+(53) WholeStageCodegenTransformer (X)
Input [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X]
Arguments: false
-(52) TakeOrderedAndProjectExecTransformer
+(54) TakeOrderedAndProjectExecTransformer
Input [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X]
Arguments: X, [o_totalprice#X DESC NULLS LAST, o_orderdate#X ASC NULLS FIRST], [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X], 0
-(53) VeloxColumnarToRowExec
+(55) VeloxColumnarToRowExec
Input [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X]
-(54) Scan parquet
+(56) Scan parquet
Output [2]: [c_custkey#X, c_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_custkey)]
ReadSchema: struct<c_custkey:bigint,c_name:string>
-(55) Filter
+(57) Filter
Input [2]: [c_custkey#X, c_name#X]
Condition : isnotnull(c_custkey#X)
-(56) BroadcastExchange
+(58) BroadcastExchange
Input [2]: [c_custkey#X, c_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(57) Scan parquet
+(59) Scan parquet
Output [4]: [o_orderkey#X, o_custkey#X, o_totalprice#X, o_orderdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_custkey), IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_totalprice:decimal(12,2),o_orderdate:date>
-(58) Filter
+(60) Filter
Input [4]: [o_orderkey#X, o_custkey#X, o_totalprice#X, o_orderdate#X]
Condition : (isnotnull(o_custkey#X) AND isnotnull(o_orderkey#X))
-(59) Scan parquet
+(61) Scan parquet
Output [2]: [l_orderkey#X, l_quantity#X]
Batched: true
Location: InMemoryFileIndex [*]
ReadSchema: struct<l_orderkey:bigint,l_quantity:decimal(12,2)>
-(60) HashAggregate
+(62) HashAggregate
Input [2]: [l_orderkey#X, l_quantity#X]
Keys [1]: [l_orderkey#X]
Functions [1]: [partial_sum(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [l_orderkey#X, sum#X, isEmpty#X]
-(61) Exchange
+(63) Exchange
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_orderkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(62) HashAggregate
+(64) HashAggregate
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
Keys [1]: [l_orderkey#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [2]: [l_orderkey#X, sum(l_quantity#X)#X AS sum(l_quantity#X)#X]
-(63) Filter
+(65) Filter
Input [2]: [l_orderkey#X, sum(l_quantity#X)#X]
Condition : (isnotnull(sum(l_quantity#X)#X) AND (sum(l_quantity#X)#X > 300.00))
-(64) Project
+(66) Project
Output [1]: [l_orderkey#X]
Input [2]: [l_orderkey#X, sum(l_quantity#X)#X]
-(65) BroadcastExchange
+(67) BroadcastExchange
Input [1]: [l_orderkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(66) BroadcastHashJoin
+(68) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(67) BroadcastHashJoin
+(69) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(68) Project
+(70) Project
Output [5]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X]
Input [6]: [c_custkey#X, c_name#X, o_orderkey#X, o_custkey#X, o_totalprice#X, o_orderdate#X]
-(69) Scan parquet
+(71) Scan parquet
Output [2]: [l_orderkey#X, l_quantity#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_quantity:decimal(12,2)>
-(70) Filter
+(72) Filter
Input [2]: [l_orderkey#X, l_quantity#X]
Condition : isnotnull(l_orderkey#X)
-(71) Scan parquet
+(73) Scan parquet
Output [2]: [l_orderkey#X, l_quantity#X]
Batched: true
Location: InMemoryFileIndex [*]
ReadSchema: struct<l_orderkey:bigint,l_quantity:decimal(12,2)>
-(72) HashAggregate
+(74) HashAggregate
Input [2]: [l_orderkey#X, l_quantity#X]
Keys [1]: [l_orderkey#X]
Functions [1]: [partial_sum(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [l_orderkey#X, sum#X, isEmpty#X]
-(73) Exchange
+(75) Exchange
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_orderkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(74) HashAggregate
+(76) HashAggregate
Input [3]: [l_orderkey#X, sum#X, isEmpty#X]
Keys [1]: [l_orderkey#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [2]: [l_orderkey#X, sum(l_quantity#X)#X AS sum(l_quantity#X)#X]
-(75) Filter
+(77) Filter
Input [2]: [l_orderkey#X, sum(l_quantity#X)#X]
Condition : (isnotnull(sum(l_quantity#X)#X) AND (sum(l_quantity#X)#X > 300.00))
-(76) Project
+(78) Project
Output [1]: [l_orderkey#X]
Input [2]: [l_orderkey#X, sum(l_quantity#X)#X]
-(77) BroadcastExchange
+(79) BroadcastExchange
Input [1]: [l_orderkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(78) BroadcastHashJoin
+(80) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(79) BroadcastExchange
+(81) BroadcastExchange
Input [2]: [l_orderkey#X, l_quantity#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(80) BroadcastHashJoin
+(82) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(81) Project
+(83) Project
Output [6]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X, l_quantity#X]
Input [7]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X, l_orderkey#X, l_quantity#X]
-(82) HashAggregate
+(84) HashAggregate
Input [6]: [c_custkey#X, c_name#X, o_orderkey#X, o_totalprice#X, o_orderdate#X, l_quantity#X]
Keys [5]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X]
Functions [1]: [partial_sum(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
-(83) Exchange
+(85) Exchange
Input [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(84) HashAggregate
+(86) HashAggregate
Input [7]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum#X, isEmpty#X]
Keys [5]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity#X)#X AS sum(l_quantity)#X]
-(85) TakeOrderedAndProject
+(87) TakeOrderedAndProject
Input [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X]
Arguments: X, [o_totalprice#X DESC NULLS LAST, o_orderdate#X ASC NULLS FIRST], [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X]
-(86) AdaptiveSparkPlan
+(88) AdaptiveSparkPlan
Output [6]: [c_name#X, c_custkey#X, o_orderkey#X, o_orderdate#X, o_totalprice#X, sum(l_quantity)#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/19.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/19.txt
index 72a9f74f8..d2ab9979f 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/19.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/19.txt
@@ -1,34 +1,35 @@
== Physical Plan ==
-AdaptiveSparkPlan (33)
+AdaptiveSparkPlan (34)
+- == Final Plan ==
- VeloxColumnarToRowExec (21)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16)
- +- ColumnarExchange (15)
- +- ^ FlushableHashAggregateExecTransformer (13)
- +- ^ ProjectExecTransformer (12)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
- :- ^ ProjectExecTransformer (3)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (10)
- +- BroadcastQueryStage (8)
- +- ColumnarBroadcastExchange (7)
- +- ^ NoopFilter (5)
- +- ^ Scan parquet (4)
+ VeloxColumnarToRowExec (22)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ FlushableHashAggregateExecTransformer (13)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
+ :- ^ ProjectExecTransformer (3)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (10)
+ +- BroadcastQueryStage (8)
+ +- ColumnarBroadcastExchange (7)
+ +- ^ NoopFilter (5)
+ +- ^ Scan parquet (4)
+- == Initial Plan ==
- HashAggregate (32)
- +- Exchange (31)
- +- HashAggregate (30)
- +- Project (29)
- +- BroadcastHashJoin Inner BuildRight (28)
- :- Project (24)
- : +- Filter (23)
- : +- Scan parquet (22)
- +- BroadcastExchange (27)
- +- Filter (26)
- +- Scan parquet (25)
+ HashAggregate (33)
+ +- Exchange (32)
+ +- HashAggregate (31)
+ +- Project (30)
+ +- BroadcastHashJoin Inner BuildRight (29)
+ :- Project (25)
+ : +- Filter (24)
+ : +- Scan parquet (23)
+ +- BroadcastExchange (28)
+ +- Filter (27)
+ +- Scan parquet (26)
(1) Scan parquet
@@ -95,91 +96,95 @@ Results [2]: [sum#X, isEmpty#X]
Input [2]: [sum#X, isEmpty#X]
Arguments: false
-(15) ColumnarExchange
+(15) VeloxAppendBatches
+Input [2]: [sum#X, isEmpty#X]
+Arguments: X
+
+(16) ColumnarExchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(16) ShuffleQueryStage
+(17) ShuffleQueryStage
Output [2]: [sum#X, isEmpty#X]
Arguments: X
-(17) InputAdapter
+(18) InputAdapter
Input [2]: [sum#X, isEmpty#X]
-(18) InputIteratorTransformer
+(19) InputIteratorTransformer
Input [2]: [sum#X, isEmpty#X]
-(19) RegularHashAggregateExecTransformer
+(20) RegularHashAggregateExecTransformer
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X]
-(20) WholeStageCodegenTransformer (X)
+(21) WholeStageCodegenTransformer (X)
Input [1]: [revenue#X]
Arguments: false
-(21) VeloxColumnarToRowExec
+(22) VeloxColumnarToRowExec
Input [1]: [revenue#X]
-(22) Scan parquet
+(23) Scan parquet
Output [6]: [l_partkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, l_shipinstruct#X, l_shipmode#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipinstruct), In(l_shipmode, [AIR,AIR REG]), EqualTo(l_shipinstruct,DELIVER IN PERSON), IsNotNull(l_partkey), Or(Or(And(GreaterThanOrEqual(l_quantity,1.00),LessThanOrEqual(l_quantity,11.00)),And(GreaterThanOrEqual(l_quantity,10.00),LessThanOrEqual(l_quantity,20.00))),And(GreaterThanOrEqual(l_quantity,20.00),LessThanOrEqual(l_quantity,30.00)))]
ReadSchema: struct<l_partkey:bigint,l_quantity:decimal(12,2),l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipinstruct:string,l_shipmode:string>
-(23) Filter
+(24) Filter
Input [6]: [l_partkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, l_shipinstruct#X, l_shipmode#X]
Condition : ((((isnotnull(l_shipinstruct#X) AND l_shipmode#X IN (AIR,AIR REG)) AND (l_shipinstruct#X = DELIVER IN PERSON)) AND isnotnull(l_partkey#X)) AND ((((l_quantity#X >= 1.00) AND (l_quantity#X <= 11.00)) OR ((l_quantity#X >= 10.00) AND (l_quantity#X <= 20.00))) OR ((l_quantity#X >= 20.00) AND (l_quantity#X <= 30.00))))
-(24) Project
+(25) Project
Output [4]: [l_partkey#X, l_quantity#X, l_extendedprice#X, l_discount#X]
Input [6]: [l_partkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, l_shipinstruct#X, l_shipmode#X]
-(25) Scan parquet
+(26) Scan parquet
Output [4]: [p_partkey#X, p_brand#X, p_size#X, p_container#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_size), GreaterThanOrEqual(p_size,1), IsNotNull(p_partkey), Or(Or(And(And(EqualTo(p_brand,Brand#X),In(p_container, [SM BOX,SM CASE,SM PACK,SM PKG])),LessThanOrEqual(p_size,5)),And(And(EqualTo(p_brand,Brand#X),In(p_container, [MED BAG,MED BOX,MED PACK,MED PKG])),LessThanOrEqual(p_size,10))),And(And(EqualTo(p_brand,Brand#X),In(p_container, [LG BOX,LG CASE,LG PACK,LG PKG])),LessThanOrEqual(p_size,15)))]
ReadSchema: struct<p_partkey:bigint,p_brand:string,p_size:int,p_container:string>
-(26) Filter
+(27) Filter
Input [4]: [p_partkey#X, p_brand#X, p_size#X, p_container#X]
Condition : (((isnotnull(p_size#X) AND (p_size#X >= 1)) AND isnotnull(p_partkey#X)) AND (((((p_brand#X = Brand#X) AND p_container#X IN (SM CASE,SM BOX,SM PACK,SM PKG)) AND (p_size#X <= 5)) OR (((p_brand#X = Brand#X) AND p_container#X IN (MED BAG,MED BOX,MED PKG,MED PACK)) AND (p_size#X <= 10))) OR (((p_brand#X = Brand#X) AND p_container#X IN (LG CASE,LG BOX,LG PACK,LG PKG)) AND (p_size#X <= 15))))
-(27) BroadcastExchange
+(28) BroadcastExchange
Input [4]: [p_partkey#X, p_brand#X, p_size#X, p_container#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(28) BroadcastHashJoin
+(29) BroadcastHashJoin
Left keys [1]: [l_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: (((((((p_brand#X = Brand#X) AND p_container#X IN (SM CASE,SM BOX,SM PACK,SM PKG)) AND (l_quantity#X >= 1.00)) AND (l_quantity#X <= 11.00)) AND (p_size#X <= 5)) OR (((((p_brand#X = Brand#X) AND p_container#X IN (MED BAG,MED BOX,MED PKG,MED PACK)) AND (l_quantity#X >= 10.00)) AND (l_quantity#X <= 20.00)) AND (p_size#X <= 10))) OR (((((p_brand#X = Brand#X) AND p_container#X IN (LG CASE,LG BOX,LG PACK,LG PKG)) AND (l_quantity#X >= 20.00)) AND (l_quantity#X <= 30.00)) AND (p_s [...]
-(29) Project
+(30) Project
Output [2]: [l_extendedprice#X, l_discount#X]
Input [8]: [l_partkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, p_partkey#X, p_brand#X, p_size#X, p_container#X]
-(30) HashAggregate
+(31) HashAggregate
Input [2]: [l_extendedprice#X, l_discount#X]
Keys: []
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [2]: [sum#X, isEmpty#X]
-(31) Exchange
+(32) Exchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X]
-(32) HashAggregate
+(33) HashAggregate
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X]
-(33) AdaptiveSparkPlan
+(34) AdaptiveSparkPlan
Output [1]: [revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/20.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/20.txt
index e09911ea6..6ebe36be3 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/20.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/20.txt
@@ -1,97 +1,100 @@
== Physical Plan ==
-AdaptiveSparkPlan (107)
+AdaptiveSparkPlan (110)
+- == Final Plan ==
- VeloxColumnarToRowExec (70)
- +- ^ SortExecTransformer (68)
- +- ^ InputIteratorTransformer (67)
- +- ShuffleQueryStage (65)
- +- ColumnarExchange (64)
- +- ^ ProjectExecTransformer (62)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (61)
- :- ^ ProjectExecTransformer (52)
- : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (51)
- : :- ^ InputIteratorTransformer (9)
- : : +- AQEShuffleRead (7)
- : : +- ShuffleQueryStage (6)
- : : +- ColumnarExchange (5)
- : : +- ^ ProjectExecTransformer (3)
- : : +- ^ NoopFilter (2)
- : : +- ^ Scan parquet (1)
- : +- ^ InputIteratorTransformer (50)
- : +- BroadcastQueryStage (48)
- : +- ColumnarBroadcastExchange (47)
- : +- ^ ProjectExecTransformer (45)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (44)
- : :- ^ InputIteratorTransformer (25)
- : : +- BroadcastQueryStage (23)
- : : +- ColumnarBroadcastExchange (22)
- : : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (20)
- : : :- ^ NoopFilter (11)
- : : : +- ^ Scan parquet (10)
- : : +- ^ InputIteratorTransformer (19)
- : : +- BroadcastQueryStage (17)
- : : +- ColumnarBroadcastExchange (16)
- : : +- ^ ProjectExecTransformer (14)
- : : +- ^ NoopFilter (13)
- : : +- ^ Scan parquet (12)
- : +- ^ FilterExecTransformer (43)
- : +- ^ ProjectExecTransformer (42)
- : +- ^ RegularHashAggregateExecTransformer (41)
- : +- ^ InputIteratorTransformer (40)
- : +- ShuffleQueryStage (38)
- : +- ColumnarExchange (37)
- : +- ^ ProjectExecTransformer (35)
- : +- ^ FlushableHashAggregateExecTransformer (34)
- : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (33)
- : :- ^ ProjectExecTransformer (28)
- : : +- ^ NoopFilter (27)
- : : +- ^ Scan parquet (26)
- : +- ^ InputIteratorTransformer (32)
- : +- BroadcastQueryStage (30)
- : +- ReusedExchange (29)
- +- ^ InputIteratorTransformer (60)
- +- BroadcastQueryStage (58)
- +- ColumnarBroadcastExchange (57)
- +- ^ ProjectExecTransformer (55)
- +- ^ NoopFilter (54)
- +- ^ Scan parquet (53)
+ VeloxColumnarToRowExec (73)
+ +- ^ SortExecTransformer (71)
+ +- ^ InputIteratorTransformer (70)
+ +- ShuffleQueryStage (68)
+ +- ColumnarExchange (67)
+ +- VeloxAppendBatches (66)
+ +- ^ ProjectExecTransformer (64)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (63)
+ :- ^ ProjectExecTransformer (54)
+ : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (53)
+ : :- ^ InputIteratorTransformer (10)
+ : : +- AQEShuffleRead (8)
+ : : +- ShuffleQueryStage (7)
+ : : +- ColumnarExchange (6)
+ : : +- VeloxAppendBatches (5)
+ : : +- ^ ProjectExecTransformer (3)
+ : : +- ^ NoopFilter (2)
+ : : +- ^ Scan parquet (1)
+ : +- ^ InputIteratorTransformer (52)
+ : +- BroadcastQueryStage (50)
+ : +- ColumnarBroadcastExchange (49)
+ : +- ^ ProjectExecTransformer (47)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (46)
+ : :- ^ InputIteratorTransformer (26)
+ : : +- BroadcastQueryStage (24)
+ : : +- ColumnarBroadcastExchange (23)
+ : : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (21)
+ : : :- ^ NoopFilter (12)
+ : : : +- ^ Scan parquet (11)
+ : : +- ^ InputIteratorTransformer (20)
+ : : +- BroadcastQueryStage (18)
+ : : +- ColumnarBroadcastExchange (17)
+ : : +- ^ ProjectExecTransformer (15)
+ : : +- ^ NoopFilter (14)
+ : : +- ^ Scan parquet (13)
+ : +- ^ FilterExecTransformer (45)
+ : +- ^ ProjectExecTransformer (44)
+ : +- ^ RegularHashAggregateExecTransformer (43)
+ : +- ^ InputIteratorTransformer (42)
+ : +- ShuffleQueryStage (40)
+ : +- ColumnarExchange (39)
+ : +- VeloxAppendBatches (38)
+ : +- ^ ProjectExecTransformer (36)
+ : +- ^ FlushableHashAggregateExecTransformer (35)
+ : +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (34)
+ : :- ^ ProjectExecTransformer (29)
+ : : +- ^ NoopFilter (28)
+ : : +- ^ Scan parquet (27)
+ : +- ^ InputIteratorTransformer (33)
+ : +- BroadcastQueryStage (31)
+ : +- ReusedExchange (30)
+ +- ^ InputIteratorTransformer (62)
+ +- BroadcastQueryStage (60)
+ +- ColumnarBroadcastExchange (59)
+ +- ^ ProjectExecTransformer (57)
+ +- ^ NoopFilter (56)
+ +- ^ Scan parquet (55)
+- == Initial Plan ==
- Sort (106)
- +- Exchange (105)
- +- Project (104)
- +- BroadcastHashJoin Inner BuildRight (103)
- :- Project (98)
- : +- ShuffledHashJoin LeftSemi BuildRight (97)
- : :- Exchange (73)
- : : +- Filter (72)
- : : +- Scan parquet (71)
- : +- Exchange (96)
- : +- Project (95)
- : +- BroadcastHashJoin Inner BuildLeft (94)
- : :- BroadcastExchange (81)
- : : +- BroadcastHashJoin LeftSemi BuildRight (80)
- : : :- Filter (75)
- : : : +- Scan parquet (74)
- : : +- BroadcastExchange (79)
- : : +- Project (78)
- : : +- Filter (77)
- : : +- Scan parquet (76)
- : +- Filter (93)
- : +- HashAggregate (92)
- : +- Exchange (91)
- : +- HashAggregate (90)
- : +- BroadcastHashJoin LeftSemi BuildRight (89)
- : :- Project (84)
- : : +- Filter (83)
- : : +- Scan parquet (82)
- : +- BroadcastExchange (88)
- : +- Project (87)
- : +- Filter (86)
- : +- Scan parquet (85)
- +- BroadcastExchange (102)
- +- Project (101)
- +- Filter (100)
- +- Scan parquet (99)
+ Sort (109)
+ +- Exchange (108)
+ +- Project (107)
+ +- BroadcastHashJoin Inner BuildRight (106)
+ :- Project (101)
+ : +- ShuffledHashJoin LeftSemi BuildRight (100)
+ : :- Exchange (76)
+ : : +- Filter (75)
+ : : +- Scan parquet (74)
+ : +- Exchange (99)
+ : +- Project (98)
+ : +- BroadcastHashJoin Inner BuildLeft (97)
+ : :- BroadcastExchange (84)
+ : : +- BroadcastHashJoin LeftSemi BuildRight (83)
+ : : :- Filter (78)
+ : : : +- Scan parquet (77)
+ : : +- BroadcastExchange (82)
+ : : +- Project (81)
+ : : +- Filter (80)
+ : : +- Scan parquet (79)
+ : +- Filter (96)
+ : +- HashAggregate (95)
+ : +- Exchange (94)
+ : +- HashAggregate (93)
+ : +- BroadcastHashJoin LeftSemi BuildRight (92)
+ : :- Project (87)
+ : : +- Filter (86)
+ : : +- Scan parquet (85)
+ : +- BroadcastExchange (91)
+ : +- Project (90)
+ : +- Filter (89)
+ : +- Scan parquet (88)
+ +- BroadcastExchange (105)
+ +- Project (104)
+ +- Filter (103)
+ +- Scan parquet (102)
(1) Scan parquet
@@ -113,448 +116,460 @@ Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Input [5]: [hash_partition_key#X, s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Arguments: false
-(5) ColumnarExchange
+(5) VeloxAppendBatches
+Input [5]: [hash_partition_key#X, s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
+Arguments: X
+
+(6) ColumnarExchange
Input [5]: [hash_partition_key#X, s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Arguments: hashpartitioning(s_suppkey#X, 1), ENSURE_REQUIREMENTS, [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X], [plan_id=X], [id=#X]
-(6) ShuffleQueryStage
+(7) ShuffleQueryStage
Output [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Arguments: X
-(7) AQEShuffleRead
+(8) AQEShuffleRead
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Arguments: local
-(8) InputAdapter
+(9) InputAdapter
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
-(9) InputIteratorTransformer
+(10) InputIteratorTransformer
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
-(10) Scan parquet
+(11) Scan parquet
Output [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_availqty), IsNotNull(ps_partkey), IsNotNull(ps_suppkey)]
ReadSchema: struct<ps_partkey:bigint,ps_suppkey:bigint,ps_availqty:int>
-(11) NoopFilter
+(12) NoopFilter
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Arguments: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
-(12) Scan parquet
+(13) Scan parquet
Output [2]: [p_partkey#X, p_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_name), StringStartsWith(p_name,forest)]
ReadSchema: struct<p_partkey:bigint,p_name:string>
-(13) NoopFilter
+(14) NoopFilter
Input [2]: [p_partkey#X, p_name#X]
Arguments: [p_partkey#X, p_name#X]
-(14) ProjectExecTransformer
+(15) ProjectExecTransformer
Output [1]: [p_partkey#X]
Input [2]: [p_partkey#X, p_name#X]
-(15) WholeStageCodegenTransformer (X)
+(16) WholeStageCodegenTransformer (X)
Input [1]: [p_partkey#X]
Arguments: false
-(16) ColumnarBroadcastExchange
+(17) ColumnarBroadcastExchange
Input [1]: [p_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(17) BroadcastQueryStage
+(18) BroadcastQueryStage
Output [1]: [p_partkey#X]
Arguments: X
-(18) InputAdapter
+(19) InputAdapter
Input [1]: [p_partkey#X]
-(19) InputIteratorTransformer
+(20) InputIteratorTransformer
Input [1]: [p_partkey#X]
-(20) BroadcastHashJoinExecTransformer
+(21) BroadcastHashJoinExecTransformer
Left keys [1]: [ps_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(21) WholeStageCodegenTransformer (X)
+(22) WholeStageCodegenTransformer (X)
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Arguments: false
-(22) ColumnarBroadcastExchange
+(23) ColumnarBroadcastExchange
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false], input[1, bigint, false]),false), [plan_id=X]
-(23) BroadcastQueryStage
+(24) BroadcastQueryStage
Output [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Arguments: X
-(24) InputAdapter
+(25) InputAdapter
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
-(25) InputIteratorTransformer
+(26) InputIteratorTransformer
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
-(26) Scan parquet
+(27) Scan parquet
Output [4]: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1994-01-01), LessThan(l_shipdate,1995-01-01), IsNotNull(l_partkey), IsNotNull(l_suppkey)]
ReadSchema: struct<l_partkey:bigint,l_suppkey:bigint,l_quantity:decimal(12,2),l_shipdate:date>
-(27) NoopFilter
+(28) NoopFilter
Input [4]: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
Arguments: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
-(28) ProjectExecTransformer
+(29) ProjectExecTransformer
Output [3]: [l_partkey#X, l_suppkey#X, l_quantity#X]
Input [4]: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
-(29) ReusedExchange [Reuses operator id: 16]
+(30) ReusedExchange [Reuses operator id: 17]
Output [1]: [p_partkey#X]
-(30) BroadcastQueryStage
+(31) BroadcastQueryStage
Output [1]: [p_partkey#X]
Arguments: X
-(31) InputAdapter
+(32) InputAdapter
Input [1]: [p_partkey#X]
-(32) InputIteratorTransformer
+(33) InputIteratorTransformer
Input [1]: [p_partkey#X]
-(33) BroadcastHashJoinExecTransformer
+(34) BroadcastHashJoinExecTransformer
Left keys [1]: [l_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(34) FlushableHashAggregateExecTransformer
+(35) FlushableHashAggregateExecTransformer
Input [3]: [l_partkey#X, l_suppkey#X, l_quantity#X]
Keys [2]: [l_partkey#X, l_suppkey#X]
Functions [1]: [partial_sum(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
-(35) ProjectExecTransformer
+(36) ProjectExecTransformer
Output [5]: [hash(l_partkey#X, l_suppkey#X, 42) AS hash_partition_key#X, l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Input [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
-(36) WholeStageCodegenTransformer (X)
+(37) WholeStageCodegenTransformer (X)
Input [5]: [hash_partition_key#X, l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: false
-(37) ColumnarExchange
+(38) VeloxAppendBatches
+Input [5]: [hash_partition_key#X, l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(39) ColumnarExchange
Input [5]: [hash_partition_key#X, l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_partkey#X, l_suppkey#X, 1), ENSURE_REQUIREMENTS, [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(38) ShuffleQueryStage
+(40) ShuffleQueryStage
Output [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: X
-(39) InputAdapter
+(41) InputAdapter
Input [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
-(40) InputIteratorTransformer
+(42) InputIteratorTransformer
Input [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
-(41) RegularHashAggregateExecTransformer
+(43) RegularHashAggregateExecTransformer
Input [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Keys [2]: [l_partkey#X, l_suppkey#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [3]: [l_partkey#X, l_suppkey#X, sum(l_quantity#X)#X]
-(42) ProjectExecTransformer
+(44) ProjectExecTransformer
Output [3]: [CheckOverflow((0.50 * promote_precision(sum(l_quantity#X)#X)), DecimalType(24,3), true) AS (0.5 * sum(l_quantity))#X, l_partkey#X, l_suppkey#X]
Input [3]: [l_partkey#X, l_suppkey#X, sum(l_quantity#X)#X]
-(43) FilterExecTransformer
+(45) FilterExecTransformer
Input [3]: [(0.5 * sum(l_quantity))#X, l_partkey#X, l_suppkey#X]
Arguments: isnotnull((0.5 * sum(l_quantity))#X)
-(44) BroadcastHashJoinExecTransformer
+(46) BroadcastHashJoinExecTransformer
Left keys [2]: [ps_partkey#X, ps_suppkey#X]
Right keys [2]: [l_partkey#X, l_suppkey#X]
Join condition: (cast(cast(ps_availqty#X as decimal(10,0)) as decimal(24,3)) > (0.5 * sum(l_quantity))#X)
-(45) ProjectExecTransformer
+(47) ProjectExecTransformer
Output [1]: [ps_suppkey#X]
Input [6]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, (0.5 * sum(l_quantity))#X, l_partkey#X, l_suppkey#X]
-(46) WholeStageCodegenTransformer (X)
+(48) WholeStageCodegenTransformer (X)
Input [1]: [ps_suppkey#X]
Arguments: false
-(47) ColumnarBroadcastExchange
+(49) ColumnarBroadcastExchange
Input [1]: [ps_suppkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(48) BroadcastQueryStage
+(50) BroadcastQueryStage
Output [1]: [ps_suppkey#X]
Arguments: X
-(49) InputAdapter
+(51) InputAdapter
Input [1]: [ps_suppkey#X]
-(50) InputIteratorTransformer
+(52) InputIteratorTransformer
Input [1]: [ps_suppkey#X]
-(51) BroadcastHashJoinExecTransformer
+(53) BroadcastHashJoinExecTransformer
Left keys [1]: [s_suppkey#X]
Right keys [1]: [ps_suppkey#X]
Join condition: None
-(52) ProjectExecTransformer
+(54) ProjectExecTransformer
Output [3]: [s_name#X, s_address#X, s_nationkey#X]
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
-(53) Scan parquet
+(55) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_name), EqualTo(n_name,CANADA), IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(54) NoopFilter
+(56) NoopFilter
Input [2]: [n_nationkey#X, n_name#X]
Arguments: [n_nationkey#X, n_name#X]
-(55) ProjectExecTransformer
+(57) ProjectExecTransformer
Output [1]: [n_nationkey#X]
Input [2]: [n_nationkey#X, n_name#X]
-(56) WholeStageCodegenTransformer (X)
+(58) WholeStageCodegenTransformer (X)
Input [1]: [n_nationkey#X]
Arguments: false
-(57) ColumnarBroadcastExchange
+(59) ColumnarBroadcastExchange
Input [1]: [n_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(58) BroadcastQueryStage
+(60) BroadcastQueryStage
Output [1]: [n_nationkey#X]
Arguments: X
-(59) InputAdapter
+(61) InputAdapter
Input [1]: [n_nationkey#X]
-(60) InputIteratorTransformer
+(62) InputIteratorTransformer
Input [1]: [n_nationkey#X]
-(61) BroadcastHashJoinExecTransformer
+(63) BroadcastHashJoinExecTransformer
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(62) ProjectExecTransformer
+(64) ProjectExecTransformer
Output [2]: [s_name#X, s_address#X]
Input [4]: [s_name#X, s_address#X, s_nationkey#X, n_nationkey#X]
-(63) WholeStageCodegenTransformer (X)
+(65) WholeStageCodegenTransformer (X)
Input [2]: [s_name#X, s_address#X]
Arguments: false
-(64) ColumnarExchange
+(66) VeloxAppendBatches
+Input [2]: [s_name#X, s_address#X]
+Arguments: X
+
+(67) ColumnarExchange
Input [2]: [s_name#X, s_address#X]
Arguments: rangepartitioning(s_name#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(65) ShuffleQueryStage
+(68) ShuffleQueryStage
Output [2]: [s_name#X, s_address#X]
Arguments: X
-(66) InputAdapter
+(69) InputAdapter
Input [2]: [s_name#X, s_address#X]
-(67) InputIteratorTransformer
+(70) InputIteratorTransformer
Input [2]: [s_name#X, s_address#X]
-(68) SortExecTransformer
+(71) SortExecTransformer
Input [2]: [s_name#X, s_address#X]
Arguments: [s_name#X ASC NULLS FIRST], true, 0
-(69) WholeStageCodegenTransformer (X)
+(72) WholeStageCodegenTransformer (X)
Input [2]: [s_name#X, s_address#X]
Arguments: false
-(70) VeloxColumnarToRowExec
+(73) VeloxColumnarToRowExec
Input [2]: [s_name#X, s_address#X]
-(71) Scan parquet
+(74) Scan parquet
Output [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_name:string,s_address:string,s_nationkey:bigint>
-(72) Filter
+(75) Filter
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Condition : isnotnull(s_nationkey#X)
-(73) Exchange
+(76) Exchange
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
Arguments: hashpartitioning(s_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(74) Scan parquet
+(77) Scan parquet
Output [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_availqty), IsNotNull(ps_partkey), IsNotNull(ps_suppkey)]
ReadSchema: struct<ps_partkey:bigint,ps_suppkey:bigint,ps_availqty:int>
-(75) Filter
+(78) Filter
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Condition : ((isnotnull(ps_availqty#X) AND isnotnull(ps_partkey#X)) AND isnotnull(ps_suppkey#X))
-(76) Scan parquet
+(79) Scan parquet
Output [2]: [p_partkey#X, p_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_name), StringStartsWith(p_name,forest)]
ReadSchema: struct<p_partkey:bigint,p_name:string>
-(77) Filter
+(80) Filter
Input [2]: [p_partkey#X, p_name#X]
Condition : (isnotnull(p_name#X) AND StartsWith(p_name#X, forest))
-(78) Project
+(81) Project
Output [1]: [p_partkey#X]
Input [2]: [p_partkey#X, p_name#X]
-(79) BroadcastExchange
+(82) BroadcastExchange
Input [1]: [p_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(80) BroadcastHashJoin
+(83) BroadcastHashJoin
Left keys [1]: [ps_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(81) BroadcastExchange
+(84) BroadcastExchange
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false], input[1, bigint, false]),false), [plan_id=X]
-(82) Scan parquet
+(85) Scan parquet
Output [4]: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1994-01-01), LessThan(l_shipdate,1995-01-01), IsNotNull(l_partkey), IsNotNull(l_suppkey)]
ReadSchema: struct<l_partkey:bigint,l_suppkey:bigint,l_quantity:decimal(12,2),l_shipdate:date>
-(83) Filter
+(86) Filter
Input [4]: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
Condition : ((((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1994-01-01)) AND (l_shipdate#X < 1995-01-01)) AND isnotnull(l_partkey#X)) AND isnotnull(l_suppkey#X))
-(84) Project
+(87) Project
Output [3]: [l_partkey#X, l_suppkey#X, l_quantity#X]
Input [4]: [l_partkey#X, l_suppkey#X, l_quantity#X, l_shipdate#X]
-(85) Scan parquet
+(88) Scan parquet
Output [2]: [p_partkey#X, p_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_name), StringStartsWith(p_name,forest)]
ReadSchema: struct<p_partkey:bigint,p_name:string>
-(86) Filter
+(89) Filter
Input [2]: [p_partkey#X, p_name#X]
Condition : (isnotnull(p_name#X) AND StartsWith(p_name#X, forest))
-(87) Project
+(90) Project
Output [1]: [p_partkey#X]
Input [2]: [p_partkey#X, p_name#X]
-(88) BroadcastExchange
+(91) BroadcastExchange
Input [1]: [p_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(89) BroadcastHashJoin
+(92) BroadcastHashJoin
Left keys [1]: [l_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(90) HashAggregate
+(93) HashAggregate
Input [3]: [l_partkey#X, l_suppkey#X, l_quantity#X]
Keys [2]: [l_partkey#X, l_suppkey#X]
Functions [1]: [partial_sum(l_quantity#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
-(91) Exchange
+(94) Exchange
Input [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_partkey#X, l_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(92) HashAggregate
+(95) HashAggregate
Input [4]: [l_partkey#X, l_suppkey#X, sum#X, isEmpty#X]
Keys [2]: [l_partkey#X, l_suppkey#X]
Functions [1]: [sum(l_quantity#X)]
Aggregate Attributes [1]: [sum(l_quantity#X)#X]
Results [3]: [CheckOverflow((0.50 * promote_precision(sum(l_quantity#X)#X)), DecimalType(24,3), true) AS (0.5 * sum(l_quantity))#X, l_partkey#X, l_suppkey#X]
-(93) Filter
+(96) Filter
Input [3]: [(0.5 * sum(l_quantity))#X, l_partkey#X, l_suppkey#X]
Condition : isnotnull((0.5 * sum(l_quantity))#X)
-(94) BroadcastHashJoin
+(97) BroadcastHashJoin
Left keys [2]: [ps_partkey#X, ps_suppkey#X]
Right keys [2]: [l_partkey#X, l_suppkey#X]
Join condition: (cast(cast(ps_availqty#X as decimal(10,0)) as decimal(24,3)) > (0.5 * sum(l_quantity))#X)
-(95) Project
+(98) Project
Output [1]: [ps_suppkey#X]
Input [6]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, (0.5 * sum(l_quantity))#X, l_partkey#X, l_suppkey#X]
-(96) Exchange
+(99) Exchange
Input [1]: [ps_suppkey#X]
Arguments: hashpartitioning(ps_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(97) ShuffledHashJoin
+(100) ShuffledHashJoin
Left keys [1]: [s_suppkey#X]
Right keys [1]: [ps_suppkey#X]
Join condition: None
-(98) Project
+(101) Project
Output [3]: [s_name#X, s_address#X, s_nationkey#X]
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_nationkey#X]
-(99) Scan parquet
+(102) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_name), EqualTo(n_name,CANADA), IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(100) Filter
+(103) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : ((isnotnull(n_name#X) AND (n_name#X = CANADA)) AND isnotnull(n_nationkey#X))
-(101) Project
+(104) Project
Output [1]: [n_nationkey#X]
Input [2]: [n_nationkey#X, n_name#X]
-(102) BroadcastExchange
+(105) BroadcastExchange
Input [1]: [n_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(103) BroadcastHashJoin
+(106) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(104) Project
+(107) Project
Output [2]: [s_name#X, s_address#X]
Input [4]: [s_name#X, s_address#X, s_nationkey#X, n_nationkey#X]
-(105) Exchange
+(108) Exchange
Input [2]: [s_name#X, s_address#X]
Arguments: rangepartitioning(s_name#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(106) Sort
+(109) Sort
Input [2]: [s_name#X, s_address#X]
Arguments: [s_name#X ASC NULLS FIRST], true, 0
-(107) AdaptiveSparkPlan
+(110) AdaptiveSparkPlan
Output [2]: [s_name#X, s_address#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/21.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/21.txt
index 10e08cfd6..eb3f64d3e 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/21.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/21.txt
@@ -1,85 +1,86 @@
== Physical Plan ==
-AdaptiveSparkPlan (92)
+AdaptiveSparkPlan (93)
+- == Final Plan ==
- VeloxColumnarToRowExec (59)
- +- TakeOrderedAndProjectExecTransformer (58)
- +- ^ RegularHashAggregateExecTransformer (56)
- +- ^ InputIteratorTransformer (55)
- +- ShuffleQueryStage (53)
- +- ColumnarExchange (52)
- +- ^ ProjectExecTransformer (50)
- +- ^ FlushableHashAggregateExecTransformer (49)
- +- ^ ProjectExecTransformer (48)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (47)
- :- ^ ProjectExecTransformer (38)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (37)
- : :- ^ ProjectExecTransformer (28)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (27)
- : : :- ^ InputIteratorTransformer (7)
- : : : +- BroadcastQueryStage (5)
- : : : +- ColumnarBroadcastExchange (4)
- : : : +- ^ NoopFilter (2)
- : : : +- ^ Scan parquet (1)
- : : +- ^ BroadcastHashJoinExecTransformer LeftAnti BuildRight (26)
- : : :- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (17)
- : : : :- ^ ProjectExecTransformer (10)
- : : : : +- ^ NoopFilter (9)
- : : : : +- ^ Scan parquet (8)
- : : : +- ^ InputIteratorTransformer (16)
- : : : +- BroadcastQueryStage (14)
- : : : +- ColumnarBroadcastExchange (13)
- : : : +- ^ Scan parquet (11)
- : : +- ^ InputIteratorTransformer (25)
- : : +- BroadcastQueryStage (23)
- : : +- ColumnarBroadcastExchange (22)
- : : +- ^ ProjectExecTransformer (20)
- : : +- ^ NoopFilter (19)
- : : +- ^ Scan parquet (18)
- : +- ^ InputIteratorTransformer (36)
- : +- BroadcastQueryStage (34)
- : +- ColumnarBroadcastExchange (33)
- : +- ^ ProjectExecTransformer (31)
- : +- ^ NoopFilter (30)
- : +- ^ Scan parquet (29)
- +- ^ InputIteratorTransformer (46)
- +- BroadcastQueryStage (44)
- +- ColumnarBroadcastExchange (43)
- +- ^ ProjectExecTransformer (41)
- +- ^ NoopFilter (40)
- +- ^ Scan parquet (39)
+ VeloxColumnarToRowExec (60)
+ +- TakeOrderedAndProjectExecTransformer (59)
+ +- ^ RegularHashAggregateExecTransformer (57)
+ +- ^ InputIteratorTransformer (56)
+ +- ShuffleQueryStage (54)
+ +- ColumnarExchange (53)
+ +- VeloxAppendBatches (52)
+ +- ^ ProjectExecTransformer (50)
+ +- ^ FlushableHashAggregateExecTransformer (49)
+ +- ^ ProjectExecTransformer (48)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (47)
+ :- ^ ProjectExecTransformer (38)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (37)
+ : :- ^ ProjectExecTransformer (28)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (27)
+ : : :- ^ InputIteratorTransformer (7)
+ : : : +- BroadcastQueryStage (5)
+ : : : +- ColumnarBroadcastExchange (4)
+ : : : +- ^ NoopFilter (2)
+ : : : +- ^ Scan parquet (1)
+ : : +- ^ BroadcastHashJoinExecTransformer LeftAnti BuildRight (26)
+ : : :- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (17)
+ : : : :- ^ ProjectExecTransformer (10)
+ : : : : +- ^ NoopFilter (9)
+ : : : : +- ^ Scan parquet (8)
+ : : : +- ^ InputIteratorTransformer (16)
+ : : : +- BroadcastQueryStage (14)
+ : : : +- ColumnarBroadcastExchange (13)
+ : : : +- ^ Scan parquet (11)
+ : : +- ^ InputIteratorTransformer (25)
+ : : +- BroadcastQueryStage (23)
+ : : +- ColumnarBroadcastExchange (22)
+ : : +- ^ ProjectExecTransformer (20)
+ : : +- ^ NoopFilter (19)
+ : : +- ^ Scan parquet (18)
+ : +- ^ InputIteratorTransformer (36)
+ : +- BroadcastQueryStage (34)
+ : +- ColumnarBroadcastExchange (33)
+ : +- ^ ProjectExecTransformer (31)
+ : +- ^ NoopFilter (30)
+ : +- ^ Scan parquet (29)
+ +- ^ InputIteratorTransformer (46)
+ +- BroadcastQueryStage (44)
+ +- ColumnarBroadcastExchange (43)
+ +- ^ ProjectExecTransformer (41)
+ +- ^ NoopFilter (40)
+ +- ^ Scan parquet (39)
+- == Initial Plan ==
- TakeOrderedAndProject (91)
- +- HashAggregate (90)
- +- Exchange (89)
- +- HashAggregate (88)
- +- Project (87)
- +- BroadcastHashJoin Inner BuildRight (86)
- :- Project (81)
- : +- BroadcastHashJoin Inner BuildRight (80)
- : :- Project (75)
- : : +- BroadcastHashJoin Inner BuildLeft (74)
- : : :- BroadcastExchange (62)
- : : : +- Filter (61)
- : : : +- Scan parquet (60)
- : : +- BroadcastHashJoin LeftAnti BuildRight (73)
- : : :- BroadcastHashJoin LeftSemi BuildRight (68)
- : : : :- Project (65)
- : : : : +- Filter (64)
- : : : : +- Scan parquet (63)
- : : : +- BroadcastExchange (67)
- : : : +- Scan parquet (66)
- : : +- BroadcastExchange (72)
- : : +- Project (71)
- : : +- Filter (70)
- : : +- Scan parquet (69)
- : +- BroadcastExchange (79)
- : +- Project (78)
- : +- Filter (77)
- : +- Scan parquet (76)
- +- BroadcastExchange (85)
- +- Project (84)
- +- Filter (83)
- +- Scan parquet (82)
+ TakeOrderedAndProject (92)
+ +- HashAggregate (91)
+ +- Exchange (90)
+ +- HashAggregate (89)
+ +- Project (88)
+ +- BroadcastHashJoin Inner BuildRight (87)
+ :- Project (82)
+ : +- BroadcastHashJoin Inner BuildRight (81)
+ : :- Project (76)
+ : : +- BroadcastHashJoin Inner BuildLeft (75)
+ : : :- BroadcastExchange (63)
+ : : : +- Filter (62)
+ : : : +- Scan parquet (61)
+ : : +- BroadcastHashJoin LeftAnti BuildRight (74)
+ : : :- BroadcastHashJoin LeftSemi BuildRight (69)
+ : : : :- Project (66)
+ : : : : +- Filter (65)
+ : : : : +- Scan parquet (64)
+ : : : +- BroadcastExchange (68)
+ : : : +- Scan parquet (67)
+ : : +- BroadcastExchange (73)
+ : : +- Project (72)
+ : : +- Filter (71)
+ : : +- Scan parquet (70)
+ : +- BroadcastExchange (80)
+ : +- Project (79)
+ : +- Filter (78)
+ : +- Scan parquet (77)
+ +- BroadcastExchange (86)
+ +- Project (85)
+ +- Filter (84)
+ +- Scan parquet (83)
(1) Scan parquet
@@ -301,194 +302,198 @@ Input [2]: [s_name#X, count#X]
Input [3]: [hash_partition_key#X, s_name#X, count#X]
Arguments: false
-(52) ColumnarExchange
+(52) VeloxAppendBatches
+Input [3]: [hash_partition_key#X, s_name#X, count#X]
+Arguments: X
+
+(53) ColumnarExchange
Input [3]: [hash_partition_key#X, s_name#X, count#X]
Arguments: hashpartitioning(s_name#X, 1), ENSURE_REQUIREMENTS, [s_name#X, count#X], [plan_id=X], [id=#X]
-(53) ShuffleQueryStage
+(54) ShuffleQueryStage
Output [2]: [s_name#X, count#X]
Arguments: X
-(54) InputAdapter
+(55) InputAdapter
Input [2]: [s_name#X, count#X]
-(55) InputIteratorTransformer
+(56) InputIteratorTransformer
Input [2]: [s_name#X, count#X]
-(56) RegularHashAggregateExecTransformer
+(57) RegularHashAggregateExecTransformer
Input [2]: [s_name#X, count#X]
Keys [1]: [s_name#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [s_name#X, count(1)#X AS numwait#X]
-(57) WholeStageCodegenTransformer (X)
+(58) WholeStageCodegenTransformer (X)
Input [2]: [s_name#X, numwait#X]
Arguments: false
-(58) TakeOrderedAndProjectExecTransformer
+(59) TakeOrderedAndProjectExecTransformer
Input [2]: [s_name#X, numwait#X]
Arguments: X, [numwait#X DESC NULLS LAST, s_name#X ASC NULLS FIRST], [s_name#X, numwait#X], 0
-(59) VeloxColumnarToRowExec
+(60) VeloxColumnarToRowExec
Input [2]: [s_name#X, numwait#X]
-(60) Scan parquet
+(61) Scan parquet
Output [3]: [s_suppkey#X, s_name#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_name:string,s_nationkey:bigint>
-(61) Filter
+(62) Filter
Input [3]: [s_suppkey#X, s_name#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(62) BroadcastExchange
+(63) BroadcastExchange
Input [3]: [s_suppkey#X, s_name#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(63) Scan parquet
+(64) Scan parquet
Output [4]: [l_orderkey#X, l_suppkey#X, l_commitdate#X, l_receiptdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_receiptdate), IsNotNull(l_commitdate), IsNotNull(l_suppkey), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_suppkey:bigint,l_commitdate:date,l_receiptdate:date>
-(64) Filter
+(65) Filter
Input [4]: [l_orderkey#X, l_suppkey#X, l_commitdate#X, l_receiptdate#X]
Condition : ((((isnotnull(l_receiptdate#X) AND isnotnull(l_commitdate#X)) AND (l_receiptdate#X > l_commitdate#X)) AND isnotnull(l_suppkey#X)) AND isnotnull(l_orderkey#X))
-(65) Project
+(66) Project
Output [2]: [l_orderkey#X, l_suppkey#X]
Input [4]: [l_orderkey#X, l_suppkey#X, l_commitdate#X, l_receiptdate#X]
-(66) Scan parquet
+(67) Scan parquet
Output [2]: [l_orderkey#X, l_suppkey#X]
Batched: true
Location: InMemoryFileIndex [*]
ReadSchema: struct<l_orderkey:bigint,l_suppkey:bigint>
-(67) BroadcastExchange
+(68) BroadcastExchange
Input [2]: [l_orderkey#X, l_suppkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(68) BroadcastHashJoin
+(69) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: NOT (l_suppkey#X = l_suppkey#X)
-(69) Scan parquet
+(70) Scan parquet
Output [4]: [l_orderkey#X, l_suppkey#X, l_commitdate#X, l_receiptdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_receiptdate), IsNotNull(l_commitdate)]
ReadSchema: struct<l_orderkey:bigint,l_suppkey:bigint,l_commitdate:date,l_receiptdate:date>
-(70) Filter
+(71) Filter
Input [4]: [l_orderkey#X, l_suppkey#X, l_commitdate#X, l_receiptdate#X]
Condition : ((isnotnull(l_receiptdate#X) AND isnotnull(l_commitdate#X)) AND (l_receiptdate#X > l_commitdate#X))
-(71) Project
+(72) Project
Output [2]: [l_orderkey#X, l_suppkey#X]
Input [4]: [l_orderkey#X, l_suppkey#X, l_commitdate#X, l_receiptdate#X]
-(72) BroadcastExchange
+(73) BroadcastExchange
Input [2]: [l_orderkey#X, l_suppkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(73) BroadcastHashJoin
+(74) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: NOT (l_suppkey#X = l_suppkey#X)
-(74) BroadcastHashJoin
+(75) BroadcastHashJoin
Left keys [1]: [s_suppkey#X]
Right keys [1]: [l_suppkey#X]
Join condition: None
-(75) Project
+(76) Project
Output [3]: [s_name#X, s_nationkey#X, l_orderkey#X]
Input [5]: [s_suppkey#X, s_name#X, s_nationkey#X, l_orderkey#X, l_suppkey#X]
-(76) Scan parquet
+(77) Scan parquet
Output [2]: [o_orderkey#X, o_orderstatus#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderstatus), EqualTo(o_orderstatus,F), IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_orderstatus:string>
-(77) Filter
+(78) Filter
Input [2]: [o_orderkey#X, o_orderstatus#X]
Condition : ((isnotnull(o_orderstatus#X) AND (o_orderstatus#X = F)) AND isnotnull(o_orderkey#X))
-(78) Project
+(79) Project
Output [1]: [o_orderkey#X]
Input [2]: [o_orderkey#X, o_orderstatus#X]
-(79) BroadcastExchange
+(80) BroadcastExchange
Input [1]: [o_orderkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(80) BroadcastHashJoin
+(81) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [o_orderkey#X]
Join condition: None
-(81) Project
+(82) Project
Output [2]: [s_name#X, s_nationkey#X]
Input [4]: [s_name#X, s_nationkey#X, l_orderkey#X, o_orderkey#X]
-(82) Scan parquet
+(83) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_name), EqualTo(n_name,SAUDI ARABIA), IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(83) Filter
+(84) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : ((isnotnull(n_name#X) AND (n_name#X = SAUDI ARABIA)) AND isnotnull(n_nationkey#X))
-(84) Project
+(85) Project
Output [1]: [n_nationkey#X]
Input [2]: [n_nationkey#X, n_name#X]
-(85) BroadcastExchange
+(86) BroadcastExchange
Input [1]: [n_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(86) BroadcastHashJoin
+(87) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(87) Project
+(88) Project
Output [1]: [s_name#X]
Input [3]: [s_name#X, s_nationkey#X, n_nationkey#X]
-(88) HashAggregate
+(89) HashAggregate
Input [1]: [s_name#X]
Keys [1]: [s_name#X]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#X]
Results [2]: [s_name#X, count#X]
-(89) Exchange
+(90) Exchange
Input [2]: [s_name#X, count#X]
Arguments: hashpartitioning(s_name#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(90) HashAggregate
+(91) HashAggregate
Input [2]: [s_name#X, count#X]
Keys [1]: [s_name#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [s_name#X, count(1)#X AS numwait#X]
-(91) TakeOrderedAndProject
+(92) TakeOrderedAndProject
Input [2]: [s_name#X, numwait#X]
Arguments: X, [numwait#X DESC NULLS LAST, s_name#X ASC NULLS FIRST], [s_name#X, numwait#X]
-(92) AdaptiveSparkPlan
+(93) AdaptiveSparkPlan
Output [2]: [s_name#X, numwait#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/22.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/22.txt
index a8c66bef1..4c149d9bb 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/22.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/22.txt
@@ -1,37 +1,39 @@
== Physical Plan ==
-AdaptiveSparkPlan (38)
+AdaptiveSparkPlan (40)
+- == Final Plan ==
- VeloxColumnarToRowExec (26)
- +- ^ SortExecTransformer (24)
- +- ^ InputIteratorTransformer (23)
- +- ShuffleQueryStage (21)
- +- ColumnarExchange (20)
- +- ^ RegularHashAggregateExecTransformer (18)
- +- ^ InputIteratorTransformer (17)
- +- ShuffleQueryStage (15)
- +- ColumnarExchange (14)
- +- ^ ProjectExecTransformer (12)
- +- ^ FlushableHashAggregateExecTransformer (11)
- +- ^ ProjectExecTransformer (10)
- +- ^ BroadcastHashJoinExecTransformer LeftAnti BuildRight (9)
- :- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (8)
- +- BroadcastQueryStage (6)
- +- ColumnarBroadcastExchange (5)
- +- ^ Scan parquet (3)
+ VeloxColumnarToRowExec (28)
+ +- ^ SortExecTransformer (26)
+ +- ^ InputIteratorTransformer (25)
+ +- ShuffleQueryStage (23)
+ +- ColumnarExchange (22)
+ +- VeloxAppendBatches (21)
+ +- ^ RegularHashAggregateExecTransformer (19)
+ +- ^ InputIteratorTransformer (18)
+ +- ShuffleQueryStage (16)
+ +- ColumnarExchange (15)
+ +- VeloxAppendBatches (14)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ FlushableHashAggregateExecTransformer (11)
+ +- ^ ProjectExecTransformer (10)
+ +- ^ BroadcastHashJoinExecTransformer LeftAnti BuildRight (9)
+ :- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (8)
+ +- BroadcastQueryStage (6)
+ +- ColumnarBroadcastExchange (5)
+ +- ^ Scan parquet (3)
+- == Initial Plan ==
- Sort (37)
- +- Exchange (36)
- +- HashAggregate (35)
- +- Exchange (34)
- +- HashAggregate (33)
- +- Project (32)
- +- BroadcastHashJoin LeftAnti BuildRight (31)
- :- Filter (28)
- : +- Scan parquet (27)
- +- BroadcastExchange (30)
- +- Scan parquet (29)
+ Sort (39)
+ +- Exchange (38)
+ +- HashAggregate (37)
+ +- Exchange (36)
+ +- HashAggregate (35)
+ +- Project (34)
+ +- BroadcastHashJoin LeftAnti BuildRight (33)
+ :- Filter (30)
+ : +- Scan parquet (29)
+ +- BroadcastExchange (32)
+ +- Scan parquet (31)
(1) Scan parquet
@@ -93,112 +95,120 @@ Input [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
Input [5]: [hash_partition_key#X, cntrycode#X, count#X, sum#X, isEmpty#X]
Arguments: false
-(14) ColumnarExchange
+(14) VeloxAppendBatches
+Input [5]: [hash_partition_key#X, cntrycode#X, count#X, sum#X, isEmpty#X]
+Arguments: X
+
+(15) ColumnarExchange
Input [5]: [hash_partition_key#X, cntrycode#X, count#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(cntrycode#X, 1), ENSURE_REQUIREMENTS, [cntrycode#X, count#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(15) ShuffleQueryStage
+(16) ShuffleQueryStage
Output [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
Arguments: X
-(16) InputAdapter
+(17) InputAdapter
Input [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
-(17) InputIteratorTransformer
+(18) InputIteratorTransformer
Input [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
-(18) RegularHashAggregateExecTransformer
+(19) RegularHashAggregateExecTransformer
Input [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
Keys [1]: [cntrycode#X]
Functions [2]: [count(1), sum(c_acctbal#X)]
Aggregate Attributes [2]: [count(1)#X, sum(c_acctbal#X)#X]
Results [3]: [cntrycode#X, count(1)#X AS numcust#X, sum(c_acctbal#X)#X AS totacctbal#X]
-(19) WholeStageCodegenTransformer (X)
+(20) WholeStageCodegenTransformer (X)
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: false
-(20) ColumnarExchange
+(21) VeloxAppendBatches
+Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
+Arguments: X
+
+(22) ColumnarExchange
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: rangepartitioning(cntrycode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(21) ShuffleQueryStage
+(23) ShuffleQueryStage
Output [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: X
-(22) InputAdapter
+(24) InputAdapter
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
-(23) InputIteratorTransformer
+(25) InputIteratorTransformer
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
-(24) SortExecTransformer
+(26) SortExecTransformer
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: [cntrycode#X ASC NULLS FIRST], true, 0
-(25) WholeStageCodegenTransformer (X)
+(27) WholeStageCodegenTransformer (X)
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: false
-(26) VeloxColumnarToRowExec
+(28) VeloxColumnarToRowExec
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
-(27) Scan parquet
+(29) Scan parquet
Output [3]: [c_custkey#X, c_phone#X, c_acctbal#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_acctbal)]
ReadSchema: struct<c_custkey:bigint,c_phone:string,c_acctbal:decimal(12,2)>
-(28) Filter
+(30) Filter
Input [3]: [c_custkey#X, c_phone#X, c_acctbal#X]
Condition : ((isnotnull(c_acctbal#X) AND substring(c_phone#X, 1, 2) IN (13,31,23,29,30,18,17)) AND (cast(c_acctbal#X as decimal(16,6)) > Subquery subquery#X, [id=#X]))
-(29) Scan parquet
+(31) Scan parquet
Output [1]: [o_custkey#X]
Batched: true
Location: InMemoryFileIndex [*]
ReadSchema: struct<o_custkey:bigint>
-(30) BroadcastExchange
+(32) BroadcastExchange
Input [1]: [o_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(31) BroadcastHashJoin
+(33) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(32) Project
+(34) Project
Output [2]: [substring(c_phone#X, 1, 2) AS cntrycode#X, c_acctbal#X]
Input [3]: [c_custkey#X, c_phone#X, c_acctbal#X]
-(33) HashAggregate
+(35) HashAggregate
Input [2]: [cntrycode#X, c_acctbal#X]
Keys [1]: [cntrycode#X]
Functions [2]: [partial_count(1), partial_sum(c_acctbal#X)]
Aggregate Attributes [3]: [count#X, sum#X, isEmpty#X]
Results [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
-(34) Exchange
+(36) Exchange
Input [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(cntrycode#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(35) HashAggregate
+(37) HashAggregate
Input [4]: [cntrycode#X, count#X, sum#X, isEmpty#X]
Keys [1]: [cntrycode#X]
Functions [2]: [count(1), sum(c_acctbal#X)]
Aggregate Attributes [2]: [count(1)#X, sum(c_acctbal#X)#X]
Results [3]: [cntrycode#X, count(1)#X AS numcust#X, sum(c_acctbal#X)#X AS totacctbal#X]
-(36) Exchange
+(38) Exchange
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: rangepartitioning(cntrycode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(37) Sort
+(39) Sort
Input [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: [cntrycode#X ASC NULLS FIRST], true, 0
-(38) AdaptiveSparkPlan
+(40) AdaptiveSparkPlan
Output [3]: [cntrycode#X, numcust#X, totacctbal#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/3.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/3.txt
index b9b7951d7..df2a6de8d 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/3.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/3.txt
@@ -1,52 +1,53 @@
== Physical Plan ==
-AdaptiveSparkPlan (53)
+AdaptiveSparkPlan (54)
+- == Final Plan ==
- VeloxColumnarToRowExec (34)
- +- TakeOrderedAndProjectExecTransformer (33)
- +- ^ ProjectExecTransformer (31)
- +- ^ RegularHashAggregateExecTransformer (30)
- +- ^ InputIteratorTransformer (29)
- +- ShuffleQueryStage (27)
- +- ColumnarExchange (26)
- +- ^ ProjectExecTransformer (24)
- +- ^ FlushableHashAggregateExecTransformer (23)
- +- ^ ProjectExecTransformer (22)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (21)
- :- ^ ProjectExecTransformer (12)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
- : :- ^ InputIteratorTransformer (8)
- : : +- BroadcastQueryStage (6)
- : : +- ColumnarBroadcastExchange (5)
- : : +- ^ ProjectExecTransformer (3)
- : : +- ^ NoopFilter (2)
- : : +- ^ Scan parquet (1)
- : +- ^ NoopFilter (10)
- : +- ^ Scan parquet (9)
- +- ^ InputIteratorTransformer (20)
- +- BroadcastQueryStage (18)
- +- ColumnarBroadcastExchange (17)
- +- ^ ProjectExecTransformer (15)
- +- ^ NoopFilter (14)
- +- ^ Scan parquet (13)
+ VeloxColumnarToRowExec (35)
+ +- TakeOrderedAndProjectExecTransformer (34)
+ +- ^ ProjectExecTransformer (32)
+ +- ^ RegularHashAggregateExecTransformer (31)
+ +- ^ InputIteratorTransformer (30)
+ +- ShuffleQueryStage (28)
+ +- ColumnarExchange (27)
+ +- VeloxAppendBatches (26)
+ +- ^ ProjectExecTransformer (24)
+ +- ^ FlushableHashAggregateExecTransformer (23)
+ +- ^ ProjectExecTransformer (22)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (21)
+ :- ^ ProjectExecTransformer (12)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
+ : :- ^ InputIteratorTransformer (8)
+ : : +- BroadcastQueryStage (6)
+ : : +- ColumnarBroadcastExchange (5)
+ : : +- ^ ProjectExecTransformer (3)
+ : : +- ^ NoopFilter (2)
+ : : +- ^ Scan parquet (1)
+ : +- ^ NoopFilter (10)
+ : +- ^ Scan parquet (9)
+ +- ^ InputIteratorTransformer (20)
+ +- BroadcastQueryStage (18)
+ +- ColumnarBroadcastExchange (17)
+ +- ^ ProjectExecTransformer (15)
+ +- ^ NoopFilter (14)
+ +- ^ Scan parquet (13)
+- == Initial Plan ==
- TakeOrderedAndProject (52)
- +- HashAggregate (51)
- +- Exchange (50)
- +- HashAggregate (49)
- +- Project (48)
- +- BroadcastHashJoin Inner BuildRight (47)
- :- Project (42)
- : +- BroadcastHashJoin Inner BuildLeft (41)
- : :- BroadcastExchange (38)
- : : +- Project (37)
- : : +- Filter (36)
- : : +- Scan parquet (35)
- : +- Filter (40)
- : +- Scan parquet (39)
- +- BroadcastExchange (46)
- +- Project (45)
- +- Filter (44)
- +- Scan parquet (43)
+ TakeOrderedAndProject (53)
+ +- HashAggregate (52)
+ +- Exchange (51)
+ +- HashAggregate (50)
+ +- Project (49)
+ +- BroadcastHashJoin Inner BuildRight (48)
+ :- Project (43)
+ : +- BroadcastHashJoin Inner BuildLeft (42)
+ : :- BroadcastExchange (39)
+ : : +- Project (38)
+ : : +- Filter (37)
+ : : +- Scan parquet (36)
+ : +- Filter (41)
+ : +- Scan parquet (40)
+ +- BroadcastExchange (47)
+ +- Project (46)
+ +- Filter (45)
+ +- Scan parquet (44)
(1) Scan parquet
@@ -159,131 +160,135 @@ Input [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Input [6]: [hash_partition_key#X, l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Arguments: false
-(26) ColumnarExchange
+(26) VeloxAppendBatches
+Input [6]: [hash_partition_key#X, l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
+Arguments: X
+
+(27) ColumnarExchange
Input [6]: [hash_partition_key#X, l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_orderkey#X, o_orderdate#X, o_shippriority#X, 1), ENSURE_REQUIREMENTS, [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(27) ShuffleQueryStage
+(28) ShuffleQueryStage
Output [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Arguments: X
-(28) InputAdapter
+(29) InputAdapter
Input [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
-(29) InputIteratorTransformer
+(30) InputIteratorTransformer
Input [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
-(30) RegularHashAggregateExecTransformer
+(31) RegularHashAggregateExecTransformer
Input [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Keys [3]: [l_orderkey#X, o_orderdate#X, o_shippriority#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [4]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
-(31) ProjectExecTransformer
+(32) ProjectExecTransformer
Output [4]: [l_orderkey#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X, o_orderdate#X, o_shippriority#X]
Input [4]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
-(32) WholeStageCodegenTransformer (X)
+(33) WholeStageCodegenTransformer (X)
Input [4]: [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X]
Arguments: false
-(33) TakeOrderedAndProjectExecTransformer
+(34) TakeOrderedAndProjectExecTransformer
Input [4]: [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X]
Arguments: X, [revenue#X DESC NULLS LAST, o_orderdate#X ASC NULLS FIRST], [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X], 0
-(34) VeloxColumnarToRowExec
+(35) VeloxColumnarToRowExec
Input [4]: [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X]
-(35) Scan parquet
+(36) Scan parquet
Output [2]: [c_custkey#X, c_mktsegment#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_mktsegment), EqualTo(c_mktsegment,BUILDING), IsNotNull(c_custkey)]
ReadSchema: struct<c_custkey:bigint,c_mktsegment:string>
-(36) Filter
+(37) Filter
Input [2]: [c_custkey#X, c_mktsegment#X]
Condition : ((isnotnull(c_mktsegment#X) AND (c_mktsegment#X = BUILDING)) AND isnotnull(c_custkey#X))
-(37) Project
+(38) Project
Output [1]: [c_custkey#X]
Input [2]: [c_custkey#X, c_mktsegment#X]
-(38) BroadcastExchange
+(39) BroadcastExchange
Input [1]: [c_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(39) Scan parquet
+(40) Scan parquet
Output [4]: [o_orderkey#X, o_custkey#X, o_orderdate#X, o_shippriority#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderdate), LessThan(o_orderdate,1995-03-15), IsNotNull(o_custkey), IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_orderdate:date,o_shippriority:int>
-(40) Filter
+(41) Filter
Input [4]: [o_orderkey#X, o_custkey#X, o_orderdate#X, o_shippriority#X]
Condition : (((isnotnull(o_orderdate#X) AND (o_orderdate#X < 1995-03-15)) AND isnotnull(o_custkey#X)) AND isnotnull(o_orderkey#X))
-(41) BroadcastHashJoin
+(42) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(42) Project
+(43) Project
Output [3]: [o_orderkey#X, o_orderdate#X, o_shippriority#X]
Input [5]: [c_custkey#X, o_orderkey#X, o_custkey#X, o_orderdate#X, o_shippriority#X]
-(43) Scan parquet
+(44) Scan parquet
Output [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThan(l_shipdate,1995-03-15), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(44) Filter
+(45) Filter
Input [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : ((isnotnull(l_shipdate#X) AND (l_shipdate#X > 1995-03-15)) AND isnotnull(l_orderkey#X))
-(45) Project
+(46) Project
Output [3]: [l_orderkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(46) BroadcastExchange
+(47) BroadcastExchange
Input [3]: [l_orderkey#X, l_extendedprice#X, l_discount#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(47) BroadcastHashJoin
+(48) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(48) Project
+(49) Project
Output [5]: [o_orderdate#X, o_shippriority#X, l_orderkey#X, l_extendedprice#X, l_discount#X]
Input [6]: [o_orderkey#X, o_orderdate#X, o_shippriority#X, l_orderkey#X, l_extendedprice#X, l_discount#X]
-(49) HashAggregate
+(50) HashAggregate
Input [5]: [o_orderdate#X, o_shippriority#X, l_orderkey#X, l_extendedprice#X, l_discount#X]
Keys [3]: [l_orderkey#X, o_orderdate#X, o_shippriority#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
-(50) Exchange
+(51) Exchange
Input [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_orderkey#X, o_orderdate#X, o_shippriority#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(51) HashAggregate
+(52) HashAggregate
Input [5]: [l_orderkey#X, o_orderdate#X, o_shippriority#X, sum#X, isEmpty#X]
Keys [3]: [l_orderkey#X, o_orderdate#X, o_shippriority#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [4]: [l_orderkey#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X, o_orderdate#X, o_shippriority#X]
-(52) TakeOrderedAndProject
+(53) TakeOrderedAndProject
Input [4]: [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X]
Arguments: X, [revenue#X DESC NULLS LAST, o_orderdate#X ASC NULLS FIRST], [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X]
-(53) AdaptiveSparkPlan
+(54) AdaptiveSparkPlan
Output [4]: [l_orderkey#X, revenue#X, o_orderdate#X, o_shippriority#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/4.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/4.txt
index 97d18724d..34cb6467c 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/4.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/4.txt
@@ -1,43 +1,45 @@
== Physical Plan ==
-AdaptiveSparkPlan (44)
+AdaptiveSparkPlan (46)
+- == Final Plan ==
- VeloxColumnarToRowExec (29)
- +- ^ SortExecTransformer (27)
- +- ^ InputIteratorTransformer (26)
- +- ShuffleQueryStage (24)
- +- ColumnarExchange (23)
- +- ^ RegularHashAggregateExecTransformer (21)
- +- ^ InputIteratorTransformer (20)
- +- ShuffleQueryStage (18)
- +- ColumnarExchange (17)
- +- ^ ProjectExecTransformer (15)
- +- ^ FlushableHashAggregateExecTransformer (14)
- +- ^ ProjectExecTransformer (13)
- +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (12)
- :- ^ ProjectExecTransformer (3)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (11)
- +- BroadcastQueryStage (9)
- +- ColumnarBroadcastExchange (8)
- +- ^ ProjectExecTransformer (6)
- +- ^ NoopFilter (5)
- +- ^ Scan parquet (4)
+ VeloxColumnarToRowExec (31)
+ +- ^ SortExecTransformer (29)
+ +- ^ InputIteratorTransformer (28)
+ +- ShuffleQueryStage (26)
+ +- ColumnarExchange (25)
+ +- VeloxAppendBatches (24)
+ +- ^ RegularHashAggregateExecTransformer (22)
+ +- ^ InputIteratorTransformer (21)
+ +- ShuffleQueryStage (19)
+ +- ColumnarExchange (18)
+ +- VeloxAppendBatches (17)
+ +- ^ ProjectExecTransformer (15)
+ +- ^ FlushableHashAggregateExecTransformer (14)
+ +- ^ ProjectExecTransformer (13)
+ +- ^ BroadcastHashJoinExecTransformer LeftSemi BuildRight (12)
+ :- ^ ProjectExecTransformer (3)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (11)
+ +- BroadcastQueryStage (9)
+ +- ColumnarBroadcastExchange (8)
+ +- ^ ProjectExecTransformer (6)
+ +- ^ NoopFilter (5)
+ +- ^ Scan parquet (4)
+- == Initial Plan ==
- Sort (43)
- +- Exchange (42)
- +- HashAggregate (41)
- +- Exchange (40)
- +- HashAggregate (39)
- +- Project (38)
- +- BroadcastHashJoin LeftSemi BuildRight (37)
- :- Project (32)
- : +- Filter (31)
- : +- Scan parquet (30)
- +- BroadcastExchange (36)
- +- Project (35)
- +- Filter (34)
- +- Scan parquet (33)
+ Sort (45)
+ +- Exchange (44)
+ +- HashAggregate (43)
+ +- Exchange (42)
+ +- HashAggregate (41)
+ +- Project (40)
+ +- BroadcastHashJoin LeftSemi BuildRight (39)
+ :- Project (34)
+ : +- Filter (33)
+ : +- Scan parquet (32)
+ +- BroadcastExchange (38)
+ +- Project (37)
+ +- Filter (36)
+ +- Scan parquet (35)
(1) Scan parquet
@@ -112,125 +114,133 @@ Input [2]: [o_orderpriority#X, count#X]
Input [3]: [hash_partition_key#X, o_orderpriority#X, count#X]
Arguments: false
-(17) ColumnarExchange
+(17) VeloxAppendBatches
+Input [3]: [hash_partition_key#X, o_orderpriority#X, count#X]
+Arguments: X
+
+(18) ColumnarExchange
Input [3]: [hash_partition_key#X, o_orderpriority#X, count#X]
Arguments: hashpartitioning(o_orderpriority#X, 1), ENSURE_REQUIREMENTS, [o_orderpriority#X, count#X], [plan_id=X], [id=#X]
-(18) ShuffleQueryStage
+(19) ShuffleQueryStage
Output [2]: [o_orderpriority#X, count#X]
Arguments: X
-(19) InputAdapter
+(20) InputAdapter
Input [2]: [o_orderpriority#X, count#X]
-(20) InputIteratorTransformer
+(21) InputIteratorTransformer
Input [2]: [o_orderpriority#X, count#X]
-(21) RegularHashAggregateExecTransformer
+(22) RegularHashAggregateExecTransformer
Input [2]: [o_orderpriority#X, count#X]
Keys [1]: [o_orderpriority#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [o_orderpriority#X, count(1)#X AS order_count#X]
-(22) WholeStageCodegenTransformer (X)
+(23) WholeStageCodegenTransformer (X)
Input [2]: [o_orderpriority#X, order_count#X]
Arguments: false
-(23) ColumnarExchange
+(24) VeloxAppendBatches
+Input [2]: [o_orderpriority#X, order_count#X]
+Arguments: X
+
+(25) ColumnarExchange
Input [2]: [o_orderpriority#X, order_count#X]
Arguments: rangepartitioning(o_orderpriority#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(24) ShuffleQueryStage
+(26) ShuffleQueryStage
Output [2]: [o_orderpriority#X, order_count#X]
Arguments: X
-(25) InputAdapter
+(27) InputAdapter
Input [2]: [o_orderpriority#X, order_count#X]
-(26) InputIteratorTransformer
+(28) InputIteratorTransformer
Input [2]: [o_orderpriority#X, order_count#X]
-(27) SortExecTransformer
+(29) SortExecTransformer
Input [2]: [o_orderpriority#X, order_count#X]
Arguments: [o_orderpriority#X ASC NULLS FIRST], true, 0
-(28) WholeStageCodegenTransformer (X)
+(30) WholeStageCodegenTransformer (X)
Input [2]: [o_orderpriority#X, order_count#X]
Arguments: false
-(29) VeloxColumnarToRowExec
+(31) VeloxColumnarToRowExec
Input [2]: [o_orderpriority#X, order_count#X]
-(30) Scan parquet
+(32) Scan parquet
Output [3]: [o_orderkey#X, o_orderdate#X, o_orderpriority#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderdate), GreaterThanOrEqual(o_orderdate,1993-07-01), LessThan(o_orderdate,1993-10-01)]
ReadSchema: struct<o_orderkey:bigint,o_orderdate:date,o_orderpriority:string>
-(31) Filter
+(33) Filter
Input [3]: [o_orderkey#X, o_orderdate#X, o_orderpriority#X]
Condition : ((isnotnull(o_orderdate#X) AND (o_orderdate#X >= 1993-07-01)) AND (o_orderdate#X < 1993-10-01))
-(32) Project
+(34) Project
Output [2]: [o_orderkey#X, o_orderpriority#X]
Input [3]: [o_orderkey#X, o_orderdate#X, o_orderpriority#X]
-(33) Scan parquet
+(35) Scan parquet
Output [3]: [l_orderkey#X, l_commitdate#X, l_receiptdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_commitdate), IsNotNull(l_receiptdate)]
ReadSchema: struct<l_orderkey:bigint,l_commitdate:date,l_receiptdate:date>
-(34) Filter
+(36) Filter
Input [3]: [l_orderkey#X, l_commitdate#X, l_receiptdate#X]
Condition : ((isnotnull(l_commitdate#X) AND isnotnull(l_receiptdate#X)) AND (l_commitdate#X < l_receiptdate#X))
-(35) Project
+(37) Project
Output [1]: [l_orderkey#X]
Input [3]: [l_orderkey#X, l_commitdate#X, l_receiptdate#X]
-(36) BroadcastExchange
+(38) BroadcastExchange
Input [1]: [l_orderkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(37) BroadcastHashJoin
+(39) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(38) Project
+(40) Project
Output [1]: [o_orderpriority#X]
Input [2]: [o_orderkey#X, o_orderpriority#X]
-(39) HashAggregate
+(41) HashAggregate
Input [1]: [o_orderpriority#X]
Keys [1]: [o_orderpriority#X]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#X]
Results [2]: [o_orderpriority#X, count#X]
-(40) Exchange
+(42) Exchange
Input [2]: [o_orderpriority#X, count#X]
Arguments: hashpartitioning(o_orderpriority#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(41) HashAggregate
+(43) HashAggregate
Input [2]: [o_orderpriority#X, count#X]
Keys [1]: [o_orderpriority#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [o_orderpriority#X, count(1)#X AS order_count#X]
-(42) Exchange
+(44) Exchange
Input [2]: [o_orderpriority#X, order_count#X]
Arguments: rangepartitioning(o_orderpriority#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(43) Sort
+(45) Sort
Input [2]: [o_orderpriority#X, order_count#X]
Arguments: [o_orderpriority#X ASC NULLS FIRST], true, 0
-(44) AdaptiveSparkPlan
+(46) AdaptiveSparkPlan
Output [2]: [o_orderpriority#X, order_count#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/5.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/5.txt
index a9b993d0a..2c9ba4c93 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/5.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/5.txt
@@ -1,91 +1,93 @@
== Physical Plan ==
-AdaptiveSparkPlan (100)
+AdaptiveSparkPlan (102)
+- == Final Plan ==
- VeloxColumnarToRowExec (65)
- +- ^ SortExecTransformer (63)
- +- ^ InputIteratorTransformer (62)
- +- ShuffleQueryStage (60)
- +- ColumnarExchange (59)
- +- ^ RegularHashAggregateExecTransformer (57)
- +- ^ InputIteratorTransformer (56)
- +- ShuffleQueryStage (54)
- +- ColumnarExchange (53)
- +- ^ ProjectExecTransformer (51)
- +- ^ FlushableHashAggregateExecTransformer (50)
- +- ^ ProjectExecTransformer (49)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (48)
- :- ^ ProjectExecTransformer (39)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (38)
- : :- ^ ProjectExecTransformer (30)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (29)
- : : :- ^ ProjectExecTransformer (21)
- : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
- : : : :- ^ ProjectExecTransformer (12)
- : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
- : : : : :- ^ InputIteratorTransformer (7)
- : : : : : +- BroadcastQueryStage (5)
- : : : : : +- ColumnarBroadcastExchange (4)
- : : : : : +- ^ NoopFilter (2)
- : : : : : +- ^ Scan parquet (1)
- : : : : +- ^ ProjectExecTransformer (10)
- : : : : +- ^ NoopFilter (9)
- : : : : +- ^ Scan parquet (8)
- : : : +- ^ InputIteratorTransformer (19)
- : : : +- BroadcastQueryStage (17)
- : : : +- ColumnarBroadcastExchange (16)
- : : : +- ^ NoopFilter (14)
- : : : +- ^ Scan parquet (13)
- : : +- ^ InputIteratorTransformer (28)
- : : +- BroadcastQueryStage (26)
- : : +- ColumnarBroadcastExchange (25)
- : : +- ^ NoopFilter (23)
- : : +- ^ Scan parquet (22)
- : +- ^ InputIteratorTransformer (37)
- : +- BroadcastQueryStage (35)
- : +- ColumnarBroadcastExchange (34)
- : +- ^ NoopFilter (32)
- : +- ^ Scan parquet (31)
- +- ^ InputIteratorTransformer (47)
- +- BroadcastQueryStage (45)
- +- ColumnarBroadcastExchange (44)
- +- ^ ProjectExecTransformer (42)
- +- ^ NoopFilter (41)
- +- ^ Scan parquet (40)
+ VeloxColumnarToRowExec (67)
+ +- ^ SortExecTransformer (65)
+ +- ^ InputIteratorTransformer (64)
+ +- ShuffleQueryStage (62)
+ +- ColumnarExchange (61)
+ +- VeloxAppendBatches (60)
+ +- ^ RegularHashAggregateExecTransformer (58)
+ +- ^ InputIteratorTransformer (57)
+ +- ShuffleQueryStage (55)
+ +- ColumnarExchange (54)
+ +- VeloxAppendBatches (53)
+ +- ^ ProjectExecTransformer (51)
+ +- ^ FlushableHashAggregateExecTransformer (50)
+ +- ^ ProjectExecTransformer (49)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (48)
+ :- ^ ProjectExecTransformer (39)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (38)
+ : :- ^ ProjectExecTransformer (30)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (29)
+ : : :- ^ ProjectExecTransformer (21)
+ : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
+ : : : :- ^ ProjectExecTransformer (12)
+ : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
+ : : : : :- ^ InputIteratorTransformer (7)
+ : : : : : +- BroadcastQueryStage (5)
+ : : : : : +- ColumnarBroadcastExchange (4)
+ : : : : : +- ^ NoopFilter (2)
+ : : : : : +- ^ Scan parquet (1)
+ : : : : +- ^ ProjectExecTransformer (10)
+ : : : : +- ^ NoopFilter (9)
+ : : : : +- ^ Scan parquet (8)
+ : : : +- ^ InputIteratorTransformer (19)
+ : : : +- BroadcastQueryStage (17)
+ : : : +- ColumnarBroadcastExchange (16)
+ : : : +- ^ NoopFilter (14)
+ : : : +- ^ Scan parquet (13)
+ : : +- ^ InputIteratorTransformer (28)
+ : : +- BroadcastQueryStage (26)
+ : : +- ColumnarBroadcastExchange (25)
+ : : +- ^ NoopFilter (23)
+ : : +- ^ Scan parquet (22)
+ : +- ^ InputIteratorTransformer (37)
+ : +- BroadcastQueryStage (35)
+ : +- ColumnarBroadcastExchange (34)
+ : +- ^ NoopFilter (32)
+ : +- ^ Scan parquet (31)
+ +- ^ InputIteratorTransformer (47)
+ +- BroadcastQueryStage (45)
+ +- ColumnarBroadcastExchange (44)
+ +- ^ ProjectExecTransformer (42)
+ +- ^ NoopFilter (41)
+ +- ^ Scan parquet (40)
+- == Initial Plan ==
- Sort (99)
- +- Exchange (98)
- +- HashAggregate (97)
- +- Exchange (96)
- +- HashAggregate (95)
- +- Project (94)
- +- BroadcastHashJoin Inner BuildRight (93)
- :- Project (88)
- : +- BroadcastHashJoin Inner BuildRight (87)
- : :- Project (83)
- : : +- BroadcastHashJoin Inner BuildRight (82)
- : : :- Project (78)
- : : : +- BroadcastHashJoin Inner BuildRight (77)
- : : : :- Project (73)
- : : : : +- BroadcastHashJoin Inner BuildLeft (72)
- : : : : :- BroadcastExchange (68)
- : : : : : +- Filter (67)
- : : : : : +- Scan parquet (66)
- : : : : +- Project (71)
- : : : : +- Filter (70)
- : : : : +- Scan parquet (69)
- : : : +- BroadcastExchange (76)
- : : : +- Filter (75)
- : : : +- Scan parquet (74)
- : : +- BroadcastExchange (81)
- : : +- Filter (80)
- : : +- Scan parquet (79)
- : +- BroadcastExchange (86)
- : +- Filter (85)
- : +- Scan parquet (84)
- +- BroadcastExchange (92)
- +- Project (91)
- +- Filter (90)
- +- Scan parquet (89)
+ Sort (101)
+ +- Exchange (100)
+ +- HashAggregate (99)
+ +- Exchange (98)
+ +- HashAggregate (97)
+ +- Project (96)
+ +- BroadcastHashJoin Inner BuildRight (95)
+ :- Project (90)
+ : +- BroadcastHashJoin Inner BuildRight (89)
+ : :- Project (85)
+ : : +- BroadcastHashJoin Inner BuildRight (84)
+ : : :- Project (80)
+ : : : +- BroadcastHashJoin Inner BuildRight (79)
+ : : : :- Project (75)
+ : : : : +- BroadcastHashJoin Inner BuildLeft (74)
+ : : : : :- BroadcastExchange (70)
+ : : : : : +- Filter (69)
+ : : : : : +- Scan parquet (68)
+ : : : : +- Project (73)
+ : : : : +- Filter (72)
+ : : : : +- Scan parquet (71)
+ : : : +- BroadcastExchange (78)
+ : : : +- Filter (77)
+ : : : +- Scan parquet (76)
+ : : +- BroadcastExchange (83)
+ : : +- Filter (82)
+ : : +- Scan parquet (81)
+ : +- BroadcastExchange (88)
+ : +- Filter (87)
+ : +- Scan parquet (86)
+ +- BroadcastExchange (94)
+ +- Project (93)
+ +- Filter (92)
+ +- Scan parquet (91)
(1) Scan parquet
@@ -312,221 +314,229 @@ Input [3]: [n_name#X, sum#X, isEmpty#X]
Input [4]: [hash_partition_key#X, n_name#X, sum#X, isEmpty#X]
Arguments: false
-(53) ColumnarExchange
+(53) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, n_name#X, sum#X, isEmpty#X]
+Arguments: X
+
+(54) ColumnarExchange
Input [4]: [hash_partition_key#X, n_name#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(n_name#X, 1), ENSURE_REQUIREMENTS, [n_name#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(54) ShuffleQueryStage
+(55) ShuffleQueryStage
Output [3]: [n_name#X, sum#X, isEmpty#X]
Arguments: X
-(55) InputAdapter
+(56) InputAdapter
Input [3]: [n_name#X, sum#X, isEmpty#X]
-(56) InputIteratorTransformer
+(57) InputIteratorTransformer
Input [3]: [n_name#X, sum#X, isEmpty#X]
-(57) RegularHashAggregateExecTransformer
+(58) RegularHashAggregateExecTransformer
Input [3]: [n_name#X, sum#X, isEmpty#X]
Keys [1]: [n_name#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [2]: [n_name#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X]
-(58) WholeStageCodegenTransformer (X)
+(59) WholeStageCodegenTransformer (X)
Input [2]: [n_name#X, revenue#X]
Arguments: false
-(59) ColumnarExchange
+(60) VeloxAppendBatches
+Input [2]: [n_name#X, revenue#X]
+Arguments: X
+
+(61) ColumnarExchange
Input [2]: [n_name#X, revenue#X]
Arguments: rangepartitioning(revenue#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(60) ShuffleQueryStage
+(62) ShuffleQueryStage
Output [2]: [n_name#X, revenue#X]
Arguments: X
-(61) InputAdapter
+(63) InputAdapter
Input [2]: [n_name#X, revenue#X]
-(62) InputIteratorTransformer
+(64) InputIteratorTransformer
Input [2]: [n_name#X, revenue#X]
-(63) SortExecTransformer
+(65) SortExecTransformer
Input [2]: [n_name#X, revenue#X]
Arguments: [revenue#X DESC NULLS LAST], true, 0
-(64) WholeStageCodegenTransformer (X)
+(66) WholeStageCodegenTransformer (X)
Input [2]: [n_name#X, revenue#X]
Arguments: false
-(65) VeloxColumnarToRowExec
+(67) VeloxColumnarToRowExec
Input [2]: [n_name#X, revenue#X]
-(66) Scan parquet
+(68) Scan parquet
Output [2]: [c_custkey#X, c_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_custkey), IsNotNull(c_nationkey)]
ReadSchema: struct<c_custkey:bigint,c_nationkey:bigint>
-(67) Filter
+(69) Filter
Input [2]: [c_custkey#X, c_nationkey#X]
Condition : (isnotnull(c_custkey#X) AND isnotnull(c_nationkey#X))
-(68) BroadcastExchange
+(70) BroadcastExchange
Input [2]: [c_custkey#X, c_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(69) Scan parquet
+(71) Scan parquet
Output [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderdate), GreaterThanOrEqual(o_orderdate,1994-01-01), LessThan(o_orderdate,1995-01-01), IsNotNull(o_custkey), IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_orderdate:date>
-(70) Filter
+(72) Filter
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Condition : ((((isnotnull(o_orderdate#X) AND (o_orderdate#X >= 1994-01-01)) AND (o_orderdate#X < 1995-01-01)) AND isnotnull(o_custkey#X)) AND isnotnull(o_orderkey#X))
-(71) Project
+(73) Project
Output [2]: [o_orderkey#X, o_custkey#X]
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
-(72) BroadcastHashJoin
+(74) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(73) Project
+(75) Project
Output [2]: [c_nationkey#X, o_orderkey#X]
Input [4]: [c_custkey#X, c_nationkey#X, o_orderkey#X, o_custkey#X]
-(74) Scan parquet
+(76) Scan parquet
Output [4]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_orderkey), IsNotNull(l_suppkey)]
ReadSchema: struct<l_orderkey:bigint,l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2)>
-(75) Filter
+(77) Filter
Input [4]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Condition : (isnotnull(l_orderkey#X) AND isnotnull(l_suppkey#X))
-(76) BroadcastExchange
+(78) BroadcastExchange
Input [4]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(77) BroadcastHashJoin
+(79) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(78) Project
+(80) Project
Output [4]: [c_nationkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Input [6]: [c_nationkey#X, o_orderkey#X, l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
-(79) Scan parquet
+(81) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(80) Filter
+(82) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(81) BroadcastExchange
+(83) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false], input[1, bigint, false]),false), [plan_id=X]
-(82) BroadcastHashJoin
+(84) BroadcastHashJoin
Left keys [2]: [l_suppkey#X, c_nationkey#X]
Right keys [2]: [s_suppkey#X, s_nationkey#X]
Join condition: None
-(83) Project
+(85) Project
Output [3]: [l_extendedprice#X, l_discount#X, s_nationkey#X]
Input [6]: [c_nationkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X, s_suppkey#X, s_nationkey#X]
-(84) Scan parquet
+(86) Scan parquet
Output [3]: [n_nationkey#X, n_name#X, n_regionkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey), IsNotNull(n_regionkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string,n_regionkey:bigint>
-(85) Filter
+(87) Filter
Input [3]: [n_nationkey#X, n_name#X, n_regionkey#X]
Condition : (isnotnull(n_nationkey#X) AND isnotnull(n_regionkey#X))
-(86) BroadcastExchange
+(88) BroadcastExchange
Input [3]: [n_nationkey#X, n_name#X, n_regionkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(87) BroadcastHashJoin
+(89) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(88) Project
+(90) Project
Output [4]: [l_extendedprice#X, l_discount#X, n_name#X, n_regionkey#X]
Input [6]: [l_extendedprice#X, l_discount#X, s_nationkey#X, n_nationkey#X, n_name#X, n_regionkey#X]
-(89) Scan parquet
+(91) Scan parquet
Output [2]: [r_regionkey#X, r_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(r_name), EqualTo(r_name,ASIA), IsNotNull(r_regionkey)]
ReadSchema: struct<r_regionkey:bigint,r_name:string>
-(90) Filter
+(92) Filter
Input [2]: [r_regionkey#X, r_name#X]
Condition : ((isnotnull(r_name#X) AND (r_name#X = ASIA)) AND isnotnull(r_regionkey#X))
-(91) Project
+(93) Project
Output [1]: [r_regionkey#X]
Input [2]: [r_regionkey#X, r_name#X]
-(92) BroadcastExchange
+(94) BroadcastExchange
Input [1]: [r_regionkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(93) BroadcastHashJoin
+(95) BroadcastHashJoin
Left keys [1]: [n_regionkey#X]
Right keys [1]: [r_regionkey#X]
Join condition: None
-(94) Project
+(96) Project
Output [3]: [l_extendedprice#X, l_discount#X, n_name#X]
Input [5]: [l_extendedprice#X, l_discount#X, n_name#X, n_regionkey#X, r_regionkey#X]
-(95) HashAggregate
+(97) HashAggregate
Input [3]: [l_extendedprice#X, l_discount#X, n_name#X]
Keys [1]: [n_name#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [n_name#X, sum#X, isEmpty#X]
-(96) Exchange
+(98) Exchange
Input [3]: [n_name#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(n_name#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(97) HashAggregate
+(99) HashAggregate
Input [3]: [n_name#X, sum#X, isEmpty#X]
Keys [1]: [n_name#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X]
Results [2]: [n_name#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true))#X AS revenue#X]
-(98) Exchange
+(100) Exchange
Input [2]: [n_name#X, revenue#X]
Arguments: rangepartitioning(revenue#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(99) Sort
+(101) Sort
Input [2]: [n_name#X, revenue#X]
Arguments: [revenue#X DESC NULLS LAST], true, 0
-(100) AdaptiveSparkPlan
+(102) AdaptiveSparkPlan
Output [2]: [n_name#X, revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/6.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/6.txt
index 7aae3dccf..b6f876d48 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/6.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/6.txt
@@ -1,22 +1,23 @@
== Physical Plan ==
-AdaptiveSparkPlan (19)
+AdaptiveSparkPlan (20)
+- == Final Plan ==
- VeloxColumnarToRowExec (12)
- +- ^ RegularHashAggregateExecTransformer (10)
- +- ^ InputIteratorTransformer (9)
- +- ShuffleQueryStage (7)
- +- ColumnarExchange (6)
- +- ^ FlushableHashAggregateExecTransformer (4)
- +- ^ ProjectExecTransformer (3)
- +- ^ NoopFilter (2)
- +- ^ Scan parquet (1)
+ VeloxColumnarToRowExec (13)
+ +- ^ RegularHashAggregateExecTransformer (11)
+ +- ^ InputIteratorTransformer (10)
+ +- ShuffleQueryStage (8)
+ +- ColumnarExchange (7)
+ +- VeloxAppendBatches (6)
+ +- ^ FlushableHashAggregateExecTransformer (4)
+ +- ^ ProjectExecTransformer (3)
+ +- ^ NoopFilter (2)
+ +- ^ Scan parquet (1)
+- == Initial Plan ==
- HashAggregate (18)
- +- Exchange (17)
- +- HashAggregate (16)
- +- Project (15)
- +- Filter (14)
- +- Scan parquet (13)
+ HashAggregate (19)
+ +- Exchange (18)
+ +- HashAggregate (17)
+ +- Project (16)
+ +- Filter (15)
+ +- Scan parquet (14)
(1) Scan parquet
@@ -45,67 +46,71 @@ Results [2]: [sum#X, isEmpty#X]
Input [2]: [sum#X, isEmpty#X]
Arguments: false
-(6) ColumnarExchange
+(6) VeloxAppendBatches
+Input [2]: [sum#X, isEmpty#X]
+Arguments: X
+
+(7) ColumnarExchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(7) ShuffleQueryStage
+(8) ShuffleQueryStage
Output [2]: [sum#X, isEmpty#X]
Arguments: X
-(8) InputAdapter
+(9) InputAdapter
Input [2]: [sum#X, isEmpty#X]
-(9) InputIteratorTransformer
+(10) InputIteratorTransformer
Input [2]: [sum#X, isEmpty#X]
-(10) RegularHashAggregateExecTransformer
+(11) RegularHashAggregateExecTransformer
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))#X]
Results [1]: [sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))#X AS revenue#X]
-(11) WholeStageCodegenTransformer (X)
+(12) WholeStageCodegenTransformer (X)
Input [1]: [revenue#X]
Arguments: false
-(12) VeloxColumnarToRowExec
+(13) VeloxColumnarToRowExec
Input [1]: [revenue#X]
-(13) Scan parquet
+(14) Scan parquet
Output [4]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), IsNotNull(l_discount), IsNotNull(l_quantity), GreaterThanOrEqual(l_shipdate,1994-01-01), LessThan(l_shipdate,1995-01-01), GreaterThanOrEqual(l_discount,0.05), LessThanOrEqual(l_discount,0.07), LessThan(l_quantity,24.00)]
ReadSchema: struct<l_quantity:decimal(12,2),l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(14) Filter
+(15) Filter
Input [4]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : (((((((isnotnull(l_shipdate#X) AND isnotnull(l_discount#X)) AND isnotnull(l_quantity#X)) AND (l_shipdate#X >= 1994-01-01)) AND (l_shipdate#X < 1995-01-01)) AND (l_discount#X >= 0.05)) AND (l_discount#X <= 0.07)) AND (l_quantity#X < 24.00))
-(15) Project
+(16) Project
Output [2]: [l_extendedprice#X, l_discount#X]
Input [4]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(16) HashAggregate
+(17) HashAggregate
Input [2]: [l_extendedprice#X, l_discount#X]
Keys: []
Functions [1]: [partial_sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [2]: [sum#X, isEmpty#X]
-(17) Exchange
+(18) Exchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X]
-(18) HashAggregate
+(19) HashAggregate
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))#X]
Results [1]: [sum(CheckOverflow((promote_precision(l_extendedprice#X) * promote_precision(l_discount#X)), DecimalType(25,4), true))#X AS revenue#X]
-(19) AdaptiveSparkPlan
+(20) AdaptiveSparkPlan
Output [1]: [revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/7.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/7.txt
index b0d68672c..8cf20f769 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/7.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/7.txt
@@ -1,85 +1,87 @@
== Physical Plan ==
-AdaptiveSparkPlan (93)
+AdaptiveSparkPlan (95)
+- == Final Plan ==
- VeloxColumnarToRowExec (60)
- +- ^ SortExecTransformer (58)
- +- ^ InputIteratorTransformer (57)
- +- ShuffleQueryStage (55)
- +- ColumnarExchange (54)
- +- ^ RegularHashAggregateExecTransformer (52)
- +- ^ InputIteratorTransformer (51)
- +- ShuffleQueryStage (49)
- +- ColumnarExchange (48)
- +- ^ ProjectExecTransformer (46)
- +- ^ FlushableHashAggregateExecTransformer (45)
- +- ^ ProjectExecTransformer (44)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (43)
- :- ^ ProjectExecTransformer (38)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (37)
- : :- ^ ProjectExecTransformer (29)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (28)
- : : :- ^ ProjectExecTransformer (20)
- : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (19)
- : : : :- ^ ProjectExecTransformer (11)
- : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (10)
- : : : : :- ^ InputIteratorTransformer (7)
- : : : : : +- BroadcastQueryStage (5)
- : : : : : +- ColumnarBroadcastExchange (4)
- : : : : : +- ^ NoopFilter (2)
- : : : : : +- ^ Scan parquet (1)
- : : : : +- ^ NoopFilter (9)
- : : : : +- ^ Scan parquet (8)
- : : : +- ^ InputIteratorTransformer (18)
- : : : +- BroadcastQueryStage (16)
- : : : +- ColumnarBroadcastExchange (15)
- : : : +- ^ NoopFilter (13)
- : : : +- ^ Scan parquet (12)
- : : +- ^ InputIteratorTransformer (27)
- : : +- BroadcastQueryStage (25)
- : : +- ColumnarBroadcastExchange (24)
- : : +- ^ NoopFilter (22)
- : : +- ^ Scan parquet (21)
- : +- ^ InputIteratorTransformer (36)
- : +- BroadcastQueryStage (34)
- : +- ColumnarBroadcastExchange (33)
- : +- ^ NoopFilter (31)
- : +- ^ Scan parquet (30)
- +- ^ InputIteratorTransformer (42)
- +- BroadcastQueryStage (40)
- +- ReusedExchange (39)
+ VeloxColumnarToRowExec (62)
+ +- ^ SortExecTransformer (60)
+ +- ^ InputIteratorTransformer (59)
+ +- ShuffleQueryStage (57)
+ +- ColumnarExchange (56)
+ +- VeloxAppendBatches (55)
+ +- ^ RegularHashAggregateExecTransformer (53)
+ +- ^ InputIteratorTransformer (52)
+ +- ShuffleQueryStage (50)
+ +- ColumnarExchange (49)
+ +- VeloxAppendBatches (48)
+ +- ^ ProjectExecTransformer (46)
+ +- ^ FlushableHashAggregateExecTransformer (45)
+ +- ^ ProjectExecTransformer (44)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (43)
+ :- ^ ProjectExecTransformer (38)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (37)
+ : :- ^ ProjectExecTransformer (29)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (28)
+ : : :- ^ ProjectExecTransformer (20)
+ : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (19)
+ : : : :- ^ ProjectExecTransformer (11)
+ : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (10)
+ : : : : :- ^ InputIteratorTransformer (7)
+ : : : : : +- BroadcastQueryStage (5)
+ : : : : : +- ColumnarBroadcastExchange (4)
+ : : : : : +- ^ NoopFilter (2)
+ : : : : : +- ^ Scan parquet (1)
+ : : : : +- ^ NoopFilter (9)
+ : : : : +- ^ Scan parquet (8)
+ : : : +- ^ InputIteratorTransformer (18)
+ : : : +- BroadcastQueryStage (16)
+ : : : +- ColumnarBroadcastExchange (15)
+ : : : +- ^ NoopFilter (13)
+ : : : +- ^ Scan parquet (12)
+ : : +- ^ InputIteratorTransformer (27)
+ : : +- BroadcastQueryStage (25)
+ : : +- ColumnarBroadcastExchange (24)
+ : : +- ^ NoopFilter (22)
+ : : +- ^ Scan parquet (21)
+ : +- ^ InputIteratorTransformer (36)
+ : +- BroadcastQueryStage (34)
+ : +- ColumnarBroadcastExchange (33)
+ : +- ^ NoopFilter (31)
+ : +- ^ Scan parquet (30)
+ +- ^ InputIteratorTransformer (42)
+ +- BroadcastQueryStage (40)
+ +- ReusedExchange (39)
+- == Initial Plan ==
- Sort (92)
- +- Exchange (91)
- +- HashAggregate (90)
- +- Exchange (89)
- +- HashAggregate (88)
- +- Project (87)
- +- BroadcastHashJoin Inner BuildRight (86)
- :- Project (82)
- : +- BroadcastHashJoin Inner BuildRight (81)
- : :- Project (77)
- : : +- BroadcastHashJoin Inner BuildRight (76)
- : : :- Project (72)
- : : : +- BroadcastHashJoin Inner BuildRight (71)
- : : : :- Project (67)
- : : : : +- BroadcastHashJoin Inner BuildLeft (66)
- : : : : :- BroadcastExchange (63)
- : : : : : +- Filter (62)
- : : : : : +- Scan parquet (61)
- : : : : +- Filter (65)
- : : : : +- Scan parquet (64)
- : : : +- BroadcastExchange (70)
- : : : +- Filter (69)
- : : : +- Scan parquet (68)
- : : +- BroadcastExchange (75)
- : : +- Filter (74)
- : : +- Scan parquet (73)
- : +- BroadcastExchange (80)
- : +- Filter (79)
- : +- Scan parquet (78)
- +- BroadcastExchange (85)
- +- Filter (84)
- +- Scan parquet (83)
+ Sort (94)
+ +- Exchange (93)
+ +- HashAggregate (92)
+ +- Exchange (91)
+ +- HashAggregate (90)
+ +- Project (89)
+ +- BroadcastHashJoin Inner BuildRight (88)
+ :- Project (84)
+ : +- BroadcastHashJoin Inner BuildRight (83)
+ : :- Project (79)
+ : : +- BroadcastHashJoin Inner BuildRight (78)
+ : : :- Project (74)
+ : : : +- BroadcastHashJoin Inner BuildRight (73)
+ : : : :- Project (69)
+ : : : : +- BroadcastHashJoin Inner BuildLeft (68)
+ : : : : :- BroadcastExchange (65)
+ : : : : : +- Filter (64)
+ : : : : : +- Scan parquet (63)
+ : : : : +- Filter (67)
+ : : : : +- Scan parquet (66)
+ : : : +- BroadcastExchange (72)
+ : : : +- Filter (71)
+ : : : +- Scan parquet (70)
+ : : +- BroadcastExchange (77)
+ : : +- Filter (76)
+ : : +- Scan parquet (75)
+ : +- BroadcastExchange (82)
+ : +- Filter (81)
+ : +- Scan parquet (80)
+ +- BroadcastExchange (87)
+ +- Filter (86)
+ +- Scan parquet (85)
(1) Scan parquet
@@ -282,213 +284,221 @@ Input [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Input [6]: [hash_partition_key#X, supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Arguments: false
-(48) ColumnarExchange
+(48) VeloxAppendBatches
+Input [6]: [hash_partition_key#X, supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
+Arguments: X
+
+(49) ColumnarExchange
Input [6]: [hash_partition_key#X, supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(supp_nation#X, cust_nation#X, l_year#X, 1), ENSURE_REQUIREMENTS, [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(49) ShuffleQueryStage
+(50) ShuffleQueryStage
Output [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Arguments: X
-(50) InputAdapter
+(51) InputAdapter
Input [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
-(51) InputIteratorTransformer
+(52) InputIteratorTransformer
Input [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
-(52) RegularHashAggregateExecTransformer
+(53) RegularHashAggregateExecTransformer
Input [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Keys [3]: [supp_nation#X, cust_nation#X, l_year#X]
Functions [1]: [sum(volume#X)]
Aggregate Attributes [1]: [sum(volume#X)#X]
Results [4]: [supp_nation#X, cust_nation#X, l_year#X, sum(volume#X)#X AS revenue#X]
-(53) WholeStageCodegenTransformer (X)
+(54) WholeStageCodegenTransformer (X)
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: false
-(54) ColumnarExchange
+(55) VeloxAppendBatches
+Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
+Arguments: X
+
+(56) ColumnarExchange
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: rangepartitioning(supp_nation#X ASC NULLS FIRST, cust_nation#X ASC NULLS FIRST, l_year#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(55) ShuffleQueryStage
+(57) ShuffleQueryStage
Output [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: X
-(56) InputAdapter
+(58) InputAdapter
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
-(57) InputIteratorTransformer
+(59) InputIteratorTransformer
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
-(58) SortExecTransformer
+(60) SortExecTransformer
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: [supp_nation#X ASC NULLS FIRST, cust_nation#X ASC NULLS FIRST, l_year#X ASC NULLS FIRST], true, 0
-(59) WholeStageCodegenTransformer (X)
+(61) WholeStageCodegenTransformer (X)
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: false
-(60) VeloxColumnarToRowExec
+(62) VeloxColumnarToRowExec
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
-(61) Scan parquet
+(63) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(62) Filter
+(64) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(63) BroadcastExchange
+(65) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(64) Scan parquet
+(66) Scan parquet
Output [5]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1995-01-01), LessThanOrEqual(l_shipdate,1996-12-31), IsNotNull(l_suppkey), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(65) Filter
+(67) Filter
Input [5]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : ((((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1995-01-01)) AND (l_shipdate#X <= 1996-12-31)) AND isnotnull(l_suppkey#X)) AND isnotnull(l_orderkey#X))
-(66) BroadcastHashJoin
+(68) BroadcastHashJoin
Left keys [1]: [s_suppkey#X]
Right keys [1]: [l_suppkey#X]
Join condition: None
-(67) Project
+(69) Project
Output [5]: [s_nationkey#X, l_orderkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Input [7]: [s_suppkey#X, s_nationkey#X, l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(68) Scan parquet
+(70) Scan parquet
Output [2]: [o_orderkey#X, o_custkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderkey), IsNotNull(o_custkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint>
-(69) Filter
+(71) Filter
Input [2]: [o_orderkey#X, o_custkey#X]
Condition : (isnotnull(o_orderkey#X) AND isnotnull(o_custkey#X))
-(70) BroadcastExchange
+(72) BroadcastExchange
Input [2]: [o_orderkey#X, o_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(71) BroadcastHashJoin
+(73) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [o_orderkey#X]
Join condition: None
-(72) Project
+(74) Project
Output [5]: [s_nationkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X, o_custkey#X]
Input [7]: [s_nationkey#X, l_orderkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X, o_orderkey#X, o_custkey#X]
-(73) Scan parquet
+(75) Scan parquet
Output [2]: [c_custkey#X, c_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_custkey), IsNotNull(c_nationkey)]
ReadSchema: struct<c_custkey:bigint,c_nationkey:bigint>
-(74) Filter
+(76) Filter
Input [2]: [c_custkey#X, c_nationkey#X]
Condition : (isnotnull(c_custkey#X) AND isnotnull(c_nationkey#X))
-(75) BroadcastExchange
+(77) BroadcastExchange
Input [2]: [c_custkey#X, c_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(76) BroadcastHashJoin
+(78) BroadcastHashJoin
Left keys [1]: [o_custkey#X]
Right keys [1]: [c_custkey#X]
Join condition: None
-(77) Project
+(79) Project
Output [5]: [s_nationkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X, c_nationkey#X]
Input [7]: [s_nationkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X, o_custkey#X, c_custkey#X, c_nationkey#X]
-(78) Scan parquet
+(80) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey), Or(EqualTo(n_name,FRANCE),EqualTo(n_name,GERMANY))]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(79) Filter
+(81) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : (isnotnull(n_nationkey#X) AND ((n_name#X = FRANCE) OR (n_name#X = GERMANY)))
-(80) BroadcastExchange
+(82) BroadcastExchange
Input [2]: [n_nationkey#X, n_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(81) BroadcastHashJoin
+(83) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(82) Project
+(84) Project
Output [5]: [l_extendedprice#X, l_discount#X, l_shipdate#X, c_nationkey#X, n_name#X]
Input [7]: [s_nationkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X, c_nationkey#X, n_nationkey#X, n_name#X]
-(83) Scan parquet
+(85) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey), Or(EqualTo(n_name,GERMANY),EqualTo(n_name,FRANCE))]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(84) Filter
+(86) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : (isnotnull(n_nationkey#X) AND ((n_name#X = GERMANY) OR (n_name#X = FRANCE)))
-(85) BroadcastExchange
+(87) BroadcastExchange
Input [2]: [n_nationkey#X, n_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(86) BroadcastHashJoin
+(88) BroadcastHashJoin
Left keys [1]: [c_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: (((n_name#X = FRANCE) AND (n_name#X = GERMANY)) OR ((n_name#X = GERMANY) AND (n_name#X = FRANCE)))
-(87) Project
+(89) Project
Output [4]: [n_name#X AS supp_nation#X, n_name#X AS cust_nation#X, year(l_shipdate#X) AS l_year#X, CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) AS volume#X]
Input [7]: [l_extendedprice#X, l_discount#X, l_shipdate#X, c_nationkey#X, n_name#X, n_nationkey#X, n_name#X]
-(88) HashAggregate
+(90) HashAggregate
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, volume#X]
Keys [3]: [supp_nation#X, cust_nation#X, l_year#X]
Functions [1]: [partial_sum(volume#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
-(89) Exchange
+(91) Exchange
Input [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(supp_nation#X, cust_nation#X, l_year#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(90) HashAggregate
+(92) HashAggregate
Input [5]: [supp_nation#X, cust_nation#X, l_year#X, sum#X, isEmpty#X]
Keys [3]: [supp_nation#X, cust_nation#X, l_year#X]
Functions [1]: [sum(volume#X)]
Aggregate Attributes [1]: [sum(volume#X)#X]
Results [4]: [supp_nation#X, cust_nation#X, l_year#X, sum(volume#X)#X AS revenue#X]
-(91) Exchange
+(93) Exchange
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: rangepartitioning(supp_nation#X ASC NULLS FIRST, cust_nation#X ASC NULLS FIRST, l_year#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(92) Sort
+(94) Sort
Input [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: [supp_nation#X ASC NULLS FIRST, cust_nation#X ASC NULLS FIRST, l_year#X ASC NULLS FIRST], true, 0
-(93) AdaptiveSparkPlan
+(95) AdaptiveSparkPlan
Output [4]: [supp_nation#X, cust_nation#X, l_year#X, revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/8.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/8.txt
index 7e20b94ed..045e283bd 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/8.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/8.txt
@@ -1,116 +1,118 @@
== Physical Plan ==
-AdaptiveSparkPlan (129)
+AdaptiveSparkPlan (131)
+- == Final Plan ==
- VeloxColumnarToRowExec (84)
- +- ^ SortExecTransformer (82)
- +- ^ InputIteratorTransformer (81)
- +- ShuffleQueryStage (79)
- +- ColumnarExchange (78)
- +- ^ ProjectExecTransformer (76)
- +- ^ RegularHashAggregateExecTransformer (75)
- +- ^ InputIteratorTransformer (74)
- +- ShuffleQueryStage (72)
- +- ColumnarExchange (71)
- +- ^ ProjectExecTransformer (69)
- +- ^ FlushableHashAggregateExecTransformer (68)
- +- ^ ProjectExecTransformer (67)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (66)
- :- ^ ProjectExecTransformer (57)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (56)
- : :- ^ ProjectExecTransformer (48)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (47)
- : : :- ^ ProjectExecTransformer (39)
- : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (38)
- : : : :- ^ ProjectExecTransformer (30)
- : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (29)
- : : : : :- ^ ProjectExecTransformer (21)
- : : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
- : : : : : :- ^ ProjectExecTransformer (12)
- : : : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
- : : : : : : :- ^ InputIteratorTransformer (8)
- : : : : : : : +- BroadcastQueryStage (6)
- : : : : : : : +- ColumnarBroadcastExchange (5)
- : : : : : : : +- ^ ProjectExecTransformer (3)
- : : : : : : : +- ^ NoopFilter (2)
- : : : : : : : +- ^ Scan parquet (1)
- : : : : : : +- ^ NoopFilter (10)
- : : : : : : +- ^ Scan parquet (9)
- : : : : : +- ^ InputIteratorTransformer (19)
- : : : : : +- BroadcastQueryStage (17)
- : : : : : +- ColumnarBroadcastExchange (16)
- : : : : : +- ^ NoopFilter (14)
- : : : : : +- ^ Scan parquet (13)
- : : : : +- ^ InputIteratorTransformer (28)
- : : : : +- BroadcastQueryStage (26)
- : : : : +- ColumnarBroadcastExchange (25)
- : : : : +- ^ NoopFilter (23)
- : : : : +- ^ Scan parquet (22)
- : : : +- ^ InputIteratorTransformer (37)
- : : : +- BroadcastQueryStage (35)
- : : : +- ColumnarBroadcastExchange (34)
- : : : +- ^ NoopFilter (32)
- : : : +- ^ Scan parquet (31)
- : : +- ^ InputIteratorTransformer (46)
- : : +- BroadcastQueryStage (44)
- : : +- ColumnarBroadcastExchange (43)
- : : +- ^ NoopFilter (41)
- : : +- ^ Scan parquet (40)
- : +- ^ InputIteratorTransformer (55)
- : +- BroadcastQueryStage (53)
- : +- ColumnarBroadcastExchange (52)
- : +- ^ NoopFilter (50)
- : +- ^ Scan parquet (49)
- +- ^ InputIteratorTransformer (65)
- +- BroadcastQueryStage (63)
- +- ColumnarBroadcastExchange (62)
- +- ^ ProjectExecTransformer (60)
- +- ^ NoopFilter (59)
- +- ^ Scan parquet (58)
+ VeloxColumnarToRowExec (86)
+ +- ^ SortExecTransformer (84)
+ +- ^ InputIteratorTransformer (83)
+ +- ShuffleQueryStage (81)
+ +- ColumnarExchange (80)
+ +- VeloxAppendBatches (79)
+ +- ^ ProjectExecTransformer (77)
+ +- ^ RegularHashAggregateExecTransformer (76)
+ +- ^ InputIteratorTransformer (75)
+ +- ShuffleQueryStage (73)
+ +- ColumnarExchange (72)
+ +- VeloxAppendBatches (71)
+ +- ^ ProjectExecTransformer (69)
+ +- ^ FlushableHashAggregateExecTransformer (68)
+ +- ^ ProjectExecTransformer (67)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (66)
+ :- ^ ProjectExecTransformer (57)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (56)
+ : :- ^ ProjectExecTransformer (48)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (47)
+ : : :- ^ ProjectExecTransformer (39)
+ : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (38)
+ : : : :- ^ ProjectExecTransformer (30)
+ : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (29)
+ : : : : :- ^ ProjectExecTransformer (21)
+ : : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
+ : : : : : :- ^ ProjectExecTransformer (12)
+ : : : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
+ : : : : : : :- ^ InputIteratorTransformer (8)
+ : : : : : : : +- BroadcastQueryStage (6)
+ : : : : : : : +- ColumnarBroadcastExchange (5)
+ : : : : : : : +- ^ ProjectExecTransformer (3)
+ : : : : : : : +- ^ NoopFilter (2)
+ : : : : : : : +- ^ Scan parquet (1)
+ : : : : : : +- ^ NoopFilter (10)
+ : : : : : : +- ^ Scan parquet (9)
+ : : : : : +- ^ InputIteratorTransformer (19)
+ : : : : : +- BroadcastQueryStage (17)
+ : : : : : +- ColumnarBroadcastExchange (16)
+ : : : : : +- ^ NoopFilter (14)
+ : : : : : +- ^ Scan parquet (13)
+ : : : : +- ^ InputIteratorTransformer (28)
+ : : : : +- BroadcastQueryStage (26)
+ : : : : +- ColumnarBroadcastExchange (25)
+ : : : : +- ^ NoopFilter (23)
+ : : : : +- ^ Scan parquet (22)
+ : : : +- ^ InputIteratorTransformer (37)
+ : : : +- BroadcastQueryStage (35)
+ : : : +- ColumnarBroadcastExchange (34)
+ : : : +- ^ NoopFilter (32)
+ : : : +- ^ Scan parquet (31)
+ : : +- ^ InputIteratorTransformer (46)
+ : : +- BroadcastQueryStage (44)
+ : : +- ColumnarBroadcastExchange (43)
+ : : +- ^ NoopFilter (41)
+ : : +- ^ Scan parquet (40)
+ : +- ^ InputIteratorTransformer (55)
+ : +- BroadcastQueryStage (53)
+ : +- ColumnarBroadcastExchange (52)
+ : +- ^ NoopFilter (50)
+ : +- ^ Scan parquet (49)
+ +- ^ InputIteratorTransformer (65)
+ +- BroadcastQueryStage (63)
+ +- ColumnarBroadcastExchange (62)
+ +- ^ ProjectExecTransformer (60)
+ +- ^ NoopFilter (59)
+ +- ^ Scan parquet (58)
+- == Initial Plan ==
- Sort (128)
- +- Exchange (127)
- +- HashAggregate (126)
- +- Exchange (125)
- +- HashAggregate (124)
- +- Project (123)
- +- BroadcastHashJoin Inner BuildRight (122)
- :- Project (117)
- : +- BroadcastHashJoin Inner BuildRight (116)
- : :- Project (112)
- : : +- BroadcastHashJoin Inner BuildRight (111)
- : : :- Project (107)
- : : : +- BroadcastHashJoin Inner BuildRight (106)
- : : : :- Project (102)
- : : : : +- BroadcastHashJoin Inner BuildRight (101)
- : : : : :- Project (97)
- : : : : : +- BroadcastHashJoin Inner BuildRight (96)
- : : : : : :- Project (92)
- : : : : : : +- BroadcastHashJoin Inner BuildLeft (91)
- : : : : : : :- BroadcastExchange (88)
- : : : : : : : +- Project (87)
- : : : : : : : +- Filter (86)
- : : : : : : : +- Scan parquet (85)
- : : : : : : +- Filter (90)
- : : : : : : +- Scan parquet (89)
- : : : : : +- BroadcastExchange (95)
- : : : : : +- Filter (94)
- : : : : : +- Scan parquet (93)
- : : : : +- BroadcastExchange (100)
- : : : : +- Filter (99)
- : : : : +- Scan parquet (98)
- : : : +- BroadcastExchange (105)
- : : : +- Filter (104)
- : : : +- Scan parquet (103)
- : : +- BroadcastExchange (110)
- : : +- Filter (109)
- : : +- Scan parquet (108)
- : +- BroadcastExchange (115)
- : +- Filter (114)
- : +- Scan parquet (113)
- +- BroadcastExchange (121)
- +- Project (120)
- +- Filter (119)
- +- Scan parquet (118)
+ Sort (130)
+ +- Exchange (129)
+ +- HashAggregate (128)
+ +- Exchange (127)
+ +- HashAggregate (126)
+ +- Project (125)
+ +- BroadcastHashJoin Inner BuildRight (124)
+ :- Project (119)
+ : +- BroadcastHashJoin Inner BuildRight (118)
+ : :- Project (114)
+ : : +- BroadcastHashJoin Inner BuildRight (113)
+ : : :- Project (109)
+ : : : +- BroadcastHashJoin Inner BuildRight (108)
+ : : : :- Project (104)
+ : : : : +- BroadcastHashJoin Inner BuildRight (103)
+ : : : : :- Project (99)
+ : : : : : +- BroadcastHashJoin Inner BuildRight (98)
+ : : : : : :- Project (94)
+ : : : : : : +- BroadcastHashJoin Inner BuildLeft (93)
+ : : : : : : :- BroadcastExchange (90)
+ : : : : : : : +- Project (89)
+ : : : : : : : +- Filter (88)
+ : : : : : : : +- Scan parquet (87)
+ : : : : : : +- Filter (92)
+ : : : : : : +- Scan parquet (91)
+ : : : : : +- BroadcastExchange (97)
+ : : : : : +- Filter (96)
+ : : : : : +- Scan parquet (95)
+ : : : : +- BroadcastExchange (102)
+ : : : : +- Filter (101)
+ : : : : +- Scan parquet (100)
+ : : : +- BroadcastExchange (107)
+ : : : +- Filter (106)
+ : : : +- Scan parquet (105)
+ : : +- BroadcastExchange (112)
+ : : +- Filter (111)
+ : : +- Scan parquet (110)
+ : +- BroadcastExchange (117)
+ : +- Filter (116)
+ : +- Scan parquet (115)
+ +- BroadcastExchange (123)
+ +- Project (122)
+ +- Filter (121)
+ +- Scan parquet (120)
(1) Scan parquet
@@ -413,273 +415,281 @@ Input [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Input [6]: [hash_partition_key#X, o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: false
-(71) ColumnarExchange
+(71) VeloxAppendBatches
+Input [6]: [hash_partition_key#X, o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
+Arguments: X
+
+(72) ColumnarExchange
Input [6]: [hash_partition_key#X, o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(o_year#X, 1), ENSURE_REQUIREMENTS, [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(72) ShuffleQueryStage
+(73) ShuffleQueryStage
Output [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: X
-(73) InputAdapter
+(74) InputAdapter
Input [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
-(74) InputIteratorTransformer
+(75) InputIteratorTransformer
Input [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
-(75) RegularHashAggregateExecTransformer
+(76) RegularHashAggregateExecTransformer
Input [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Keys [1]: [o_year#X]
Functions [2]: [sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END), sum(volume#X)]
Aggregate Attributes [2]: [sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END)#X, sum(volume#X)#X]
Results [3]: [o_year#X, sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END)#X, sum(volume#X)#X]
-(76) ProjectExecTransformer
+(77) ProjectExecTransformer
Output [2]: [o_year#X, CheckOverflow((promote_precision(sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END)#X) / promote_precision(sum(volume#X)#X)), DecimalType(38,6), true) AS mkt_share#X]
Input [3]: [o_year#X, sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END)#X, sum(volume#X)#X]
-(77) WholeStageCodegenTransformer (X)
+(78) WholeStageCodegenTransformer (X)
Input [2]: [o_year#X, mkt_share#X]
Arguments: false
-(78) ColumnarExchange
+(79) VeloxAppendBatches
+Input [2]: [o_year#X, mkt_share#X]
+Arguments: X
+
+(80) ColumnarExchange
Input [2]: [o_year#X, mkt_share#X]
Arguments: rangepartitioning(o_year#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(79) ShuffleQueryStage
+(81) ShuffleQueryStage
Output [2]: [o_year#X, mkt_share#X]
Arguments: X
-(80) InputAdapter
+(82) InputAdapter
Input [2]: [o_year#X, mkt_share#X]
-(81) InputIteratorTransformer
+(83) InputIteratorTransformer
Input [2]: [o_year#X, mkt_share#X]
-(82) SortExecTransformer
+(84) SortExecTransformer
Input [2]: [o_year#X, mkt_share#X]
Arguments: [o_year#X ASC NULLS FIRST], true, 0
-(83) WholeStageCodegenTransformer (X)
+(85) WholeStageCodegenTransformer (X)
Input [2]: [o_year#X, mkt_share#X]
Arguments: false
-(84) VeloxColumnarToRowExec
+(86) VeloxColumnarToRowExec
Input [2]: [o_year#X, mkt_share#X]
-(85) Scan parquet
+(87) Scan parquet
Output [2]: [p_partkey#X, p_type#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_type), EqualTo(p_type,ECONOMY ANODIZED STEEL), IsNotNull(p_partkey)]
ReadSchema: struct<p_partkey:bigint,p_type:string>
-(86) Filter
+(88) Filter
Input [2]: [p_partkey#X, p_type#X]
Condition : ((isnotnull(p_type#X) AND (p_type#X = ECONOMY ANODIZED STEEL)) AND isnotnull(p_partkey#X))
-(87) Project
+(89) Project
Output [1]: [p_partkey#X]
Input [2]: [p_partkey#X, p_type#X]
-(88) BroadcastExchange
+(90) BroadcastExchange
Input [1]: [p_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(89) Scan parquet
+(91) Scan parquet
Output [5]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_partkey), IsNotNull(l_suppkey), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_partkey:bigint,l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2)>
-(90) Filter
+(92) Filter
Input [5]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Condition : ((isnotnull(l_partkey#X) AND isnotnull(l_suppkey#X)) AND isnotnull(l_orderkey#X))
-(91) BroadcastHashJoin
+(93) BroadcastHashJoin
Left keys [1]: [p_partkey#X]
Right keys [1]: [l_partkey#X]
Join condition: None
-(92) Project
+(94) Project
Output [4]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
Input [6]: [p_partkey#X, l_orderkey#X, l_partkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X]
-(93) Scan parquet
+(95) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(94) Filter
+(96) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(95) BroadcastExchange
+(97) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(96) BroadcastHashJoin
+(98) BroadcastHashJoin
Left keys [1]: [l_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(97) Project
+(99) Project
Output [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, s_nationkey#X]
Input [6]: [l_orderkey#X, l_suppkey#X, l_extendedprice#X, l_discount#X, s_suppkey#X, s_nationkey#X]
-(98) Scan parquet
+(100) Scan parquet
Output [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderdate), GreaterThanOrEqual(o_orderdate,1995-01-01), LessThanOrEqual(o_orderdate,1996-12-31), IsNotNull(o_orderkey), IsNotNull(o_custkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_orderdate:date>
-(99) Filter
+(101) Filter
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Condition : ((((isnotnull(o_orderdate#X) AND (o_orderdate#X >= 1995-01-01)) AND (o_orderdate#X <= 1996-12-31)) AND isnotnull(o_orderkey#X)) AND isnotnull(o_custkey#X))
-(100) BroadcastExchange
+(102) BroadcastExchange
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(101) BroadcastHashJoin
+(103) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [o_orderkey#X]
Join condition: None
-(102) Project
+(104) Project
Output [5]: [l_extendedprice#X, l_discount#X, s_nationkey#X, o_custkey#X, o_orderdate#X]
Input [7]: [l_orderkey#X, l_extendedprice#X, l_discount#X, s_nationkey#X, o_orderkey#X, o_custkey#X, o_orderdate#X]
-(103) Scan parquet
+(105) Scan parquet
Output [2]: [c_custkey#X, c_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_custkey), IsNotNull(c_nationkey)]
ReadSchema: struct<c_custkey:bigint,c_nationkey:bigint>
-(104) Filter
+(106) Filter
Input [2]: [c_custkey#X, c_nationkey#X]
Condition : (isnotnull(c_custkey#X) AND isnotnull(c_nationkey#X))
-(105) BroadcastExchange
+(107) BroadcastExchange
Input [2]: [c_custkey#X, c_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(106) BroadcastHashJoin
+(108) BroadcastHashJoin
Left keys [1]: [o_custkey#X]
Right keys [1]: [c_custkey#X]
Join condition: None
-(107) Project
+(109) Project
Output [5]: [l_extendedprice#X, l_discount#X, s_nationkey#X, o_orderdate#X, c_nationkey#X]
Input [7]: [l_extendedprice#X, l_discount#X, s_nationkey#X, o_custkey#X, o_orderdate#X, c_custkey#X, c_nationkey#X]
-(108) Scan parquet
+(110) Scan parquet
Output [2]: [n_nationkey#X, n_regionkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey), IsNotNull(n_regionkey)]
ReadSchema: struct<n_nationkey:bigint,n_regionkey:bigint>
-(109) Filter
+(111) Filter
Input [2]: [n_nationkey#X, n_regionkey#X]
Condition : (isnotnull(n_nationkey#X) AND isnotnull(n_regionkey#X))
-(110) BroadcastExchange
+(112) BroadcastExchange
Input [2]: [n_nationkey#X, n_regionkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(111) BroadcastHashJoin
+(113) BroadcastHashJoin
Left keys [1]: [c_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(112) Project
+(114) Project
Output [5]: [l_extendedprice#X, l_discount#X, s_nationkey#X, o_orderdate#X, n_regionkey#X]
Input [7]: [l_extendedprice#X, l_discount#X, s_nationkey#X, o_orderdate#X, c_nationkey#X, n_nationkey#X, n_regionkey#X]
-(113) Scan parquet
+(115) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(114) Filter
+(116) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : isnotnull(n_nationkey#X)
-(115) BroadcastExchange
+(117) BroadcastExchange
Input [2]: [n_nationkey#X, n_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(116) BroadcastHashJoin
+(118) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(117) Project
+(119) Project
Output [5]: [l_extendedprice#X, l_discount#X, o_orderdate#X, n_regionkey#X, n_name#X]
Input [7]: [l_extendedprice#X, l_discount#X, s_nationkey#X, o_orderdate#X, n_regionkey#X, n_nationkey#X, n_name#X]
-(118) Scan parquet
+(120) Scan parquet
Output [2]: [r_regionkey#X, r_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(r_name), EqualTo(r_name,AMERICA), IsNotNull(r_regionkey)]
ReadSchema: struct<r_regionkey:bigint,r_name:string>
-(119) Filter
+(121) Filter
Input [2]: [r_regionkey#X, r_name#X]
Condition : ((isnotnull(r_name#X) AND (r_name#X = AMERICA)) AND isnotnull(r_regionkey#X))
-(120) Project
+(122) Project
Output [1]: [r_regionkey#X]
Input [2]: [r_regionkey#X, r_name#X]
-(121) BroadcastExchange
+(123) BroadcastExchange
Input [1]: [r_regionkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(122) BroadcastHashJoin
+(124) BroadcastHashJoin
Left keys [1]: [n_regionkey#X]
Right keys [1]: [r_regionkey#X]
Join condition: None
-(123) Project
+(125) Project
Output [3]: [year(o_orderdate#X) AS o_year#X, CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) AS volume#X, n_name#X AS nation#X]
Input [6]: [l_extendedprice#X, l_discount#X, o_orderdate#X, n_regionkey#X, n_name#X, r_regionkey#X]
-(124) HashAggregate
+(126) HashAggregate
Input [3]: [o_year#X, volume#X, nation#X]
Keys [1]: [o_year#X]
Functions [2]: [partial_sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END), partial_sum(volume#X)]
Aggregate Attributes [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Results [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
-(125) Exchange
+(127) Exchange
Input [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(o_year#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(126) HashAggregate
+(128) HashAggregate
Input [5]: [o_year#X, sum#X, isEmpty#X, sum#X, isEmpty#X]
Keys [1]: [o_year#X]
Functions [2]: [sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END), sum(volume#X)]
Aggregate Attributes [2]: [sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END)#X, sum(volume#X)#X]
Results [2]: [o_year#X, CheckOverflow((promote_precision(sum(CASE WHEN (nation#X = BRAZIL) THEN volume#X ELSE 0.0000 END)#X) / promote_precision(sum(volume#X)#X)), DecimalType(38,6), true) AS mkt_share#X]
-(127) Exchange
+(129) Exchange
Input [2]: [o_year#X, mkt_share#X]
Arguments: rangepartitioning(o_year#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(128) Sort
+(130) Sort
Input [2]: [o_year#X, mkt_share#X]
Arguments: [o_year#X ASC NULLS FIRST], true, 0
-(129) AdaptiveSparkPlan
+(131) AdaptiveSparkPlan
Output [2]: [o_year#X, mkt_share#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/9.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/9.txt
index 8a30fbdfb..abdd1d7b9 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/9.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark32/9.txt
@@ -1,89 +1,91 @@
== Physical Plan ==
-AdaptiveSparkPlan (98)
+AdaptiveSparkPlan (100)
+- == Final Plan ==
- VeloxColumnarToRowExec (64)
- +- ^ SortExecTransformer (62)
- +- ^ InputIteratorTransformer (61)
- +- ShuffleQueryStage (59)
- +- ColumnarExchange (58)
- +- ^ RegularHashAggregateExecTransformer (56)
- +- ^ InputIteratorTransformer (55)
- +- ShuffleQueryStage (53)
- +- ColumnarExchange (52)
- +- ^ ProjectExecTransformer (50)
- +- ^ FlushableHashAggregateExecTransformer (49)
- +- ^ ProjectExecTransformer (48)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (47)
- :- ^ ProjectExecTransformer (39)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (38)
- : :- ^ ProjectExecTransformer (30)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (29)
- : : :- ^ ProjectExecTransformer (21)
- : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
- : : : :- ^ ProjectExecTransformer (12)
- : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
- : : : : :- ^ InputIteratorTransformer (8)
- : : : : : +- BroadcastQueryStage (6)
- : : : : : +- ColumnarBroadcastExchange (5)
- : : : : : +- ^ ProjectExecTransformer (3)
- : : : : : +- ^ NoopFilter (2)
- : : : : : +- ^ Scan parquet (1)
- : : : : +- ^ NoopFilter (10)
- : : : : +- ^ Scan parquet (9)
- : : : +- ^ InputIteratorTransformer (19)
- : : : +- BroadcastQueryStage (17)
- : : : +- ColumnarBroadcastExchange (16)
- : : : +- ^ NoopFilter (14)
- : : : +- ^ Scan parquet (13)
- : : +- ^ InputIteratorTransformer (28)
- : : +- BroadcastQueryStage (26)
- : : +- ColumnarBroadcastExchange (25)
- : : +- ^ NoopFilter (23)
- : : +- ^ Scan parquet (22)
- : +- ^ InputIteratorTransformer (37)
- : +- BroadcastQueryStage (35)
- : +- ColumnarBroadcastExchange (34)
- : +- ^ NoopFilter (32)
- : +- ^ Scan parquet (31)
- +- ^ InputIteratorTransformer (46)
- +- BroadcastQueryStage (44)
- +- ColumnarBroadcastExchange (43)
- +- ^ NoopFilter (41)
- +- ^ Scan parquet (40)
+ VeloxColumnarToRowExec (66)
+ +- ^ SortExecTransformer (64)
+ +- ^ InputIteratorTransformer (63)
+ +- ShuffleQueryStage (61)
+ +- ColumnarExchange (60)
+ +- VeloxAppendBatches (59)
+ +- ^ RegularHashAggregateExecTransformer (57)
+ +- ^ InputIteratorTransformer (56)
+ +- ShuffleQueryStage (54)
+ +- ColumnarExchange (53)
+ +- VeloxAppendBatches (52)
+ +- ^ ProjectExecTransformer (50)
+ +- ^ FlushableHashAggregateExecTransformer (49)
+ +- ^ ProjectExecTransformer (48)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (47)
+ :- ^ ProjectExecTransformer (39)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (38)
+ : :- ^ ProjectExecTransformer (30)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (29)
+ : : :- ^ ProjectExecTransformer (21)
+ : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
+ : : : :- ^ ProjectExecTransformer (12)
+ : : : : +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
+ : : : : :- ^ InputIteratorTransformer (8)
+ : : : : : +- BroadcastQueryStage (6)
+ : : : : : +- ColumnarBroadcastExchange (5)
+ : : : : : +- ^ ProjectExecTransformer (3)
+ : : : : : +- ^ NoopFilter (2)
+ : : : : : +- ^ Scan parquet (1)
+ : : : : +- ^ NoopFilter (10)
+ : : : : +- ^ Scan parquet (9)
+ : : : +- ^ InputIteratorTransformer (19)
+ : : : +- BroadcastQueryStage (17)
+ : : : +- ColumnarBroadcastExchange (16)
+ : : : +- ^ NoopFilter (14)
+ : : : +- ^ Scan parquet (13)
+ : : +- ^ InputIteratorTransformer (28)
+ : : +- BroadcastQueryStage (26)
+ : : +- ColumnarBroadcastExchange (25)
+ : : +- ^ NoopFilter (23)
+ : : +- ^ Scan parquet (22)
+ : +- ^ InputIteratorTransformer (37)
+ : +- BroadcastQueryStage (35)
+ : +- ColumnarBroadcastExchange (34)
+ : +- ^ NoopFilter (32)
+ : +- ^ Scan parquet (31)
+ +- ^ InputIteratorTransformer (46)
+ +- BroadcastQueryStage (44)
+ +- ColumnarBroadcastExchange (43)
+ +- ^ NoopFilter (41)
+ +- ^ Scan parquet (40)
+- == Initial Plan ==
- Sort (97)
- +- Exchange (96)
- +- HashAggregate (95)
- +- Exchange (94)
- +- HashAggregate (93)
- +- Project (92)
- +- BroadcastHashJoin Inner BuildRight (91)
- :- Project (87)
- : +- BroadcastHashJoin Inner BuildRight (86)
- : :- Project (82)
- : : +- BroadcastHashJoin Inner BuildRight (81)
- : : :- Project (77)
- : : : +- BroadcastHashJoin Inner BuildRight (76)
- : : : :- Project (72)
- : : : : +- BroadcastHashJoin Inner BuildLeft (71)
- : : : : :- BroadcastExchange (68)
- : : : : : +- Project (67)
- : : : : : +- Filter (66)
- : : : : : +- Scan parquet (65)
- : : : : +- Filter (70)
- : : : : +- Scan parquet (69)
- : : : +- BroadcastExchange (75)
- : : : +- Filter (74)
- : : : +- Scan parquet (73)
- : : +- BroadcastExchange (80)
- : : +- Filter (79)
- : : +- Scan parquet (78)
- : +- BroadcastExchange (85)
- : +- Filter (84)
- : +- Scan parquet (83)
- +- BroadcastExchange (90)
- +- Filter (89)
- +- Scan parquet (88)
+ Sort (99)
+ +- Exchange (98)
+ +- HashAggregate (97)
+ +- Exchange (96)
+ +- HashAggregate (95)
+ +- Project (94)
+ +- BroadcastHashJoin Inner BuildRight (93)
+ :- Project (89)
+ : +- BroadcastHashJoin Inner BuildRight (88)
+ : :- Project (84)
+ : : +- BroadcastHashJoin Inner BuildRight (83)
+ : : :- Project (79)
+ : : : +- BroadcastHashJoin Inner BuildRight (78)
+ : : : :- Project (74)
+ : : : : +- BroadcastHashJoin Inner BuildLeft (73)
+ : : : : :- BroadcastExchange (70)
+ : : : : : +- Project (69)
+ : : : : : +- Filter (68)
+ : : : : : +- Scan parquet (67)
+ : : : : +- Filter (72)
+ : : : : +- Scan parquet (71)
+ : : : +- BroadcastExchange (77)
+ : : : +- Filter (76)
+ : : : +- Scan parquet (75)
+ : : +- BroadcastExchange (82)
+ : : +- Filter (81)
+ : : +- Scan parquet (80)
+ : +- BroadcastExchange (87)
+ : +- Filter (86)
+ : +- Scan parquet (85)
+ +- BroadcastExchange (92)
+ +- Filter (91)
+ +- Scan parquet (90)
(1) Scan parquet
@@ -306,217 +308,225 @@ Input [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
Input [5]: [hash_partition_key#X, nation#X, o_year#X, sum#X, isEmpty#X]
Arguments: false
-(52) ColumnarExchange
+(52) VeloxAppendBatches
+Input [5]: [hash_partition_key#X, nation#X, o_year#X, sum#X, isEmpty#X]
+Arguments: X
+
+(53) ColumnarExchange
Input [5]: [hash_partition_key#X, nation#X, o_year#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(nation#X, o_year#X, 1), ENSURE_REQUIREMENTS, [nation#X, o_year#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(53) ShuffleQueryStage
+(54) ShuffleQueryStage
Output [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
Arguments: X
-(54) InputAdapter
+(55) InputAdapter
Input [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
-(55) InputIteratorTransformer
+(56) InputIteratorTransformer
Input [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
-(56) RegularHashAggregateExecTransformer
+(57) RegularHashAggregateExecTransformer
Input [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
Keys [2]: [nation#X, o_year#X]
Functions [1]: [sum(amount#X)]
Aggregate Attributes [1]: [sum(amount#X)#X]
Results [3]: [nation#X, o_year#X, sum(amount#X)#X AS sum_profit#X]
-(57) WholeStageCodegenTransformer (X)
+(58) WholeStageCodegenTransformer (X)
Input [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: false
-(58) ColumnarExchange
+(59) VeloxAppendBatches
+Input [3]: [nation#X, o_year#X, sum_profit#X]
+Arguments: X
+
+(60) ColumnarExchange
Input [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: rangepartitioning(nation#X ASC NULLS FIRST, o_year#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(59) ShuffleQueryStage
+(61) ShuffleQueryStage
Output [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: X
-(60) InputAdapter
+(62) InputAdapter
Input [3]: [nation#X, o_year#X, sum_profit#X]
-(61) InputIteratorTransformer
+(63) InputIteratorTransformer
Input [3]: [nation#X, o_year#X, sum_profit#X]
-(62) SortExecTransformer
+(64) SortExecTransformer
Input [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: [nation#X ASC NULLS FIRST, o_year#X DESC NULLS LAST], true, 0
-(63) WholeStageCodegenTransformer (X)
+(65) WholeStageCodegenTransformer (X)
Input [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: false
-(64) VeloxColumnarToRowExec
+(66) VeloxColumnarToRowExec
Input [3]: [nation#X, o_year#X, sum_profit#X]
-(65) Scan parquet
+(67) Scan parquet
Output [2]: [p_partkey#X, p_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_name), StringContains(p_name,green), IsNotNull(p_partkey)]
ReadSchema: struct<p_partkey:bigint,p_name:string>
-(66) Filter
+(68) Filter
Input [2]: [p_partkey#X, p_name#X]
Condition : ((isnotnull(p_name#X) AND Contains(p_name#X, green)) AND isnotnull(p_partkey#X))
-(67) Project
+(69) Project
Output [1]: [p_partkey#X]
Input [2]: [p_partkey#X, p_name#X]
-(68) BroadcastExchange
+(70) BroadcastExchange
Input [1]: [p_partkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(69) Scan parquet
+(71) Scan parquet
Output [6]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_partkey), IsNotNull(l_suppkey), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_partkey:bigint,l_suppkey:bigint,l_quantity:decimal(12,2),l_extendedprice:decimal(12,2),l_discount:decimal(12,2)>
-(70) Filter
+(72) Filter
Input [6]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X]
Condition : ((isnotnull(l_partkey#X) AND isnotnull(l_suppkey#X)) AND isnotnull(l_orderkey#X))
-(71) BroadcastHashJoin
+(73) BroadcastHashJoin
Left keys [1]: [p_partkey#X]
Right keys [1]: [l_partkey#X]
Join condition: None
-(72) Project
+(74) Project
Output [6]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X]
Input [7]: [p_partkey#X, l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X]
-(73) Scan parquet
+(75) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(74) Filter
+(76) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(75) BroadcastExchange
+(77) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(76) BroadcastHashJoin
+(78) BroadcastHashJoin
Left keys [1]: [l_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(77) Project
+(79) Project
Output [7]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, s_nationkey#X]
Input [8]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, s_suppkey#X, s_nationkey#X]
-(78) Scan parquet
+(80) Scan parquet
Output [3]: [ps_partkey#X, ps_suppkey#X, ps_supplycost#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_suppkey), IsNotNull(ps_partkey)]
ReadSchema: struct<ps_partkey:bigint,ps_suppkey:bigint,ps_supplycost:decimal(12,2)>
-(79) Filter
+(81) Filter
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_supplycost#X]
Condition : (isnotnull(ps_suppkey#X) AND isnotnull(ps_partkey#X))
-(80) BroadcastExchange
+(82) BroadcastExchange
Input [3]: [ps_partkey#X, ps_suppkey#X, ps_supplycost#X]
Arguments: HashedRelationBroadcastMode(List(input[1, bigint, false], input[0, bigint, false]),false), [plan_id=X]
-(81) BroadcastHashJoin
+(83) BroadcastHashJoin
Left keys [2]: [l_suppkey#X, l_partkey#X]
Right keys [2]: [ps_suppkey#X, ps_partkey#X]
Join condition: None
-(82) Project
+(84) Project
Output [6]: [l_orderkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, s_nationkey#X, ps_supplycost#X]
Input [10]: [l_orderkey#X, l_partkey#X, l_suppkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, s_nationkey#X, ps_partkey#X, ps_suppkey#X, ps_supplycost#X]
-(83) Scan parquet
+(85) Scan parquet
Output [2]: [o_orderkey#X, o_orderdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_orderdate:date>
-(84) Filter
+(86) Filter
Input [2]: [o_orderkey#X, o_orderdate#X]
Condition : isnotnull(o_orderkey#X)
-(85) BroadcastExchange
+(87) BroadcastExchange
Input [2]: [o_orderkey#X, o_orderdate#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(86) BroadcastHashJoin
+(88) BroadcastHashJoin
Left keys [1]: [l_orderkey#X]
Right keys [1]: [o_orderkey#X]
Join condition: None
-(87) Project
+(89) Project
Output [6]: [l_quantity#X, l_extendedprice#X, l_discount#X, s_nationkey#X, ps_supplycost#X, o_orderdate#X]
Input [8]: [l_orderkey#X, l_quantity#X, l_extendedprice#X, l_discount#X, s_nationkey#X, ps_supplycost#X, o_orderkey#X, o_orderdate#X]
-(88) Scan parquet
+(90) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(89) Filter
+(91) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : isnotnull(n_nationkey#X)
-(90) BroadcastExchange
+(92) BroadcastExchange
Input [2]: [n_nationkey#X, n_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(91) BroadcastHashJoin
+(93) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(92) Project
+(94) Project
Output [3]: [n_name#X AS nation#X, year(o_orderdate#X) AS o_year#X, CheckOverflow((promote_precision(cast(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2), true))), DecimalType(26,4), true) as decimal(27,4))) - promote_precision(cast(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(l_quantity#X)), DecimalType(25,4), true) as d [...]
Input [8]: [l_quantity#X, l_extendedprice#X, l_discount#X, s_nationkey#X, ps_supplycost#X, o_orderdate#X, n_nationkey#X, n_name#X]
-(93) HashAggregate
+(95) HashAggregate
Input [3]: [nation#X, o_year#X, amount#X]
Keys [2]: [nation#X, o_year#X]
Functions [1]: [partial_sum(amount#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
-(94) Exchange
+(96) Exchange
Input [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(nation#X, o_year#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(95) HashAggregate
+(97) HashAggregate
Input [4]: [nation#X, o_year#X, sum#X, isEmpty#X]
Keys [2]: [nation#X, o_year#X]
Functions [1]: [sum(amount#X)]
Aggregate Attributes [1]: [sum(amount#X)#X]
Results [3]: [nation#X, o_year#X, sum(amount#X)#X AS sum_profit#X]
-(96) Exchange
+(98) Exchange
Input [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: rangepartitioning(nation#X ASC NULLS FIRST, o_year#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(97) Sort
+(99) Sort
Input [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: [nation#X ASC NULLS FIRST, o_year#X DESC NULLS LAST], true, 0
-(98) AdaptiveSparkPlan
+(100) AdaptiveSparkPlan
Output [3]: [nation#X, o_year#X, sum_profit#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/1.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/1.txt
index 63b30bb5d..afea15af5 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/1.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/1.txt
@@ -1,29 +1,31 @@
== Physical Plan ==
-AdaptiveSparkPlan (28)
+AdaptiveSparkPlan (30)
+- == Final Plan ==
- VeloxColumnarToRowExec (19)
- +- ^ SortExecTransformer (17)
- +- ^ InputIteratorTransformer (16)
- +- ShuffleQueryStage (14), Statistics(X)
- +- ColumnarExchange (13)
- +- ^ RegularHashAggregateExecTransformer (11)
- +- ^ InputIteratorTransformer (10)
- +- ShuffleQueryStage (8), Statistics(X)
- +- ColumnarExchange (7)
- +- ^ ProjectExecTransformer (5)
- +- ^ FlushableHashAggregateExecTransformer (4)
- +- ^ ProjectExecTransformer (3)
- +- ^ NoopFilter (2)
- +- ^ Scan parquet (1)
+ VeloxColumnarToRowExec (21)
+ +- ^ SortExecTransformer (19)
+ +- ^ InputIteratorTransformer (18)
+ +- ShuffleQueryStage (16), Statistics(X)
+ +- ColumnarExchange (15)
+ +- VeloxAppendBatches (14)
+ +- ^ RegularHashAggregateExecTransformer (12)
+ +- ^ InputIteratorTransformer (11)
+ +- ShuffleQueryStage (9), Statistics(X)
+ +- ColumnarExchange (8)
+ +- VeloxAppendBatches (7)
+ +- ^ ProjectExecTransformer (5)
+ +- ^ FlushableHashAggregateExecTransformer (4)
+ +- ^ ProjectExecTransformer (3)
+ +- ^ NoopFilter (2)
+ +- ^ Scan parquet (1)
+- == Initial Plan ==
- Sort (27)
- +- Exchange (26)
- +- HashAggregate (25)
- +- Exchange (24)
- +- HashAggregate (23)
- +- Project (22)
- +- Filter (21)
- +- Scan parquet (20)
+ Sort (29)
+ +- Exchange (28)
+ +- HashAggregate (27)
+ +- Exchange (26)
+ +- HashAggregate (25)
+ +- Project (24)
+ +- Filter (23)
+ +- Scan parquet (22)
(1) Scan parquet
@@ -56,97 +58,105 @@ Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X,
Input [18]: [hash_partition_key#X, l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: false
-(7) ColumnarExchange
+(7) VeloxAppendBatches
+Input [18]: [hash_partition_key#X, l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
+Arguments: X
+
+(8) ColumnarExchange
Input [18]: [hash_partition_key#X, l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: hashpartitioning(l_returnflag#X, l_linestatus#X, 1), ENSURE_REQUIREMENTS, [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X], [plan_id=X], [id=#X]
-(8) ShuffleQueryStage
+(9) ShuffleQueryStage
Output [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: X
-(9) InputAdapter
+(10) InputAdapter
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
-(10) InputIteratorTransformer
+(11) InputIteratorTransformer
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
-(11) RegularHashAggregateExecTransformer
+(12) RegularHashAggregateExecTransformer
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Keys [2]: [l_returnflag#X, l_linestatus#X]
Functions [8]: [sum(l_quantity#X), sum(l_extendedprice#X), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4))), sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), D [...]
Aggregate Attributes [8]: [sum(l_quantity#X)#X, sum(l_extendedprice#X)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as de [...]
Results [10]: [l_returnflag#X, l_linestatus#X, sum(l_quantity#X)#X AS sum_qty#X, sum(l_extendedprice#X)#X AS sum_base_price#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS sum_disc_price#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_ [...]
-(12) WholeStageCodegenTransformer (X)
+(13) WholeStageCodegenTransformer (X)
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: false
-(13) ColumnarExchange
+(14) VeloxAppendBatches
+Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
+Arguments: X
+
+(15) ColumnarExchange
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: rangepartitioning(l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(14) ShuffleQueryStage
+(16) ShuffleQueryStage
Output [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: X
-(15) InputAdapter
+(17) InputAdapter
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
-(16) InputIteratorTransformer
+(18) InputIteratorTransformer
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
-(17) SortExecTransformer
+(19) SortExecTransformer
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: [l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST], true, 0
-(18) WholeStageCodegenTransformer (X)
+(20) WholeStageCodegenTransformer (X)
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: false
-(19) VeloxColumnarToRowExec
+(21) VeloxColumnarToRowExec
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
-(20) Scan parquet
+(22) Scan parquet
Output [7]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), LessThanOrEqual(l_shipdate,1998-09-02)]
ReadSchema: struct<l_quantity:decimal(12,2),l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_tax:decimal(12,2),l_returnflag:string,l_linestatus:string,l_shipdate:date>
-(21) Filter
+(23) Filter
Input [7]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X, l_shipdate#X]
Condition : (isnotnull(l_shipdate#X) AND (l_shipdate#X <= 1998-09-02))
-(22) Project
+(24) Project
Output [6]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X]
Input [7]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X, l_shipdate#X]
-(23) HashAggregate
+(25) HashAggregate
Input [6]: [l_quantity#X, l_extendedprice#X, l_discount#X, l_tax#X, l_returnflag#X, l_linestatus#X]
Keys [2]: [l_returnflag#X, l_linestatus#X]
Functions [8]: [partial_sum(l_quantity#X), partial_sum(l_extendedprice#X), partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4))), partial_sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_d [...]
Aggregate Attributes [15]: [sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Results [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
-(24) Exchange
+(26) Exchange
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Arguments: hashpartitioning(l_returnflag#X, l_linestatus#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(25) HashAggregate
+(27) HashAggregate
Input [17]: [l_returnflag#X, l_linestatus#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, isEmpty#X, sum#X, count#X, sum#X, count#X, sum#X, count#X, count#X]
Keys [2]: [l_returnflag#X, l_linestatus#X]
Functions [8]: [sum(l_quantity#X), sum(l_extendedprice#X), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4))), sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), D [...]
Aggregate Attributes [8]: [sum(l_quantity#X)#X, sum(l_extendedprice#X)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as de [...]
Results [10]: [l_returnflag#X, l_linestatus#X, sum(l_quantity#X)#X AS sum_qty#X, sum(l_extendedprice#X)#X AS sum_base_price#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS sum_disc_price#X, sum(CheckOverflow((promote_precision(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_ [...]
-(26) Exchange
+(28) Exchange
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: rangepartitioning(l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(27) Sort
+(29) Sort
Input [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: [l_returnflag#X ASC NULLS FIRST, l_linestatus#X ASC NULLS FIRST], true, 0
-(28) AdaptiveSparkPlan
+(30) AdaptiveSparkPlan
Output [10]: [l_returnflag#X, l_linestatus#X, sum_qty#X, sum_base_price#X, sum_disc_price#X, sum_charge#X, avg_qty#X, avg_price#X, avg_disc#X, count_order#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/10.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/10.txt
index 18c2ef9bc..7a509e435 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/10.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/10.txt
@@ -1,64 +1,65 @@
== Physical Plan ==
-AdaptiveSparkPlan (67)
+AdaptiveSparkPlan (68)
+- == Final Plan ==
- VeloxColumnarToRowExec (43)
- +- TakeOrderedAndProjectExecTransformer (42)
- +- ^ ProjectExecTransformer (40)
- +- ^ RegularHashAggregateExecTransformer (39)
- +- ^ InputIteratorTransformer (38)
- +- ShuffleQueryStage (36), Statistics(X)
- +- ColumnarExchange (35)
- +- ^ ProjectExecTransformer (33)
- +- ^ FlushableHashAggregateExecTransformer (32)
- +- ^ ProjectExecTransformer (31)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (30)
- :- ^ ProjectExecTransformer (22)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (21)
- : :- ^ ProjectExecTransformer (12)
- : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
- : : :- ^ NoopFilter (2)
- : : : +- ^ Scan parquet (1)
- : : +- ^ InputIteratorTransformer (10)
- : : +- BroadcastQueryStage (8), Statistics(X)
- : : +- ColumnarBroadcastExchange (7)
- : : +- ^ ProjectExecTransformer (5)
- : : +- ^ NoopFilter (4)
- : : +- ^ Scan parquet (3)
- : +- ^ InputIteratorTransformer (20)
- : +- BroadcastQueryStage (18), Statistics(X)
- : +- ColumnarBroadcastExchange (17)
- : +- ^ ProjectExecTransformer (15)
- : +- ^ NoopFilter (14)
- : +- ^ Scan parquet (13)
- +- ^ InputIteratorTransformer (29)
- +- BroadcastQueryStage (27), Statistics(X)
- +- ColumnarBroadcastExchange (26)
- +- ^ NoopFilter (24)
- +- ^ Scan parquet (23)
+ VeloxColumnarToRowExec (44)
+ +- TakeOrderedAndProjectExecTransformer (43)
+ +- ^ ProjectExecTransformer (41)
+ +- ^ RegularHashAggregateExecTransformer (40)
+ +- ^ InputIteratorTransformer (39)
+ +- ShuffleQueryStage (37), Statistics(X)
+ +- ColumnarExchange (36)
+ +- VeloxAppendBatches (35)
+ +- ^ ProjectExecTransformer (33)
+ +- ^ FlushableHashAggregateExecTransformer (32)
+ +- ^ ProjectExecTransformer (31)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (30)
+ :- ^ ProjectExecTransformer (22)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (21)
+ : :- ^ ProjectExecTransformer (12)
+ : : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
+ : : :- ^ NoopFilter (2)
+ : : : +- ^ Scan parquet (1)
+ : : +- ^ InputIteratorTransformer (10)
+ : : +- BroadcastQueryStage (8), Statistics(X)
+ : : +- ColumnarBroadcastExchange (7)
+ : : +- ^ ProjectExecTransformer (5)
+ : : +- ^ NoopFilter (4)
+ : : +- ^ Scan parquet (3)
+ : +- ^ InputIteratorTransformer (20)
+ : +- BroadcastQueryStage (18), Statistics(X)
+ : +- ColumnarBroadcastExchange (17)
+ : +- ^ ProjectExecTransformer (15)
+ : +- ^ NoopFilter (14)
+ : +- ^ Scan parquet (13)
+ +- ^ InputIteratorTransformer (29)
+ +- BroadcastQueryStage (27), Statistics(X)
+ +- ColumnarBroadcastExchange (26)
+ +- ^ NoopFilter (24)
+ +- ^ Scan parquet (23)
+- == Initial Plan ==
- TakeOrderedAndProject (66)
- +- HashAggregate (65)
- +- Exchange (64)
- +- HashAggregate (63)
- +- Project (62)
- +- BroadcastHashJoin Inner BuildRight (61)
- :- Project (57)
- : +- BroadcastHashJoin Inner BuildRight (56)
- : :- Project (51)
- : : +- BroadcastHashJoin Inner BuildRight (50)
- : : :- Filter (45)
- : : : +- Scan parquet (44)
- : : +- BroadcastExchange (49)
- : : +- Project (48)
- : : +- Filter (47)
- : : +- Scan parquet (46)
- : +- BroadcastExchange (55)
- : +- Project (54)
- : +- Filter (53)
- : +- Scan parquet (52)
- +- BroadcastExchange (60)
- +- Filter (59)
- +- Scan parquet (58)
+ TakeOrderedAndProject (67)
+ +- HashAggregate (66)
+ +- Exchange (65)
+ +- HashAggregate (64)
+ +- Project (63)
+ +- BroadcastHashJoin Inner BuildRight (62)
+ :- Project (58)
+ : +- BroadcastHashJoin Inner BuildRight (57)
+ : :- Project (52)
+ : : +- BroadcastHashJoin Inner BuildRight (51)
+ : : :- Filter (46)
+ : : : +- Scan parquet (45)
+ : : +- BroadcastExchange (50)
+ : : +- Project (49)
+ : : +- Filter (48)
+ : : +- Scan parquet (47)
+ : +- BroadcastExchange (56)
+ : +- Project (55)
+ : +- Filter (54)
+ : +- Scan parquet (53)
+ +- BroadcastExchange (61)
+ +- Filter (60)
+ +- Scan parquet (59)
(1) Scan parquet
@@ -209,155 +210,159 @@ Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X
Input [10]: [hash_partition_key#X, c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: false
-(35) ColumnarExchange
+(35) VeloxAppendBatches
+Input [10]: [hash_partition_key#X, c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
+Arguments: X
+
+(36) ColumnarExchange
Input [10]: [hash_partition_key#X, c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, 1), ENSURE_REQUIREMENTS, [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(36) ShuffleQueryStage
+(37) ShuffleQueryStage
Output [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: X
-(37) InputAdapter
+(38) InputAdapter
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
-(38) InputIteratorTransformer
+(39) InputIteratorTransformer
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
-(39) RegularHashAggregateExecTransformer
+(40) RegularHashAggregateExecTransformer
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Keys [7]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
Results [8]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
-(40) ProjectExecTransformer
+(41) ProjectExecTransformer
Output [8]: [c_custkey#X, c_name#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Input [8]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
-(41) WholeStageCodegenTransformer (X)
+(42) WholeStageCodegenTransformer (X)
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: false
-(42) TakeOrderedAndProjectExecTransformer
+(43) TakeOrderedAndProjectExecTransformer
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: X, [revenue#X DESC NULLS LAST], [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X], 0
-(43) VeloxColumnarToRowExec
+(44) VeloxColumnarToRowExec
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
-(44) Scan parquet
+(45) Scan parquet
Output [7]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(c_custkey), IsNotNull(c_nationkey)]
ReadSchema: struct<c_custkey:bigint,c_name:string,c_address:string,c_nationkey:bigint,c_phone:string,c_acctbal:decimal(12,2),c_comment:string>
-(45) Filter
+(46) Filter
Input [7]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X]
Condition : (isnotnull(c_custkey#X) AND isnotnull(c_nationkey#X))
-(46) Scan parquet
+(47) Scan parquet
Output [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderdate), GreaterThanOrEqual(o_orderdate,1993-10-01), LessThan(o_orderdate,1994-01-01), IsNotNull(o_custkey), IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_orderdate:date>
-(47) Filter
+(48) Filter
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
Condition : ((((isnotnull(o_orderdate#X) AND (o_orderdate#X >= 1993-10-01)) AND (o_orderdate#X < 1994-01-01)) AND isnotnull(o_custkey#X)) AND isnotnull(o_orderkey#X))
-(48) Project
+(49) Project
Output [2]: [o_orderkey#X, o_custkey#X]
Input [3]: [o_orderkey#X, o_custkey#X, o_orderdate#X]
-(49) BroadcastExchange
+(50) BroadcastExchange
Input [2]: [o_orderkey#X, o_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[1, bigint, true]),false), [plan_id=X]
-(50) BroadcastHashJoin
+(51) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(51) Project
+(52) Project
Output [8]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, o_orderkey#X]
Input [9]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, o_orderkey#X, o_custkey#X]
-(52) Scan parquet
+(53) Scan parquet
Output [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_returnflag#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_returnflag), EqualTo(l_returnflag,R), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_returnflag:string>
-(53) Filter
+(54) Filter
Input [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_returnflag#X]
Condition : ((isnotnull(l_returnflag#X) AND (l_returnflag#X = R)) AND isnotnull(l_orderkey#X))
-(54) Project
+(55) Project
Output [3]: [l_orderkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_orderkey#X, l_extendedprice#X, l_discount#X, l_returnflag#X]
-(55) BroadcastExchange
+(56) BroadcastExchange
Input [3]: [l_orderkey#X, l_extendedprice#X, l_discount#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(56) BroadcastHashJoin
+(57) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(57) Project
+(58) Project
Output [9]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X]
Input [11]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, o_orderkey#X, l_orderkey#X, l_extendedprice#X, l_discount#X]
-(58) Scan parquet
+(59) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(59) Filter
+(60) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : isnotnull(n_nationkey#X)
-(60) BroadcastExchange
+(61) BroadcastExchange
Input [2]: [n_nationkey#X, n_name#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(61) BroadcastHashJoin
+(62) BroadcastHashJoin
Left keys [1]: [c_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(62) Project
+(63) Project
Output [9]: [c_custkey#X, c_name#X, c_address#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X, n_name#X]
Input [11]: [c_custkey#X, c_name#X, c_address#X, c_nationkey#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X, n_nationkey#X, n_name#X]
-(63) HashAggregate
+(64) HashAggregate
Input [9]: [c_custkey#X, c_name#X, c_address#X, c_phone#X, c_acctbal#X, c_comment#X, l_extendedprice#X, l_discount#X, n_name#X]
Keys [7]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
-(64) Exchange
+(65) Exchange
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(65) HashAggregate
+(66) HashAggregate
Input [9]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X, sum#X, isEmpty#X]
Keys [7]: [c_custkey#X, c_name#X, c_acctbal#X, c_phone#X, n_name#X, c_address#X, c_comment#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
Results [8]: [c_custkey#X, c_name#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
-(66) TakeOrderedAndProject
+(67) TakeOrderedAndProject
Input [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: X, [revenue#X DESC NULLS LAST], [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
-(67) AdaptiveSparkPlan
+(68) AdaptiveSparkPlan
Output [8]: [c_custkey#X, c_name#X, revenue#X, c_acctbal#X, n_name#X, c_address#X, c_phone#X, c_comment#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/11.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/11.txt
index 6afbee587..1c4e102aa 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/11.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/11.txt
@@ -1,55 +1,57 @@
== Physical Plan ==
-AdaptiveSparkPlan (58)
+AdaptiveSparkPlan (60)
+- == Final Plan ==
- VeloxColumnarToRowExec (38)
- +- ^ SortExecTransformer (36)
- +- ^ InputIteratorTransformer (35)
- +- ShuffleQueryStage (33), Statistics(X)
- +- ColumnarExchange (32)
- +- ^ FilterExecTransformer (30)
- +- ^ RegularHashAggregateExecTransformer (29)
- +- ^ InputIteratorTransformer (28)
- +- ShuffleQueryStage (26), Statistics(X)
- +- ColumnarExchange (25)
- +- ^ ProjectExecTransformer (23)
- +- ^ FlushableHashAggregateExecTransformer (22)
- +- ^ ProjectExecTransformer (21)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
- :- ^ ProjectExecTransformer (11)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
- : :- ^ NoopFilter (2)
- : : +- ^ Scan parquet (1)
- : +- ^ InputIteratorTransformer (9)
- : +- BroadcastQueryStage (7), Statistics(X)
- : +- ColumnarBroadcastExchange (6)
- : +- ^ NoopFilter (4)
- : +- ^ Scan parquet (3)
- +- ^ InputIteratorTransformer (19)
- +- BroadcastQueryStage (17), Statistics(X)
- +- ColumnarBroadcastExchange (16)
- +- ^ ProjectExecTransformer (14)
- +- ^ NoopFilter (13)
- +- ^ Scan parquet (12)
+ VeloxColumnarToRowExec (40)
+ +- ^ SortExecTransformer (38)
+ +- ^ InputIteratorTransformer (37)
+ +- ShuffleQueryStage (35), Statistics(X)
+ +- ColumnarExchange (34)
+ +- VeloxAppendBatches (33)
+ +- ^ FilterExecTransformer (31)
+ +- ^ RegularHashAggregateExecTransformer (30)
+ +- ^ InputIteratorTransformer (29)
+ +- ShuffleQueryStage (27), Statistics(X)
+ +- ColumnarExchange (26)
+ +- VeloxAppendBatches (25)
+ +- ^ ProjectExecTransformer (23)
+ +- ^ FlushableHashAggregateExecTransformer (22)
+ +- ^ ProjectExecTransformer (21)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (20)
+ :- ^ ProjectExecTransformer (11)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
+ : :- ^ NoopFilter (2)
+ : : +- ^ Scan parquet (1)
+ : +- ^ InputIteratorTransformer (9)
+ : +- BroadcastQueryStage (7), Statistics(X)
+ : +- ColumnarBroadcastExchange (6)
+ : +- ^ NoopFilter (4)
+ : +- ^ Scan parquet (3)
+ +- ^ InputIteratorTransformer (19)
+ +- BroadcastQueryStage (17), Statistics(X)
+ +- ColumnarBroadcastExchange (16)
+ +- ^ ProjectExecTransformer (14)
+ +- ^ NoopFilter (13)
+ +- ^ Scan parquet (12)
+- == Initial Plan ==
- Sort (57)
- +- Exchange (56)
- +- Filter (55)
- +- HashAggregate (54)
- +- Exchange (53)
- +- HashAggregate (52)
- +- Project (51)
- +- BroadcastHashJoin Inner BuildRight (50)
- :- Project (45)
- : +- BroadcastHashJoin Inner BuildRight (44)
- : :- Filter (40)
- : : +- Scan parquet (39)
- : +- BroadcastExchange (43)
- : +- Filter (42)
- : +- Scan parquet (41)
- +- BroadcastExchange (49)
- +- Project (48)
- +- Filter (47)
- +- Scan parquet (46)
+ Sort (59)
+ +- Exchange (58)
+ +- Filter (57)
+ +- HashAggregate (56)
+ +- Exchange (55)
+ +- HashAggregate (54)
+ +- Project (53)
+ +- BroadcastHashJoin Inner BuildRight (52)
+ :- Project (47)
+ : +- BroadcastHashJoin Inner BuildRight (46)
+ : :- Filter (42)
+ : : +- Scan parquet (41)
+ : +- BroadcastExchange (45)
+ : +- Filter (44)
+ : +- Scan parquet (43)
+ +- BroadcastExchange (51)
+ +- Project (50)
+ +- Filter (49)
+ +- Scan parquet (48)
(1) Scan parquet
@@ -158,379 +160,392 @@ Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Input [4]: [hash_partition_key#X, ps_partkey#X, sum#X, isEmpty#X]
Arguments: false
-(25) ColumnarExchange
+(25) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, ps_partkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(26) ColumnarExchange
Input [4]: [hash_partition_key#X, ps_partkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(ps_partkey#X, 1), ENSURE_REQUIREMENTS, [ps_partkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(26) ShuffleQueryStage
+(27) ShuffleQueryStage
Output [3]: [ps_partkey#X, sum#X, isEmpty#X]
Arguments: X
-(27) InputAdapter
+(28) InputAdapter
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
-(28) InputIteratorTransformer
+(29) InputIteratorTransformer
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
-(29) RegularHashAggregateExecTransformer
+(30) RegularHashAggregateExecTransformer
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Keys [1]: [ps_partkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X]
Results [2]: [ps_partkey#X, sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X AS value#X]
-(30) FilterExecTransformer
+(31) FilterExecTransformer
Input [2]: [ps_partkey#X, value#X]
Arguments: (isnotnull(value#X) AND (cast(value#X as decimal(38,6)) > Subquery subquery#X, [id=#X]))
-(31) WholeStageCodegenTransformer (X)
+(32) WholeStageCodegenTransformer (X)
Input [2]: [ps_partkey#X, value#X]
Arguments: false
-(32) ColumnarExchange
+(33) VeloxAppendBatches
+Input [2]: [ps_partkey#X, value#X]
+Arguments: X
+
+(34) ColumnarExchange
Input [2]: [ps_partkey#X, value#X]
Arguments: rangepartitioning(value#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(33) ShuffleQueryStage
+(35) ShuffleQueryStage
Output [2]: [ps_partkey#X, value#X]
Arguments: X
-(34) InputAdapter
+(36) InputAdapter
Input [2]: [ps_partkey#X, value#X]
-(35) InputIteratorTransformer
+(37) InputIteratorTransformer
Input [2]: [ps_partkey#X, value#X]
-(36) SortExecTransformer
+(38) SortExecTransformer
Input [2]: [ps_partkey#X, value#X]
Arguments: [value#X DESC NULLS LAST], true, 0
-(37) WholeStageCodegenTransformer (X)
+(39) WholeStageCodegenTransformer (X)
Input [2]: [ps_partkey#X, value#X]
Arguments: false
-(38) VeloxColumnarToRowExec
+(40) VeloxColumnarToRowExec
Input [2]: [ps_partkey#X, value#X]
-(39) Scan parquet
+(41) Scan parquet
Output [4]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_suppkey)]
ReadSchema: struct<ps_partkey:bigint,ps_suppkey:bigint,ps_availqty:int,ps_supplycost:decimal(12,2)>
-(40) Filter
+(42) Filter
Input [4]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Condition : isnotnull(ps_suppkey#X)
-(41) Scan parquet
+(43) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(42) Filter
+(44) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(43) BroadcastExchange
+(45) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(44) BroadcastHashJoin
+(46) BroadcastHashJoin
Left keys [1]: [ps_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(45) Project
+(47) Project
Output [4]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X, s_nationkey#X]
Input [6]: [ps_partkey#X, ps_suppkey#X, ps_availqty#X, ps_supplycost#X, s_suppkey#X, s_nationkey#X]
-(46) Scan parquet
+(48) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_name), EqualTo(n_name,GERMANY), IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(47) Filter
+(49) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : ((isnotnull(n_name#X) AND (n_name#X = GERMANY)) AND isnotnull(n_nationkey#X))
-(48) Project
+(50) Project
Output [1]: [n_nationkey#X]
Input [2]: [n_nationkey#X, n_name#X]
-(49) BroadcastExchange
+(51) BroadcastExchange
Input [1]: [n_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(50) BroadcastHashJoin
+(52) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(51) Project
+(53) Project
Output [3]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X]
Input [5]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X, s_nationkey#X, n_nationkey#X]
-(52) HashAggregate
+(54) HashAggregate
Input [3]: [ps_partkey#X, ps_availqty#X, ps_supplycost#X]
Keys [1]: [ps_partkey#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [ps_partkey#X, sum#X, isEmpty#X]
-(53) Exchange
+(55) Exchange
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(ps_partkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(54) HashAggregate
+(56) HashAggregate
Input [3]: [ps_partkey#X, sum#X, isEmpty#X]
Keys [1]: [ps_partkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X]
Results [2]: [ps_partkey#X, sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X AS value#X]
-(55) Filter
+(57) Filter
Input [2]: [ps_partkey#X, value#X]
Condition : (isnotnull(value#X) AND (cast(value#X as decimal(38,6)) > Subquery subquery#X, [id=#X]))
-(56) Exchange
+(58) Exchange
Input [2]: [ps_partkey#X, value#X]
Arguments: rangepartitioning(value#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(57) Sort
+(59) Sort
Input [2]: [ps_partkey#X, value#X]
Arguments: [value#X DESC NULLS LAST], true, 0
-(58) AdaptiveSparkPlan
+(60) AdaptiveSparkPlan
Output [2]: [ps_partkey#X, value#X]
Arguments: isFinalPlan=true
===== Subqueries =====
-Subquery:1 Hosting operator id = 30 Hosting Expression = Subquery subquery#X, [id=#X]
-AdaptiveSparkPlan (99)
+Subquery:1 Hosting operator id = 31 Hosting Expression = Subquery subquery#X, [id=#X]
+AdaptiveSparkPlan (102)
+- == Final Plan ==
- VeloxColumnarToRowExec (82)
- +- ^ ProjectExecTransformer (80)
- +- ^ RegularHashAggregateExecTransformer (79)
- +- ^ InputIteratorTransformer (78)
- +- ShuffleQueryStage (76), Statistics(X)
- +- ColumnarExchange (75)
- +- ^ FlushableHashAggregateExecTransformer (73)
- +- ^ ProjectExecTransformer (72)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (71)
- :- ^ ProjectExecTransformer (66)
- : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (65)
- : :- ^ NoopFilter (60)
- : : +- ^ Scan parquet (59)
- : +- ^ InputIteratorTransformer (64)
- : +- BroadcastQueryStage (62), Statistics(X)
- : +- ReusedExchange (61)
- +- ^ InputIteratorTransformer (70)
- +- BroadcastQueryStage (68), Statistics(X)
- +- ReusedExchange (67)
+ VeloxColumnarToRowExec (85)
+ +- ^ ProjectExecTransformer (83)
+ +- ^ RegularHashAggregateExecTransformer (82)
+ +- ^ InputIteratorTransformer (81)
+ +- ShuffleQueryStage (79), Statistics(X)
+ +- ColumnarExchange (78)
+ +- VeloxAppendBatches (77)
+ +- ^ FlushableHashAggregateExecTransformer (75)
+ +- ^ ProjectExecTransformer (74)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (73)
+ :- ^ ProjectExecTransformer (68)
+ : +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (67)
+ : :- ^ NoopFilter (62)
+ : : +- ^ Scan parquet (61)
+ : +- ^ InputIteratorTransformer (66)
+ : +- BroadcastQueryStage (64), Statistics(X)
+ : +- ReusedExchange (63)
+ +- ^ InputIteratorTransformer (72)
+ +- BroadcastQueryStage (70), Statistics(X)
+ +- ReusedExchange (69)
+- == Initial Plan ==
- HashAggregate (98)
- +- Exchange (97)
- +- HashAggregate (96)
- +- Project (95)
- +- BroadcastHashJoin Inner BuildRight (94)
- :- Project (89)
- : +- BroadcastHashJoin Inner BuildRight (88)
- : :- Filter (84)
- : : +- Scan parquet (83)
- : +- BroadcastExchange (87)
- : +- Filter (86)
- : +- Scan parquet (85)
- +- BroadcastExchange (93)
- +- Project (92)
- +- Filter (91)
- +- Scan parquet (90)
-
-
-(59) Scan parquet
+ HashAggregate (101)
+ +- Exchange (100)
+ +- HashAggregate (99)
+ +- Project (98)
+ +- BroadcastHashJoin Inner BuildRight (97)
+ :- Project (92)
+ : +- BroadcastHashJoin Inner BuildRight (91)
+ : :- Filter (87)
+ : : +- Scan parquet (86)
+ : +- BroadcastExchange (90)
+ : +- Filter (89)
+ : +- Scan parquet (88)
+ +- BroadcastExchange (96)
+ +- Project (95)
+ +- Filter (94)
+ +- Scan parquet (93)
+
+
+(61) Scan parquet
Output [3]: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_suppkey)]
ReadSchema: struct<ps_suppkey:bigint,ps_availqty:int,ps_supplycost:decimal(12,2)>
-(60) NoopFilter
+(62) NoopFilter
Input [3]: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Arguments: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
-(61) ReusedExchange [Reuses operator id: 6]
+(63) ReusedExchange [Reuses operator id: 6]
Output [2]: [s_suppkey#X, s_nationkey#X]
-(62) BroadcastQueryStage
+(64) BroadcastQueryStage
Output [2]: [s_suppkey#X, s_nationkey#X]
Arguments: X
-(63) InputAdapter
+(65) InputAdapter
Input [2]: [s_suppkey#X, s_nationkey#X]
-(64) InputIteratorTransformer
+(66) InputIteratorTransformer
Input [2]: [s_suppkey#X, s_nationkey#X]
-(65) BroadcastHashJoinExecTransformer
+(67) BroadcastHashJoinExecTransformer
Left keys [1]: [ps_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(66) ProjectExecTransformer
+(68) ProjectExecTransformer
Output [3]: [ps_availqty#X, ps_supplycost#X, s_nationkey#X]
Input [5]: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X, s_suppkey#X, s_nationkey#X]
-(67) ReusedExchange [Reuses operator id: 16]
+(69) ReusedExchange [Reuses operator id: 16]
Output [1]: [n_nationkey#X]
-(68) BroadcastQueryStage
+(70) BroadcastQueryStage
Output [1]: [n_nationkey#X]
Arguments: X
-(69) InputAdapter
+(71) InputAdapter
Input [1]: [n_nationkey#X]
-(70) InputIteratorTransformer
+(72) InputIteratorTransformer
Input [1]: [n_nationkey#X]
-(71) BroadcastHashJoinExecTransformer
+(73) BroadcastHashJoinExecTransformer
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(72) ProjectExecTransformer
+(74) ProjectExecTransformer
Output [3]: [ps_availqty#X, ps_supplycost#X, CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)) AS _pre_X#X]
Input [4]: [ps_availqty#X, ps_supplycost#X, s_nationkey#X, n_nationkey#X]
-(73) FlushableHashAggregateExecTransformer
+(75) FlushableHashAggregateExecTransformer
Input [3]: [ps_availqty#X, ps_supplycost#X, _pre_X#X]
Keys: []
Functions [1]: [partial_sum(_pre_X#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [2]: [sum#X, isEmpty#X]
-(74) WholeStageCodegenTransformer (X)
+(76) WholeStageCodegenTransformer (X)
Input [2]: [sum#X, isEmpty#X]
Arguments: false
-(75) ColumnarExchange
+(77) VeloxAppendBatches
+Input [2]: [sum#X, isEmpty#X]
+Arguments: X
+
+(78) ColumnarExchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(76) ShuffleQueryStage
+(79) ShuffleQueryStage
Output [2]: [sum#X, isEmpty#X]
Arguments: X
-(77) InputAdapter
+(80) InputAdapter
Input [2]: [sum#X, isEmpty#X]
-(78) InputIteratorTransformer
+(81) InputIteratorTransformer
Input [2]: [sum#X, isEmpty#X]
-(79) RegularHashAggregateExecTransformer
+(82) RegularHashAggregateExecTransformer
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X]
Results [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X]
-(80) ProjectExecTransformer
+(83) ProjectExecTransformer
Output [1]: [CheckOverflow((promote_precision(cast(sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X as decimal(38,10))) * 0.0001000000), DecimalType(38,6)) AS (sum((ps_supplycost * ps_availqty)) * 0.0001000000)#X]
Input [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X]
-(81) WholeStageCodegenTransformer (X)
+(84) WholeStageCodegenTransformer (X)
Input [1]: [(sum((ps_supplycost * ps_availqty)) * 0.0001000000)#X]
Arguments: false
-(82) VeloxColumnarToRowExec
+(85) VeloxColumnarToRowExec
Input [1]: [(sum((ps_supplycost * ps_availqty)) * 0.0001000000)#X]
-(83) Scan parquet
+(86) Scan parquet
Output [3]: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(ps_suppkey)]
ReadSchema: struct<ps_suppkey:bigint,ps_availqty:int,ps_supplycost:decimal(12,2)>
-(84) Filter
+(87) Filter
Input [3]: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X]
Condition : isnotnull(ps_suppkey#X)
-(85) Scan parquet
+(88) Scan parquet
Output [2]: [s_suppkey#X, s_nationkey#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey), IsNotNull(s_nationkey)]
ReadSchema: struct<s_suppkey:bigint,s_nationkey:bigint>
-(86) Filter
+(89) Filter
Input [2]: [s_suppkey#X, s_nationkey#X]
Condition : (isnotnull(s_suppkey#X) AND isnotnull(s_nationkey#X))
-(87) BroadcastExchange
+(90) BroadcastExchange
Input [2]: [s_suppkey#X, s_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(88) BroadcastHashJoin
+(91) BroadcastHashJoin
Left keys [1]: [ps_suppkey#X]
Right keys [1]: [s_suppkey#X]
Join condition: None
-(89) Project
+(92) Project
Output [3]: [ps_availqty#X, ps_supplycost#X, s_nationkey#X]
Input [5]: [ps_suppkey#X, ps_availqty#X, ps_supplycost#X, s_suppkey#X, s_nationkey#X]
-(90) Scan parquet
+(93) Scan parquet
Output [2]: [n_nationkey#X, n_name#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(n_name), EqualTo(n_name,GERMANY), IsNotNull(n_nationkey)]
ReadSchema: struct<n_nationkey:bigint,n_name:string>
-(91) Filter
+(94) Filter
Input [2]: [n_nationkey#X, n_name#X]
Condition : ((isnotnull(n_name#X) AND (n_name#X = GERMANY)) AND isnotnull(n_nationkey#X))
-(92) Project
+(95) Project
Output [1]: [n_nationkey#X]
Input [2]: [n_nationkey#X, n_name#X]
-(93) BroadcastExchange
+(96) BroadcastExchange
Input [1]: [n_nationkey#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, true]),false), [plan_id=X]
-(94) BroadcastHashJoin
+(97) BroadcastHashJoin
Left keys [1]: [s_nationkey#X]
Right keys [1]: [n_nationkey#X]
Join condition: None
-(95) Project
+(98) Project
Output [2]: [ps_availqty#X, ps_supplycost#X]
Input [4]: [ps_availqty#X, ps_supplycost#X, s_nationkey#X, n_nationkey#X]
-(96) HashAggregate
+(99) HashAggregate
Input [2]: [ps_availqty#X, ps_supplycost#X]
Keys: []
Functions [1]: [partial_sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [2]: [sum#X, isEmpty#X]
-(97) Exchange
+(100) Exchange
Input [2]: [sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X]
-(98) HashAggregate
+(101) HashAggregate
Input [2]: [sum#X, isEmpty#X]
Keys: []
Functions [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X]
Results [1]: [CheckOverflow((promote_precision(cast(sum(CheckOverflow((promote_precision(ps_supplycost#X) * promote_precision(cast(ps_availqty#X as decimal(12,2)))), DecimalType(23,2)))#X as decimal(38,10))) * 0.0001000000), DecimalType(38,6)) AS (sum((ps_supplycost * ps_availqty)) * 0.0001000000)#X]
-(99) AdaptiveSparkPlan
+(102) AdaptiveSparkPlan
Output [1]: [(sum((ps_supplycost * ps_availqty)) * 0.0001000000)#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/12.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/12.txt
index c65916d66..12d0f6f0f 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/12.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/12.txt
@@ -1,41 +1,43 @@
== Physical Plan ==
-AdaptiveSparkPlan (42)
+AdaptiveSparkPlan (44)
+- == Final Plan ==
- VeloxColumnarToRowExec (28)
- +- ^ SortExecTransformer (26)
- +- ^ InputIteratorTransformer (25)
- +- ShuffleQueryStage (23), Statistics(X)
- +- ColumnarExchange (22)
- +- ^ RegularHashAggregateExecTransformer (20)
- +- ^ InputIteratorTransformer (19)
- +- ShuffleQueryStage (17), Statistics(X)
- +- ColumnarExchange (16)
- +- ^ ProjectExecTransformer (14)
- +- ^ FlushableHashAggregateExecTransformer (13)
- +- ^ ProjectExecTransformer (12)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
- :- ^ InputIteratorTransformer (7)
- : +- BroadcastQueryStage (5), Statistics(X)
- : +- ColumnarBroadcastExchange (4)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ ProjectExecTransformer (10)
- +- ^ NoopFilter (9)
- +- ^ Scan parquet (8)
+ VeloxColumnarToRowExec (30)
+ +- ^ SortExecTransformer (28)
+ +- ^ InputIteratorTransformer (27)
+ +- ShuffleQueryStage (25), Statistics(X)
+ +- ColumnarExchange (24)
+ +- VeloxAppendBatches (23)
+ +- ^ RegularHashAggregateExecTransformer (21)
+ +- ^ InputIteratorTransformer (20)
+ +- ShuffleQueryStage (18), Statistics(X)
+ +- ColumnarExchange (17)
+ +- VeloxAppendBatches (16)
+ +- ^ ProjectExecTransformer (14)
+ +- ^ FlushableHashAggregateExecTransformer (13)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (11)
+ :- ^ InputIteratorTransformer (7)
+ : +- BroadcastQueryStage (5), Statistics(X)
+ : +- ColumnarBroadcastExchange (4)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ ProjectExecTransformer (10)
+ +- ^ NoopFilter (9)
+ +- ^ Scan parquet (8)
+- == Initial Plan ==
- Sort (41)
- +- Exchange (40)
- +- HashAggregate (39)
- +- Exchange (38)
- +- HashAggregate (37)
- +- Project (36)
- +- BroadcastHashJoin Inner BuildLeft (35)
- :- BroadcastExchange (31)
- : +- Filter (30)
- : +- Scan parquet (29)
- +- Project (34)
- +- Filter (33)
- +- Scan parquet (32)
+ Sort (43)
+ +- Exchange (42)
+ +- HashAggregate (41)
+ +- Exchange (40)
+ +- HashAggregate (39)
+ +- Project (38)
+ +- BroadcastHashJoin Inner BuildLeft (37)
+ :- BroadcastExchange (33)
+ : +- Filter (32)
+ : +- Scan parquet (31)
+ +- Project (36)
+ +- Filter (35)
+ +- Scan parquet (34)
(1) Scan parquet
@@ -106,121 +108,129 @@ Input [3]: [l_shipmode#X, sum#X, sum#X]
Input [4]: [hash_partition_key#X, l_shipmode#X, sum#X, sum#X]
Arguments: false
-(16) ColumnarExchange
+(16) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, l_shipmode#X, sum#X, sum#X]
+Arguments: X
+
+(17) ColumnarExchange
Input [4]: [hash_partition_key#X, l_shipmode#X, sum#X, sum#X]
Arguments: hashpartitioning(l_shipmode#X, 1), ENSURE_REQUIREMENTS, [l_shipmode#X, sum#X, sum#X], [plan_id=X], [id=#X]
-(17) ShuffleQueryStage
+(18) ShuffleQueryStage
Output [3]: [l_shipmode#X, sum#X, sum#X]
Arguments: X
-(18) InputAdapter
+(19) InputAdapter
Input [3]: [l_shipmode#X, sum#X, sum#X]
-(19) InputIteratorTransformer
+(20) InputIteratorTransformer
Input [3]: [l_shipmode#X, sum#X, sum#X]
-(20) RegularHashAggregateExecTransformer
+(21) RegularHashAggregateExecTransformer
Input [3]: [l_shipmode#X, sum#X, sum#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]
-(21) WholeStageCodegenTransformer (X)
+(22) WholeStageCodegenTransformer (X)
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: false
-(22) ColumnarExchange
+(23) VeloxAppendBatches
+Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
+Arguments: X
+
+(24) ColumnarExchange
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: rangepartitioning(l_shipmode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(23) ShuffleQueryStage
+(25) ShuffleQueryStage
Output [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: X
-(24) InputAdapter
+(26) InputAdapter
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
-(25) InputIteratorTransformer
+(27) InputIteratorTransformer
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
-(26) SortExecTransformer
+(28) SortExecTransformer
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: [l_shipmode#X ASC NULLS FIRST], true, 0
-(27) WholeStageCodegenTransformer (X)
+(29) WholeStageCodegenTransformer (X)
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: false
-(28) VeloxColumnarToRowExec
+(30) VeloxColumnarToRowExec
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
-(29) Scan parquet
+(31) Scan parquet
Output [2]: [o_orderkey#X, o_orderpriority#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_orderkey)]
ReadSchema: struct<o_orderkey:bigint,o_orderpriority:string>
-(30) Filter
+(32) Filter
Input [2]: [o_orderkey#X, o_orderpriority#X]
Condition : isnotnull(o_orderkey#X)
-(31) BroadcastExchange
+(33) BroadcastExchange
Input [2]: [o_orderkey#X, o_orderpriority#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(32) Scan parquet
+(34) Scan parquet
Output [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_commitdate), IsNotNull(l_receiptdate), IsNotNull(l_shipdate), In(l_shipmode, [MAIL,SHIP]), GreaterThanOrEqual(l_receiptdate,1994-01-01), LessThan(l_receiptdate,1995-01-01), IsNotNull(l_orderkey)]
ReadSchema: struct<l_orderkey:bigint,l_shipdate:date,l_commitdate:date,l_receiptdate:date,l_shipmode:string>
-(33) Filter
+(35) Filter
Input [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
Condition : ((((((((isnotnull(l_commitdate#X) AND isnotnull(l_receiptdate#X)) AND isnotnull(l_shipdate#X)) AND l_shipmode#X IN (MAIL,SHIP)) AND (l_commitdate#X < l_receiptdate#X)) AND (l_shipdate#X < l_commitdate#X)) AND (l_receiptdate#X >= 1994-01-01)) AND (l_receiptdate#X < 1995-01-01)) AND isnotnull(l_orderkey#X))
-(34) Project
+(36) Project
Output [2]: [l_orderkey#X, l_shipmode#X]
Input [5]: [l_orderkey#X, l_shipdate#X, l_commitdate#X, l_receiptdate#X, l_shipmode#X]
-(35) BroadcastHashJoin
+(37) BroadcastHashJoin
Left keys [1]: [o_orderkey#X]
Right keys [1]: [l_orderkey#X]
Join condition: None
-(36) Project
+(38) Project
Output [2]: [o_orderpriority#X, l_shipmode#X]
Input [4]: [o_orderkey#X, o_orderpriority#X, l_orderkey#X, l_shipmode#X]
-(37) HashAggregate
+(39) HashAggregate
Input [2]: [o_orderpriority#X, l_shipmode#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [partial_sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), partial_sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum#X, sum#X]
Results [3]: [l_shipmode#X, sum#X, sum#X]
-(38) Exchange
+(40) Exchange
Input [3]: [l_shipmode#X, sum#X, sum#X]
Arguments: hashpartitioning(l_shipmode#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(39) HashAggregate
+(41) HashAggregate
Input [3]: [l_shipmode#X, sum#X, sum#X]
Keys [1]: [l_shipmode#X]
Functions [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END), sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)]
Aggregate Attributes [2]: [sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X]
Results [3]: [l_shipmode#X, sum(CASE WHEN ((o_orderpriority#X = 1-URGENT) OR (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS high_line_count#X, sum(CASE WHEN (NOT (o_orderpriority#X = 1-URGENT) AND NOT (o_orderpriority#X = 2-HIGH)) THEN 1 ELSE 0 END)#X AS low_line_count#X]
-(40) Exchange
+(42) Exchange
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: rangepartitioning(l_shipmode#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(41) Sort
+(43) Sort
Input [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: [l_shipmode#X ASC NULLS FIRST], true, 0
-(42) AdaptiveSparkPlan
+(44) AdaptiveSparkPlan
Output [3]: [l_shipmode#X, high_line_count#X, low_line_count#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/13.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/13.txt
index 9ff054f93..1dfb80d78 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/13.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/13.txt
@@ -1,49 +1,52 @@
== Physical Plan ==
-AdaptiveSparkPlan (52)
+AdaptiveSparkPlan (55)
+- == Final Plan ==
- VeloxColumnarToRowExec (36)
- +- ^ SortExecTransformer (34)
- +- ^ InputIteratorTransformer (33)
- +- ShuffleQueryStage (31), Statistics(X)
- +- ColumnarExchange (30)
- +- ^ RegularHashAggregateExecTransformer (28)
- +- ^ InputIteratorTransformer (27)
- +- ShuffleQueryStage (25), Statistics(X)
- +- ColumnarExchange (24)
- +- ^ ProjectExecTransformer (22)
- +- ^ FlushableHashAggregateExecTransformer (21)
- +- ^ ProjectExecTransformer (20)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16), Statistics(X)
- +- ColumnarExchange (15)
- +- ^ ProjectExecTransformer (13)
- +- ^ FlushableHashAggregateExecTransformer (12)
- +- ^ ProjectExecTransformer (11)
- +- ^ BroadcastHashJoinExecTransformer LeftOuter BuildRight (10)
- :- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (9)
- +- BroadcastQueryStage (7), Statistics(X)
- +- ColumnarBroadcastExchange (6)
- +- ^ ProjectExecTransformer (4)
- +- ^ NoopFilter (3)
- +- ^ Scan parquet (2)
+ VeloxColumnarToRowExec (39)
+ +- ^ SortExecTransformer (37)
+ +- ^ InputIteratorTransformer (36)
+ +- ShuffleQueryStage (34), Statistics(X)
+ +- ColumnarExchange (33)
+ +- VeloxAppendBatches (32)
+ +- ^ RegularHashAggregateExecTransformer (30)
+ +- ^ InputIteratorTransformer (29)
+ +- ShuffleQueryStage (27), Statistics(X)
+ +- ColumnarExchange (26)
+ +- VeloxAppendBatches (25)
+ +- ^ ProjectExecTransformer (23)
+ +- ^ FlushableHashAggregateExecTransformer (22)
+ +- ^ ProjectExecTransformer (21)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17), Statistics(X)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ ProjectExecTransformer (13)
+ +- ^ FlushableHashAggregateExecTransformer (12)
+ +- ^ ProjectExecTransformer (11)
+ +- ^ BroadcastHashJoinExecTransformer LeftOuter BuildRight (10)
+ :- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (9)
+ +- BroadcastQueryStage (7), Statistics(X)
+ +- ColumnarBroadcastExchange (6)
+ +- ^ ProjectExecTransformer (4)
+ +- ^ NoopFilter (3)
+ +- ^ Scan parquet (2)
+- == Initial Plan ==
- Sort (51)
- +- Exchange (50)
- +- HashAggregate (49)
- +- Exchange (48)
- +- HashAggregate (47)
- +- HashAggregate (46)
- +- Exchange (45)
- +- HashAggregate (44)
- +- Project (43)
- +- BroadcastHashJoin LeftOuter BuildRight (42)
- :- Scan parquet (37)
- +- BroadcastExchange (41)
- +- Project (40)
- +- Filter (39)
- +- Scan parquet (38)
+ Sort (54)
+ +- Exchange (53)
+ +- HashAggregate (52)
+ +- Exchange (51)
+ +- HashAggregate (50)
+ +- HashAggregate (49)
+ +- Exchange (48)
+ +- HashAggregate (47)
+ +- Project (46)
+ +- BroadcastHashJoin LeftOuter BuildRight (45)
+ :- Scan parquet (40)
+ +- BroadcastExchange (44)
+ +- Project (43)
+ +- Filter (42)
+ +- Scan parquet (41)
(1) Scan parquet
@@ -109,174 +112,186 @@ Input [2]: [c_custkey#X, count#X]
Input [3]: [hash_partition_key#X, c_custkey#X, count#X]
Arguments: false
-(15) ColumnarExchange
+(15) VeloxAppendBatches
+Input [3]: [hash_partition_key#X, c_custkey#X, count#X]
+Arguments: X
+
+(16) ColumnarExchange
Input [3]: [hash_partition_key#X, c_custkey#X, count#X]
Arguments: hashpartitioning(c_custkey#X, 1), ENSURE_REQUIREMENTS, [c_custkey#X, count#X], [plan_id=X], [id=#X]
-(16) ShuffleQueryStage
+(17) ShuffleQueryStage
Output [2]: [c_custkey#X, count#X]
Arguments: X
-(17) InputAdapter
+(18) InputAdapter
Input [2]: [c_custkey#X, count#X]
-(18) InputIteratorTransformer
+(19) InputIteratorTransformer
Input [2]: [c_custkey#X, count#X]
-(19) RegularHashAggregateExecTransformer
+(20) RegularHashAggregateExecTransformer
Input [2]: [c_custkey#X, count#X]
Keys [1]: [c_custkey#X]
Functions [1]: [count(o_orderkey#X)]
Aggregate Attributes [1]: [count(o_orderkey#X)#X]
Results [2]: [c_custkey#X, count(o_orderkey#X)#X]
-(20) ProjectExecTransformer
+(21) ProjectExecTransformer
Output [1]: [count(o_orderkey#X)#X AS c_count#X]
Input [2]: [c_custkey#X, count(o_orderkey#X)#X]
-(21) FlushableHashAggregateExecTransformer
+(22) FlushableHashAggregateExecTransformer
Input [1]: [c_count#X]
Keys [1]: [c_count#X]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#X]
Results [2]: [c_count#X, count#X]
-(22) ProjectExecTransformer
+(23) ProjectExecTransformer
Output [3]: [hash(c_count#X, 42) AS hash_partition_key#X, c_count#X, count#X]
Input [2]: [c_count#X, count#X]
-(23) WholeStageCodegenTransformer (X)
+(24) WholeStageCodegenTransformer (X)
Input [3]: [hash_partition_key#X, c_count#X, count#X]
Arguments: false
-(24) ColumnarExchange
+(25) VeloxAppendBatches
+Input [3]: [hash_partition_key#X, c_count#X, count#X]
+Arguments: X
+
+(26) ColumnarExchange
Input [3]: [hash_partition_key#X, c_count#X, count#X]
Arguments: hashpartitioning(c_count#X, 1), ENSURE_REQUIREMENTS, [c_count#X, count#X], [plan_id=X], [id=#X]
-(25) ShuffleQueryStage
+(27) ShuffleQueryStage
Output [2]: [c_count#X, count#X]
Arguments: X
-(26) InputAdapter
+(28) InputAdapter
Input [2]: [c_count#X, count#X]
-(27) InputIteratorTransformer
+(29) InputIteratorTransformer
Input [2]: [c_count#X, count#X]
-(28) RegularHashAggregateExecTransformer
+(30) RegularHashAggregateExecTransformer
Input [2]: [c_count#X, count#X]
Keys [1]: [c_count#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [c_count#X, count(1)#X AS custdist#X]
-(29) WholeStageCodegenTransformer (X)
+(31) WholeStageCodegenTransformer (X)
Input [2]: [c_count#X, custdist#X]
Arguments: false
-(30) ColumnarExchange
+(32) VeloxAppendBatches
+Input [2]: [c_count#X, custdist#X]
+Arguments: X
+
+(33) ColumnarExchange
Input [2]: [c_count#X, custdist#X]
Arguments: rangepartitioning(custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(31) ShuffleQueryStage
+(34) ShuffleQueryStage
Output [2]: [c_count#X, custdist#X]
Arguments: X
-(32) InputAdapter
+(35) InputAdapter
Input [2]: [c_count#X, custdist#X]
-(33) InputIteratorTransformer
+(36) InputIteratorTransformer
Input [2]: [c_count#X, custdist#X]
-(34) SortExecTransformer
+(37) SortExecTransformer
Input [2]: [c_count#X, custdist#X]
Arguments: [custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST], true, 0
-(35) WholeStageCodegenTransformer (X)
+(38) WholeStageCodegenTransformer (X)
Input [2]: [c_count#X, custdist#X]
Arguments: false
-(36) VeloxColumnarToRowExec
+(39) VeloxColumnarToRowExec
Input [2]: [c_count#X, custdist#X]
-(37) Scan parquet
+(40) Scan parquet
Output [1]: [c_custkey#X]
Batched: true
Location: InMemoryFileIndex [*]
ReadSchema: struct<c_custkey:bigint>
-(38) Scan parquet
+(41) Scan parquet
Output [3]: [o_orderkey#X, o_custkey#X, o_comment#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(o_comment), IsNotNull(o_custkey)]
ReadSchema: struct<o_orderkey:bigint,o_custkey:bigint,o_comment:string>
-(39) Filter
+(42) Filter
Input [3]: [o_orderkey#X, o_custkey#X, o_comment#X]
Condition : ((isnotnull(o_comment#X) AND NOT o_comment#X LIKE %special%requests%) AND isnotnull(o_custkey#X))
-(40) Project
+(43) Project
Output [2]: [o_orderkey#X, o_custkey#X]
Input [3]: [o_orderkey#X, o_custkey#X, o_comment#X]
-(41) BroadcastExchange
+(44) BroadcastExchange
Input [2]: [o_orderkey#X, o_custkey#X]
Arguments: HashedRelationBroadcastMode(List(input[1, bigint, true]),false), [plan_id=X]
-(42) BroadcastHashJoin
+(45) BroadcastHashJoin
Left keys [1]: [c_custkey#X]
Right keys [1]: [o_custkey#X]
Join condition: None
-(43) Project
+(46) Project
Output [2]: [c_custkey#X, o_orderkey#X]
Input [3]: [c_custkey#X, o_orderkey#X, o_custkey#X]
-(44) HashAggregate
+(47) HashAggregate
Input [2]: [c_custkey#X, o_orderkey#X]
Keys [1]: [c_custkey#X]
Functions [1]: [partial_count(o_orderkey#X)]
Aggregate Attributes [1]: [count#X]
Results [2]: [c_custkey#X, count#X]
-(45) Exchange
+(48) Exchange
Input [2]: [c_custkey#X, count#X]
Arguments: hashpartitioning(c_custkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(46) HashAggregate
+(49) HashAggregate
Input [2]: [c_custkey#X, count#X]
Keys [1]: [c_custkey#X]
Functions [1]: [count(o_orderkey#X)]
Aggregate Attributes [1]: [count(o_orderkey#X)#X]
Results [1]: [count(o_orderkey#X)#X AS c_count#X]
-(47) HashAggregate
+(50) HashAggregate
Input [1]: [c_count#X]
Keys [1]: [c_count#X]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#X]
Results [2]: [c_count#X, count#X]
-(48) Exchange
+(51) Exchange
Input [2]: [c_count#X, count#X]
Arguments: hashpartitioning(c_count#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(49) HashAggregate
+(52) HashAggregate
Input [2]: [c_count#X, count#X]
Keys [1]: [c_count#X]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#X]
Results [2]: [c_count#X, count(1)#X AS custdist#X]
-(50) Exchange
+(53) Exchange
Input [2]: [c_count#X, custdist#X]
Arguments: rangepartitioning(custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(51) Sort
+(54) Sort
Input [2]: [c_count#X, custdist#X]
Arguments: [custdist#X DESC NULLS LAST, c_count#X DESC NULLS LAST], true, 0
-(52) AdaptiveSparkPlan
+(55) AdaptiveSparkPlan
Output [2]: [c_count#X, custdist#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/14.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/14.txt
index cc14a0347..6f71859cb 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/14.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/14.txt
@@ -1,35 +1,36 @@
== Physical Plan ==
-AdaptiveSparkPlan (34)
+AdaptiveSparkPlan (35)
+- == Final Plan ==
- VeloxColumnarToRowExec (22)
- +- ^ ProjectExecTransformer (20)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16), Statistics(X)
- +- ColumnarExchange (15)
- +- ^ FlushableHashAggregateExecTransformer (13)
- +- ^ ProjectExecTransformer (12)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
- :- ^ ProjectExecTransformer (3)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (10)
- +- BroadcastQueryStage (8), Statistics(X)
- +- ColumnarBroadcastExchange (7)
- +- ^ NoopFilter (5)
- +- ^ Scan parquet (4)
+ VeloxColumnarToRowExec (23)
+ +- ^ ProjectExecTransformer (21)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17), Statistics(X)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ FlushableHashAggregateExecTransformer (13)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (11)
+ :- ^ ProjectExecTransformer (3)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (10)
+ +- BroadcastQueryStage (8), Statistics(X)
+ +- ColumnarBroadcastExchange (7)
+ +- ^ NoopFilter (5)
+ +- ^ Scan parquet (4)
+- == Initial Plan ==
- HashAggregate (33)
- +- Exchange (32)
- +- HashAggregate (31)
- +- Project (30)
- +- BroadcastHashJoin Inner BuildRight (29)
- :- Project (25)
- : +- Filter (24)
- : +- Scan parquet (23)
- +- BroadcastExchange (28)
- +- Filter (27)
- +- Scan parquet (26)
+ HashAggregate (34)
+ +- Exchange (33)
+ +- HashAggregate (32)
+ +- Project (31)
+ +- BroadcastHashJoin Inner BuildRight (30)
+ :- Project (26)
+ : +- Filter (25)
+ : +- Scan parquet (24)
+ +- BroadcastExchange (29)
+ +- Filter (28)
+ +- Scan parquet (27)
(1) Scan parquet
@@ -96,95 +97,99 @@ Results [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: false
-(15) ColumnarExchange
+(15) VeloxAppendBatches
+Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
+Arguments: X
+
+(16) ColumnarExchange
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(16) ShuffleQueryStage
+(17) ShuffleQueryStage
Output [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: X
-(17) InputAdapter
+(18) InputAdapter
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
-(18) InputIteratorTransformer
+(19) InputIteratorTransformer
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
-(19) RegularHashAggregateExecTransformer
+(20) RegularHashAggregateExecTransformer
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Keys: []
Functions [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)) [...]
Aggregate Attributes [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), Decim [...]
Results [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)) [...]
-(20) ProjectExecTransformer
+(21) ProjectExecTransformer
Output [1]: [CheckOverflow((promote_precision(CheckOverflow((100.0000 * promote_precision(sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END)#X)), DecimalType(38,6))) / promote_precision(cast(sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13 [...]
Input [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))) [...]
-(21) WholeStageCodegenTransformer (X)
+(22) WholeStageCodegenTransformer (X)
Input [1]: [promo_revenue#X]
Arguments: false
-(22) VeloxColumnarToRowExec
+(23) VeloxColumnarToRowExec
Input [1]: [promo_revenue#X]
-(23) Scan parquet
+(24) Scan parquet
Output [4]: [l_partkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1995-09-01), LessThan(l_shipdate,1995-10-01), IsNotNull(l_partkey)]
ReadSchema: struct<l_partkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(24) Filter
+(25) Filter
Input [4]: [l_partkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : (((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1995-09-01)) AND (l_shipdate#X < 1995-10-01)) AND isnotnull(l_partkey#X))
-(25) Project
+(26) Project
Output [3]: [l_partkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_partkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(26) Scan parquet
+(27) Scan parquet
Output [2]: [p_partkey#X, p_type#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(p_partkey)]
ReadSchema: struct<p_partkey:bigint,p_type:string>
-(27) Filter
+(28) Filter
Input [2]: [p_partkey#X, p_type#X]
Condition : isnotnull(p_partkey#X)
-(28) BroadcastExchange
+(29) BroadcastExchange
Input [2]: [p_partkey#X, p_type#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(29) BroadcastHashJoin
+(30) BroadcastHashJoin
Left keys [1]: [l_partkey#X]
Right keys [1]: [p_partkey#X]
Join condition: None
-(30) Project
+(31) Project
Output [3]: [l_extendedprice#X, l_discount#X, p_type#X]
Input [5]: [l_partkey#X, l_extendedprice#X, l_discount#X, p_partkey#X, p_type#X]
-(31) HashAggregate
+(32) HashAggregate
Input [3]: [l_extendedprice#X, l_discount#X, p_type#X]
Keys: []
Functions [2]: [partial_sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END), partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), De [...]
Aggregate Attributes [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Results [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
-(32) Exchange
+(33) Exchange
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=X]
-(33) HashAggregate
+(34) HashAggregate
Input [4]: [sum#X, isEmpty#X, sum#X, isEmpty#X]
Keys: []
Functions [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END), sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)) [...]
Aggregate Attributes [2]: [sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END)#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), Decim [...]
Results [1]: [CheckOverflow((promote_precision(CheckOverflow((100.0000 * promote_precision(sum(CASE WHEN StartsWith(p_type#X, PROMO) THEN CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) ELSE 0.0000 END)#X)), DecimalType(38,6))) / promote_precision(cast(sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(1 [...]
-(34) AdaptiveSparkPlan
+(35) AdaptiveSparkPlan
Output [1]: [promo_revenue#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/15.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/15.txt
index 2d463a561..40b342510 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/15.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/15.txt
@@ -1,42 +1,44 @@
== Physical Plan ==
-AdaptiveSparkPlan (41)
+AdaptiveSparkPlan (43)
+- == Final Plan ==
- VeloxColumnarToRowExec (26)
- +- AQEShuffleRead (25)
- +- ShuffleQueryStage (24), Statistics(X)
- +- ColumnarExchange (23)
- +- ^ ProjectExecTransformer (21)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (20)
- :- ^ InputIteratorTransformer (7)
- : +- BroadcastQueryStage (5), Statistics(X)
- : +- ColumnarBroadcastExchange (4)
- : +- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ FilterExecTransformer (19)
- +- ^ RegularHashAggregateExecTransformer (18)
- +- ^ InputIteratorTransformer (17)
- +- ShuffleQueryStage (15), Statistics(X)
- +- ColumnarExchange (14)
- +- ^ ProjectExecTransformer (12)
- +- ^ FlushableHashAggregateExecTransformer (11)
- +- ^ ProjectExecTransformer (10)
- +- ^ NoopFilter (9)
- +- ^ Scan parquet (8)
+ VeloxColumnarToRowExec (28)
+ +- AQEShuffleRead (27)
+ +- ShuffleQueryStage (26), Statistics(X)
+ +- ColumnarExchange (25)
+ +- VeloxAppendBatches (24)
+ +- ^ ProjectExecTransformer (22)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildLeft (21)
+ :- ^ InputIteratorTransformer (7)
+ : +- BroadcastQueryStage (5), Statistics(X)
+ : +- ColumnarBroadcastExchange (4)
+ : +- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ FilterExecTransformer (20)
+ +- ^ RegularHashAggregateExecTransformer (19)
+ +- ^ InputIteratorTransformer (18)
+ +- ShuffleQueryStage (16), Statistics(X)
+ +- ColumnarExchange (15)
+ +- VeloxAppendBatches (14)
+ +- ^ ProjectExecTransformer (12)
+ +- ^ FlushableHashAggregateExecTransformer (11)
+ +- ^ ProjectExecTransformer (10)
+ +- ^ NoopFilter (9)
+ +- ^ Scan parquet (8)
+- == Initial Plan ==
- Sort (40)
- +- Exchange (39)
- +- Project (38)
- +- BroadcastHashJoin Inner BuildLeft (37)
- :- BroadcastExchange (29)
- : +- Filter (28)
- : +- Scan parquet (27)
- +- Filter (36)
- +- HashAggregate (35)
- +- Exchange (34)
- +- HashAggregate (33)
- +- Project (32)
- +- Filter (31)
- +- Scan parquet (30)
+ Sort (42)
+ +- Exchange (41)
+ +- Project (40)
+ +- BroadcastHashJoin Inner BuildLeft (39)
+ :- BroadcastExchange (31)
+ : +- Filter (30)
+ : +- Scan parquet (29)
+ +- Filter (38)
+ +- HashAggregate (37)
+ +- Exchange (36)
+ +- HashAggregate (35)
+ +- Project (34)
+ +- Filter (33)
+ +- Scan parquet (32)
(1) Scan parquet
@@ -98,284 +100,297 @@ Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: false
-(14) ColumnarExchange
+(14) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(15) ColumnarExchange
Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_suppkey#X, 1), ENSURE_REQUIREMENTS, [l_suppkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(15) ShuffleQueryStage
+(16) ShuffleQueryStage
Output [3]: [l_suppkey#X, sum#X, isEmpty#X]
Arguments: X
-(16) InputAdapter
+(17) InputAdapter
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(17) InputIteratorTransformer
+(18) InputIteratorTransformer
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(18) RegularHashAggregateExecTransformer
+(19) RegularHashAggregateExecTransformer
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
Results [2]: [l_suppkey#X AS supplier_no#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS total_revenue#X]
-(19) FilterExecTransformer
+(20) FilterExecTransformer
Input [2]: [supplier_no#X, total_revenue#X]
Arguments: (isnotnull(total_revenue#X) AND (total_revenue#X = Subquery subquery#X, [id=#X]))
-(20) BroadcastHashJoinExecTransformer
+(21) BroadcastHashJoinExecTransformer
Left keys [1]: [s_suppkey#X]
Right keys [1]: [supplier_no#X]
Join condition: None
-(21) ProjectExecTransformer
+(22) ProjectExecTransformer
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Input [6]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, supplier_no#X, total_revenue#X]
-(22) WholeStageCodegenTransformer (X)
+(23) WholeStageCodegenTransformer (X)
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: false
-(23) ColumnarExchange
+(24) VeloxAppendBatches
+Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
+Arguments: X
+
+(25) ColumnarExchange
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: rangepartitioning(s_suppkey#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X], [id=#X]
-(24) ShuffleQueryStage
+(26) ShuffleQueryStage
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: X
-(25) AQEShuffleRead
+(27) AQEShuffleRead
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: local
-(26) VeloxColumnarToRowExec
+(28) VeloxColumnarToRowExec
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
-(27) Scan parquet
+(29) Scan parquet
Output [4]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(s_suppkey)]
ReadSchema: struct<s_suppkey:bigint,s_name:string,s_address:string,s_phone:string>
-(28) Filter
+(30) Filter
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X]
Condition : isnotnull(s_suppkey#X)
-(29) BroadcastExchange
+(31) BroadcastExchange
Input [4]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X]
Arguments: HashedRelationBroadcastMode(List(input[0, bigint, false]),false), [plan_id=X]
-(30) Scan parquet
+(32) Scan parquet
Output [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1996-01-01), LessThan(l_shipdate,1996-04-01), IsNotNull(l_suppkey)]
ReadSchema: struct<l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(31) Filter
+(33) Filter
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : (((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1996-01-01)) AND (l_shipdate#X < 1996-04-01)) AND isnotnull(l_suppkey#X))
-(32) Project
+(34) Project
Output [3]: [l_suppkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(33) HashAggregate
+(35) HashAggregate
Input [3]: [l_suppkey#X, l_extendedprice#X, l_discount#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(34) Exchange
+(36) Exchange
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(35) HashAggregate
+(37) HashAggregate
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
Results [2]: [l_suppkey#X AS supplier_no#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS total_revenue#X]
-(36) Filter
+(38) Filter
Input [2]: [supplier_no#X, total_revenue#X]
Condition : (isnotnull(total_revenue#X) AND (total_revenue#X = Subquery subquery#X, [id=#X]))
-(37) BroadcastHashJoin
+(39) BroadcastHashJoin
Left keys [1]: [s_suppkey#X]
Right keys [1]: [supplier_no#X]
Join condition: None
-(38) Project
+(40) Project
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Input [6]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, supplier_no#X, total_revenue#X]
-(39) Exchange
+(41) Exchange
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: rangepartitioning(s_suppkey#X ASC NULLS FIRST, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(40) Sort
+(42) Sort
Input [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: [s_suppkey#X ASC NULLS FIRST], true, 0
-(41) AdaptiveSparkPlan
+(43) AdaptiveSparkPlan
Output [5]: [s_suppkey#X, s_name#X, s_address#X, s_phone#X, total_revenue#X]
Arguments: isFinalPlan=true
===== Subqueries =====
-Subquery:1 Hosting operator id = 19 Hosting Expression = Subquery subquery#X, [id=#X]
-AdaptiveSparkPlan (66)
+Subquery:1 Hosting operator id = 20 Hosting Expression = Subquery subquery#X, [id=#X]
+AdaptiveSparkPlan (69)
+- == Final Plan ==
- VeloxColumnarToRowExec (57)
- +- ^ RegularHashAggregateExecTransformer (55)
- +- ^ RegularHashAggregateExecTransformer (54)
- +- ^ ProjectExecTransformer (53)
- +- ^ RegularHashAggregateExecTransformer (52)
- +- ^ InputIteratorTransformer (51)
- +- ShuffleQueryStage (49), Statistics(X)
- +- ColumnarExchange (48)
- +- ^ ProjectExecTransformer (46)
- +- ^ FlushableHashAggregateExecTransformer (45)
- +- ^ ProjectExecTransformer (44)
- +- ^ NoopFilter (43)
- +- ^ Scan parquet (42)
+ VeloxColumnarToRowExec (60)
+ +- ^ RegularHashAggregateExecTransformer (58)
+ +- ^ RegularHashAggregateExecTransformer (57)
+ +- ^ ProjectExecTransformer (56)
+ +- ^ RegularHashAggregateExecTransformer (55)
+ +- ^ InputIteratorTransformer (54)
+ +- ShuffleQueryStage (52), Statistics(X)
+ +- ColumnarExchange (51)
+ +- VeloxAppendBatches (50)
+ +- ^ ProjectExecTransformer (48)
+ +- ^ FlushableHashAggregateExecTransformer (47)
+ +- ^ ProjectExecTransformer (46)
+ +- ^ NoopFilter (45)
+ +- ^ Scan parquet (44)
+- == Initial Plan ==
- HashAggregate (65)
- +- HashAggregate (64)
- +- HashAggregate (63)
- +- Exchange (62)
- +- HashAggregate (61)
- +- Project (60)
- +- Filter (59)
- +- Scan parquet (58)
+ HashAggregate (68)
+ +- HashAggregate (67)
+ +- HashAggregate (66)
+ +- Exchange (65)
+ +- HashAggregate (64)
+ +- Project (63)
+ +- Filter (62)
+ +- Scan parquet (61)
-(42) Scan parquet
+(44) Scan parquet
Output [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1996-01-01), LessThan(l_shipdate,1996-04-01)]
ReadSchema: struct<l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(43) NoopFilter
+(45) NoopFilter
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Arguments: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(44) ProjectExecTransformer
+(46) ProjectExecTransformer
Output [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)) AS _pre_X#X]
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(45) FlushableHashAggregateExecTransformer
+(47) FlushableHashAggregateExecTransformer
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, _pre_X#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [partial_sum(_pre_X#X)]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(46) ProjectExecTransformer
+(48) ProjectExecTransformer
Output [4]: [hash(l_suppkey#X, 42) AS hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(47) WholeStageCodegenTransformer (X)
+(49) WholeStageCodegenTransformer (X)
Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: false
-(48) ColumnarExchange
+(50) VeloxAppendBatches
+Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
+Arguments: X
+
+(51) ColumnarExchange
Input [4]: [hash_partition_key#X, l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_suppkey#X, 1), ENSURE_REQUIREMENTS, [l_suppkey#X, sum#X, isEmpty#X], [plan_id=X], [id=#X]
-(49) ShuffleQueryStage
+(52) ShuffleQueryStage
Output [3]: [l_suppkey#X, sum#X, isEmpty#X]
Arguments: X
-(50) InputAdapter
+(53) InputAdapter
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(51) InputIteratorTransformer
+(54) InputIteratorTransformer
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(52) RegularHashAggregateExecTransformer
+(55) RegularHashAggregateExecTransformer
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
Results [2]: [l_suppkey#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
-(53) ProjectExecTransformer
+(56) ProjectExecTransformer
Output [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS total_revenue#X]
Input [2]: [l_suppkey#X, sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
-(54) RegularHashAggregateExecTransformer
+(57) RegularHashAggregateExecTransformer
Input [1]: [total_revenue#X]
Keys: []
Functions [1]: [partial_max(total_revenue#X)]
Aggregate Attributes [1]: [max#X]
Results [1]: [max#X]
-(55) RegularHashAggregateExecTransformer
+(58) RegularHashAggregateExecTransformer
Input [1]: [max#X]
Keys: []
Functions [1]: [max(total_revenue#X)]
Aggregate Attributes [1]: [max(total_revenue#X)#X]
Results [1]: [max(total_revenue#X)#X AS max(total_revenue)#X]
-(56) WholeStageCodegenTransformer (X)
+(59) WholeStageCodegenTransformer (X)
Input [1]: [max(total_revenue)#X]
Arguments: false
-(57) VeloxColumnarToRowExec
+(60) VeloxColumnarToRowExec
Input [1]: [max(total_revenue)#X]
-(58) Scan parquet
+(61) Scan parquet
Output [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Batched: true
Location: InMemoryFileIndex [*]
PushedFilters: [IsNotNull(l_shipdate), GreaterThanOrEqual(l_shipdate,1996-01-01), LessThan(l_shipdate,1996-04-01)]
ReadSchema: struct<l_suppkey:bigint,l_extendedprice:decimal(12,2),l_discount:decimal(12,2),l_shipdate:date>
-(59) Filter
+(62) Filter
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
Condition : ((isnotnull(l_shipdate#X) AND (l_shipdate#X >= 1996-01-01)) AND (l_shipdate#X < 1996-04-01))
-(60) Project
+(63) Project
Output [3]: [l_suppkey#X, l_extendedprice#X, l_discount#X]
Input [4]: [l_suppkey#X, l_extendedprice#X, l_discount#X, l_shipdate#X]
-(61) HashAggregate
+(64) HashAggregate
Input [3]: [l_suppkey#X, l_extendedprice#X, l_discount#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [partial_sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [2]: [sum#X, isEmpty#X]
Results [3]: [l_suppkey#X, sum#X, isEmpty#X]
-(62) Exchange
+(65) Exchange
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Arguments: hashpartitioning(l_suppkey#X, 1), ENSURE_REQUIREMENTS, [plan_id=X]
-(63) HashAggregate
+(66) HashAggregate
Input [3]: [l_suppkey#X, sum#X, isEmpty#X]
Keys [1]: [l_suppkey#X]
Functions [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))]
Aggregate Attributes [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X]
Results [1]: [sum(CheckOverflow((promote_precision(cast(l_extendedprice#X as decimal(13,2))) * promote_precision(CheckOverflow((1.00 - promote_precision(cast(l_discount#X as decimal(13,2)))), DecimalType(13,2)))), DecimalType(26,4)))#X AS total_revenue#X]
-(64) HashAggregate
+(67) HashAggregate
Input [1]: [total_revenue#X]
Keys: []
Functions [1]: [partial_max(total_revenue#X)]
Aggregate Attributes [1]: [max#X]
Results [1]: [max#X]
-(65) HashAggregate
+(68) HashAggregate
Input [1]: [max#X]
Keys: []
Functions [1]: [max(total_revenue#X)]
Aggregate Attributes [1]: [max(total_revenue#X)#X]
Results [1]: [max(total_revenue#X)#X AS max(total_revenue)#X]
-(66) AdaptiveSparkPlan
+(69) AdaptiveSparkPlan
Output [1]: [max(total_revenue)#X]
Arguments: isFinalPlan=true
\ No newline at end of file
diff --git a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/16.txt b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/16.txt
index c4d21c42e..d579c1a70 100644
--- a/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/16.txt
+++ b/backends-velox/src/test/resources/tpch-approved-plan/v1-bhj-ras/spark33/16.txt
@@ -1,53 +1,56 @@
== Physical Plan ==
-AdaptiveSparkPlan (56)
+AdaptiveSparkPlan (59)
+- == Final Plan ==
- VeloxColumnarToRowExec (35)
- +- ^ SortExecTransformer (33)
- +- ^ InputIteratorTransformer (32)
- +- ShuffleQueryStage (30), Statistics(X)
- +- ColumnarExchange (29)
- +- ^ RegularHashAggregateExecTransformer (27)
- +- ^ InputIteratorTransformer (26)
- +- ShuffleQueryStage (24), Statistics(X)
- +- ColumnarExchange (23)
- +- ^ ProjectExecTransformer (21)
- +- ^ FlushableHashAggregateExecTransformer (20)
- +- ^ RegularHashAggregateExecTransformer (19)
- +- ^ InputIteratorTransformer (18)
- +- ShuffleQueryStage (16), Statistics(X)
- +- ColumnarExchange (15)
- +- ^ ProjectExecTransformer (13)
- +- ^ FlushableHashAggregateExecTransformer (12)
- +- ^ ProjectExecTransformer (11)
- +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
- :- ^ NoopFilter (2)
- : +- ^ Scan parquet (1)
- +- ^ InputIteratorTransformer (9)
- +- BroadcastQueryStage (7), Statistics(X)
- +- ColumnarBroadcastExchange (6)
- +- ^ NoopFilter (4)
- +- ^ Scan parquet (3)
+ VeloxColumnarToRowExec (38)
+ +- ^ SortExecTransformer (36)
+ +- ^ InputIteratorTransformer (35)
+ +- ShuffleQueryStage (33), Statistics(X)
+ +- ColumnarExchange (32)
+ +- VeloxAppendBatches (31)
+ +- ^ RegularHashAggregateExecTransformer (29)
+ +- ^ InputIteratorTransformer (28)
+ +- ShuffleQueryStage (26), Statistics(X)
+ +- ColumnarExchange (25)
+ +- VeloxAppendBatches (24)
+ +- ^ ProjectExecTransformer (22)
+ +- ^ FlushableHashAggregateExecTransformer (21)
+ +- ^ RegularHashAggregateExecTransformer (20)
+ +- ^ InputIteratorTransformer (19)
+ +- ShuffleQueryStage (17), Statistics(X)
+ +- ColumnarExchange (16)
+ +- VeloxAppendBatches (15)
+ +- ^ ProjectExecTransformer (13)
+ +- ^ FlushableHashAggregateExecTransformer (12)
+ +- ^ ProjectExecTransformer (11)
+ +- ^ BroadcastHashJoinExecTransformer Inner BuildRight (10)
+ :- ^ NoopFilter (2)
+ : +- ^ Scan parquet (1)
+ +- ^ InputIteratorTransformer (9)
... 109435 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org