You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Steve Carlin (Code Review)" <ge...@cloudera.org> on 2025/08/20 15:31:20 UTC
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Steve Carlin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23316
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
This commit registers the equivalent union expressions in the value
transfer graph when the physical union node is created for the Calcite
planner.
Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
---
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q05.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q10a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q54.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q71.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test
9 files changed, 553 insertions(+), 354 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/23316/1
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Sep 2025 22:58:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 3: Code-Review+2
(3 comments)
LGTM.
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
File java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java:
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@85
PS2, Line 85: if (unionAll) {
> Good point. Just added it.
Done
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@119
PS2, Line 119:
> Done
Done
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@134
PS2, Line 134: for (int i = 0; i < outputExprs.size(); ++i) {
> It does create a new TupleDescriptor which I believe creates new SlotDescri
If a new TupleDescriptor is created, then what you have is fine. Marking this done for now.
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 3
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 23 Sep 2025 23:59:08 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 3
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Sep 2025 18:09:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Hello Aman Sinha, Fang-Yu Rao, Riza Suminto, Joe McDonnell, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23316
to look at the new patch set (#3).
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
This commit registers the equivalent union expressions in the value
transfer graph when the physical union node is created for the Calcite
planner.
Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
---
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaUnionRel.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaValuesRel.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q05.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q10a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q54.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q71.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test
11 files changed, 530 insertions(+), 319 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/23316/3
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 3
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20208/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 Aug 2025 16:00:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/12524/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Sep 2025 18:09:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Hello Aman Sinha, Fang-Yu Rao, Riza Suminto, Joe McDonnell, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23316
to look at the new patch set (#2).
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
This commit registers the equivalent union expressions in the value
transfer graph when the physical union node is created for the Calcite
planner.
Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
---
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q05.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q10a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q54.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q71.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test
9 files changed, 520 insertions(+), 315 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/23316/2
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 2
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20397/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 2
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 Sep 2025 03:56:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
This commit registers the equivalent union expressions in the value
transfer graph when the physical union node is created for the Calcite
planner.
Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Reviewed-on: http://gerrit.cloudera.org:8080/23316
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaUnionRel.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaValuesRel.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q05.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q10a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q54.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q71.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test
11 files changed, 530 insertions(+), 319 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 5
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Sep 2025 18:09:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20565/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 3
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 23 Sep 2025 21:06:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
File java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java:
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@85
PS2, Line 85: registerUnionValueTransfers(analyzer, outputExprs, childrenPlanNodes);
> Is there any additional check to be done for Union Distinct vs Union All ?
Good point. Just added it.
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@119
PS2, Line 119: * Register the union value transfers which allows runtime filter generators
> Could you expand the comments to indicate that the value transfer is only b
Done
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@134
PS2, Line 134: SlotRef inputExpr = nodeWithExprs.outputExprs_.get(i).unwrapSlotRef(false);
> My understanding is that the slotRef of the left child of a UnionAll is the
It does create a new TupleDescriptor which I believe creates new SlotDescriptors and new SlotRefs.
If I am understand you correctly, I'm not sure if that's possible? Are you suggesting that one copy can be prevented? Not sure how that would work across multiple nodes, but I can do some research. Perhaps I should file a Jira?
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 2
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Mon, 22 Sep 2025 18:10:22 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/23316 )
Change subject: IMPALA-14106: Calcite planner: Register equivalent union expressions in value transfer graph
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
File java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java:
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@85
PS2, Line 85: registerUnionValueTransfers(analyzer, outputExprs, childrenPlanNodes);
Is there any additional check to be done for Union Distinct vs Union All ? I think only for the latter we should be creating the value transfer but I am not sure if we use the same Union plan node for both. If we only use a UnionAll plan node for both and expect that the Distinct part is done below by an Aggregate, then it should be ok.
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@119
PS2, Line 119: * Register the union value transfers which allows runtime filter generators
Could you expand the comments to indicate that the value transfer is only between the output expr of the Union and each of the child inputs, and not a value transfer between the child inputs themselves.
So if you have
UNION ALL which has output exprs out_expr1, out_expr2 and it has 2 children: LHS: produces lhs_expr1, lhs_expr2 RHS: produces rhs_expr1, rhs_expr2
The value transfer is NOT between lhs_expr1 <-> rhs_expr1, but rather out_expr1 <-> lhs_expr1 and out_expr1 <-> rhs_expr1.
This is different from an equality Join, because there we do create a value transfer between the join columns coming from each child.
http://gerrit.cloudera.org:8080/#/c/23316/2/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java@134
PS2, Line 134: SlotRef inputExpr = nodeWithExprs.outputExprs_.get(i).unwrapSlotRef(false);
My understanding is that the slotRef of the left child of a UnionAll is the same as the slotRef of the output expr of the UnionAll. Do you see a difference ? in other words, if I have SELECT a, b UNION ALL c, d. then the output expr of the UnionAll is already a, b.
--
To view, visit http://gerrit.cloudera.org:8080/23316
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c858ae82a1cb7b89b0ae4e70205d8eeaeb28687
Gerrit-Change-Number: 23316
Gerrit-PatchSet: 2
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sat, 20 Sep 2025 19:12:48 +0000
Gerrit-HasComments: Yes