You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Steve Carlin (Code Review)" <ge...@cloudera.org> on 2026/05/18 13:37:06 UTC

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Steve Carlin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24318


Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................

IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Calcite planner contains different semantics for pivot and unpivot.

In some cases, it fails. In some cases, it is returning different results.

This commit disables the feature for the Calcite planner until the
results can by synchronized.

Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
---
M java/calcite-planner/src/main/codegen/templates/Parser.jj
1 file changed, 8 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/24318/1
-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/24318/1/java/calcite-planner/src/main/codegen/templates/Parser.jj
File java/calcite-planner/src/main/codegen/templates/Parser.jj:

http://gerrit.cloudera.org:8080/#/c/24318/1/java/calcite-planner/src/main/codegen/templates/Parser.jj@2873
PS1, Line 2873:     <PIVOT>
> A lengthy explanation is needed...
That's tricky. Disabling it makes sense.



-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 20:39:00 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/22489/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 14:00:09 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/24318/1/java/calcite-planner/src/main/codegen/templates/Parser.jj
File java/calcite-planner/src/main/codegen/templates/Parser.jj:

http://gerrit.cloudera.org:8080/#/c/24318/1/java/calcite-planner/src/main/codegen/templates/Parser.jj@2873
PS1, Line 2873:     <PIVOT>
> Before IMPALA-2083, did these work with Calcite, or fail later?
A lengthy explanation is needed...

The PIVOT command was working with Calcite but there was a bug.  The Aggregate that came out of the command used a FILTER within the aggregate which was not handled properly.  There is a rule that allows you to change the FILTER to a CASE.  If I were to add that change, the PIVOT command will work, but....

...it works using SQL Server syntax.  I think whoever developed it came out of SQL server.  An inner select is needed.  Many cases in the test_pivot_clause.py file do not parse correctly.  Even worse, some of them that do parse return results that are compatible with SQL Server, but are different from what is developed on the original planner.



-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 19:11:11 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1: Code-Review+2

Disabling it makes sense for now


-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 22:09:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13511/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 17:27:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................

IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Calcite planner contains different semantics for pivot and unpivot.

In some cases, it fails. In some cases, it is returning different results.

This commit disables the feature for the Calcite planner until the
results can by synchronized.

Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Reviewed-on: http://gerrit.cloudera.org:8080/24318
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
---
M java/calcite-planner/src/main/codegen/templates/Parser.jj
1 file changed, 8 insertions(+), 2 deletions(-)

Approvals:
  Michael Smith: Looks good to me, but someone else must approve
  Impala Public Jenkins: Verified
  Joe McDonnell: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 2
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/24318/1/java/calcite-planner/src/main/codegen/templates/Parser.jj
File java/calcite-planner/src/main/codegen/templates/Parser.jj:

http://gerrit.cloudera.org:8080/#/c/24318/1/java/calcite-planner/src/main/codegen/templates/Parser.jj@2873
PS1, Line 2873:     <PIVOT>
Before IMPALA-2083, did these work with Calcite, or fail later?



-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 18:33:37 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24318 )

Change subject: IMPALA-2083: (addendum) Disable pivot and unpivot in Calcite planner
......................................................................


Patch Set 1: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24318
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd0e65721b38bca965a53f948ffc4c8a9f140ebb
Gerrit-Change-Number: 24318
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2026 21:55:59 +0000
Gerrit-HasComments: No