You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Peter Rozsa (Code Review)" <ge...@cloudera.org> on 2023/04/24 12:12:58 UTC
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Peter Rozsa has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19790
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
IMPALA-12042: Invalid casts in set operations calculation
This change fixes invalid casts in set operations where the common type
calculation fails to determine the incompatibility between 3 or more
types. The previous solution does not check every possible combinations
between types (nCr); the current approach provides a solution to this
problem by grouping the available types in the statement's slots and
reducing the available type set by calculating every combination until
only one type remains or any type combination yield an invalid type.
Tests:
- new test case added to AnalyzeStmtTests
Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
A fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java
M fe/src/main/java/org/apache/impala/catalog/Type.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
4 files changed, 211 insertions(+), 36 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/19790/1
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
IMPALA-12042: Invalid casts in set operations calculation
This change fixes invalid casts in set operations where the common type
calculation fails to determine the incompatibility between 3 or more
types. The previous solution does not check every possible combinations
between types (nCr); the current approach provides a solution to this
problem by grouping the available types in the statement's slots and
reducing the available type set by calculating every combination until
only one type remains or any type combination yield an invalid type.
Tests:
- new test case added to AnalyzeStmtTests
Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
A fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java
M fe/src/main/java/org/apache/impala/catalog/Type.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
4 files changed, 212 insertions(+), 36 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/19790/2
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12948/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 14:29:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19790/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19790/4//COMMIT_MSG@9
PS4, Line 9: This patch adds an additional type check for every expression
We could add the example from the Jira ticket.
http://gerrit.cloudera.org:8080/#/c/19790/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:
http://gerrit.cloudera.org:8080/#/c/19790/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3317
PS4, Line 3317: getCompatibleType(compatibleType, widestExprs.get(i), exprLists.get(j).get(i));
We could check that this returns 'compatibleType' in a precondition check.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 14:44:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 May 2023 12:24:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19790/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19790/4//COMMIT_MSG@9
PS4, Line 9: This patch adds an additional type check for every expression
> We could add the example from the Jira ticket.
Done
http://gerrit.cloudera.org:8080/#/c/19790/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:
http://gerrit.cloudera.org:8080/#/c/19790/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3317
PS4, Line 3317: Type commonType = getCompatibleType(compatibleType,
> We could check that this returns 'compatibleType' in a precondition check.
Done
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 15:26:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19790/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java:
http://gerrit.cloudera.org:8080/#/c/19790/1/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@3563
PS1, Line 3563: "Incompatible return types 'BOOLEAN' and 'DECIMAL(4,1)' of exprs 'TRUE' and '111.0'");
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Apr 2023 12:13:49 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12848/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Apr 2023 12:44:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 11:53:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 6: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 11:54:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 May 2023 07:10:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 6: Code-Review+2
The build error seems unrelated, restarting the test.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 May 2023 07:09:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9297/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 May 2023 07:10:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
IMPALA-12042: Fix invalid casts in set operations calculation
This patch adds an additional type check for every expression
listed in set operations, avoiding invalid casts.
Test:
- test case added to AnalyzeStmtsTest.java
Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
2 files changed, 7 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/19790/4
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9295/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 11:54:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has removed a vote on this change.
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Removed -Verified by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has removed a vote on this change.
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12936/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 May 2023 17:20:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
IMPALA-12042: Fix invalid casts in set operations calculation
This patch adds an additional type check for every expression
listed in set operations, avoiding invalid casts. For example:
values (true), (123), (111.0) produced an invalid cast from
BOOLEAN to DECIMAL(4,1), now, the additional check detects the
incompatibility before creating the cast expressions.
Test:
- test case added to AnalyzeStmtsTest.java
Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Reviewed-on: http://gerrit.cloudera.org:8080/19790
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
2 files changed, 9 insertions(+), 0 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 8
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 6: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9295/
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 17:06:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12949/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 15:46:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
Patch Set 3:
(22 comments)
http://gerrit.cloudera.org:8080/#/c/19790/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19790/3//COMMIT_MSG@15
PS3, Line 15: yield
Nit: yields
http://gerrit.cloudera.org:8080/#/c/19790/3//COMMIT_MSG@47
PS3, Line 47: 1) A B C D E
Could you describe how to read this example? I don't understand it.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java
File fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java:
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@61
PS3, Line 61: t
Nit: '.' at the end of the sentence.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@67
PS3, Line 67: firstList.size()
Could be 'size', defined on L63.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@109
PS3, Line 109: maximal size
Some more context is needed here, otherwise it's difficult to understand that this is about type sizes. Although the algorithm is described in the commit message it should also be described here completely because when someone is reading the code later they won't see the commit message.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@112
PS3, Line 112: originated
Did you mean 'are originated'?
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@117
PS3, Line 117: m
Nit: '.' at the end of the sentence.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@133
PS3, Line 133:
Nit: unneeded space.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@135
PS3, Line 135:
Nit: unneeded space.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@143
PS3, Line 143: getLeftmostRightExpression
Can't we get a misleading error message this way? It can be that it is not 'expr' and the leftmost right expression that are incompatible, can't it? Shouldn't we give a custom error message that lists the types from which the common type was deduced and that it is incompatible with this type?
This also applies to other calls to getCompatibleType(), on L198 and L212.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@150
PS3, Line 150: * Expression lists could contain the same type multiple times, which could
Some context would be useful also here, about parametric types. It could be explained that we group them by their primitive types (and scale in case of Decimals) - the sentence beginning on L152 refers to "the grouping" as something known but it hasn't been explained yet.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@158
PS3, Line 158: filteredExpr
Nit: filteredExprs? There are several expressions.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@163
PS3, Line 163: if (type instanceof ScalarType) {
This check is unnecessary, type.isDecimal() on the next line is enough.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@172
PS3, Line 172: orderedExpr.get()
Could use 'expr'.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@193
PS3, Line 193: private TypeCompatibility compatibilityReducer(Set<TypeCompatibility> combination)
These two reducers could be binary functions and the extraction of the two elements could be moved to the caller (getCompatibilities()). This reduces code duplication and also expresses the intent of binary reduction more clearly.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@241
PS3, Line 241: private int getTypeHashCode(Type type) {
Could be static.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@294
PS3, Line 294: left_
I don't think we should shadow the fields in the parent class, so we could call this 'leftExpr_'. I'd also set the these parent fields explicitly by using the 3-argument constructor - we could either set them to the types of the expressions or null, explaining that they are not used.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@323
PS3, Line 323: Ordered
Shouldn't it be 'HashableExpr'? It doesn't provide an ordering (which would be useful with tree-based containers) but a hash and equals function.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java
File fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java:
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java@69
PS3, Line 69: NumericLiteral
This will not necessarily be a DECIMAL, it can be an integer also, depending on the value (see https://github.com/apache/impala/blob/f826f7212438604990fa707315195d77b3bf2a5d/fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java#L113). You could use a constructor where you provide the type explicitly.
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java@82
PS3, Line 82: NumericLiteral
Providing explicit types in the constructor of NumericLiteral would be safer in my opinion (applies to the other calls in this function, too).
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java@120
PS3, Line 120: Dynamic
ParametricTypes?
http://gerrit.cloudera.org:8080/#/c/19790/3/fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java@136
PS3, Line 136: Assert.assertThat(actualTypes, Matchers.contains(expectedTypes));
Don't we want to test that the list of types doesn't contain any other type?
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 11:23:35 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
IMPALA-12042: Fix invalid casts in set operations calculation
This patch adds an additional type check for every expression
listed in set operations, avoiding invalid casts. For example:
values (true), (123), (111.0) produced an invalid cast from
BOOLEAN to DECIMAL(4,1), now, the additional check detects the
incompatibility before creating the cast expressions.
Test:
- test case added to AnalyzeStmtsTest.java
Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
2 files changed, 9 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/19790/5
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
IMPALA-12042: Invalid casts in set operations calculation
This change fixes invalid casts in set operations where the common type
calculation fails to determine the incompatibility between 3 or more
types. The previous solution does not check every possible combination
between types; the new approach provides a solution to this
problem by grouping the available types in the statement's slots and
reducing the available type set by calculating every combination until
only one type remains or any type combination yield an invalid type.
The algorithm has 4 steps:
1) The initial list of expressions is transposed; this format orders
the expressions in a way that every slot has its respective list of
expressions, making the type combination creation easier.
2) A simplification step groups the expressions by types; every group
stores the first maximal size type, this can be done because every
parametric type has the property to cast a smaller size type to a
bigger size type, for example, VARCHAR(10) could be cast to
VARCHAR(11) without any concern. This step significantly reduces the
number of intermediate-type combinations, regardless of the size of
the expression list, the output's upper bound is the number of available
types (DECIMAL's with different scales are treated as separate types).
3) The root type compatibilities are calculated from the simplified
list of expressions and stored in a set. The root type compatibility
stores the original expressions that are used for error reporting.
After the first round of compatibility calculation, regular
compatibilities are getting calculated, which contain their parent
compatibilities. With this tree-like structure, in case of type
compatibility errors, the original expressions can be traced back and
reported properly. The compatibility calculation goes until the size
of the set goes to one or a compatibility error happens.
4) If the common type is deduced, a final step goes through the list of
all expressions and validates that every expression is compatible with
it. This step is required because of the intermediate "type-casting"
condition; if the combination of two types turns out to be a third,
different type, the remaining combination set might not deduce the
incompatibility. Example:
1) A B C D E
2) B D F (A, B) -> B, (C, D) -> D, (D, E) -> F (new type introduced)
3) D F (B, D) -> D, (D, F) -> F
4) F (D, F) -> F
(A, F) check is missing because A is sorted out in the first iteration,
and might cause incompatibility.
This final step will check unconditionally that every expression
is compatible with the common type.
Tests:
- new test case added to AnalyzeStmtTests
- unit test added to SetCompatibilityHandler
Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
A fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java
M fe/src/main/java/org/apache/impala/analysis/SetOperationStmt.java
M fe/src/main/java/org/apache/impala/catalog/Type.java
M fe/src/main/java/org/apache/impala/planner/PlanFragment.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A fe/src/test/java/org/apache/impala/analysis/SetCompatibilityHandlerTest.java
7 files changed, 514 insertions(+), 55 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/19790/3
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-12042: Fix invalid casts in set operations calculation
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Fix invalid casts in set operations calculation
......................................................................
Patch Set 5: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 10:55:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12846/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Apr 2023 12:34:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12042: Invalid casts in set operations calculation
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19790 )
Change subject: IMPALA-12042: Invalid casts in set operations calculation
......................................................................
Patch Set 2:
(16 comments)
http://gerrit.cloudera.org:8080/#/c/19790/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19790/2//COMMIT_MSG@11
PS2, Line 11: combinations
Nit: combination
http://gerrit.cloudera.org:8080/#/c/19790/2//COMMIT_MSG@12
PS2, Line 12: (nCr)
What does this mean?
http://gerrit.cloudera.org:8080/#/c/19790/2//COMMIT_MSG@12
PS2, Line 12: current
The current approach could mean the one used before this change, "the new approach" or "the approach introduced by this change" would be better.
http://gerrit.cloudera.org:8080/#/c/19790/2//COMMIT_MSG@13
PS2, Line 13: problem by grouping the available types in the statement's slots and
Could you explain the algorithm in a bit more detail? How we group types (unifying parametric types) and why this approach is correct - will it catch every inconsistency? Does the order of the elements matter?
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@3285
PS2, Line 3285: Throw
Nit: we could change it to "Throws".
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java
File fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java:
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@48
PS2, Line 48: Restructuring
Transpose would be more descriptive.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@49
PS2, Line 49: restructuredList
Could be 'transposedList'.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@50
PS2, Line 50: int
Nit: could be final.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@50
PS2, Line 50: int bound = firstList.size();
We could add a precondition check somewhere that checks that all lists have the same length.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@93
PS2, Line 93: common
Nit: "a common type".
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@94
PS2, Line 94: type
You could write more about how types are grouped, i.e. that this is about unifying for example CHAR(n) for all n etc.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@96
PS2, Line 96: combinations
You could mention that after unifying parametric types (CHAR, VARCHAR) the remaining number of types is small so we can work with the combination.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@126
PS2, Line 126: compatibleTypes.add(new TypeWithOrigin(t2.originExpr_, compatibleType));
What if 'compatibleType' is neither 't1' nor 't2'? For example if t1 is VARCHAR(2), t2 is VARCHAR(3) we can end up here:
https://github.com/apache/impala/blob/c8aa5796d93510723342055cc70cf8d00abae754/fe/src/main/java/org/apache/impala/catalog/ScalarType.java#L468
And the common type will be VARCHAR(3).
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@149
PS2, Line 149: private final Expr originExpr_;
I would put these private members after the overridden public functions OR before the constructors.
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/main/java/org/apache/impala/analysis/SetCompatibilityHandler.java@170
PS2, Line 170: * Helper class to make 'Type' class ordering possible in set operations.
Could you explain in more detail while this is needed and how it's achieved?
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java:
http://gerrit.cloudera.org:8080/#/c/19790/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@3561
PS2, Line 3561: // Transitive compatibility is not allowed (boolean -> tinyint -> decimal(4,1))
You could add: "Regression test for IMPALA-12042: ...".
--
To view, visit http://gerrit.cloudera.org:8080/19790
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02df42c67deda37b7f71db267dc761778a9caa2b
Gerrit-Change-Number: 19790
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Apr 2023 13:39:53 +0000
Gerrit-HasComments: Yes