You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Peter Rozsa (Code Review)" <ge...@cloudera.org> on 2023/07/11 11:20:56 UTC

[Impala-ASF-CR] IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

Peter Rozsa has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20184


Change subject: IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check
......................................................................

IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

Change-Id: I39d13f177482f74ec39570118adab609444c6929
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
M fe/src/main/java/org/apache/impala/analysis/TimestampLiteral.java
M fe/src/test/java/org/apache/impala/analysis/NumericLiteralTest.java
M testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
12 files changed, 94 insertions(+), 54 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/20184/1
-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 2:

(12 comments)

http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@7
PS2, Line 7:  
Add (Addendum), as the Jira ticket is not specifically for this change.


http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@9
PS2, Line 9: fixes
Could you describe what the problem was? A test could be added for a problematic query.


http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@10
PS2, Line 10: now
We often describe the state before the change as "now" in commit messages. This may not be a good habit, but I think we should be more explicit here, we could say "after this change".


http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@12
PS2, Line 12:  
I'd put the following sentences in a new paragraph, as they address a different problem.


http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@13
PS2, Line 13: now
See comment on L10.


http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@15
PS2, Line 15: ,
Nit: comma not needed.


http://gerrit.cloudera.org:8080/#/c/20184/2//COMMIT_MSG@16
PS2, Line 16: another one contains non-constant expressions with regular casts in one
Could also add an example of what didn't use to work but works now.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java
File fe/src/main/java/org/apache/impala/analysis/CastExpr.java:

http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@55
PS2, Line 55: defined
Nit: . (full stop) at the end.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@56
PS2, Line 56:   private TypeCompatibility compatibility_ = TypeCompatibility.DEFAULT;
Could be final.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/Expr.java
File fe/src/main/java/org/apache/impala/analysis/Expr.java:

http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/Expr.java@1115
PS2, Line 1115: castExpr
Optional: could be 'thisCastExpr'.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
File fe/src/main/java/org/apache/impala/analysis/NullLiteral.java:

http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/NullLiteral.java@48
PS2, Line 48: TypeCompatibility.DEFAULT
Optional: you could use the 1-argument overload inherited from Expr.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
File fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java:

http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java@470
PS2, Line 470:       return new CastExpr(targetType, this);
Don't we need to use 'compatibility' here?



-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jul 2023 16:05:07 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check
......................................................................

IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

This change fixes the expression substitution with an unsafely casted
expression; now the CastExrp contains the level of compatibility which
was used to create the expression, therefore substitutions can recreate
the expression as is. The patch also fixes the column level
compatibility check in set operations; now compatibility level is set
to the regular level initially for every column check, it will allow
cases, where one column contains unsafely casted constant values and
another one contains non-constant expressions with regular casts in one
set operation.

Tests:
 - test case added to insert-unsafe.test

Change-Id: I39d13f177482f74ec39570118adab609444c6929
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
M fe/src/main/java/org/apache/impala/analysis/TimestampLiteral.java
M fe/src/test/java/org/apache/impala/analysis/NumericLiteralTest.java
M testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
12 files changed, 94 insertions(+), 54 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/20184/2
-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9534/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 13:48:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13518/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jul 2023 12:09:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 3:

(8 comments)

http://gerrit.cloudera.org:8080/#/c/20184/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/20184/3//COMMIT_MSG@10
PS3, Line 10: CastExrp
Nit: type, should be CastExpr.


http://gerrit.cloudera.org:8080/#/c/20184/3//COMMIT_MSG@12
PS3, Line 12: Before this change
Optional: I'd start the commit message with the problem and then describe how it's fixed.


http://gerrit.cloudera.org:8080/#/c/20184/3//COMMIT_MSG@16
PS3, Line 16: 'select "1", "1" union
            : select 1, "1"'
Nit: I'd put the example on a separate line.


http://gerrit.cloudera.org:8080/#/c/20184/3//COMMIT_MSG@21
PS3, Line 21: it will allow cases
Could you make it clearer that this didn't work before the patch? I'd start with the problem and then explain how it's solved.


http://gerrit.cloudera.org:8080/#/c/20184/3//COMMIT_MSG@24
PS3, Line 24: 'select "1", 1 union select 1, int_col from
            : unsafe_insert'
Nit: I'd put the example on a separate line.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java
File fe/src/main/java/org/apache/impala/analysis/CastExpr.java:

http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/CastExpr.java@56
PS2, Line 56:   private TypeCompatibility compatibility_ = TypeCompatibility.DEFAULT;
> Unfortunately, it can't, it's initialized with a default value, and assigne
In this case I think it would be more correct if there wasn't a default value here but it would be initialised in each constructor. This is a value that should never change.


http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
File fe/src/main/java/org/apache/impala/analysis/NullLiteral.java:

http://gerrit.cloudera.org:8080/#/c/20184/2/fe/src/main/java/org/apache/impala/analysis/NullLiteral.java@48
PS2, Line 48: TypeCompatibility.DEFAULT
> The 1-argument base method has a throws clause, that would generate 14 more
Ok.


http://gerrit.cloudera.org:8080/#/c/20184/3/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
File testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test:

http://gerrit.cloudera.org:8080/#/c/20184/3/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test@213
PS3, Line 213: ====
Could add a comment that this is a case where one column is needs unsafe compatibility and the other needs default compatibility.



-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Jul 2023 14:15:22 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 20:49:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................

IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

This change fixes the expression substitution with an unsafely casted
expression; after this change the CastExrp contains the level of
compatibility which was used to create the expression, therefore
substitutions can recreate the expression as is. Before this change, the
information regarding compatibility level was not present at the
substitution part and cases where the root type must be preserved, the
unsafely casted expressions got casted with regular compatibility,
causing IllegalStateException. For example: 'select "1", "1" union
select 1, "1"'

The patch also fixes the column-level compatibility check in set
operations; after this change compatibility level is set to the regular
level initially for every column check, it will allow cases where
one column contains unsafely casted constant values and another one
contains non-constant expressions with regular casts in one set
operation. For example: 'select "1", 1 union select 1, int_col from
unsafe_insert'

Tests:
 - test case added to insert-unsafe.test

Change-Id: I39d13f177482f74ec39570118adab609444c6929
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
M fe/src/main/java/org/apache/impala/analysis/TimestampLiteral.java
M fe/src/test/java/org/apache/impala/analysis/NumericLiteralTest.java
M testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
12 files changed, 98 insertions(+), 55 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/20184/3
-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 4: Code-Review+1

(4 comments)

Thanks, only some nits.

http://gerrit.cloudera.org:8080/#/c/20184/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/20184/4//COMMIT_MSG@18
PS4, Line 18: Set
Could start it with "Also, ..." to indicate that this is an additional issue that this change resolves.


http://gerrit.cloudera.org:8080/#/c/20184/4/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
File testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test:

http://gerrit.cloudera.org:8080/#/c/20184/4/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test@214
PS4, Line 214: # Mixing unsafe and regular compatibility on column level, unsafe union between 1 (TINYINT) and "1" (STRING)
Nit: it would be better to keep the line width of the comment to 90.


http://gerrit.cloudera.org:8080/#/c/20184/4/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test@215
PS4, Line 215: "
Nit: full stop (.) at the end.


http://gerrit.cloudera.org:8080/#/c/20184/4/testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test@219
PS4, Line 219: casts
Nit: full stop (.) at the end.



-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 10:39:35 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................

IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Expression substitution recreates cast expressions without considering
the compatibility level introduced by IMPALA-10173. In unsafe mode, the
recreation causes IllegalStateException. This change fixes this
behavior by storing the compatibility level in each CastExpr, and
reusing it when the expression substitution recreates the cast
expression.

For example: 'select "1", "1" union select 1, "1"'

Set operation's common type calculations did not distinguish
compatibility levels for each column slot, if one column slot's common
type was considered unsafe, every other slot was treated as unsafe.
This change fixes this behavior by reinitializing the compatibility
level for every column slot, enabling cases where one column slot
contains unsafely casted constant values and another contains
non-constant expressions with regular casts.
These queries failed before this change with 'Unsafe implicit cast is
prohibited for non-const expression' error.

For example: 'select "1", 1 union select 1, int_col from unsafe_insert'

Tests:
 - test cases added to insert-unsafe.test

Change-Id: I39d13f177482f74ec39570118adab609444c6929
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
M fe/src/main/java/org/apache/impala/analysis/TimestampLiteral.java
M fe/src/test/java/org/apache/impala/analysis/NumericLiteralTest.java
M testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
12 files changed, 103 insertions(+), 55 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/20184/4
-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13529/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Jul 2023 15:33:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................

IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Expression substitution recreates cast expressions without considering
the compatibility level introduced by IMPALA-10173. In unsafe mode, the
recreation causes IllegalStateException. This change fixes this
behavior by storing the compatibility level in each CastExpr, and
reusing it when the expression substitution recreates the cast
expression.

For example: 'select "1", "1" union select 1, "1"'

Also, Set operation's common type calculations did not distinguish
compatibility levels for each column slot, if one column slot's common
type was considered unsafe, every other slot was treated as unsafe.
This change fixes this behavior by reinitializing the compatibility
level for every column slot, enabling cases where one column slot
contains unsafely casted constant values and another contains
non-constant expressions with regular casts.
These queries failed before this change with 'Unsafe implicit cast is
prohibited for non-const expression' error.

For example: 'select "1", 1 union select 1, int_col from unsafe_insert'

Tests:
 - test cases added to insert-unsafe.test

Change-Id: I39d13f177482f74ec39570118adab609444c6929
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
M fe/src/main/java/org/apache/impala/analysis/TimestampLiteral.java
M fe/src/test/java/org/apache/impala/analysis/NumericLiteralTest.java
M testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
12 files changed, 103 insertions(+), 55 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/20184/5
-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................

IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Expression substitution recreates cast expressions without considering
the compatibility level introduced by IMPALA-10173. In unsafe mode, the
recreation causes IllegalStateException. This change fixes this
behavior by storing the compatibility level in each CastExpr, and
reusing it when the expression substitution recreates the cast
expression.

For example: 'select "1", "1" union select 1, "1"'

Also, Set operation's common type calculations did not distinguish
compatibility levels for each column slot, if one column slot's common
type was considered unsafe, every other slot was treated as unsafe.
This change fixes this behavior by reinitializing the compatibility
level for every column slot, enabling cases where one column slot
contains unsafely casted constant values and another contains
non-constant expressions with regular casts.
These queries failed before this change with 'Unsafe implicit cast is
prohibited for non-const expression' error.

For example: 'select "1", 1 union select 1, int_col from unsafe_insert'

Tests:
 - test cases added to insert-unsafe.test

Change-Id: I39d13f177482f74ec39570118adab609444c6929
Reviewed-on: http://gerrit.cloudera.org:8080/20184
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/main/java/org/apache/impala/analysis/DateLiteral.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/NullLiteral.java
M fe/src/main/java/org/apache/impala/analysis/NumericLiteral.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/StringLiteral.java
M fe/src/main/java/org/apache/impala/analysis/TimestampLiteral.java
M fe/src/test/java/org/apache/impala/analysis/NumericLiteralTest.java
M testdata/workloads/functional-query/queries/QueryTest/insert-unsafe.test
12 files changed, 103 insertions(+), 55 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13649/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 13:08:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 5: Code-Review+2

Thanks Péter.


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 13:47:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 13:48:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: (Addendum) Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13644/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Jul 2023 07:32:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20184 )

Change subject: IMPALA-10173: Fix substitution for unsafe expressions, column-level compatibility check
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/13514/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20184
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I39d13f177482f74ec39570118adab609444c6929
Gerrit-Change-Number: 20184
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jul 2023 11:43:48 +0000
Gerrit-HasComments: No