You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Andrew Sherman (Code Review)" <ge...@cloudera.org> on 2022/12/20 18:47:20 UTC

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Andrew Sherman has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19380


Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
21 files changed, 1,030 insertions(+), 108 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 6: Code-Review+2

Carry forward +2


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Jan 2023 17:49:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Hello Tamas Mate, Gabor Kaszab, Zoltan Borok-Nagy, Gergely Fürnstáhl, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19380

to look at the new patch set (#4).

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
18 files changed, 1,035 insertions(+), 104 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/4
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 8: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 8
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jan 2023 17:37:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12048/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 20:21:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Hello Tamas Mate, Gabor Kaszab, Zoltan Borok-Nagy, Gergely Fürnstáhl, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19380

to look at the new patch set (#7).

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
16 files changed, 1,036 insertions(+), 98 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/7
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 7
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 5: Code-Review+2

LGTM and nice tests!


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Jan 2023 14:06:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19380/1/tests/util/iceberg_util.py
File tests/util/iceberg_util.py:

http://gerrit.cloudera.org:8080/#/c/19380/1/tests/util/iceberg_util.py@28
PS1, Line 28:     2020-08-30 22:58:08.440000000	8270633197658268308	NULL	TRUE
tab used for whitespace



-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 18:48:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12046/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 19:07:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12122/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 22:38:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8962/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 8
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jan 2023 17:37:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12129/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Jan 2023 22:48:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 5: Code-Review+1

Thanks for the work, LGTM!


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Jan 2023 08:12:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 6:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/12139/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Jan 2023 18:09:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@1309
PS2, Line 1309: timeTravelTable.readSchema();
> I did that initially, but then I split it out. 
Splitting out to a separate function is nice, but I would call it in the constructor.

Deferred construction puts in extra layer of complexity to the usage of the object (we can forget to call this in the future).

In my opinion, it is worth it if
- the deferred part is computation-heavy
- and the object is "sane" and usable without it in some cases.


http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java:

http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@453
PS4, Line 453:         Path resolvedPath = analyzer_.resolvePathWithMasking(slotRef.getRawPath(),
             :             PathType.SLOT_REF, null);
We could add a similar overload for these guys too.


http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java:

http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java@197
PS4, Line 197: class ForwardingFeIcebergTable implements FeIcebergTable {
             :   private final FeIcebergTable base;
I agree 100% with the pattern, I just wish there was a better way to do it. C++ private inheritance comes close, but still need to "reimplement"/change/extend the API with forwarding to the base class.



-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Jan 2023 08:46:21 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
16 files changed, 1,036 insertions(+), 99 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/5
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 8: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 8
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jan 2023 22:47:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 7: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 7
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Jan 2023 16:55:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Hello Tamas Mate, Gabor Kaszab, Zoltan Borok-Nagy, Gergely Fürnstáhl, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19380

to look at the new patch set (#3).

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
18 files changed, 1,035 insertions(+), 104 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 7:

The gerrit-verify-dryrun tests run the dockerised tests which use local catalog mode.
This exposed a bug where my time travel changes were broken in local catalog mode. 
Fix this in LocalIcebergTable by calling the existing transfromToTHdfsTable() method.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 7
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Jan 2023 23:43:44 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19380

to look at the new patch set (#2).

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
21 files changed, 1,030 insertions(+), 108 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 2:

(5 comments)

Left a few nit comments, otherwise the change looks great!

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@1309
PS2, Line 1309: timeTravelTable.readSchema();
Could it be part of the constructor of IcebergTimeTravelTable?


http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java
File fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java@95
PS2, Line 95: , null)
Maybe we could overload Analyzer.resolvePath() so we wouldn't need to change unrelated code paths.


http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/TableRef.java
File fe/src/main/java/org/apache/impala/analysis/TableRef.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/TableRef.java@471
PS2, Line 471: so we
             :     // cannot check it yet.
"so we cannot check if the table supports time travel"?


http://gerrit.cloudera.org:8080/#/c/19380/2/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:

http://gerrit.cloudera.org:8080/#/c/19380/2/tests/authorization/test_ranger.py@1844
PS2, Line 1844:   
nit: +2 indent


http://gerrit.cloudera.org:8080/#/c/19380/2/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:

http://gerrit.cloudera.org:8080/#/c/19380/2/tests/query_test/test_iceberg.py@456
PS2, Line 456: # FIXME use new ts??
Please resolve this. Does the following query work?

 SELECT * FROM  {tbl} FOR SYSTEM_TIME AS OF <ts_5>
 MINUS
 SELECT *, NULL FROM  {tbl} FOR SYSTEM_TIME AS OF <ts_4>

Or since the second subquery has NULL the whole query returns every row? Then maybe we could

 SELECT i FROM  {tbl} FOR SYSTEM_TIME AS OF <ts_5>
 MINUS
 SELECT i FROM  {tbl} FOR SYSTEM_TIME AS OF <ts_4>



-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 05 Jan 2023 16:57:08 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 2:

(5 comments)

Thanks for review comments.

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@1309
PS2, Line 1309: timeTravelTable.readSchema();
> Could it be part of the constructor of IcebergTimeTravelTable?
I did that initially, but then I split it out. 
I think I prefer having the constructor be simple, and having work done in a method. 
If you feel strongly I will include it, but otherwise I'll leave this.


http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java
File fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java@95
PS2, Line 95: , null)
> Maybe we could overload Analyzer.resolvePath() so we wouldn't need to chang
Yes, that's nicer, done, thanks.


http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/TableRef.java
File fe/src/main/java/org/apache/impala/analysis/TableRef.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/TableRef.java@471
PS2, Line 471: so we
             :     // cannot check it yet.
> "so we cannot check if the table supports time travel"?
Done


http://gerrit.cloudera.org:8080/#/c/19380/2/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:

http://gerrit.cloudera.org:8080/#/c/19380/2/tests/authorization/test_ranger.py@1844
PS2, Line 1844:   
> nit: +2 indent
Done


http://gerrit.cloudera.org:8080/#/c/19380/2/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:

http://gerrit.cloudera.org:8080/#/c/19380/2/tests/query_test/test_iceberg.py@456
PS2, Line 456: # FIXME use new ts??
> Please resolve this. Does the following query work?
Thanks, done, it worked.



-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 22:37:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12121/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 22:22:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 4:

(3 comments)

Thanks for the comments

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/19380/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@1309
PS2, Line 1309: timeTravelSpec.analyze(this);
> Splitting out to a separate function is nice, but I would call it in the co
OK I will inline the readSchema() call, thanks!


http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java:

http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@453
PS4, Line 453:         Path resolvedPath = analyzer_.resolvePathWithMasking(slotRef.getRawPath(),
             :             PathType.SLOT_REF, null);
> We could add a similar overload for these guys too.
Done


http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java:

http://gerrit.cloudera.org:8080/#/c/19380/4/fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java@197
PS4, Line 197: class ForwardingFeIcebergTable implements FeIcebergTable {
             :   private final FeIcebergTable base;
> I agree 100% with the pattern, I just wish there was a better way to do it.
It is a lot of boilerplate, but at least the boilerplate code is in one place so is easy to maintain.



-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Jan 2023 22:29:20 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 6:

Verify tests fail in test_time_travel.
It seems like this test is being run with local catalog mode.
I can reproduce by running  with local catalog mode.
I will investigate further...


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Jan 2023 04:04:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Hello Tamas Mate, Gabor Kaszab, Zoltan Borok-Nagy, Gergely Fürnstáhl, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19380

to look at the new patch set (#6).

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
16 files changed, 1,036 insertions(+), 98 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/6
-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8958/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Jan 2023 17:53:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8958/


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Jan 2023 23:05:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12149/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 7
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Jan 2023 23:56:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................

IMPALA-10893: Use old schema during iceberg time travel.

Before this change the schema used during Iceberg Time Travel was the
current schema of the table. With this change we will use the schema
from the point specified by the Time Travel parameters.

The parameters used by an Iceberg Time Travel query are part of the FROM
clause of the query. Previously analysis of the Time Travel parameters
took place after the table Path was resolved, at which point some
schema information is cached. In order to use the old schema during
iceberg time travel however we need to ensure that the version of the
Table that is used is always the version specified by the Time Travel
parameters. To do this we have to move the analysis of the Time Travel
parameters inside the code that resolves the Path.

Add a new implementation of FeIcebergTable that represents an Iceberg
table involved in Time Travel. This is implemented by embedding a
reference to the base Iceberg Table. All methods that are not Time
Travel related are delegated to the base table. The Time Travel related
methods use the historic Iceberg schema.

TESTING:
- Add a new file iceberg_util.py to hold the snapshot utility code that
  was developed for the in-progress IMPALA-11482.
- Extend the existing Iceberg Time Travel tests to check the schema.
- Add a test that shows time travel working with columns masking.
  The column masking configuration is not tightly coupled to the schema
  so it is possible to mask historical columns.

Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Reviewed-on: http://gerrit.cloudera.org:8080/19380
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/authorization/test_ranger.py
M tests/common/iceberg_test_suite.py
M tests/query_test/test_iceberg.py
A tests/util/iceberg_util.py
16 files changed, 1,036 insertions(+), 98 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 9
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19380/3/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:

http://gerrit.cloudera.org:8080/#/c/19380/3/tests/query_test/test_iceberg.py@458
PS3, Line 458:  
flake8: W291 trailing whitespace


http://gerrit.cloudera.org:8080/#/c/19380/3/tests/query_test/test_iceberg.py@458
PS3, Line 458:                         SELECT *, NULL FROM {tbl} FOR SYSTEM_TIME 
line has trailing whitespace



-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 22:03:56 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10893: Use old schema during iceberg time travel.

Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19380 )

Change subject: IMPALA-10893: Use old schema during iceberg time travel.
......................................................................


Patch Set 4: Code-Review+1

Nice work, LGTM, added a few nitpicks


-- 
To view, visit http://gerrit.cloudera.org:8080/19380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab
Gerrit-Change-Number: 19380
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 09 Jan 2023 08:46:39 +0000
Gerrit-HasComments: No