You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2021/09/30 02:40:24 UTC

[Impala-ASF-CR] IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17888

Change subject: IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
......................................................................

IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Ranger column-masking/row-filtering policies can have complex
expressions that reference other tables (e.g. by using subqueries). So
when loading metadata of a table, we also need to load all tables
referenced by its related policies (i.e. "policy tables"). This requires
getting all its columns so we can get the column-masking policies.
However, for a failed loaded table in LocalCatalog mode, the table is
represented by a FailedLoadLocalTable which has a null ColumnMap. This
causes a NullPointerException when collecting policy tables for it.

This patch skips collecting policy tables on failed loaded tables. Also
add some null checks in LocalTable in case its ColumnMap is null. So
FailedLoadLocalTable can have the same robustness as IncompleteTable
which is the failed table representation in the legacy catalog mode.

Tests:
- Manually verified the NPE disappear in logs
- Added FE unit test

Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
---
M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java
3 files changed, 40 insertions(+), 12 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/17888/1
--
To view, visit http://gerrit.cloudera.org:8080/17888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Gerrit-Change-Number: 17888
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.

Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17888 )

Change subject: IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
......................................................................

IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Tests:
- Manually verified the NPE disappear in logs
- Added FE unit test

Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Reviewed-on: http://gerrit.cloudera.org:8080/17888
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/test/java/org/apache/impala/analysis/StmtMetadataLoaderTest.java
3 files changed, 40 insertions(+), 12 deletions(-)

Approvals:
Joe McDonnell: Looks good to me, approved
Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/17888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Gerrit-Change-Number: 17888
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17888 )

Change subject: IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
......................................................................


Patch Set 1: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Gerrit-Change-Number: 17888
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 01 Oct 2021 21:44:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17888 )

Change subject: IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9532/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Gerrit-Change-Number: 17888
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Sep 2021 03:04:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.

Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17888 )

Change subject: IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
......................................................................


Patch Set 1: Code-Review+2

This makes sense to me


-- 
To view, visit http://gerrit.cloudera.org:8080/17888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Gerrit-Change-Number: 17888
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 01 Oct 2021 15:27:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17888 )

Change subject: IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7511/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/17888
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Gerrit-Change-Number: 17888
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 01 Oct 2021 15:28:08 +0000
Gerrit-HasComments: No