You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/10/01 22:56:00 UTC
[jira] [Commented] (IMPALA-10936)
StmtMetadataLoader::collectPolicyTables() should handle
FailedLoadLocalTable without NPE
[ https://issues.apache.org/jira/browse/IMPALA-10936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423440#comment-17423440 ]
ASF subversion and git services commented on IMPALA-10936:
----------------------------------------------------------
Commit 4b8bb041b95529a8147981e7ca2c392e07611b2f in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4b8bb04 ]
IMPALA-10936: Fix NPE in collecting policy tables on FailedLoadLocalTable
Ranger column-masking/row-filtering policies can have complex
expressions that reference other tables (e.g. by using subqueries). So
when loading metadata of a table, we also need to load all tables
referenced by its related policies (i.e. "policy tables"). This requires
getting all its columns so we can get the column-masking policies.
However, for a failed loaded table in LocalCatalog mode, the table is
represented by a FailedLoadLocalTable which has a null ColumnMap. This
causes a NullPointerException when collecting policy tables for it.
This patch skips collecting policy tables on failed loaded tables. Also
add some null checks in LocalTable in case its ColumnMap is null. So
FailedLoadLocalTable can have the same robustness as IncompleteTable
which is the failed table representation in the legacy catalog mode.
Tests:
- Manually verified the NPE disappear in logs
- Added FE unit test
Change-Id: I7a6cd567685920211f05f2fdaac96bc98da41ac6
Reviewed-on: http://gerrit.cloudera.org:8080/17888
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> StmtMetadataLoader::collectPolicyTables() should handle FailedLoadLocalTable without NPE
> ----------------------------------------------------------------------------------------
>
> Key: IMPALA-10936
> URL: https://issues.apache.org/jira/browse/IMPALA-10936
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.1.0
> Reporter: Joe McDonnell
> Assignee: Quanlong Huang
> Priority: Major
>
> If table loading fails, then it result in a FailedLoadLocalTable, which uses this constructor for LocalTable:
> {noformat}
> protected LocalTable(LocalDb db, String tblName) {
> this.db_ = Preconditions.checkNotNull(db);
> this.name_ = tblName;
> this.ref_ = null;
> this.msTable_ = null;
> this.cols_ = null;
> this.tableStats_ = null;
> }{noformat}
> Since cols_ and other fields are null, the code in StmtMetadataLoader::collectPolicyTables(), which calls LocalTable::getColumnsInHiveOrder() will throw a NullPointerException when trying to access cols_ in getNonClusteringColumns().
> This will fail the query with a NullPointerException, even though the underlying problem came from failing to load the table metadata. We should handle this to avoid the NullPointerException. That may happen as part of StmtMetadataLoader::collectPolicyTables() or StmtMetadataLoader::getMissingTables().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org