[jira] [Resolved] (IMPALA-11281) Consider loading the table metadata for a ResetMetadataStmt

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-11281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang resolved IMPALA-11281.
-------------------------------------
    Fix Version/s: Impala 4.2.0
       Resolution: Fixed

Resolving this. There is a follow-up item tracked in IMPALA-11501. Thank [~fangyurao] !

> Consider loading the table metadata for a ResetMetadataStmt
> -----------------------------------------------------------
>
>                 Key: IMPALA-11281
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11281
>             Project: IMPALA
>          Issue Type: Bug
>            Reporter: Fang-Yu Rao
>            Assignee: Fang-Yu Rao
>            Priority: Major
>             Fix For: Impala 4.2.0
>
>
> Currently when a {{ResetMetadataStmt}} that has a non-null '{{tableName_}}' is being analyzed, we do not add its '{{{}tableName_{}}}' to the given '{{{}tblRefs{}}}' in {{collectTableRefs()}} if its '{{{}partitionSpec_{}}}' is null ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java#L131]).
> When the metadata of a table is not fully loaded, we won't populate the column names of a table in its corresponding {{AuthorizableTable}} ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java#L227L229]) since the table is an {{{}IncompleteTable{}}}.
> If the column names are not populated in the corresponding {{AuthorizableTable}} of a table in a {{{}ResetMetadataStmt{}}}, then the logic in [RangerAuthorizationChecker#authorizeByTableMasking()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L669-L684] that is supposed to block the metadata update when there are policies defined on the columns could not take effect since in this case [((AuthorizableTable) authorizable).getColumns()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L663] returns an empty list. As a result, such an update would be allowed if there is no other authorization error.
> To reproduce the issue, we could comment out all the test cases in [RangerAuditLogTest#testAuditsForColumnMasking()|https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java#L261] except for the following one. The following test case would fail since the query "{{{}invalidate metadata functional.alltypestiny{}}}" won't result in any authorization error. This test case could succeed with its previous test cases enabled because in the previous test cases, there is at least one invocation to {{SelectStmt#collectTableRefs()}} that triggers the metadata loading of the table '{{{}functional.alltypestiny{}}}'.
> {code:java}
>       // Updates on metadata fails by column-masking policies.
>       authzError(events -> {
>         assertEquals(1, events.size());
>         assertEquals("invalidate metadata functional.alltypestiny",
>             events.get(0).getRequestData());
>         assertEventEquals("@table", "refresh", "functional/alltypestiny", 0,
>             events.get(0));
>         // Make sure it's denied by a column masking policy.
>         assertTrue(columnMaskingPolicyIds.contains(events.get(0).getPolicyId()));
>       }, "invalidate metadata functional.alltypestiny", onServer(TPrivilegeLevel.ALL));
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)