You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Fang-Yu Rao (Jira)" <ji...@apache.org> on 2022/05/04 22:07:00 UTC
[jira] [Updated] (IMPALA-11281) Consider loading the table metadata for a ResetMetadataStmt
[ https://issues.apache.org/jira/browse/IMPALA-11281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fang-Yu Rao updated IMPALA-11281:
---------------------------------
Description:
Currently when a {{ResetMetadataStmt}} is being analyzed, we do not add its '{{{}tableName_{}}}' to the given '{{{}tblRefs{}}}' if its '{{{}partitionSpec_{}}}' is null ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java#L131]).
When the metadata of a table is not fully loaded, we won't populate the column names of a table in its corresponding {{AuthorizableTable}} ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java#L227L229]) since the table is an {{{}IncompleteTable{}}}.
If the column names are not populated in the corresponding {{AuthorizableTable}} of a table in a {{{}ResetMetadataStmt{}}}, then the logic in [RangerAuthorizationChecker#authorizeByTableMasking()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L669-L684] that is supposed to block the metadata update when there are policies defined on the columns could not take effect since in this case [((AuthorizableTable) authorizable).getColumns()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L663] returns an empty list. As a result, such an update would be allowed if there is no other authorization error.
To reproduce the issue, we could comment out all the test cases in [RangerAuditLogTest#testAuditsForColumnMasking()|https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java#L261] except for the following one. The following test case would fail since the query "{{{}invalidate metadata functional.alltypestiny{}}}" won't result in any authorization error. This test case could succeed with its previous test cases enabled because in the previous test cases, there is at least one invocation to {{SelectStmt#collectTableRefs()}} that triggers the metadata loading of the table '{{{}functional.alltypestiny{}}}'.
{code:java}
// Updates on metadata fails by column-masking policies.
authzError(events -> {
assertEquals(1, events.size());
assertEquals("invalidate metadata functional.alltypestiny",
events.get(0).getRequestData());
assertEventEquals("@table", "refresh", "functional/alltypestiny", 0,
events.get(0));
// Make sure it's denied by a column masking policy.
assertTrue(columnMaskingPolicyIds.contains(events.get(0).getPolicyId()));
}, "invalidate metadata functional.alltypestiny", onServer(TPrivilegeLevel.ALL));
{code}
was:
Currently when a {{ResetMetadataStmt}} is being analyzed, we do not add its '{{{}tableName_{}}}' to the given '{{{}tblRefs{}}}' if its '{{{}partitionSpec_{}}}' is null ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java#L131]).
When the metadata of a table is not fully loaded, we won't populate the column names of a table in its corresponding {{AuthorizableTable}} ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java#L227L229]) since the table is an {{{}IncompleteTable{}}}.
If the column names are not populated in the corresponding {{AuthorizableTable}} of a table in a {{{}ResetMetadataStmt{}}}, then the logic in [RangerAuthorizationChecker#authorizeByTableMasking()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L669-L684] that is supposed to block the metadata update when there are policies defined on the columns could not take effect since in this case [((AuthorizableTable) authorizable).getColumns()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L663] returns an empty list. That is, such an update would be allowed if there is no other authorization error.
To reproduce the issue, we could comment out all the test cases in [RangerAuditLogTest#testAuditsForColumnMasking()|https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java#L261] except for the following one. The following test case would fail since the query "{{{}invalidate metadata functional.alltypestiny{}}}" won't result in any authorization error. This test case could succeed with its previous test cases enabled because in the previous test cases, there is at least one invocation to {{SelectStmt#collectTableRefs()}} that triggers the metadata loading of the table '{{{}functional.alltypestiny{}}}'.
{code:java}
// Updates on metadata fails by column-masking policies.
authzError(events -> {
assertEquals(1, events.size());
assertEquals("invalidate metadata functional.alltypestiny",
events.get(0).getRequestData());
assertEventEquals("@table", "refresh", "functional/alltypestiny", 0,
events.get(0));
// Make sure it's denied by a column masking policy.
assertTrue(columnMaskingPolicyIds.contains(events.get(0).getPolicyId()));
}, "invalidate metadata functional.alltypestiny", onServer(TPrivilegeLevel.ALL));
{code}
> Consider loading the table metadata for a ResetMetadataStmt
> -----------------------------------------------------------
>
> Key: IMPALA-11281
> URL: https://issues.apache.org/jira/browse/IMPALA-11281
> Project: IMPALA
> Issue Type: Bug
> Reporter: Fang-Yu Rao
> Assignee: Fang-Yu Rao
> Priority: Major
>
> Currently when a {{ResetMetadataStmt}} is being analyzed, we do not add its '{{{}tableName_{}}}' to the given '{{{}tblRefs{}}}' if its '{{{}partitionSpec_{}}}' is null ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java#L131]).
> When the metadata of a table is not fully loaded, we won't populate the column names of a table in its corresponding {{AuthorizableTable}} ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java#L227L229]) since the table is an {{{}IncompleteTable{}}}.
> If the column names are not populated in the corresponding {{AuthorizableTable}} of a table in a {{{}ResetMetadataStmt{}}}, then the logic in [RangerAuthorizationChecker#authorizeByTableMasking()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L669-L684] that is supposed to block the metadata update when there are policies defined on the columns could not take effect since in this case [((AuthorizableTable) authorizable).getColumns()|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L663] returns an empty list. As a result, such an update would be allowed if there is no other authorization error.
> To reproduce the issue, we could comment out all the test cases in [RangerAuditLogTest#testAuditsForColumnMasking()|https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java#L261] except for the following one. The following test case would fail since the query "{{{}invalidate metadata functional.alltypestiny{}}}" won't result in any authorization error. This test case could succeed with its previous test cases enabled because in the previous test cases, there is at least one invocation to {{SelectStmt#collectTableRefs()}} that triggers the metadata loading of the table '{{{}functional.alltypestiny{}}}'.
> {code:java}
> // Updates on metadata fails by column-masking policies.
> authzError(events -> {
> assertEquals(1, events.size());
> assertEquals("invalidate metadata functional.alltypestiny",
> events.get(0).getRequestData());
> assertEventEquals("@table", "refresh", "functional/alltypestiny", 0,
> events.get(0));
> // Make sure it's denied by a column masking policy.
> assertTrue(columnMaskingPolicyIds.contains(events.get(0).getPolicyId()));
> }, "invalidate metadata functional.alltypestiny", onServer(TPrivilegeLevel.ALL));
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org