You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2021/03/28 12:15:00 UTC
[jira] [Commented] (RANGER-3225) Hive plugin may not block updates
when unmask policy exists
[ https://issues.apache.org/jira/browse/RANGER-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17310183#comment-17310183 ]
Quanlong Huang commented on RANGER-3225:
----------------------------------------
Some logs when enabling DEBUG log of org.apache.ranger.authorization.hive.authorizer:
{code:java}
2021-03-28 12:10:17,122 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-806]: Executing command(queryId=hive_20210328121016_25387a96-f4f3-49e4-a3e0-a7b616969711):
explain authorization insert into table my_tbl values (0, 'foo', 'bar')
2021-03-28 12:10:17,122 INFO org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook: [HiveServer2-Background-Pool: Thread-806]: Received pre-hook notification for: hive_20210328121016_25387a96-f4f3-49e4-a3e0-a7b616969711
2021-03-28 12:10:17,127 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-806]: Starting task [Stage-4:EXPLAIN] in serial mode
2021-03-28 12:10:17,127 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: ==> RangerHiveAuthorizer.initUserRoles()
2021-03-28 12:10:17,127 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: 'checkPrivileges':{'hiveOpType':QUERY, 'inputHObjs':[], 'outputHObjs':['HivePrivilegeObject':{'type':TABLE_OR_VIEW, 'dbName':default, 'objectType':TABLE_OR_VIEW, 'objectName':my_tbl, 'columns':[], 'partKeys':[], 'commandParams':[], 'actionType':INSERT, 'owner':admin}], 'context':{'clientType':HIVESERVER2, 'commandString':, 'ipAddress':172.27.99.193, 'forwardedAddresses':null, 'sessionString':b3496223-9fec-4615-801f-24f8cda04287}, 'user':admin, 'groups':[hueDefaultUsers]}
2021-03-28 12:10:17,127 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: RangerHiveAuthorizer.checkPrivileges: Unexpected operation type[QUERY] received with empty input objects list!
2021-03-28 12:10:17,127 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: RangerHiveAuthorizer.buildRequestContextWithAllAccessedResources() - AllRequestedHiveResources={default/my_tbl; }
2021-03-28 12:10:17,127 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: request: RangerAccessRequestImpl={resource={RangerResourceImpl={ownerUser={admin} elements={database=default; table=my_tbl; } }} accessType={update} user={admin} userGroups={hueDefaultUsers } userRoles={} accessTime={Sun Mar 28 12:10:17 UTC 2021} clientIPAddress={null} forwardedAddresses={} remoteIPAddress={172.27.99.193} clientType={HIVESERVER2} action={QUERY} requestData={} sessionId={b3496223-9fec-4615-801f-24f8cda04287} resourceMatchingScope={SELF} clusterName={null} clusterType={null} context={REQUESTED_RESOURCES={AllRequestedHiveResources={default/my_tbl; } } } }
2021-03-28 12:10:17,128 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: isBlockAccessIfRowfilterColumnMaskSpecified(QUERY, RangerAccessRequestImpl={resource={RangerResourceImpl={ownerUser={admin} elements={database=default; table=my_tbl; } }} accessType={update} user={admin} userGroups={hueDefaultUsers } userRoles={} accessTime={Sun Mar 28 12:10:17 UTC 2021} clientIPAddress={172.27.99.193} forwardedAddresses={} remoteIPAddress={172.27.99.193} clientType={HIVESERVER2} action={QUERY} requestData={} sessionId={b3496223-9fec-4615-801f-24f8cda04287} resourceMatchingScope={SELF} clusterName={Cluster 1} clusterType={} context={REQUESTED_RESOURCES={AllRequestedHiveResources={default/my_tbl; } } token:OWNER={admin} token:USER={admin} } }): true
2021-03-28 12:10:17,128 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: ==> getRowFilterResult(request=RangerAccessRequestImpl={resource={RangerResourceImpl={ownerUser={null} elements={database=default; table=my_tbl; } }} accessType={select} user={admin} userGroups={hueDefaultUsers } userRoles={} accessTime={Sun Mar 28 12:10:17 UTC 2021} clientIPAddress={172.27.99.193} forwardedAddresses={} remoteIPAddress={172.27.99.193} clientType={HIVESERVER2} action={QUERY} requestData={} sessionId={b3496223-9fec-4615-801f-24f8cda04287} resourceMatchingScope={SELF} clusterName={Cluster 1} clusterType={} context={REQUESTED_RESOURCES={AllRequestedHiveResources={default/my_tbl; } } token:OWNER={admin} token:USER={admin} } })
2021-03-28 12:10:17,128 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: <== getRowFilterResult(request=RangerAccessRequestImpl={resource={RangerResourceImpl={ownerUser={null} elements={database=default; table=my_tbl; } }} accessType={select} user={admin} userGroups={hueDefaultUsers } userRoles={} accessTime={Sun Mar 28 12:10:17 UTC 2021} clientIPAddress={172.27.99.193} forwardedAddresses={} remoteIPAddress={172.27.99.193} clientType={HIVESERVER2} action={QUERY} requestData={} sessionId={b3496223-9fec-4615-801f-24f8cda04287} resourceMatchingScope={SELF} clusterName={Cluster 1} clusterType={} context={REQUESTED_RESOURCES={AllRequestedHiveResources={default/my_tbl; } } token:OWNER={admin} token:USER={admin} } }): ret=RangerAccessResult={isAccessDetermined={false} isAllowed={false} isAuditedDetermined={false} isAudited={false} auditLogId={null} policyType={2} policyId={-1} zoneName={null} auditPolicyId={-1} policyVersion={null} evaluatedPoliciesCount={0} reason={null} additionalInfo={}}
2021-03-28 12:10:17,128 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: ==> getDataMaskResult(request=RangerAccessRequestImpl={resource={RangerResourceImpl={ownerUser={null} elements={database=default; table=my_tbl; } }} accessType={select} user={admin} userGroups={hueDefaultUsers } userRoles={} accessTime={Sun Mar 28 12:10:17 UTC 2021} clientIPAddress={172.27.99.193} forwardedAddresses={} remoteIPAddress={172.27.99.193} clientType={HIVESERVER2} action={QUERY} requestData={} sessionId={b3496223-9fec-4615-801f-24f8cda04287} resourceMatchingScope={SELF_OR_DESCENDANTS} clusterName={Cluster 1} clusterType={} context={REQUESTED_RESOURCES={AllRequestedHiveResources={default/my_tbl; } } token:OWNER={admin} token:USER={admin} } })
2021-03-28 12:10:17,128 DEBUG org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer: [HiveServer2-Background-Pool: Thread-806]: <== getDataMaskResult(request=RangerAccessRequestImpl={resource={RangerResourceImpl={ownerUser={null} elements={database=default; table=my_tbl; } }} accessType={select} user={admin} userGroups={hueDefaultUsers } userRoles={} accessTime={Sun Mar 28 12:10:17 UTC 2021} clientIPAddress={172.27.99.193} forwardedAddresses={} remoteIPAddress={172.27.99.193} clientType={HIVESERVER2} action={QUERY} requestData={} sessionId={b3496223-9fec-4615-801f-24f8cda04287} resourceMatchingScope={SELF_OR_DESCENDANTS} clusterName={Cluster 1} clusterType={} context={REQUESTED_RESOURCES={AllRequestedHiveResources={default/my_tbl; } } token:OWNER={admin} token:USER={admin} } }): ret=RangerAccessResult={isAccessDetermined={true} isAllowed={true} isAuditedDetermined={true} isAudited={true} auditLogId={null} policyType={1} policyId={65} zoneName={null} auditPolicyId={65} policyVersion={7} evaluatedPoliciesCount={1} reason={null} additionalInfo={maskType=MASK_NONE, maskedValue=, maskCondition=null, }}
2021-03-28 12:10:17,129 INFO org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook: [HiveServer2-Background-Pool: Thread-806]: Received post-hook notification for: hive_20210328121016_25387a96-f4f3-49e4-a3e0-a7b616969711
2021-03-28 12:10:17,180 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-806]: Completed executing command(queryId=hive_20210328121016_25387a96-f4f3-49e4-a3e0-a7b616969711); Time taken: 0.059 seconds
2021-03-28 12:10:17,180 INFO org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-806]: OK {code}
> Hive plugin may not block updates when unmask policy exists
> -----------------------------------------------------------
>
> Key: RANGER-3225
> URL: https://issues.apache.org/jira/browse/RANGER-3225
> Project: Ranger
> Issue Type: Bug
> Components: plugins
> Affects Versions: 0.6.3, 1.0.0, 0.7.1, 1.1.0, 1.2.0, 2.1.0
> Reporter: Quanlong Huang
> Priority: Major
>
> Per RANGER-1087 and RANGER-1100, table modifications(insert/delete/update) should be blocked when row-filter/column-masking policy is enabled for the user. However, when there are no row-filtering policies on the table, and there are both mask and unmask policies on the columns, updates may not be blocked.
> The cause is we just check one column masking policy of the table, regardless whether it's an unmask (MASK_TYPE_NONE) policy:
> {code:java}
> // check if masking is enabled for any column in the table/view
> request.setResourceMatchingScope(RangerAccessRequest.ResourceMatchingScope.SELF_OR_DESCENDANTS);
> RangerAccessResult dataMaskResult = getDataMaskResult(request);
> if (isDataMaskEnabled(dataMaskResult)) {
> // block the update
> }{code}
> [https://github.com/apache/ranger/blob/58b51a39ebe2e7dc4d253658e423f0afb6a74987/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L978-L982]
> When the picked policy is an unmasked policy, isDataMaskEnabled() returns false on it.
> {code:java}
> private boolean isDataMaskEnabled(RangerAccessResult result) {
> return result != null && result.isMaskEnabled();
> }{code}
> Codes for RangerAccessResult#isMaskEnabled():
> {code:java}
> public boolean isMaskEnabled() {
> return StringUtils.isNotEmpty(this.getMaskType()) && !StringUtils.equalsIgnoreCase(this.getMaskType(), RangerPolicy.MASK_TYPE_NONE);
> }
> {code}
> It's undeterminded which column masking policy will be matched. When re-creating some policies, or disabling and then re-enabling some policies, the result changes. In theory, we should check all column masking policies of the table until we find a real mask policy.
> *How to reproduce*
> Create a table with 3 columns (id int, name string, addr string). Add a redact policy on "name". Add an unmask policy on "id". Check whether updates will be blocked:
> {code:sql}
> explain authorization insert into table my_tbl values (0, 'foo', 'bar'); {code}
> The result could be OK, or
> {code:java}
> Permission denied: user [admin] does not have [UPDATE] privilege on [default/my_tbl]{code}
> cc [~madhan], [~jcamachorodriguez]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)