[jira] [Commented] (SENTRY-2140) Attribute based access control

["Anthony Young-Garner (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463937#comment-16463937 ] 

Anthony Young-Garner commented on SENTRY-2140:
----------------------------------------------

In reviewing the comments on the [scope doc|https://docs.google.com/document/d/1gCtL7-lpaewbiMM6G_mxbkTvCCMEK3sHI5dV_vWMrLA/edit#heading=h.u0inu6d1lh16] for SENTRY-2140, we realized it was important to explain and clarify some of the thinking either described in or behind the prose in the document. To this end, Alexander Kolbasov, Sergio Peña and I had a meeting earlier this week to discuss some of the open questions:
 
1) Why should attributes reside in a source external to Sentry?
 
Users want to assign attributes to arbitrary data and want to be able to apply policies based on these. The intended users already have full-featured systems that apply user-specified metadata to structured and unstructured data in Hive, Impala, Spark, etc. These systems provide user interfaces, scripting capabilities and metadata lifecycle management that far exceed the capabilities that might reasonably be added to Sentry so my expectation is that users will leverage these existing systems in applying attributes to data.
 
2) Why wouldn't attribute based privileges be granted in beeline rather than through the Sentry CLI?
 
In the intended users groups, there are different levels of admins - data admins and org admins. They want to administer things differently and have different skill sets. Data admins who grant role based access to data are not the same admins who assign policies to attributes and apply attributes to data. However, it may be both possible and useful in the scope of the project to allow data admins to at least view attribute based policies through beeline.
 
3) This isn't full ABAC? ABAC applies only to columns and RBAC is still being used for everything else?
 
Yes, the ABAC solution is overlaid on the existing RBAC solution. The column value mask is applied independently of the RBAC privilege but the RBAC privilege is required before the ABAC privilege is considered. You must have the RBAC privilege to see a column, the ABAC privilege only affects how you can see the column that the RBAC privilege gives you permission to see.
 
4) Couldn't we implement masking with RBAC rather than ABAC?
 
Yes, technically we can implement masking with RBAC. However, the main value users see in using attributes is that it solves an administrative complexity / management scalability problem. Having to assign, maintain and update masks individually across thousands or tens of thousands of columns is administratively intractable.
 
5) The file-based attribute provider doesn't seem useful in production. What is its purpose?
 
The file-based attribute provider is a reference implementation, similar to the Hadoop Java KeyStore Key Provider in Hadoop. It demonstrates how to implement a provider using any external attribute source.
 
6) Why is a cache needed?
 
I wanted to split the question about a) the architectural need for a cache from b) the question about how that cache is implemented. 
 
6a. So, first, the architectural need for a cache is driven by the fact that attributes reside in an external system (as discussed in item 1). Since the system of record for attributes is external to Sentry, attribute data must be either retrieved (pulled) or sent (pushed) from the external system to Sentry. Rather than retrieving attribute data each time it is needed from the external system, Sentry will retain some copy of the attribute data locally (to reduce load on the external system and also to speed up access to attribute data). A local copy of remote data kept for purposes of faster data access is a cache by definition, however it is implemented.
 
6b. Given that we will definitely have a de facto cache of some type (however implemented) in the design (as discussed in 6a), we investigated using the existing Sentry datastore as a cache (see SENTRY-2196). We've implemented this in prototype sufficiently to understand that this will work as long as the schema in which attributes are stored has no joins with the rest of the Sentry datastore schema. However, in discussion with Impala committers, we understood that the Impala team was hoping to avoid increasing the number of direct calls that require copying data from the Sentry database into Impala (this significantly impacts Impala performance and requires manually refreshing privilege and role information in Impala). We hypothesized that a dedicated, distributed cache might improve performance without the need for manual refreshes. In initial tests using Infinispan to share Sentry roles and privileges with Impala, we found that performance in time increased by at least an order of magnitude and scalability (in number of roles and privileges that Impala could support) increased by multiple orders of magnitude. Therefore, we thought we might introduce the use of Infinispan in Sentry with the ABAC project but over time (in subsequent projects) expand its use to RBAC also. As a last detail, we expect that we still want to persist the cached data to the Sentry datastore to ensure that attribute data is highly available even if the external attribute source is not highly available.
 
Where there is ambiguity or lack of detail in the scope doc on these questions, Steve or I will update the scope doc. 

> Attribute based access control
> ------------------------------
>
>                 Key: SENTRY-2140
>                 URL: https://issues.apache.org/jira/browse/SENTRY-2140
>             Project: Sentry
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Steve Moist
>            Priority: Major
>         Attachments: Sentry ABAC Proposal v1.1.pdf, Sentry ABAC Proposal.pdf
>
>
> As a user, I want to have finer grain control over which users/roles can view data in Hive.  Some information such as Social Security Number is considered very confidential information.  I want to be able to tag columns in Hive with "attributes" that prevent users/roles from not accessing or seeing the data.  For users/roles that have that attribute, they should be able to see that information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)