[DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Madhan Neethiraj <mn...@hortonworks.com>]

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

>My question was: do real customers ask for exception on Deny list?
It is mostly the real-world scenarios. E.g. 
Compliance policy dictates customer info can’t be used after 1 year. So the policy will deny access after “EXPIRY DATE” to all, but you still want the ETL user (e.g. Falcon) to be allowed to delete after the expiry date. So you will need to exclude Falcon from this policy. There could be another policy to allow Falcon user to allow delete only for this resource.
Allow everyone from “Finance Group” to the entire database, except for users from “Intern” group. Intern group or users might have permissions for certain tables or columns and this is managed my different policies.
So far we had only “Allow”, which works in majority of the use cases. But the moment we will add “Deny” we will introduce conflicting policies.  Exception gives the admin to ability to break the dead-lock.

Bosco


From:  Dilli Dorai
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Friday, October 16, 2015 at 1:42 PM
To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Bosco,
Explicit Deny is fine and I would expect some customers would ask for it.
My question was: do real customers ask for exception on Deny list?
We may have a case of over engineering here.
Thanks
Dilli

On Fri, Oct 16, 2015 at 1:40 PM, Don Bosco Durai <bo...@apache.org> wrote:
Dilli, there are few recent use cases that come up, which requires explicit deny, otherwise the policies might become too complex. Most obvious ones are supporting tag based policy where you want to restrict access at a global level. Other use cases are integrating with external fraud monitoring tools, which want to deny users/groups within Hadoop using API calls. I feel, this will be great addition to Ranger feature set.

Thanks

Bosco


From:  Dilli Dorai
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 11:26 AM

To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Do we have a realistic customer asking for exception on Deny set?


On Thu, Oct 15, 2015 at 10:32 AM, Balaji Ganesan <ba...@gmail.com> wrote:
I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality. 

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple 
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups. 


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 10:24 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 9:01 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Dilli Dorai <di...@gmail.com>]

Bosco,
Explicit Deny is fine and I would expect some customers would ask for it.
My question was: do real customers ask for exception on Deny list?
We may have a case of over engineering here.
Thanks
Dilli

On Fri, Oct 16, 2015 at 1:40 PM, Don Bosco Durai <bo...@apache.org> wrote:

> Dilli, there are few recent use cases that come up, which requires
> explicit deny, otherwise the policies might become too complex. Most
> obvious ones are supporting tag based policy where you want to restrict
> access at a global level. Other use cases are integrating with external
> fraud monitoring tools, which want to deny users/groups within Hadoop using
> API calls. I feel, this will be great addition to Ranger feature set.
>
> Thanks
>
> Bosco
>
>
> From: Dilli Dorai
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Thursday, October 15, 2015 at 11:26 AM
>
> To: <us...@ranger.incubator.apache.org>
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> Do we have a realistic customer asking for exception on Deny set?
>
>
> On Thu, Oct 15, 2015 at 10:32 AM, Balaji Ganesan <
> balaji.ganesan03@gmail.com> wrote:
>
>> I think we completely missed my earlier point. My question was around
>> whether we need to have exceptions in Deny and Allow beyond just Allow or
>> Deny policy items. I used the MSFT example to see if they have implemented
>> exceptions to Deny, which I did not find any. I am not suggesting that we
>> ask the users to order the policies and Ranger follow the order.
>>
>> We can open a separate thread on whether Ranger needs to use an explicit
>> hierarchy for policy evaluation or not. For the current discussion, I don't
>> see a specific use case where we would need a deny or allow exception and
>> not fulfilled by just allow or deny functionality.
>>
>> Let us take the use case we discussed earlier. The group "interns" got
>> denied,but we want to exclude one user ("Scott") from that, I am suggesting
>> to have the particular user ("Scott") included in the Allow section of
>> policy and group "interns" included in Deny section of the same policy.
>> Ranger should evaluate the both Allow and Deny in the same policy and
>> deduce that user "Scott" can access the resource while anyone else in group
>> "interns" is denied to access the resource. The policy creation in the UI
>> is much simpler than having to specify "Scott" as an exception to Deny and
>> then specifying "Scott" for Allow.
>>
>> On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
>> mneethiraj@hortonworks.com> wrote:
>>
>>> Microsoft DACL also talks about exceptions, but the model relies on the
>>> users to setup ACE in specific order – see below.
>>>
>>> From
>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
>>> :
>>>
>>> "In most cases, you can control access to an object by using
>>> access-allowed ACEs; you do not need to explicitly deny access to an
>>> object. The exception is when an ACE allows access to a group and you want
>>> to deny access to a member of the group. To do this, place an access-denied
>>> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
>>> Note that the order of the ACEs
>>> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
>>> important because the system reads the ACEs in sequence until access is
>>> granted or denied. The user's access-denied ACE must appear first;
>>> otherwise, when the system reads the group's access allowed ACE, it will
>>> grant access to the restricted user."
>>>
>>> This approach of relying on ordering of policy items (equivalent of ACEs
>>> above) can work for Ranger as well. However Microsoft DACL model evaluates
>>> only one DACL to determine access to an object; compare this with Ranger
>>> where there could be multiple policies that can be evaluated for a resource
>>> – for example to determine access to a finance.invoice.ssn column, Ranger
>>> would evaluate policies that are applicable for finance database,
>>> finance.invoice table and finance.invoice.ssn column. To guarantee
>>> consistent authorization result, that relies on proper ordering of
>>> allow/deny policy items in each policy, would require the policies to be
>>> evaluated in a specific order – either most-specific-to-most-generic
>>> (column policy, table policy, database policy)  or most-generic-to
>>> most-specific. And it might be very challenging, if not impossible, to
>>> ensure a policy order given our support for wildcard specification – for
>>> example, should a policy for “/finance/*2015/*” be evaluated before a
>>> policy for “/finance/invoice*/*” while determining access for a file named
>>> “/finance/invoice2015/vendor1.txt”?
>>>
>>> I think this approach would make the policy authoring very difficult and
>>> confusing, in addition to being not able to support certain scenarios like
>>> “deny at a higher level (like for a database), even if access might be
>>> allowed at a lower level (for a table/column)”, “allow at a higher level,
>>> but deny at a lower level). The current implementation (in tag-policy
>>> branch of Ranger) is much less confusing and offers building blocks that
>>> can be used to support more usecases and can guarantee consistent results
>>> for a given set of policies. If we are after simplicity, the DACL model
>>> does not seem to be the right choice..
>>>
>>> Thanks,
>>> Madhan
>>>
>>>
>>> From: Balaji Ganesan
>>> Reply-To: "user@ranger.incubator.apache.org"
>>> Date: Wednesday, October 14, 2015 at 10:31 AM
>>>
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> >> My proposal is to just keep the policy to have only allow and deny
>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>> "intern" group, then anyone in intern group would not be allowed while
>>> everyone else in finance group will get access. If there is a person in
>>> intern group who needs access, then user need to be taken out of the intern
>>> group or we need to add only the users who specifically need to be denied.
>>>
>>> <<Users who are not comfortable with the idea of “excludes” can continue
>>> to use only allow and deny in the policies. Users who are comfortable with
>>> “excludes” can choose to use it to simplify their policy management.>>
>>>
>>> The question here is whether exceptions are really needed in the
>>> product. We need to keep the product simple
>>> Questions is what use cases can exceptions clearly solve above and
>>> beyond simply access grant and access deny policy line items. As an
>>> example, I am looking at MSFT for how they have have implemented
>>> deny-access
>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
>>> The use case MSFT is trying to solve it denying access to specific users
>>> who are part of the group which is provided access. I think most of the
>>> deny-access use cases would be for specific users, not for large sets of
>>> groups.
>>>
>>>
>>> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
>>> mneethiraj@hortonworks.com> wrote:
>>>
>>>> >> For argument sakes, finance could be the higher level group and
>>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>>> add all groups which needs access or specify deny groups which don't need
>>>> access
>>>>
>>>> The groups may not fall into nice hierarchies – for example, interns
>>>> group might consist of users from various orgs in a company (not just from
>>>> finance group). In such cases, the only choice is to setup Ranger policies
>>>> with individual users (and not groups).
>>>>
>>>> >> My proposal is to just keep the policy to have only allow and deny
>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>> "intern" group, then anyone in intern group would not be allowed while
>>>> everyone else in finance group will get access. If there is a person in
>>>> intern group who needs access, then user need to be taken out of the intern
>>>> group or we need to add only the users who specifically need to be denied.
>>>>
>>>> Users who are not comfortable with the idea of “excludes” can continue
>>>> to use only allow and deny in the policies. Users who are comfortable with
>>>> “excludes” can choose to use it to simplify their policy management.
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>> From: Balaji Ganesan
>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>>>
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> <<Are you suggesting that the security admins create policies that
>>>> list individual users, instead of using groups? Wouldn’t that make security
>>>> administration more painful, in a reasonably large organization? For every
>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>> users and groups to alleviate this issue.>>
>>>> For argument sakes, finance could be the higher level group and intern
>>>> could be a sub group in a hierarchy. There could be n number of sub groups.
>>>> We could provide access by adding n-1 groups to the policy. We can argue
>>>> whether n<10 or n>50,  depending on the answer it would make sense to add
>>>> all groups which needs access or specify deny groups which don't need access
>>>>
>>>> My proposal is to just keep the policy to have only allow and deny with
>>>> NO exceptions. In your use case, if we allow "finance" group and deny
>>>> "intern" group, then anyone in intern group would not be allowed while
>>>> everyone else in finance group will get access. If there is a person in
>>>> intern group who needs access, then user need to be taken out of the intern
>>>> group or we need to add only the users who specifically need to be denied.
>>>>
>>>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Balaji,
>>>>>
>>>>> >> why cannot user clearly specific users or groups that would need
>>>>> specific access. Why is there a need to give access to finance group as a
>>>>> whole when we know there is a subset of user who do not need access to the
>>>>> the finance database? or clearly specify users who need to be denied?
>>>>> Are you suggesting that the security admins create policies that list
>>>>> individual users, instead of using groups? Wouldn’t that make security
>>>>> administration more painful, in a reasonably large organization? For every
>>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>>> users and groups to alleviate this issue.
>>>>>
>>>>>
>>>>> From: Balaji Ganesan
>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>>>
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>>>
>>>>> We clearly need to show case use cases for deny exclude and allow
>>>>> exclude. In my opinion it is very confusing to user to construct such a
>>>>> policy
>>>>>
>>>>> <<Let’s say one of the users, scott who is in interns and finance
>>>>> groups, works on an assignment that requires select access to finance
>>>>> database. To enable this access, the authorization policy for the database
>>>>> should be updated by adding a deny-exclude, as shown below:>>
>>>>> In the wiki, we have created a "deny" policy for intern group and an
>>>>> exception for Scott. First of all, why cannot user clearly specific users
>>>>> or groups that would need specific access. Why is there a need to give
>>>>> access to finance group as a whole when we know there is a subset of user
>>>>> who do not need access to the the finance database? or clearly specify
>>>>> users who need to be denied? It is confusing to think about exceptions for
>>>>> deny, deny, exceptions for allow and allow.
>>>>>
>>>>>
>>>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Bosco,
>>>>>>
>>>>>> Thanks for the review and comments. The wiki
>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>>>> has been updated to address your comments. Please review.
>>>>>>
>>>>>> Thanks,
>>>>>> Madhan
>>>>>>
>>>>>> From: Don Bosco Durai
>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>> deny-conditions and exceptions
>>>>>>
>>>>>> Madhan, thanks for putting this document together. It is looking good.
>>>>>>
>>>>>> Can I make a few suggestions:
>>>>>>
>>>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for
>>>>>>    "HDFS policy that allows all finance group users to access contents of
>>>>>>    /finance folder, but denies access to users in interns group. Users in
>>>>>>    interns group will be denied the access even if they are part of finance
>>>>>>    group.”
>>>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny
>>>>>>    all users from “interns” group from accessing table “Employees"
>>>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>>>    good. I think, this is sort of a new concept.
>>>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart.
>>>>>>    Can we create flow chart diagram. It will be easy to understand
>>>>>>
>>>>>> Thanks again. Let me know if you need help in the documentation.
>>>>>>
>>>>>> Bosco
>>>>>>
>>>>>>
>>>>>> From: Madhan Neethiraj
>>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>> Subject: [DISCUSS] Policy model enhancement to support
>>>>>> deny-conditions and exceptions
>>>>>>
>>>>>> All,
>>>>>>
>>>>>> Apache Ranger policy model enhancement to support deny-conditions and
>>>>>> exceptions (RANGER-606
>>>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy
>>>>>> branch <https://github.com/apache/incubator-ranger/tree/tag-policy>.
>>>>>> This enhancement adds the capability to explicitly deny access to resources
>>>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>>>
>>>>>> An overview of the implementation, along with few examples is
>>>>>> available in Apache wiki page here
>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>>>> Please review.
>>>>>>
>>>>>> Thanks,
>>>>>> Madhan
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

Dilli, there are few recent use cases that come up, which requires explicit deny, otherwise the policies might become too complex. Most obvious ones are supporting tag based policy where you want to restrict access at a global level. Other use cases are integrating with external fraud monitoring tools, which want to deny users/groups within Hadoop using API calls. I feel, this will be great addition to Ranger feature set.

Thanks

Bosco


From:  Dilli Dorai
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 11:26 AM
To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Do we have a realistic customer asking for exception on Deny set?


On Thu, Oct 15, 2015 at 10:32 AM, Balaji Ganesan <ba...@gmail.com> wrote:
I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality. 

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple 
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups. 


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 10:24 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 9:01 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Dilli Dorai <di...@gmail.com>]

Do we have a realistic customer asking for exception on Deny set?


On Thu, Oct 15, 2015 at 10:32 AM, Balaji Ganesan <balaji.ganesan03@gmail.com
> wrote:

> I think we completely missed my earlier point. My question was around
> whether we need to have exceptions in Deny and Allow beyond just Allow or
> Deny policy items. I used the MSFT example to see if they have implemented
> exceptions to Deny, which I did not find any. I am not suggesting that we
> ask the users to order the policies and Ranger follow the order.
>
> We can open a separate thread on whether Ranger needs to use an explicit
> hierarchy for policy evaluation or not. For the current discussion, I don't
> see a specific use case where we would need a deny or allow exception and
> not fulfilled by just allow or deny functionality.
>
> Let us take the use case we discussed earlier. The group "interns" got
> denied,but we want to exclude one user ("Scott") from that, I am suggesting
> to have the particular user ("Scott") included in the Allow section of
> policy and group "interns" included in Deny section of the same policy.
> Ranger should evaluate the both Allow and Deny in the same policy and
> deduce that user "Scott" can access the resource while anyone else in group
> "interns" is denied to access the resource. The policy creation in the UI
> is much simpler than having to specify "Scott" as an exception to Deny and
> then specifying "Scott" for Allow.
>
> On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
> mneethiraj@hortonworks.com> wrote:
>
>> Microsoft DACL also talks about exceptions, but the model relies on the
>> users to setup ACE in specific order – see below.
>>
>> From
>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
>> :
>>
>> "In most cases, you can control access to an object by using
>> access-allowed ACEs; you do not need to explicitly deny access to an
>> object. The exception is when an ACE allows access to a group and you want
>> to deny access to a member of the group. To do this, place an access-denied
>> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
>> Note that the order of the ACEs
>> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
>> important because the system reads the ACEs in sequence until access is
>> granted or denied. The user's access-denied ACE must appear first;
>> otherwise, when the system reads the group's access allowed ACE, it will
>> grant access to the restricted user."
>>
>> This approach of relying on ordering of policy items (equivalent of ACEs
>> above) can work for Ranger as well. However Microsoft DACL model evaluates
>> only one DACL to determine access to an object; compare this with Ranger
>> where there could be multiple policies that can be evaluated for a resource
>> – for example to determine access to a finance.invoice.ssn column, Ranger
>> would evaluate policies that are applicable for finance database,
>> finance.invoice table and finance.invoice.ssn column. To guarantee
>> consistent authorization result, that relies on proper ordering of
>> allow/deny policy items in each policy, would require the policies to be
>> evaluated in a specific order – either most-specific-to-most-generic
>> (column policy, table policy, database policy)  or most-generic-to
>> most-specific. And it might be very challenging, if not impossible, to
>> ensure a policy order given our support for wildcard specification – for
>> example, should a policy for “/finance/*2015/*” be evaluated before a
>> policy for “/finance/invoice*/*” while determining access for a file named
>> “/finance/invoice2015/vendor1.txt”?
>>
>> I think this approach would make the policy authoring very difficult and
>> confusing, in addition to being not able to support certain scenarios like
>> “deny at a higher level (like for a database), even if access might be
>> allowed at a lower level (for a table/column)”, “allow at a higher level,
>> but deny at a lower level). The current implementation (in tag-policy
>> branch of Ranger) is much less confusing and offers building blocks that
>> can be used to support more usecases and can guarantee consistent results
>> for a given set of policies. If we are after simplicity, the DACL model
>> does not seem to be the right choice..
>>
>> Thanks,
>> Madhan
>>
>>
>> From: Balaji Ganesan
>> Reply-To: "user@ranger.incubator.apache.org"
>> Date: Wednesday, October 14, 2015 at 10:31 AM
>>
>> To: "user@ranger.incubator.apache.org"
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> >> My proposal is to just keep the policy to have only allow and deny
>> with NO exceptions. In your use case, if we allow "finance" group and deny
>> "intern" group, then anyone in intern group would not be allowed while
>> everyone else in finance group will get access. If there is a person in
>> intern group who needs access, then user need to be taken out of the intern
>> group or we need to add only the users who specifically need to be denied.
>>
>> <<Users who are not comfortable with the idea of “excludes” can continue
>> to use only allow and deny in the policies. Users who are comfortable with
>> “excludes” can choose to use it to simplify their policy management.>>
>>
>> The question here is whether exceptions are really needed in the product.
>> We need to keep the product simple
>> Questions is what use cases can exceptions clearly solve above and beyond
>> simply access grant and access deny policy line items. As an example, I am
>> looking at MSFT for how they have have implemented deny-access
>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
>> The use case MSFT is trying to solve it denying access to specific users
>> who are part of the group which is provided access. I think most of the
>> deny-access use cases would be for specific users, not for large sets of
>> groups.
>>
>>
>> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
>> mneethiraj@hortonworks.com> wrote:
>>
>>> >> For argument sakes, finance could be the higher level group and
>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>> add all groups which needs access or specify deny groups which don't need
>>> access
>>>
>>> The groups may not fall into nice hierarchies – for example, interns
>>> group might consist of users from various orgs in a company (not just from
>>> finance group). In such cases, the only choice is to setup Ranger policies
>>> with individual users (and not groups).
>>>
>>> >> My proposal is to just keep the policy to have only allow and deny
>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>> "intern" group, then anyone in intern group would not be allowed while
>>> everyone else in finance group will get access. If there is a person in
>>> intern group who needs access, then user need to be taken out of the intern
>>> group or we need to add only the users who specifically need to be denied.
>>>
>>> Users who are not comfortable with the idea of “excludes” can continue
>>> to use only allow and deny in the policies. Users who are comfortable with
>>> “excludes” can choose to use it to simplify their policy management.
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Balaji Ganesan
>>> Reply-To: "user@ranger.incubator.apache.org"
>>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>>
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> <<Are you suggesting that the security admins create policies that list
>>> individual users, instead of using groups? Wouldn’t that make security
>>> administration more painful, in a reasonably large organization? For every
>>> change in users role (or group) or an employee joining/leaving the org, all
>>> security policies have to be reviewed and updated..  Many ACLs support both
>>> users and groups to alleviate this issue.>>
>>> For argument sakes, finance could be the higher level group and intern
>>> could be a sub group in a hierarchy. There could be n number of sub groups.
>>> We could provide access by adding n-1 groups to the policy. We can argue
>>> whether n<10 or n>50,  depending on the answer it would make sense to add
>>> all groups which needs access or specify deny groups which don't need access
>>>
>>> My proposal is to just keep the policy to have only allow and deny with
>>> NO exceptions. In your use case, if we allow "finance" group and deny
>>> "intern" group, then anyone in intern group would not be allowed while
>>> everyone else in finance group will get access. If there is a person in
>>> intern group who needs access, then user need to be taken out of the intern
>>> group or we need to add only the users who specifically need to be denied.
>>>
>>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Balaji,
>>>>
>>>> >> why cannot user clearly specific users or groups that would need
>>>> specific access. Why is there a need to give access to finance group as a
>>>> whole when we know there is a subset of user who do not need access to the
>>>> the finance database? or clearly specify users who need to be denied?
>>>> Are you suggesting that the security admins create policies that list
>>>> individual users, instead of using groups? Wouldn’t that make security
>>>> administration more painful, in a reasonably large organization? For every
>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>> users and groups to alleviate this issue.
>>>>
>>>>
>>>> From: Balaji Ganesan
>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>>
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>>
>>>> We clearly need to show case use cases for deny exclude and allow
>>>> exclude. In my opinion it is very confusing to user to construct such a
>>>> policy
>>>>
>>>> <<Let’s say one of the users, scott who is in interns and finance
>>>> groups, works on an assignment that requires select access to finance
>>>> database. To enable this access, the authorization policy for the database
>>>> should be updated by adding a deny-exclude, as shown below:>>
>>>> In the wiki, we have created a "deny" policy for intern group and an
>>>> exception for Scott. First of all, why cannot user clearly specific users
>>>> or groups that would need specific access. Why is there a need to give
>>>> access to finance group as a whole when we know there is a subset of user
>>>> who do not need access to the the finance database? or clearly specify
>>>> users who need to be denied? It is confusing to think about exceptions for
>>>> deny, deny, exceptions for allow and allow.
>>>>
>>>>
>>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Bosco,
>>>>>
>>>>> Thanks for the review and comments. The wiki
>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>>> has been updated to address your comments. Please review.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>> From: Don Bosco Durai
>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> Madhan, thanks for putting this document together. It is looking good.
>>>>>
>>>>> Can I make a few suggestions:
>>>>>
>>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for
>>>>>    "HDFS policy that allows all finance group users to access contents of
>>>>>    /finance folder, but denies access to users in interns group. Users in
>>>>>    interns group will be denied the access even if they are part of finance
>>>>>    group.”
>>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny
>>>>>    all users from “interns” group from accessing table “Employees"
>>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>>    good. I think, this is sort of a new concept.
>>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can
>>>>>    we create flow chart diagram. It will be easy to understand
>>>>>
>>>>> Thanks again. Let me know if you need help in the documentation.
>>>>>
>>>>> Bosco
>>>>>
>>>>>
>>>>> From: Madhan Neethiraj
>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> All,
>>>>>
>>>>> Apache Ranger policy model enhancement to support deny-conditions and
>>>>> exceptions (RANGER-606
>>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy
>>>>> branch <https://github.com/apache/incubator-ranger/tree/tag-policy>.
>>>>> This enhancement adds the capability to explicitly deny access to resources
>>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>>
>>>>> An overview of the implementation, along with few examples is
>>>>> available in Apache wiki page here
>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>>> Please review.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Alok Lal <al...@hortonworks.com>]

My description confuses between policy and policy item.  My bad!  Let’s try this.

Policy:
Resource: <Finance specific resource>
Allow Policy Item:
Access <read/write/create/etc.>
Group: Finance
User: Scott

Above policy allows access to everyone in Finance or user Scott.  Since Interns are not mentioned in Allow they are not going to get access.

Thanks



From: Balaji Ganesan <ba...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Wednesday, October 21, 2015 at 3:28 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Unless I am missing something, this use case is trivial.  Won’t the following two policies do?

  *   Policy1
     *   Allow: Finance
  *   Policy 2:
     *   Allow: Scott>>

I did not get your point here. Do we need a deny policy (and an exception) here or not? Can you elaborate?



On Wed, Oct 21, 2015 at 3:19 PM, Alok Lal <al...@hortonworks.com>> wrote:
@Balaji,
Since scott is a Finance Intern he is part of Finance.  So he does NOT need to specified on Allow.

> Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.

But let us modify the use case as you have suggested, i.e. as follows:

Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is an intern who is doing a project for CFO and hence needs access to the budget data.

Unless I am missing something, this use case is trivial.  Won’t the following two policies do?

  *   Policy1
     *   Allow: Finance
  *   Policy 2:
     *   Allow: Scott

Thanks

From: Balaji Ganesan <ba...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Wednesday, October 21, 2015 at 3:12 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Thanks Alok for taking time to put this down. For the record, I zone out after couple of paragraphs in an email, so please limit it next time you send one :-)

Let us take you use case

<<Our use case is as follows:
Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.
Madhan’s model proposes to address this with the following policy:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception – Scott>>

Actually, for this use case, the policy in the current implementation. would need  to look like this

  *   Allow – Finance, Scott
  *   Deny – Intern
     *   Deny exception – Scott

This is part which is confusing. Scott not only needs to be mentioned in the deny exception, but also need to be added to access policy for Scott to be given access. If we read it carefully it makes sense, but it is not intuitive enough for any layman user creating the policy. What I was suggesting was to make the policy definition simpler, the policy should like this

  *   Allow – Finance, Scott
  *   Deny – Interns

The policy evaluation, as you had pointed out, changes in this case. We are indeed making exceptions for users who are explicitly given access or denied, very similar to the suggestion from Abhay. I agree on some of the disadvantages with this simplistic mode (not sure if I would call this imperative vs declarative), but we can get feedback from community and users as well evolve.

On Fri, Oct 16, 2015 at 8:21 PM, Alok Lal <al...@hortonworks.com>> wrote:
@Balaji,
I know how much you love long emails.  Here goes! :)

You wrote:

> Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

Here’s why what you say can’t be done without also imposing a “hidden/implicit rules”.

Our use case is as follows:
Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.

Madhan’s model proposes to address this with the following policy:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception – Scott

Which would, as the use case requires, allow access to everyone in Finance unless they are an intern, except to Scott.  Please note that policy’s description alone is sufficient for anyone to know what it would do.

Now let us consider the policy that you have suggested:

  *   Allow – Finance, Scott
  *   Deny – Interns

For the above to honor our use case we would also need to apply the following additional rule:

  *   Allow access if user has been explicitly been given access even if he has been denied.  (This BTW was what Abhay’s had suggested.)

Unless we have the above rule we can’t disambiguate what to do about Scott who is part of both Finance and Interns — one of which has been given access and another has been denied.  One must have a way to break the tie in the form of a rule.  @Balaji which do you feel is better?

  *   Your combination of (simpler policy + simple rule) or
  *   Madhan’s complicated policy without any additional rules

Perhaps the decision is not obvious at this stage.  Now, consider the following use case which is only a slight modification to above.

New use case:
Finance department is very large.  Some Interns are in Finance.  Everyone from Finance has access to Budget forecast except its interns.  CEO has asked every department to allocate people to his favorite new project - Compliance.  Finance has asked some of its interns to be part of Compliance project and wants to give then access to Budget forecast.

Madhan’s Model can address this as follows:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception: Compliance

Which again is straight forward to interpret and verify.

You model would have to be as follows:

  *   Allow – Finance, Compliance
  *   Deny – Intern

How do we interpret this policy given that some interns are in Finance, some of those working on Compliance project are in Finance?  We would need an additional rule as follows:

  *   If a user is explicitly allowed access because of a greater number groups (Finance and Compliance) than he is denied access (Intern) then allow wins and he should be allowed access.

So now we have two rules.

The point is this: Domain we want to model is complex.  And we have two choices.

  *   Keep complexity in the structure of your model itself.  This makes it easy to reason about it without the need of an additional codec (set of rules).  In computer science this is often referred to as a Declarative specification, i.e. you expend your energies only in stating your intent.  You are NOT concerned about how something works.  This is Madhan’s style — naturally as programmers have a bias towards getting to declarative specification.
  *   Keep the model simple, move complexity to a set of rules that must be consulted in a known order to get at the result.  This is often called as an Imperative specification.  To understand and reason about something it is not enough to just define your data but you also have to know the rules and know how they are applied.  This has advantage of course.  The rules can be easily re-ordered to yield a system that does something entirely different.  Disadvantages are obvious: as the number of rules grow system becomes hard to reason about and hence brittle.

I can tell you from years of computer science research the move away from an imperative style is towards declarative style.  We should think long and hard before we rule in favor of an imperative specification in this day and age.

Thanks
- Alok

From: Don Bosco Durai <bo...@apache.org>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, October 16, 2015 at 2:10 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>

Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

> I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger?

I am okay with this. But we have to document and advertise this so that users know that this feature exists and they can enable it.

Bosco



From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Friday, October 16, 2015 at 1:47 PM
To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.>>
We all know from experience that expecting Admin to be smart enough does not always hold true. The premise of doing a UI, and not a command line tool, is the admins do not have expertise or training to construct access policies for Hadoop easily. That is big differentiator for Ranger, which is the ease of use and centralized policy management.

<<Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
If there are only few customers who would need this, the deny or allow exception functionality need not be exposed for the majority. If there is option available, users will use it even if they could have met their use case using simpler methods.

<<I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.>>
I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger?

On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org>> wrote:
>That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.

>Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.
Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.

>I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.
I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.

Thanks

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:54 PM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:

  1.  When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
  2.  We have to document the flow. And I think, Madhan’s flow is simple.
     *   First Deny Policies, except those who are in exception list
     *   Then Allow Policies, except those who are in the exception list
  3.  Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
  4.  If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.

The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:32 AM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality.

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs<https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups.


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 10:24 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 9:01 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Bosco,

Thanks for the review and comments. The wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies> has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:

  1.  Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
  2.  Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
  3.  The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
  4.  Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
  5.  Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand

Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

*<<Unless I am missing something, this use case is trivial.  Won’t the
following two policies do?*

   - *Policy1*
      - *Allow: Finance*
   - *Policy 2:*
      -
*Allow: Scott>> *

I did not get your point here. Do we need a deny policy (and an exception)
here or not? Can you elaborate?



On Wed, Oct 21, 2015 at 3:19 PM, Alok Lal <al...@hortonworks.com> wrote:

> @Balaji,
> Since scott is a Finance Intern he is part of Finance.  So he does NOT
> need to specified on Allow.
>
> *> Finance department is very large.  Some Interns are in Finance.  Only
> people in Finance have access to Budget forecasts.  Interns are not given
> access to Budget.  Scott is a finance intern who is doing a project for CFO
> and hence needs access to the budget data.*
>
>
> But let us modify the use case as you have suggested, i.e. as follows:
>
> *Finance department is very large.  Some Interns are in Finance.  Only
> people in Finance have access to Budget forecasts.  Interns are not given
> access to Budget.  Scott is an intern who is doing a project for CFO and
> hence needs access to the budget data.*
>
>
> Unless I am missing something, this use case is trivial.  Won’t the
> following two policies do?
>
>    - Policy1
>       - Allow: Finance
>    - Policy 2:
>       - Allow: Scott
>
> Thanks
>
> From: Balaji Ganesan <ba...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Wednesday, October 21, 2015 at 3:12 PM
>
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> Thanks Alok for taking time to put this down. For the record, I zone out
> after couple of paragraphs in an email, so please limit it next time you
> send one :-)
>
> Let us take you use case
>
> *<<Our use case is as follows:*
>
> *Finance department is very large.  Some Interns are in Finance.  Only
> people in Finance have access to Budget forecasts.  Interns are not given
> access to Budget.  Scott is a finance intern who is doing a project for CFO
> and hence needs access to the budget data.*
>
> *Madhan’s model proposes to address this with the following policy:*
>
>    - *Allow – Finance*
>    - *Deny – Intern*
>       - *Deny exception – Scott>>*
>
> Actually, for this use case, the policy in the current implementation.
> would need  to look like this
>
>    - Allow – Finance, Scott
>    - Deny – Intern
>       - Deny exception – Scott
>
> This is part which is confusing. Scott not only needs to be mentioned in
> the deny exception, but also need to be added to access policy for Scott to
> be given access. If we read it carefully it makes sense, but it is not
> intuitive enough for any layman user creating the policy. What I was
> suggesting was to make the policy definition simpler, the policy should
> like this
>
>    - Allow – Finance, Scott
>    - Deny – Interns
>
> The policy evaluation, as you had pointed out, changes in this case. We
> are indeed making exceptions for users who are explicitly given access or
> denied, very similar to the suggestion from Abhay. I agree on some of the
> disadvantages with this simplistic mode (not sure if I would call this
> imperative vs declarative), but we can get feedback from community and
> users as well evolve.
>
> On Fri, Oct 16, 2015 at 8:21 PM, Alok Lal <al...@hortonworks.com> wrote:
>
>> @Balaji,
>> I know how much you love long emails.  Here goes! :)
>>
>> You wrote:
>>
>> > Let us take the use case we discussed earlier. The group "interns" got
>> denied,but we want to exclude one user ("Scott") from that, I am suggesting
>> to have the particular user ("Scott") included in the Allow section of
>> policy and group "interns" included in Deny section of the same policy.
>> Ranger should evaluate the both Allow and Deny in the same policy and
>> deduce that user "Scott" can access the resource while anyone else in group
>> "interns" is denied to access the resource. The policy creation in the UI
>> is much simpler than having to specify "Scott" as an exception to Deny and
>> then specifying "Scott" for Allow.
>>
>> Here’s why what you say can’t be done without also imposing a
>> “hidden/implicit rules”.
>>
>> Our use case is as follows:
>>
>> Finance department is very large.  Some Interns are in Finance.  Only
>> people in Finance have access to Budget forecasts.  Interns are not given
>> access to Budget.  Scott is a finance intern who is doing a project for CFO
>> and hence needs access to the budget data.
>>
>>
>> Madhan’s model proposes to address this with the following policy:
>>
>>    - Allow – Finance
>>    - Deny – Intern
>>       - Deny exception – Scott
>>
>> Which would, as the use case requires, allow access to everyone in
>> Finance unless they are an intern, except to Scott.  Please note that
>> policy’s description alone is sufficient for anyone to know what it
>> would do.
>>
>> Now let us consider the policy that you have suggested:
>>
>>    - Allow – Finance, Scott
>>    - Deny – Interns
>>
>> For the above to honor our use case we would also need to apply the
>> following additional rule:
>>
>>    - Allow access if user has been explicitly been given access even if
>>    he has been denied.  (This BTW was what Abhay’s had suggested.)
>>
>> Unless we have the above rule we can’t disambiguate what to do about
>> Scott who is part of both Finance and Interns — one of which has been given
>> access and another has been denied.  One must have a way to break the
>> tie in the form of a rule.  @Balaji which do you feel is better?
>>
>>    - Your combination of (simpler policy + simple rule) or
>>    - Madhan’s complicated policy without any additional rules
>>
>> Perhaps the decision is not obvious at this stage.  Now, consider the
>> following use case which is only a slight modification to above.
>>
>> New use case:
>>
>> Finance department is very large.  Some Interns are in Finance.  Everyone
>> from Finance has access to Budget forecast except its interns.  CEO has
>> asked every department to allocate people to his favorite new project -
>> Compliance.  Finance has asked some of its interns to be part of Compliance
>> project and wants to give then access to Budget forecast.
>>
>>
>> Madhan’s Model can address this as follows:
>>
>>    - Allow – Finance
>>    - Deny – Intern
>>       - Deny exception: Compliance
>>
>> Which again is straight forward to interpret and verify.
>>
>> You model would have to be as follows:
>>
>>    - Allow – Finance, Compliance
>>    - Deny – Intern
>>
>> How do we interpret this policy given that some interns are in Finance,
>> some of those working on Compliance project are in Finance?  We would need
>> an additional rule as follows:
>>
>>    - If a user is explicitly allowed access because of a greater number
>>    groups (Finance and Compliance) than he is denied access (Intern) then
>>    allow wins and he should be allowed access.
>>
>> So now we have two rules.
>>
>> The point is this: Domain we want to model is complex.  And we have two
>> choices.
>>
>>    - Keep complexity in the structure of your model itself.  This makes
>>    it easy to reason about it without the need of an additional codec
>>    (set of rules).  In computer science this is often referred to as a
>>    Declarative specification, i.e. you expend your energies only in
>>    stating your intent.  You are NOT concerned about how something
>>    works.  This is Madhan’s style — naturally as programmers have a bias
>>    towards getting to declarative specification.
>>    - Keep the model simple, move complexity to a set of rules that must
>>    be consulted in a known order to get at the result.  This is often called
>>    as an Imperative specification.  To understand and reason about
>>    something it is not enough to just define your data but you also have to
>>    know the rules and know how they are applied.  This has advantage of
>>    course.  The rules can be easily re-ordered to yield a system that does
>>    something entirely different.  Disadvantages are obvious: as the number of
>>    rules grow system becomes hard to reason about and hence brittle.
>>
>> I can tell you from years of computer science research the move away from
>> an imperative style is towards declarative style.  We should think long and
>> hard before we rule in favor of an imperative specification in this day and
>> age.
>>
>> Thanks
>> - Alok
>>
>> From: Don Bosco Durai <bo...@apache.org>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Friday, October 16, 2015 at 2:10 PM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>>
>>
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> > I agree that we need a UI setting to enable/disable deny and
>> exceptions. For general majority, they should not even see any thing
>> related to deny or exceptions. Can we agree on this as a needed feature for
>> Ranger?
>>
>> I am okay with this. But we have to document and advertise this so that
>> users know that this feature exists and they can enable it.
>>
>> Bosco
>>
>>
>>
>> From: Balaji Ganesan
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Friday, October 16, 2015 at 1:47 PM
>> To: <us...@ranger.incubator.apache.org>
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> <<I am not sure how tag based policy will simplify. I feel, “Deny” is
>> advanced feature and I would expect the Admin to be smart enough to know
>> what he/she is doing. Maybe in the future we can give more tools to help
>> the Admins.>>
>> We all know from experience that expecting Admin to be smart enough does
>> not always hold true. The premise of doing a UI, and not a command line
>> tool, is the admins do not have expertise or training to construct access
>> policies for Hadoop easily. That is big differentiator for Ranger, which is
>> the ease of use and centralized policy management.
>>
>> <<Based on the Madhan’s examples, yes it is needed. Without it, there
>> are few use cases we can’t meet. We can use a convoluted to meet these
>> requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
>> If there are only few customers who would need this, the deny or allow
>> exception functionality need not be exposed for the majority. If there is
>> option available, users will use it even if they could have met their use
>> case using simpler methods.
>>
>> <<I agree with this. We can make it optional and have an UI setting to
>> enable it. Generally, users won’t know this feature exists. I would put a
>> band at the both of the policy edit page and put a link with “Deny feature
>> is disabled. Click here to enable it”.>>
>> I agree that we need a UI setting to enable/disable deny and exceptions.
>> For general majority, they should not even see any thing related to deny or
>> exceptions. Can we agree on this as a needed feature for Ranger?
>>
>> On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>>
>>> >That is the concern here. How can we expect Administrators to ensure
>>> they are keeping policies in sync between high level and lower level
>>> resources? The use case is Scott and finance group should get access while
>>> everyone else in intern group should. The intern group is part of the
>>> finance group. We can expect the administrators to achieve this by creating
>>> a higher level tag based policy and a resource based policy.
>>>
>>> I am not sure how tag based policy will simplify. I feel, “Deny” is
>>> advanced feature and I would expect the Admin to be smart enough to know
>>> what he/she is doing. Maybe in the future we can give more tools to help
>>> the Admins.
>>>
>>> >Not sure if we are discussing the same thing here. My question was
>>> around the need to have exceptions for deny and allow. If it indeed going
>>> to be complicated, then should we not such areas which add more
>>> complication.
>>> Based on the Madhan’s examples, yes it is needed. Without it, there are
>>> few use cases we can’t meet. We can use a convoluted to meet these
>>> requirement, but I feel, “Exceptions/Exclude” might be a simpler option.
>>>
>>> >I think we should provide both deny and exceptions policies as
>>> optional. The exceptions should be turned off by default and should not
>>> appear in the UI. Users who really need exceptions can turn it on.
>>> I agree with this. We can make it optional and have an UI setting to
>>> enable it. Generally, users won’t know this feature exists. I would put a
>>> band at the both of the policy edit page and put a link with “Deny feature
>>> is disabled. Click here to enable it”.
>>>
>>> Thanks
>>>
>>> Bosco
>>>
>>>
>>> From: Balaji Ganesan
>>> Reply-To: <us...@ranger.incubator.apache.org>
>>> Date: Thursday, October 15, 2015 at 10:54 PM
>>>
>>> To: <us...@ranger.incubator.apache.org>
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> <<I think, Madhan’s use case is not exactly the same. In his case, he
>>> wants to “Deny” all interns at a high level, but not necessary want to
>>> Allow (globally) “interns” in this policy (which could be bad in some
>>> cases). Scott’s permissions to any sub resources might be managed by other
>>> policies.>>
>>>
>>> That is the concern here. How can we expect Administrators to ensure
>>> they are keeping policies in sync between high level and lower level
>>> resources? The use case is Scott and finance group should get access while
>>> everyone else in intern group should. The intern group is part of the
>>> finance group. We can expect the administrators to achieve this by creating
>>> a higher level tag based policy and a resource based policy.
>>>
>>> <<When we decided to support “Deny”, we know it would add
>>> complications. We have to live it and doing partial work (because of UI) is
>>> not a good idea.>>
>>>
>>> Not sure if we are discussing the same thing here. My question was
>>> around the need to have exceptions for deny and allow. If it indeed going
>>> to be complicated, then should we not such areas which add more
>>> complication.
>>>
>>> <<The good thing about Madhan’s design is, it is full backward
>>> compatibility and it is optional to use deny and even exception. And it
>>> provides a lot of flexibility for expert users to model their policies.>>
>>>
>>> I think we should provide both deny and exceptions policies as optional.
>>> The exceptions should be turned off by default and should not appear in the
>>> UI. Users who really need exceptions can turn it on.
>>>
>>> On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>
>>> wrote:
>>>
>>>> >Allow and Deny in the same policy and deduce that user "Scott" can
>>>> access the resource while anyone else in group "interns" is denied to
>>>> access the resource. The policy creation in the UI is much simpler than
>>>> having to specify "Scott" as an exception to Deny and then specifying
>>>> "Scott" for Allow.
>>>>
>>>> I think, Madhan’s use case is not exactly the same. In his case, he
>>>> wants to “Deny” all interns at a high level, but not necessary want to
>>>> Allow (globally) “interns” in this policy (which could be bad in some
>>>> cases). Scott’s permissions to any sub resources might be managed by other
>>>> policies.
>>>>
>>>> The main thing here is:
>>>>
>>>>    1. When we decided to support “Deny”, we know it would add
>>>>    complications. We have to live it and doing partial work (because of UI) is
>>>>    not a good idea.
>>>>    2. We have to document the flow. And I think, Madhan’s flow is
>>>>    simple.
>>>>       1. First Deny Policies, except those who are in exception list
>>>>       2. Then Allow Policies, except those who are in the exception
>>>>       list
>>>>    3. Finally, you are correct. For most users and use cases, you can
>>>>    model the policies with our existing ones. So very few will use “Deny” and
>>>>    those would use it, would have to understand how it works. And I don’t
>>>>    think it is so complex, but they just need know the about 2 rules
>>>>    4. If we feel the UI is confusing it, then we need to fix the UI.
>>>>    But I feel, compared some of the other system which support “Deny” and
>>>>    hierarchy, ours is pretty elegant and scalable solution.
>>>>
>>>> The good thing about Madhan’s design is, it is full backward
>>>> compatibility and it is optional to use deny and even exception. And it
>>>> provides a lot of flexibility for expert users to model their policies.
>>>>
>>>> Bosco
>>>>
>>>>
>>>> From: Balaji Ganesan
>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>> Date: Thursday, October 15, 2015 at 10:32 AM
>>>>
>>>> To: <us...@ranger.incubator.apache.org>
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> I think we completely missed my earlier point. My question was around
>>>> whether we need to have exceptions in Deny and Allow beyond just Allow or
>>>> Deny policy items. I used the MSFT example to see if they have implemented
>>>> exceptions to Deny, which I did not find any. I am not suggesting that we
>>>> ask the users to order the policies and Ranger follow the order.
>>>>
>>>> We can open a separate thread on whether Ranger needs to use an
>>>> explicit hierarchy for policy evaluation or not. For the current
>>>> discussion, I don't see a specific use case where we would need a deny or
>>>> allow exception and not fulfilled by just allow or deny functionality.
>>>>
>>>> Let us take the use case we discussed earlier. The group "interns" got
>>>> denied,but we want to exclude one user ("Scott") from that, I am suggesting
>>>> to have the particular user ("Scott") included in the Allow section of
>>>> policy and group "interns" included in Deny section of the same policy.
>>>> Ranger should evaluate the both Allow and Deny in the same policy and
>>>> deduce that user "Scott" can access the resource while anyone else in group
>>>> "interns" is denied to access the resource. The policy creation in the UI
>>>> is much simpler than having to specify "Scott" as an exception to Deny and
>>>> then specifying "Scott" for Allow.
>>>>
>>>> On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
>>>> mneethiraj@hortonworks.com> wrote:
>>>>
>>>>> Microsoft DACL also talks about exceptions, but the model relies on
>>>>> the users to setup ACE in specific order – see below.
>>>>>
>>>>> From
>>>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
>>>>> :
>>>>>
>>>>> "In most cases, you can control access to an object by using
>>>>> access-allowed ACEs; you do not need to explicitly deny access to an
>>>>> object. The exception is when an ACE allows access to a group and you want
>>>>> to deny access to a member of the group. To do this, place an access-denied
>>>>> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
>>>>> Note that the order of the ACEs
>>>>> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
>>>>> important because the system reads the ACEs in sequence until access is
>>>>> granted or denied. The user's access-denied ACE must appear first;
>>>>> otherwise, when the system reads the group's access allowed ACE, it will
>>>>> grant access to the restricted user."
>>>>>
>>>>> This approach of relying on ordering of policy items (equivalent of
>>>>> ACEs above) can work for Ranger as well. However Microsoft DACL model
>>>>> evaluates only one DACL to determine access to an object; compare this with
>>>>> Ranger where there could be multiple policies that can be evaluated for a
>>>>> resource – for example to determine access to a finance.invoice.ssn column,
>>>>> Ranger would evaluate policies that are applicable for finance database,
>>>>> finance.invoice table and finance.invoice.ssn column. To guarantee
>>>>> consistent authorization result, that relies on proper ordering of
>>>>> allow/deny policy items in each policy, would require the policies to be
>>>>> evaluated in a specific order – either most-specific-to-most-generic
>>>>> (column policy, table policy, database policy)  or most-generic-to
>>>>> most-specific. And it might be very challenging, if not impossible, to
>>>>> ensure a policy order given our support for wildcard specification – for
>>>>> example, should a policy for “/finance/*2015/*” be evaluated before a
>>>>> policy for “/finance/invoice*/*” while determining access for a file named
>>>>> “/finance/invoice2015/vendor1.txt”?
>>>>>
>>>>> I think this approach would make the policy authoring very difficult
>>>>> and confusing, in addition to being not able to support certain scenarios
>>>>> like “deny at a higher level (like for a database), even if access might be
>>>>> allowed at a lower level (for a table/column)”, “allow at a higher level,
>>>>> but deny at a lower level). The current implementation (in tag-policy
>>>>> branch of Ranger) is much less confusing and offers building blocks that
>>>>> can be used to support more usecases and can guarantee consistent results
>>>>> for a given set of policies. If we are after simplicity, the DACL model
>>>>> does not seem to be the right choice..
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>>
>>>>> From: Balaji Ganesan
>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>> Date: Wednesday, October 14, 2015 at 10:31 AM
>>>>>
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> >> My proposal is to just keep the policy to have only allow and deny
>>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>>> "intern" group, then anyone in intern group would not be allowed while
>>>>> everyone else in finance group will get access. If there is a person in
>>>>> intern group who needs access, then user need to be taken out of the intern
>>>>> group or we need to add only the users who specifically need to be denied.
>>>>>
>>>>> <<Users who are not comfortable with the idea of “excludes” can
>>>>> continue to use only allow and deny in the policies. Users who are
>>>>> comfortable with “excludes” can choose to use it to simplify their policy
>>>>> management.>>
>>>>>
>>>>> The question here is whether exceptions are really needed in the
>>>>> product. We need to keep the product simple
>>>>> Questions is what use cases can exceptions clearly solve above and
>>>>> beyond simply access grant and access deny policy line items. As an
>>>>> example, I am looking at MSFT for how they have have implemented
>>>>> deny-access
>>>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
>>>>> The use case MSFT is trying to solve it denying access to specific users
>>>>> who are part of the group which is provided access. I think most of the
>>>>> deny-access use cases would be for specific users, not for large sets of
>>>>> groups.
>>>>>
>>>>>
>>>>> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
>>>>> mneethiraj@hortonworks.com> wrote:
>>>>>
>>>>>> >> For argument sakes, finance could be the higher level group and
>>>>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>>>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>>>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>>>>> add all groups which needs access or specify deny groups which don't need
>>>>>> access
>>>>>>
>>>>>> The groups may not fall into nice hierarchies – for example, interns
>>>>>> group might consist of users from various orgs in a company (not just from
>>>>>> finance group). In such cases, the only choice is to setup Ranger policies
>>>>>> with individual users (and not groups).
>>>>>>
>>>>>> >> My proposal is to just keep the policy to have only allow and deny
>>>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>>>> "intern" group, then anyone in intern group would not be allowed while
>>>>>> everyone else in finance group will get access. If there is a person in
>>>>>> intern group who needs access, then user need to be taken out of the intern
>>>>>> group or we need to add only the users who specifically need to be denied.
>>>>>>
>>>>>> Users who are not comfortable with the idea of “excludes” can
>>>>>> continue to use only allow and deny in the policies. Users who are
>>>>>> comfortable with “excludes” can choose to use it to simplify their policy
>>>>>> management.
>>>>>>
>>>>>> Thanks,
>>>>>> Madhan
>>>>>>
>>>>>> From: Balaji Ganesan
>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>>>>>
>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>> deny-conditions and exceptions
>>>>>>
>>>>>> <<Are you suggesting that the security admins create policies that
>>>>>> list individual users, instead of using groups? Wouldn’t that make security
>>>>>> administration more painful, in a reasonably large organization? For every
>>>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>>>> users and groups to alleviate this issue.>>
>>>>>> For argument sakes, finance could be the higher level group and
>>>>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>>>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>>>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>>>>> add all groups which needs access or specify deny groups which don't need
>>>>>> access
>>>>>>
>>>>>> My proposal is to just keep the policy to have only allow and deny
>>>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>>>> "intern" group, then anyone in intern group would not be allowed while
>>>>>> everyone else in finance group will get access. If there is a person in
>>>>>> intern group who needs access, then user need to be taken out of the intern
>>>>>> group or we need to add only the users who specifically need to be denied.
>>>>>>
>>>>>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Balaji,
>>>>>>>
>>>>>>> >> why cannot user clearly specific users or groups that would need
>>>>>>> specific access. Why is there a need to give access to finance group as a
>>>>>>> whole when we know there is a subset of user who do not need access to the
>>>>>>> the finance database? or clearly specify users who need to be denied?
>>>>>>> Are you suggesting that the security admins create policies that
>>>>>>> list individual users, instead of using groups? Wouldn’t that make security
>>>>>>> administration more painful, in a reasonably large organization? For every
>>>>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>>>>> users and groups to alleviate this issue.
>>>>>>>
>>>>>>>
>>>>>>> From: Balaji Ganesan
>>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>>>>>
>>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>>> deny-conditions and exceptions
>>>>>>>
>>>>>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>>>>>
>>>>>>> We clearly need to show case use cases for deny exclude and allow
>>>>>>> exclude. In my opinion it is very confusing to user to construct such a
>>>>>>> policy
>>>>>>>
>>>>>>> <<Let’s say one of the users, scott who is in interns and finance
>>>>>>> groups, works on an assignment that requires select access to finance
>>>>>>> database. To enable this access, the authorization policy for the database
>>>>>>> should be updated by adding a deny-exclude, as shown below:>>
>>>>>>> In the wiki, we have created a "deny" policy for intern group and an
>>>>>>> exception for Scott. First of all, why cannot user clearly specific users
>>>>>>> or groups that would need specific access. Why is there a need to give
>>>>>>> access to finance group as a whole when we know there is a subset of user
>>>>>>> who do not need access to the the finance database? or clearly specify
>>>>>>> users who need to be denied? It is confusing to think about exceptions for
>>>>>>> deny, deny, exceptions for allow and allow.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <madhan@apache.org
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Bosco,
>>>>>>>>
>>>>>>>> Thanks for the review and comments. The wiki
>>>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>>>>>> has been updated to address your comments. Please review.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Madhan
>>>>>>>>
>>>>>>>> From: Don Bosco Durai
>>>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>>>> deny-conditions and exceptions
>>>>>>>>
>>>>>>>> Madhan, thanks for putting this document together. It is looking
>>>>>>>> good.
>>>>>>>>
>>>>>>>> Can I make a few suggestions:
>>>>>>>>
>>>>>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for
>>>>>>>>    "HDFS policy that allows all finance group users to access contents of
>>>>>>>>    /finance folder, but denies access to users in interns group. Users in
>>>>>>>>    interns group will be denied the access even if they are part of finance
>>>>>>>>    group.”
>>>>>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny
>>>>>>>>    all users from “interns” group from accessing table “Employees"
>>>>>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>>>>>    good. I think, this is sort of a new concept.
>>>>>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart.
>>>>>>>>    Can we create flow chart diagram. It will be easy to understand
>>>>>>>>
>>>>>>>> Thanks again. Let me know if you need help in the documentation.
>>>>>>>>
>>>>>>>> Bosco
>>>>>>>>
>>>>>>>>
>>>>>>>> From: Madhan Neethiraj
>>>>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>>>> Subject: [DISCUSS] Policy model enhancement to support
>>>>>>>> deny-conditions and exceptions
>>>>>>>>
>>>>>>>> All,
>>>>>>>>
>>>>>>>> Apache Ranger policy model enhancement to support deny-conditions
>>>>>>>> and exceptions (RANGER-606
>>>>>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available
>>>>>>>> in  tag-policy branch
>>>>>>>> <https://github.com/apache/incubator-ranger/tree/tag-policy>. This
>>>>>>>> enhancement adds the capability to explicitly deny access to resources
>>>>>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>>>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>>>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>>>>>
>>>>>>>> An overview of the implementation, along with few examples is
>>>>>>>> available in Apache wiki page here
>>>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>>>>>> Please review.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Madhan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Alok Lal <al...@hortonworks.com>]

@Balaji,
Since scott is a Finance Intern he is part of Finance.  So he does NOT need to specified on Allow.

> Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.

But let us modify the use case as you have suggested, i.e. as follows:

Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is an intern who is doing a project for CFO and hence needs access to the budget data.

Unless I am missing something, this use case is trivial.  Won’t the following two policies do?

  *   Policy1
     *   Allow: Finance
  *   Policy 2:
     *   Allow: Scott

Thanks

From: Balaji Ganesan <ba...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Wednesday, October 21, 2015 at 3:12 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Thanks Alok for taking time to put this down. For the record, I zone out after couple of paragraphs in an email, so please limit it next time you send one :-)

Let us take you use case

<<Our use case is as follows:
Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.
Madhan’s model proposes to address this with the following policy:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception – Scott>>

Actually, for this use case, the policy in the current implementation. would need  to look like this

  *   Allow – Finance, Scott
  *   Deny – Intern
     *   Deny exception – Scott

This is part which is confusing. Scott not only needs to be mentioned in the deny exception, but also need to be added to access policy for Scott to be given access. If we read it carefully it makes sense, but it is not intuitive enough for any layman user creating the policy. What I was suggesting was to make the policy definition simpler, the policy should like this

  *   Allow – Finance, Scott
  *   Deny – Interns

The policy evaluation, as you had pointed out, changes in this case. We are indeed making exceptions for users who are explicitly given access or denied, very similar to the suggestion from Abhay. I agree on some of the disadvantages with this simplistic mode (not sure if I would call this imperative vs declarative), but we can get feedback from community and users as well evolve.

On Fri, Oct 16, 2015 at 8:21 PM, Alok Lal <al...@hortonworks.com>> wrote:
@Balaji,
I know how much you love long emails.  Here goes! :)

You wrote:

> Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

Here’s why what you say can’t be done without also imposing a “hidden/implicit rules”.

Our use case is as follows:
Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.

Madhan’s model proposes to address this with the following policy:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception – Scott

Which would, as the use case requires, allow access to everyone in Finance unless they are an intern, except to Scott.  Please note that policy’s description alone is sufficient for anyone to know what it would do.

Now let us consider the policy that you have suggested:

  *   Allow – Finance, Scott
  *   Deny – Interns

For the above to honor our use case we would also need to apply the following additional rule:

  *   Allow access if user has been explicitly been given access even if he has been denied.  (This BTW was what Abhay’s had suggested.)

Unless we have the above rule we can’t disambiguate what to do about Scott who is part of both Finance and Interns — one of which has been given access and another has been denied.  One must have a way to break the tie in the form of a rule.  @Balaji which do you feel is better?

  *   Your combination of (simpler policy + simple rule) or
  *   Madhan’s complicated policy without any additional rules

Perhaps the decision is not obvious at this stage.  Now, consider the following use case which is only a slight modification to above.

New use case:
Finance department is very large.  Some Interns are in Finance.  Everyone from Finance has access to Budget forecast except its interns.  CEO has asked every department to allocate people to his favorite new project - Compliance.  Finance has asked some of its interns to be part of Compliance project and wants to give then access to Budget forecast.

Madhan’s Model can address this as follows:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception: Compliance

Which again is straight forward to interpret and verify.

You model would have to be as follows:

  *   Allow – Finance, Compliance
  *   Deny – Intern

How do we interpret this policy given that some interns are in Finance, some of those working on Compliance project are in Finance?  We would need an additional rule as follows:

  *   If a user is explicitly allowed access because of a greater number groups (Finance and Compliance) than he is denied access (Intern) then allow wins and he should be allowed access.

So now we have two rules.

The point is this: Domain we want to model is complex.  And we have two choices.

  *   Keep complexity in the structure of your model itself.  This makes it easy to reason about it without the need of an additional codec (set of rules).  In computer science this is often referred to as a Declarative specification, i.e. you expend your energies only in stating your intent.  You are NOT concerned about how something works.  This is Madhan’s style — naturally as programmers have a bias towards getting to declarative specification.
  *   Keep the model simple, move complexity to a set of rules that must be consulted in a known order to get at the result.  This is often called as an Imperative specification.  To understand and reason about something it is not enough to just define your data but you also have to know the rules and know how they are applied.  This has advantage of course.  The rules can be easily re-ordered to yield a system that does something entirely different.  Disadvantages are obvious: as the number of rules grow system becomes hard to reason about and hence brittle.

I can tell you from years of computer science research the move away from an imperative style is towards declarative style.  We should think long and hard before we rule in favor of an imperative specification in this day and age.

Thanks
- Alok

From: Don Bosco Durai <bo...@apache.org>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, October 16, 2015 at 2:10 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>

Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

> I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger?

I am okay with this. But we have to document and advertise this so that users know that this feature exists and they can enable it.

Bosco



From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Friday, October 16, 2015 at 1:47 PM
To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.>>
We all know from experience that expecting Admin to be smart enough does not always hold true. The premise of doing a UI, and not a command line tool, is the admins do not have expertise or training to construct access policies for Hadoop easily. That is big differentiator for Ranger, which is the ease of use and centralized policy management.

<<Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
If there are only few customers who would need this, the deny or allow exception functionality need not be exposed for the majority. If there is option available, users will use it even if they could have met their use case using simpler methods.

<<I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.>>
I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger?

On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org>> wrote:
>That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.

>Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.
Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.

>I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.
I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.

Thanks

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:54 PM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:

  1.  When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
  2.  We have to document the flow. And I think, Madhan’s flow is simple.
     *   First Deny Policies, except those who are in exception list
     *   Then Allow Policies, except those who are in the exception list
  3.  Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
  4.  If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.

The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:32 AM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality.

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs<https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups.


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 10:24 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 9:01 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Bosco,

Thanks for the review and comments. The wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies> has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:

  1.  Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
  2.  Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
  3.  The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
  4.  Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
  5.  Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand

Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

Thanks Alok for taking time to put this down. For the record, I zone out
after couple of paragraphs in an email, so please limit it next time you
send one :-)

Let us take you use case

*<<Our use case is as follows:*

*Finance department is very large.  Some Interns are in Finance.  Only
people in Finance have access to Budget forecasts.  Interns are not given
access to Budget.  Scott is a finance intern who is doing a project for CFO
and hence needs access to the budget data.*

*Madhan’s model proposes to address this with the following policy:*

   - *Allow – Finance*
   - *Deny – Intern*
      - *Deny exception – Scott>>*

Actually, for this use case, the policy in the current implementation.
would need  to look like this

   - Allow – Finance, Scott
   - Deny – Intern
      - Deny exception – Scott

This is part which is confusing. Scott not only needs to be mentioned in
the deny exception, but also need to be added to access policy for Scott to
be given access. If we read it carefully it makes sense, but it is not
intuitive enough for any layman user creating the policy. What I was
suggesting was to make the policy definition simpler, the policy should
like this

   - Allow – Finance, Scott
   - Deny – Interns

The policy evaluation, as you had pointed out, changes in this case. We are
indeed making exceptions for users who are explicitly given access or
denied, very similar to the suggestion from Abhay. I agree on some of the
disadvantages with this simplistic mode (not sure if I would call this
imperative vs declarative), but we can get feedback from community and
users as well evolve.

On Fri, Oct 16, 2015 at 8:21 PM, Alok Lal <al...@hortonworks.com> wrote:

> @Balaji,
> I know how much you love long emails.  Here goes! :)
>
> You wrote:
>
> > Let us take the use case we discussed earlier. The group "interns" got
> denied,but we want to exclude one user ("Scott") from that, I am suggesting
> to have the particular user ("Scott") included in the Allow section of
> policy and group "interns" included in Deny section of the same policy.
> Ranger should evaluate the both Allow and Deny in the same policy and
> deduce that user "Scott" can access the resource while anyone else in group
> "interns" is denied to access the resource. The policy creation in the UI
> is much simpler than having to specify "Scott" as an exception to Deny and
> then specifying "Scott" for Allow.
>
> Here’s why what you say can’t be done without also imposing a
> “hidden/implicit rules”.
>
> Our use case is as follows:
>
> Finance department is very large.  Some Interns are in Finance.  Only
> people in Finance have access to Budget forecasts.  Interns are not given
> access to Budget.  Scott is a finance intern who is doing a project for CFO
> and hence needs access to the budget data.
>
>
> Madhan’s model proposes to address this with the following policy:
>
>    - Allow – Finance
>    - Deny – Intern
>       - Deny exception – Scott
>
> Which would, as the use case requires, allow access to everyone in Finance
> unless they are an intern, except to Scott.  Please note that policy’s
> description alone is sufficient for anyone to know what it would do.
>
> Now let us consider the policy that you have suggested:
>
>    - Allow – Finance, Scott
>    - Deny – Interns
>
> For the above to honor our use case we would also need to apply the
> following additional rule:
>
>    - Allow access if user has been explicitly been given access even if
>    he has been denied.  (This BTW was what Abhay’s had suggested.)
>
> Unless we have the above rule we can’t disambiguate what to do about Scott
> who is part of both Finance and Interns — one of which has been given
> access and another has been denied.  One must have a way to break the tie
> in the form of a rule.  @Balaji which do you feel is better?
>
>    - Your combination of (simpler policy + simple rule) or
>    - Madhan’s complicated policy without any additional rules
>
> Perhaps the decision is not obvious at this stage.  Now, consider the
> following use case which is only a slight modification to above.
>
> New use case:
>
> Finance department is very large.  Some Interns are in Finance.  Everyone
> from Finance has access to Budget forecast except its interns.  CEO has
> asked every department to allocate people to his favorite new project -
> Compliance.  Finance has asked some of its interns to be part of Compliance
> project and wants to give then access to Budget forecast.
>
>
> Madhan’s Model can address this as follows:
>
>    - Allow – Finance
>    - Deny – Intern
>       - Deny exception: Compliance
>
> Which again is straight forward to interpret and verify.
>
> You model would have to be as follows:
>
>    - Allow – Finance, Compliance
>    - Deny – Intern
>
> How do we interpret this policy given that some interns are in Finance,
> some of those working on Compliance project are in Finance?  We would need
> an additional rule as follows:
>
>    - If a user is explicitly allowed access because of a greater number
>    groups (Finance and Compliance) than he is denied access (Intern) then
>    allow wins and he should be allowed access.
>
> So now we have two rules.
>
> The point is this: Domain we want to model is complex.  And we have two
> choices.
>
>    - Keep complexity in the structure of your model itself.  This makes
>    it easy to reason about it without the need of an additional codec
>    (set of rules).  In computer science this is often referred to as a
>    Declarative specification, i.e. you expend your energies only in
>    stating your intent.  You are NOT concerned about how something
>    works.  This is Madhan’s style — naturally as programmers have a bias
>    towards getting to declarative specification.
>    - Keep the model simple, move complexity to a set of rules that must
>    be consulted in a known order to get at the result.  This is often called
>    as an Imperative specification.  To understand and reason about
>    something it is not enough to just define your data but you also have to
>    know the rules and know how they are applied.  This has advantage of
>    course.  The rules can be easily re-ordered to yield a system that does
>    something entirely different.  Disadvantages are obvious: as the number of
>    rules grow system becomes hard to reason about and hence brittle.
>
> I can tell you from years of computer science research the move away from
> an imperative style is towards declarative style.  We should think long and
> hard before we rule in favor of an imperative specification in this day and
> age.
>
> Thanks
> - Alok
>
> From: Don Bosco Durai <bo...@apache.org>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Friday, October 16, 2015 at 2:10 PM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> > I agree that we need a UI setting to enable/disable deny and exceptions.
> For general majority, they should not even see any thing related to deny or
> exceptions. Can we agree on this as a needed feature for Ranger?
>
> I am okay with this. But we have to document and advertise this so that
> users know that this feature exists and they can enable it.
>
> Bosco
>
>
>
> From: Balaji Ganesan
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Friday, October 16, 2015 at 1:47 PM
> To: <us...@ranger.incubator.apache.org>
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> <<I am not sure how tag based policy will simplify. I feel, “Deny” is
> advanced feature and I would expect the Admin to be smart enough to know
> what he/she is doing. Maybe in the future we can give more tools to help
> the Admins.>>
> We all know from experience that expecting Admin to be smart enough does
> not always hold true. The premise of doing a UI, and not a command line
> tool, is the admins do not have expertise or training to construct access
> policies for Hadoop easily. That is big differentiator for Ranger, which is
> the ease of use and centralized policy management.
>
> <<Based on the Madhan’s examples, yes it is needed. Without it, there are
> few use cases we can’t meet. We can use a convoluted to meet these
> requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
> If there are only few customers who would need this, the deny or allow
> exception functionality need not be exposed for the majority. If there is
> option available, users will use it even if they could have met their use
> case using simpler methods.
>
> <<I agree with this. We can make it optional and have an UI setting to
> enable it. Generally, users won’t know this feature exists. I would put a
> band at the both of the policy edit page and put a link with “Deny feature
> is disabled. Click here to enable it”.>>
> I agree that we need a UI setting to enable/disable deny and exceptions.
> For general majority, they should not even see any thing related to deny or
> exceptions. Can we agree on this as a needed feature for Ranger?
>
> On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org> wrote:
>
>> >That is the concern here. How can we expect Administrators to ensure
>> they are keeping policies in sync between high level and lower level
>> resources? The use case is Scott and finance group should get access while
>> everyone else in intern group should. The intern group is part of the
>> finance group. We can expect the administrators to achieve this by creating
>> a higher level tag based policy and a resource based policy.
>>
>> I am not sure how tag based policy will simplify. I feel, “Deny” is
>> advanced feature and I would expect the Admin to be smart enough to know
>> what he/she is doing. Maybe in the future we can give more tools to help
>> the Admins.
>>
>> >Not sure if we are discussing the same thing here. My question was
>> around the need to have exceptions for deny and allow. If it indeed going
>> to be complicated, then should we not such areas which add more
>> complication.
>> Based on the Madhan’s examples, yes it is needed. Without it, there are
>> few use cases we can’t meet. We can use a convoluted to meet these
>> requirement, but I feel, “Exceptions/Exclude” might be a simpler option.
>>
>> >I think we should provide both deny and exceptions policies as optional.
>> The exceptions should be turned off by default and should not appear in the
>> UI. Users who really need exceptions can turn it on.
>> I agree with this. We can make it optional and have an UI setting to
>> enable it. Generally, users won’t know this feature exists. I would put a
>> band at the both of the policy edit page and put a link with “Deny feature
>> is disabled. Click here to enable it”.
>>
>> Thanks
>>
>> Bosco
>>
>>
>> From: Balaji Ganesan
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Thursday, October 15, 2015 at 10:54 PM
>>
>> To: <us...@ranger.incubator.apache.org>
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> <<I think, Madhan’s use case is not exactly the same. In his case, he
>> wants to “Deny” all interns at a high level, but not necessary want to
>> Allow (globally) “interns” in this policy (which could be bad in some
>> cases). Scott’s permissions to any sub resources might be managed by other
>> policies.>>
>>
>> That is the concern here. How can we expect Administrators to ensure they
>> are keeping policies in sync between high level and lower level resources?
>> The use case is Scott and finance group should get access while everyone
>> else in intern group should. The intern group is part of the finance group.
>> We can expect the administrators to achieve this by creating a higher level
>> tag based policy and a resource based policy.
>>
>> <<When we decided to support “Deny”, we know it would add complications.
>> We have to live it and doing partial work (because of UI) is not a good
>> idea.>>
>>
>> Not sure if we are discussing the same thing here. My question was around
>> the need to have exceptions for deny and allow. If it indeed going to be
>> complicated, then should we not such areas which add more complication.
>>
>> <<The good thing about Madhan’s design is, it is full backward
>> compatibility and it is optional to use deny and even exception. And it
>> provides a lot of flexibility for expert users to model their policies.>>
>>
>> I think we should provide both deny and exceptions policies as optional.
>> The exceptions should be turned off by default and should not appear in the
>> UI. Users who really need exceptions can turn it on.
>>
>> On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>>
>>> >Allow and Deny in the same policy and deduce that user "Scott" can
>>> access the resource while anyone else in group "interns" is denied to
>>> access the resource. The policy creation in the UI is much simpler than
>>> having to specify "Scott" as an exception to Deny and then specifying
>>> "Scott" for Allow.
>>>
>>> I think, Madhan’s use case is not exactly the same. In his case, he
>>> wants to “Deny” all interns at a high level, but not necessary want to
>>> Allow (globally) “interns” in this policy (which could be bad in some
>>> cases). Scott’s permissions to any sub resources might be managed by other
>>> policies.
>>>
>>> The main thing here is:
>>>
>>>    1. When we decided to support “Deny”, we know it would add
>>>    complications. We have to live it and doing partial work (because of UI) is
>>>    not a good idea.
>>>    2. We have to document the flow. And I think, Madhan’s flow is
>>>    simple.
>>>       1. First Deny Policies, except those who are in exception list
>>>       2. Then Allow Policies, except those who are in the exception list
>>>    3. Finally, you are correct. For most users and use cases, you can
>>>    model the policies with our existing ones. So very few will use “Deny” and
>>>    those would use it, would have to understand how it works. And I don’t
>>>    think it is so complex, but they just need know the about 2 rules
>>>    4. If we feel the UI is confusing it, then we need to fix the UI.
>>>    But I feel, compared some of the other system which support “Deny” and
>>>    hierarchy, ours is pretty elegant and scalable solution.
>>>
>>> The good thing about Madhan’s design is, it is full backward
>>> compatibility and it is optional to use deny and even exception. And it
>>> provides a lot of flexibility for expert users to model their policies.
>>>
>>> Bosco
>>>
>>>
>>> From: Balaji Ganesan
>>> Reply-To: <us...@ranger.incubator.apache.org>
>>> Date: Thursday, October 15, 2015 at 10:32 AM
>>>
>>> To: <us...@ranger.incubator.apache.org>
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> I think we completely missed my earlier point. My question was around
>>> whether we need to have exceptions in Deny and Allow beyond just Allow or
>>> Deny policy items. I used the MSFT example to see if they have implemented
>>> exceptions to Deny, which I did not find any. I am not suggesting that we
>>> ask the users to order the policies and Ranger follow the order.
>>>
>>> We can open a separate thread on whether Ranger needs to use an explicit
>>> hierarchy for policy evaluation or not. For the current discussion, I don't
>>> see a specific use case where we would need a deny or allow exception and
>>> not fulfilled by just allow or deny functionality.
>>>
>>> Let us take the use case we discussed earlier. The group "interns" got
>>> denied,but we want to exclude one user ("Scott") from that, I am suggesting
>>> to have the particular user ("Scott") included in the Allow section of
>>> policy and group "interns" included in Deny section of the same policy.
>>> Ranger should evaluate the both Allow and Deny in the same policy and
>>> deduce that user "Scott" can access the resource while anyone else in group
>>> "interns" is denied to access the resource. The policy creation in the UI
>>> is much simpler than having to specify "Scott" as an exception to Deny and
>>> then specifying "Scott" for Allow.
>>>
>>> On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
>>> mneethiraj@hortonworks.com> wrote:
>>>
>>>> Microsoft DACL also talks about exceptions, but the model relies on the
>>>> users to setup ACE in specific order – see below.
>>>>
>>>> From
>>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
>>>> :
>>>>
>>>> "In most cases, you can control access to an object by using
>>>> access-allowed ACEs; you do not need to explicitly deny access to an
>>>> object. The exception is when an ACE allows access to a group and you want
>>>> to deny access to a member of the group. To do this, place an access-denied
>>>> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
>>>> Note that the order of the ACEs
>>>> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
>>>> important because the system reads the ACEs in sequence until access is
>>>> granted or denied. The user's access-denied ACE must appear first;
>>>> otherwise, when the system reads the group's access allowed ACE, it will
>>>> grant access to the restricted user."
>>>>
>>>> This approach of relying on ordering of policy items (equivalent of
>>>> ACEs above) can work for Ranger as well. However Microsoft DACL model
>>>> evaluates only one DACL to determine access to an object; compare this with
>>>> Ranger where there could be multiple policies that can be evaluated for a
>>>> resource – for example to determine access to a finance.invoice.ssn column,
>>>> Ranger would evaluate policies that are applicable for finance database,
>>>> finance.invoice table and finance.invoice.ssn column. To guarantee
>>>> consistent authorization result, that relies on proper ordering of
>>>> allow/deny policy items in each policy, would require the policies to be
>>>> evaluated in a specific order – either most-specific-to-most-generic
>>>> (column policy, table policy, database policy)  or most-generic-to
>>>> most-specific. And it might be very challenging, if not impossible, to
>>>> ensure a policy order given our support for wildcard specification – for
>>>> example, should a policy for “/finance/*2015/*” be evaluated before a
>>>> policy for “/finance/invoice*/*” while determining access for a file named
>>>> “/finance/invoice2015/vendor1.txt”?
>>>>
>>>> I think this approach would make the policy authoring very difficult
>>>> and confusing, in addition to being not able to support certain scenarios
>>>> like “deny at a higher level (like for a database), even if access might be
>>>> allowed at a lower level (for a table/column)”, “allow at a higher level,
>>>> but deny at a lower level). The current implementation (in tag-policy
>>>> branch of Ranger) is much less confusing and offers building blocks that
>>>> can be used to support more usecases and can guarantee consistent results
>>>> for a given set of policies. If we are after simplicity, the DACL model
>>>> does not seem to be the right choice..
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>>
>>>> From: Balaji Ganesan
>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>> Date: Wednesday, October 14, 2015 at 10:31 AM
>>>>
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> >> My proposal is to just keep the policy to have only allow and deny
>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>> "intern" group, then anyone in intern group would not be allowed while
>>>> everyone else in finance group will get access. If there is a person in
>>>> intern group who needs access, then user need to be taken out of the intern
>>>> group or we need to add only the users who specifically need to be denied.
>>>>
>>>> <<Users who are not comfortable with the idea of “excludes” can
>>>> continue to use only allow and deny in the policies. Users who are
>>>> comfortable with “excludes” can choose to use it to simplify their policy
>>>> management.>>
>>>>
>>>> The question here is whether exceptions are really needed in the
>>>> product. We need to keep the product simple
>>>> Questions is what use cases can exceptions clearly solve above and
>>>> beyond simply access grant and access deny policy line items. As an
>>>> example, I am looking at MSFT for how they have have implemented
>>>> deny-access
>>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
>>>> The use case MSFT is trying to solve it denying access to specific users
>>>> who are part of the group which is provided access. I think most of the
>>>> deny-access use cases would be for specific users, not for large sets of
>>>> groups.
>>>>
>>>>
>>>> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
>>>> mneethiraj@hortonworks.com> wrote:
>>>>
>>>>> >> For argument sakes, finance could be the higher level group and
>>>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>>>> add all groups which needs access or specify deny groups which don't need
>>>>> access
>>>>>
>>>>> The groups may not fall into nice hierarchies – for example, interns
>>>>> group might consist of users from various orgs in a company (not just from
>>>>> finance group). In such cases, the only choice is to setup Ranger policies
>>>>> with individual users (and not groups).
>>>>>
>>>>> >> My proposal is to just keep the policy to have only allow and deny
>>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>>> "intern" group, then anyone in intern group would not be allowed while
>>>>> everyone else in finance group will get access. If there is a person in
>>>>> intern group who needs access, then user need to be taken out of the intern
>>>>> group or we need to add only the users who specifically need to be denied.
>>>>>
>>>>> Users who are not comfortable with the idea of “excludes” can continue
>>>>> to use only allow and deny in the policies. Users who are comfortable with
>>>>> “excludes” can choose to use it to simplify their policy management.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>> From: Balaji Ganesan
>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>>>>
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> <<Are you suggesting that the security admins create policies that
>>>>> list individual users, instead of using groups? Wouldn’t that make security
>>>>> administration more painful, in a reasonably large organization? For every
>>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>>> users and groups to alleviate this issue.>>
>>>>> For argument sakes, finance could be the higher level group and intern
>>>>> could be a sub group in a hierarchy. There could be n number of sub groups.
>>>>> We could provide access by adding n-1 groups to the policy. We can argue
>>>>> whether n<10 or n>50,  depending on the answer it would make sense to add
>>>>> all groups which needs access or specify deny groups which don't need access
>>>>>
>>>>> My proposal is to just keep the policy to have only allow and deny
>>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>>> "intern" group, then anyone in intern group would not be allowed while
>>>>> everyone else in finance group will get access. If there is a person in
>>>>> intern group who needs access, then user need to be taken out of the intern
>>>>> group or we need to add only the users who specifically need to be denied.
>>>>>
>>>>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Balaji,
>>>>>>
>>>>>> >> why cannot user clearly specific users or groups that would need
>>>>>> specific access. Why is there a need to give access to finance group as a
>>>>>> whole when we know there is a subset of user who do not need access to the
>>>>>> the finance database? or clearly specify users who need to be denied?
>>>>>> Are you suggesting that the security admins create policies that list
>>>>>> individual users, instead of using groups? Wouldn’t that make security
>>>>>> administration more painful, in a reasonably large organization? For every
>>>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>>>> users and groups to alleviate this issue.
>>>>>>
>>>>>>
>>>>>> From: Balaji Ganesan
>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>>>>
>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>> deny-conditions and exceptions
>>>>>>
>>>>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>>>>
>>>>>> We clearly need to show case use cases for deny exclude and allow
>>>>>> exclude. In my opinion it is very confusing to user to construct such a
>>>>>> policy
>>>>>>
>>>>>> <<Let’s say one of the users, scott who is in interns and finance
>>>>>> groups, works on an assignment that requires select access to finance
>>>>>> database. To enable this access, the authorization policy for the database
>>>>>> should be updated by adding a deny-exclude, as shown below:>>
>>>>>> In the wiki, we have created a "deny" policy for intern group and an
>>>>>> exception for Scott. First of all, why cannot user clearly specific users
>>>>>> or groups that would need specific access. Why is there a need to give
>>>>>> access to finance group as a whole when we know there is a subset of user
>>>>>> who do not need access to the the finance database? or clearly specify
>>>>>> users who need to be denied? It is confusing to think about exceptions for
>>>>>> deny, deny, exceptions for allow and allow.
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Bosco,
>>>>>>>
>>>>>>> Thanks for the review and comments. The wiki
>>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>>>>> has been updated to address your comments. Please review.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Madhan
>>>>>>>
>>>>>>> From: Don Bosco Durai
>>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>>> deny-conditions and exceptions
>>>>>>>
>>>>>>> Madhan, thanks for putting this document together. It is looking
>>>>>>> good.
>>>>>>>
>>>>>>> Can I make a few suggestions:
>>>>>>>
>>>>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for
>>>>>>>    "HDFS policy that allows all finance group users to access contents of
>>>>>>>    /finance folder, but denies access to users in interns group. Users in
>>>>>>>    interns group will be denied the access even if they are part of finance
>>>>>>>    group.”
>>>>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny
>>>>>>>    all users from “interns” group from accessing table “Employees"
>>>>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>>>>    good. I think, this is sort of a new concept.
>>>>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart.
>>>>>>>    Can we create flow chart diagram. It will be easy to understand
>>>>>>>
>>>>>>> Thanks again. Let me know if you need help in the documentation.
>>>>>>>
>>>>>>> Bosco
>>>>>>>
>>>>>>>
>>>>>>> From: Madhan Neethiraj
>>>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>>> Subject: [DISCUSS] Policy model enhancement to support
>>>>>>> deny-conditions and exceptions
>>>>>>>
>>>>>>> All,
>>>>>>>
>>>>>>> Apache Ranger policy model enhancement to support deny-conditions
>>>>>>> and exceptions (RANGER-606
>>>>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in
>>>>>>>  tag-policy branch
>>>>>>> <https://github.com/apache/incubator-ranger/tree/tag-policy>. This
>>>>>>> enhancement adds the capability to explicitly deny access to resources
>>>>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>>>>
>>>>>>> An overview of the implementation, along with few examples is
>>>>>>> available in Apache wiki page here
>>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>>>>> Please review.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Madhan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Alok Lal <al...@hortonworks.com>]

@Balaji,
I know how much you love long emails.  Here goes! :)

You wrote:

> Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

Here’s why what you say can’t be done without also imposing a “hidden/implicit rules”.

Our use case is as follows:
Finance department is very large.  Some Interns are in Finance.  Only people in Finance have access to Budget forecasts.  Interns are not given access to Budget.  Scott is a finance intern who is doing a project for CFO and hence needs access to the budget data.

Madhan’s model proposes to address this with the following policy:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception – Scott

Which would, as the use case requires, allow access to everyone in Finance unless they are an intern, except to Scott.  Please note that policy’s description alone is sufficient for anyone to know what it would do.

Now let us consider the policy that you have suggested:

  *   Allow – Finance, Scott
  *   Deny – Interns

For the above to honor our use case we would also need to apply the following additional rule:

  *   Allow access if user has been explicitly been given access even if he has been denied.  (This BTW was what Abhay’s had suggested.)

Unless we have the above rule we can’t disambiguate what to do about Scott who is part of both Finance and Interns — one of which has been given access and another has been denied.  One must have a way to break the tie in the form of a rule.  @Balaji which do you feel is better?

  *   Your combination of (simpler policy + simple rule) or
  *   Madhan’s complicated policy without any additional rules

Perhaps the decision is not obvious at this stage.  Now, consider the following use case which is only a slight modification to above.

New use case:
Finance department is very large.  Some Interns are in Finance.  Everyone from Finance has access to Budget forecast except its interns.  CEO has asked every department to allocate people to his favorite new project - Compliance.  Finance has asked some of its interns to be part of Compliance project and wants to give then access to Budget forecast.

Madhan’s Model can address this as follows:

  *   Allow – Finance
  *   Deny – Intern
     *   Deny exception: Compliance

Which again is straight forward to interpret and verify.

You model would have to be as follows:

  *   Allow – Finance, Compliance
  *   Deny – Intern

How do we interpret this policy given that some interns are in Finance, some of those working on Compliance project are in Finance?  We would need an additional rule as follows:

  *   If a user is explicitly allowed access because of a greater number groups (Finance and Compliance) than he is denied access (Intern) then allow wins and he should be allowed access.

So now we have two rules.

The point is this: Domain we want to model is complex.  And we have two choices.

  *   Keep complexity in the structure of your model itself.  This makes it easy to reason about it without the need of an additional codec (set of rules).  In computer science this is often referred to as a Declarative specification, i.e. you expend your energies only in stating your intent.  You are NOT concerned about how something works.  This is Madhan’s style — naturally as programmers have a bias towards getting to declarative specification.
  *   Keep the model simple, move complexity to a set of rules that must be consulted in a known order to get at the result.  This is often called as an Imperative specification.  To understand and reason about something it is not enough to just define your data but you also have to know the rules and know how they are applied.  This has advantage of course.  The rules can be easily re-ordered to yield a system that does something entirely different.  Disadvantages are obvious: as the number of rules grow system becomes hard to reason about and hence brittle.

I can tell you from years of computer science research the move away from an imperative style is towards declarative style.  We should think long and hard before we rule in favor of an imperative specification in this day and age.

Thanks
- Alok

From: Don Bosco Durai <bo...@apache.org>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, October 16, 2015 at 2:10 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

> I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger?

I am okay with this. But we have to document and advertise this so that users know that this feature exists and they can enable it.

Bosco



From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Friday, October 16, 2015 at 1:47 PM
To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.>>
We all know from experience that expecting Admin to be smart enough does not always hold true. The premise of doing a UI, and not a command line tool, is the admins do not have expertise or training to construct access policies for Hadoop easily. That is big differentiator for Ranger, which is the ease of use and centralized policy management.

<<Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
If there are only few customers who would need this, the deny or allow exception functionality need not be exposed for the majority. If there is option available, users will use it even if they could have met their use case using simpler methods.

<<I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.>>
I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger?

On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org>> wrote:
>That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.

>Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.
Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.

>I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.
I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.

Thanks

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:54 PM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:

  1.  When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
  2.  We have to document the flow. And I think, Madhan’s flow is simple.
     *   First Deny Policies, except those who are in exception list
     *   Then Allow Policies, except those who are in the exception list
  3.  Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
  4.  If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.

The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:32 AM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality.

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs<https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups.


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 10:24 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 9:01 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Bosco,

Thanks for the review and comments. The wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies> has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:

  1.  Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
  2.  Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
  3.  The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
  4.  Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
  5.  Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand

Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

> I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger? 

I am okay with this. But we have to document and advertise this so that users know that this feature exists and they can enable it.

Bosco



From:  Balaji Ganesan
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Friday, October 16, 2015 at 1:47 PM
To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.>>
We all know from experience that expecting Admin to be smart enough does not always hold true. The premise of doing a UI, and not a command line tool, is the admins do not have expertise or training to construct access policies for Hadoop easily. That is big differentiator for Ranger, which is the ease of use and centralized policy management. 

<<Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
If there are only few customers who would need this, the deny or allow exception functionality need not be exposed for the majority. If there is option available, users will use it even if they could have met their use case using simpler methods. 

<<I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.>>
I agree that we need a UI setting to enable/disable deny and exceptions. For general majority, they should not even see any thing related to deny or exceptions. Can we agree on this as a needed feature for Ranger? 

On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org> wrote:
>That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy. 

I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.

>Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication. 
Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.

>I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on. 
I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.

Thanks

Bosco


From:  Balaji Ganesan
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 10:54 PM

To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy. 

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication. 

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on. 

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:
When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
We have to document the flow. And I think, Madhan’s flow is simple. 
First Deny Policies, except those who are in exception list
Then Allow Policies, except those who are in the exception list
Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.
The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From:  Balaji Ganesan
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 10:32 AM

To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality. 

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple 
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups. 


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 10:24 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 9:01 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

<<I am not sure how tag based policy will simplify. I feel, “Deny” is
advanced feature and I would expect the Admin to be smart enough to know
what he/she is doing. Maybe in the future we can give more tools to help
the Admins.>>
We all know from experience that expecting Admin to be smart enough does
not always hold true. The premise of doing a UI, and not a command line
tool, is the admins do not have expertise or training to construct access
policies for Hadoop easily. That is big differentiator for Ranger, which is
the ease of use and centralized policy management.

<<Based on the Madhan’s examples, yes it is needed. Without it, there are
few use cases we can’t meet. We can use a convoluted to meet these
requirement, but I feel, “Exceptions/Exclude” might be a simpler option.>>
If there are only few customers who would need this, the deny or allow
exception functionality need not be exposed for the majority. If there is
option available, users will use it even if they could have met their use
case using simpler methods.

<<I agree with this. We can make it optional and have an UI setting to
enable it. Generally, users won’t know this feature exists. I would put a
band at the both of the policy edit page and put a link with “Deny feature
is disabled. Click here to enable it”.>>
I agree that we need a UI setting to enable/disable deny and exceptions.
For general majority, they should not even see any thing related to deny or
exceptions. Can we agree on this as a needed feature for Ranger?

On Fri, Oct 16, 2015 at 1:23 PM, Don Bosco Durai <bo...@apache.org> wrote:

> >That is the concern here. How can we expect Administrators to ensure they
> are keeping policies in sync between high level and lower level resources?
> The use case is Scott and finance group should get access while everyone
> else in intern group should. The intern group is part of the finance group.
> We can expect the administrators to achieve this by creating a higher level
> tag based policy and a resource based policy.
>
> I am not sure how tag based policy will simplify. I feel, “Deny” is
> advanced feature and I would expect the Admin to be smart enough to know
> what he/she is doing. Maybe in the future we can give more tools to help
> the Admins.
>
> >Not sure if we are discussing the same thing here. My question was around
> the need to have exceptions for deny and allow. If it indeed going to be
> complicated, then should we not such areas which add more complication.
> Based on the Madhan’s examples, yes it is needed. Without it, there are
> few use cases we can’t meet. We can use a convoluted to meet these
> requirement, but I feel, “Exceptions/Exclude” might be a simpler option.
>
> >I think we should provide both deny and exceptions policies as optional.
> The exceptions should be turned off by default and should not appear in the
> UI. Users who really need exceptions can turn it on.
> I agree with this. We can make it optional and have an UI setting to
> enable it. Generally, users won’t know this feature exists. I would put a
> band at the both of the policy edit page and put a link with “Deny feature
> is disabled. Click here to enable it”.
>
> Thanks
>
> Bosco
>
>
> From: Balaji Ganesan
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Thursday, October 15, 2015 at 10:54 PM
>
> To: <us...@ranger.incubator.apache.org>
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> <<I think, Madhan’s use case is not exactly the same. In his case, he
> wants to “Deny” all interns at a high level, but not necessary want to
> Allow (globally) “interns” in this policy (which could be bad in some
> cases). Scott’s permissions to any sub resources might be managed by other
> policies.>>
>
> That is the concern here. How can we expect Administrators to ensure they
> are keeping policies in sync between high level and lower level resources?
> The use case is Scott and finance group should get access while everyone
> else in intern group should. The intern group is part of the finance group.
> We can expect the administrators to achieve this by creating a higher level
> tag based policy and a resource based policy.
>
> <<When we decided to support “Deny”, we know it would add complications.
> We have to live it and doing partial work (because of UI) is not a good
> idea.>>
>
> Not sure if we are discussing the same thing here. My question was around
> the need to have exceptions for deny and allow. If it indeed going to be
> complicated, then should we not such areas which add more complication.
>
> <<The good thing about Madhan’s design is, it is full backward
> compatibility and it is optional to use deny and even exception. And it
> provides a lot of flexibility for expert users to model their policies.>>
>
> I think we should provide both deny and exceptions policies as optional.
> The exceptions should be turned off by default and should not appear in the
> UI. Users who really need exceptions can turn it on.
>
> On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>
> wrote:
>
>> >Allow and Deny in the same policy and deduce that user "Scott" can
>> access the resource while anyone else in group "interns" is denied to
>> access the resource. The policy creation in the UI is much simpler than
>> having to specify "Scott" as an exception to Deny and then specifying
>> "Scott" for Allow.
>>
>> I think, Madhan’s use case is not exactly the same. In his case, he wants
>> to “Deny” all interns at a high level, but not necessary want to Allow
>> (globally) “interns” in this policy (which could be bad in some cases).
>> Scott’s permissions to any sub resources might be managed by other policies.
>>
>> The main thing here is:
>>
>>    1. When we decided to support “Deny”, we know it would add
>>    complications. We have to live it and doing partial work (because of UI) is
>>    not a good idea.
>>    2. We have to document the flow. And I think, Madhan’s flow is
>>    simple.
>>       1. First Deny Policies, except those who are in exception list
>>       2. Then Allow Policies, except those who are in the exception list
>>    3. Finally, you are correct. For most users and use cases, you can
>>    model the policies with our existing ones. So very few will use “Deny” and
>>    those would use it, would have to understand how it works. And I don’t
>>    think it is so complex, but they just need know the about 2 rules
>>    4. If we feel the UI is confusing it, then we need to fix the UI. But
>>    I feel, compared some of the other system which support “Deny” and
>>    hierarchy, ours is pretty elegant and scalable solution.
>>
>> The good thing about Madhan’s design is, it is full backward
>> compatibility and it is optional to use deny and even exception. And it
>> provides a lot of flexibility for expert users to model their policies.
>>
>> Bosco
>>
>>
>> From: Balaji Ganesan
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Thursday, October 15, 2015 at 10:32 AM
>>
>> To: <us...@ranger.incubator.apache.org>
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> I think we completely missed my earlier point. My question was around
>> whether we need to have exceptions in Deny and Allow beyond just Allow or
>> Deny policy items. I used the MSFT example to see if they have implemented
>> exceptions to Deny, which I did not find any. I am not suggesting that we
>> ask the users to order the policies and Ranger follow the order.
>>
>> We can open a separate thread on whether Ranger needs to use an explicit
>> hierarchy for policy evaluation or not. For the current discussion, I don't
>> see a specific use case where we would need a deny or allow exception and
>> not fulfilled by just allow or deny functionality.
>>
>> Let us take the use case we discussed earlier. The group "interns" got
>> denied,but we want to exclude one user ("Scott") from that, I am suggesting
>> to have the particular user ("Scott") included in the Allow section of
>> policy and group "interns" included in Deny section of the same policy.
>> Ranger should evaluate the both Allow and Deny in the same policy and
>> deduce that user "Scott" can access the resource while anyone else in group
>> "interns" is denied to access the resource. The policy creation in the UI
>> is much simpler than having to specify "Scott" as an exception to Deny and
>> then specifying "Scott" for Allow.
>>
>> On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
>> mneethiraj@hortonworks.com> wrote:
>>
>>> Microsoft DACL also talks about exceptions, but the model relies on the
>>> users to setup ACE in specific order – see below.
>>>
>>> From
>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
>>> :
>>>
>>> "In most cases, you can control access to an object by using
>>> access-allowed ACEs; you do not need to explicitly deny access to an
>>> object. The exception is when an ACE allows access to a group and you want
>>> to deny access to a member of the group. To do this, place an access-denied
>>> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
>>> Note that the order of the ACEs
>>> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
>>> important because the system reads the ACEs in sequence until access is
>>> granted or denied. The user's access-denied ACE must appear first;
>>> otherwise, when the system reads the group's access allowed ACE, it will
>>> grant access to the restricted user."
>>>
>>> This approach of relying on ordering of policy items (equivalent of ACEs
>>> above) can work for Ranger as well. However Microsoft DACL model evaluates
>>> only one DACL to determine access to an object; compare this with Ranger
>>> where there could be multiple policies that can be evaluated for a resource
>>> – for example to determine access to a finance.invoice.ssn column, Ranger
>>> would evaluate policies that are applicable for finance database,
>>> finance.invoice table and finance.invoice.ssn column. To guarantee
>>> consistent authorization result, that relies on proper ordering of
>>> allow/deny policy items in each policy, would require the policies to be
>>> evaluated in a specific order – either most-specific-to-most-generic
>>> (column policy, table policy, database policy)  or most-generic-to
>>> most-specific. And it might be very challenging, if not impossible, to
>>> ensure a policy order given our support for wildcard specification – for
>>> example, should a policy for “/finance/*2015/*” be evaluated before a
>>> policy for “/finance/invoice*/*” while determining access for a file named
>>> “/finance/invoice2015/vendor1.txt”?
>>>
>>> I think this approach would make the policy authoring very difficult and
>>> confusing, in addition to being not able to support certain scenarios like
>>> “deny at a higher level (like for a database), even if access might be
>>> allowed at a lower level (for a table/column)”, “allow at a higher level,
>>> but deny at a lower level). The current implementation (in tag-policy
>>> branch of Ranger) is much less confusing and offers building blocks that
>>> can be used to support more usecases and can guarantee consistent results
>>> for a given set of policies. If we are after simplicity, the DACL model
>>> does not seem to be the right choice..
>>>
>>> Thanks,
>>> Madhan
>>>
>>>
>>> From: Balaji Ganesan
>>> Reply-To: "user@ranger.incubator.apache.org"
>>> Date: Wednesday, October 14, 2015 at 10:31 AM
>>>
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> >> My proposal is to just keep the policy to have only allow and deny
>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>> "intern" group, then anyone in intern group would not be allowed while
>>> everyone else in finance group will get access. If there is a person in
>>> intern group who needs access, then user need to be taken out of the intern
>>> group or we need to add only the users who specifically need to be denied.
>>>
>>> <<Users who are not comfortable with the idea of “excludes” can continue
>>> to use only allow and deny in the policies. Users who are comfortable with
>>> “excludes” can choose to use it to simplify their policy management.>>
>>>
>>> The question here is whether exceptions are really needed in the
>>> product. We need to keep the product simple
>>> Questions is what use cases can exceptions clearly solve above and
>>> beyond simply access grant and access deny policy line items. As an
>>> example, I am looking at MSFT for how they have have implemented
>>> deny-access
>>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
>>> The use case MSFT is trying to solve it denying access to specific users
>>> who are part of the group which is provided access. I think most of the
>>> deny-access use cases would be for specific users, not for large sets of
>>> groups.
>>>
>>>
>>> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
>>> mneethiraj@hortonworks.com> wrote:
>>>
>>>> >> For argument sakes, finance could be the higher level group and
>>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>>> add all groups which needs access or specify deny groups which don't need
>>>> access
>>>>
>>>> The groups may not fall into nice hierarchies – for example, interns
>>>> group might consist of users from various orgs in a company (not just from
>>>> finance group). In such cases, the only choice is to setup Ranger policies
>>>> with individual users (and not groups).
>>>>
>>>> >> My proposal is to just keep the policy to have only allow and deny
>>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>>> "intern" group, then anyone in intern group would not be allowed while
>>>> everyone else in finance group will get access. If there is a person in
>>>> intern group who needs access, then user need to be taken out of the intern
>>>> group or we need to add only the users who specifically need to be denied.
>>>>
>>>> Users who are not comfortable with the idea of “excludes” can continue
>>>> to use only allow and deny in the policies. Users who are comfortable with
>>>> “excludes” can choose to use it to simplify their policy management.
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>> From: Balaji Ganesan
>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>>>
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> <<Are you suggesting that the security admins create policies that
>>>> list individual users, instead of using groups? Wouldn’t that make security
>>>> administration more painful, in a reasonably large organization? For every
>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>> users and groups to alleviate this issue.>>
>>>> For argument sakes, finance could be the higher level group and intern
>>>> could be a sub group in a hierarchy. There could be n number of sub groups.
>>>> We could provide access by adding n-1 groups to the policy. We can argue
>>>> whether n<10 or n>50,  depending on the answer it would make sense to add
>>>> all groups which needs access or specify deny groups which don't need access
>>>>
>>>> My proposal is to just keep the policy to have only allow and deny with
>>>> NO exceptions. In your use case, if we allow "finance" group and deny
>>>> "intern" group, then anyone in intern group would not be allowed while
>>>> everyone else in finance group will get access. If there is a person in
>>>> intern group who needs access, then user need to be taken out of the intern
>>>> group or we need to add only the users who specifically need to be denied.
>>>>
>>>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Balaji,
>>>>>
>>>>> >> why cannot user clearly specific users or groups that would need
>>>>> specific access. Why is there a need to give access to finance group as a
>>>>> whole when we know there is a subset of user who do not need access to the
>>>>> the finance database? or clearly specify users who need to be denied?
>>>>> Are you suggesting that the security admins create policies that list
>>>>> individual users, instead of using groups? Wouldn’t that make security
>>>>> administration more painful, in a reasonably large organization? For every
>>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>>> users and groups to alleviate this issue.
>>>>>
>>>>>
>>>>> From: Balaji Ganesan
>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>>>
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>>>
>>>>> We clearly need to show case use cases for deny exclude and allow
>>>>> exclude. In my opinion it is very confusing to user to construct such a
>>>>> policy
>>>>>
>>>>> <<Let’s say one of the users, scott who is in interns and finance
>>>>> groups, works on an assignment that requires select access to finance
>>>>> database. To enable this access, the authorization policy for the database
>>>>> should be updated by adding a deny-exclude, as shown below:>>
>>>>> In the wiki, we have created a "deny" policy for intern group and an
>>>>> exception for Scott. First of all, why cannot user clearly specific users
>>>>> or groups that would need specific access. Why is there a need to give
>>>>> access to finance group as a whole when we know there is a subset of user
>>>>> who do not need access to the the finance database? or clearly specify
>>>>> users who need to be denied? It is confusing to think about exceptions for
>>>>> deny, deny, exceptions for allow and allow.
>>>>>
>>>>>
>>>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Bosco,
>>>>>>
>>>>>> Thanks for the review and comments. The wiki
>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>>>> has been updated to address your comments. Please review.
>>>>>>
>>>>>> Thanks,
>>>>>> Madhan
>>>>>>
>>>>>> From: Don Bosco Durai
>>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>>> deny-conditions and exceptions
>>>>>>
>>>>>> Madhan, thanks for putting this document together. It is looking good.
>>>>>>
>>>>>> Can I make a few suggestions:
>>>>>>
>>>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for
>>>>>>    "HDFS policy that allows all finance group users to access contents of
>>>>>>    /finance folder, but denies access to users in interns group. Users in
>>>>>>    interns group will be denied the access even if they are part of finance
>>>>>>    group.”
>>>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny
>>>>>>    all users from “interns” group from accessing table “Employees"
>>>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>>>    good. I think, this is sort of a new concept.
>>>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart.
>>>>>>    Can we create flow chart diagram. It will be easy to understand
>>>>>>
>>>>>> Thanks again. Let me know if you need help in the documentation.
>>>>>>
>>>>>> Bosco
>>>>>>
>>>>>>
>>>>>> From: Madhan Neethiraj
>>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>>>> To: "user@ranger.incubator.apache.org"
>>>>>> Subject: [DISCUSS] Policy model enhancement to support
>>>>>> deny-conditions and exceptions
>>>>>>
>>>>>> All,
>>>>>>
>>>>>> Apache Ranger policy model enhancement to support deny-conditions and
>>>>>> exceptions (RANGER-606
>>>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy
>>>>>> branch <https://github.com/apache/incubator-ranger/tree/tag-policy>.
>>>>>> This enhancement adds the capability to explicitly deny access to resources
>>>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>>>
>>>>>> An overview of the implementation, along with few examples is
>>>>>> available in Apache wiki page here
>>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>>>> Please review.
>>>>>>
>>>>>> Thanks,
>>>>>> Madhan
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

>That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy. 

I am not sure how tag based policy will simplify. I feel, “Deny” is advanced feature and I would expect the Admin to be smart enough to know what he/she is doing. Maybe in the future we can give more tools to help the Admins.

>Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication. 
Based on the Madhan’s examples, yes it is needed. Without it, there are few use cases we can’t meet. We can use a convoluted to meet these requirement, but I feel, “Exceptions/Exclude” might be a simpler option.

>I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on. 
I agree with this. We can make it optional and have an UI setting to enable it. Generally, users won’t know this feature exists. I would put a band at the both of the policy edit page and put a link with “Deny feature is disabled. Click here to enable it”.

Thanks

Bosco


From:  Balaji Ganesan
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 10:54 PM
To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy. 

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication. 

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on. 

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:
When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
We have to document the flow. And I think, Madhan’s flow is simple. 
First Deny Policies, except those who are in exception list
Then Allow Policies, except those who are in the exception list
Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.
The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From:  Balaji Ganesan
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 10:32 AM

To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality. 

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple 
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups. 


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 10:24 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 9:01 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

Abhay, would you be able to give an example for these? It will be easy to understand.

Thanks

Bosco


From:  Abhay Kulkarni
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Friday, October 16, 2015 at 10:21 AM
To:  "user@ranger.incubator.apache.org"
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

As I understand it, Ranger considers a policy for a user and a policy for a group of which a user is a member exactly the same for allow/deny decision. If Ranger can treat a policy written explicitly for a specific user as overriding another written (implicitly) for the user (through group membership), then perhaps it will be easier for administrators to reason about the evaluation. Essentially, in such design, allow/deny exceptions become implicitly (by Ranger Policy Evaluation) rather than explicitly (by administrator) defined.

-Abhay

From: Balaji Ganesan <ba...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, October 15, 2015 at 10:54 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy. 

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication. 

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on. 

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:
When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
We have to document the flow. And I think, Madhan’s flow is simple. 
First Deny Policies, except those who are in exception list
Then Allow Policies, except those who are in the exception list
Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.
The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>
Date: Thursday, October 15, 2015 at 10:32 AM 

To: <us...@ranger.incubator.apache.org>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order. 

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality. 

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Wednesday, October 14, 2015 at 10:31 AM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple 
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups. 


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 10:24 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 9:01 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Abhay Kulkarni <ak...@hortonworks.com>]

As I understand it, Ranger considers a policy for a user and a policy for a group of which a user is a member exactly the same for allow/deny decision. If Ranger can treat a policy written explicitly for a specific user as overriding another written (implicitly) for the user (through group membership), then perhaps it will be easier for administrators to reason about the evaluation. Essentially, in such design, allow/deny exceptions become implicitly (by Ranger Policy Evaluation) rather than explicitly (by administrator) defined.

-Abhay

From: Balaji Ganesan <ba...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:54 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.>>

That is the concern here. How can we expect Administrators to ensure they are keeping policies in sync between high level and lower level resources? The use case is Scott and finance group should get access while everyone else in intern group should. The intern group is part of the finance group. We can expect the administrators to achieve this by creating a higher level tag based policy and a resource based policy.

<<When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around the need to have exceptions for deny and allow. If it indeed going to be complicated, then should we not such areas which add more complication.

<<The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional. The exceptions should be turned off by default and should not appear in the UI. Users who really need exceptions can turn it on.

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org>> wrote:
>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:

  1.  When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
  2.  We have to document the flow. And I think, Madhan’s flow is simple.
     *   First Deny Policies, except those who are in exception list
     *   Then Allow Policies, except those who are in the exception list
  3.  Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
  4.  If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.

The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From: Balaji Ganesan
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, October 15, 2015 at 10:32 AM

To: <us...@ranger.incubator.apache.org>>
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality.

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow.

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs<https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups.


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 10:24 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 9:01 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Bosco,

Thanks for the review and comments. The wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies> has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:

  1.  Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
  2.  Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
  3.  The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
  4.  Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
  5.  Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand

Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

<<I think, Madhan’s use case is not exactly the same. In his case, he wants
to “Deny” all interns at a high level, but not necessary want to Allow
(globally) “interns” in this policy (which could be bad in some cases).
Scott’s permissions to any sub resources might be managed by other
policies.>>

That is the concern here. How can we expect Administrators to ensure they
are keeping policies in sync between high level and lower level resources?
The use case is Scott and finance group should get access while everyone
else in intern group should. The intern group is part of the finance group.
We can expect the administrators to achieve this by creating a higher level
tag based policy and a resource based policy.

<<When we decided to support “Deny”, we know it would add complications. We
have to live it and doing partial work (because of UI) is not a good idea.>>

Not sure if we are discussing the same thing here. My question was around
the need to have exceptions for deny and allow. If it indeed going to be
complicated, then should we not such areas which add more complication.

<<The good thing about Madhan’s design is, it is full backward
compatibility and it is optional to use deny and even exception. And it
provides a lot of flexibility for expert users to model their policies.>>

I think we should provide both deny and exceptions policies as optional.
The exceptions should be turned off by default and should not appear in the
UI. Users who really need exceptions can turn it on.

On Thu, Oct 15, 2015 at 11:35 AM, Don Bosco Durai <bo...@apache.org> wrote:

> >Allow and Deny in the same policy and deduce that user "Scott" can access
> the resource while anyone else in group "interns" is denied to access the
> resource. The policy creation in the UI is much simpler than having to
> specify "Scott" as an exception to Deny and then specifying "Scott" for
> Allow.
>
> I think, Madhan’s use case is not exactly the same. In his case, he wants
> to “Deny” all interns at a high level, but not necessary want to Allow
> (globally) “interns” in this policy (which could be bad in some cases).
> Scott’s permissions to any sub resources might be managed by other policies.
>
> The main thing here is:
>
>    1. When we decided to support “Deny”, we know it would add
>    complications. We have to live it and doing partial work (because of UI) is
>    not a good idea.
>    2. We have to document the flow. And I think, Madhan’s flow is simple.
>       1. First Deny Policies, except those who are in exception list
>       2. Then Allow Policies, except those who are in the exception list
>    3. Finally, you are correct. For most users and use cases, you can
>    model the policies with our existing ones. So very few will use “Deny” and
>    those would use it, would have to understand how it works. And I don’t
>    think it is so complex, but they just need know the about 2 rules
>    4. If we feel the UI is confusing it, then we need to fix the UI. But
>    I feel, compared some of the other system which support “Deny” and
>    hierarchy, ours is pretty elegant and scalable solution.
>
> The good thing about Madhan’s design is, it is full backward compatibility
> and it is optional to use deny and even exception. And it provides a lot of
> flexibility for expert users to model their policies.
>
> Bosco
>
>
> From: Balaji Ganesan
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Thursday, October 15, 2015 at 10:32 AM
>
> To: <us...@ranger.incubator.apache.org>
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> I think we completely missed my earlier point. My question was around
> whether we need to have exceptions in Deny and Allow beyond just Allow or
> Deny policy items. I used the MSFT example to see if they have implemented
> exceptions to Deny, which I did not find any. I am not suggesting that we
> ask the users to order the policies and Ranger follow the order.
>
> We can open a separate thread on whether Ranger needs to use an explicit
> hierarchy for policy evaluation or not. For the current discussion, I don't
> see a specific use case where we would need a deny or allow exception and
> not fulfilled by just allow or deny functionality.
>
> Let us take the use case we discussed earlier. The group "interns" got
> denied,but we want to exclude one user ("Scott") from that, I am suggesting
> to have the particular user ("Scott") included in the Allow section of
> policy and group "interns" included in Deny section of the same policy.
> Ranger should evaluate the both Allow and Deny in the same policy and
> deduce that user "Scott" can access the resource while anyone else in group
> "interns" is denied to access the resource. The policy creation in the UI
> is much simpler than having to specify "Scott" as an exception to Deny and
> then specifying "Scott" for Allow.
>
> On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
> mneethiraj@hortonworks.com> wrote:
>
>> Microsoft DACL also talks about exceptions, but the model relies on the
>> users to setup ACE in specific order – see below.
>>
>> From
>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
>> :
>>
>> "In most cases, you can control access to an object by using
>> access-allowed ACEs; you do not need to explicitly deny access to an
>> object. The exception is when an ACE allows access to a group and you want
>> to deny access to a member of the group. To do this, place an access-denied
>> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
>> Note that the order of the ACEs
>> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
>> important because the system reads the ACEs in sequence until access is
>> granted or denied. The user's access-denied ACE must appear first;
>> otherwise, when the system reads the group's access allowed ACE, it will
>> grant access to the restricted user."
>>
>> This approach of relying on ordering of policy items (equivalent of ACEs
>> above) can work for Ranger as well. However Microsoft DACL model evaluates
>> only one DACL to determine access to an object; compare this with Ranger
>> where there could be multiple policies that can be evaluated for a resource
>> – for example to determine access to a finance.invoice.ssn column, Ranger
>> would evaluate policies that are applicable for finance database,
>> finance.invoice table and finance.invoice.ssn column. To guarantee
>> consistent authorization result, that relies on proper ordering of
>> allow/deny policy items in each policy, would require the policies to be
>> evaluated in a specific order – either most-specific-to-most-generic
>> (column policy, table policy, database policy)  or most-generic-to
>> most-specific. And it might be very challenging, if not impossible, to
>> ensure a policy order given our support for wildcard specification – for
>> example, should a policy for “/finance/*2015/*” be evaluated before a
>> policy for “/finance/invoice*/*” while determining access for a file named
>> “/finance/invoice2015/vendor1.txt”?
>>
>> I think this approach would make the policy authoring very difficult and
>> confusing, in addition to being not able to support certain scenarios like
>> “deny at a higher level (like for a database), even if access might be
>> allowed at a lower level (for a table/column)”, “allow at a higher level,
>> but deny at a lower level). The current implementation (in tag-policy
>> branch of Ranger) is much less confusing and offers building blocks that
>> can be used to support more usecases and can guarantee consistent results
>> for a given set of policies. If we are after simplicity, the DACL model
>> does not seem to be the right choice..
>>
>> Thanks,
>> Madhan
>>
>>
>> From: Balaji Ganesan
>> Reply-To: "user@ranger.incubator.apache.org"
>> Date: Wednesday, October 14, 2015 at 10:31 AM
>>
>> To: "user@ranger.incubator.apache.org"
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> >> My proposal is to just keep the policy to have only allow and deny
>> with NO exceptions. In your use case, if we allow "finance" group and deny
>> "intern" group, then anyone in intern group would not be allowed while
>> everyone else in finance group will get access. If there is a person in
>> intern group who needs access, then user need to be taken out of the intern
>> group or we need to add only the users who specifically need to be denied.
>>
>> <<Users who are not comfortable with the idea of “excludes” can continue
>> to use only allow and deny in the policies. Users who are comfortable with
>> “excludes” can choose to use it to simplify their policy management.>>
>>
>> The question here is whether exceptions are really needed in the product.
>> We need to keep the product simple
>> Questions is what use cases can exceptions clearly solve above and beyond
>> simply access grant and access deny policy line items. As an example, I am
>> looking at MSFT for how they have have implemented deny-access
>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
>> The use case MSFT is trying to solve it denying access to specific users
>> who are part of the group which is provided access. I think most of the
>> deny-access use cases would be for specific users, not for large sets of
>> groups.
>>
>>
>> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
>> mneethiraj@hortonworks.com> wrote:
>>
>>> >> For argument sakes, finance could be the higher level group and
>>> intern could be a sub group in a hierarchy. There could be n number of sub
>>> groups. We could provide access by adding n-1 groups to the policy. We can
>>> argue whether n<10 or n>50,  depending on the answer it would make sense to
>>> add all groups which needs access or specify deny groups which don't need
>>> access
>>>
>>> The groups may not fall into nice hierarchies – for example, interns
>>> group might consist of users from various orgs in a company (not just from
>>> finance group). In such cases, the only choice is to setup Ranger policies
>>> with individual users (and not groups).
>>>
>>> >> My proposal is to just keep the policy to have only allow and deny
>>> with NO exceptions. In your use case, if we allow "finance" group and deny
>>> "intern" group, then anyone in intern group would not be allowed while
>>> everyone else in finance group will get access. If there is a person in
>>> intern group who needs access, then user need to be taken out of the intern
>>> group or we need to add only the users who specifically need to be denied.
>>>
>>> Users who are not comfortable with the idea of “excludes” can continue
>>> to use only allow and deny in the policies. Users who are comfortable with
>>> “excludes” can choose to use it to simplify their policy management.
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Balaji Ganesan
>>> Reply-To: "user@ranger.incubator.apache.org"
>>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>>
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> <<Are you suggesting that the security admins create policies that list
>>> individual users, instead of using groups? Wouldn’t that make security
>>> administration more painful, in a reasonably large organization? For every
>>> change in users role (or group) or an employee joining/leaving the org, all
>>> security policies have to be reviewed and updated..  Many ACLs support both
>>> users and groups to alleviate this issue.>>
>>> For argument sakes, finance could be the higher level group and intern
>>> could be a sub group in a hierarchy. There could be n number of sub groups.
>>> We could provide access by adding n-1 groups to the policy. We can argue
>>> whether n<10 or n>50,  depending on the answer it would make sense to add
>>> all groups which needs access or specify deny groups which don't need access
>>>
>>> My proposal is to just keep the policy to have only allow and deny with
>>> NO exceptions. In your use case, if we allow "finance" group and deny
>>> "intern" group, then anyone in intern group would not be allowed while
>>> everyone else in finance group will get access. If there is a person in
>>> intern group who needs access, then user need to be taken out of the intern
>>> group or we need to add only the users who specifically need to be denied.
>>>
>>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Balaji,
>>>>
>>>> >> why cannot user clearly specific users or groups that would need
>>>> specific access. Why is there a need to give access to finance group as a
>>>> whole when we know there is a subset of user who do not need access to the
>>>> the finance database? or clearly specify users who need to be denied?
>>>> Are you suggesting that the security admins create policies that list
>>>> individual users, instead of using groups? Wouldn’t that make security
>>>> administration more painful, in a reasonably large organization? For every
>>>> change in users role (or group) or an employee joining/leaving the org, all
>>>> security policies have to be reviewed and updated..  Many ACLs support both
>>>> users and groups to alleviate this issue.
>>>>
>>>>
>>>> From: Balaji Ganesan
>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>>
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>>
>>>> We clearly need to show case use cases for deny exclude and allow
>>>> exclude. In my opinion it is very confusing to user to construct such a
>>>> policy
>>>>
>>>> <<Let’s say one of the users, scott who is in interns and finance
>>>> groups, works on an assignment that requires select access to finance
>>>> database. To enable this access, the authorization policy for the database
>>>> should be updated by adding a deny-exclude, as shown below:>>
>>>> In the wiki, we have created a "deny" policy for intern group and an
>>>> exception for Scott. First of all, why cannot user clearly specific users
>>>> or groups that would need specific access. Why is there a need to give
>>>> access to finance group as a whole when we know there is a subset of user
>>>> who do not need access to the the finance database? or clearly specify
>>>> users who need to be denied? It is confusing to think about exceptions for
>>>> deny, deny, exceptions for allow and allow.
>>>>
>>>>
>>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Bosco,
>>>>>
>>>>> Thanks for the review and comments. The wiki
>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>>> has been updated to address your comments. Please review.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>> From: Don Bosco Durai
>>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> Madhan, thanks for putting this document together. It is looking good.
>>>>>
>>>>> Can I make a few suggestions:
>>>>>
>>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for
>>>>>    "HDFS policy that allows all finance group users to access contents of
>>>>>    /finance folder, but denies access to users in interns group. Users in
>>>>>    interns group will be denied the access even if they are part of finance
>>>>>    group.”
>>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny
>>>>>    all users from “interns” group from accessing table “Employees"
>>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>>    good. I think, this is sort of a new concept.
>>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can
>>>>>    we create flow chart diagram. It will be easy to understand
>>>>>
>>>>> Thanks again. Let me know if you need help in the documentation.
>>>>>
>>>>> Bosco
>>>>>
>>>>>
>>>>> From: Madhan Neethiraj
>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>>> To: "user@ranger.incubator.apache.org"
>>>>> Subject: [DISCUSS] Policy model enhancement to support
>>>>> deny-conditions and exceptions
>>>>>
>>>>> All,
>>>>>
>>>>> Apache Ranger policy model enhancement to support deny-conditions and
>>>>> exceptions (RANGER-606
>>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy
>>>>> branch <https://github.com/apache/incubator-ranger/tree/tag-policy>.
>>>>> This enhancement adds the capability to explicitly deny access to resources
>>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>>
>>>>> An overview of the implementation, along with few examples is
>>>>> available in Apache wiki page here
>>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>>> Please review.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

>Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

I think, Madhan’s use case is not exactly the same. In his case, he wants to “Deny” all interns at a high level, but not necessary want to Allow (globally) “interns” in this policy (which could be bad in some cases). Scott’s permissions to any sub resources might be managed by other policies.

The main thing here is:
When we decided to support “Deny”, we know it would add complications. We have to live it and doing partial work (because of UI) is not a good idea.
We have to document the flow. And I think, Madhan’s flow is simple. 
First Deny Policies, except those who are in exception list
Then Allow Policies, except those who are in the exception list
Finally, you are correct. For most users and use cases, you can model the policies with our existing ones. So very few will use “Deny” and those would use it, would have to understand how it works. And I don’t think it is so complex, but they just need know the about 2 rules
If we feel the UI is confusing it, then we need to fix the UI. But I feel, compared some of the other system which support “Deny” and hierarchy, ours is pretty elegant and scalable solution.
The good thing about Madhan’s design is, it is full backward compatibility and it is optional to use deny and even exception. And it provides a lot of flexibility for expert users to model their policies.

Bosco


From:  Balaji Ganesan
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Thursday, October 15, 2015 at 10:32 AM
To:  <us...@ranger.incubator.apache.org>
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

I think we completely missed my earlier point. My question was around whether we need to have exceptions in Deny and Allow beyond just Allow or Deny policy items. I used the MSFT example to see if they have implemented exceptions to Deny, which I did not find any. I am not suggesting that we ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit hierarchy for policy evaluation or not. For the current discussion, I don't see a specific use case where we would need a deny or allow exception and not fulfilled by just allow or deny functionality. 

Let us take the use case we discussed earlier. The group "interns" got denied,but we want to exclude one user ("Scott") from that, I am suggesting to have the particular user ("Scott") included in the Allow section of policy and group "interns" included in Deny section of the same policy. Ranger should evaluate the both Allow and Deny in the same policy and deduce that user "Scott" can access the resource while anyone else in group "interns" is denied to access the resource. The policy creation in the UI is much simpler than having to specify "Scott" as an exception to Deny and then specifying "Scott" for Allow. 

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

>From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Wednesday, October 14, 2015 at 10:31 AM

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple 
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups. 


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 10:24 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied. 

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org"
Date: Tuesday, October 13, 2015 at 9:01 PM 

To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

I think we completely missed my earlier point. My question was around
whether we need to have exceptions in Deny and Allow beyond just Allow or
Deny policy items. I used the MSFT example to see if they have implemented
exceptions to Deny, which I did not find any. I am not suggesting that we
ask the users to order the policies and Ranger follow the order.

We can open a separate thread on whether Ranger needs to use an explicit
hierarchy for policy evaluation or not. For the current discussion, I don't
see a specific use case where we would need a deny or allow exception and
not fulfilled by just allow or deny functionality.

Let us take the use case we discussed earlier. The group "interns" got
denied,but we want to exclude one user ("Scott") from that, I am suggesting
to have the particular user ("Scott") included in the Allow section of
policy and group "interns" included in Deny section of the same policy.
Ranger should evaluate the both Allow and Deny in the same policy and
deduce that user "Scott" can access the resource while anyone else in group
"interns" is denied to access the resource. The policy creation in the UI
is much simpler than having to specify "Scott" as an exception to Deny and
then specifying "Scott" for Allow.

On Wed, Oct 14, 2015 at 11:36 AM, Madhan Neethiraj <
mneethiraj@hortonworks.com> wrote:

> Microsoft DACL also talks about exceptions, but the model relies on the
> users to setup ACE in specific order – see below.
>
> From
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx
> :
>
> "In most cases, you can control access to an object by using
> access-allowed ACEs; you do not need to explicitly deny access to an
> object. The exception is when an ACE allows access to a group and you want
> to deny access to a member of the group. To do this, place an access-denied
> ACE for the user in the DACL ahead of the access-allowed ACE for the group.
> Note that the order of the ACEs
> <https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is
> important because the system reads the ACEs in sequence until access is
> granted or denied. The user's access-denied ACE must appear first;
> otherwise, when the system reads the group's access allowed ACE, it will
> grant access to the restricted user."
>
> This approach of relying on ordering of policy items (equivalent of ACEs
> above) can work for Ranger as well. However Microsoft DACL model evaluates
> only one DACL to determine access to an object; compare this with Ranger
> where there could be multiple policies that can be evaluated for a resource
> – for example to determine access to a finance.invoice.ssn column, Ranger
> would evaluate policies that are applicable for finance database,
> finance.invoice table and finance.invoice.ssn column. To guarantee
> consistent authorization result, that relies on proper ordering of
> allow/deny policy items in each policy, would require the policies to be
> evaluated in a specific order – either most-specific-to-most-generic
> (column policy, table policy, database policy)  or most-generic-to
> most-specific. And it might be very challenging, if not impossible, to
> ensure a policy order given our support for wildcard specification – for
> example, should a policy for “/finance/*2015/*” be evaluated before a
> policy for “/finance/invoice*/*” while determining access for a file named
> “/finance/invoice2015/vendor1.txt”?
>
> I think this approach would make the policy authoring very difficult and
> confusing, in addition to being not able to support certain scenarios like
> “deny at a higher level (like for a database), even if access might be
> allowed at a lower level (for a table/column)”, “allow at a higher level,
> but deny at a lower level). The current implementation (in tag-policy
> branch of Ranger) is much less confusing and offers building blocks that
> can be used to support more usecases and can guarantee consistent results
> for a given set of policies. If we are after simplicity, the DACL model
> does not seem to be the right choice..
>
> Thanks,
> Madhan
>
>
> From: Balaji Ganesan
> Reply-To: "user@ranger.incubator.apache.org"
> Date: Wednesday, October 14, 2015 at 10:31 AM
>
> To: "user@ranger.incubator.apache.org"
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> >> My proposal is to just keep the policy to have only allow and deny with
> NO exceptions. In your use case, if we allow "finance" group and deny
> "intern" group, then anyone in intern group would not be allowed while
> everyone else in finance group will get access. If there is a person in
> intern group who needs access, then user need to be taken out of the intern
> group or we need to add only the users who specifically need to be denied.
>
> <<Users who are not comfortable with the idea of “excludes” can continue
> to use only allow and deny in the policies. Users who are comfortable with
> “excludes” can choose to use it to simplify their policy management.>>
>
> The question here is whether exceptions are really needed in the product.
> We need to keep the product simple
> Questions is what use cases can exceptions clearly solve above and beyond
> simply access grant and access deny policy line items. As an example, I am
> looking at MSFT for how they have have implemented deny-access
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
> The use case MSFT is trying to solve it denying access to specific users
> who are part of the group which is provided access. I think most of the
> deny-access use cases would be for specific users, not for large sets of
> groups.
>
>
> On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
> mneethiraj@hortonworks.com> wrote:
>
>> >> For argument sakes, finance could be the higher level group and intern
>> could be a sub group in a hierarchy. There could be n number of sub groups.
>> We could provide access by adding n-1 groups to the policy. We can argue
>> whether n<10 or n>50,  depending on the answer it would make sense to add
>> all groups which needs access or specify deny groups which don't need access
>>
>> The groups may not fall into nice hierarchies – for example, interns
>> group might consist of users from various orgs in a company (not just from
>> finance group). In such cases, the only choice is to setup Ranger policies
>> with individual users (and not groups).
>>
>> >> My proposal is to just keep the policy to have only allow and deny
>> with NO exceptions. In your use case, if we allow "finance" group and deny
>> "intern" group, then anyone in intern group would not be allowed while
>> everyone else in finance group will get access. If there is a person in
>> intern group who needs access, then user need to be taken out of the intern
>> group or we need to add only the users who specifically need to be denied.
>>
>> Users who are not comfortable with the idea of “excludes” can continue to
>> use only allow and deny in the policies. Users who are comfortable with
>> “excludes” can choose to use it to simplify their policy management.
>>
>> Thanks,
>> Madhan
>>
>> From: Balaji Ganesan
>> Reply-To: "user@ranger.incubator.apache.org"
>> Date: Tuesday, October 13, 2015 at 10:24 PM
>>
>> To: "user@ranger.incubator.apache.org"
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> <<Are you suggesting that the security admins create policies that list
>> individual users, instead of using groups? Wouldn’t that make security
>> administration more painful, in a reasonably large organization? For every
>> change in users role (or group) or an employee joining/leaving the org, all
>> security policies have to be reviewed and updated..  Many ACLs support both
>> users and groups to alleviate this issue.>>
>> For argument sakes, finance could be the higher level group and intern
>> could be a sub group in a hierarchy. There could be n number of sub groups.
>> We could provide access by adding n-1 groups to the policy. We can argue
>> whether n<10 or n>50,  depending on the answer it would make sense to add
>> all groups which needs access or specify deny groups which don't need access
>>
>> My proposal is to just keep the policy to have only allow and deny with
>> NO exceptions. In your use case, if we allow "finance" group and deny
>> "intern" group, then anyone in intern group would not be allowed while
>> everyone else in finance group will get access. If there is a person in
>> intern group who needs access, then user need to be taken out of the intern
>> group or we need to add only the users who specifically need to be denied.
>>
>> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Balaji,
>>>
>>> >> why cannot user clearly specific users or groups that would need
>>> specific access. Why is there a need to give access to finance group as a
>>> whole when we know there is a subset of user who do not need access to the
>>> the finance database? or clearly specify users who need to be denied?
>>> Are you suggesting that the security admins create policies that list
>>> individual users, instead of using groups? Wouldn’t that make security
>>> administration more painful, in a reasonably large organization? For every
>>> change in users role (or group) or an employee joining/leaving the org, all
>>> security policies have to be reviewed and updated..  Many ACLs support both
>>> users and groups to alleviate this issue.
>>>
>>>
>>> From: Balaji Ganesan
>>> Reply-To: "user@ranger.incubator.apache.org"
>>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>>
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>>
>>> We clearly need to show case use cases for deny exclude and allow
>>> exclude. In my opinion it is very confusing to user to construct such a
>>> policy
>>>
>>> <<Let’s say one of the users, scott who is in interns and finance
>>> groups, works on an assignment that requires select access to finance
>>> database. To enable this access, the authorization policy for the database
>>> should be updated by adding a deny-exclude, as shown below:>>
>>> In the wiki, we have created a "deny" policy for intern group and an
>>> exception for Scott. First of all, why cannot user clearly specific users
>>> or groups that would need specific access. Why is there a need to give
>>> access to finance group as a whole when we know there is a subset of user
>>> who do not need access to the the finance database? or clearly specify
>>> users who need to be denied? It is confusing to think about exceptions for
>>> deny, deny, exceptions for allow and allow.
>>>
>>>
>>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Bosco,
>>>>
>>>> Thanks for the review and comments. The wiki
>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>>> has been updated to address your comments. Please review.
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>> From: Don Bosco Durai
>>>> Reply-To: "user@ranger.incubator.apache.org"
>>>> Date: Monday, October 12, 2015 at 6:20 PM
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>>> deny-conditions and exceptions
>>>>
>>>> Madhan, thanks for putting this document together. It is looking good.
>>>>
>>>> Can I make a few suggestions:
>>>>
>>>>    1. Call out each use case as separate section. E.g. 2.2.3 for "HDFS
>>>>    policy that allows all finance group users to access contents of /finance
>>>>    folder, but denies access to users in interns group. Users in interns group
>>>>    will be denied the access even if they are part of finance group.”
>>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny all
>>>>    users from “interns” group from accessing table “Employees"
>>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>>    good. I think, this is sort of a new concept.
>>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can
>>>>    we create flow chart diagram. It will be easy to understand
>>>>
>>>> Thanks again. Let me know if you need help in the documentation.
>>>>
>>>> Bosco
>>>>
>>>>
>>>> From: Madhan Neethiraj
>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>> Date: Monday, October 12, 2015 at 5:46 PM
>>>> To: "user@ranger.incubator.apache.org"
>>>> Subject: [DISCUSS] Policy model enhancement to support deny-conditions
>>>> and exceptions
>>>>
>>>> All,
>>>>
>>>> Apache Ranger policy model enhancement to support deny-conditions and
>>>> exceptions (RANGER-606
>>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy
>>>> branch <https://github.com/apache/incubator-ranger/tree/tag-policy>.
>>>> This enhancement adds the capability to explicitly deny access to resources
>>>> based on users/groups, access-types and custom-conditions. It also supports
>>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>>> but exclude specific users/groups who might be part of the wider groups.
>>>>
>>>> An overview of the implementation, along with few examples is available
>>>> in Apache wiki page here
>>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>>> Please review.
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>>
>>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Madhan Neethiraj <mn...@hortonworks.com>]

Microsoft DACL also talks about exceptions, but the model relies on the users to setup ACE in specific order – see below.

From  https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx:
"In most cases, you can control access to an object by using access-allowed ACEs; you do not need to explicitly deny access to an object. The exception is when an ACE allows access to a group and you want to deny access to a member of the group. To do this, place an access-denied ACE for the user in the DACL ahead of the access-allowed ACE for the group. Note that the order of the ACEs<https://msdn.microsoft.com/en-us/library/windows/desktop/aa379298(v=vs.85).aspx> is important because the system reads the ACEs in sequence until access is granted or denied. The user's access-denied ACE must appear first; otherwise, when the system reads the group's access allowed ACE, it will grant access to the restricted user."

This approach of relying on ordering of policy items (equivalent of ACEs above) can work for Ranger as well. However Microsoft DACL model evaluates only one DACL to determine access to an object; compare this with Ranger where there could be multiple policies that can be evaluated for a resource – for example to determine access to a finance.invoice.ssn column, Ranger would evaluate policies that are applicable for finance database, finance.invoice table and finance.invoice.ssn column. To guarantee consistent authorization result, that relies on proper ordering of allow/deny policy items in each policy, would require the policies to be evaluated in a specific order – either most-specific-to-most-generic (column policy, table policy, database policy)  or most-generic-to most-specific. And it might be very challenging, if not impossible, to ensure a policy order given our support for wildcard specification – for example, should a policy for “/finance/*2015/*” be evaluated before a policy for “/finance/invoice*/*” while determining access for a file named “/finance/invoice2015/vendor1.txt”?

I think this approach would make the policy authoring very difficult and confusing, in addition to being not able to support certain scenarios like “deny at a higher level (like for a database), even if access might be allowed at a lower level (for a table/column)”, “allow at a higher level, but deny at a lower level). The current implementation (in tag-policy branch of Ranger) is much less confusing and offers building blocks that can be used to support more usecases and can guarantee consistent results for a given set of policies. If we are after simplicity, the DACL model does not seem to be the right choice..

Thanks,
Madhan


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Wednesday, October 14, 2015 at 10:31 AM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

<<Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product. We need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond simply access grant and access deny policy line items. As an example, I am looking at MSFT for how they have have implemented deny-access https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx. The use case MSFT is trying to solve it denying access to specific users who are part of the group which is provided access. I think most of the deny-access use cases would be for specific users, not for large sets of groups.


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <mn...@hortonworks.com>> wrote:
>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 10:24 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 9:01 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Bosco,

Thanks for the review and comments. The wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies> has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:

  1.  Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
  2.  Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
  3.  The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
  4.  Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
  5.  Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand

Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

>> My proposal is to just keep the policy to have only allow and deny with
NO exceptions. In your use case, if we allow "finance" group and deny
"intern" group, then anyone in intern group would not be allowed while
everyone else in finance group will get access. If there is a person in
intern group who needs access, then user need to be taken out of the intern
group or we need to add only the users who specifically need to be denied.

<<Users who are not comfortable with the idea of “excludes” can continue to
use only allow and deny in the policies. Users who are comfortable with
“excludes” can choose to use it to simplify their policy management.>>

The question here is whether exceptions are really needed in the product.
We need to keep the product simple
Questions is what use cases can exceptions clearly solve above and beyond
simply access grant and access deny policy line items. As an example, I am
looking at MSFT for how they have have implemented deny-access
https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597(v=vs.85).aspx.
The use case MSFT is trying to solve it denying access to specific users
who are part of the group which is provided access. I think most of the
deny-access use cases would be for specific users, not for large sets of
groups.


On Tue, Oct 13, 2015 at 11:47 PM, Madhan Neethiraj <
mneethiraj@hortonworks.com> wrote:

> >> For argument sakes, finance could be the higher level group and intern
> could be a sub group in a hierarchy. There could be n number of sub groups.
> We could provide access by adding n-1 groups to the policy. We can argue
> whether n<10 or n>50,  depending on the answer it would make sense to add
> all groups which needs access or specify deny groups which don't need access
>
> The groups may not fall into nice hierarchies – for example, interns group
> might consist of users from various orgs in a company (not just from
> finance group). In such cases, the only choice is to setup Ranger policies
> with individual users (and not groups).
>
> >> My proposal is to just keep the policy to have only allow and deny with
> NO exceptions. In your use case, if we allow "finance" group and deny
> "intern" group, then anyone in intern group would not be allowed while
> everyone else in finance group will get access. If there is a person in
> intern group who needs access, then user need to be taken out of the intern
> group or we need to add only the users who specifically need to be denied.
>
> Users who are not comfortable with the idea of “excludes” can continue to
> use only allow and deny in the policies. Users who are comfortable with
> “excludes” can choose to use it to simplify their policy management.
>
> Thanks,
> Madhan
>
> From: Balaji Ganesan
> Reply-To: "user@ranger.incubator.apache.org"
> Date: Tuesday, October 13, 2015 at 10:24 PM
>
> To: "user@ranger.incubator.apache.org"
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> <<Are you suggesting that the security admins create policies that list
> individual users, instead of using groups? Wouldn’t that make security
> administration more painful, in a reasonably large organization? For every
> change in users role (or group) or an employee joining/leaving the org, all
> security policies have to be reviewed and updated..  Many ACLs support both
> users and groups to alleviate this issue.>>
> For argument sakes, finance could be the higher level group and intern
> could be a sub group in a hierarchy. There could be n number of sub groups.
> We could provide access by adding n-1 groups to the policy. We can argue
> whether n<10 or n>50,  depending on the answer it would make sense to add
> all groups which needs access or specify deny groups which don't need access
>
> My proposal is to just keep the policy to have only allow and deny with NO
> exceptions. In your use case, if we allow "finance" group and deny "intern"
> group, then anyone in intern group would not be allowed while everyone else
> in finance group will get access. If there is a person in intern group who
> needs access, then user need to be taken out of the intern group or we need
> to add only the users who specifically need to be denied.
>
> On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Balaji,
>>
>> >> why cannot user clearly specific users or groups that would need
>> specific access. Why is there a need to give access to finance group as a
>> whole when we know there is a subset of user who do not need access to the
>> the finance database? or clearly specify users who need to be denied?
>> Are you suggesting that the security admins create policies that list
>> individual users, instead of using groups? Wouldn’t that make security
>> administration more painful, in a reasonably large organization? For every
>> change in users role (or group) or an employee joining/leaving the org, all
>> security policies have to be reviewed and updated..  Many ACLs support both
>> users and groups to alleviate this issue.
>>
>>
>> From: Balaji Ganesan
>> Reply-To: "user@ranger.incubator.apache.org"
>> Date: Tuesday, October 13, 2015 at 9:01 PM
>>
>> To: "user@ranger.incubator.apache.org"
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> Madhan, Fantastic job in putting together in the wiki. Thank you.
>>
>> We clearly need to show case use cases for deny exclude and allow
>> exclude. In my opinion it is very confusing to user to construct such a
>> policy
>>
>> <<Let’s say one of the users, scott who is in interns and finance groups,
>> works on an assignment that requires select access to finance database. To
>> enable this access, the authorization policy for the database should be
>> updated by adding a deny-exclude, as shown below:>>
>> In the wiki, we have created a "deny" policy for intern group and an
>> exception for Scott. First of all, why cannot user clearly specific users
>> or groups that would need specific access. Why is there a need to give
>> access to finance group as a whole when we know there is a subset of user
>> who do not need access to the the finance database? or clearly specify
>> users who need to be denied? It is confusing to think about exceptions for
>> deny, deny, exceptions for allow and allow.
>>
>>
>> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Bosco,
>>>
>>> Thanks for the review and comments. The wiki
>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>>> has been updated to address your comments. Please review.
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Don Bosco Durai
>>> Reply-To: "user@ranger.incubator.apache.org"
>>> Date: Monday, October 12, 2015 at 6:20 PM
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: Re: [DISCUSS] Policy model enhancement to support
>>> deny-conditions and exceptions
>>>
>>> Madhan, thanks for putting this document together. It is looking good.
>>>
>>> Can I make a few suggestions:
>>>
>>>    1. Call out each use case as separate section. E.g. 2.2.3 for "HDFS
>>>    policy that allows all finance group users to access contents of /finance
>>>    folder, but denies access to users in interns group. Users in interns group
>>>    will be denied the access even if they are part of finance group.”
>>>    2. Can we also add a simple use case of global “Deny”. E.g Deny all
>>>    users from “interns” group from accessing table “Employees"
>>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>>    4. Probably one small paragraph to explain “Exceptions” will be
>>>    good. I think, this is sort of a new concept.
>>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can
>>>    we create flow chart diagram. It will be easy to understand
>>>
>>> Thanks again. Let me know if you need help in the documentation.
>>>
>>> Bosco
>>>
>>>
>>> From: Madhan Neethiraj
>>> Reply-To: <us...@ranger.incubator.apache.org>
>>> Date: Monday, October 12, 2015 at 5:46 PM
>>> To: "user@ranger.incubator.apache.org"
>>> Subject: [DISCUSS] Policy model enhancement to support deny-conditions
>>> and exceptions
>>>
>>> All,
>>>
>>> Apache Ranger policy model enhancement to support deny-conditions and
>>> exceptions (RANGER-606
>>> <https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy
>>> branch <https://github.com/apache/incubator-ranger/tree/tag-policy>.
>>> This enhancement adds the capability to explicitly deny access to resources
>>> based on users/groups, access-types and custom-conditions. It also supports
>>> allow/deny to be specified for a wider group (like employees, public, etc)
>>> but exclude specific users/groups who might be part of the wider groups.
>>>
>>> An overview of the implementation, along with few examples is available
>>> in Apache wiki page here
>>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>>> Please review.
>>>
>>> Thanks,
>>> Madhan
>>>
>>>
>>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Madhan Neethiraj <mn...@hortonworks.com>]

>> For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

The groups may not fall into nice hierarchies – for example, interns group might consist of users from various orgs in a company (not just from finance group). In such cases, the only choice is to setup Ranger policies with individual users (and not groups).

>> My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

Users who are not comfortable with the idea of “excludes” can continue to use only allow and deny in the policies. Users who are comfortable with “excludes” can choose to use it to simplify their policy management.

Thanks,
Madhan

From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 10:24 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

<<Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern could be a sub group in a hierarchy. There could be n number of sub groups. We could provide access by adding n-1 groups to the policy. We can argue whether n<10 or n>50,  depending on the answer it would make sense to add all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO exceptions. In your use case, if we allow "finance" group and deny "intern" group, then anyone in intern group would not be allowed while everyone else in finance group will get access. If there is a person in intern group who needs access, then user need to be taken out of the intern group or we need to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From: Balaji Ganesan
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Tuesday, October 13, 2015 at 9:01 PM

To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Bosco,

Thanks for the review and comments. The wiki<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies> has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:

  1.  Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
  2.  Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
  3.  The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
  4.  Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
  5.  Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand

Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606<https://issues.apache.org/jira/browse/RANGER-606>) is available in  tag-policy branch<https://github.com/apache/incubator-ranger/tree/tag-policy>. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

<<Are you suggesting that the security admins create policies that list
individual users, instead of using groups? Wouldn’t that make security
administration more painful, in a reasonably large organization? For every
change in users role (or group) or an employee joining/leaving the org, all
security policies have to be reviewed and updated..  Many ACLs support both
users and groups to alleviate this issue.>>
For argument sakes, finance could be the higher level group and intern
could be a sub group in a hierarchy. There could be n number of sub groups.
We could provide access by adding n-1 groups to the policy. We can argue
whether n<10 or n>50,  depending on the answer it would make sense to add
all groups which needs access or specify deny groups which don't need access

My proposal is to just keep the policy to have only allow and deny with NO
exceptions. In your use case, if we allow "finance" group and deny "intern"
group, then anyone in intern group would not be allowed while everyone else
in finance group will get access. If there is a person in intern group who
needs access, then user need to be taken out of the intern group or we need
to add only the users who specifically need to be denied.

On Tue, Oct 13, 2015 at 9:48 PM, Madhan Neethiraj <ma...@apache.org> wrote:

> Balaji,
>
> >> why cannot user clearly specific users or groups that would need
> specific access. Why is there a need to give access to finance group as a
> whole when we know there is a subset of user who do not need access to the
> the finance database? or clearly specify users who need to be denied?
> Are you suggesting that the security admins create policies that list
> individual users, instead of using groups? Wouldn’t that make security
> administration more painful, in a reasonably large organization? For every
> change in users role (or group) or an employee joining/leaving the org, all
> security policies have to be reviewed and updated..  Many ACLs support both
> users and groups to alleviate this issue.
>
>
> From: Balaji Ganesan
> Reply-To: "user@ranger.incubator.apache.org"
> Date: Tuesday, October 13, 2015 at 9:01 PM
>
> To: "user@ranger.incubator.apache.org"
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> Madhan, Fantastic job in putting together in the wiki. Thank you.
>
> We clearly need to show case use cases for deny exclude and allow exclude.
> In my opinion it is very confusing to user to construct such a policy
>
> <<Let’s say one of the users, scott who is in interns and finance groups,
> works on an assignment that requires select access to finance database. To
> enable this access, the authorization policy for the database should be
> updated by adding a deny-exclude, as shown below:>>
> In the wiki, we have created a "deny" policy for intern group and an
> exception for Scott. First of all, why cannot user clearly specific users
> or groups that would need specific access. Why is there a need to give
> access to finance group as a whole when we know there is a subset of user
> who do not need access to the the finance database? or clearly specify
> users who need to be denied? It is confusing to think about exceptions for
> deny, deny, exceptions for allow and allow.
>
>
> On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Bosco,
>>
>> Thanks for the review and comments. The wiki
>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
>> has been updated to address your comments. Please review.
>>
>> Thanks,
>> Madhan
>>
>> From: Don Bosco Durai
>> Reply-To: "user@ranger.incubator.apache.org"
>> Date: Monday, October 12, 2015 at 6:20 PM
>> To: "user@ranger.incubator.apache.org"
>> Subject: Re: [DISCUSS] Policy model enhancement to support
>> deny-conditions and exceptions
>>
>> Madhan, thanks for putting this document together. It is looking good.
>>
>> Can I make a few suggestions:
>>
>>    1. Call out each use case as separate section. E.g. 2.2.3 for "HDFS
>>    policy that allows all finance group users to access contents of /finance
>>    folder, but denies access to users in interns group. Users in interns group
>>    will be denied the access even if they are part of finance group.”
>>    2. Can we also add a simple use case of global “Deny”. E.g Deny all
>>    users from “interns” group from accessing table “Employees"
>>    3. The label “Exceptions”, can we make it more explicit. E.g.
>>    “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
>>    4. Probably one small paragraph to explain “Exceptions” will be good.
>>    I think, this is sort of a new concept.
>>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we
>>    create flow chart diagram. It will be easy to understand
>>
>> Thanks again. Let me know if you need help in the documentation.
>>
>> Bosco
>>
>>
>> From: Madhan Neethiraj
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Monday, October 12, 2015 at 5:46 PM
>> To: "user@ranger.incubator.apache.org"
>> Subject: [DISCUSS] Policy model enhancement to support deny-conditions
>> and exceptions
>>
>> All,
>>
>> Apache Ranger policy model enhancement to support deny-conditions and
>> exceptions (RANGER-606 <https://issues.apache.org/jira/browse/RANGER-606>)
>> is available in  tag-policy branch
>> <https://github.com/apache/incubator-ranger/tree/tag-policy>. This
>> enhancement adds the capability to explicitly deny access to resources
>> based on users/groups, access-types and custom-conditions. It also supports
>> allow/deny to be specified for a wider group (like employees, public, etc)
>> but exclude specific users/groups who might be part of the wider groups.
>>
>> An overview of the implementation, along with few examples is available
>> in Apache wiki page here
>> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
>> Please review.
>>
>> Thanks,
>> Madhan
>>
>>
>>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Madhan Neethiraj <ma...@apache.org>]

Balaji,

>> why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied?
Are you suggesting that the security admins create policies that list individual users, instead of using groups? Wouldn’t that make security administration more painful, in a reasonably large organization? For every change in users role (or group) or an employee joining/leaving the org, all security policies have to be reviewed and updated..  Many ACLs support both users and groups to alleviate this issue.


From:  Balaji Ganesan
Reply-To:  "user@ranger.incubator.apache.org"
Date:  Tuesday, October 13, 2015 at 9:01 PM
To:  "user@ranger.incubator.apache.org"
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude. In my opinion it is very confusing to user to construct such a policy 

<<Let’s say one of the users, scott who is in interns and finance groups, works on an assignment that requires select access to finance database. To enable this access, the authorization policy for the database should be updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an exception for Scott. First of all, why cannot user clearly specific users or groups that would need specific access. Why is there a need to give access to finance group as a whole when we know there is a subset of user who do not need access to the the finance database? or clearly specify users who need to be denied? It is confusing to think about exceptions for deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From: Don Bosco Durai
Reply-To: "user@ranger.incubator.apache.org"
Date: Monday, October 12, 2015 at 6:20 PM
To: "user@ranger.incubator.apache.org"
Subject: Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Balaji Ganesan <ba...@gmail.com>]

Madhan, Fantastic job in putting together in the wiki. Thank you.

We clearly need to show case use cases for deny exclude and allow exclude.
In my opinion it is very confusing to user to construct such a policy

<<Let’s say one of the users, scott who is in interns and finance groups,
works on an assignment that requires select access to finance database. To
enable this access, the authorization policy for the database should be
updated by adding a deny-exclude, as shown below:>>
In the wiki, we have created a "deny" policy for intern group and an
exception for Scott. First of all, why cannot user clearly specific users
or groups that would need specific access. Why is there a need to give
access to finance group as a whole when we know there is a subset of user
who do not need access to the the finance database? or clearly specify
users who need to be denied? It is confusing to think about exceptions for
deny, deny, exceptions for allow and allow.


On Tue, Oct 13, 2015 at 2:43 PM, Madhan Neethiraj <ma...@apache.org> wrote:

> Bosco,
>
> Thanks for the review and comments. The wiki
> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>
> has been updated to address your comments. Please review.
>
> Thanks,
> Madhan
>
> From: Don Bosco Durai
> Reply-To: "user@ranger.incubator.apache.org"
> Date: Monday, October 12, 2015 at 6:20 PM
> To: "user@ranger.incubator.apache.org"
> Subject: Re: [DISCUSS] Policy model enhancement to support
> deny-conditions and exceptions
>
> Madhan, thanks for putting this document together. It is looking good.
>
> Can I make a few suggestions:
>
>    1. Call out each use case as separate section. E.g. 2.2.3 for "HDFS
>    policy that allows all finance group users to access contents of /finance
>    folder, but denies access to users in interns group. Users in interns group
>    will be denied the access even if they are part of finance group.”
>    2. Can we also add a simple use case of global “Deny”. E.g Deny all
>    users from “interns” group from accessing table “Employees"
>    3. The label “Exceptions”, can we make it more explicit. E.g. “Exclude
>    from Allow Conditions” and “Exclude from Deny Conditions”
>    4. Probably one small paragraph to explain “Exceptions” will be good.
>    I think, this is sort of a new concept.
>    5. Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we
>    create flow chart diagram. It will be easy to understand
>
> Thanks again. Let me know if you need help in the documentation.
>
> Bosco
>
>
> From: Madhan Neethiraj
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Monday, October 12, 2015 at 5:46 PM
> To: "user@ranger.incubator.apache.org"
> Subject: [DISCUSS] Policy model enhancement to support deny-conditions
> and exceptions
>
> All,
>
> Apache Ranger policy model enhancement to support deny-conditions and
> exceptions (RANGER-606 <https://issues.apache.org/jira/browse/RANGER-606>)
> is available in  tag-policy branch
> <https://github.com/apache/incubator-ranger/tree/tag-policy>. This
> enhancement adds the capability to explicitly deny access to resources
> based on users/groups, access-types and custom-conditions. It also supports
> allow/deny to be specified for a wider group (like employees, public, etc)
> but exclude specific users/groups who might be part of the wider groups.
>
> An overview of the implementation, along with few examples is available in
> Apache wiki page here
> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+exceptions+in+Ranger+policies>.
> Please review.
>
> Thanks,
> Madhan
>
>
>

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Madhan Neethiraj <ma...@apache.org>]

Bosco,

Thanks for the review and comments. The wiki has been updated to address your comments. Please review.

Thanks,
Madhan

From:  Don Bosco Durai
Reply-To:  "user@ranger.incubator.apache.org"
Date:  Monday, October 12, 2015 at 6:20 PM
To:  "user@ranger.incubator.apache.org"
Subject:  Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From: Madhan Neethiraj
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, October 12, 2015 at 5:46 PM
To: "user@ranger.incubator.apache.org"
Subject: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan

Re: [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

[Don Bosco Durai <bo...@apache.org>]

Madhan, thanks for putting this document together. It is looking good.

Can I make a few suggestions:
Call out each use case as separate section. E.g. 2.2.3 for "HDFS policy that allows all finance group users to access contents of /finance folder, but denies access to users in interns group. Users in interns group will be denied the access even if they are part of finance group.”
Can we also add a simple use case of global “Deny”. E.g Deny all users from “interns” group from accessing table “Employees"
The label “Exceptions”, can we make it more explicit. E.g. “Exclude from Allow Conditions” and “Exclude from Deny Conditions”
Probably one small paragraph to explain “Exceptions” will be good. I think, this is sort of a new concept.
Section 3 “Policy Evaluation”, it seems to be a flow chart. Can we create flow chart diagram. It will be easy to understand
Thanks again. Let me know if you need help in the documentation.

Bosco


From:  Madhan Neethiraj
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Monday, October 12, 2015 at 5:46 PM
To:  "user@ranger.incubator.apache.org"
Subject:  [DISCUSS] Policy model enhancement to support deny-conditions and exceptions

All,

Apache Ranger policy model enhancement to support deny-conditions and exceptions (RANGER-606) is available in  tag-policy branch. This enhancement adds the capability to explicitly deny access to resources based on users/groups, access-types and custom-conditions. It also supports allow/deny to be specified for a wider group (like employees, public, etc) but exclude specific users/groups who might be part of the wider groups.

An overview of the implementation, along with few examples is available in Apache wiki page here. Please review.

Thanks,
Madhan