You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Aruna Sivaram <si...@gmail.com> on 2015/12/10 06:45:25 UTC
Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a
custom plugin which we plan to integrate with the ranger framework. This
custom plugin will be used for access control of our components. In order
to explore this possibility, i have written a custom plugin as per the
example given in the link
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class
(CustomServiceAuthoriser), will not be extending any of the hadoop security
classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
have created the repository and policies for the same through the rest web
service.
I have written the custom service authoriser as per the example and
deployed the same on the sandbox. I have a service named CustomService.
What i wanted to know is how the customservice will communicate with my
customserviceauthoriser which is contained in my plugin. Currently i dont
find any documentation which talks about the mode of communication or
rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin
classes would be logging. This will help us in debugging the flow. I see a
lot of log statements in the ranger plugin code base but am unable to find
the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control
of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
RE: Queries on the developement for a new custom plugin
Posted by "veera.valluri@bnpparibasfortis.com" <ve...@bnpparibasfortis.com>.
Well explained, Madhan.
From: Madhan Neethiraj [mailto:mneethiraj@hortonworks.com] On Behalf Of Madhan Neethiraj
Sent: Monday, December 21, 2015 4:27 PM
To: user@ranger.incubator.apache.org
Subject: Re: Queries on the developement for a new custom plugin
Aruna,
>> If user "aruna" does any other action, other than submit-app on the resource "default". Then the audit log will show the result as "denied". In this case, should the policy id which is 656 be shown along with the "denied" result, or the id will be a blank field.
Lets say the action was “write”. In this case, policy #656 would not “deny” the access. The policy simply was not able to make an authorization decision – since the attributes of the request (user, group, access-type) do not match with the policy. In such case, the policy engine will look for the next policy that matches the request. Following scenario should help clarify this.
In addition to 656, consider there is another policy with: “id=657; resource=*; user=john; access-type=write”; here the resource and access-type in the policy match the request, but the username does not. Hence this policy can not make the authorization decision. In this case, should we expect the audit log to contain both 656 and 657 as policyIds? Remember, there could be multiple such policies that do not match the request.
>> Under what cases will the audit log show result as "denied" along with policy id.?
Ranger policy model was recently enhanced to be able to actively deny access – refer the documentation here<https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>. Only when such a policy explicitly denies an access, audit log will record its policyId.
Hope this helps.
Madhan
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Monday, December 21, 2015 at 5:59 AM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
let's say i have a policy with following details
Policy id : 656
Resource name : default
User : aruna
Access type : submit-app
If user "aruna" does any other action, other than submit-app on the resource "default". Then the audit log will show the result as "denied". In this case, should the policy id which is 656 be shown along with the "denied" result, or the id will be a blank field. That's my confusion.
Under what cases will the audit log show result as "denied" along with policy id.?
Thanks
Aruna
On 17 Dec 2015 22:30, "Madhan Neethiraj" <ma...@apache.org>> wrote:
Aruna,
>> However, the policyId still shows blank. Is this the right behavior?
Ranger populates policyId field in audit log only when that policy makes the authorization decision – either allow or deny. In this particular case, no policy explicitly allowed or denied the access. The end result is “deny” - only because there was no policy to allow the access. Hence the policyId is left blank. Hope this helps. To help understand this further, try to answer this question: which policyId should be recorded in the audit log in this case? Please consider that there could be multiple policies applicable for the resource being accessed – like queue=*, queue=test*, queue=t*.
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Thursday, December 17, 2015 at 12:17 AM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
I was able to fix this issue for the 3rd case. Thing is that there was a spelling mismatch between the actionType specified in the servicedef and the actionType that is sent through the request. Now, with the change i am getting the Result as "Denied" instead of "allowed". However, the policyId still shows blank. Is this the right behavior?
I am attaching a screenshot for your reference.
Regards
Aruna
On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <ma...@apache.org>> wrote:
Aruna,
>> 2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns “Denied"
Since there was no policy that allowed the access, the request was denied. Hence the policyId is blank.
>> 3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied”
The audit log clearly says that the access was granted by policyId=656. Can you please send the screenshot of the policy? Another possibility is if your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as shown below); can you please check if this is the case?
"accessTypes":
[
{
"itemId": 1,
"name": "SUBMIT_APP",
"label": “SUBMIT_APP”,
"impliedGrants":
[
"ADMIN_QUEUE"
]
},
{
"itemId": 2,
"name": “ADMIN_QUEUE",
"label": "ADMIN_QUEUE"
}
]
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Monday, December 14, 2015 at 10:47 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
Let assume that I have created a custom policy named TestPolicy, Policy Id : 656 for a user called "network3" giving "submit-app" permissions for a queue called "default".
1st case --> , if the same user "network3" does a "submit-app" on the "default" queue, then the audit log shows the correct result with the Result as "Allowed", the isAccessAllowed method returns "Allowed"
2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns "Denied"
3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied" (I have created a policy giving only "submit-app" permissions on "default" queue to network3 user). In such a scenario, what should be done? Currently my custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed Method.
and following are the values of the RangerAccessRequestImpl object in this scenario.
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
RangerAccessResourceImpl resource = new RangerAccessResourceImpl();
resource.setValue("queue", default);
request.setResource(resource);
request.setAccessType("admin-queue");
request.setUser("network3");
request.setAccessTime(new Date());
request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
request.setAction("admin-queue");
request.setClientIPAddress("10.0.2.15");
RangerAccessResult result = plugin.isAccessAllowed(request);
return result == null ? false : result.getIsAllowed();
In this scenario, the isAccessAllowed returns "Allowed", ideally it should return "denied".
I have attached the screenshot for your reference, which highlights the 3 cases that i have pointed out.
Regards
Aruna
On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org>> wrote:
Aruna,
>> However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
1. Audit log (in access tab) should include the ID of the policy that allowed the access. Can you please check?
2. What was the return from isAccessAllowed(request)? Allow or Deny?
If above does not help to identify the issue, can you please send the return value from isAccessAllowed(request) and the screenshot of the corresponding audit log?
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Monday, December 14, 2015 at 7:07 AM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Bosco,
Thanks for a very detailed explanation, it has given me a very good perspective on how ranger works/can be used.
Our use case is that we plan to use apache ranger for authorization of a home grown application that is non hadoop in nature. As we are using ranger for the authorization for all the hadoop based components, we plan to use ranger for all custom based applications in order to ensure consistency as well as use the auditing / GUI and various other features of ranger.
Good news is that, I was able to write a custom plugin & custom authoriser in ranger for our service and it is also able to sync successfully (as seen in the plugin tab of the audit GUI). The audit UI (access tab) is also showing the details of the policy that has been accessed.
However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
Thank you once again.
Regards
Aruna
On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org>> wrote:
Aruna, can you give more detail on what you are trying to achieve?
I was searching for integration design diagram, but couldn’t find one. We will work on creating one. In the meanwhile, here is the high level.
1. Ranger plugins run within the component process.
2. It gives a light java library, which does the following:
* Provides method to check access. RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
* Pulls policies from Ranger Admin (implicit)
* Does auditing (implicit)
1. Ranger community works closely with the Hadoop component community for writing plugins for Hadoop components. Currently, there are close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN, Storm, Knox and KMS). We are working with the other communities to support more.
2. The framework is generic and so you can use Ranger to provide access control to your home grown application also. The wiki page https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives how to implement custom stack.
3. Ranger design philosophy is not to change the authorization of the component, but have the component define an interface and abstract all the actions the component supports. This way the components are free to support any actions they want to and evolve them over the period of time. This also help other security providers to provide similar implementation.
So if you are planning to write a custom plugin, I assume you are having your app and you want to use Ranger to provide the access control and audit. And if that is the case, some of the questions you have asked applies mostly on your side.
1. log4j: Since Ranger plugin is embedded within the process, it uses the component’s logging framework. So out here, it will be yours
2. If you want to use Ranger plugin, first you need to create an interface in your application for authorization. We recommend you provide the default/native simple implementation. Ranger will implement the same interface, but Ranger implementation will use the policies from Ranger Admin. You can review the following sample implementations: https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java and https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
3. Once your interface is defined, then the wiki pages tells how to define a service using JSON. Which can be loaded in Ranger Admin. After that you can create the policies from Ranger Admin UI, the REST APIs for managing polices are automatically possible, the Ranger Admin audit UI will start showing the audits, etc.
Please give your use case, so we can guide you better.
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Thursday, December 10, 2015 at 11:12 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
======================================================
BNP Paribas Fortis disclaimer:
http://www.bnpparibasfortis.com/e-mail-disclaimer.html
BNP Paribas Fortis privacy policy:
http://www.bnpparibasfortis.com/privacy-policy.html
======================================================
Re: Queries on the developement for a new custom plugin
Posted by Aruna Sivaram <si...@gmail.com>.
Thanks madhan, for the detailed explanation. I am now able to understand
the audit logging much better.
Appreciate your help and support.
Regards
Aruna
On 21 Dec 2015 20:57, "Madhan Neethiraj" <ma...@apache.org> wrote:
> Aruna,
>
> >> If user "aruna" does any other action, other than submit-app on the
> resource "default". Then the audit log will show the result as "denied". In
> this case, should the policy id which is 656 be shown along with the
> "denied" result, or the id will be a blank field.
> Lets say the action was “write”. In this case, policy #656 would not
> “deny” the access. The policy simply was not able to make an authorization
> decision – since the attributes of the request (user, group, access-type)
> do not match with the policy. In such case, the policy engine will look for
> the next policy that matches the request. Following scenario should help
> clarify this.
>
> In addition to 656, consider there is another policy with: “id=657;
> resource=*; user=john; access-type=write”; here the resource and
> access-type in the policy match the request, but the username does not.
> Hence this policy can not make the authorization decision. In this case,
> should we expect the audit log to contain both 656 and 657 as policyIds?
> Remember, there could be multiple such policies that do not match the
> request.
>
>
> >> Under what cases will the audit log show result as "denied" along with
> policy id.?
> Ranger policy model was recently enhanced to be able to actively deny
> access – refer the documentation here
> <https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies>.
> Only when such a policy explicitly denies an access, audit log will record
> its policyId.
>
> Hope this helps.
>
> Madhan
>
> From: Aruna Sivaram <si...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Monday, December 21, 2015 at 5:59 AM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Queries on the developement for a new custom plugin
>
> Madhan,
>
> let's say i have a policy with following details
>
> Policy id : 656
> Resource name : default
> User : aruna
> Access type : submit-app
>
> If user "aruna" does any other action, other than submit-app on the
> resource "default". Then the audit log will show the result as "denied". In
> this case, should the policy id which is 656 be shown along with the
> "denied" result, or the id will be a blank field. That's my confusion.
>
> Under what cases will the audit log show result as "denied" along with
> policy id.?
>
> Thanks
> Aruna
> On 17 Dec 2015 22:30, "Madhan Neethiraj" <ma...@apache.org> wrote:
>
>> Aruna,
>>
>> >> However, the policyId still shows blank. Is this the right behavior?
>> Ranger populates policyId field in audit log only when that policy makes
>> the authorization decision – either allow or deny. In this particular case,
>> no policy explicitly allowed or denied the access. The end result is “deny”
>> - only because there was no policy to allow the access. Hence the policyId
>> is left blank. Hope this helps. To help understand this further, try to
>> answer this question: which policyId should be recorded in the audit log in
>> this case? Please consider that there could be multiple policies applicable
>> for the resource being accessed – like queue=*, queue=test*, queue=t*.
>>
>> Thanks,
>> Madhan
>>
>> From: Aruna Sivaram <si...@gmail.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Thursday, December 17, 2015 at 12:17 AM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: Queries on the developement for a new custom plugin
>>
>> Madhan,
>>
>> I was able to fix this issue for the 3rd case. Thing is that there was a
>> spelling mismatch between the actionType specified in the servicedef and
>> the actionType that is sent through the request. Now, with the change i am
>> getting the Result as "Denied" instead of "allowed". However, the policyId
>> still shows blank. Is this the right behavior?
>>
>> I am attaching a screenshot for your reference.
>>
>> Regards
>> Aruna
>>
>> On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Aruna,
>>>
>>> >> 2nd case --> if a different user "it3" does a "submit-app" on the
>>> "default" queue, then the audit log shows the result as "denied". However,
>>> the policy id is blank, is this the right behavior? the isAccessAllowed
>>> returns “Denied"
>>>
>>> Since there was no policy that allowed the access, the request was
>>> denied. Hence the policyId is blank.
>>>
>>>
>>>
>>> >> 3rd case --> if the user "network3" does a "admin-queue" on the
>>> "default" queue, then the audit log shows "allowed", instead of "denied”
>>>
>>> The audit log clearly says that the access was granted by policyId=656.
>>> Can you please send the screenshot of the policy? Another possibility is if
>>> your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as
>>> shown below); can you please check if this is the case?
>>>
>>> "accessTypes":
>>>
>>> [
>>>
>>> {
>>>
>>> "itemId": 1,
>>>
>>> "name": "SUBMIT_APP",
>>>
>>> "label": “SUBMIT_APP”,
>>>
>>> "impliedGrants":
>>>
>>> [
>>>
>>> "ADMIN_QUEUE"
>>>
>>> ]
>>>
>>> },
>>>
>>>
>>> {
>>>
>>> "itemId": 2,
>>>
>>> "name": “ADMIN_QUEUE",
>>>
>>> "label": "ADMIN_QUEUE"
>>>
>>> }
>>>
>>> ]
>>>
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Aruna Sivaram <si...@gmail.com>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Monday, December 14, 2015 at 10:47 PM
>>>
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Re: Queries on the developement for a new custom plugin
>>>
>>> Madhan,
>>>
>>> Let assume that I have created a custom policy named TestPolicy, Policy
>>> Id : 656 for a user called "network3" giving "submit-app" permissions for
>>> a queue called "default".
>>>
>>> 1st case --> , if the same user "network3" does a "submit-app" on the
>>> "default" queue, then the audit log shows the correct result with the
>>> Result as "Allowed", the isAccessAllowed method returns "Allowed"
>>>
>>> 2nd case --> if a different user "it3" does a "submit-app" on the
>>> "default" queue, then the audit log shows the result as "denied". However,
>>> the policy id is blank, is this the right behavior? the isAccessAllowed
>>> returns "Denied"
>>>
>>> 3rd case --> if the user "network3" does a "admin-queue" on the
>>> "default" queue, then the audit log shows "allowed", instead of "denied" (I
>>> have created a policy giving only "submit-app" permissions on "default"
>>> queue to network3 user). In such a scenario, what should be done? Currently
>>> my custom authoriser passes the RangerAccessRequestImpl to the
>>> isAccessAllowed Method.
>>> and following are the values of the RangerAccessRequestImpl object in
>>> this scenario.
>>>
>>> RangerAccessRequestImpl request = new
>>> RangerAccessRequestImpl();
>>> RangerAccessResourceImpl resource = new
>>> RangerAccessResourceImpl();
>>> resource.setValue("queue", default);
>>> request.setResource(resource);
>>> request.setAccessType("admin-queue");
>>> request.setUser("network3");
>>> request.setAccessTime(new Date());
>>>
>>> request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
>>> request.setAction("admin-queue");
>>> request.setClientIPAddress("10.0.2.15");
>>> RangerAccessResult result = plugin.isAccessAllowed(request);
>>> return result == null ? false : result.getIsAllowed();
>>> In this scenario, the isAccessAllowed returns "Allowed", ideally it
>>> should return "denied".
>>>
>>>
>>> I have attached the screenshot for your reference, which highlights the
>>> 3 cases that i have pointed out.
>>>
>>> Regards
>>> Aruna
>>>
>>> On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Aruna,
>>>>
>>>> >> However, in cases, where a user cannot access a resource, the audit
>>>> UI (access tab) should show "denied". However, in my case, it stills shows
>>>> as "allowed" instead of "denied". It would be helpful if you could tell me
>>>> what needs to be done in such a case. I am invoking the
>>>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
>>>> know what needs to be passed to this API when a user is not allowed to
>>>> access a particular resource.
>>>>
>>>> 1. Audit log (in access tab) should include the ID of the policy that
>>>> allowed the access. Can you please check?
>>>> 2. What was the return from isAccessAllowed(request)? Allow or Deny?
>>>>
>>>> If above does not help to identify the issue, can you please send the
>>>> return value from isAccessAllowed(request) and the screenshot of the
>>>> corresponding audit log?
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>> From: Aruna Sivaram <si...@gmail.com>
>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Date: Monday, December 14, 2015 at 7:07 AM
>>>>
>>>> To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Re: Queries on the developement for a new custom plugin
>>>>
>>>> Bosco,
>>>>
>>>> Thanks for a very detailed explanation, it has given me a very good
>>>> perspective on how ranger works/can be used.
>>>>
>>>> Our use case is that we plan to use apache ranger for authorization of
>>>> a home grown application that is non hadoop in nature. As we are using
>>>> ranger for the authorization for all the hadoop based components, we plan
>>>> to use ranger for all custom based applications in order to ensure
>>>> consistency as well as use the auditing / GUI and various other features of
>>>> ranger.
>>>>
>>>> Good news is that, I was able to write a custom plugin & custom
>>>> authoriser in ranger for our service and it is also able to sync
>>>> successfully (as seen in the plugin tab of the audit GUI). The audit UI
>>>> (access tab) is also showing the details of the policy that has been
>>>> accessed.
>>>>
>>>> However, in cases, where a user cannot access a resource, the audit UI
>>>> (access tab) should show "denied". However, in my case, it stills shows as
>>>> "allowed" instead of "denied". It would be helpful if you could tell me
>>>> what needs to be done in such a case. I am invoking the
>>>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
>>>> know what needs to be passed to this API when a user is not allowed to
>>>> access a particular resource.
>>>>
>>>>
>>>> Thank you once again.
>>>>
>>>> Regards
>>>> Aruna
>>>>
>>>>
>>>> On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org>
>>>> wrote:
>>>>
>>>>> Aruna, can you give more detail on what you are trying to achieve?
>>>>>
>>>>> I was searching for integration design diagram, but couldn’t find one.
>>>>> We will work on creating one. In the meanwhile, here is the high level.
>>>>>
>>>>>
>>>>> 1. Ranger plugins run within the component process.
>>>>> 2. It gives a light java library, which does the following:
>>>>> 1. Provides method to check access.
>>>>> RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
>>>>> 2. Pulls policies from Ranger Admin (implicit)
>>>>> 3. Does auditing (implicit)
>>>>> 3. Ranger community works closely with the Hadoop component
>>>>> community for writing plugins for Hadoop components. Currently, there are
>>>>> close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka,
>>>>> Solr, YARN, Storm, Knox and KMS). We are working with the other communities
>>>>> to support more.
>>>>> 4. The framework is generic and so you can use Ranger to provide
>>>>> access control to your home grown application also. The wiki page
>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives
>>>>> how to implement custom stack.
>>>>> 5. Ranger design philosophy is not to change the authorization of
>>>>> the component, but have the component define an interface and abstract all
>>>>> the actions the component supports. This way the components are free to
>>>>> support any actions they want to and evolve them over the period of time.
>>>>> This also help other security providers to provide similar implementation.
>>>>>
>>>>> So if you are planning to write a custom plugin, I assume you are
>>>>> having your app and you want to use Ranger to provide the access control
>>>>> and audit. And if that is the case, some of the questions you have asked
>>>>> applies mostly on your side.
>>>>>
>>>>> 1. log4j: Since Ranger plugin is embedded within the process, it
>>>>> uses the component’s logging framework. So out here, it will be yours
>>>>> 2. If you want to use Ranger plugin, first you need to create an
>>>>> interface in your application for authorization. We recommend you provide
>>>>> the default/native simple implementation. Ranger will implement the same
>>>>> interface, but Ranger implementation will use the policies from Ranger
>>>>> Admin. You can review the following sample implementations:
>>>>> https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>>>>> and
>>>>> https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
>>>>> 3. Once your interface is defined, then the wiki pages tells how
>>>>> to define a service using JSON. Which can be loaded in Ranger Admin. After
>>>>> that you can create the policies from Ranger Admin UI, the REST APIs for
>>>>> managing polices are automatically possible, the Ranger Admin audit UI will
>>>>> start showing the audits, etc.
>>>>>
>>>>> Please give your use case, so we can guide you better.
>>>>>
>>>>> Thanks
>>>>>
>>>>> Bosco
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> From: Ramesh Mani <rm...@hortonworks.com>
>>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>>> Date: Thursday, December 10, 2015 at 11:12 PM
>>>>>
>>>>> To: "user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Subject: Re: Queries on the developement for a new custom plugin
>>>>>
>>>>> Please find the answer below.
>>>>>
>>>>> From: Aruna Sivaram <si...@gmail.com>
>>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Date: Thursday, December 10, 2015 at 8:34 PM
>>>>> To: "user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Subject: Re: Queries on the developement for a new custom plugin
>>>>>
>>>>> Thanks for your quick response.Please find my queries inline.
>>>>>
>>>>>
>>>>> On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>
>>>>> wrote:
>>>>>
>>>>> If you have created a CustomService, and if you want to authorize the
>>>>>> access of components/resources in your CustomService, then you need to have
>>>>>> default authorizer in your CustomService to do authorization, which you
>>>>>> will extend in your Ranger custom plugin and will be called when your
>>>>>> CustomService needs authorization check.
>>>>>>
>>>>>
>>>>> [*Aruna] : It would be great if you could provide a snippet/example
>>>>> where the authoriser code present in the plugin is invoked by the existing
>>>>> services (hive/hdfs/storm/kafka) for authorisation. This would give me a
>>>>> better picture as to how it exactly works.*
>>>>> [RM] Each service dictates how to the register the custom authorizer
>>>>> with it. So you need to refer the respective service’s authorization
>>>>> mechanism.
>>>>>
>>>>>
>>>>>> For logging, as you had seen it uses log4j you can have log4j
>>>>>> appender in the log4.properties and get the log, all hadoop components have
>>>>>> their log4j properties file where it specify the location it puts the log.
>>>>>> By default it is /var/log/hadoop/ for hadoop.
>>>>>>
>>>>>
>>>>> *[Aruna] : there are many log4j.properties in the ranger code base and
>>>>> there is no specific log4j.properties for each of the plugins. Hence, which
>>>>> log4j.properties do i need to modify. My aim is to see all the logs
>>>>> generated by my custom plugin. This would enable me to debug the code
>>>>> better.*
>>>>> [RM] Here also each service provides it log4j.properties file and that
>>>>> is where you define appender for ranger also.
>>>>>
>>>>> [Aruna ] : The other thing i wanted to know was how does one start
>>>>> the plugin or rather initiate the plugin which polls for the policies. I
>>>>> see that the enable-<service>-plugin.sh scripts sets the environment and
>>>>> copies property files to the right locations, but i dont see where the
>>>>> authoriser is instantiated in order to invoke the init() method of the
>>>>> RangerbasePlugin.
>>>>> [RM] This is already in the wiki page.
>>>>>
>>>>>
>>>>>
>>>>>> Thanks for all your help
>>>>>>
>>>>>
>>>>> Aruna
>>>>>
>>>>>>
>>>>>> From: Aruna Sivaram <si...@gmail.com>
>>>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>>>> user@ranger.incubator.apache.org>
>>>>>> Date: Wednesday, December 9, 2015 at 9:45 PM
>>>>>> To: "user@ranger.incubator.apache.org" <
>>>>>> user@ranger.incubator.apache.org>
>>>>>> Subject: Queries on the developement for a new custom plugin
>>>>>>
>>>>>> I am using ranger 0.5 for the access control. We are planning to
>>>>>> develop a custom plugin which we plan to integrate with the ranger
>>>>>> framework. This custom plugin will be used for access control of our
>>>>>> components. In order to explore this possibility, i have written a custom
>>>>>> plugin as per the example given in the link
>>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>>>>>>
>>>>>> Since this is a custom plugin, my authoriser class
>>>>>> (CustomServiceAuthoriser), will not be extending any of the hadoop security
>>>>>> classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
>>>>>> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
>>>>>> have created the repository and policies for the same through the rest web
>>>>>> service.
>>>>>>
>>>>>>
>>>>>> I have written the custom service authoriser as per the example and
>>>>>> deployed the same on the sandbox. I have a service named CustomService.
>>>>>> What i wanted to know is how the customservice will communicate with my
>>>>>> customserviceauthoriser which is contained in my plugin. Currently i dont
>>>>>> find any documentation which talks about the mode of communication or
>>>>>> rather how the plugin class will be invoked by the service.
>>>>>>
>>>>>> I am relatively new to ranger so may be I am missing something ?
>>>>>>
>>>>>> Also, i would like to know the location of the log where each of the
>>>>>> plugin classes would be logging. This will help us in debugging the flow. I
>>>>>> see a lot of log statements in the ranger plugin code base but am unable to
>>>>>> find the location of the logs.
>>>>>>
>>>>>> Secondly, can ranger be used to develop custom plugins for access
>>>>>> control of non hadoop components?
>>>>>>
>>>>>> Any help from your end would be appreciated
>>>>>>
>>>>>> --
>>>>>> Regards
>>>>>> Aruna Sivaram
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards
>>>>> Aruna Sivaram
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards
>>>> Aruna Sivaram
>>>>
>>>
>>>
>>>
>>> --
>>> Regards
>>> Aruna Sivaram
>>>
>>
>>
>>
>> --
>> Regards
>> Aruna Sivaram
>>
>
Re: Queries on the developement for a new custom plugin
Posted by Madhan Neethiraj <ma...@apache.org>.
Aruna,
>> If user "aruna" does any other action, other than submit-app on the resource "default". Then the audit log will show the result as "denied". In this case, should the policy id which is 656 be shown along with the "denied" result, or the id will be a blank field.
Lets say the action was “write”. In this case, policy #656 would not “deny” the access. The policy simply was not able to make an authorization decision – since the attributes of the request (user, group, access-type) do not match with the policy. In such case, the policy engine will look for the next policy that matches the request. Following scenario should help clarify this.
In addition to 656, consider there is another policy with: “id=657; resource=*; user=john; access-type=write”; here the resource and access-type in the policy match the request, but the username does not. Hence this policy can not make the authorization decision. In this case, should we expect the audit log to contain both 656 and 657 as policyIds? Remember, there could be multiple such policies that do not match the request.
>> Under what cases will the audit log show result as "denied" along with policy id.?
Ranger policy model was recently enhanced to be able to actively deny access – refer the documentation here. Only when such a policy explicitly denies an access, audit log will record its policyId.
Hope this helps.
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 21, 2015 at 5:59 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
let's say i have a policy with following details
Policy id : 656
Resource name : default
User : aruna
Access type : submit-app
If user "aruna" does any other action, other than submit-app on the resource "default". Then the audit log will show the result as "denied". In this case, should the policy id which is 656 be shown along with the "denied" result, or the id will be a blank field. That's my confusion.
Under what cases will the audit log show result as "denied" along with policy id.?
Thanks
Aruna
On 17 Dec 2015 22:30, "Madhan Neethiraj" <ma...@apache.org> wrote:
Aruna,
>> However, the policyId still shows blank. Is this the right behavior?
Ranger populates policyId field in audit log only when that policy makes the authorization decision – either allow or deny. In this particular case, no policy explicitly allowed or denied the access. The end result is “deny” - only because there was no policy to allow the access. Hence the policyId is left blank. Hope this helps. To help understand this further, try to answer this question: which policyId should be recorded in the audit log in this case? Please consider that there could be multiple policies applicable for the resource being accessed – like queue=*, queue=test*, queue=t*.
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 17, 2015 at 12:17 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
I was able to fix this issue for the 3rd case. Thing is that there was a spelling mismatch between the actionType specified in the servicedef and the actionType that is sent through the request. Now, with the change i am getting the Result as "Denied" instead of "allowed". However, the policyId still shows blank. Is this the right behavior?
I am attaching a screenshot for your reference.
Regards
Aruna
On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <ma...@apache.org> wrote:
Aruna,
>> 2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns “Denied"
Since there was no policy that allowed the access, the request was denied. Hence the policyId is blank.
>> 3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied”
The audit log clearly says that the access was granted by policyId=656. Can you please send the screenshot of the policy? Another possibility is if your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as shown below); can you please check if this is the case?
"accessTypes":
[
{
"itemId": 1,
"name": "SUBMIT_APP",
"label": “SUBMIT_APP”,
"impliedGrants":
[
"ADMIN_QUEUE"
]
},
{
"itemId": 2,
"name": “ADMIN_QUEUE",
"label": "ADMIN_QUEUE"
}
]
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 10:47 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
Let assume that I have created a custom policy named TestPolicy, Policy Id : 656 for a user called "network3" giving "submit-app" permissions for a queue called "default".
1st case --> , if the same user "network3" does a "submit-app" on the "default" queue, then the audit log shows the correct result with the Result as "Allowed", the isAccessAllowed method returns "Allowed"
2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns "Denied"
3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied" (I have created a policy giving only "submit-app" permissions on "default" queue to network3 user). In such a scenario, what should be done? Currently my custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed Method.
and following are the values of the RangerAccessRequestImpl object in this scenario.
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
RangerAccessResourceImpl resource = new RangerAccessResourceImpl();
resource.setValue("queue", default);
request.setResource(resource);
request.setAccessType("admin-queue");
request.setUser("network3");
request.setAccessTime(new Date());
request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
request.setAction("admin-queue");
request.setClientIPAddress("10.0.2.15");
RangerAccessResult result = plugin.isAccessAllowed(request);
return result == null ? false : result.getIsAllowed();
In this scenario, the isAccessAllowed returns "Allowed", ideally it should return "denied".
I have attached the screenshot for your reference, which highlights the 3 cases that i have pointed out.
Regards
Aruna
On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Aruna,
>> However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
1. Audit log (in access tab) should include the ID of the policy that allowed the access. Can you please check?
2. What was the return from isAccessAllowed(request)? Allow or Deny?
If above does not help to identify the issue, can you please send the return value from isAccessAllowed(request) and the screenshot of the corresponding audit log?
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 7:07 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Bosco,
Thanks for a very detailed explanation, it has given me a very good perspective on how ranger works/can be used.
Our use case is that we plan to use apache ranger for authorization of a home grown application that is non hadoop in nature. As we are using ranger for the authorization for all the hadoop based components, we plan to use ranger for all custom based applications in order to ensure consistency as well as use the auditing / GUI and various other features of ranger.
Good news is that, I was able to write a custom plugin & custom authoriser in ranger for our service and it is also able to sync successfully (as seen in the plugin tab of the audit GUI). The audit UI (access tab) is also showing the details of the policy that has been accessed.
However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
Thank you once again.
Regards
Aruna
On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org> wrote:
Aruna, can you give more detail on what you are trying to achieve?
I was searching for integration design diagram, but couldn’t find one. We will work on creating one. In the meanwhile, here is the high level.
Ranger plugins run within the component process.
It gives a light java library, which does the following:
Provides method to check access. RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
Pulls policies from Ranger Admin (implicit)
Does auditing (implicit)
Ranger community works closely with the Hadoop component community for writing plugins for Hadoop components. Currently, there are close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN, Storm, Knox and KMS). We are working with the other communities to support more.
The framework is generic and so you can use Ranger to provide access control to your home grown application also. The wiki page https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives how to implement custom stack.
Ranger design philosophy is not to change the authorization of the component, but have the component define an interface and abstract all the actions the component supports. This way the components are free to support any actions they want to and evolve them over the period of time. This also help other security providers to provide similar implementation.
So if you are planning to write a custom plugin, I assume you are having your app and you want to use Ranger to provide the access control and audit. And if that is the case, some of the questions you have asked applies mostly on your side.
log4j: Since Ranger plugin is embedded within the process, it uses the component’s logging framework. So out here, it will be yours
If you want to use Ranger plugin, first you need to create an interface in your application for authorization. We recommend you provide the default/native simple implementation. Ranger will implement the same interface, but Ranger implementation will use the policies from Ranger Admin. You can review the following sample implementations: https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java and https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
Once your interface is defined, then the wiki pages tells how to define a service using JSON. Which can be loaded in Ranger Admin. After that you can create the policies from Ranger Admin UI, the REST APIs for managing polices are automatically possible, the Ranger Admin audit UI will start showing the audits, etc.
Please give your use case, so we can guide you better.
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 11:12 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Aruna Sivaram <si...@gmail.com>.
Madhan,
let's say i have a policy with following details
Policy id : 656
Resource name : default
User : aruna
Access type : submit-app
If user "aruna" does any other action, other than submit-app on the
resource "default". Then the audit log will show the result as "denied". In
this case, should the policy id which is 656 be shown along with the
"denied" result, or the id will be a blank field. That's my confusion.
Under what cases will the audit log show result as "denied" along with
policy id.?
Thanks
Aruna
On 17 Dec 2015 22:30, "Madhan Neethiraj" <ma...@apache.org> wrote:
> Aruna,
>
> >> However, the policyId still shows blank. Is this the right behavior?
> Ranger populates policyId field in audit log only when that policy makes
> the authorization decision – either allow or deny. In this particular case,
> no policy explicitly allowed or denied the access. The end result is “deny”
> - only because there was no policy to allow the access. Hence the policyId
> is left blank. Hope this helps. To help understand this further, try to
> answer this question: which policyId should be recorded in the audit log in
> this case? Please consider that there could be multiple policies applicable
> for the resource being accessed – like queue=*, queue=test*, queue=t*.
>
> Thanks,
> Madhan
>
> From: Aruna Sivaram <si...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Thursday, December 17, 2015 at 12:17 AM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Queries on the developement for a new custom plugin
>
> Madhan,
>
> I was able to fix this issue for the 3rd case. Thing is that there was a
> spelling mismatch between the actionType specified in the servicedef and
> the actionType that is sent through the request. Now, with the change i am
> getting the Result as "Denied" instead of "allowed". However, the policyId
> still shows blank. Is this the right behavior?
>
> I am attaching a screenshot for your reference.
>
> Regards
> Aruna
>
> On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Aruna,
>>
>> >> 2nd case --> if a different user "it3" does a "submit-app" on the
>> "default" queue, then the audit log shows the result as "denied". However,
>> the policy id is blank, is this the right behavior? the isAccessAllowed
>> returns “Denied"
>>
>> Since there was no policy that allowed the access, the request was
>> denied. Hence the policyId is blank.
>>
>>
>>
>> >> 3rd case --> if the user "network3" does a "admin-queue" on the
>> "default" queue, then the audit log shows "allowed", instead of "denied”
>>
>> The audit log clearly says that the access was granted by policyId=656.
>> Can you please send the screenshot of the policy? Another possibility is if
>> your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as
>> shown below); can you please check if this is the case?
>>
>> "accessTypes":
>>
>> [
>>
>> {
>>
>> "itemId": 1,
>>
>> "name": "SUBMIT_APP",
>>
>> "label": “SUBMIT_APP”,
>>
>> "impliedGrants":
>>
>> [
>>
>> "ADMIN_QUEUE"
>>
>> ]
>>
>> },
>>
>>
>> {
>>
>> "itemId": 2,
>>
>> "name": “ADMIN_QUEUE",
>>
>> "label": "ADMIN_QUEUE"
>>
>> }
>>
>> ]
>>
>>
>> Thanks,
>> Madhan
>>
>> From: Aruna Sivaram <si...@gmail.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Monday, December 14, 2015 at 10:47 PM
>>
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: Queries on the developement for a new custom plugin
>>
>> Madhan,
>>
>> Let assume that I have created a custom policy named TestPolicy, Policy
>> Id : 656 for a user called "network3" giving "submit-app" permissions for
>> a queue called "default".
>>
>> 1st case --> , if the same user "network3" does a "submit-app" on the
>> "default" queue, then the audit log shows the correct result with the
>> Result as "Allowed", the isAccessAllowed method returns "Allowed"
>>
>> 2nd case --> if a different user "it3" does a "submit-app" on the
>> "default" queue, then the audit log shows the result as "denied". However,
>> the policy id is blank, is this the right behavior? the isAccessAllowed
>> returns "Denied"
>>
>> 3rd case --> if the user "network3" does a "admin-queue" on the "default"
>> queue, then the audit log shows "allowed", instead of "denied" (I have
>> created a policy giving only "submit-app" permissions on "default" queue to
>> network3 user). In such a scenario, what should be done? Currently my
>> custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed
>> Method.
>> and following are the values of the RangerAccessRequestImpl object in
>> this scenario.
>>
>> RangerAccessRequestImpl request = new
>> RangerAccessRequestImpl();
>> RangerAccessResourceImpl resource = new
>> RangerAccessResourceImpl();
>> resource.setValue("queue", default);
>> request.setResource(resource);
>> request.setAccessType("admin-queue");
>> request.setUser("network3");
>> request.setAccessTime(new Date());
>>
>> request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
>> request.setAction("admin-queue");
>> request.setClientIPAddress("10.0.2.15");
>> RangerAccessResult result = plugin.isAccessAllowed(request);
>> return result == null ? false : result.getIsAllowed();
>> In this scenario, the isAccessAllowed returns "Allowed", ideally it
>> should return "denied".
>>
>>
>> I have attached the screenshot for your reference, which highlights the 3
>> cases that i have pointed out.
>>
>> Regards
>> Aruna
>>
>> On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Aruna,
>>>
>>> >> However, in cases, where a user cannot access a resource, the audit
>>> UI (access tab) should show "denied". However, in my case, it stills shows
>>> as "allowed" instead of "denied". It would be helpful if you could tell me
>>> what needs to be done in such a case. I am invoking the
>>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
>>> know what needs to be passed to this API when a user is not allowed to
>>> access a particular resource.
>>>
>>> 1. Audit log (in access tab) should include the ID of the policy that
>>> allowed the access. Can you please check?
>>> 2. What was the return from isAccessAllowed(request)? Allow or Deny?
>>>
>>> If above does not help to identify the issue, can you please send the
>>> return value from isAccessAllowed(request) and the screenshot of the
>>> corresponding audit log?
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Aruna Sivaram <si...@gmail.com>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Monday, December 14, 2015 at 7:07 AM
>>>
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Re: Queries on the developement for a new custom plugin
>>>
>>> Bosco,
>>>
>>> Thanks for a very detailed explanation, it has given me a very good
>>> perspective on how ranger works/can be used.
>>>
>>> Our use case is that we plan to use apache ranger for authorization of a
>>> home grown application that is non hadoop in nature. As we are using ranger
>>> for the authorization for all the hadoop based components, we plan to use
>>> ranger for all custom based applications in order to ensure consistency as
>>> well as use the auditing / GUI and various other features of ranger.
>>>
>>> Good news is that, I was able to write a custom plugin & custom
>>> authoriser in ranger for our service and it is also able to sync
>>> successfully (as seen in the plugin tab of the audit GUI). The audit UI
>>> (access tab) is also showing the details of the policy that has been
>>> accessed.
>>>
>>> However, in cases, where a user cannot access a resource, the audit UI
>>> (access tab) should show "denied". However, in my case, it stills shows as
>>> "allowed" instead of "denied". It would be helpful if you could tell me
>>> what needs to be done in such a case. I am invoking the
>>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
>>> know what needs to be passed to this API when a user is not allowed to
>>> access a particular resource.
>>>
>>>
>>> Thank you once again.
>>>
>>> Regards
>>> Aruna
>>>
>>>
>>> On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org>
>>> wrote:
>>>
>>>> Aruna, can you give more detail on what you are trying to achieve?
>>>>
>>>> I was searching for integration design diagram, but couldn’t find one.
>>>> We will work on creating one. In the meanwhile, here is the high level.
>>>>
>>>>
>>>> 1. Ranger plugins run within the component process.
>>>> 2. It gives a light java library, which does the following:
>>>> 1. Provides method to check access.
>>>> RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
>>>> 2. Pulls policies from Ranger Admin (implicit)
>>>> 3. Does auditing (implicit)
>>>> 3. Ranger community works closely with the Hadoop component
>>>> community for writing plugins for Hadoop components. Currently, there are
>>>> close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka,
>>>> Solr, YARN, Storm, Knox and KMS). We are working with the other communities
>>>> to support more.
>>>> 4. The framework is generic and so you can use Ranger to provide
>>>> access control to your home grown application also. The wiki page
>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives
>>>> how to implement custom stack.
>>>> 5. Ranger design philosophy is not to change the authorization of
>>>> the component, but have the component define an interface and abstract all
>>>> the actions the component supports. This way the components are free to
>>>> support any actions they want to and evolve them over the period of time.
>>>> This also help other security providers to provide similar implementation.
>>>>
>>>> So if you are planning to write a custom plugin, I assume you are
>>>> having your app and you want to use Ranger to provide the access control
>>>> and audit. And if that is the case, some of the questions you have asked
>>>> applies mostly on your side.
>>>>
>>>> 1. log4j: Since Ranger plugin is embedded within the process, it
>>>> uses the component’s logging framework. So out here, it will be yours
>>>> 2. If you want to use Ranger plugin, first you need to create an
>>>> interface in your application for authorization. We recommend you provide
>>>> the default/native simple implementation. Ranger will implement the same
>>>> interface, but Ranger implementation will use the policies from Ranger
>>>> Admin. You can review the following sample implementations:
>>>> https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>>>> and
>>>> https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
>>>> 3. Once your interface is defined, then the wiki pages tells how to
>>>> define a service using JSON. Which can be loaded in Ranger Admin. After
>>>> that you can create the policies from Ranger Admin UI, the REST APIs for
>>>> managing polices are automatically possible, the Ranger Admin audit UI will
>>>> start showing the audits, etc.
>>>>
>>>> Please give your use case, so we can guide you better.
>>>>
>>>> Thanks
>>>>
>>>> Bosco
>>>>
>>>>
>>>>
>>>>
>>>> From: Ramesh Mani <rm...@hortonworks.com>
>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>> Date: Thursday, December 10, 2015 at 11:12 PM
>>>>
>>>> To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Re: Queries on the developement for a new custom plugin
>>>>
>>>> Please find the answer below.
>>>>
>>>> From: Aruna Sivaram <si...@gmail.com>
>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Date: Thursday, December 10, 2015 at 8:34 PM
>>>> To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Re: Queries on the developement for a new custom plugin
>>>>
>>>> Thanks for your quick response.Please find my queries inline.
>>>>
>>>>
>>>> On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>
>>>> wrote:
>>>>
>>>> If you have created a CustomService, and if you want to authorize the
>>>>> access of components/resources in your CustomService, then you need to have
>>>>> default authorizer in your CustomService to do authorization, which you
>>>>> will extend in your Ranger custom plugin and will be called when your
>>>>> CustomService needs authorization check.
>>>>>
>>>>
>>>> [*Aruna] : It would be great if you could provide a snippet/example
>>>> where the authoriser code present in the plugin is invoked by the existing
>>>> services (hive/hdfs/storm/kafka) for authorisation. This would give me a
>>>> better picture as to how it exactly works.*
>>>> [RM] Each service dictates how to the register the custom authorizer
>>>> with it. So you need to refer the respective service’s authorization
>>>> mechanism.
>>>>
>>>>
>>>>> For logging, as you had seen it uses log4j you can have log4j appender
>>>>> in the log4.properties and get the log, all hadoop components have their
>>>>> log4j properties file where it specify the location it puts the log.
>>>>> By default it is /var/log/hadoop/ for hadoop.
>>>>>
>>>>
>>>> *[Aruna] : there are many log4j.properties in the ranger code base and
>>>> there is no specific log4j.properties for each of the plugins. Hence, which
>>>> log4j.properties do i need to modify. My aim is to see all the logs
>>>> generated by my custom plugin. This would enable me to debug the code
>>>> better.*
>>>> [RM] Here also each service provides it log4j.properties file and that
>>>> is where you define appender for ranger also.
>>>>
>>>> [Aruna ] : The other thing i wanted to know was how does one start the
>>>> plugin or rather initiate the plugin which polls for the policies. I see
>>>> that the enable-<service>-plugin.sh scripts sets the environment and copies
>>>> property files to the right locations, but i dont see where the authoriser
>>>> is instantiated in order to invoke the init() method of the
>>>> RangerbasePlugin.
>>>> [RM] This is already in the wiki page.
>>>>
>>>>
>>>>
>>>>> Thanks for all your help
>>>>>
>>>>
>>>> Aruna
>>>>
>>>>>
>>>>> From: Aruna Sivaram <si...@gmail.com>
>>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Date: Wednesday, December 9, 2015 at 9:45 PM
>>>>> To: "user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Subject: Queries on the developement for a new custom plugin
>>>>>
>>>>> I am using ranger 0.5 for the access control. We are planning to
>>>>> develop a custom plugin which we plan to integrate with the ranger
>>>>> framework. This custom plugin will be used for access control of our
>>>>> components. In order to explore this possibility, i have written a custom
>>>>> plugin as per the example given in the link
>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>>>>>
>>>>> Since this is a custom plugin, my authoriser class
>>>>> (CustomServiceAuthoriser), will not be extending any of the hadoop security
>>>>> classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
>>>>> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
>>>>> have created the repository and policies for the same through the rest web
>>>>> service.
>>>>>
>>>>>
>>>>> I have written the custom service authoriser as per the example and
>>>>> deployed the same on the sandbox. I have a service named CustomService.
>>>>> What i wanted to know is how the customservice will communicate with my
>>>>> customserviceauthoriser which is contained in my plugin. Currently i dont
>>>>> find any documentation which talks about the mode of communication or
>>>>> rather how the plugin class will be invoked by the service.
>>>>>
>>>>> I am relatively new to ranger so may be I am missing something ?
>>>>>
>>>>> Also, i would like to know the location of the log where each of the
>>>>> plugin classes would be logging. This will help us in debugging the flow. I
>>>>> see a lot of log statements in the ranger plugin code base but am unable to
>>>>> find the location of the logs.
>>>>>
>>>>> Secondly, can ranger be used to develop custom plugins for access
>>>>> control of non hadoop components?
>>>>>
>>>>> Any help from your end would be appreciated
>>>>>
>>>>> --
>>>>> Regards
>>>>> Aruna Sivaram
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards
>>>> Aruna Sivaram
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards
>>> Aruna Sivaram
>>>
>>
>>
>>
>> --
>> Regards
>> Aruna Sivaram
>>
>
>
>
> --
> Regards
> Aruna Sivaram
>
Re: Queries on the developement for a new custom plugin
Posted by Madhan Neethiraj <ma...@apache.org>.
Aruna,
>> However, the policyId still shows blank. Is this the right behavior?
Ranger populates policyId field in audit log only when that policy makes the authorization decision – either allow or deny. In this particular case, no policy explicitly allowed or denied the access. The end result is “deny” - only because there was no policy to allow the access. Hence the policyId is left blank. Hope this helps. To help understand this further, try to answer this question: which policyId should be recorded in the audit log in this case? Please consider that there could be multiple policies applicable for the resource being accessed – like queue=*, queue=test*, queue=t*.
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 17, 2015 at 12:17 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
I was able to fix this issue for the 3rd case. Thing is that there was a spelling mismatch between the actionType specified in the servicedef and the actionType that is sent through the request. Now, with the change i am getting the Result as "Denied" instead of "allowed". However, the policyId still shows blank. Is this the right behavior?
I am attaching a screenshot for your reference.
Regards
Aruna
On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <ma...@apache.org> wrote:
Aruna,
>> 2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns “Denied"
Since there was no policy that allowed the access, the request was denied. Hence the policyId is blank.
>> 3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied”
The audit log clearly says that the access was granted by policyId=656. Can you please send the screenshot of the policy? Another possibility is if your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as shown below); can you please check if this is the case?
"accessTypes":
[
{
"itemId": 1,
"name": "SUBMIT_APP",
"label": “SUBMIT_APP”,
"impliedGrants":
[
"ADMIN_QUEUE"
]
},
{
"itemId": 2,
"name": “ADMIN_QUEUE",
"label": "ADMIN_QUEUE"
}
]
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 10:47 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
Let assume that I have created a custom policy named TestPolicy, Policy Id : 656 for a user called "network3" giving "submit-app" permissions for a queue called "default".
1st case --> , if the same user "network3" does a "submit-app" on the "default" queue, then the audit log shows the correct result with the Result as "Allowed", the isAccessAllowed method returns "Allowed"
2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns "Denied"
3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied" (I have created a policy giving only "submit-app" permissions on "default" queue to network3 user). In such a scenario, what should be done? Currently my custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed Method.
and following are the values of the RangerAccessRequestImpl object in this scenario.
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
RangerAccessResourceImpl resource = new RangerAccessResourceImpl();
resource.setValue("queue", default);
request.setResource(resource);
request.setAccessType("admin-queue");
request.setUser("network3");
request.setAccessTime(new Date());
request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
request.setAction("admin-queue");
request.setClientIPAddress("10.0.2.15");
RangerAccessResult result = plugin.isAccessAllowed(request);
return result == null ? false : result.getIsAllowed();
In this scenario, the isAccessAllowed returns "Allowed", ideally it should return "denied".
I have attached the screenshot for your reference, which highlights the 3 cases that i have pointed out.
Regards
Aruna
On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Aruna,
>> However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
1. Audit log (in access tab) should include the ID of the policy that allowed the access. Can you please check?
2. What was the return from isAccessAllowed(request)? Allow or Deny?
If above does not help to identify the issue, can you please send the return value from isAccessAllowed(request) and the screenshot of the corresponding audit log?
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 7:07 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Bosco,
Thanks for a very detailed explanation, it has given me a very good perspective on how ranger works/can be used.
Our use case is that we plan to use apache ranger for authorization of a home grown application that is non hadoop in nature. As we are using ranger for the authorization for all the hadoop based components, we plan to use ranger for all custom based applications in order to ensure consistency as well as use the auditing / GUI and various other features of ranger.
Good news is that, I was able to write a custom plugin & custom authoriser in ranger for our service and it is also able to sync successfully (as seen in the plugin tab of the audit GUI). The audit UI (access tab) is also showing the details of the policy that has been accessed.
However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
Thank you once again.
Regards
Aruna
On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org> wrote:
Aruna, can you give more detail on what you are trying to achieve?
I was searching for integration design diagram, but couldn’t find one. We will work on creating one. In the meanwhile, here is the high level.
Ranger plugins run within the component process.
It gives a light java library, which does the following:
Provides method to check access. RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
Pulls policies from Ranger Admin (implicit)
Does auditing (implicit)
Ranger community works closely with the Hadoop component community for writing plugins for Hadoop components. Currently, there are close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN, Storm, Knox and KMS). We are working with the other communities to support more.
The framework is generic and so you can use Ranger to provide access control to your home grown application also. The wiki page https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives how to implement custom stack.
Ranger design philosophy is not to change the authorization of the component, but have the component define an interface and abstract all the actions the component supports. This way the components are free to support any actions they want to and evolve them over the period of time. This also help other security providers to provide similar implementation.
So if you are planning to write a custom plugin, I assume you are having your app and you want to use Ranger to provide the access control and audit. And if that is the case, some of the questions you have asked applies mostly on your side.
log4j: Since Ranger plugin is embedded within the process, it uses the component’s logging framework. So out here, it will be yours
If you want to use Ranger plugin, first you need to create an interface in your application for authorization. We recommend you provide the default/native simple implementation. Ranger will implement the same interface, but Ranger implementation will use the policies from Ranger Admin. You can review the following sample implementations: https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java and https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
Once your interface is defined, then the wiki pages tells how to define a service using JSON. Which can be loaded in Ranger Admin. After that you can create the policies from Ranger Admin UI, the REST APIs for managing polices are automatically possible, the Ranger Admin audit UI will start showing the audits, etc.
Please give your use case, so we can guide you better.
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 11:12 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Aruna Sivaram <si...@gmail.com>.
Madhan,
I was able to fix this issue for the 3rd case. Thing is that there was a
spelling mismatch between the actionType specified in the servicedef and
the actionType that is sent through the request. Now, with the change i am
getting the Result as "Denied" instead of "allowed". However, the policyId
still shows blank. Is this the right behavior?
I am attaching a screenshot for your reference.
Regards
Aruna
On Wed, Dec 16, 2015 at 5:25 AM, Madhan Neethiraj <ma...@apache.org> wrote:
> Aruna,
>
> >> 2nd case --> if a different user "it3" does a "submit-app" on the
> "default" queue, then the audit log shows the result as "denied". However,
> the policy id is blank, is this the right behavior? the isAccessAllowed
> returns “Denied"
>
> Since there was no policy that allowed the access, the request was denied.
> Hence the policyId is blank.
>
>
>
> >> 3rd case --> if the user "network3" does a "admin-queue" on the
> "default" queue, then the audit log shows "allowed", instead of "denied”
>
> The audit log clearly says that the access was granted by policyId=656.
> Can you please send the screenshot of the policy? Another possibility is if
> your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as
> shown below); can you please check if this is the case?
>
> "accessTypes":
>
> [
>
> {
>
> "itemId": 1,
>
> "name": "SUBMIT_APP",
>
> "label": “SUBMIT_APP”,
>
> "impliedGrants":
>
> [
>
> "ADMIN_QUEUE"
>
> ]
>
> },
>
>
> {
>
> "itemId": 2,
>
> "name": “ADMIN_QUEUE",
>
> "label": "ADMIN_QUEUE"
>
> }
>
> ]
>
>
> Thanks,
> Madhan
>
> From: Aruna Sivaram <si...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Monday, December 14, 2015 at 10:47 PM
>
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Queries on the developement for a new custom plugin
>
> Madhan,
>
> Let assume that I have created a custom policy named TestPolicy, Policy Id
> : 656 for a user called "network3" giving "submit-app" permissions for a
> queue called "default".
>
> 1st case --> , if the same user "network3" does a "submit-app" on the
> "default" queue, then the audit log shows the correct result with the
> Result as "Allowed", the isAccessAllowed method returns "Allowed"
>
> 2nd case --> if a different user "it3" does a "submit-app" on the
> "default" queue, then the audit log shows the result as "denied". However,
> the policy id is blank, is this the right behavior? the isAccessAllowed
> returns "Denied"
>
> 3rd case --> if the user "network3" does a "admin-queue" on the "default"
> queue, then the audit log shows "allowed", instead of "denied" (I have
> created a policy giving only "submit-app" permissions on "default" queue to
> network3 user). In such a scenario, what should be done? Currently my
> custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed
> Method.
> and following are the values of the RangerAccessRequestImpl object in this
> scenario.
>
> RangerAccessRequestImpl request = new
> RangerAccessRequestImpl();
> RangerAccessResourceImpl resource = new
> RangerAccessResourceImpl();
> resource.setValue("queue", default);
> request.setResource(resource);
> request.setAccessType("admin-queue");
> request.setUser("network3");
> request.setAccessTime(new Date());
>
> request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
> request.setAction("admin-queue");
> request.setClientIPAddress("10.0.2.15");
> RangerAccessResult result = plugin.isAccessAllowed(request);
> return result == null ? false : result.getIsAllowed();
> In this scenario, the isAccessAllowed returns "Allowed", ideally it should
> return "denied".
>
>
> I have attached the screenshot for your reference, which highlights the 3
> cases that i have pointed out.
>
> Regards
> Aruna
>
> On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Aruna,
>>
>> >> However, in cases, where a user cannot access a resource, the audit UI
>> (access tab) should show "denied". However, in my case, it stills shows as
>> "allowed" instead of "denied". It would be helpful if you could tell me
>> what needs to be done in such a case. I am invoking the
>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
>> know what needs to be passed to this API when a user is not allowed to
>> access a particular resource.
>>
>> 1. Audit log (in access tab) should include the ID of the policy that
>> allowed the access. Can you please check?
>> 2. What was the return from isAccessAllowed(request)? Allow or Deny?
>>
>> If above does not help to identify the issue, can you please send the
>> return value from isAccessAllowed(request) and the screenshot of the
>> corresponding audit log?
>>
>> Thanks,
>> Madhan
>>
>> From: Aruna Sivaram <si...@gmail.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Monday, December 14, 2015 at 7:07 AM
>>
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: Queries on the developement for a new custom plugin
>>
>> Bosco,
>>
>> Thanks for a very detailed explanation, it has given me a very good
>> perspective on how ranger works/can be used.
>>
>> Our use case is that we plan to use apache ranger for authorization of a
>> home grown application that is non hadoop in nature. As we are using ranger
>> for the authorization for all the hadoop based components, we plan to use
>> ranger for all custom based applications in order to ensure consistency as
>> well as use the auditing / GUI and various other features of ranger.
>>
>> Good news is that, I was able to write a custom plugin & custom
>> authoriser in ranger for our service and it is also able to sync
>> successfully (as seen in the plugin tab of the audit GUI). The audit UI
>> (access tab) is also showing the details of the policy that has been
>> accessed.
>>
>> However, in cases, where a user cannot access a resource, the audit UI
>> (access tab) should show "denied". However, in my case, it stills shows as
>> "allowed" instead of "denied". It would be helpful if you could tell me
>> what needs to be done in such a case. I am invoking the
>> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
>> know what needs to be passed to this API when a user is not allowed to
>> access a particular resource.
>>
>>
>> Thank you once again.
>>
>> Regards
>> Aruna
>>
>>
>> On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>>
>>> Aruna, can you give more detail on what you are trying to achieve?
>>>
>>> I was searching for integration design diagram, but couldn’t find one.
>>> We will work on creating one. In the meanwhile, here is the high level.
>>>
>>>
>>> 1. Ranger plugins run within the component process.
>>> 2. It gives a light java library, which does the following:
>>> 1. Provides method to check access.
>>> RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
>>> 2. Pulls policies from Ranger Admin (implicit)
>>> 3. Does auditing (implicit)
>>> 3. Ranger community works closely with the Hadoop component
>>> community for writing plugins for Hadoop components. Currently, there are
>>> close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka,
>>> Solr, YARN, Storm, Knox and KMS). We are working with the other communities
>>> to support more.
>>> 4. The framework is generic and so you can use Ranger to provide
>>> access control to your home grown application also. The wiki page
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives
>>> how to implement custom stack.
>>> 5. Ranger design philosophy is not to change the authorization of
>>> the component, but have the component define an interface and abstract all
>>> the actions the component supports. This way the components are free to
>>> support any actions they want to and evolve them over the period of time.
>>> This also help other security providers to provide similar implementation.
>>>
>>> So if you are planning to write a custom plugin, I assume you are having
>>> your app and you want to use Ranger to provide the access control and
>>> audit. And if that is the case, some of the questions you have asked
>>> applies mostly on your side.
>>>
>>> 1. log4j: Since Ranger plugin is embedded within the process, it
>>> uses the component’s logging framework. So out here, it will be yours
>>> 2. If you want to use Ranger plugin, first you need to create an
>>> interface in your application for authorization. We recommend you provide
>>> the default/native simple implementation. Ranger will implement the same
>>> interface, but Ranger implementation will use the policies from Ranger
>>> Admin. You can review the following sample implementations:
>>> https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>>> and
>>> https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
>>> 3. Once your interface is defined, then the wiki pages tells how to
>>> define a service using JSON. Which can be loaded in Ranger Admin. After
>>> that you can create the policies from Ranger Admin UI, the REST APIs for
>>> managing polices are automatically possible, the Ranger Admin audit UI will
>>> start showing the audits, etc.
>>>
>>> Please give your use case, so we can guide you better.
>>>
>>> Thanks
>>>
>>> Bosco
>>>
>>>
>>>
>>>
>>> From: Ramesh Mani <rm...@hortonworks.com>
>>> Reply-To: <us...@ranger.incubator.apache.org>
>>> Date: Thursday, December 10, 2015 at 11:12 PM
>>>
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Re: Queries on the developement for a new custom plugin
>>>
>>> Please find the answer below.
>>>
>>> From: Aruna Sivaram <si...@gmail.com>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Thursday, December 10, 2015 at 8:34 PM
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Re: Queries on the developement for a new custom plugin
>>>
>>> Thanks for your quick response.Please find my queries inline.
>>>
>>>
>>> On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>
>>> wrote:
>>>
>>> If you have created a CustomService, and if you want to authorize the
>>>> access of components/resources in your CustomService, then you need to have
>>>> default authorizer in your CustomService to do authorization, which you
>>>> will extend in your Ranger custom plugin and will be called when your
>>>> CustomService needs authorization check.
>>>>
>>>
>>> [*Aruna] : It would be great if you could provide a snippet/example
>>> where the authoriser code present in the plugin is invoked by the existing
>>> services (hive/hdfs/storm/kafka) for authorisation. This would give me a
>>> better picture as to how it exactly works.*
>>> [RM] Each service dictates how to the register the custom authorizer
>>> with it. So you need to refer the respective service’s authorization
>>> mechanism.
>>>
>>>
>>>> For logging, as you had seen it uses log4j you can have log4j appender
>>>> in the log4.properties and get the log, all hadoop components have their
>>>> log4j properties file where it specify the location it puts the log.
>>>> By default it is /var/log/hadoop/ for hadoop.
>>>>
>>>
>>> *[Aruna] : there are many log4j.properties in the ranger code base and
>>> there is no specific log4j.properties for each of the plugins. Hence, which
>>> log4j.properties do i need to modify. My aim is to see all the logs
>>> generated by my custom plugin. This would enable me to debug the code
>>> better.*
>>> [RM] Here also each service provides it log4j.properties file and that
>>> is where you define appender for ranger also.
>>>
>>> [Aruna ] : The other thing i wanted to know was how does one start the
>>> plugin or rather initiate the plugin which polls for the policies. I see
>>> that the enable-<service>-plugin.sh scripts sets the environment and copies
>>> property files to the right locations, but i dont see where the authoriser
>>> is instantiated in order to invoke the init() method of the
>>> RangerbasePlugin.
>>> [RM] This is already in the wiki page.
>>>
>>>
>>>
>>>> Thanks for all your help
>>>>
>>>
>>> Aruna
>>>
>>>>
>>>> From: Aruna Sivaram <si...@gmail.com>
>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Date: Wednesday, December 9, 2015 at 9:45 PM
>>>> To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Queries on the developement for a new custom plugin
>>>>
>>>> I am using ranger 0.5 for the access control. We are planning to
>>>> develop a custom plugin which we plan to integrate with the ranger
>>>> framework. This custom plugin will be used for access control of our
>>>> components. In order to explore this possibility, i have written a custom
>>>> plugin as per the example given in the link
>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>>>>
>>>> Since this is a custom plugin, my authoriser class
>>>> (CustomServiceAuthoriser), will not be extending any of the hadoop security
>>>> classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
>>>> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
>>>> have created the repository and policies for the same through the rest web
>>>> service.
>>>>
>>>>
>>>> I have written the custom service authoriser as per the example and
>>>> deployed the same on the sandbox. I have a service named CustomService.
>>>> What i wanted to know is how the customservice will communicate with my
>>>> customserviceauthoriser which is contained in my plugin. Currently i dont
>>>> find any documentation which talks about the mode of communication or
>>>> rather how the plugin class will be invoked by the service.
>>>>
>>>> I am relatively new to ranger so may be I am missing something ?
>>>>
>>>> Also, i would like to know the location of the log where each of the
>>>> plugin classes would be logging. This will help us in debugging the flow. I
>>>> see a lot of log statements in the ranger plugin code base but am unable to
>>>> find the location of the logs.
>>>>
>>>> Secondly, can ranger be used to develop custom plugins for access
>>>> control of non hadoop components?
>>>>
>>>> Any help from your end would be appreciated
>>>>
>>>> --
>>>> Regards
>>>> Aruna Sivaram
>>>>
>>>
>>>
>>>
>>> --
>>> Regards
>>> Aruna Sivaram
>>>
>>>
>>
>>
>> --
>> Regards
>> Aruna Sivaram
>>
>
>
>
> --
> Regards
> Aruna Sivaram
>
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Madhan Neethiraj <ma...@apache.org>.
Aruna,
>> 2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns “Denied"
Since there was no policy that allowed the access, the request was denied. Hence the policyId is blank.
>> 3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied”
The audit log clearly says that the access was granted by policyId=656. Can you please send the screenshot of the policy? Another possibility is if your service-def has “ADMIN_QUEUE” as an impliedGrant for “SUBMIT_APP” (as shown below); can you please check if this is the case?
"accessTypes":
[
{
"itemId": 1,
"name": "SUBMIT_APP",
"label": “SUBMIT_APP”,
"impliedGrants":
[
"ADMIN_QUEUE"
]
},
{
"itemId": 2,
"name": “ADMIN_QUEUE",
"label": "ADMIN_QUEUE"
}
]
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 10:47 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Madhan,
Let assume that I have created a custom policy named TestPolicy, Policy Id : 656 for a user called "network3" giving "submit-app" permissions for a queue called "default".
1st case --> , if the same user "network3" does a "submit-app" on the "default" queue, then the audit log shows the correct result with the Result as "Allowed", the isAccessAllowed method returns "Allowed"
2nd case --> if a different user "it3" does a "submit-app" on the "default" queue, then the audit log shows the result as "denied". However, the policy id is blank, is this the right behavior? the isAccessAllowed returns "Denied"
3rd case --> if the user "network3" does a "admin-queue" on the "default" queue, then the audit log shows "allowed", instead of "denied" (I have created a policy giving only "submit-app" permissions on "default" queue to network3 user). In such a scenario, what should be done? Currently my custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed Method.
and following are the values of the RangerAccessRequestImpl object in this scenario.
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
RangerAccessResourceImpl resource = new RangerAccessResourceImpl();
resource.setValue("queue", default);
request.setResource(resource);
request.setAccessType("admin-queue");
request.setUser("network3");
request.setAccessTime(new Date());
request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
request.setAction("admin-queue");
request.setClientIPAddress("10.0.2.15");
RangerAccessResult result = plugin.isAccessAllowed(request);
return result == null ? false : result.getIsAllowed();
In this scenario, the isAccessAllowed returns "Allowed", ideally it should return "denied".
I have attached the screenshot for your reference, which highlights the 3 cases that i have pointed out.
Regards
Aruna
On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Aruna,
>> However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
1. Audit log (in access tab) should include the ID of the policy that allowed the access. Can you please check?
2. What was the return from isAccessAllowed(request)? Allow or Deny?
If above does not help to identify the issue, can you please send the return value from isAccessAllowed(request) and the screenshot of the corresponding audit log?
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 7:07 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Bosco,
Thanks for a very detailed explanation, it has given me a very good perspective on how ranger works/can be used.
Our use case is that we plan to use apache ranger for authorization of a home grown application that is non hadoop in nature. As we are using ranger for the authorization for all the hadoop based components, we plan to use ranger for all custom based applications in order to ensure consistency as well as use the auditing / GUI and various other features of ranger.
Good news is that, I was able to write a custom plugin & custom authoriser in ranger for our service and it is also able to sync successfully (as seen in the plugin tab of the audit GUI). The audit UI (access tab) is also showing the details of the policy that has been accessed.
However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
Thank you once again.
Regards
Aruna
On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org> wrote:
Aruna, can you give more detail on what you are trying to achieve?
I was searching for integration design diagram, but couldn’t find one. We will work on creating one. In the meanwhile, here is the high level.
Ranger plugins run within the component process.
It gives a light java library, which does the following:
Provides method to check access. RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
Pulls policies from Ranger Admin (implicit)
Does auditing (implicit)
Ranger community works closely with the Hadoop component community for writing plugins for Hadoop components. Currently, there are close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN, Storm, Knox and KMS). We are working with the other communities to support more.
The framework is generic and so you can use Ranger to provide access control to your home grown application also. The wiki page https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives how to implement custom stack.
Ranger design philosophy is not to change the authorization of the component, but have the component define an interface and abstract all the actions the component supports. This way the components are free to support any actions they want to and evolve them over the period of time. This also help other security providers to provide similar implementation.
So if you are planning to write a custom plugin, I assume you are having your app and you want to use Ranger to provide the access control and audit. And if that is the case, some of the questions you have asked applies mostly on your side.
log4j: Since Ranger plugin is embedded within the process, it uses the component’s logging framework. So out here, it will be yours
If you want to use Ranger plugin, first you need to create an interface in your application for authorization. We recommend you provide the default/native simple implementation. Ranger will implement the same interface, but Ranger implementation will use the policies from Ranger Admin. You can review the following sample implementations: https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java and https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
Once your interface is defined, then the wiki pages tells how to define a service using JSON. Which can be loaded in Ranger Admin. After that you can create the policies from Ranger Admin UI, the REST APIs for managing polices are automatically possible, the Ranger Admin audit UI will start showing the audits, etc.
Please give your use case, so we can guide you better.
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 11:12 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Aruna Sivaram <si...@gmail.com>.
Madhan,
Let assume that I have created a custom policy named TestPolicy, Policy Id
: 656 for a user called "network3" giving "submit-app" permissions for a
queue called "default".
1st case --> , if the same user "network3" does a "submit-app" on the
"default" queue, then the audit log shows the correct result with the
Result as "Allowed", the isAccessAllowed method returns "Allowed"
2nd case --> if a different user "it3" does a "submit-app" on the "default"
queue, then the audit log shows the result as "denied". However, the policy
id is blank, is this the right behavior? the isAccessAllowed returns
"Denied"
3rd case --> if the user "network3" does a "admin-queue" on the "default"
queue, then the audit log shows "allowed", instead of "denied" (I have
created a policy giving only "submit-app" permissions on "default" queue to
network3 user). In such a scenario, what should be done? Currently my
custom authoriser passes the RangerAccessRequestImpl to the isAccessAllowed
Method.
and following are the values of the RangerAccessRequestImpl object in this
scenario.
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
RangerAccessResourceImpl resource = new
RangerAccessResourceImpl();
resource.setValue("queue", default);
request.setResource(resource);
request.setAccessType("admin-queue");
request.setUser("network3");
request.setAccessTime(new Date());
request.setAccessTime(org.apache.ranger.authorization.utils.StringUtil.getUTCDate());
request.setAction("admin-queue");
request.setClientIPAddress("10.0.2.15");
RangerAccessResult result = plugin.isAccessAllowed(request);
return result == null ? false : result.getIsAllowed();
In this scenario, the isAccessAllowed returns "Allowed", ideally it should
return "denied".
I have attached the screenshot for your reference, which highlights the 3
cases that i have pointed out.
Regards
Aruna
On Mon, Dec 14, 2015 at 10:01 PM, Madhan Neethiraj <ma...@apache.org>
wrote:
> Aruna,
>
> >> However, in cases, where a user cannot access a resource, the audit UI
> (access tab) should show "denied". However, in my case, it stills shows as
> "allowed" instead of "denied". It would be helpful if you could tell me
> what needs to be done in such a case. I am invoking the
> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
> know what needs to be passed to this API when a user is not allowed to
> access a particular resource.
>
> 1. Audit log (in access tab) should include the ID of the policy that
> allowed the access. Can you please check?
> 2. What was the return from isAccessAllowed(request)? Allow or Deny?
>
> If above does not help to identify the issue, can you please send the
> return value from isAccessAllowed(request) and the screenshot of the
> corresponding audit log?
>
> Thanks,
> Madhan
>
> From: Aruna Sivaram <si...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Monday, December 14, 2015 at 7:07 AM
>
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Queries on the developement for a new custom plugin
>
> Bosco,
>
> Thanks for a very detailed explanation, it has given me a very good
> perspective on how ranger works/can be used.
>
> Our use case is that we plan to use apache ranger for authorization of a
> home grown application that is non hadoop in nature. As we are using ranger
> for the authorization for all the hadoop based components, we plan to use
> ranger for all custom based applications in order to ensure consistency as
> well as use the auditing / GUI and various other features of ranger.
>
> Good news is that, I was able to write a custom plugin & custom authoriser
> in ranger for our service and it is also able to sync successfully (as seen
> in the plugin tab of the audit GUI). The audit UI (access tab) is also
> showing the details of the policy that has been accessed.
>
> However, in cases, where a user cannot access a resource, the audit UI
> (access tab) should show "denied". However, in my case, it stills shows as
> "allowed" instead of "denied". It would be helpful if you could tell me
> what needs to be done in such a case. I am invoking the
> isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
> know what needs to be passed to this API when a user is not allowed to
> access a particular resource.
>
>
> Thank you once again.
>
> Regards
> Aruna
>
>
> On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org>
> wrote:
>
>> Aruna, can you give more detail on what you are trying to achieve?
>>
>> I was searching for integration design diagram, but couldn’t find one. We
>> will work on creating one. In the meanwhile, here is the high level.
>>
>>
>> 1. Ranger plugins run within the component process.
>> 2. It gives a light java library, which does the following:
>> 1. Provides method to check access.
>> RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
>> 2. Pulls policies from Ranger Admin (implicit)
>> 3. Does auditing (implicit)
>> 3. Ranger community works closely with the Hadoop component community
>> for writing plugins for Hadoop components. Currently, there are close to 9
>> plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN,
>> Storm, Knox and KMS). We are working with the other communities to support
>> more.
>> 4. The framework is generic and so you can use Ranger to provide
>> access control to your home grown application also. The wiki page
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives
>> how to implement custom stack.
>> 5. Ranger design philosophy is not to change the authorization of the
>> component, but have the component define an interface and abstract all the
>> actions the component supports. This way the components are free to support
>> any actions they want to and evolve them over the period of time. This also
>> help other security providers to provide similar implementation.
>>
>> So if you are planning to write a custom plugin, I assume you are having
>> your app and you want to use Ranger to provide the access control and
>> audit. And if that is the case, some of the questions you have asked
>> applies mostly on your side.
>>
>> 1. log4j: Since Ranger plugin is embedded within the process, it uses
>> the component’s logging framework. So out here, it will be yours
>> 2. If you want to use Ranger plugin, first you need to create an
>> interface in your application for authorization. We recommend you provide
>> the default/native simple implementation. Ranger will implement the same
>> interface, but Ranger implementation will use the policies from Ranger
>> Admin. You can review the following sample implementations:
>> https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>> and
>> https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
>> 3. Once your interface is defined, then the wiki pages tells how to
>> define a service using JSON. Which can be loaded in Ranger Admin. After
>> that you can create the policies from Ranger Admin UI, the REST APIs for
>> managing polices are automatically possible, the Ranger Admin audit UI will
>> start showing the audits, etc.
>>
>> Please give your use case, so we can guide you better.
>>
>> Thanks
>>
>> Bosco
>>
>>
>>
>>
>> From: Ramesh Mani <rm...@hortonworks.com>
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Thursday, December 10, 2015 at 11:12 PM
>>
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: Queries on the developement for a new custom plugin
>>
>> Please find the answer below.
>>
>> From: Aruna Sivaram <si...@gmail.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Thursday, December 10, 2015 at 8:34 PM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: Queries on the developement for a new custom plugin
>>
>> Thanks for your quick response.Please find my queries inline.
>>
>>
>> On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>
>> wrote:
>>
>> If you have created a CustomService, and if you want to authorize the
>>> access of components/resources in your CustomService, then you need to have
>>> default authorizer in your CustomService to do authorization, which you
>>> will extend in your Ranger custom plugin and will be called when your
>>> CustomService needs authorization check.
>>>
>>
>> [*Aruna] : It would be great if you could provide a snippet/example
>> where the authoriser code present in the plugin is invoked by the existing
>> services (hive/hdfs/storm/kafka) for authorisation. This would give me a
>> better picture as to how it exactly works.*
>> [RM] Each service dictates how to the register the custom authorizer
>> with it. So you need to refer the respective service’s authorization
>> mechanism.
>>
>>
>>> For logging, as you had seen it uses log4j you can have log4j appender
>>> in the log4.properties and get the log, all hadoop components have their
>>> log4j properties file where it specify the location it puts the log.
>>> By default it is /var/log/hadoop/ for hadoop.
>>>
>>
>> *[Aruna] : there are many log4j.properties in the ranger code base and
>> there is no specific log4j.properties for each of the plugins. Hence, which
>> log4j.properties do i need to modify. My aim is to see all the logs
>> generated by my custom plugin. This would enable me to debug the code
>> better.*
>> [RM] Here also each service provides it log4j.properties file and that is
>> where you define appender for ranger also.
>>
>> [Aruna ] : The other thing i wanted to know was how does one start the
>> plugin or rather initiate the plugin which polls for the policies. I see
>> that the enable-<service>-plugin.sh scripts sets the environment and copies
>> property files to the right locations, but i dont see where the authoriser
>> is instantiated in order to invoke the init() method of the
>> RangerbasePlugin.
>> [RM] This is already in the wiki page.
>>
>>
>>
>>> Thanks for all your help
>>>
>>
>> Aruna
>>
>>>
>>> From: Aruna Sivaram <si...@gmail.com>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Wednesday, December 9, 2015 at 9:45 PM
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Queries on the developement for a new custom plugin
>>>
>>> I am using ranger 0.5 for the access control. We are planning to develop
>>> a custom plugin which we plan to integrate with the ranger framework. This
>>> custom plugin will be used for access control of our components. In order
>>> to explore this possibility, i have written a custom plugin as per the
>>> example given in the link
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>>>
>>> Since this is a custom plugin, my authoriser class
>>> (CustomServiceAuthoriser), will not be extending any of the hadoop security
>>> classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
>>> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
>>> have created the repository and policies for the same through the rest web
>>> service.
>>>
>>>
>>> I have written the custom service authoriser as per the example and
>>> deployed the same on the sandbox. I have a service named CustomService.
>>> What i wanted to know is how the customservice will communicate with my
>>> customserviceauthoriser which is contained in my plugin. Currently i dont
>>> find any documentation which talks about the mode of communication or
>>> rather how the plugin class will be invoked by the service.
>>>
>>> I am relatively new to ranger so may be I am missing something ?
>>>
>>> Also, i would like to know the location of the log where each of the
>>> plugin classes would be logging. This will help us in debugging the flow. I
>>> see a lot of log statements in the ranger plugin code base but am unable to
>>> find the location of the logs.
>>>
>>> Secondly, can ranger be used to develop custom plugins for access
>>> control of non hadoop components?
>>>
>>> Any help from your end would be appreciated
>>>
>>> --
>>> Regards
>>> Aruna Sivaram
>>>
>>
>>
>>
>> --
>> Regards
>> Aruna Sivaram
>>
>>
>
>
> --
> Regards
> Aruna Sivaram
>
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Madhan Neethiraj <ma...@apache.org>.
Aruna,
>> However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
1. Audit log (in access tab) should include the ID of the policy that allowed the access. Can you please check?
2. What was the return from isAccessAllowed(request)? Allow or Deny?
If above does not help to identify the issue, can you please send the return value from isAccessAllowed(request) and the screenshot of the corresponding audit log?
Thanks,
Madhan
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, December 14, 2015 at 7:07 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Bosco,
Thanks for a very detailed explanation, it has given me a very good perspective on how ranger works/can be used.
Our use case is that we plan to use apache ranger for authorization of a home grown application that is non hadoop in nature. As we are using ranger for the authorization for all the hadoop based components, we plan to use ranger for all custom based applications in order to ensure consistency as well as use the auditing / GUI and various other features of ranger.
Good news is that, I was able to write a custom plugin & custom authoriser in ranger for our service and it is also able to sync successfully (as seen in the plugin tab of the audit GUI). The audit UI (access tab) is also showing the details of the policy that has been accessed.
However, in cases, where a user cannot access a resource, the audit UI (access tab) should show "denied". However, in my case, it stills shows as "allowed" instead of "denied". It would be helpful if you could tell me what needs to be done in such a case. I am invoking the isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me know what needs to be passed to this API when a user is not allowed to access a particular resource.
Thank you once again.
Regards
Aruna
On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org> wrote:
Aruna, can you give more detail on what you are trying to achieve?
I was searching for integration design diagram, but couldn’t find one. We will work on creating one. In the meanwhile, here is the high level.
Ranger plugins run within the component process.
It gives a light java library, which does the following:
Provides method to check access. RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
Pulls policies from Ranger Admin (implicit)
Does auditing (implicit)
Ranger community works closely with the Hadoop component community for writing plugins for Hadoop components. Currently, there are close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN, Storm, Knox and KMS). We are working with the other communities to support more.
The framework is generic and so you can use Ranger to provide access control to your home grown application also. The wiki page https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives how to implement custom stack.
Ranger design philosophy is not to change the authorization of the component, but have the component define an interface and abstract all the actions the component supports. This way the components are free to support any actions they want to and evolve them over the period of time. This also help other security providers to provide similar implementation.
So if you are planning to write a custom plugin, I assume you are having your app and you want to use Ranger to provide the access control and audit. And if that is the case, some of the questions you have asked applies mostly on your side.
log4j: Since Ranger plugin is embedded within the process, it uses the component’s logging framework. So out here, it will be yours
If you want to use Ranger plugin, first you need to create an interface in your application for authorization. We recommend you provide the default/native simple implementation. Ranger will implement the same interface, but Ranger implementation will use the policies from Ranger Admin. You can review the following sample implementations: https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java and https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
Once your interface is defined, then the wiki pages tells how to define a service using JSON. Which can be loaded in Ranger Admin. After that you can create the policies from Ranger Admin UI, the REST APIs for managing polices are automatically possible, the Ranger Admin audit UI will start showing the audits, etc.
Please give your use case, so we can guide you better.
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 11:12 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Aruna Sivaram <si...@gmail.com>.
Bosco,
Thanks for a very detailed explanation, it has given me a very good
perspective on how ranger works/can be used.
Our use case is that we plan to use apache ranger for authorization of a
home grown application that is non hadoop in nature. As we are using ranger
for the authorization for all the hadoop based components, we plan to use
ranger for all custom based applications in order to ensure consistency as
well as use the auditing / GUI and various other features of ranger.
Good news is that, I was able to write a custom plugin & custom authoriser
in ranger for our service and it is also able to sync successfully (as seen
in the plugin tab of the audit GUI). The audit UI (access tab) is also
showing the details of the policy that has been accessed.
However, in cases, where a user cannot access a resource, the audit UI
(access tab) should show "denied". However, in my case, it stills shows as
"allowed" instead of "denied". It would be helpful if you could tell me
what needs to be done in such a case. I am invoking the
isAccessAllowed(<RangerAccessRequest>) from my custom authoriser. Let me
know what needs to be passed to this API when a user is not allowed to
access a particular resource.
Thank you once again.
Regards
Aruna
On Fri, Dec 11, 2015 at 11:41 PM, Don Bosco Durai <bo...@apache.org> wrote:
> Aruna, can you give more detail on what you are trying to achieve?
>
> I was searching for integration design diagram, but couldn’t find one. We
> will work on creating one. In the meanwhile, here is the high level.
>
>
> 1. Ranger plugins run within the component process.
> 2. It gives a light java library, which does the following:
> 1. Provides method to check access.
> RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
> 2. Pulls policies from Ranger Admin (implicit)
> 3. Does auditing (implicit)
> 3. Ranger community works closely with the Hadoop component community
> for writing plugins for Hadoop components. Currently, there are close to 9
> plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN,
> Storm, Knox and KMS). We are working with the other communities to support
> more.
> 4. The framework is generic and so you can use Ranger to provide
> access control to your home grown application also. The wiki page
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives
> how to implement custom stack.
> 5. Ranger design philosophy is not to change the authorization of the
> component, but have the component define an interface and abstract all the
> actions the component supports. This way the components are free to support
> any actions they want to and evolve them over the period of time. This also
> help other security providers to provide similar implementation.
>
> So if you are planning to write a custom plugin, I assume you are having
> your app and you want to use Ranger to provide the access control and
> audit. And if that is the case, some of the questions you have asked
> applies mostly on your side.
>
> 1. log4j: Since Ranger plugin is embedded within the process, it uses
> the component’s logging framework. So out here, it will be yours
> 2. If you want to use Ranger plugin, first you need to create an
> interface in your application for authorization. We recommend you provide
> the default/native simple implementation. Ranger will implement the same
> interface, but Ranger implementation will use the policies from Ranger
> Admin. You can review the following sample implementations:
> https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
> and
> https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
> 3. Once your interface is defined, then the wiki pages tells how to
> define a service using JSON. Which can be loaded in Ranger Admin. After
> that you can create the policies from Ranger Admin UI, the REST APIs for
> managing polices are automatically possible, the Ranger Admin audit UI will
> start showing the audits, etc.
>
> Please give your use case, so we can guide you better.
>
> Thanks
>
> Bosco
>
>
>
>
> From: Ramesh Mani <rm...@hortonworks.com>
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Thursday, December 10, 2015 at 11:12 PM
>
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Queries on the developement for a new custom plugin
>
> Please find the answer below.
>
> From: Aruna Sivaram <si...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Thursday, December 10, 2015 at 8:34 PM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Queries on the developement for a new custom plugin
>
> Thanks for your quick response.Please find my queries inline.
>
>
> On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>
> wrote:
>
> If you have created a CustomService, and if you want to authorize the
>> access of components/resources in your CustomService, then you need to have
>> default authorizer in your CustomService to do authorization, which you
>> will extend in your Ranger custom plugin and will be called when your
>> CustomService needs authorization check.
>>
>
> [*Aruna] : It would be great if you could provide a snippet/example where
> the authoriser code present in the plugin is invoked by the existing
> services (hive/hdfs/storm/kafka) for authorisation. This would give me a
> better picture as to how it exactly works.*
> [RM] Each service dictates how to the register the custom authorizer with
> it. So you need to refer the respective service’s authorization mechanism.
>
>
>> For logging, as you had seen it uses log4j you can have log4j appender in
>> the log4.properties and get the log, all hadoop components have their log4j
>> properties file where it specify the location it puts the log.
>> By default it is /var/log/hadoop/ for hadoop.
>>
>
> *[Aruna] : there are many log4j.properties in the ranger code base and
> there is no specific log4j.properties for each of the plugins. Hence, which
> log4j.properties do i need to modify. My aim is to see all the logs
> generated by my custom plugin. This would enable me to debug the code
> better.*
> [RM] Here also each service provides it log4j.properties file and that is
> where you define appender for ranger also.
>
> [Aruna ] : The other thing i wanted to know was how does one start the
> plugin or rather initiate the plugin which polls for the policies. I see
> that the enable-<service>-plugin.sh scripts sets the environment and copies
> property files to the right locations, but i dont see where the authoriser
> is instantiated in order to invoke the init() method of the
> RangerbasePlugin.
> [RM] This is already in the wiki page.
>
>
>
>> Thanks for all your help
>>
>
> Aruna
>
>>
>> From: Aruna Sivaram <si...@gmail.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Wednesday, December 9, 2015 at 9:45 PM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Queries on the developement for a new custom plugin
>>
>> I am using ranger 0.5 for the access control. We are planning to develop
>> a custom plugin which we plan to integrate with the ranger framework. This
>> custom plugin will be used for access control of our components. In order
>> to explore this possibility, i have written a custom plugin as per the
>> example given in the link
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>>
>> Since this is a custom plugin, my authoriser class
>> (CustomServiceAuthoriser), will not be extending any of the hadoop security
>> classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
>> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
>> have created the repository and policies for the same through the rest web
>> service.
>>
>>
>> I have written the custom service authoriser as per the example and
>> deployed the same on the sandbox. I have a service named CustomService.
>> What i wanted to know is how the customservice will communicate with my
>> customserviceauthoriser which is contained in my plugin. Currently i dont
>> find any documentation which talks about the mode of communication or
>> rather how the plugin class will be invoked by the service.
>>
>> I am relatively new to ranger so may be I am missing something ?
>>
>> Also, i would like to know the location of the log where each of the
>> plugin classes would be logging. This will help us in debugging the flow. I
>> see a lot of log statements in the ranger plugin code base but am unable to
>> find the location of the logs.
>>
>> Secondly, can ranger be used to develop custom plugins for access control
>> of non hadoop components?
>>
>> Any help from your end would be appreciated
>>
>> --
>> Regards
>> Aruna Sivaram
>>
>
>
>
> --
> Regards
> Aruna Sivaram
>
>
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Don Bosco Durai <bo...@apache.org>.
Aruna, can you give more detail on what you are trying to achieve?
I was searching for integration design diagram, but couldn’t find one. We will work on creating one. In the meanwhile, here is the high level.
Ranger plugins run within the component process.
It gives a light java library, which does the following:
Provides method to check access. RangerBasePlugin.isAccessAllow() (explicit, you have to call it)
Pulls policies from Ranger Admin (implicit)
Does auditing (implicit)
Ranger community works closely with the Hadoop component community for writing plugins for Hadoop components. Currently, there are close to 9 plugins available as part of Ranger (HDFS, Hive, Hbase, Kafka, Solr, YARN, Storm, Knox and KMS). We are working with the other communities to support more.
The framework is generic and so you can use Ranger to provide access control to your home grown application also. The wiki page https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 gives how to implement custom stack.
Ranger design philosophy is not to change the authorization of the component, but have the component define an interface and abstract all the actions the component supports. This way the components are free to support any actions they want to and evolve them over the period of time. This also help other security providers to provide similar implementation.
So if you are planning to write a custom plugin, I assume you are having your app and you want to use Ranger to provide the access control and audit. And if that is the case, some of the questions you have asked applies mostly on your side.
log4j: Since Ranger plugin is embedded within the process, it uses the component’s logging framework. So out here, it will be yours
If you want to use Ranger plugin, first you need to create an interface in your application for authorization. We recommend you provide the default/native simple implementation. Ranger will implement the same interface, but Ranger implementation will use the policies from Ranger Admin. You can review the following sample implementations: https://github.com/apache/incubator-ranger/blob/master/plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java and https://github.com/apache/incubator-ranger/blob/master/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
Once your interface is defined, then the wiki pages tells how to define a service using JSON. Which can be loaded in Ranger Admin. After that you can create the policies from Ranger Admin UI, the REST APIs for managing polices are automatically possible, the Ranger Admin audit UI will start showing the audits, etc.
Please give your use case, so we can guide you better.
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 11:12 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Ramesh Mani <rm...@hortonworks.com>.
Please find the answer below.
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Thursday, December 10, 2015 at 8:34 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Queries on the developement for a new custom plugin
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com>> wrote:
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
[Aruna] : It would be great if you could provide a snippet/example where the authoriser code present in the plugin is invoked by the existing services (hive/hdfs/storm/kafka) for authorisation. This would give me a better picture as to how it exactly works.
[RM] Each service dictates how to the register the custom authorizer with it. So you need to refer the respective service’s authorization mechanism.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
[Aruna] : there are many log4j.properties in the ranger code base and there is no specific log4j.properties for each of the plugins. Hence, which log4j.properties do i need to modify. My aim is to see all the logs generated by my custom plugin. This would enable me to debug the code better.
[RM] Here also each service provides it log4j.properties file and that is where you define appender for ranger also.
[Aruna ] : The other thing i wanted to know was how does one start the plugin or rather initiate the plugin which polls for the policies. I see that the enable-<service>-plugin.sh scripts sets the environment and copies property files to the right locations, but i dont see where the authoriser is instantiated in order to invoke the init() method of the RangerbasePlugin.
[RM] This is already in the wiki page.
Thanks for all your help
Aruna
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Aruna Sivaram <si...@gmail.com>.
Thanks for your quick response.Please find my queries inline.
On Fri, Dec 11, 2015 at 4:55 AM, Ramesh Mani <rm...@hortonworks.com> wrote:
If you have created a CustomService, and if you want to authorize the
> access of components/resources in your CustomService, then you need to have
> default authorizer in your CustomService to do authorization, which you
> will extend in your Ranger custom plugin and will be called when your
> CustomService needs authorization check.
>
[*Aruna] : It would be great if you could provide a snippet/example where
the authoriser code present in the plugin is invoked by the existing
services (hive/hdfs/storm/kafka) for authorisation. This would give me a
better picture as to how it exactly works.*
> For logging, as you had seen it uses log4j you can have log4j appender in
> the log4.properties and get the log, all hadoop components have their log4j
> properties file where it specify the location it puts the log.
> By default it is /var/log/hadoop/ for hadoop.
>
*[Aruna] : there are many log4j.properties in the ranger code base and
there is no specific log4j.properties for each of the plugins. Hence, which
log4j.properties do i need to modify. My aim is to see all the logs
generated by my custom plugin. This would enable me to debug the code
better.*
[Aruna ] : The other thing i wanted to know was how does one start the
plugin or rather initiate the plugin which polls for the policies. I see
that the enable-<service>-plugin.sh scripts sets the environment and copies
property files to the right locations, but i dont see where the authoriser
is instantiated in order to invoke the init() method of the
RangerbasePlugin.
> Thanks for all your help
>
Aruna
>
> From: Aruna Sivaram <si...@gmail.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Wednesday, December 9, 2015 at 9:45 PM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Queries on the developement for a new custom plugin
>
> I am using ranger 0.5 for the access control. We are planning to develop a
> custom plugin which we plan to integrate with the ranger framework. This
> custom plugin will be used for access control of our components. In order
> to explore this possibility, i have written a custom plugin as per the
> example given in the link
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>
> Since this is a custom plugin, my authoriser class
> (CustomServiceAuthoriser), will not be extending any of the hadoop security
> classes (Eg in case of storm or hive, the RangerStormAuthoriser implements
> IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I
> have created the repository and policies for the same through the rest web
> service.
>
>
> I have written the custom service authoriser as per the example and
> deployed the same on the sandbox. I have a service named CustomService.
> What i wanted to know is how the customservice will communicate with my
> customserviceauthoriser which is contained in my plugin. Currently i dont
> find any documentation which talks about the mode of communication or
> rather how the plugin class will be invoked by the service.
>
> I am relatively new to ranger so may be I am missing something ?
>
> Also, i would like to know the location of the log where each of the
> plugin classes would be logging. This will help us in debugging the flow. I
> see a lot of log statements in the ranger plugin code base but am unable to
> find the location of the logs.
>
> Secondly, can ranger be used to develop custom plugins for access control
> of non hadoop components?
>
> Any help from your end would be appreciated
>
> --
> Regards
> Aruna Sivaram
>
--
Regards
Aruna Sivaram
Re: Queries on the developement for a new custom plugin
Posted by Ramesh Mani <rm...@hortonworks.com>.
If you have created a CustomService, and if you want to authorize the access of components/resources in your CustomService, then you need to have default authorizer in your CustomService to do authorization, which you will extend in your Ranger custom plugin and will be called when your CustomService needs authorization check.
For logging, as you had seen it uses log4j you can have log4j appender in the log4.properties and get the log, all hadoop components have their log4j properties file where it specify the location it puts the log.
By default it is /var/log/hadoop/ for hadoop.
From: Aruna Sivaram <si...@gmail.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Wednesday, December 9, 2015 at 9:45 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Queries on the developement for a new custom plugin
I am using ranger 0.5 for the access control. We are planning to develop a custom plugin which we plan to integrate with the ranger framework. This custom plugin will be used for access control of our components. In order to explore this possibility, i have written a custom plugin as per the example given in the link https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
Since this is a custom plugin, my authoriser class (CustomServiceAuthoriser), will not be extending any of the hadoop security classes (Eg in case of storm or hive, the RangerStormAuthoriser implements IAuthorizer and RangerYarnAuthorizer extends YarnAuthorizationProvider. I have created the repository and policies for the same through the rest web service.
I have written the custom service authoriser as per the example and deployed the same on the sandbox. I have a service named CustomService. What i wanted to know is how the customservice will communicate with my customserviceauthoriser which is contained in my plugin. Currently i dont find any documentation which talks about the mode of communication or rather how the plugin class will be invoked by the service.
I am relatively new to ranger so may be I am missing something ?
Also, i would like to know the location of the log where each of the plugin classes would be logging. This will help us in debugging the flow. I see a lot of log statements in the ranger plugin code base but am unable to find the location of the logs.
Secondly, can ranger be used to develop custom plugins for access control of non hadoop components?
Any help from your end would be appreciated
--
Regards
Aruna Sivaram