You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by rohit sinha <ta...@gmail.com> on 2016/04/06 19:47:30 UTC
Integrating new services with Ranger
Hello,
I am looking into integrating an external service with Apache Ranger for
authorization.
I looked up the wiki but there is no information about integrating new
services.
Can someone give me some info which might be helpful in identifying
different components which needs to be developed and other required stuff
Thanks.
Re: Integrating new services with Ranger
Posted by Ramesh Mani <rm...@hortonworks.com>.
Rohit,
Please check the the folder where you have the security.xml is in the
class path. This might be your issue
Regards,
Ramesh
On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello Madhan,
>Thanks a lot for your reply.
>
>I am looking into the integration and I working towards developing the
>components outside of ranger trunk for initial development purpose. After
>taking an overview of the codebase it seems that it's possible to have
>plugins outside of the ranger trunk too.
>
>I was able to add my service to Ranger Amin through the curl request and
>connect to Ranger from my Authorizer but I see the following error:
>
>com.sun.jersey.api.client.ClientHandlerException:
>> java.lang.IllegalArgumentException: URI is not absolute
>> at
>>
>>com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URL
>>ConnectionClientHandler.java:151)
>> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>> at
>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> at
>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> at
>> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> at
>>
>>org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIf
>>Updated(RangerAdminRESTClient.java:94)
>> at
>>
>>org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(P
>>olicyRefresher.java:215)
>> at
>>
>>org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.
>>java:183)
>> at
>>
>>org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:15
>>6)
>> Caused by: java.lang.IllegalArgumentException: URI is not absolute
>> at java.net.URI.toURL(URI.java:1095)
>> at
>>
>>com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(UR
>>LConnectionClientHandler.java:159)
>> at
>>
>>com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URL
>>ConnectionClientHandler.java:149)
>> ... 8 more
>> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not exist
>>or
>> not readble 'null'
>
>
>A quick search on google pointed towards misconfiguration of service name
>in security.xml which I have double checked.
>
>Any pointers to debug this will be appreciated.
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>mneethiraj@hortonworks.com> wrote:
>
>> Rohit,
>>
>> You are right. REPOSITORY_NAME referenced in the doc is the name of the
>> service instance in Ranger Admin, which contains the policies for the
>> component (in this case HBase). The plugin reads this value from a
>> configuration named ranger.plugin.hbase.service.name (in file
>> ranger-hbase-security.xml).
>>
>> >> After doing this I don't see anything in the Audit -> Plugins
>> An entry will be created here for every policy download form plugins.
>> Plugins download the policies at the following events:
>> - during the component startup (HBase/HiveServer/...)
>> - when there is a policy change in service instance
>>
>> Hope this helps.
>>
>> Madhan
>>
>>
>>
>> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Thanks for sharing the SampleApp. I was able to run it understand the
>> >integration point.
>> >I also tried to enable HBase ranger plugin. When I added the service
>>from
>> >the Ranger Admin UI I was able to talk to HBase and the resource
>> completion
>> >worked. After that I deleted the HBase plugin from the UI and tried to
>> >enable it from the the command line following instructions mentioned
>>here:
>> >
>>
>>https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+In
>>stallation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >
>> >After doing this I don't see anything in the Audit -> Plugins. In the
>> >instructions I do see warning
>> >
>> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not,
>> the
>> >> hbase-plugin will not be able to communicate with Ranger admin.
>> >
>> >Does this mean I need to add the service from the Ranger Admin Panel
>>or I
>> >did something wrong in the enabling the plugin ?
>> >
>> >Any help will be highly appreciated.
>> >
>> >Thanks.
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
>> >wrote:
>> >
>> >> Awesome. Thanks a lot.
>> >>
>> >> Thanks,
>> >> Rohit Sinha
>> >>
>> >>
>> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>> >>
>> >>> It is optional. It is easy to setup and helps a lot while debugging
>> >>> during initial setup.
>> >>>
>> >>> Bosco
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>> >>>
>> >>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
>>helpful
>> in
>> >>> >getting some understanding of the integration. I am going through
>>the
>> >>> >SampleApp docs now.
>> >>> >
>> >>> >Is auditing an optional feature ? From the documentation it looks
>>like
>> >>> it's
>> >>> >not and Solr installation is a requirement. I was wondering if I
>>can
>> have
>> >>> >auditing off and skip Solr installation for initial integration
>> purpose.
>> >>> >
>> >>> >Thanks.
>> >>> >
>> >>> >Thanks,
>> >>> >Rohit Sinha
>> >>> >
>> >>> >
>> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj
>><ma...@apache.org>
>> >>> wrote:
>> >>> >
>> >>> >> Rohit,
>> >>> >>
>> >>> >> In addition to the details in the wiki, I would recommend
>>reviewing
>> the
>> >>> >> following sample application to understand the details of adding
>> Ranger
>> >>> >> authorization to an application.
>> >>> >>
>> >>> >> - README.txt:
>> >>> >>
>> >>>
>>
>>https://github.com/apache/incubator-ranger/blob/master/ranger-examples/RE
>>ADME.txt
>> >>> >> - Application sources:
>> >>> >>
>> >>>
>>
>>https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sa
>>mpleapp
>> >>> >> .
>> >>> >>
>> >>> >> Madhan
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >>> vperiasamy@hortonworks.com
>> >>> >> on behalf of vel@apache.org> wrote:
>> >>> >>
>> >>> >> >Rohit -
>> >>> >> >
>> >>>
>>
>>https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >>> >> >explains how to add a custom plugin for Ranger.
>> >>> >> >
>> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
>> wrote:
>> >>> >> >
>> >>> >> >>Hello,
>> >>> >> >>I am looking into integrating an external service with Apache
>> Ranger
>> >>> for
>> >>> >> >>authorization.
>> >>> >> >>I looked up the wiki but there is no information about
>>integrating
>> >>> new
>> >>> >> >>services.
>> >>> >> >>Can someone give me some info which might be helpful in
>> identifying
>> >>> >> >>different components which needs to be developed and other
>> required
>> >>> stuff
>> >>> >> >>
>> >>> >> >>Thanks.
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
So, the issue which was causing the connection issue with the following
error:
2016-04-10 05:54:27,608 [timed-executor-pool-0] INFO
> org.apache.ranger.plugin.client.BaseClient (BaseClient.
> java:104) - Init Login: security not enabled, using username
> 2016-04-10 05:54:27,610 [http-bio-6080-exec-8] ERROR org.apache.ranger.biz
> .ServiceMgr (ServiceMgr.java:129) -
> ==> ServiceMgr.validateConfig Error:org.apache.ranger.plugin.client.HadoopException:
> java.lang.IllegalAccess
> Error: tried to access method com.google.common.base.Stopwatch.<init>()V
> from class com.example.app.security.authorization.ranger.lookup.
> client.Client
was due to multiple guava dependency in ranger-admin classpath. My lookup
jar was packing guava 11 which was causing the issue. I got it resolved
now.
If someone can point me to the documentation which explains the different
properties in the service definition json that will great.
Thanks.
Thanks,
Rohit Sinha
On Sat, Apr 9, 2016 at 11:14 PM, rohit sinha <ta...@gmail.com>
wrote:
> Hello,
>
> Thanks for sharing the details. I understand how the policy request are
> served. My concern was that since there is no mechanism to ensure that the
> service making authorization request is a legal service anyone can make
> authorization request to check authorization for different users on
> different resources (with the correct service name and resource hierarchy).
> I am not sure if this is a big issue or not since the auth request are
> basically read only.
>
> Anyways, I have been working on the integration and I am trying to make
> the resource lookup work. I have implemented the testConnection part in my
> implementation for RangerBaseService.
>
> My test connection works sometimes and other times it randomly throws the
> following exception:
>
> 2016-04-10 05:54:27,608 [timed-executor-pool-0] INFO
>> org.apache.ranger.plugin.client.BaseClient (BaseClient.
>> java:104) - Init Login: security not enabled, using username
>> 2016-04-10 05:54:27,610 [http-bio-6080-exec-8] ERROR
>> org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:129) -
>> ==> ServiceMgr.validateConfig
>> Error:org.apache.ranger.plugin.client.HadoopException:
>> java.lang.IllegalAccess
>> Error: tried to access method com.google.common.base.Stopwatch.<init>()V
>> from class
>> com.example.app.security.authorization.ranger.lookup.client.Client
>
>
> I see this in the UI when the testConnection fails and also in xa_portal
> logs. So, far I have not been able to identify when the failure happens.
> Any idea what can be causing this ?
>
> Also, can you point me to the documentation which explains the different
> properties in the service definition json ? I don't understand few of the
> json elements like what does recursiveSupported actually does ?
>
> Thanks.
>
> Thanks,
> Rohit Sinha
>
>
> On Fri, Apr 8, 2016 at 6:32 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Rohit,
>>
>> Ranger plugins download policies from Ranger Admin using the serviceName.
>> The plugins must be configured with the correct name of the service
>> containing
>> policies to enforce (refer to earlier discussion in this email on
>> configuration
>> ranger.plugin.<pluginType>.service.name).
>>
>> If a plugin is configured with incorrect serviceName (for example a HBase
>> plugin
>> configured with HDFS serviceName), it will end up failing/denying all
>> Authorization requests - as the resources for each service are different
>> i.e.
>>
>> Resources for HBase: { table/column-family/column }
>> Resources for HDFS: { path }
>>
>> Madhan
>>
>>
>>
>> On 4/8/16, 6:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Hello Madhan,
>> >Thanks a lot for the prompt reply.
>> >This answers how does ranger authenticate itself to the service where it
>> >wants to do lookup. I am still confused about the other part.
>> >
>> >If I have HBase and HDFS policy how does Ranger makes sure that the
>> service
>> >talking to ranger is one and not another for performing authorization
>> >checks ?
>> >
>> >Thanks.
>> >
>> >On Friday, April 8, 2016, Madhan Neethiraj <ma...@apache.org> wrote:
>> >
>> >> Rohit,
>> >>
>> >> Your question at the end is the answer you are looking for!
>> >>
>> >> The username and password given while adding a service is used by
>> Ranger
>> >> Admin
>> >> while performing lookup. In case of secure cluster, the username
>> should be
>> >> the
>> >> Kerberos principal. This user must have enough permissions in the
>> component
>> >> (via a Ranger policy?) to perform the lookup.
>> >>
>> >>
>> >>
>> >>
>> >> Hope this help.
>> >>
>> >> Madhan
>> >>
>> >> On 4/8/16, 5:31 PM, "rohit sinha" <talktorohit54@gmail.com
>> <javascript:;>>
>> >> wrote:
>> >>
>> >> >Hello Madhan,
>> >> >
>> >> >I am having some difficulty understanding how ranger and services
>> identify
>> >> >each other on a secure cluster.
>> >> >
>> >> >For example, if I have a resource lookup plugin running in ranger for
>> >> HBase
>> >> >and a ranger authorization plugin running in HBase and when they make
>> >> >request to each other how does ranger make sure that the talking
>> service
>> >> in
>> >> >HBase and also how does HBase identifies the talking service in
>> ranger.
>> >> >
>> >> >Also, for every service we take username and password while the
>> service is
>> >> >being added. What is it used for ?
>> >> >
>> >> >Thanks.
>> >> >
>> >> >
>> >> >Thanks,
>> >> >Rohit Sinha
>> >> >
>> >> >
>> >> >On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <madhan@apache.org
>> >> <javascript:;>> wrote:
>> >> >
>> >> >> Rohit,
>> >> >>
>> >> >> >> 1. Can you point me to some resource where I can see how to
>> configure
>> >> >> Ranger on Kerberos without KMS.
>> >> >> Ranger configuration does not depend on presence or absence of KMS.
>> It
>> >> will
>> >> >> help if you can provide more details on what you are trying to do or
>> >> what
>> >> >> difference/difficulty you see due to presence/absence of KMS.
>> >> >>
>> >> >> >> 2. Also, how can I list all the policies for a resource in my
>> service
>> >> >>
>> >> >>
>> >> >>
>> >> >> RangerBasePlugin is designed for authorizing accesses and don’t
>> provide
>> >> >> APIs to
>> >> >> search the policies. You can use REST APIs of Ranger Admin to search
>> >> >> policies.
>> >> >>
>> >> >>
>> >> >> For example: curl -f -X GET -H "Accept: application/json" -u
>> >> >> admin-user:admin-pass
>> >> >>
>> >>
>> http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
>> >> >>
>> >> >> For details of the REST APIs, please refer to:
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
>> >> >>
>> >> >> Thanks,
>> >> >> Madhan
>> >> >>
>> >> >> On 4/7/16, 9:28 PM, "rohit sinha" <talktorohit54@gmail.com
>> >> <javascript:;>> wrote:
>> >> >>
>> >> >> >Hello Madhan,
>> >> >> >Thanks a lot for the prompt reply.
>> >> >> >
>> >> >> >1. Can you point me to some resource where I can see how to
>> configure
>> >> >> >Ranger on Kerberos without KMS.
>> >> >> >
>> >> >> >2. Also, how can I list all the policies for a resource in my
>> service.
>> >> I
>> >> >> >don't see any api exposed by RangerBasePlugin for this. I was
>> looking
>> >> into
>> >> >> >the code base and saw that ServiceStore has some APIs for this. How
>> >> can I
>> >> >> >access this ?
>> >> >> >
>> >> >> >Thanks.
>> >> >> >
>> >> >> >Thanks,
>> >> >> >Rohit Sinha
>> >> >> >
>> >> >> >
>> >> >> >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <
>> madhan@apache.org
>> >> <javascript:;>>
>> >> >> wrote:
>> >> >> >
>> >> >> >> Rohit,
>> >> >> >>
>> >> >> >> Thanks for sharing your feedback on Ranger integration. Good to
>> know
>> >> >> >> that you are able to get Ranger authorization working in your
>> >> >> application.
>> >> >> >>
>> >> >> >> >> 1. Is it possible to have Ranger running on an unencrypted
>> HDFS
>> >> >> >> >> with secure Hadoop through Kerberos ?
>> >> >> >> Yes. Ranger works with secure Hadoop - either unencrypted or
>> >> encrypted.
>> >> >> >>
>> >> >> >> >> 2. Currently, I see the following error log for policy cache
>> file.
>> >> >> >> >> Isn’t the policy cache file created automatically ?
>> >> >> >> Yes, the policy cache file is created automatically. I guess the
>> >> error
>> >> >> >>
>> >> >> >> you see could be cause by non-existent directory where the cache
>> file
>> >> >> >> would be saved - in this case
>> /etc/ranger/myservicedev/policycache/.
>> >> >> >> Please ensure that this directory exists and has write
>> permission for
>> >> >> >> the user that runs the application.
>> >> >> >>
>> >> >> >> The directory location can be specified via configuration
>> >> >> >> ranger.plugin.myservice.policy.cache.dir in
>> >> >> ranger-myservice-security.xml.
>> >> >> >>
>> >> >> >> Hope this helps.
>> >> >> >>
>> >> >> >> Madhan
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> On 4/7/16, 7:38 PM, "rohit sinha" <talktorohit54@gmail.com
>> >> <javascript:;>> wrote:
>> >> >> >>
>> >> >> >> >Hello,
>> >> >> >> >
>> >> >> >> >Thanks a lot for your prompt replies. It was really helpful.
>> >> >> >> >Yes, it seems like there was some misconfiguration issue. I was
>> >> able to
>> >> >> >> >make it work and I have a very basic integration up and running
>> >> with my
>> >> >> >> >service now. I have worked on integration/analysis of other
>> >> >> authorization
>> >> >> >> >models and I must say that Ranger so far has been the easiest to
>> >> >> integrate
>> >> >> >> >with (at the basic level) and the documentation helped a lot.
>> >> >> Appreciate
>> >> >> >> >your efforts in building it so far.
>> >> >> >> >
>> >> >> >> >I have two question which I am trying to figure out. Will
>> >> appreciate if
>> >> >> >> >someone can provide some pointers.
>> >> >> >> >
>> >> >> >> >1. Kerberos: From the documentation found here
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
>> >> >> >> >I understand that Ranger works with KMS on an encrypted HDFS.
>> Is it
>> >> >> >> >possible to have Ranger running on an unencrypted HDFS with
>> secure
>> >> >> Hadoop
>> >> >> >> >through Kerberos ? Something where I can talk to Ranger by
>> giving a
>> >> >> >> >principal and keytab ?
>> >> >> >> >
>> >> >> >> >2. Currently, I see the following error log for policy cache
>> file.
>> >> >> Isn't
>> >> >> >> >the policy cache file created automatically ? Am I missing some
>> >> >> >> >configuration.
>> >> >> >> >
>> >> >> >> >16/04/07 21:26:18 INFO util.PolicyRefresher:
>> >> >> >> >> PolicyRefresher(serviceName=myservice): found updated version.
>> >> >> >> >> lastKnownVersion=-1; newVersion=1
>> >> >> >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save
>> >> >> policies to
>> >> >> >> >> cache file
>> >> >> >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
>> >> >> >> >> java.io.FileNotFoundException:
>> >> >> >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json
>> (No
>> >> >> such
>> >> >> >> file
>> >> >> >> >> or directory)
>> >> >> >> >> at java.io.FileOutputStream.open(Native Method)
>> >> >> >> >> at
>> >> java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> >> >> >> >> at
>> >> java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> >> >> >> >> at java.io.FileWriter.<init>(FileWriter.java:90)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
>> >> >> >> >> .... internal stacktrace....
>> >> >> >> >> at java.lang.Thread.run(Thread.java:745)
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >Thanks.
>> >> >> >> >
>> >> >> >> >Thanks,
>> >> >> >> >Rohit Sinha
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <
>> >> madhan@apache.org <javascript:;>>
>> >> >> >> wrote:
>> >> >> >> >
>> >> >> >> >> Rohit,
>> >> >> >> >>
>> >> >> >> >> To download policies from Ranger Admin, Ranger plugins
>> require the
>> >> >> URL
>> >> >> >> to
>> >> >> >> >> Ranger Admin and the name of the service containing the
>> policies.
>> >> >> These
>> >> >> >> >> values are read from following configurations from a file
>> named
>> >> >> >> >> ranger-<pluginType>-security.xml
>> >> >> >> >>
>> >> >> >> >> ranger.plugin.<pluginType>.policy.rest.url
>> >> >> >> >> ranger.plugin.<pluginType>.service.name
>> >> >> >> >>
>> >> >> >> >> For example, these are specified in
>> >> >> conf/ranger-sampleapp-security.xml
>> >> >> >> for
>> >> >> >> >> the sample application.
>> >> >> >> >>
>> >> >> >> >> Can you please review the configuration for your plugin for
>> the
>> >> >> above?
>> >> >> >> >>
>> >> >> >> >> Hope this helps.
>> >> >> >> >>
>> >> >> >> >> Madhan
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <talktorohit54@gmail.com
>> >> <javascript:;>> wrote:
>> >> >> >> >>
>> >> >> >> >> >Hello Madhan,
>> >> >> >> >> >Thanks a lot for your reply.
>> >> >> >> >> >
>> >> >> >> >> >I am looking into the integration and I working towards
>> >> developing
>> >> >> the
>> >> >> >> >> >components outside of ranger trunk for initial development
>> >> purpose.
>> >> >> >> After
>> >> >> >> >> >taking an overview of the codebase it seems that it's
>> possible to
>> >> >> have
>> >> >> >> >> >plugins outside of the ranger trunk too.
>> >> >> >> >> >
>> >> >> >> >> >I was able to add my service to Ranger Amin through the curl
>> >> request
>> >> >> >> and
>> >> >> >> >> >connect to Ranger from my Authorizer but I see the following
>> >> error:
>> >> >> >> >> >
>> >> >> >> >> >com.sun.jersey.api.client.ClientHandlerException:
>> >> >> >> >> >> java.lang.IllegalArgumentException: URI is not absolute
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
>> >> >> >> >> >> at
>> >> >> com.sun.jersey.api.client.Client.handle(Client.java:648)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >>
>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >>
>> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
>> >> >> >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not
>> >> absolute
>> >> >> >> >> >> at java.net.URI.toURL(URI.java:1095)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
>> >> >> >> >> >> at
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
>> >> >> >> >> >> ... 8 more
>> >> >> >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file
>> does
>> >> not
>> >> >> >> exist
>> >> >> >> >> or
>> >> >> >> >> >> not readble 'null'
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >A quick search on google pointed towards misconfiguration of
>> >> service
>> >> >> >> name
>> >> >> >> >> >in security.xml which I have double checked.
>> >> >> >> >> >
>> >> >> >> >> >Any pointers to debug this will be appreciated.
>> >> >> >> >> >
>> >> >> >> >> >Thanks.
>> >> >> >> >> >
>> >> >> >> >> >Thanks,
>> >> >> >> >> >Rohit Sinha
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>> >> >> >> >> >mneethiraj@hortonworks.com <javascript:;>> wrote:
>> >> >> >> >> >
>> >> >> >> >> >> Rohit,
>> >> >> >> >> >>
>> >> >> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the
>> >> name
>> >> >> of
>> >> >> >> the
>> >> >> >> >> >> service instance in Ranger Admin, which contains the
>> policies
>> >> for
>> >> >> the
>> >> >> >> >> >> component (in this case HBase). The plugin reads this value
>> >> from a
>> >> >> >> >> >> configuration named ranger.plugin.hbase.service.name (in
>> file
>> >> >> >> >> >> ranger-hbase-security.xml).
>> >> >> >> >> >>
>> >> >> >> >> >> >> After doing this I don't see anything in the Audit ->
>> >> Plugins
>> >> >> >> >> >> An entry will be created here for every policy download
>> form
>> >> >> plugins.
>> >> >> >> >> >> Plugins download the policies at the following events:
>> >> >> >> >> >> - during the component startup (HBase/HiveServer/...)
>> >> >> >> >> >> - when there is a policy change in service instance
>> >> >> >> >> >>
>> >> >> >> >> >> Hope this helps.
>> >> >> >> >> >>
>> >> >> >> >> >> Madhan
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <talktorohit54@gmail.com
>> >> <javascript:;>>
>> >> >> wrote:
>> >> >> >> >> >>
>> >> >> >> >> >> >Thanks for sharing the SampleApp. I was able to run it
>> >> understand
>> >> >> >> the
>> >> >> >> >> >> >integration point.
>> >> >> >> >> >> >I also tried to enable HBase ranger plugin. When I added
>> the
>> >> >> service
>> >> >> >> >> from
>> >> >> >> >> >> >the Ranger Admin UI I was able to talk to HBase and the
>> >> resource
>> >> >> >> >> >> completion
>> >> >> >> >> >> >worked. After that I deleted the HBase plugin from the UI
>> and
>> >> >> tried
>> >> >> >> to
>> >> >> >> >> >> >enable it from the the command line following instructions
>> >> >> mentioned
>> >> >> >> >> here:
>> >> >> >> >> >> >
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >> >> >> >> >> >
>> >> >> >> >> >> >After doing this I don't see anything in the Audit ->
>> >> Plugins. In
>> >> >> >> the
>> >> >> >> >> >> >instructions I do see warning
>> >> >> >> >> >> >
>> >> >> >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger
>> >> Admin.
>> >> >> If
>> >> >> >> not,
>> >> >> >> >> >> the
>> >> >> >> >> >> >> hbase-plugin will not be able to communicate with Ranger
>> >> admin.
>> >> >> >> >> >> >
>> >> >> >> >> >> >Does this mean I need to add the service from the Ranger
>> Admin
>> >> >> Panel
>> >> >> >> >> or I
>> >> >> >> >> >> >did something wrong in the enabling the plugin ?
>> >> >> >> >> >> >
>> >> >> >> >> >> >Any help will be highly appreciated.
>> >> >> >> >> >> >
>> >> >> >> >> >> >Thanks.
>> >> >> >> >> >> >
>> >> >> >> >> >> >Thanks,
>> >> >> >> >> >> >Rohit Sinha
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
>> >> >> >> talktorohit54@gmail.com <javascript:;>>
>> >> >> >> >> >> >wrote:
>> >> >> >> >> >> >
>> >> >> >> >> >> >> Awesome. Thanks a lot.
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Thanks,
>> >> >> >> >> >> >> Rohit Sinha
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
>> >> >> >> bosco@apache.org <javascript:;>>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>> It is optional. It is easy to setup and helps a lot
>> while
>> >> >> >> debugging
>> >> >> >> >> >> >>> during initial setup.
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> Bosco
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <
>> >> talktorohit54@gmail.com <javascript:;>>
>> >> >> >> wrote:
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> >Thanks a lot for the prompt replies. Really
>> appreciate it.
>> >> >> >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was
>> >> really
>> >> >> >> >> helpful
>> >> >> >> >> >> in
>> >> >> >> >> >> >>> >getting some understanding of the integration. I am
>> going
>> >> >> >> through
>> >> >> >> >> the
>> >> >> >> >> >> >>> >SampleApp docs now.
>> >> >> >> >> >> >>> >
>> >> >> >> >> >> >>> >Is auditing an optional feature ? From the
>> documentation
>> >> it
>> >> >> >> looks
>> >> >> >> >> like
>> >> >> >> >> >> >>> it's
>> >> >> >> >> >> >>> >not and Solr installation is a requirement. I was
>> >> wondering
>> >> >> if I
>> >> >> >> >> can
>> >> >> >> >> >> have
>> >> >> >> >> >> >>> >auditing off and skip Solr installation for initial
>> >> >> integration
>> >> >> >> >> >> purpose.
>> >> >> >> >> >> >>> >
>> >> >> >> >> >> >>> >Thanks.
>> >> >> >> >> >> >>> >
>> >> >> >> >> >> >>> >Thanks,
>> >> >> >> >> >> >>> >Rohit Sinha
>> >> >> >> >> >> >>> >
>> >> >> >> >> >> >>> >
>> >> >> >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
>> >> >> >> >> madhan@apache.org <javascript:;>>
>> >> >> >> >> >> >>> wrote:
>> >> >> >> >> >> >>> >
>> >> >> >> >> >> >>> >> Rohit,
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >> In addition to the details in the wiki, I would
>> >> recommend
>> >> >> >> >> reviewing
>> >> >> >> >> >> the
>> >> >> >> >> >> >>> >> following sample application to understand the
>> details
>> >> of
>> >> >> >> adding
>> >> >> >> >> >> Ranger
>> >> >> >> >> >> >>> >> authorization to an application.
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >> - README.txt:
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >> >> >> >> >> >>> >> - Application sources:
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >> >> >> >> >> >>> >> .
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >> Madhan
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >> >> >> >> >> >>> vperiasamy@hortonworks.com <javascript:;>
>> >> >> >> >> >> >>> >> on behalf of vel@apache.org <javascript:;>> wrote:
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >> >Rohit -
>> >> >> >> >> >> >>> >> >
>> >> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >> >> >> >> >> >>> >> >explains how to add a custom plugin for Ranger.
>> >> >> >> >> >> >>> >> >
>> >> >> >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <
>> >> >> talktorohit54@gmail.com <javascript:;>>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >> >>> >> >
>> >> >> >> >> >> >>> >> >>Hello,
>> >> >> >> >> >> >>> >> >>I am looking into integrating an external service
>> with
>> >> >> >> Apache
>> >> >> >> >> >> Ranger
>> >> >> >> >> >> >>> for
>> >> >> >> >> >> >>> >> >>authorization.
>> >> >> >> >> >> >>> >> >>I looked up the wiki but there is no information
>> about
>> >> >> >> >> integrating
>> >> >> >> >> >> >>> new
>> >> >> >> >> >> >>> >> >>services.
>> >> >> >> >> >> >>> >> >>Can someone give me some info which might be
>> helpful
>> >> in
>> >> >> >> >> >> identifying
>> >> >> >> >> >> >>> >> >>different components which needs to be developed
>> and
>> >> >> other
>> >> >> >> >> >> required
>> >> >> >> >> >> >>> stuff
>> >> >> >> >> >> >>> >> >>
>> >> >> >> >> >> >>> >> >>Thanks.
>> >> >> >> >> >> >>> >> >
>> >> >> >> >> >> >>> >> >
>> >> >> >> >> >> >>> >> >
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>> >>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >>
>> >> >>
>> >>
>> >>
>> >
>> >--
>> >Thanks,
>> >Rohit Sinha
>>
>>
>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Hello,
Thanks for sharing the details. I understand how the policy request are
served. My concern was that since there is no mechanism to ensure that the
service making authorization request is a legal service anyone can make
authorization request to check authorization for different users on
different resources (with the correct service name and resource hierarchy).
I am not sure if this is a big issue or not since the auth request are
basically read only.
Anyways, I have been working on the integration and I am trying to make the
resource lookup work. I have implemented the testConnection part in my
implementation for RangerBaseService.
My test connection works sometimes and other times it randomly throws the
following exception:
2016-04-10 05:54:27,608 [timed-executor-pool-0] INFO
> org.apache.ranger.plugin.client.BaseClient (BaseClient.
> java:104) - Init Login: security not enabled, using username
> 2016-04-10 05:54:27,610 [http-bio-6080-exec-8] ERROR
> org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:129) -
> ==> ServiceMgr.validateConfig
> Error:org.apache.ranger.plugin.client.HadoopException:
> java.lang.IllegalAccess
> Error: tried to access method com.google.common.base.Stopwatch.<init>()V
> from class
> com.example.app.security.authorization.ranger.lookup.client.Client
I see this in the UI when the testConnection fails and also in xa_portal
logs. So, far I have not been able to identify when the failure happens.
Any idea what can be causing this ?
Also, can you point me to the documentation which explains the different
properties in the service definition json ? I don't understand few of the
json elements like what does recursiveSupported actually does ?
Thanks.
Thanks,
Rohit Sinha
On Fri, Apr 8, 2016 at 6:32 PM, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> Ranger plugins download policies from Ranger Admin using the serviceName.
> The plugins must be configured with the correct name of the service
> containing
> policies to enforce (refer to earlier discussion in this email on
> configuration
> ranger.plugin.<pluginType>.service.name).
>
> If a plugin is configured with incorrect serviceName (for example a HBase
> plugin
> configured with HDFS serviceName), it will end up failing/denying all
> Authorization requests - as the resources for each service are different
> i.e.
>
> Resources for HBase: { table/column-family/column }
> Resources for HDFS: { path }
>
> Madhan
>
>
>
> On 4/8/16, 6:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Hello Madhan,
> >Thanks a lot for the prompt reply.
> >This answers how does ranger authenticate itself to the service where it
> >wants to do lookup. I am still confused about the other part.
> >
> >If I have HBase and HDFS policy how does Ranger makes sure that the
> service
> >talking to ranger is one and not another for performing authorization
> >checks ?
> >
> >Thanks.
> >
> >On Friday, April 8, 2016, Madhan Neethiraj <ma...@apache.org> wrote:
> >
> >> Rohit,
> >>
> >> Your question at the end is the answer you are looking for!
> >>
> >> The username and password given while adding a service is used by Ranger
> >> Admin
> >> while performing lookup. In case of secure cluster, the username should
> be
> >> the
> >> Kerberos principal. This user must have enough permissions in the
> component
> >> (via a Ranger policy?) to perform the lookup.
> >>
> >>
> >>
> >>
> >> Hope this help.
> >>
> >> Madhan
> >>
> >> On 4/8/16, 5:31 PM, "rohit sinha" <talktorohit54@gmail.com
> <javascript:;>>
> >> wrote:
> >>
> >> >Hello Madhan,
> >> >
> >> >I am having some difficulty understanding how ranger and services
> identify
> >> >each other on a secure cluster.
> >> >
> >> >For example, if I have a resource lookup plugin running in ranger for
> >> HBase
> >> >and a ranger authorization plugin running in HBase and when they make
> >> >request to each other how does ranger make sure that the talking
> service
> >> in
> >> >HBase and also how does HBase identifies the talking service in ranger.
> >> >
> >> >Also, for every service we take username and password while the
> service is
> >> >being added. What is it used for ?
> >> >
> >> >Thanks.
> >> >
> >> >
> >> >Thanks,
> >> >Rohit Sinha
> >> >
> >> >
> >> >On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <madhan@apache.org
> >> <javascript:;>> wrote:
> >> >
> >> >> Rohit,
> >> >>
> >> >> >> 1. Can you point me to some resource where I can see how to
> configure
> >> >> Ranger on Kerberos without KMS.
> >> >> Ranger configuration does not depend on presence or absence of KMS.
> It
> >> will
> >> >> help if you can provide more details on what you are trying to do or
> >> what
> >> >> difference/difficulty you see due to presence/absence of KMS.
> >> >>
> >> >> >> 2. Also, how can I list all the policies for a resource in my
> service
> >> >>
> >> >>
> >> >>
> >> >> RangerBasePlugin is designed for authorizing accesses and don’t
> provide
> >> >> APIs to
> >> >> search the policies. You can use REST APIs of Ranger Admin to search
> >> >> policies.
> >> >>
> >> >>
> >> >> For example: curl -f -X GET -H "Accept: application/json" -u
> >> >> admin-user:admin-pass
> >> >>
> >>
> http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
> >> >>
> >> >> For details of the REST APIs, please refer to:
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
> >> >>
> >> >> Thanks,
> >> >> Madhan
> >> >>
> >> >> On 4/7/16, 9:28 PM, "rohit sinha" <talktorohit54@gmail.com
> >> <javascript:;>> wrote:
> >> >>
> >> >> >Hello Madhan,
> >> >> >Thanks a lot for the prompt reply.
> >> >> >
> >> >> >1. Can you point me to some resource where I can see how to
> configure
> >> >> >Ranger on Kerberos without KMS.
> >> >> >
> >> >> >2. Also, how can I list all the policies for a resource in my
> service.
> >> I
> >> >> >don't see any api exposed by RangerBasePlugin for this. I was
> looking
> >> into
> >> >> >the code base and saw that ServiceStore has some APIs for this. How
> >> can I
> >> >> >access this ?
> >> >> >
> >> >> >Thanks.
> >> >> >
> >> >> >Thanks,
> >> >> >Rohit Sinha
> >> >> >
> >> >> >
> >> >> >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <madhan@apache.org
> >> <javascript:;>>
> >> >> wrote:
> >> >> >
> >> >> >> Rohit,
> >> >> >>
> >> >> >> Thanks for sharing your feedback on Ranger integration. Good to
> know
> >> >> >> that you are able to get Ranger authorization working in your
> >> >> application.
> >> >> >>
> >> >> >> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
> >> >> >> >> with secure Hadoop through Kerberos ?
> >> >> >> Yes. Ranger works with secure Hadoop - either unencrypted or
> >> encrypted.
> >> >> >>
> >> >> >> >> 2. Currently, I see the following error log for policy cache
> file.
> >> >> >> >> Isn’t the policy cache file created automatically ?
> >> >> >> Yes, the policy cache file is created automatically. I guess the
> >> error
> >> >> >>
> >> >> >> you see could be cause by non-existent directory where the cache
> file
> >> >> >> would be saved - in this case
> /etc/ranger/myservicedev/policycache/.
> >> >> >> Please ensure that this directory exists and has write permission
> for
> >> >> >> the user that runs the application.
> >> >> >>
> >> >> >> The directory location can be specified via configuration
> >> >> >> ranger.plugin.myservice.policy.cache.dir in
> >> >> ranger-myservice-security.xml.
> >> >> >>
> >> >> >> Hope this helps.
> >> >> >>
> >> >> >> Madhan
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On 4/7/16, 7:38 PM, "rohit sinha" <talktorohit54@gmail.com
> >> <javascript:;>> wrote:
> >> >> >>
> >> >> >> >Hello,
> >> >> >> >
> >> >> >> >Thanks a lot for your prompt replies. It was really helpful.
> >> >> >> >Yes, it seems like there was some misconfiguration issue. I was
> >> able to
> >> >> >> >make it work and I have a very basic integration up and running
> >> with my
> >> >> >> >service now. I have worked on integration/analysis of other
> >> >> authorization
> >> >> >> >models and I must say that Ranger so far has been the easiest to
> >> >> integrate
> >> >> >> >with (at the basic level) and the documentation helped a lot.
> >> >> Appreciate
> >> >> >> >your efforts in building it so far.
> >> >> >> >
> >> >> >> >I have two question which I am trying to figure out. Will
> >> appreciate if
> >> >> >> >someone can provide some pointers.
> >> >> >> >
> >> >> >> >1. Kerberos: From the documentation found here
> >> >> >> >
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
> >> >> >> >I understand that Ranger works with KMS on an encrypted HDFS. Is
> it
> >> >> >> >possible to have Ranger running on an unencrypted HDFS with
> secure
> >> >> Hadoop
> >> >> >> >through Kerberos ? Something where I can talk to Ranger by
> giving a
> >> >> >> >principal and keytab ?
> >> >> >> >
> >> >> >> >2. Currently, I see the following error log for policy cache
> file.
> >> >> Isn't
> >> >> >> >the policy cache file created automatically ? Am I missing some
> >> >> >> >configuration.
> >> >> >> >
> >> >> >> >16/04/07 21:26:18 INFO util.PolicyRefresher:
> >> >> >> >> PolicyRefresher(serviceName=myservice): found updated version.
> >> >> >> >> lastKnownVersion=-1; newVersion=1
> >> >> >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save
> >> >> policies to
> >> >> >> >> cache file
> >> >> >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
> >> >> >> >> java.io.FileNotFoundException:
> >> >> >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json
> (No
> >> >> such
> >> >> >> file
> >> >> >> >> or directory)
> >> >> >> >> at java.io.FileOutputStream.open(Native Method)
> >> >> >> >> at
> >> java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> >> >> >> >> at
> >> java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> >> >> >> >> at java.io.FileWriter.<init>(FileWriter.java:90)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
> >> >> >> >> .... internal stacktrace....
> >> >> >> >> at java.lang.Thread.run(Thread.java:745)
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >Thanks.
> >> >> >> >
> >> >> >> >Thanks,
> >> >> >> >Rohit Sinha
> >> >> >> >
> >> >> >> >
> >> >> >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <
> >> madhan@apache.org <javascript:;>>
> >> >> >> wrote:
> >> >> >> >
> >> >> >> >> Rohit,
> >> >> >> >>
> >> >> >> >> To download policies from Ranger Admin, Ranger plugins require
> the
> >> >> URL
> >> >> >> to
> >> >> >> >> Ranger Admin and the name of the service containing the
> policies.
> >> >> These
> >> >> >> >> values are read from following configurations from a file named
> >> >> >> >> ranger-<pluginType>-security.xml
> >> >> >> >>
> >> >> >> >> ranger.plugin.<pluginType>.policy.rest.url
> >> >> >> >> ranger.plugin.<pluginType>.service.name
> >> >> >> >>
> >> >> >> >> For example, these are specified in
> >> >> conf/ranger-sampleapp-security.xml
> >> >> >> for
> >> >> >> >> the sample application.
> >> >> >> >>
> >> >> >> >> Can you please review the configuration for your plugin for the
> >> >> above?
> >> >> >> >>
> >> >> >> >> Hope this helps.
> >> >> >> >>
> >> >> >> >> Madhan
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <talktorohit54@gmail.com
> >> <javascript:;>> wrote:
> >> >> >> >>
> >> >> >> >> >Hello Madhan,
> >> >> >> >> >Thanks a lot for your reply.
> >> >> >> >> >
> >> >> >> >> >I am looking into the integration and I working towards
> >> developing
> >> >> the
> >> >> >> >> >components outside of ranger trunk for initial development
> >> purpose.
> >> >> >> After
> >> >> >> >> >taking an overview of the codebase it seems that it's
> possible to
> >> >> have
> >> >> >> >> >plugins outside of the ranger trunk too.
> >> >> >> >> >
> >> >> >> >> >I was able to add my service to Ranger Amin through the curl
> >> request
> >> >> >> and
> >> >> >> >> >connect to Ranger from my Authorizer but I see the following
> >> error:
> >> >> >> >> >
> >> >> >> >> >com.sun.jersey.api.client.ClientHandlerException:
> >> >> >> >> >> java.lang.IllegalArgumentException: URI is not absolute
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
> >> >> >> >> >> at
> >> >> com.sun.jersey.api.client.Client.handle(Client.java:648)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >>
> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
> >> >> >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not
> >> absolute
> >> >> >> >> >> at java.net.URI.toURL(URI.java:1095)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> >> >> >> >> >> at
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> >> >> >> >> >> ... 8 more
> >> >> >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does
> >> not
> >> >> >> exist
> >> >> >> >> or
> >> >> >> >> >> not readble 'null'
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >A quick search on google pointed towards misconfiguration of
> >> service
> >> >> >> name
> >> >> >> >> >in security.xml which I have double checked.
> >> >> >> >> >
> >> >> >> >> >Any pointers to debug this will be appreciated.
> >> >> >> >> >
> >> >> >> >> >Thanks.
> >> >> >> >> >
> >> >> >> >> >Thanks,
> >> >> >> >> >Rohit Sinha
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
> >> >> >> >> >mneethiraj@hortonworks.com <javascript:;>> wrote:
> >> >> >> >> >
> >> >> >> >> >> Rohit,
> >> >> >> >> >>
> >> >> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the
> >> name
> >> >> of
> >> >> >> the
> >> >> >> >> >> service instance in Ranger Admin, which contains the
> policies
> >> for
> >> >> the
> >> >> >> >> >> component (in this case HBase). The plugin reads this value
> >> from a
> >> >> >> >> >> configuration named ranger.plugin.hbase.service.name (in
> file
> >> >> >> >> >> ranger-hbase-security.xml).
> >> >> >> >> >>
> >> >> >> >> >> >> After doing this I don't see anything in the Audit ->
> >> Plugins
> >> >> >> >> >> An entry will be created here for every policy download form
> >> >> plugins.
> >> >> >> >> >> Plugins download the policies at the following events:
> >> >> >> >> >> - during the component startup (HBase/HiveServer/...)
> >> >> >> >> >> - when there is a policy change in service instance
> >> >> >> >> >>
> >> >> >> >> >> Hope this helps.
> >> >> >> >> >>
> >> >> >> >> >> Madhan
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <talktorohit54@gmail.com
> >> <javascript:;>>
> >> >> wrote:
> >> >> >> >> >>
> >> >> >> >> >> >Thanks for sharing the SampleApp. I was able to run it
> >> understand
> >> >> >> the
> >> >> >> >> >> >integration point.
> >> >> >> >> >> >I also tried to enable HBase ranger plugin. When I added
> the
> >> >> service
> >> >> >> >> from
> >> >> >> >> >> >the Ranger Admin UI I was able to talk to HBase and the
> >> resource
> >> >> >> >> >> completion
> >> >> >> >> >> >worked. After that I deleted the HBase plugin from the UI
> and
> >> >> tried
> >> >> >> to
> >> >> >> >> >> >enable it from the the command line following instructions
> >> >> mentioned
> >> >> >> >> here:
> >> >> >> >> >> >
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
> >> >> >> >> >> >
> >> >> >> >> >> >After doing this I don't see anything in the Audit ->
> >> Plugins. In
> >> >> >> the
> >> >> >> >> >> >instructions I do see warning
> >> >> >> >> >> >
> >> >> >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger
> >> Admin.
> >> >> If
> >> >> >> not,
> >> >> >> >> >> the
> >> >> >> >> >> >> hbase-plugin will not be able to communicate with Ranger
> >> admin.
> >> >> >> >> >> >
> >> >> >> >> >> >Does this mean I need to add the service from the Ranger
> Admin
> >> >> Panel
> >> >> >> >> or I
> >> >> >> >> >> >did something wrong in the enabling the plugin ?
> >> >> >> >> >> >
> >> >> >> >> >> >Any help will be highly appreciated.
> >> >> >> >> >> >
> >> >> >> >> >> >Thanks.
> >> >> >> >> >> >
> >> >> >> >> >> >Thanks,
> >> >> >> >> >> >Rohit Sinha
> >> >> >> >> >> >
> >> >> >> >> >> >
> >> >> >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
> >> >> >> talktorohit54@gmail.com <javascript:;>>
> >> >> >> >> >> >wrote:
> >> >> >> >> >> >
> >> >> >> >> >> >> Awesome. Thanks a lot.
> >> >> >> >> >> >>
> >> >> >> >> >> >> Thanks,
> >> >> >> >> >> >> Rohit Sinha
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
> >> >> >> bosco@apache.org <javascript:;>>
> >> >> >> >> >> wrote:
> >> >> >> >> >> >>
> >> >> >> >> >> >>> It is optional. It is easy to setup and helps a lot
> while
> >> >> >> debugging
> >> >> >> >> >> >>> during initial setup.
> >> >> >> >> >> >>>
> >> >> >> >> >> >>> Bosco
> >> >> >> >> >> >>>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <
> >> talktorohit54@gmail.com <javascript:;>>
> >> >> >> wrote:
> >> >> >> >> >> >>>
> >> >> >> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate
> it.
> >> >> >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was
> >> really
> >> >> >> >> helpful
> >> >> >> >> >> in
> >> >> >> >> >> >>> >getting some understanding of the integration. I am
> going
> >> >> >> through
> >> >> >> >> the
> >> >> >> >> >> >>> >SampleApp docs now.
> >> >> >> >> >> >>> >
> >> >> >> >> >> >>> >Is auditing an optional feature ? From the
> documentation
> >> it
> >> >> >> looks
> >> >> >> >> like
> >> >> >> >> >> >>> it's
> >> >> >> >> >> >>> >not and Solr installation is a requirement. I was
> >> wondering
> >> >> if I
> >> >> >> >> can
> >> >> >> >> >> have
> >> >> >> >> >> >>> >auditing off and skip Solr installation for initial
> >> >> integration
> >> >> >> >> >> purpose.
> >> >> >> >> >> >>> >
> >> >> >> >> >> >>> >Thanks.
> >> >> >> >> >> >>> >
> >> >> >> >> >> >>> >Thanks,
> >> >> >> >> >> >>> >Rohit Sinha
> >> >> >> >> >> >>> >
> >> >> >> >> >> >>> >
> >> >> >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
> >> >> >> >> madhan@apache.org <javascript:;>>
> >> >> >> >> >> >>> wrote:
> >> >> >> >> >> >>> >
> >> >> >> >> >> >>> >> Rohit,
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >> In addition to the details in the wiki, I would
> >> recommend
> >> >> >> >> reviewing
> >> >> >> >> >> the
> >> >> >> >> >> >>> >> following sample application to understand the
> details
> >> of
> >> >> >> adding
> >> >> >> >> >> Ranger
> >> >> >> >> >> >>> >> authorization to an application.
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >> - README.txt:
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>>
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >> >> >> >> >> >>> >> - Application sources:
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>>
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >> >> >> >> >> >>> >> .
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >> Madhan
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
> >> >> >> >> >> >>> vperiasamy@hortonworks.com <javascript:;>
> >> >> >> >> >> >>> >> on behalf of vel@apache.org <javascript:;>> wrote:
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >> >Rohit -
> >> >> >> >> >> >>> >> >
> >> >> >> >> >> >>>
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >> >> >> >> >> >>> >> >explains how to add a custom plugin for Ranger.
> >> >> >> >> >> >>> >> >
> >> >> >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <
> >> >> talktorohit54@gmail.com <javascript:;>>
> >> >> >> >> >> wrote:
> >> >> >> >> >> >>> >> >
> >> >> >> >> >> >>> >> >>Hello,
> >> >> >> >> >> >>> >> >>I am looking into integrating an external service
> with
> >> >> >> Apache
> >> >> >> >> >> Ranger
> >> >> >> >> >> >>> for
> >> >> >> >> >> >>> >> >>authorization.
> >> >> >> >> >> >>> >> >>I looked up the wiki but there is no information
> about
> >> >> >> >> integrating
> >> >> >> >> >> >>> new
> >> >> >> >> >> >>> >> >>services.
> >> >> >> >> >> >>> >> >>Can someone give me some info which might be
> helpful
> >> in
> >> >> >> >> >> identifying
> >> >> >> >> >> >>> >> >>different components which needs to be developed
> and
> >> >> other
> >> >> >> >> >> required
> >> >> >> >> >> >>> stuff
> >> >> >> >> >> >>> >> >>
> >> >> >> >> >> >>> >> >>Thanks.
> >> >> >> >> >> >>> >> >
> >> >> >> >> >> >>> >> >
> >> >> >> >> >> >>> >> >
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>> >>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >>
> >> >>
> >>
> >>
> >
> >--
> >Thanks,
> >Rohit Sinha
>
>
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
Ranger plugins download policies from Ranger Admin using the serviceName.
The plugins must be configured with the correct name of the service containing
policies to enforce (refer to earlier discussion in this email on configuration
ranger.plugin.<pluginType>.service.name).
If a plugin is configured with incorrect serviceName (for example a HBase plugin
configured with HDFS serviceName), it will end up failing/denying all
Authorization requests - as the resources for each service are different i.e.
Resources for HBase: { table/column-family/column }
Resources for HDFS: { path }
Madhan
On 4/8/16, 6:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello Madhan,
>Thanks a lot for the prompt reply.
>This answers how does ranger authenticate itself to the service where it
>wants to do lookup. I am still confused about the other part.
>
>If I have HBase and HDFS policy how does Ranger makes sure that the service
>talking to ranger is one and not another for performing authorization
>checks ?
>
>Thanks.
>
>On Friday, April 8, 2016, Madhan Neethiraj <ma...@apache.org> wrote:
>
>> Rohit,
>>
>> Your question at the end is the answer you are looking for!
>>
>> The username and password given while adding a service is used by Ranger
>> Admin
>> while performing lookup. In case of secure cluster, the username should be
>> the
>> Kerberos principal. This user must have enough permissions in the component
>> (via a Ranger policy?) to perform the lookup.
>>
>>
>>
>>
>> Hope this help.
>>
>> Madhan
>>
>> On 4/8/16, 5:31 PM, "rohit sinha" <talktorohit54@gmail.com <javascript:;>>
>> wrote:
>>
>> >Hello Madhan,
>> >
>> >I am having some difficulty understanding how ranger and services identify
>> >each other on a secure cluster.
>> >
>> >For example, if I have a resource lookup plugin running in ranger for
>> HBase
>> >and a ranger authorization plugin running in HBase and when they make
>> >request to each other how does ranger make sure that the talking service
>> in
>> >HBase and also how does HBase identifies the talking service in ranger.
>> >
>> >Also, for every service we take username and password while the service is
>> >being added. What is it used for ?
>> >
>> >Thanks.
>> >
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <madhan@apache.org
>> <javascript:;>> wrote:
>> >
>> >> Rohit,
>> >>
>> >> >> 1. Can you point me to some resource where I can see how to configure
>> >> Ranger on Kerberos without KMS.
>> >> Ranger configuration does not depend on presence or absence of KMS. It
>> will
>> >> help if you can provide more details on what you are trying to do or
>> what
>> >> difference/difficulty you see due to presence/absence of KMS.
>> >>
>> >> >> 2. Also, how can I list all the policies for a resource in my service
>> >>
>> >>
>> >>
>> >> RangerBasePlugin is designed for authorizing accesses and don’t provide
>> >> APIs to
>> >> search the policies. You can use REST APIs of Ranger Admin to search
>> >> policies.
>> >>
>> >>
>> >> For example: curl -f -X GET -H "Accept: application/json" -u
>> >> admin-user:admin-pass
>> >>
>> http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
>> >>
>> >> For details of the REST APIs, please refer to:
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
>> >>
>> >> Thanks,
>> >> Madhan
>> >>
>> >> On 4/7/16, 9:28 PM, "rohit sinha" <talktorohit54@gmail.com
>> <javascript:;>> wrote:
>> >>
>> >> >Hello Madhan,
>> >> >Thanks a lot for the prompt reply.
>> >> >
>> >> >1. Can you point me to some resource where I can see how to configure
>> >> >Ranger on Kerberos without KMS.
>> >> >
>> >> >2. Also, how can I list all the policies for a resource in my service.
>> I
>> >> >don't see any api exposed by RangerBasePlugin for this. I was looking
>> into
>> >> >the code base and saw that ServiceStore has some APIs for this. How
>> can I
>> >> >access this ?
>> >> >
>> >> >Thanks.
>> >> >
>> >> >Thanks,
>> >> >Rohit Sinha
>> >> >
>> >> >
>> >> >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <madhan@apache.org
>> <javascript:;>>
>> >> wrote:
>> >> >
>> >> >> Rohit,
>> >> >>
>> >> >> Thanks for sharing your feedback on Ranger integration. Good to know
>> >> >> that you are able to get Ranger authorization working in your
>> >> application.
>> >> >>
>> >> >> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
>> >> >> >> with secure Hadoop through Kerberos ?
>> >> >> Yes. Ranger works with secure Hadoop - either unencrypted or
>> encrypted.
>> >> >>
>> >> >> >> 2. Currently, I see the following error log for policy cache file.
>> >> >> >> Isn’t the policy cache file created automatically ?
>> >> >> Yes, the policy cache file is created automatically. I guess the
>> error
>> >> >>
>> >> >> you see could be cause by non-existent directory where the cache file
>> >> >> would be saved - in this case /etc/ranger/myservicedev/policycache/.
>> >> >> Please ensure that this directory exists and has write permission for
>> >> >> the user that runs the application.
>> >> >>
>> >> >> The directory location can be specified via configuration
>> >> >> ranger.plugin.myservice.policy.cache.dir in
>> >> ranger-myservice-security.xml.
>> >> >>
>> >> >> Hope this helps.
>> >> >>
>> >> >> Madhan
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> On 4/7/16, 7:38 PM, "rohit sinha" <talktorohit54@gmail.com
>> <javascript:;>> wrote:
>> >> >>
>> >> >> >Hello,
>> >> >> >
>> >> >> >Thanks a lot for your prompt replies. It was really helpful.
>> >> >> >Yes, it seems like there was some misconfiguration issue. I was
>> able to
>> >> >> >make it work and I have a very basic integration up and running
>> with my
>> >> >> >service now. I have worked on integration/analysis of other
>> >> authorization
>> >> >> >models and I must say that Ranger so far has been the easiest to
>> >> integrate
>> >> >> >with (at the basic level) and the documentation helped a lot.
>> >> Appreciate
>> >> >> >your efforts in building it so far.
>> >> >> >
>> >> >> >I have two question which I am trying to figure out. Will
>> appreciate if
>> >> >> >someone can provide some pointers.
>> >> >> >
>> >> >> >1. Kerberos: From the documentation found here
>> >> >> >
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
>> >> >> >I understand that Ranger works with KMS on an encrypted HDFS. Is it
>> >> >> >possible to have Ranger running on an unencrypted HDFS with secure
>> >> Hadoop
>> >> >> >through Kerberos ? Something where I can talk to Ranger by giving a
>> >> >> >principal and keytab ?
>> >> >> >
>> >> >> >2. Currently, I see the following error log for policy cache file.
>> >> Isn't
>> >> >> >the policy cache file created automatically ? Am I missing some
>> >> >> >configuration.
>> >> >> >
>> >> >> >16/04/07 21:26:18 INFO util.PolicyRefresher:
>> >> >> >> PolicyRefresher(serviceName=myservice): found updated version.
>> >> >> >> lastKnownVersion=-1; newVersion=1
>> >> >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save
>> >> policies to
>> >> >> >> cache file
>> >> >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
>> >> >> >> java.io.FileNotFoundException:
>> >> >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No
>> >> such
>> >> >> file
>> >> >> >> or directory)
>> >> >> >> at java.io.FileOutputStream.open(Native Method)
>> >> >> >> at
>> java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> >> >> >> at
>> java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> >> >> >> at java.io.FileWriter.<init>(FileWriter.java:90)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
>> >> >> >> .... internal stacktrace....
>> >> >> >> at java.lang.Thread.run(Thread.java:745)
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >Thanks.
>> >> >> >
>> >> >> >Thanks,
>> >> >> >Rohit Sinha
>> >> >> >
>> >> >> >
>> >> >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <
>> madhan@apache.org <javascript:;>>
>> >> >> wrote:
>> >> >> >
>> >> >> >> Rohit,
>> >> >> >>
>> >> >> >> To download policies from Ranger Admin, Ranger plugins require the
>> >> URL
>> >> >> to
>> >> >> >> Ranger Admin and the name of the service containing the policies.
>> >> These
>> >> >> >> values are read from following configurations from a file named
>> >> >> >> ranger-<pluginType>-security.xml
>> >> >> >>
>> >> >> >> ranger.plugin.<pluginType>.policy.rest.url
>> >> >> >> ranger.plugin.<pluginType>.service.name
>> >> >> >>
>> >> >> >> For example, these are specified in
>> >> conf/ranger-sampleapp-security.xml
>> >> >> for
>> >> >> >> the sample application.
>> >> >> >>
>> >> >> >> Can you please review the configuration for your plugin for the
>> >> above?
>> >> >> >>
>> >> >> >> Hope this helps.
>> >> >> >>
>> >> >> >> Madhan
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <talktorohit54@gmail.com
>> <javascript:;>> wrote:
>> >> >> >>
>> >> >> >> >Hello Madhan,
>> >> >> >> >Thanks a lot for your reply.
>> >> >> >> >
>> >> >> >> >I am looking into the integration and I working towards
>> developing
>> >> the
>> >> >> >> >components outside of ranger trunk for initial development
>> purpose.
>> >> >> After
>> >> >> >> >taking an overview of the codebase it seems that it's possible to
>> >> have
>> >> >> >> >plugins outside of the ranger trunk too.
>> >> >> >> >
>> >> >> >> >I was able to add my service to Ranger Amin through the curl
>> request
>> >> >> and
>> >> >> >> >connect to Ranger from my Authorizer but I see the following
>> error:
>> >> >> >> >
>> >> >> >> >com.sun.jersey.api.client.ClientHandlerException:
>> >> >> >> >> java.lang.IllegalArgumentException: URI is not absolute
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
>> >> >> >> >> at
>> >> com.sun.jersey.api.client.Client.handle(Client.java:648)
>> >> >> >> >> at
>> >> >> >> >>
>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> >> >> >> >> at
>> >> >> >> >>
>> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >>
>> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
>> >> >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not
>> absolute
>> >> >> >> >> at java.net.URI.toURL(URI.java:1095)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
>> >> >> >> >> at
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
>> >> >> >> >> ... 8 more
>> >> >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does
>> not
>> >> >> exist
>> >> >> >> or
>> >> >> >> >> not readble 'null'
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >A quick search on google pointed towards misconfiguration of
>> service
>> >> >> name
>> >> >> >> >in security.xml which I have double checked.
>> >> >> >> >
>> >> >> >> >Any pointers to debug this will be appreciated.
>> >> >> >> >
>> >> >> >> >Thanks.
>> >> >> >> >
>> >> >> >> >Thanks,
>> >> >> >> >Rohit Sinha
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>> >> >> >> >mneethiraj@hortonworks.com <javascript:;>> wrote:
>> >> >> >> >
>> >> >> >> >> Rohit,
>> >> >> >> >>
>> >> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the
>> name
>> >> of
>> >> >> the
>> >> >> >> >> service instance in Ranger Admin, which contains the policies
>> for
>> >> the
>> >> >> >> >> component (in this case HBase). The plugin reads this value
>> from a
>> >> >> >> >> configuration named ranger.plugin.hbase.service.name (in file
>> >> >> >> >> ranger-hbase-security.xml).
>> >> >> >> >>
>> >> >> >> >> >> After doing this I don't see anything in the Audit ->
>> Plugins
>> >> >> >> >> An entry will be created here for every policy download form
>> >> plugins.
>> >> >> >> >> Plugins download the policies at the following events:
>> >> >> >> >> - during the component startup (HBase/HiveServer/...)
>> >> >> >> >> - when there is a policy change in service instance
>> >> >> >> >>
>> >> >> >> >> Hope this helps.
>> >> >> >> >>
>> >> >> >> >> Madhan
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <talktorohit54@gmail.com
>> <javascript:;>>
>> >> wrote:
>> >> >> >> >>
>> >> >> >> >> >Thanks for sharing the SampleApp. I was able to run it
>> understand
>> >> >> the
>> >> >> >> >> >integration point.
>> >> >> >> >> >I also tried to enable HBase ranger plugin. When I added the
>> >> service
>> >> >> >> from
>> >> >> >> >> >the Ranger Admin UI I was able to talk to HBase and the
>> resource
>> >> >> >> >> completion
>> >> >> >> >> >worked. After that I deleted the HBase plugin from the UI and
>> >> tried
>> >> >> to
>> >> >> >> >> >enable it from the the command line following instructions
>> >> mentioned
>> >> >> >> here:
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >> >> >> >> >
>> >> >> >> >> >After doing this I don't see anything in the Audit ->
>> Plugins. In
>> >> >> the
>> >> >> >> >> >instructions I do see warning
>> >> >> >> >> >
>> >> >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger
>> Admin.
>> >> If
>> >> >> not,
>> >> >> >> >> the
>> >> >> >> >> >> hbase-plugin will not be able to communicate with Ranger
>> admin.
>> >> >> >> >> >
>> >> >> >> >> >Does this mean I need to add the service from the Ranger Admin
>> >> Panel
>> >> >> >> or I
>> >> >> >> >> >did something wrong in the enabling the plugin ?
>> >> >> >> >> >
>> >> >> >> >> >Any help will be highly appreciated.
>> >> >> >> >> >
>> >> >> >> >> >Thanks.
>> >> >> >> >> >
>> >> >> >> >> >Thanks,
>> >> >> >> >> >Rohit Sinha
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
>> >> >> talktorohit54@gmail.com <javascript:;>>
>> >> >> >> >> >wrote:
>> >> >> >> >> >
>> >> >> >> >> >> Awesome. Thanks a lot.
>> >> >> >> >> >>
>> >> >> >> >> >> Thanks,
>> >> >> >> >> >> Rohit Sinha
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
>> >> >> bosco@apache.org <javascript:;>>
>> >> >> >> >> wrote:
>> >> >> >> >> >>
>> >> >> >> >> >>> It is optional. It is easy to setup and helps a lot while
>> >> >> debugging
>> >> >> >> >> >>> during initial setup.
>> >> >> >> >> >>>
>> >> >> >> >> >>> Bosco
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <
>> talktorohit54@gmail.com <javascript:;>>
>> >> >> wrote:
>> >> >> >> >> >>>
>> >> >> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >> >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was
>> really
>> >> >> >> helpful
>> >> >> >> >> in
>> >> >> >> >> >>> >getting some understanding of the integration. I am going
>> >> >> through
>> >> >> >> the
>> >> >> >> >> >>> >SampleApp docs now.
>> >> >> >> >> >>> >
>> >> >> >> >> >>> >Is auditing an optional feature ? From the documentation
>> it
>> >> >> looks
>> >> >> >> like
>> >> >> >> >> >>> it's
>> >> >> >> >> >>> >not and Solr installation is a requirement. I was
>> wondering
>> >> if I
>> >> >> >> can
>> >> >> >> >> have
>> >> >> >> >> >>> >auditing off and skip Solr installation for initial
>> >> integration
>> >> >> >> >> purpose.
>> >> >> >> >> >>> >
>> >> >> >> >> >>> >Thanks.
>> >> >> >> >> >>> >
>> >> >> >> >> >>> >Thanks,
>> >> >> >> >> >>> >Rohit Sinha
>> >> >> >> >> >>> >
>> >> >> >> >> >>> >
>> >> >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
>> >> >> >> madhan@apache.org <javascript:;>>
>> >> >> >> >> >>> wrote:
>> >> >> >> >> >>> >
>> >> >> >> >> >>> >> Rohit,
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >> In addition to the details in the wiki, I would
>> recommend
>> >> >> >> reviewing
>> >> >> >> >> the
>> >> >> >> >> >>> >> following sample application to understand the details
>> of
>> >> >> adding
>> >> >> >> >> Ranger
>> >> >> >> >> >>> >> authorization to an application.
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >> - README.txt:
>> >> >> >> >> >>> >>
>> >> >> >> >> >>>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >> >> >> >> >>> >> - Application sources:
>> >> >> >> >> >>> >>
>> >> >> >> >> >>>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >> >> >> >> >>> >> .
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >> Madhan
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >> >> >> >> >>> vperiasamy@hortonworks.com <javascript:;>
>> >> >> >> >> >>> >> on behalf of vel@apache.org <javascript:;>> wrote:
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >> >Rohit -
>> >> >> >> >> >>> >> >
>> >> >> >> >> >>>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >> >> >> >> >>> >> >explains how to add a custom plugin for Ranger.
>> >> >> >> >> >>> >> >
>> >> >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <
>> >> talktorohit54@gmail.com <javascript:;>>
>> >> >> >> >> wrote:
>> >> >> >> >> >>> >> >
>> >> >> >> >> >>> >> >>Hello,
>> >> >> >> >> >>> >> >>I am looking into integrating an external service with
>> >> >> Apache
>> >> >> >> >> Ranger
>> >> >> >> >> >>> for
>> >> >> >> >> >>> >> >>authorization.
>> >> >> >> >> >>> >> >>I looked up the wiki but there is no information about
>> >> >> >> integrating
>> >> >> >> >> >>> new
>> >> >> >> >> >>> >> >>services.
>> >> >> >> >> >>> >> >>Can someone give me some info which might be helpful
>> in
>> >> >> >> >> identifying
>> >> >> >> >> >>> >> >>different components which needs to be developed and
>> >> other
>> >> >> >> >> required
>> >> >> >> >> >>> stuff
>> >> >> >> >> >>> >> >>
>> >> >> >> >> >>> >> >>Thanks.
>> >> >> >> >> >>> >> >
>> >> >> >> >> >>> >> >
>> >> >> >> >> >>> >> >
>> >> >> >> >> >>> >>
>> >> >> >> >> >>> >>
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>
>
>--
>Thanks,
>Rohit Sinha
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Hello Madhan,
Thanks a lot for the prompt reply.
This answers how does ranger authenticate itself to the service where it
wants to do lookup. I am still confused about the other part.
If I have HBase and HDFS policy how does Ranger makes sure that the service
talking to ranger is one and not another for performing authorization
checks ?
Thanks.
On Friday, April 8, 2016, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> Your question at the end is the answer you are looking for!
>
> The username and password given while adding a service is used by Ranger
> Admin
> while performing lookup. In case of secure cluster, the username should be
> the
> Kerberos principal. This user must have enough permissions in the component
> (via a Ranger policy?) to perform the lookup.
>
>
>
>
> Hope this help.
>
> Madhan
>
> On 4/8/16, 5:31 PM, "rohit sinha" <talktorohit54@gmail.com <javascript:;>>
> wrote:
>
> >Hello Madhan,
> >
> >I am having some difficulty understanding how ranger and services identify
> >each other on a secure cluster.
> >
> >For example, if I have a resource lookup plugin running in ranger for
> HBase
> >and a ranger authorization plugin running in HBase and when they make
> >request to each other how does ranger make sure that the talking service
> in
> >HBase and also how does HBase identifies the talking service in ranger.
> >
> >Also, for every service we take username and password while the service is
> >being added. What is it used for ?
> >
> >Thanks.
> >
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <madhan@apache.org
> <javascript:;>> wrote:
> >
> >> Rohit,
> >>
> >> >> 1. Can you point me to some resource where I can see how to configure
> >> Ranger on Kerberos without KMS.
> >> Ranger configuration does not depend on presence or absence of KMS. It
> will
> >> help if you can provide more details on what you are trying to do or
> what
> >> difference/difficulty you see due to presence/absence of KMS.
> >>
> >> >> 2. Also, how can I list all the policies for a resource in my service
> >>
> >>
> >>
> >> RangerBasePlugin is designed for authorizing accesses and don’t provide
> >> APIs to
> >> search the policies. You can use REST APIs of Ranger Admin to search
> >> policies.
> >>
> >>
> >> For example: curl -f -X GET -H "Accept: application/json" -u
> >> admin-user:admin-pass
> >>
> http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
> >>
> >> For details of the REST APIs, please refer to:
> >>
> https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
> >>
> >> Thanks,
> >> Madhan
> >>
> >> On 4/7/16, 9:28 PM, "rohit sinha" <talktorohit54@gmail.com
> <javascript:;>> wrote:
> >>
> >> >Hello Madhan,
> >> >Thanks a lot for the prompt reply.
> >> >
> >> >1. Can you point me to some resource where I can see how to configure
> >> >Ranger on Kerberos without KMS.
> >> >
> >> >2. Also, how can I list all the policies for a resource in my service.
> I
> >> >don't see any api exposed by RangerBasePlugin for this. I was looking
> into
> >> >the code base and saw that ServiceStore has some APIs for this. How
> can I
> >> >access this ?
> >> >
> >> >Thanks.
> >> >
> >> >Thanks,
> >> >Rohit Sinha
> >> >
> >> >
> >> >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <madhan@apache.org
> <javascript:;>>
> >> wrote:
> >> >
> >> >> Rohit,
> >> >>
> >> >> Thanks for sharing your feedback on Ranger integration. Good to know
> >> >> that you are able to get Ranger authorization working in your
> >> application.
> >> >>
> >> >> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
> >> >> >> with secure Hadoop through Kerberos ?
> >> >> Yes. Ranger works with secure Hadoop - either unencrypted or
> encrypted.
> >> >>
> >> >> >> 2. Currently, I see the following error log for policy cache file.
> >> >> >> Isn’t the policy cache file created automatically ?
> >> >> Yes, the policy cache file is created automatically. I guess the
> error
> >> >>
> >> >> you see could be cause by non-existent directory where the cache file
> >> >> would be saved - in this case /etc/ranger/myservicedev/policycache/.
> >> >> Please ensure that this directory exists and has write permission for
> >> >> the user that runs the application.
> >> >>
> >> >> The directory location can be specified via configuration
> >> >> ranger.plugin.myservice.policy.cache.dir in
> >> ranger-myservice-security.xml.
> >> >>
> >> >> Hope this helps.
> >> >>
> >> >> Madhan
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On 4/7/16, 7:38 PM, "rohit sinha" <talktorohit54@gmail.com
> <javascript:;>> wrote:
> >> >>
> >> >> >Hello,
> >> >> >
> >> >> >Thanks a lot for your prompt replies. It was really helpful.
> >> >> >Yes, it seems like there was some misconfiguration issue. I was
> able to
> >> >> >make it work and I have a very basic integration up and running
> with my
> >> >> >service now. I have worked on integration/analysis of other
> >> authorization
> >> >> >models and I must say that Ranger so far has been the easiest to
> >> integrate
> >> >> >with (at the basic level) and the documentation helped a lot.
> >> Appreciate
> >> >> >your efforts in building it so far.
> >> >> >
> >> >> >I have two question which I am trying to figure out. Will
> appreciate if
> >> >> >someone can provide some pointers.
> >> >> >
> >> >> >1. Kerberos: From the documentation found here
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
> >> >> >I understand that Ranger works with KMS on an encrypted HDFS. Is it
> >> >> >possible to have Ranger running on an unencrypted HDFS with secure
> >> Hadoop
> >> >> >through Kerberos ? Something where I can talk to Ranger by giving a
> >> >> >principal and keytab ?
> >> >> >
> >> >> >2. Currently, I see the following error log for policy cache file.
> >> Isn't
> >> >> >the policy cache file created automatically ? Am I missing some
> >> >> >configuration.
> >> >> >
> >> >> >16/04/07 21:26:18 INFO util.PolicyRefresher:
> >> >> >> PolicyRefresher(serviceName=myservice): found updated version.
> >> >> >> lastKnownVersion=-1; newVersion=1
> >> >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save
> >> policies to
> >> >> >> cache file
> >> >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
> >> >> >> java.io.FileNotFoundException:
> >> >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No
> >> such
> >> >> file
> >> >> >> or directory)
> >> >> >> at java.io.FileOutputStream.open(Native Method)
> >> >> >> at
> java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> >> >> >> at
> java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> >> >> >> at java.io.FileWriter.<init>(FileWriter.java:90)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
> >> >> >> .... internal stacktrace....
> >> >> >> at java.lang.Thread.run(Thread.java:745)
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >Thanks.
> >> >> >
> >> >> >Thanks,
> >> >> >Rohit Sinha
> >> >> >
> >> >> >
> >> >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <
> madhan@apache.org <javascript:;>>
> >> >> wrote:
> >> >> >
> >> >> >> Rohit,
> >> >> >>
> >> >> >> To download policies from Ranger Admin, Ranger plugins require the
> >> URL
> >> >> to
> >> >> >> Ranger Admin and the name of the service containing the policies.
> >> These
> >> >> >> values are read from following configurations from a file named
> >> >> >> ranger-<pluginType>-security.xml
> >> >> >>
> >> >> >> ranger.plugin.<pluginType>.policy.rest.url
> >> >> >> ranger.plugin.<pluginType>.service.name
> >> >> >>
> >> >> >> For example, these are specified in
> >> conf/ranger-sampleapp-security.xml
> >> >> for
> >> >> >> the sample application.
> >> >> >>
> >> >> >> Can you please review the configuration for your plugin for the
> >> above?
> >> >> >>
> >> >> >> Hope this helps.
> >> >> >>
> >> >> >> Madhan
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <talktorohit54@gmail.com
> <javascript:;>> wrote:
> >> >> >>
> >> >> >> >Hello Madhan,
> >> >> >> >Thanks a lot for your reply.
> >> >> >> >
> >> >> >> >I am looking into the integration and I working towards
> developing
> >> the
> >> >> >> >components outside of ranger trunk for initial development
> purpose.
> >> >> After
> >> >> >> >taking an overview of the codebase it seems that it's possible to
> >> have
> >> >> >> >plugins outside of the ranger trunk too.
> >> >> >> >
> >> >> >> >I was able to add my service to Ranger Amin through the curl
> request
> >> >> and
> >> >> >> >connect to Ranger from my Authorizer but I see the following
> error:
> >> >> >> >
> >> >> >> >com.sun.jersey.api.client.ClientHandlerException:
> >> >> >> >> java.lang.IllegalArgumentException: URI is not absolute
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
> >> >> >> >> at
> >> com.sun.jersey.api.client.Client.handle(Client.java:648)
> >> >> >> >> at
> >> >> >> >>
> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
> >> >> >> >> at
> >> >> >> >>
> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> >> >> >> >> at
> >> >> >> >>
> >> >>
> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
> >> >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not
> absolute
> >> >> >> >> at java.net.URI.toURL(URI.java:1095)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> >> >> >> >> at
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> >> >> >> >> ... 8 more
> >> >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does
> not
> >> >> exist
> >> >> >> or
> >> >> >> >> not readble 'null'
> >> >> >> >
> >> >> >> >
> >> >> >> >A quick search on google pointed towards misconfiguration of
> service
> >> >> name
> >> >> >> >in security.xml which I have double checked.
> >> >> >> >
> >> >> >> >Any pointers to debug this will be appreciated.
> >> >> >> >
> >> >> >> >Thanks.
> >> >> >> >
> >> >> >> >Thanks,
> >> >> >> >Rohit Sinha
> >> >> >> >
> >> >> >> >
> >> >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
> >> >> >> >mneethiraj@hortonworks.com <javascript:;>> wrote:
> >> >> >> >
> >> >> >> >> Rohit,
> >> >> >> >>
> >> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the
> name
> >> of
> >> >> the
> >> >> >> >> service instance in Ranger Admin, which contains the policies
> for
> >> the
> >> >> >> >> component (in this case HBase). The plugin reads this value
> from a
> >> >> >> >> configuration named ranger.plugin.hbase.service.name (in file
> >> >> >> >> ranger-hbase-security.xml).
> >> >> >> >>
> >> >> >> >> >> After doing this I don't see anything in the Audit ->
> Plugins
> >> >> >> >> An entry will be created here for every policy download form
> >> plugins.
> >> >> >> >> Plugins download the policies at the following events:
> >> >> >> >> - during the component startup (HBase/HiveServer/...)
> >> >> >> >> - when there is a policy change in service instance
> >> >> >> >>
> >> >> >> >> Hope this helps.
> >> >> >> >>
> >> >> >> >> Madhan
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <talktorohit54@gmail.com
> <javascript:;>>
> >> wrote:
> >> >> >> >>
> >> >> >> >> >Thanks for sharing the SampleApp. I was able to run it
> understand
> >> >> the
> >> >> >> >> >integration point.
> >> >> >> >> >I also tried to enable HBase ranger plugin. When I added the
> >> service
> >> >> >> from
> >> >> >> >> >the Ranger Admin UI I was able to talk to HBase and the
> resource
> >> >> >> >> completion
> >> >> >> >> >worked. After that I deleted the HBase plugin from the UI and
> >> tried
> >> >> to
> >> >> >> >> >enable it from the the command line following instructions
> >> mentioned
> >> >> >> here:
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
> >> >> >> >> >
> >> >> >> >> >After doing this I don't see anything in the Audit ->
> Plugins. In
> >> >> the
> >> >> >> >> >instructions I do see warning
> >> >> >> >> >
> >> >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger
> Admin.
> >> If
> >> >> not,
> >> >> >> >> the
> >> >> >> >> >> hbase-plugin will not be able to communicate with Ranger
> admin.
> >> >> >> >> >
> >> >> >> >> >Does this mean I need to add the service from the Ranger Admin
> >> Panel
> >> >> >> or I
> >> >> >> >> >did something wrong in the enabling the plugin ?
> >> >> >> >> >
> >> >> >> >> >Any help will be highly appreciated.
> >> >> >> >> >
> >> >> >> >> >Thanks.
> >> >> >> >> >
> >> >> >> >> >Thanks,
> >> >> >> >> >Rohit Sinha
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
> >> >> talktorohit54@gmail.com <javascript:;>>
> >> >> >> >> >wrote:
> >> >> >> >> >
> >> >> >> >> >> Awesome. Thanks a lot.
> >> >> >> >> >>
> >> >> >> >> >> Thanks,
> >> >> >> >> >> Rohit Sinha
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
> >> >> bosco@apache.org <javascript:;>>
> >> >> >> >> wrote:
> >> >> >> >> >>
> >> >> >> >> >>> It is optional. It is easy to setup and helps a lot while
> >> >> debugging
> >> >> >> >> >>> during initial setup.
> >> >> >> >> >>>
> >> >> >> >> >>> Bosco
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <
> talktorohit54@gmail.com <javascript:;>>
> >> >> wrote:
> >> >> >> >> >>>
> >> >> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
> >> >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was
> really
> >> >> >> helpful
> >> >> >> >> in
> >> >> >> >> >>> >getting some understanding of the integration. I am going
> >> >> through
> >> >> >> the
> >> >> >> >> >>> >SampleApp docs now.
> >> >> >> >> >>> >
> >> >> >> >> >>> >Is auditing an optional feature ? From the documentation
> it
> >> >> looks
> >> >> >> like
> >> >> >> >> >>> it's
> >> >> >> >> >>> >not and Solr installation is a requirement. I was
> wondering
> >> if I
> >> >> >> can
> >> >> >> >> have
> >> >> >> >> >>> >auditing off and skip Solr installation for initial
> >> integration
> >> >> >> >> purpose.
> >> >> >> >> >>> >
> >> >> >> >> >>> >Thanks.
> >> >> >> >> >>> >
> >> >> >> >> >>> >Thanks,
> >> >> >> >> >>> >Rohit Sinha
> >> >> >> >> >>> >
> >> >> >> >> >>> >
> >> >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
> >> >> >> madhan@apache.org <javascript:;>>
> >> >> >> >> >>> wrote:
> >> >> >> >> >>> >
> >> >> >> >> >>> >> Rohit,
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> In addition to the details in the wiki, I would
> recommend
> >> >> >> reviewing
> >> >> >> >> the
> >> >> >> >> >>> >> following sample application to understand the details
> of
> >> >> adding
> >> >> >> >> Ranger
> >> >> >> >> >>> >> authorization to an application.
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> - README.txt:
> >> >> >> >> >>> >>
> >> >> >> >> >>>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >> >> >> >> >>> >> - Application sources:
> >> >> >> >> >>> >>
> >> >> >> >> >>>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >> >> >> >> >>> >> .
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> Madhan
> >> >> >> >> >>> >>
> >> >> >> >> >>> >>
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
> >> >> >> >> >>> vperiasamy@hortonworks.com <javascript:;>
> >> >> >> >> >>> >> on behalf of vel@apache.org <javascript:;>> wrote:
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> >Rohit -
> >> >> >> >> >>> >> >
> >> >> >> >> >>>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >> >> >> >> >>> >> >explains how to add a custom plugin for Ranger.
> >> >> >> >> >>> >> >
> >> >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <
> >> talktorohit54@gmail.com <javascript:;>>
> >> >> >> >> wrote:
> >> >> >> >> >>> >> >
> >> >> >> >> >>> >> >>Hello,
> >> >> >> >> >>> >> >>I am looking into integrating an external service with
> >> >> Apache
> >> >> >> >> Ranger
> >> >> >> >> >>> for
> >> >> >> >> >>> >> >>authorization.
> >> >> >> >> >>> >> >>I looked up the wiki but there is no information about
> >> >> >> integrating
> >> >> >> >> >>> new
> >> >> >> >> >>> >> >>services.
> >> >> >> >> >>> >> >>Can someone give me some info which might be helpful
> in
> >> >> >> >> identifying
> >> >> >> >> >>> >> >>different components which needs to be developed and
> >> other
> >> >> >> >> required
> >> >> >> >> >>> stuff
> >> >> >> >> >>> >> >>
> >> >> >> >> >>> >> >>Thanks.
> >> >> >> >> >>> >> >
> >> >> >> >> >>> >> >
> >> >> >> >> >>> >> >
> >> >> >> >> >>> >>
> >> >> >> >> >>> >>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >>
> >> >>
> >>
> >>
>
>
--
Thanks,
Rohit Sinha
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
Your question at the end is the answer you are looking for!
The username and password given while adding a service is used by Ranger Admin
while performing lookup. In case of secure cluster, the username should be the
Kerberos principal. This user must have enough permissions in the component
(via a Ranger policy?) to perform the lookup.
Hope this help.
Madhan
On 4/8/16, 5:31 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello Madhan,
>
>I am having some difficulty understanding how ranger and services identify
>each other on a secure cluster.
>
>For example, if I have a resource lookup plugin running in ranger for HBase
>and a ranger authorization plugin running in HBase and when they make
>request to each other how does ranger make sure that the talking service in
>HBase and also how does HBase identifies the talking service in ranger.
>
>Also, for every service we take username and password while the service is
>being added. What is it used for ?
>
>Thanks.
>
>
>Thanks,
>Rohit Sinha
>
>
>On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <ma...@apache.org> wrote:
>
>> Rohit,
>>
>> >> 1. Can you point me to some resource where I can see how to configure
>> Ranger on Kerberos without KMS.
>> Ranger configuration does not depend on presence or absence of KMS. It will
>> help if you can provide more details on what you are trying to do or what
>> difference/difficulty you see due to presence/absence of KMS.
>>
>> >> 2. Also, how can I list all the policies for a resource in my service
>>
>>
>>
>> RangerBasePlugin is designed for authorizing accesses and don’t provide
>> APIs to
>> search the policies. You can use REST APIs of Ranger Admin to search
>> policies.
>>
>>
>> For example: curl -f -X GET -H "Accept: application/json" -u
>> admin-user:admin-pass
>> http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
>>
>> For details of the REST APIs, please refer to:
>> https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
>>
>> Thanks,
>> Madhan
>>
>> On 4/7/16, 9:28 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Hello Madhan,
>> >Thanks a lot for the prompt reply.
>> >
>> >1. Can you point me to some resource where I can see how to configure
>> >Ranger on Kerberos without KMS.
>> >
>> >2. Also, how can I list all the policies for a resource in my service. I
>> >don't see any api exposed by RangerBasePlugin for this. I was looking into
>> >the code base and saw that ServiceStore has some APIs for this. How can I
>> >access this ?
>> >
>> >Thanks.
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>> >
>> >> Rohit,
>> >>
>> >> Thanks for sharing your feedback on Ranger integration. Good to know
>> >> that you are able to get Ranger authorization working in your
>> application.
>> >>
>> >> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
>> >> >> with secure Hadoop through Kerberos ?
>> >> Yes. Ranger works with secure Hadoop - either unencrypted or encrypted.
>> >>
>> >> >> 2. Currently, I see the following error log for policy cache file.
>> >> >> Isn’t the policy cache file created automatically ?
>> >> Yes, the policy cache file is created automatically. I guess the error
>> >>
>> >> you see could be cause by non-existent directory where the cache file
>> >> would be saved - in this case /etc/ranger/myservicedev/policycache/.
>> >> Please ensure that this directory exists and has write permission for
>> >> the user that runs the application.
>> >>
>> >> The directory location can be specified via configuration
>> >> ranger.plugin.myservice.policy.cache.dir in
>> ranger-myservice-security.xml.
>> >>
>> >> Hope this helps.
>> >>
>> >> Madhan
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On 4/7/16, 7:38 PM, "rohit sinha" <ta...@gmail.com> wrote:
>> >>
>> >> >Hello,
>> >> >
>> >> >Thanks a lot for your prompt replies. It was really helpful.
>> >> >Yes, it seems like there was some misconfiguration issue. I was able to
>> >> >make it work and I have a very basic integration up and running with my
>> >> >service now. I have worked on integration/analysis of other
>> authorization
>> >> >models and I must say that Ranger so far has been the easiest to
>> integrate
>> >> >with (at the basic level) and the documentation helped a lot.
>> Appreciate
>> >> >your efforts in building it so far.
>> >> >
>> >> >I have two question which I am trying to figure out. Will appreciate if
>> >> >someone can provide some pointers.
>> >> >
>> >> >1. Kerberos: From the documentation found here
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
>> >> >I understand that Ranger works with KMS on an encrypted HDFS. Is it
>> >> >possible to have Ranger running on an unencrypted HDFS with secure
>> Hadoop
>> >> >through Kerberos ? Something where I can talk to Ranger by giving a
>> >> >principal and keytab ?
>> >> >
>> >> >2. Currently, I see the following error log for policy cache file.
>> Isn't
>> >> >the policy cache file created automatically ? Am I missing some
>> >> >configuration.
>> >> >
>> >> >16/04/07 21:26:18 INFO util.PolicyRefresher:
>> >> >> PolicyRefresher(serviceName=myservice): found updated version.
>> >> >> lastKnownVersion=-1; newVersion=1
>> >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save
>> policies to
>> >> >> cache file
>> >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
>> >> >> java.io.FileNotFoundException:
>> >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No
>> such
>> >> file
>> >> >> or directory)
>> >> >> at java.io.FileOutputStream.open(Native Method)
>> >> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> >> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> >> >> at java.io.FileWriter.<init>(FileWriter.java:90)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
>> >> >> .... internal stacktrace....
>> >> >> at java.lang.Thread.run(Thread.java:745)
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >Thanks.
>> >> >
>> >> >Thanks,
>> >> >Rohit Sinha
>> >> >
>> >> >
>> >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <ma...@apache.org>
>> >> wrote:
>> >> >
>> >> >> Rohit,
>> >> >>
>> >> >> To download policies from Ranger Admin, Ranger plugins require the
>> URL
>> >> to
>> >> >> Ranger Admin and the name of the service containing the policies.
>> These
>> >> >> values are read from following configurations from a file named
>> >> >> ranger-<pluginType>-security.xml
>> >> >>
>> >> >> ranger.plugin.<pluginType>.policy.rest.url
>> >> >> ranger.plugin.<pluginType>.service.name
>> >> >>
>> >> >> For example, these are specified in
>> conf/ranger-sampleapp-security.xml
>> >> for
>> >> >> the sample application.
>> >> >>
>> >> >> Can you please review the configuration for your plugin for the
>> above?
>> >> >>
>> >> >> Hope this helps.
>> >> >>
>> >> >> Madhan
>> >> >>
>> >> >>
>> >> >>
>> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
>> >> >>
>> >> >> >Hello Madhan,
>> >> >> >Thanks a lot for your reply.
>> >> >> >
>> >> >> >I am looking into the integration and I working towards developing
>> the
>> >> >> >components outside of ranger trunk for initial development purpose.
>> >> After
>> >> >> >taking an overview of the codebase it seems that it's possible to
>> have
>> >> >> >plugins outside of the ranger trunk too.
>> >> >> >
>> >> >> >I was able to add my service to Ranger Amin through the curl request
>> >> and
>> >> >> >connect to Ranger from my Authorizer but I see the following error:
>> >> >> >
>> >> >> >com.sun.jersey.api.client.ClientHandlerException:
>> >> >> >> java.lang.IllegalArgumentException: URI is not absolute
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
>> >> >> >> at
>> com.sun.jersey.api.client.Client.handle(Client.java:648)
>> >> >> >> at
>> >> >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> >> >> >> at
>> >> >> >>
>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> >> >> >> at
>> >> >> >>
>> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
>> >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute
>> >> >> >> at java.net.URI.toURL(URI.java:1095)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
>> >> >> >> at
>> >> >> >>
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
>> >> >> >> ... 8 more
>> >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not
>> >> exist
>> >> >> or
>> >> >> >> not readble 'null'
>> >> >> >
>> >> >> >
>> >> >> >A quick search on google pointed towards misconfiguration of service
>> >> name
>> >> >> >in security.xml which I have double checked.
>> >> >> >
>> >> >> >Any pointers to debug this will be appreciated.
>> >> >> >
>> >> >> >Thanks.
>> >> >> >
>> >> >> >Thanks,
>> >> >> >Rohit Sinha
>> >> >> >
>> >> >> >
>> >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>> >> >> >mneethiraj@hortonworks.com> wrote:
>> >> >> >
>> >> >> >> Rohit,
>> >> >> >>
>> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the name
>> of
>> >> the
>> >> >> >> service instance in Ranger Admin, which contains the policies for
>> the
>> >> >> >> component (in this case HBase). The plugin reads this value from a
>> >> >> >> configuration named ranger.plugin.hbase.service.name (in file
>> >> >> >> ranger-hbase-security.xml).
>> >> >> >>
>> >> >> >> >> After doing this I don't see anything in the Audit -> Plugins
>> >> >> >> An entry will be created here for every policy download form
>> plugins.
>> >> >> >> Plugins download the policies at the following events:
>> >> >> >> - during the component startup (HBase/HiveServer/...)
>> >> >> >> - when there is a policy change in service instance
>> >> >> >>
>> >> >> >> Hope this helps.
>> >> >> >>
>> >> >> >> Madhan
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com>
>> wrote:
>> >> >> >>
>> >> >> >> >Thanks for sharing the SampleApp. I was able to run it understand
>> >> the
>> >> >> >> >integration point.
>> >> >> >> >I also tried to enable HBase ranger plugin. When I added the
>> service
>> >> >> from
>> >> >> >> >the Ranger Admin UI I was able to talk to HBase and the resource
>> >> >> >> completion
>> >> >> >> >worked. After that I deleted the HBase plugin from the UI and
>> tried
>> >> to
>> >> >> >> >enable it from the the command line following instructions
>> mentioned
>> >> >> here:
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >> >> >> >
>> >> >> >> >After doing this I don't see anything in the Audit -> Plugins. In
>> >> the
>> >> >> >> >instructions I do see warning
>> >> >> >> >
>> >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin.
>> If
>> >> not,
>> >> >> >> the
>> >> >> >> >> hbase-plugin will not be able to communicate with Ranger admin.
>> >> >> >> >
>> >> >> >> >Does this mean I need to add the service from the Ranger Admin
>> Panel
>> >> >> or I
>> >> >> >> >did something wrong in the enabling the plugin ?
>> >> >> >> >
>> >> >> >> >Any help will be highly appreciated.
>> >> >> >> >
>> >> >> >> >Thanks.
>> >> >> >> >
>> >> >> >> >Thanks,
>> >> >> >> >Rohit Sinha
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
>> >> talktorohit54@gmail.com>
>> >> >> >> >wrote:
>> >> >> >> >
>> >> >> >> >> Awesome. Thanks a lot.
>> >> >> >> >>
>> >> >> >> >> Thanks,
>> >> >> >> >> Rohit Sinha
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
>> >> bosco@apache.org>
>> >> >> >> wrote:
>> >> >> >> >>
>> >> >> >> >>> It is optional. It is easy to setup and helps a lot while
>> >> debugging
>> >> >> >> >>> during initial setup.
>> >> >> >> >>>
>> >> >> >> >>> Bosco
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com>
>> >> wrote:
>> >> >> >> >>>
>> >> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
>> >> >> helpful
>> >> >> >> in
>> >> >> >> >>> >getting some understanding of the integration. I am going
>> >> through
>> >> >> the
>> >> >> >> >>> >SampleApp docs now.
>> >> >> >> >>> >
>> >> >> >> >>> >Is auditing an optional feature ? From the documentation it
>> >> looks
>> >> >> like
>> >> >> >> >>> it's
>> >> >> >> >>> >not and Solr installation is a requirement. I was wondering
>> if I
>> >> >> can
>> >> >> >> have
>> >> >> >> >>> >auditing off and skip Solr installation for initial
>> integration
>> >> >> >> purpose.
>> >> >> >> >>> >
>> >> >> >> >>> >Thanks.
>> >> >> >> >>> >
>> >> >> >> >>> >Thanks,
>> >> >> >> >>> >Rohit Sinha
>> >> >> >> >>> >
>> >> >> >> >>> >
>> >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
>> >> >> madhan@apache.org>
>> >> >> >> >>> wrote:
>> >> >> >> >>> >
>> >> >> >> >>> >> Rohit,
>> >> >> >> >>> >>
>> >> >> >> >>> >> In addition to the details in the wiki, I would recommend
>> >> >> reviewing
>> >> >> >> the
>> >> >> >> >>> >> following sample application to understand the details of
>> >> adding
>> >> >> >> Ranger
>> >> >> >> >>> >> authorization to an application.
>> >> >> >> >>> >>
>> >> >> >> >>> >> - README.txt:
>> >> >> >> >>> >>
>> >> >> >> >>>
>> >> >> >>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >> >> >> >>> >> - Application sources:
>> >> >> >> >>> >>
>> >> >> >> >>>
>> >> >> >>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >> >> >> >>> >> .
>> >> >> >> >>> >>
>> >> >> >> >>> >> Madhan
>> >> >> >> >>> >>
>> >> >> >> >>> >>
>> >> >> >> >>> >>
>> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >> >> >> >>> vperiasamy@hortonworks.com
>> >> >> >> >>> >> on behalf of vel@apache.org> wrote:
>> >> >> >> >>> >>
>> >> >> >> >>> >> >Rohit -
>> >> >> >> >>> >> >
>> >> >> >> >>>
>> >> >> >>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >> >> >> >>> >> >explains how to add a custom plugin for Ranger.
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <
>> talktorohit54@gmail.com>
>> >> >> >> wrote:
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >>Hello,
>> >> >> >> >>> >> >>I am looking into integrating an external service with
>> >> Apache
>> >> >> >> Ranger
>> >> >> >> >>> for
>> >> >> >> >>> >> >>authorization.
>> >> >> >> >>> >> >>I looked up the wiki but there is no information about
>> >> >> integrating
>> >> >> >> >>> new
>> >> >> >> >>> >> >>services.
>> >> >> >> >>> >> >>Can someone give me some info which might be helpful in
>> >> >> >> identifying
>> >> >> >> >>> >> >>different components which needs to be developed and
>> other
>> >> >> >> required
>> >> >> >> >>> stuff
>> >> >> >> >>> >> >>
>> >> >> >> >>> >> >>Thanks.
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >
>> >> >> >> >>> >> >
>> >> >> >> >>> >>
>> >> >> >> >>> >>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Hello Madhan,
I am having some difficulty understanding how ranger and services identify
each other on a secure cluster.
For example, if I have a resource lookup plugin running in ranger for HBase
and a ranger authorization plugin running in HBase and when they make
request to each other how does ranger make sure that the talking service in
HBase and also how does HBase identifies the talking service in ranger.
Also, for every service we take username and password while the service is
being added. What is it used for ?
Thanks.
Thanks,
Rohit Sinha
On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> >> 1. Can you point me to some resource where I can see how to configure
> Ranger on Kerberos without KMS.
> Ranger configuration does not depend on presence or absence of KMS. It will
> help if you can provide more details on what you are trying to do or what
> difference/difficulty you see due to presence/absence of KMS.
>
> >> 2. Also, how can I list all the policies for a resource in my service
>
>
>
> RangerBasePlugin is designed for authorizing accesses and don’t provide
> APIs to
> search the policies. You can use REST APIs of Ranger Admin to search
> policies.
>
>
> For example: curl -f -X GET -H "Accept: application/json" -u
> admin-user:admin-pass
> http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
>
> For details of the REST APIs, please refer to:
> https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
>
> Thanks,
> Madhan
>
> On 4/7/16, 9:28 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Hello Madhan,
> >Thanks a lot for the prompt reply.
> >
> >1. Can you point me to some resource where I can see how to configure
> >Ranger on Kerberos without KMS.
> >
> >2. Also, how can I list all the policies for a resource in my service. I
> >don't see any api exposed by RangerBasePlugin for this. I was looking into
> >the code base and saw that ServiceStore has some APIs for this. How can I
> >access this ?
> >
> >Thanks.
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
> >
> >> Rohit,
> >>
> >> Thanks for sharing your feedback on Ranger integration. Good to know
> >> that you are able to get Ranger authorization working in your
> application.
> >>
> >> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
> >> >> with secure Hadoop through Kerberos ?
> >> Yes. Ranger works with secure Hadoop - either unencrypted or encrypted.
> >>
> >> >> 2. Currently, I see the following error log for policy cache file.
> >> >> Isn’t the policy cache file created automatically ?
> >> Yes, the policy cache file is created automatically. I guess the error
> >>
> >> you see could be cause by non-existent directory where the cache file
> >> would be saved - in this case /etc/ranger/myservicedev/policycache/.
> >> Please ensure that this directory exists and has write permission for
> >> the user that runs the application.
> >>
> >> The directory location can be specified via configuration
> >> ranger.plugin.myservice.policy.cache.dir in
> ranger-myservice-security.xml.
> >>
> >> Hope this helps.
> >>
> >> Madhan
> >>
> >>
> >>
> >>
> >>
> >> On 4/7/16, 7:38 PM, "rohit sinha" <ta...@gmail.com> wrote:
> >>
> >> >Hello,
> >> >
> >> >Thanks a lot for your prompt replies. It was really helpful.
> >> >Yes, it seems like there was some misconfiguration issue. I was able to
> >> >make it work and I have a very basic integration up and running with my
> >> >service now. I have worked on integration/analysis of other
> authorization
> >> >models and I must say that Ranger so far has been the easiest to
> integrate
> >> >with (at the basic level) and the documentation helped a lot.
> Appreciate
> >> >your efforts in building it so far.
> >> >
> >> >I have two question which I am trying to figure out. Will appreciate if
> >> >someone can provide some pointers.
> >> >
> >> >1. Kerberos: From the documentation found here
> >> >
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
> >> >I understand that Ranger works with KMS on an encrypted HDFS. Is it
> >> >possible to have Ranger running on an unencrypted HDFS with secure
> Hadoop
> >> >through Kerberos ? Something where I can talk to Ranger by giving a
> >> >principal and keytab ?
> >> >
> >> >2. Currently, I see the following error log for policy cache file.
> Isn't
> >> >the policy cache file created automatically ? Am I missing some
> >> >configuration.
> >> >
> >> >16/04/07 21:26:18 INFO util.PolicyRefresher:
> >> >> PolicyRefresher(serviceName=myservice): found updated version.
> >> >> lastKnownVersion=-1; newVersion=1
> >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save
> policies to
> >> >> cache file
> >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
> >> >> java.io.FileNotFoundException:
> >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No
> such
> >> file
> >> >> or directory)
> >> >> at java.io.FileOutputStream.open(Native Method)
> >> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> >> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> >> >> at java.io.FileWriter.<init>(FileWriter.java:90)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
> >> >> .... internal stacktrace....
> >> >> at java.lang.Thread.run(Thread.java:745)
> >> >
> >> >
> >> >
> >> >
> >> >Thanks.
> >> >
> >> >Thanks,
> >> >Rohit Sinha
> >> >
> >> >
> >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <ma...@apache.org>
> >> wrote:
> >> >
> >> >> Rohit,
> >> >>
> >> >> To download policies from Ranger Admin, Ranger plugins require the
> URL
> >> to
> >> >> Ranger Admin and the name of the service containing the policies.
> These
> >> >> values are read from following configurations from a file named
> >> >> ranger-<pluginType>-security.xml
> >> >>
> >> >> ranger.plugin.<pluginType>.policy.rest.url
> >> >> ranger.plugin.<pluginType>.service.name
> >> >>
> >> >> For example, these are specified in
> conf/ranger-sampleapp-security.xml
> >> for
> >> >> the sample application.
> >> >>
> >> >> Can you please review the configuration for your plugin for the
> above?
> >> >>
> >> >> Hope this helps.
> >> >>
> >> >> Madhan
> >> >>
> >> >>
> >> >>
> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
> >> >>
> >> >> >Hello Madhan,
> >> >> >Thanks a lot for your reply.
> >> >> >
> >> >> >I am looking into the integration and I working towards developing
> the
> >> >> >components outside of ranger trunk for initial development purpose.
> >> After
> >> >> >taking an overview of the codebase it seems that it's possible to
> have
> >> >> >plugins outside of the ranger trunk too.
> >> >> >
> >> >> >I was able to add my service to Ranger Amin through the curl request
> >> and
> >> >> >connect to Ranger from my Authorizer but I see the following error:
> >> >> >
> >> >> >com.sun.jersey.api.client.ClientHandlerException:
> >> >> >> java.lang.IllegalArgumentException: URI is not absolute
> >> >> >> at
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
> >> >> >> at
> com.sun.jersey.api.client.Client.handle(Client.java:648)
> >> >> >> at
> >> >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
> >> >> >> at
> >> >> >>
> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> >> >> >> at
> >> >> >>
> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
> >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute
> >> >> >> at java.net.URI.toURL(URI.java:1095)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> >> >> >> at
> >> >> >>
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> >> >> >> ... 8 more
> >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not
> >> exist
> >> >> or
> >> >> >> not readble 'null'
> >> >> >
> >> >> >
> >> >> >A quick search on google pointed towards misconfiguration of service
> >> name
> >> >> >in security.xml which I have double checked.
> >> >> >
> >> >> >Any pointers to debug this will be appreciated.
> >> >> >
> >> >> >Thanks.
> >> >> >
> >> >> >Thanks,
> >> >> >Rohit Sinha
> >> >> >
> >> >> >
> >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
> >> >> >mneethiraj@hortonworks.com> wrote:
> >> >> >
> >> >> >> Rohit,
> >> >> >>
> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the name
> of
> >> the
> >> >> >> service instance in Ranger Admin, which contains the policies for
> the
> >> >> >> component (in this case HBase). The plugin reads this value from a
> >> >> >> configuration named ranger.plugin.hbase.service.name (in file
> >> >> >> ranger-hbase-security.xml).
> >> >> >>
> >> >> >> >> After doing this I don't see anything in the Audit -> Plugins
> >> >> >> An entry will be created here for every policy download form
> plugins.
> >> >> >> Plugins download the policies at the following events:
> >> >> >> - during the component startup (HBase/HiveServer/...)
> >> >> >> - when there is a policy change in service instance
> >> >> >>
> >> >> >> Hope this helps.
> >> >> >>
> >> >> >> Madhan
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com>
> wrote:
> >> >> >>
> >> >> >> >Thanks for sharing the SampleApp. I was able to run it understand
> >> the
> >> >> >> >integration point.
> >> >> >> >I also tried to enable HBase ranger plugin. When I added the
> service
> >> >> from
> >> >> >> >the Ranger Admin UI I was able to talk to HBase and the resource
> >> >> >> completion
> >> >> >> >worked. After that I deleted the HBase plugin from the UI and
> tried
> >> to
> >> >> >> >enable it from the the command line following instructions
> mentioned
> >> >> here:
> >> >> >> >
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
> >> >> >> >
> >> >> >> >After doing this I don't see anything in the Audit -> Plugins. In
> >> the
> >> >> >> >instructions I do see warning
> >> >> >> >
> >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin.
> If
> >> not,
> >> >> >> the
> >> >> >> >> hbase-plugin will not be able to communicate with Ranger admin.
> >> >> >> >
> >> >> >> >Does this mean I need to add the service from the Ranger Admin
> Panel
> >> >> or I
> >> >> >> >did something wrong in the enabling the plugin ?
> >> >> >> >
> >> >> >> >Any help will be highly appreciated.
> >> >> >> >
> >> >> >> >Thanks.
> >> >> >> >
> >> >> >> >Thanks,
> >> >> >> >Rohit Sinha
> >> >> >> >
> >> >> >> >
> >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
> >> talktorohit54@gmail.com>
> >> >> >> >wrote:
> >> >> >> >
> >> >> >> >> Awesome. Thanks a lot.
> >> >> >> >>
> >> >> >> >> Thanks,
> >> >> >> >> Rohit Sinha
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
> >> bosco@apache.org>
> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >>> It is optional. It is easy to setup and helps a lot while
> >> debugging
> >> >> >> >>> during initial setup.
> >> >> >> >>>
> >> >> >> >>> Bosco
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com>
> >> wrote:
> >> >> >> >>>
> >> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
> >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
> >> >> helpful
> >> >> >> in
> >> >> >> >>> >getting some understanding of the integration. I am going
> >> through
> >> >> the
> >> >> >> >>> >SampleApp docs now.
> >> >> >> >>> >
> >> >> >> >>> >Is auditing an optional feature ? From the documentation it
> >> looks
> >> >> like
> >> >> >> >>> it's
> >> >> >> >>> >not and Solr installation is a requirement. I was wondering
> if I
> >> >> can
> >> >> >> have
> >> >> >> >>> >auditing off and skip Solr installation for initial
> integration
> >> >> >> purpose.
> >> >> >> >>> >
> >> >> >> >>> >Thanks.
> >> >> >> >>> >
> >> >> >> >>> >Thanks,
> >> >> >> >>> >Rohit Sinha
> >> >> >> >>> >
> >> >> >> >>> >
> >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
> >> >> madhan@apache.org>
> >> >> >> >>> wrote:
> >> >> >> >>> >
> >> >> >> >>> >> Rohit,
> >> >> >> >>> >>
> >> >> >> >>> >> In addition to the details in the wiki, I would recommend
> >> >> reviewing
> >> >> >> the
> >> >> >> >>> >> following sample application to understand the details of
> >> adding
> >> >> >> Ranger
> >> >> >> >>> >> authorization to an application.
> >> >> >> >>> >>
> >> >> >> >>> >> - README.txt:
> >> >> >> >>> >>
> >> >> >> >>>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >> >> >> >>> >> - Application sources:
> >> >> >> >>> >>
> >> >> >> >>>
> >> >> >>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >> >> >> >>> >> .
> >> >> >> >>> >>
> >> >> >> >>> >> Madhan
> >> >> >> >>> >>
> >> >> >> >>> >>
> >> >> >> >>> >>
> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
> >> >> >> >>> vperiasamy@hortonworks.com
> >> >> >> >>> >> on behalf of vel@apache.org> wrote:
> >> >> >> >>> >>
> >> >> >> >>> >> >Rohit -
> >> >> >> >>> >> >
> >> >> >> >>>
> >> >> >>
> >> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >> >> >> >>> >> >explains how to add a custom plugin for Ranger.
> >> >> >> >>> >> >
> >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <
> talktorohit54@gmail.com>
> >> >> >> wrote:
> >> >> >> >>> >> >
> >> >> >> >>> >> >>Hello,
> >> >> >> >>> >> >>I am looking into integrating an external service with
> >> Apache
> >> >> >> Ranger
> >> >> >> >>> for
> >> >> >> >>> >> >>authorization.
> >> >> >> >>> >> >>I looked up the wiki but there is no information about
> >> >> integrating
> >> >> >> >>> new
> >> >> >> >>> >> >>services.
> >> >> >> >>> >> >>Can someone give me some info which might be helpful in
> >> >> >> identifying
> >> >> >> >>> >> >>different components which needs to be developed and
> other
> >> >> >> required
> >> >> >> >>> stuff
> >> >> >> >>> >> >>
> >> >> >> >>> >> >>Thanks.
> >> >> >> >>> >> >
> >> >> >> >>> >> >
> >> >> >> >>> >> >
> >> >> >> >>> >>
> >> >> >> >>> >>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>
> >> >> >>
> >> >>
> >> >>
> >>
> >>
>
>
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
>> 1. Can you point me to some resource where I can see how to configure Ranger on Kerberos without KMS.
Ranger configuration does not depend on presence or absence of KMS. It will
help if you can provide more details on what you are trying to do or what
difference/difficulty you see due to presence/absence of KMS.
>> 2. Also, how can I list all the policies for a resource in my service
RangerBasePlugin is designed for authorizing accesses and don’t provide APIs to
search the policies. You can use REST APIs of Ranger Admin to search policies.
For example: curl -f -X GET -H "Accept: application/json" -u admin-user:admin-pass http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1
For details of the REST APIs, please refer to: https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management
Thanks,
Madhan
On 4/7/16, 9:28 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello Madhan,
>Thanks a lot for the prompt reply.
>
>1. Can you point me to some resource where I can see how to configure
>Ranger on Kerberos without KMS.
>
>2. Also, how can I list all the policies for a resource in my service. I
>don't see any api exposed by RangerBasePlugin for this. I was looking into
>the code base and saw that ServiceStore has some APIs for this. How can I
>access this ?
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <ma...@apache.org> wrote:
>
>> Rohit,
>>
>> Thanks for sharing your feedback on Ranger integration. Good to know
>> that you are able to get Ranger authorization working in your application.
>>
>> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
>> >> with secure Hadoop through Kerberos ?
>> Yes. Ranger works with secure Hadoop - either unencrypted or encrypted.
>>
>> >> 2. Currently, I see the following error log for policy cache file.
>> >> Isn’t the policy cache file created automatically ?
>> Yes, the policy cache file is created automatically. I guess the error
>>
>> you see could be cause by non-existent directory where the cache file
>> would be saved - in this case /etc/ranger/myservicedev/policycache/.
>> Please ensure that this directory exists and has write permission for
>> the user that runs the application.
>>
>> The directory location can be specified via configuration
>> ranger.plugin.myservice.policy.cache.dir in ranger-myservice-security.xml.
>>
>> Hope this helps.
>>
>> Madhan
>>
>>
>>
>>
>>
>> On 4/7/16, 7:38 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Hello,
>> >
>> >Thanks a lot for your prompt replies. It was really helpful.
>> >Yes, it seems like there was some misconfiguration issue. I was able to
>> >make it work and I have a very basic integration up and running with my
>> >service now. I have worked on integration/analysis of other authorization
>> >models and I must say that Ranger so far has been the easiest to integrate
>> >with (at the basic level) and the documentation helped a lot. Appreciate
>> >your efforts in building it so far.
>> >
>> >I have two question which I am trying to figure out. Will appreciate if
>> >someone can provide some pointers.
>> >
>> >1. Kerberos: From the documentation found here
>> >
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
>> >I understand that Ranger works with KMS on an encrypted HDFS. Is it
>> >possible to have Ranger running on an unencrypted HDFS with secure Hadoop
>> >through Kerberos ? Something where I can talk to Ranger by giving a
>> >principal and keytab ?
>> >
>> >2. Currently, I see the following error log for policy cache file. Isn't
>> >the policy cache file created automatically ? Am I missing some
>> >configuration.
>> >
>> >16/04/07 21:26:18 INFO util.PolicyRefresher:
>> >> PolicyRefresher(serviceName=myservice): found updated version.
>> >> lastKnownVersion=-1; newVersion=1
>> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save policies to
>> >> cache file
>> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
>> >> java.io.FileNotFoundException:
>> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No such
>> file
>> >> or directory)
>> >> at java.io.FileOutputStream.open(Native Method)
>> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> >> at java.io.FileWriter.<init>(FileWriter.java:90)
>> >> at
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
>> >> at
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
>> >> at
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
>> >> at
>> >>
>> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
>> >> .... internal stacktrace....
>> >> at java.lang.Thread.run(Thread.java:745)
>> >
>> >
>> >
>> >
>> >Thanks.
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>> >
>> >> Rohit,
>> >>
>> >> To download policies from Ranger Admin, Ranger plugins require the URL
>> to
>> >> Ranger Admin and the name of the service containing the policies. These
>> >> values are read from following configurations from a file named
>> >> ranger-<pluginType>-security.xml
>> >>
>> >> ranger.plugin.<pluginType>.policy.rest.url
>> >> ranger.plugin.<pluginType>.service.name
>> >>
>> >> For example, these are specified in conf/ranger-sampleapp-security.xml
>> for
>> >> the sample application.
>> >>
>> >> Can you please review the configuration for your plugin for the above?
>> >>
>> >> Hope this helps.
>> >>
>> >> Madhan
>> >>
>> >>
>> >>
>> >> On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
>> >>
>> >> >Hello Madhan,
>> >> >Thanks a lot for your reply.
>> >> >
>> >> >I am looking into the integration and I working towards developing the
>> >> >components outside of ranger trunk for initial development purpose.
>> After
>> >> >taking an overview of the codebase it seems that it's possible to have
>> >> >plugins outside of the ranger trunk too.
>> >> >
>> >> >I was able to add my service to Ranger Amin through the curl request
>> and
>> >> >connect to Ranger from my Authorizer but I see the following error:
>> >> >
>> >> >com.sun.jersey.api.client.ClientHandlerException:
>> >> >> java.lang.IllegalArgumentException: URI is not absolute
>> >> >> at
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
>> >> >> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>> >> >> at
>> >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> >> >> at
>> >> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> >> >> at
>> >> >>
>> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
>> >> >> at
>> >> >>
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
>> >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute
>> >> >> at java.net.URI.toURL(URI.java:1095)
>> >> >> at
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
>> >> >> at
>> >> >>
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
>> >> >> ... 8 more
>> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not
>> exist
>> >> or
>> >> >> not readble 'null'
>> >> >
>> >> >
>> >> >A quick search on google pointed towards misconfiguration of service
>> name
>> >> >in security.xml which I have double checked.
>> >> >
>> >> >Any pointers to debug this will be appreciated.
>> >> >
>> >> >Thanks.
>> >> >
>> >> >Thanks,
>> >> >Rohit Sinha
>> >> >
>> >> >
>> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>> >> >mneethiraj@hortonworks.com> wrote:
>> >> >
>> >> >> Rohit,
>> >> >>
>> >> >> You are right. REPOSITORY_NAME referenced in the doc is the name of
>> the
>> >> >> service instance in Ranger Admin, which contains the policies for the
>> >> >> component (in this case HBase). The plugin reads this value from a
>> >> >> configuration named ranger.plugin.hbase.service.name (in file
>> >> >> ranger-hbase-security.xml).
>> >> >>
>> >> >> >> After doing this I don't see anything in the Audit -> Plugins
>> >> >> An entry will be created here for every policy download form plugins.
>> >> >> Plugins download the policies at the following events:
>> >> >> - during the component startup (HBase/HiveServer/...)
>> >> >> - when there is a policy change in service instance
>> >> >>
>> >> >> Hope this helps.
>> >> >>
>> >> >> Madhan
>> >> >>
>> >> >>
>> >> >>
>> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
>> >> >>
>> >> >> >Thanks for sharing the SampleApp. I was able to run it understand
>> the
>> >> >> >integration point.
>> >> >> >I also tried to enable HBase ranger plugin. When I added the service
>> >> from
>> >> >> >the Ranger Admin UI I was able to talk to HBase and the resource
>> >> >> completion
>> >> >> >worked. After that I deleted the HBase plugin from the UI and tried
>> to
>> >> >> >enable it from the the command line following instructions mentioned
>> >> here:
>> >> >> >
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >> >> >
>> >> >> >After doing this I don't see anything in the Audit -> Plugins. In
>> the
>> >> >> >instructions I do see warning
>> >> >> >
>> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If
>> not,
>> >> >> the
>> >> >> >> hbase-plugin will not be able to communicate with Ranger admin.
>> >> >> >
>> >> >> >Does this mean I need to add the service from the Ranger Admin Panel
>> >> or I
>> >> >> >did something wrong in the enabling the plugin ?
>> >> >> >
>> >> >> >Any help will be highly appreciated.
>> >> >> >
>> >> >> >Thanks.
>> >> >> >
>> >> >> >Thanks,
>> >> >> >Rohit Sinha
>> >> >> >
>> >> >> >
>> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
>> talktorohit54@gmail.com>
>> >> >> >wrote:
>> >> >> >
>> >> >> >> Awesome. Thanks a lot.
>> >> >> >>
>> >> >> >> Thanks,
>> >> >> >> Rohit Sinha
>> >> >> >>
>> >> >> >>
>> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
>> bosco@apache.org>
>> >> >> wrote:
>> >> >> >>
>> >> >> >>> It is optional. It is easy to setup and helps a lot while
>> debugging
>> >> >> >>> during initial setup.
>> >> >> >>>
>> >> >> >>> Bosco
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com>
>> wrote:
>> >> >> >>>
>> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
>> >> helpful
>> >> >> in
>> >> >> >>> >getting some understanding of the integration. I am going
>> through
>> >> the
>> >> >> >>> >SampleApp docs now.
>> >> >> >>> >
>> >> >> >>> >Is auditing an optional feature ? From the documentation it
>> looks
>> >> like
>> >> >> >>> it's
>> >> >> >>> >not and Solr installation is a requirement. I was wondering if I
>> >> can
>> >> >> have
>> >> >> >>> >auditing off and skip Solr installation for initial integration
>> >> >> purpose.
>> >> >> >>> >
>> >> >> >>> >Thanks.
>> >> >> >>> >
>> >> >> >>> >Thanks,
>> >> >> >>> >Rohit Sinha
>> >> >> >>> >
>> >> >> >>> >
>> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
>> >> madhan@apache.org>
>> >> >> >>> wrote:
>> >> >> >>> >
>> >> >> >>> >> Rohit,
>> >> >> >>> >>
>> >> >> >>> >> In addition to the details in the wiki, I would recommend
>> >> reviewing
>> >> >> the
>> >> >> >>> >> following sample application to understand the details of
>> adding
>> >> >> Ranger
>> >> >> >>> >> authorization to an application.
>> >> >> >>> >>
>> >> >> >>> >> - README.txt:
>> >> >> >>> >>
>> >> >> >>>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >> >> >>> >> - Application sources:
>> >> >> >>> >>
>> >> >> >>>
>> >> >>
>> >>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >> >> >>> >> .
>> >> >> >>> >>
>> >> >> >>> >> Madhan
>> >> >> >>> >>
>> >> >> >>> >>
>> >> >> >>> >>
>> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >> >> >>> vperiasamy@hortonworks.com
>> >> >> >>> >> on behalf of vel@apache.org> wrote:
>> >> >> >>> >>
>> >> >> >>> >> >Rohit -
>> >> >> >>> >> >
>> >> >> >>>
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >> >> >>> >> >explains how to add a custom plugin for Ranger.
>> >> >> >>> >> >
>> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
>> >> >> wrote:
>> >> >> >>> >> >
>> >> >> >>> >> >>Hello,
>> >> >> >>> >> >>I am looking into integrating an external service with
>> Apache
>> >> >> Ranger
>> >> >> >>> for
>> >> >> >>> >> >>authorization.
>> >> >> >>> >> >>I looked up the wiki but there is no information about
>> >> integrating
>> >> >> >>> new
>> >> >> >>> >> >>services.
>> >> >> >>> >> >>Can someone give me some info which might be helpful in
>> >> >> identifying
>> >> >> >>> >> >>different components which needs to be developed and other
>> >> >> required
>> >> >> >>> stuff
>> >> >> >>> >> >>
>> >> >> >>> >> >>Thanks.
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >>
>> >> >> >>> >>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>
>> >> >>
>> >>
>> >>
>>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Hello Madhan,
Thanks a lot for the prompt reply.
1. Can you point me to some resource where I can see how to configure
Ranger on Kerberos without KMS.
2. Also, how can I list all the policies for a resource in my service. I
don't see any api exposed by RangerBasePlugin for this. I was looking into
the code base and saw that ServiceStore has some APIs for this. How can I
access this ?
Thanks.
Thanks,
Rohit Sinha
On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> Thanks for sharing your feedback on Ranger integration. Good to know
> that you are able to get Ranger authorization working in your application.
>
> >> 1. Is it possible to have Ranger running on an unencrypted HDFS
> >> with secure Hadoop through Kerberos ?
> Yes. Ranger works with secure Hadoop - either unencrypted or encrypted.
>
> >> 2. Currently, I see the following error log for policy cache file.
> >> Isn’t the policy cache file created automatically ?
> Yes, the policy cache file is created automatically. I guess the error
>
> you see could be cause by non-existent directory where the cache file
> would be saved - in this case /etc/ranger/myservicedev/policycache/.
> Please ensure that this directory exists and has write permission for
> the user that runs the application.
>
> The directory location can be specified via configuration
> ranger.plugin.myservice.policy.cache.dir in ranger-myservice-security.xml.
>
> Hope this helps.
>
> Madhan
>
>
>
>
>
> On 4/7/16, 7:38 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Hello,
> >
> >Thanks a lot for your prompt replies. It was really helpful.
> >Yes, it seems like there was some misconfiguration issue. I was able to
> >make it work and I have a very basic integration up and running with my
> >service now. I have worked on integration/analysis of other authorization
> >models and I must say that Ranger so far has been the easiest to integrate
> >with (at the basic level) and the documentation helped a lot. Appreciate
> >your efforts in building it so far.
> >
> >I have two question which I am trying to figure out. Will appreciate if
> >someone can provide some pointers.
> >
> >1. Kerberos: From the documentation found here
> >
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
> >I understand that Ranger works with KMS on an encrypted HDFS. Is it
> >possible to have Ranger running on an unencrypted HDFS with secure Hadoop
> >through Kerberos ? Something where I can talk to Ranger by giving a
> >principal and keytab ?
> >
> >2. Currently, I see the following error log for policy cache file. Isn't
> >the policy cache file created automatically ? Am I missing some
> >configuration.
> >
> >16/04/07 21:26:18 INFO util.PolicyRefresher:
> >> PolicyRefresher(serviceName=myservice): found updated version.
> >> lastKnownVersion=-1; newVersion=1
> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save policies to
> >> cache file
> '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
> >> java.io.FileNotFoundException:
> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No such
> file
> >> or directory)
> >> at java.io.FileOutputStream.open(Native Method)
> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> >> at java.io.FileWriter.<init>(FileWriter.java:90)
> >> at
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
> >> at
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
> >> at
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
> >> at
> >>
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
> >> .... internal stacktrace....
> >> at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> >
> >Thanks.
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <ma...@apache.org>
> wrote:
> >
> >> Rohit,
> >>
> >> To download policies from Ranger Admin, Ranger plugins require the URL
> to
> >> Ranger Admin and the name of the service containing the policies. These
> >> values are read from following configurations from a file named
> >> ranger-<pluginType>-security.xml
> >>
> >> ranger.plugin.<pluginType>.policy.rest.url
> >> ranger.plugin.<pluginType>.service.name
> >>
> >> For example, these are specified in conf/ranger-sampleapp-security.xml
> for
> >> the sample application.
> >>
> >> Can you please review the configuration for your plugin for the above?
> >>
> >> Hope this helps.
> >>
> >> Madhan
> >>
> >>
> >>
> >> On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
> >>
> >> >Hello Madhan,
> >> >Thanks a lot for your reply.
> >> >
> >> >I am looking into the integration and I working towards developing the
> >> >components outside of ranger trunk for initial development purpose.
> After
> >> >taking an overview of the codebase it seems that it's possible to have
> >> >plugins outside of the ranger trunk too.
> >> >
> >> >I was able to add my service to Ranger Amin through the curl request
> and
> >> >connect to Ranger from my Authorizer but I see the following error:
> >> >
> >> >com.sun.jersey.api.client.ClientHandlerException:
> >> >> java.lang.IllegalArgumentException: URI is not absolute
> >> >> at
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
> >> >> at com.sun.jersey.api.client.Client.handle(Client.java:648)
> >> >> at
> >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
> >> >> at
> >> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> >> >> at
> >> >>
> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
> >> >> at
> >> >>
> >>
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
> >> >> at
> >> >>
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
> >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute
> >> >> at java.net.URI.toURL(URI.java:1095)
> >> >> at
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> >> >> at
> >> >>
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> >> >> ... 8 more
> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not
> exist
> >> or
> >> >> not readble 'null'
> >> >
> >> >
> >> >A quick search on google pointed towards misconfiguration of service
> name
> >> >in security.xml which I have double checked.
> >> >
> >> >Any pointers to debug this will be appreciated.
> >> >
> >> >Thanks.
> >> >
> >> >Thanks,
> >> >Rohit Sinha
> >> >
> >> >
> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
> >> >mneethiraj@hortonworks.com> wrote:
> >> >
> >> >> Rohit,
> >> >>
> >> >> You are right. REPOSITORY_NAME referenced in the doc is the name of
> the
> >> >> service instance in Ranger Admin, which contains the policies for the
> >> >> component (in this case HBase). The plugin reads this value from a
> >> >> configuration named ranger.plugin.hbase.service.name (in file
> >> >> ranger-hbase-security.xml).
> >> >>
> >> >> >> After doing this I don't see anything in the Audit -> Plugins
> >> >> An entry will be created here for every policy download form plugins.
> >> >> Plugins download the policies at the following events:
> >> >> - during the component startup (HBase/HiveServer/...)
> >> >> - when there is a policy change in service instance
> >> >>
> >> >> Hope this helps.
> >> >>
> >> >> Madhan
> >> >>
> >> >>
> >> >>
> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
> >> >>
> >> >> >Thanks for sharing the SampleApp. I was able to run it understand
> the
> >> >> >integration point.
> >> >> >I also tried to enable HBase ranger plugin. When I added the service
> >> from
> >> >> >the Ranger Admin UI I was able to talk to HBase and the resource
> >> >> completion
> >> >> >worked. After that I deleted the HBase plugin from the UI and tried
> to
> >> >> >enable it from the the command line following instructions mentioned
> >> here:
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
> >> >> >
> >> >> >After doing this I don't see anything in the Audit -> Plugins. In
> the
> >> >> >instructions I do see warning
> >> >> >
> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If
> not,
> >> >> the
> >> >> >> hbase-plugin will not be able to communicate with Ranger admin.
> >> >> >
> >> >> >Does this mean I need to add the service from the Ranger Admin Panel
> >> or I
> >> >> >did something wrong in the enabling the plugin ?
> >> >> >
> >> >> >Any help will be highly appreciated.
> >> >> >
> >> >> >Thanks.
> >> >> >
> >> >> >Thanks,
> >> >> >Rohit Sinha
> >> >> >
> >> >> >
> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <
> talktorohit54@gmail.com>
> >> >> >wrote:
> >> >> >
> >> >> >> Awesome. Thanks a lot.
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Rohit Sinha
> >> >> >>
> >> >> >>
> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <
> bosco@apache.org>
> >> >> wrote:
> >> >> >>
> >> >> >>> It is optional. It is easy to setup and helps a lot while
> debugging
> >> >> >>> during initial setup.
> >> >> >>>
> >> >> >>> Bosco
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com>
> wrote:
> >> >> >>>
> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
> >> helpful
> >> >> in
> >> >> >>> >getting some understanding of the integration. I am going
> through
> >> the
> >> >> >>> >SampleApp docs now.
> >> >> >>> >
> >> >> >>> >Is auditing an optional feature ? From the documentation it
> looks
> >> like
> >> >> >>> it's
> >> >> >>> >not and Solr installation is a requirement. I was wondering if I
> >> can
> >> >> have
> >> >> >>> >auditing off and skip Solr installation for initial integration
> >> >> purpose.
> >> >> >>> >
> >> >> >>> >Thanks.
> >> >> >>> >
> >> >> >>> >Thanks,
> >> >> >>> >Rohit Sinha
> >> >> >>> >
> >> >> >>> >
> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
> >> madhan@apache.org>
> >> >> >>> wrote:
> >> >> >>> >
> >> >> >>> >> Rohit,
> >> >> >>> >>
> >> >> >>> >> In addition to the details in the wiki, I would recommend
> >> reviewing
> >> >> the
> >> >> >>> >> following sample application to understand the details of
> adding
> >> >> Ranger
> >> >> >>> >> authorization to an application.
> >> >> >>> >>
> >> >> >>> >> - README.txt:
> >> >> >>> >>
> >> >> >>>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >> >> >>> >> - Application sources:
> >> >> >>> >>
> >> >> >>>
> >> >>
> >>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >> >> >>> >> .
> >> >> >>> >>
> >> >> >>> >> Madhan
> >> >> >>> >>
> >> >> >>> >>
> >> >> >>> >>
> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
> >> >> >>> vperiasamy@hortonworks.com
> >> >> >>> >> on behalf of vel@apache.org> wrote:
> >> >> >>> >>
> >> >> >>> >> >Rohit -
> >> >> >>> >> >
> >> >> >>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >> >> >>> >> >explains how to add a custom plugin for Ranger.
> >> >> >>> >> >
> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
> >> >> wrote:
> >> >> >>> >> >
> >> >> >>> >> >>Hello,
> >> >> >>> >> >>I am looking into integrating an external service with
> Apache
> >> >> Ranger
> >> >> >>> for
> >> >> >>> >> >>authorization.
> >> >> >>> >> >>I looked up the wiki but there is no information about
> >> integrating
> >> >> >>> new
> >> >> >>> >> >>services.
> >> >> >>> >> >>Can someone give me some info which might be helpful in
> >> >> identifying
> >> >> >>> >> >>different components which needs to be developed and other
> >> >> required
> >> >> >>> stuff
> >> >> >>> >> >>
> >> >> >>> >> >>Thanks.
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >>
> >> >> >>> >>
> >> >> >>>
> >> >> >>>
> >> >> >>
> >> >>
> >>
> >>
>
>
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
Thanks for sharing your feedback on Ranger integration. Good to know
that you are able to get Ranger authorization working in your application.
>> 1. Is it possible to have Ranger running on an unencrypted HDFS
>> with secure Hadoop through Kerberos ?
Yes. Ranger works with secure Hadoop - either unencrypted or encrypted.
>> 2. Currently, I see the following error log for policy cache file.
>> Isn’t the policy cache file created automatically ?
Yes, the policy cache file is created automatically. I guess the error
you see could be cause by non-existent directory where the cache file
would be saved - in this case /etc/ranger/myservicedev/policycache/.
Please ensure that this directory exists and has write permission for
the user that runs the application.
The directory location can be specified via configuration
ranger.plugin.myservice.policy.cache.dir in ranger-myservice-security.xml.
Hope this helps.
Madhan
On 4/7/16, 7:38 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello,
>
>Thanks a lot for your prompt replies. It was really helpful.
>Yes, it seems like there was some misconfiguration issue. I was able to
>make it work and I have a very basic integration up and running with my
>service now. I have worked on integration/analysis of other authorization
>models and I must say that Ranger so far has been the easiest to integrate
>with (at the basic level) and the documentation helped a lot. Appreciate
>your efforts in building it so far.
>
>I have two question which I am trying to figure out. Will appreciate if
>someone can provide some pointers.
>
>1. Kerberos: From the documentation found here
>https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
>I understand that Ranger works with KMS on an encrypted HDFS. Is it
>possible to have Ranger running on an unencrypted HDFS with secure Hadoop
>through Kerberos ? Something where I can talk to Ranger by giving a
>principal and keytab ?
>
>2. Currently, I see the following error log for policy cache file. Isn't
>the policy cache file created automatically ? Am I missing some
>configuration.
>
>16/04/07 21:26:18 INFO util.PolicyRefresher:
>> PolicyRefresher(serviceName=myservice): found updated version.
>> lastKnownVersion=-1; newVersion=1
>> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save policies to
>> cache file '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
>> java.io.FileNotFoundException:
>> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No such file
>> or directory)
>> at java.io.FileOutputStream.open(Native Method)
>> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> at java.io.FileWriter.<init>(FileWriter.java:90)
>> at
>> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
>> at
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
>> at
>> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
>> at
>> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
>> .... internal stacktrace....
>> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <ma...@apache.org> wrote:
>
>> Rohit,
>>
>> To download policies from Ranger Admin, Ranger plugins require the URL to
>> Ranger Admin and the name of the service containing the policies. These
>> values are read from following configurations from a file named
>> ranger-<pluginType>-security.xml
>>
>> ranger.plugin.<pluginType>.policy.rest.url
>> ranger.plugin.<pluginType>.service.name
>>
>> For example, these are specified in conf/ranger-sampleapp-security.xml for
>> the sample application.
>>
>> Can you please review the configuration for your plugin for the above?
>>
>> Hope this helps.
>>
>> Madhan
>>
>>
>>
>> On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Hello Madhan,
>> >Thanks a lot for your reply.
>> >
>> >I am looking into the integration and I working towards developing the
>> >components outside of ranger trunk for initial development purpose. After
>> >taking an overview of the codebase it seems that it's possible to have
>> >plugins outside of the ranger trunk too.
>> >
>> >I was able to add my service to Ranger Amin through the curl request and
>> >connect to Ranger from my Authorizer but I see the following error:
>> >
>> >com.sun.jersey.api.client.ClientHandlerException:
>> >> java.lang.IllegalArgumentException: URI is not absolute
>> >> at
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
>> >> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>> >> at
>> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> >> at
>> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> >> at
>> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> >> at
>> >>
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
>> >> at
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
>> >> at
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
>> >> at
>> >>
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
>> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute
>> >> at java.net.URI.toURL(URI.java:1095)
>> >> at
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
>> >> at
>> >>
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
>> >> ... 8 more
>> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not exist
>> or
>> >> not readble 'null'
>> >
>> >
>> >A quick search on google pointed towards misconfiguration of service name
>> >in security.xml which I have double checked.
>> >
>> >Any pointers to debug this will be appreciated.
>> >
>> >Thanks.
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>> >mneethiraj@hortonworks.com> wrote:
>> >
>> >> Rohit,
>> >>
>> >> You are right. REPOSITORY_NAME referenced in the doc is the name of the
>> >> service instance in Ranger Admin, which contains the policies for the
>> >> component (in this case HBase). The plugin reads this value from a
>> >> configuration named ranger.plugin.hbase.service.name (in file
>> >> ranger-hbase-security.xml).
>> >>
>> >> >> After doing this I don't see anything in the Audit -> Plugins
>> >> An entry will be created here for every policy download form plugins.
>> >> Plugins download the policies at the following events:
>> >> - during the component startup (HBase/HiveServer/...)
>> >> - when there is a policy change in service instance
>> >>
>> >> Hope this helps.
>> >>
>> >> Madhan
>> >>
>> >>
>> >>
>> >> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
>> >>
>> >> >Thanks for sharing the SampleApp. I was able to run it understand the
>> >> >integration point.
>> >> >I also tried to enable HBase ranger plugin. When I added the service
>> from
>> >> >the Ranger Admin UI I was able to talk to HBase and the resource
>> >> completion
>> >> >worked. After that I deleted the HBase plugin from the UI and tried to
>> >> >enable it from the the command line following instructions mentioned
>> here:
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >> >
>> >> >After doing this I don't see anything in the Audit -> Plugins. In the
>> >> >instructions I do see warning
>> >> >
>> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not,
>> >> the
>> >> >> hbase-plugin will not be able to communicate with Ranger admin.
>> >> >
>> >> >Does this mean I need to add the service from the Ranger Admin Panel
>> or I
>> >> >did something wrong in the enabling the plugin ?
>> >> >
>> >> >Any help will be highly appreciated.
>> >> >
>> >> >Thanks.
>> >> >
>> >> >Thanks,
>> >> >Rohit Sinha
>> >> >
>> >> >
>> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
>> >> >wrote:
>> >> >
>> >> >> Awesome. Thanks a lot.
>> >> >>
>> >> >> Thanks,
>> >> >> Rohit Sinha
>> >> >>
>> >> >>
>> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org>
>> >> wrote:
>> >> >>
>> >> >>> It is optional. It is easy to setup and helps a lot while debugging
>> >> >>> during initial setup.
>> >> >>>
>> >> >>> Bosco
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>> >> >>>
>> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
>> helpful
>> >> in
>> >> >>> >getting some understanding of the integration. I am going through
>> the
>> >> >>> >SampleApp docs now.
>> >> >>> >
>> >> >>> >Is auditing an optional feature ? From the documentation it looks
>> like
>> >> >>> it's
>> >> >>> >not and Solr installation is a requirement. I was wondering if I
>> can
>> >> have
>> >> >>> >auditing off and skip Solr installation for initial integration
>> >> purpose.
>> >> >>> >
>> >> >>> >Thanks.
>> >> >>> >
>> >> >>> >Thanks,
>> >> >>> >Rohit Sinha
>> >> >>> >
>> >> >>> >
>> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
>> madhan@apache.org>
>> >> >>> wrote:
>> >> >>> >
>> >> >>> >> Rohit,
>> >> >>> >>
>> >> >>> >> In addition to the details in the wiki, I would recommend
>> reviewing
>> >> the
>> >> >>> >> following sample application to understand the details of adding
>> >> Ranger
>> >> >>> >> authorization to an application.
>> >> >>> >>
>> >> >>> >> - README.txt:
>> >> >>> >>
>> >> >>>
>> >>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >> >>> >> - Application sources:
>> >> >>> >>
>> >> >>>
>> >>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >> >>> >> .
>> >> >>> >>
>> >> >>> >> Madhan
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >> >>> vperiasamy@hortonworks.com
>> >> >>> >> on behalf of vel@apache.org> wrote:
>> >> >>> >>
>> >> >>> >> >Rohit -
>> >> >>> >> >
>> >> >>>
>> >>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >> >>> >> >explains how to add a custom plugin for Ranger.
>> >> >>> >> >
>> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
>> >> wrote:
>> >> >>> >> >
>> >> >>> >> >>Hello,
>> >> >>> >> >>I am looking into integrating an external service with Apache
>> >> Ranger
>> >> >>> for
>> >> >>> >> >>authorization.
>> >> >>> >> >>I looked up the wiki but there is no information about
>> integrating
>> >> >>> new
>> >> >>> >> >>services.
>> >> >>> >> >>Can someone give me some info which might be helpful in
>> >> identifying
>> >> >>> >> >>different components which needs to be developed and other
>> >> required
>> >> >>> stuff
>> >> >>> >> >>
>> >> >>> >> >>Thanks.
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >>
>> >> >>> >>
>> >> >>>
>> >> >>>
>> >> >>
>> >>
>>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Hello,
Thanks a lot for your prompt replies. It was really helpful.
Yes, it seems like there was some misconfiguration issue. I was able to
make it work and I have a very basic integration up and running with my
service now. I have worked on integration/analysis of other authorization
models and I must say that Ranger so far has been the easiest to integrate
with (at the basic level) and the documentation helped a lot. Appreciate
your efforts in building it so far.
I have two question which I am trying to figure out. Will appreciate if
someone can provide some pointers.
1. Kerberos: From the documentation found here
https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm
I understand that Ranger works with KMS on an encrypted HDFS. Is it
possible to have Ranger running on an unencrypted HDFS with secure Hadoop
through Kerberos ? Something where I can talk to Ranger by giving a
principal and keytab ?
2. Currently, I see the following error log for policy cache file. Isn't
the policy cache file created automatically ? Am I missing some
configuration.
16/04/07 21:26:18 INFO util.PolicyRefresher:
> PolicyRefresher(serviceName=myservice): found updated version.
> lastKnownVersion=-1; newVersion=1
> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save policies to
> cache file '/etc/ranger/myservicedev/policycache/myservice_myservice.json'
> java.io.FileNotFoundException:
> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No such file
> or directory)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at java.io.FileWriter.<init>(FileWriter.java:90)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105)
> .... internal stacktrace....
> at java.lang.Thread.run(Thread.java:745)
Thanks.
Thanks,
Rohit Sinha
On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> To download policies from Ranger Admin, Ranger plugins require the URL to
> Ranger Admin and the name of the service containing the policies. These
> values are read from following configurations from a file named
> ranger-<pluginType>-security.xml
>
> ranger.plugin.<pluginType>.policy.rest.url
> ranger.plugin.<pluginType>.service.name
>
> For example, these are specified in conf/ranger-sampleapp-security.xml for
> the sample application.
>
> Can you please review the configuration for your plugin for the above?
>
> Hope this helps.
>
> Madhan
>
>
>
> On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Hello Madhan,
> >Thanks a lot for your reply.
> >
> >I am looking into the integration and I working towards developing the
> >components outside of ranger trunk for initial development purpose. After
> >taking an overview of the codebase it seems that it's possible to have
> >plugins outside of the ranger trunk too.
> >
> >I was able to add my service to Ranger Amin through the curl request and
> >connect to Ranger from my Authorizer but I see the following error:
> >
> >com.sun.jersey.api.client.ClientHandlerException:
> >> java.lang.IllegalArgumentException: URI is not absolute
> >> at
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
> >> at com.sun.jersey.api.client.Client.handle(Client.java:648)
> >> at
> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
> >> at
> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> >> at
> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
> >> at
> >>
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
> >> at
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
> >> at
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
> >> at
> >>
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute
> >> at java.net.URI.toURL(URI.java:1095)
> >> at
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> >> at
> >>
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> >> ... 8 more
> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not exist
> or
> >> not readble 'null'
> >
> >
> >A quick search on google pointed towards misconfiguration of service name
> >in security.xml which I have double checked.
> >
> >Any pointers to debug this will be appreciated.
> >
> >Thanks.
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
> >mneethiraj@hortonworks.com> wrote:
> >
> >> Rohit,
> >>
> >> You are right. REPOSITORY_NAME referenced in the doc is the name of the
> >> service instance in Ranger Admin, which contains the policies for the
> >> component (in this case HBase). The plugin reads this value from a
> >> configuration named ranger.plugin.hbase.service.name (in file
> >> ranger-hbase-security.xml).
> >>
> >> >> After doing this I don't see anything in the Audit -> Plugins
> >> An entry will be created here for every policy download form plugins.
> >> Plugins download the policies at the following events:
> >> - during the component startup (HBase/HiveServer/...)
> >> - when there is a policy change in service instance
> >>
> >> Hope this helps.
> >>
> >> Madhan
> >>
> >>
> >>
> >> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
> >>
> >> >Thanks for sharing the SampleApp. I was able to run it understand the
> >> >integration point.
> >> >I also tried to enable HBase ranger plugin. When I added the service
> from
> >> >the Ranger Admin UI I was able to talk to HBase and the resource
> >> completion
> >> >worked. After that I deleted the HBase plugin from the UI and tried to
> >> >enable it from the the command line following instructions mentioned
> here:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
> >> >
> >> >After doing this I don't see anything in the Audit -> Plugins. In the
> >> >instructions I do see warning
> >> >
> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not,
> >> the
> >> >> hbase-plugin will not be able to communicate with Ranger admin.
> >> >
> >> >Does this mean I need to add the service from the Ranger Admin Panel
> or I
> >> >did something wrong in the enabling the plugin ?
> >> >
> >> >Any help will be highly appreciated.
> >> >
> >> >Thanks.
> >> >
> >> >Thanks,
> >> >Rohit Sinha
> >> >
> >> >
> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
> >> >wrote:
> >> >
> >> >> Awesome. Thanks a lot.
> >> >>
> >> >> Thanks,
> >> >> Rohit Sinha
> >> >>
> >> >>
> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org>
> >> wrote:
> >> >>
> >> >>> It is optional. It is easy to setup and helps a lot while debugging
> >> >>> during initial setup.
> >> >>>
> >> >>> Bosco
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
> >> >>>
> >> >>> >Thanks a lot for the prompt replies. Really appreciate it.
> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really
> helpful
> >> in
> >> >>> >getting some understanding of the integration. I am going through
> the
> >> >>> >SampleApp docs now.
> >> >>> >
> >> >>> >Is auditing an optional feature ? From the documentation it looks
> like
> >> >>> it's
> >> >>> >not and Solr installation is a requirement. I was wondering if I
> can
> >> have
> >> >>> >auditing off and skip Solr installation for initial integration
> >> purpose.
> >> >>> >
> >> >>> >Thanks.
> >> >>> >
> >> >>> >Thanks,
> >> >>> >Rohit Sinha
> >> >>> >
> >> >>> >
> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <
> madhan@apache.org>
> >> >>> wrote:
> >> >>> >
> >> >>> >> Rohit,
> >> >>> >>
> >> >>> >> In addition to the details in the wiki, I would recommend
> reviewing
> >> the
> >> >>> >> following sample application to understand the details of adding
> >> Ranger
> >> >>> >> authorization to an application.
> >> >>> >>
> >> >>> >> - README.txt:
> >> >>> >>
> >> >>>
> >>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >> >>> >> - Application sources:
> >> >>> >>
> >> >>>
> >>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >> >>> >> .
> >> >>> >>
> >> >>> >> Madhan
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
> >> >>> vperiasamy@hortonworks.com
> >> >>> >> on behalf of vel@apache.org> wrote:
> >> >>> >>
> >> >>> >> >Rohit -
> >> >>> >> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >> >>> >> >explains how to add a custom plugin for Ranger.
> >> >>> >> >
> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
> >> wrote:
> >> >>> >> >
> >> >>> >> >>Hello,
> >> >>> >> >>I am looking into integrating an external service with Apache
> >> Ranger
> >> >>> for
> >> >>> >> >>authorization.
> >> >>> >> >>I looked up the wiki but there is no information about
> integrating
> >> >>> new
> >> >>> >> >>services.
> >> >>> >> >>Can someone give me some info which might be helpful in
> >> identifying
> >> >>> >> >>different components which needs to be developed and other
> >> required
> >> >>> stuff
> >> >>> >> >>
> >> >>> >> >>Thanks.
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >>
> >> >>> >>
> >> >>>
> >> >>>
> >> >>
> >>
>
>
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
To download policies from Ranger Admin, Ranger plugins require the URL to Ranger Admin and the name of the service containing the policies. These values are read from following configurations from a file named ranger-<pluginType>-security.xml
ranger.plugin.<pluginType>.policy.rest.url
ranger.plugin.<pluginType>.service.name
For example, these are specified in conf/ranger-sampleapp-security.xml for the sample application.
Can you please review the configuration for your plugin for the above?
Hope this helps.
Madhan
On 4/7/16, 5:40 AM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello Madhan,
>Thanks a lot for your reply.
>
>I am looking into the integration and I working towards developing the
>components outside of ranger trunk for initial development purpose. After
>taking an overview of the codebase it seems that it's possible to have
>plugins outside of the ranger trunk too.
>
>I was able to add my service to Ranger Amin through the curl request and
>connect to Ranger from my Authorizer but I see the following error:
>
>com.sun.jersey.api.client.ClientHandlerException:
>> java.lang.IllegalArgumentException: URI is not absolute
>> at
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
>> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>> at
>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
>> at
>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> at
>> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
>> at
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
>> at
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
>> at
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
>> at
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
>> Caused by: java.lang.IllegalArgumentException: URI is not absolute
>> at java.net.URI.toURL(URI.java:1095)
>> at
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
>> at
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
>> ... 8 more
>> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not exist or
>> not readble 'null'
>
>
>A quick search on google pointed towards misconfiguration of service name
>in security.xml which I have double checked.
>
>Any pointers to debug this will be appreciated.
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
>mneethiraj@hortonworks.com> wrote:
>
>> Rohit,
>>
>> You are right. REPOSITORY_NAME referenced in the doc is the name of the
>> service instance in Ranger Admin, which contains the policies for the
>> component (in this case HBase). The plugin reads this value from a
>> configuration named ranger.plugin.hbase.service.name (in file
>> ranger-hbase-security.xml).
>>
>> >> After doing this I don't see anything in the Audit -> Plugins
>> An entry will be created here for every policy download form plugins.
>> Plugins download the policies at the following events:
>> - during the component startup (HBase/HiveServer/...)
>> - when there is a policy change in service instance
>>
>> Hope this helps.
>>
>> Madhan
>>
>>
>>
>> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Thanks for sharing the SampleApp. I was able to run it understand the
>> >integration point.
>> >I also tried to enable HBase ranger plugin. When I added the service from
>> >the Ranger Admin UI I was able to talk to HBase and the resource
>> completion
>> >worked. After that I deleted the HBase plugin from the UI and tried to
>> >enable it from the the command line following instructions mentioned here:
>> >
>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>> >
>> >After doing this I don't see anything in the Audit -> Plugins. In the
>> >instructions I do see warning
>> >
>> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not,
>> the
>> >> hbase-plugin will not be able to communicate with Ranger admin.
>> >
>> >Does this mean I need to add the service from the Ranger Admin Panel or I
>> >did something wrong in the enabling the plugin ?
>> >
>> >Any help will be highly appreciated.
>> >
>> >Thanks.
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
>> >wrote:
>> >
>> >> Awesome. Thanks a lot.
>> >>
>> >> Thanks,
>> >> Rohit Sinha
>> >>
>> >>
>> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>> >>
>> >>> It is optional. It is easy to setup and helps a lot while debugging
>> >>> during initial setup.
>> >>>
>> >>> Bosco
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>> >>>
>> >>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >>> >The "Ranger Stacks - How to add a custom plugin?" was really helpful
>> in
>> >>> >getting some understanding of the integration. I am going through the
>> >>> >SampleApp docs now.
>> >>> >
>> >>> >Is auditing an optional feature ? From the documentation it looks like
>> >>> it's
>> >>> >not and Solr installation is a requirement. I was wondering if I can
>> have
>> >>> >auditing off and skip Solr installation for initial integration
>> purpose.
>> >>> >
>> >>> >Thanks.
>> >>> >
>> >>> >Thanks,
>> >>> >Rohit Sinha
>> >>> >
>> >>> >
>> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org>
>> >>> wrote:
>> >>> >
>> >>> >> Rohit,
>> >>> >>
>> >>> >> In addition to the details in the wiki, I would recommend reviewing
>> the
>> >>> >> following sample application to understand the details of adding
>> Ranger
>> >>> >> authorization to an application.
>> >>> >>
>> >>> >> - README.txt:
>> >>> >>
>> >>>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >>> >> - Application sources:
>> >>> >>
>> >>>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >>> >> .
>> >>> >>
>> >>> >> Madhan
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> >>> vperiasamy@hortonworks.com
>> >>> >> on behalf of vel@apache.org> wrote:
>> >>> >>
>> >>> >> >Rohit -
>> >>> >> >
>> >>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >>> >> >explains how to add a custom plugin for Ranger.
>> >>> >> >
>> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
>> wrote:
>> >>> >> >
>> >>> >> >>Hello,
>> >>> >> >>I am looking into integrating an external service with Apache
>> Ranger
>> >>> for
>> >>> >> >>authorization.
>> >>> >> >>I looked up the wiki but there is no information about integrating
>> >>> new
>> >>> >> >>services.
>> >>> >> >>Can someone give me some info which might be helpful in
>> identifying
>> >>> >> >>different components which needs to be developed and other
>> required
>> >>> stuff
>> >>> >> >>
>> >>> >> >>Thanks.
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Hello Madhan,
Thanks a lot for your reply.
I am looking into the integration and I working towards developing the
components outside of ranger trunk for initial development purpose. After
taking an overview of the codebase it seems that it's possible to have
plugins outside of the ranger trunk too.
I was able to add my service to Ranger Amin through the curl request and
connect to Ranger from my Authorizer but I see the following error:
com.sun.jersey.api.client.ClientHandlerException:
> java.lang.IllegalArgumentException: URI is not absolute
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
> at com.sun.jersey.api.client.Client.handle(Client.java:648)
> at
> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
> at
> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> at
> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
> at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156)
> Caused by: java.lang.IllegalArgumentException: URI is not absolute
> at java.net.URI.toURL(URI.java:1095)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> ... 8 more
> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not exist or
> not readble 'null'
A quick search on google pointed towards misconfiguration of service name
in security.xml which I have double checked.
Any pointers to debug this will be appreciated.
Thanks.
Thanks,
Rohit Sinha
On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj <
mneethiraj@hortonworks.com> wrote:
> Rohit,
>
> You are right. REPOSITORY_NAME referenced in the doc is the name of the
> service instance in Ranger Admin, which contains the policies for the
> component (in this case HBase). The plugin reads this value from a
> configuration named ranger.plugin.hbase.service.name (in file
> ranger-hbase-security.xml).
>
> >> After doing this I don't see anything in the Audit -> Plugins
> An entry will be created here for every policy download form plugins.
> Plugins download the policies at the following events:
> - during the component startup (HBase/HiveServer/...)
> - when there is a policy change in service instance
>
> Hope this helps.
>
> Madhan
>
>
>
> On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Thanks for sharing the SampleApp. I was able to run it understand the
> >integration point.
> >I also tried to enable HBase ranger plugin. When I added the service from
> >the Ranger Admin UI I was able to talk to HBase and the resource
> completion
> >worked. After that I deleted the HBase plugin from the UI and tried to
> >enable it from the the command line following instructions mentioned here:
> >
> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
> >
> >After doing this I don't see anything in the Audit -> Plugins. In the
> >instructions I do see warning
> >
> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not,
> the
> >> hbase-plugin will not be able to communicate with Ranger admin.
> >
> >Does this mean I need to add the service from the Ranger Admin Panel or I
> >did something wrong in the enabling the plugin ?
> >
> >Any help will be highly appreciated.
> >
> >Thanks.
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
> >wrote:
> >
> >> Awesome. Thanks a lot.
> >>
> >> Thanks,
> >> Rohit Sinha
> >>
> >>
> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org>
> wrote:
> >>
> >>> It is optional. It is easy to setup and helps a lot while debugging
> >>> during initial setup.
> >>>
> >>> Bosco
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
> >>>
> >>> >Thanks a lot for the prompt replies. Really appreciate it.
> >>> >The "Ranger Stacks - How to add a custom plugin?" was really helpful
> in
> >>> >getting some understanding of the integration. I am going through the
> >>> >SampleApp docs now.
> >>> >
> >>> >Is auditing an optional feature ? From the documentation it looks like
> >>> it's
> >>> >not and Solr installation is a requirement. I was wondering if I can
> have
> >>> >auditing off and skip Solr installation for initial integration
> purpose.
> >>> >
> >>> >Thanks.
> >>> >
> >>> >Thanks,
> >>> >Rohit Sinha
> >>> >
> >>> >
> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org>
> >>> wrote:
> >>> >
> >>> >> Rohit,
> >>> >>
> >>> >> In addition to the details in the wiki, I would recommend reviewing
> the
> >>> >> following sample application to understand the details of adding
> Ranger
> >>> >> authorization to an application.
> >>> >>
> >>> >> - README.txt:
> >>> >>
> >>>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >>> >> - Application sources:
> >>> >>
> >>>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >>> >> .
> >>> >>
> >>> >> Madhan
> >>> >>
> >>> >>
> >>> >>
> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
> >>> vperiasamy@hortonworks.com
> >>> >> on behalf of vel@apache.org> wrote:
> >>> >>
> >>> >> >Rohit -
> >>> >> >
> >>>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >>> >> >explains how to add a custom plugin for Ranger.
> >>> >> >
> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com>
> wrote:
> >>> >> >
> >>> >> >>Hello,
> >>> >> >>I am looking into integrating an external service with Apache
> Ranger
> >>> for
> >>> >> >>authorization.
> >>> >> >>I looked up the wiki but there is no information about integrating
> >>> new
> >>> >> >>services.
> >>> >> >>Can someone give me some info which might be helpful in
> identifying
> >>> >> >>different components which needs to be developed and other
> required
> >>> stuff
> >>> >> >>
> >>> >> >>Thanks.
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >>
> >>> >>
> >>>
> >>>
> >>
>
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <mn...@hortonworks.com>.
Rohit,
You are right. REPOSITORY_NAME referenced in the doc is the name of the service instance in Ranger Admin, which contains the policies for the component (in this case HBase). The plugin reads this value from a configuration named ranger.plugin.hbase.service.name (in file ranger-hbase-security.xml).
>> After doing this I don't see anything in the Audit -> Plugins
An entry will be created here for every policy download form plugins. Plugins download the policies at the following events:
- during the component startup (HBase/HiveServer/...)
- when there is a policy change in service instance
Hope this helps.
Madhan
On 4/6/16, 7:13 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Thanks for sharing the SampleApp. I was able to run it understand the
>integration point.
>I also tried to enable HBase ranger plugin. When I added the service from
>the Ranger Admin UI I was able to talk to HBase and the resource completion
>worked. After that I deleted the HBase plugin from the UI and tried to
>enable it from the the command line following instructions mentioned here:
>https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
>
>After doing this I don't see anything in the Audit -> Plugins. In the
>instructions I do see warning
>
>> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not, the
>> hbase-plugin will not be able to communicate with Ranger admin.
>
>Does this mean I need to add the service from the Ranger Admin Panel or I
>did something wrong in the enabling the plugin ?
>
>Any help will be highly appreciated.
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
>wrote:
>
>> Awesome. Thanks a lot.
>>
>> Thanks,
>> Rohit Sinha
>>
>>
>> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org> wrote:
>>
>>> It is optional. It is easy to setup and helps a lot while debugging
>>> during initial setup.
>>>
>>> Bosco
>>>
>>>
>>>
>>>
>>>
>>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>>
>>> >Thanks a lot for the prompt replies. Really appreciate it.
>>> >The "Ranger Stacks - How to add a custom plugin?" was really helpful in
>>> >getting some understanding of the integration. I am going through the
>>> >SampleApp docs now.
>>> >
>>> >Is auditing an optional feature ? From the documentation it looks like
>>> it's
>>> >not and Solr installation is a requirement. I was wondering if I can have
>>> >auditing off and skip Solr installation for initial integration purpose.
>>> >
>>> >Thanks.
>>> >
>>> >Thanks,
>>> >Rohit Sinha
>>> >
>>> >
>>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>> >
>>> >> Rohit,
>>> >>
>>> >> In addition to the details in the wiki, I would recommend reviewing the
>>> >> following sample application to understand the details of adding Ranger
>>> >> authorization to an application.
>>> >>
>>> >> - README.txt:
>>> >>
>>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>>> >> - Application sources:
>>> >>
>>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>>> >> .
>>> >>
>>> >> Madhan
>>> >>
>>> >>
>>> >>
>>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>>> vperiasamy@hortonworks.com
>>> >> on behalf of vel@apache.org> wrote:
>>> >>
>>> >> >Rohit -
>>> >> >
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>>> >> >explains how to add a custom plugin for Ranger.
>>> >> >
>>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
>>> >> >
>>> >> >>Hello,
>>> >> >>I am looking into integrating an external service with Apache Ranger
>>> for
>>> >> >>authorization.
>>> >> >>I looked up the wiki but there is no information about integrating
>>> new
>>> >> >>services.
>>> >> >>Can someone give me some info which might be helpful in identifying
>>> >> >>different components which needs to be developed and other required
>>> stuff
>>> >> >>
>>> >> >>Thanks.
>>> >> >
>>> >> >
>>> >> >
>>> >>
>>> >>
>>>
>>>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Thanks for sharing the SampleApp. I was able to run it understand the
integration point.
I also tried to enable HBase ranger plugin. When I added the service from
the Ranger Admin UI I was able to talk to HBase and the resource completion
worked. After that I deleted the HBase plugin from the UI and tried to
enable it from the the command line following instructions mentioned here:
https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1)
After doing this I don't see anything in the Audit -> Plugins. In the
instructions I do see warning
> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not, the
> hbase-plugin will not be able to communicate with Ranger admin.
Does this mean I need to add the service from the Ranger Admin Panel or I
did something wrong in the enabling the plugin ?
Any help will be highly appreciated.
Thanks.
Thanks,
Rohit Sinha
On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <ta...@gmail.com>
wrote:
> Awesome. Thanks a lot.
>
> Thanks,
> Rohit Sinha
>
>
> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org> wrote:
>
>> It is optional. It is easy to setup and helps a lot while debugging
>> during initial setup.
>>
>> Bosco
>>
>>
>>
>>
>>
>> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Thanks a lot for the prompt replies. Really appreciate it.
>> >The "Ranger Stacks - How to add a custom plugin?" was really helpful in
>> >getting some understanding of the integration. I am going through the
>> >SampleApp docs now.
>> >
>> >Is auditing an optional feature ? From the documentation it looks like
>> it's
>> >not and Solr installation is a requirement. I was wondering if I can have
>> >auditing off and skip Solr installation for initial integration purpose.
>> >
>> >Thanks.
>> >
>> >Thanks,
>> >Rohit Sinha
>> >
>> >
>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>> >
>> >> Rohit,
>> >>
>> >> In addition to the details in the wiki, I would recommend reviewing the
>> >> following sample application to understand the details of adding Ranger
>> >> authorization to an application.
>> >>
>> >> - README.txt:
>> >>
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> >> - Application sources:
>> >>
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> >> .
>> >>
>> >> Madhan
>> >>
>> >>
>> >>
>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <
>> vperiasamy@hortonworks.com
>> >> on behalf of vel@apache.org> wrote:
>> >>
>> >> >Rohit -
>> >> >
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >> >explains how to add a custom plugin for Ranger.
>> >> >
>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
>> >> >
>> >> >>Hello,
>> >> >>I am looking into integrating an external service with Apache Ranger
>> for
>> >> >>authorization.
>> >> >>I looked up the wiki but there is no information about integrating
>> new
>> >> >>services.
>> >> >>Can someone give me some info which might be helpful in identifying
>> >> >>different components which needs to be developed and other required
>> stuff
>> >> >>
>> >> >>Thanks.
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>>
>>
>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Awesome. Thanks a lot.
Thanks,
Rohit Sinha
On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <bo...@apache.org> wrote:
> It is optional. It is easy to setup and helps a lot while debugging during
> initial setup.
>
> Bosco
>
>
>
>
>
> On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Thanks a lot for the prompt replies. Really appreciate it.
> >The "Ranger Stacks - How to add a custom plugin?" was really helpful in
> >getting some understanding of the integration. I am going through the
> >SampleApp docs now.
> >
> >Is auditing an optional feature ? From the documentation it looks like
> it's
> >not and Solr installation is a requirement. I was wondering if I can have
> >auditing off and skip Solr installation for initial integration purpose.
> >
> >Thanks.
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org>
> wrote:
> >
> >> Rohit,
> >>
> >> In addition to the details in the wiki, I would recommend reviewing the
> >> following sample application to understand the details of adding Ranger
> >> authorization to an application.
> >>
> >> - README.txt:
> >>
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> >> - Application sources:
> >>
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> >> .
> >>
> >> Madhan
> >>
> >>
> >>
> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <vperiasamy@hortonworks.com
> >> on behalf of vel@apache.org> wrote:
> >>
> >> >Rohit -
> >> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >> >explains how to add a custom plugin for Ranger.
> >> >
> >> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
> >> >
> >> >>Hello,
> >> >>I am looking into integrating an external service with Apache Ranger
> for
> >> >>authorization.
> >> >>I looked up the wiki but there is no information about integrating new
> >> >>services.
> >> >>Can someone give me some info which might be helpful in identifying
> >> >>different components which needs to be developed and other required
> stuff
> >> >>
> >> >>Thanks.
> >> >
> >> >
> >> >
> >>
> >>
>
>
Re: Integrating new services with Ranger
Posted by Don Bosco Durai <bo...@apache.org>.
It is optional. It is easy to setup and helps a lot while debugging during initial setup.
Bosco
On 4/6/16, 12:23 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Thanks a lot for the prompt replies. Really appreciate it.
>The "Ranger Stacks - How to add a custom plugin?" was really helpful in
>getting some understanding of the integration. I am going through the
>SampleApp docs now.
>
>Is auditing an optional feature ? From the documentation it looks like it's
>not and Solr installation is a requirement. I was wondering if I can have
>auditing off and skip Solr installation for initial integration purpose.
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org> wrote:
>
>> Rohit,
>>
>> In addition to the details in the wiki, I would recommend reviewing the
>> following sample application to understand the details of adding Ranger
>> authorization to an application.
>>
>> - README.txt:
>> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
>> - Application sources:
>> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
>> .
>>
>> Madhan
>>
>>
>>
>> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <vperiasamy@hortonworks.com
>> on behalf of vel@apache.org> wrote:
>>
>> >Rohit -
>> >https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>> >explains how to add a custom plugin for Ranger.
>> >
>> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
>> >
>> >>Hello,
>> >>I am looking into integrating an external service with Apache Ranger for
>> >>authorization.
>> >>I looked up the wiki but there is no information about integrating new
>> >>services.
>> >>Can someone give me some info which might be helpful in identifying
>> >>different components which needs to be developed and other required stuff
>> >>
>> >>Thanks.
>> >
>> >
>> >
>>
>>
Re: Integrating new services with Ranger
Posted by rohit sinha <ta...@gmail.com>.
Thanks a lot for the prompt replies. Really appreciate it.
The "Ranger Stacks - How to add a custom plugin?" was really helpful in
getting some understanding of the integration. I am going through the
SampleApp docs now.
Is auditing an optional feature ? From the documentation it looks like it's
not and Solr installation is a requirement. I was wondering if I can have
auditing off and skip Solr installation for initial integration purpose.
Thanks.
Thanks,
Rohit Sinha
On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> In addition to the details in the wiki, I would recommend reviewing the
> following sample application to understand the details of adding Ranger
> authorization to an application.
>
> - README.txt:
> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
> - Application sources:
> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp
> .
>
> Madhan
>
>
>
> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <vperiasamy@hortonworks.com
> on behalf of vel@apache.org> wrote:
>
> >Rohit -
> >https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
> >explains how to add a custom plugin for Ranger.
> >
> >On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
> >
> >>Hello,
> >>I am looking into integrating an external service with Apache Ranger for
> >>authorization.
> >>I looked up the wiki but there is no information about integrating new
> >>services.
> >>Can someone give me some info which might be helpful in identifying
> >>different components which needs to be developed and other required stuff
> >>
> >>Thanks.
> >
> >
> >
>
>
Re: Integrating new services with Ranger
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
In addition to the details in the wiki, I would recommend reviewing the following sample application to understand the details of adding Ranger authorization to an application.
- README.txt: https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt
- Application sources: https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp.
Madhan
On 4/6/16, 11:22 AM, "Velmurugan Periasamy" <vperiasamy@hortonworks.com on behalf of vel@apache.org> wrote:
>Rohit -
>https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
>explains how to add a custom plugin for Ranger.
>
>On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
>
>>Hello,
>>I am looking into integrating an external service with Apache Ranger for
>>authorization.
>>I looked up the wiki but there is no information about integrating new
>>services.
>>Can someone give me some info which might be helpful in identifying
>>different components which needs to be developed and other required stuff
>>
>>Thanks.
>
>
>
Re: Integrating new services with Ranger
Posted by Velmurugan Periasamy <ve...@apache.org>.
Rohit -
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207
explains how to add a custom plugin for Ranger.
On 4/6/16, 10:47 AM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello,
>I am looking into integrating an external service with Apache Ranger for
>authorization.
>I looked up the wiki but there is no information about integrating new
>services.
>Can someone give me some info which might be helpful in identifying
>different components which needs to be developed and other required stuff
>
>Thanks.