You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Aneela Saleem <an...@platalytics.com> on 2015/12/05 16:43:07 UTC

Hbase policy issue

Hi!

I have made a policy on hbase table *emp *with column family *f1 *and
column *name.*
I gave mike user the permission of read and  roger have all permissions.

But when i scan table using user mike or roger even. It does not allow
*ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
Insufficient permissions for user ‘roger',action: scannerOpen,
tableName:emp, family:f1*

No audits are being shown and hbase logs have following exception

2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
continue to use last known version of policies (-1)
com.sun.jersey.api.client.ClientHandlerException:
org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
field "isEnabled" (Class
org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
as ignorable
 at [Source:
sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line:
1, column: 938] (through reference chain:
org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
        at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
        at
org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
        at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
        at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
        at
org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
Unrecognized field "isEnabled" (Class
org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
as ignorable
 at [Source:
sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line:
1, column: 938] (through reference chain:
org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at
org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
        at
org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
        at
org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
        at
org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
        at
org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at
org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at
org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at
org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at
org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at
org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at
org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at
org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at
org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at
org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at
org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at
org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at
org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
        at
org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
        at
org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
        at
com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
        ... 5 more
2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file
does not exist or not readble
'/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'



Thanks

Re: Hbase policy issue

Posted by Aneela Saleem <an...@platalytics.com>.
I have created JIRA, you can find here
<https://issues.apache.org/jira/browse/RANGER-769>

On Mon, Dec 7, 2015 at 3:35 AM, Aneela Saleem <an...@platalytics.com>
wrote:

> Sure Bosco!
>
> On Mon, Dec 7, 2015 at 3:05 AM, Don Bosco Durai <bo...@apache.org> wrote:
>
>> Yes, thanks for verifying it. We should have http-core in the plugin
>> folder.
>>
>> Would you be able to do the change and upload the patch for review? One
>> of the commuters can commit the code behalf of you.
>>
>> We might have to update src/main/assembly/hbase-agent.xml. Can you also
>> create a JIRA for this?
>>
>> Thanks
>>
>> Bosco
>>
>>
>> From: Aneela Saleem <an...@platalytics.com>
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Sunday, December 6, 2015 at 12:13 PM
>> To: <us...@ranger.incubator.apache.org>
>>
>> Subject: Re: Hbase policy issue
>>
>> Bosco!
>>
>> I have Reverted changes, place httpcore-4.1.3.jar in hbase/lib again.
>> Place httpcore-4.2.5.jar in ranger-hbase-plugin-impl folder. And now it's
>> also working.
>> Bosco! there is no http-core by default in hbase-plugin and http-client
>> in hdfs plugin. I think it should be there by default.
>>
>> Thanks
>>
>> On Sun, Dec 6, 2015 at 11:59 PM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>>
>>> Aneela, thanks
>>>
>>> Madhan, we should have both the jars in our plugin impl folder and make
>>> sure it works.
>>>
>>> Aneela, if you don't mind, can you do one experiment? Can you revert
>>> your change? Have the old httpcore in the hbase lib folder, but add the
>>> 4.2.5 in the ranger-impl folder (check under lib folder) and try it out?
>>>
>>> If it works, then we don't have to manually update the hbase libraries.
>>>
>>> Thanks
>>>
>>> Bosco
>>>
>>>
>>> _____________________________
>>> From: Aneela Saleem <an...@platalytics.com>
>>> Sent: Sunday, December 6, 2015 5:09 AM
>>> Subject: Re: Hbase policy issue
>>> To: <us...@ranger.incubator.apache.org>
>>>
>>>
>>>
>>> The issue was due to incompatible http-core version. Hbase by default
>>> has old version oh http-core. Replace old one with
>>> httpcore-4.4.2.jar that is compatible with httpclient-4.2.5.jar.
>>>
>>> On Sat, Dec 5, 2015 at 10:43 PM, Aneela Saleem <an...@platalytics.com>
>>> wrote:
>>>
>>>> Hi Madhan!
>>>>
>>>> In my Hadoop directory, The command *find . -name
>>>> "ranger-plugins-common*.jar" | xargs ls –l * produces following output
>>>> ./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar
>>>>
>>>>
>>>>
>>>> And above authorization issue is resolved now.
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Aneela,
>>>>>
>>>>> Was the earlier issue with HBase authorization resolved now? This
>>>>> exception should not impact the authorization.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>> From: Aneela Saleem <an...@platalytics.com>
>>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Date: Saturday, December 5, 2015 at 9:28 AM
>>>>> To: " user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Subject: Re: Hbase policy issue
>>>>>
>>>>> Hi Madhan!
>>>>>
>>>>> I reinstallled hbase plugin and checked the logs. Now the exception
>>>>> changed to following
>>>>>
>>>>> 2015-12-05 22:25:42,251 FATAL
>>>>> [org.apache.ranger.audit.queue.AuditBatchQueue0]
>>>>> destination.SolrAuditDestination: Can't connect to Solr server. URL=[
>>>>> http://192.168.23.126:6083/solr/ranger_audits]
>>>>> java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
>>>>>         at
>>>>> org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
>>>>>
>>>>>         at
>>>>> org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
>>>>>
>>>>>         at
>>>>> org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
>>>>>
>>>>>         at
>>>>> org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
>>>>>
>>>>>         at
>>>>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
>>>>>
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:356)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
>>>>>         at java.lang.Thread.run(Thread.java:745)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <
>>>>> hafizmujadid00@gmail.com> wrote:
>>>>>
>>>>>> hi Madhan!
>>>>>>
>>>>>> I have compiled ranger master branch code. Both the hbase and
>>>>>> hbase-admin have same versions. I have hbase version 1.1.2.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Aneela,
>>>>>>>
>>>>>>> It looks like the Ranger HBase plugin used in this environment is
>>>>>>> from an earlier version than the Ranger Admin. Can you please send the
>>>>>>> version details to troubleshoot this further?
>>>>>>>
>>>>>>> While we look at fixing the issue in the code, you should be able to
>>>>>>> continue by using Ranger HBase plugin from the same version as Ranger
>>>>>>> Admin.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Madhan
>>>>>>>
>>>>>>> From: Aneela Saleem < aneela@platalytics.com>
>>>>>>> Reply-To: " user@ranger.incubator.apache.org" <
>>>>>>> user@ranger.incubator.apache.org>
>>>>>>> Date: Saturday, December 5, 2015 at 7:43 AM
>>>>>>> To: " user@ranger.incubator.apache.org" <
>>>>>>> user@ranger.incubator.apache.org>
>>>>>>> Subject: Hbase policy issue
>>>>>>>
>>>>>>> Hi!
>>>>>>>
>>>>>>> I have made a policy on hbase table *emp *with column family *f1 *
>>>>>>> and column *name.*
>>>>>>> I gave mike user the permission of read and  roger have all
>>>>>>> permissions.
>>>>>>>
>>>>>>> But when i scan table using user mike or roger even. It does not
>>>>>>> allow
>>>>>>> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>>>>>> Insufficient permissions for user ‘roger',action: scannerOpen,
>>>>>>> tableName:emp, family:f1*
>>>>>>>
>>>>>>> No audits are being shown and hbase logs have following exception
>>>>>>>
>>>>>>> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
>>>>>>> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
>>>>>>> continue to use last known version of policies (-1)
>>>>>>> com.sun.jersey.api.client.ClientHandlerException:
>>>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>>>>>> field "isEnabled" (Class
>>>>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>>>>> as ignorable
>>>>>>>  at [Source:
>>>>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>>>>> line: 1, column: 938] (through reference chain:
>>>>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>>>>
>>>>>>>         at
>>>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>>>>>>>         at
>>>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>>>>>>>         at
>>>>>>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
>>>>>>> Caused by:
>>>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>>>>>> field "isEnabled" (Class
>>>>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>>>>> as ignorable
>>>>>>>  at [Source:
>>>>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>>>>> line: 1, column: 938] (through reference chain:
>>>>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>>>
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>>>>>>>         at
>>>>>>> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>>>>>>>         at
>>>>>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>>>>>>>
>>>>>>>         at
>>>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>>>>>>>         ... 5 more
>>>>>>> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher:
>>>>>>> cache file does not exist or not readble
>>>>>>> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards: HAFIZ MUJADID
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>

Re: Hbase policy issue

Posted by Aneela Saleem <an...@platalytics.com>.
Sure Bosco!

On Mon, Dec 7, 2015 at 3:05 AM, Don Bosco Durai <bo...@apache.org> wrote:

> Yes, thanks for verifying it. We should have http-core in the plugin
> folder.
>
> Would you be able to do the change and upload the patch for review? One of
> the commuters can commit the code behalf of you.
>
> We might have to update src/main/assembly/hbase-agent.xml. Can you also
> create a JIRA for this?
>
> Thanks
>
> Bosco
>
>
> From: Aneela Saleem <an...@platalytics.com>
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Sunday, December 6, 2015 at 12:13 PM
> To: <us...@ranger.incubator.apache.org>
>
> Subject: Re: Hbase policy issue
>
> Bosco!
>
> I have Reverted changes, place httpcore-4.1.3.jar in hbase/lib again.
> Place httpcore-4.2.5.jar in ranger-hbase-plugin-impl folder. And now it's
> also working.
> Bosco! there is no http-core by default in hbase-plugin and http-client in
> hdfs plugin. I think it should be there by default.
>
> Thanks
>
> On Sun, Dec 6, 2015 at 11:59 PM, Don Bosco Durai <bo...@apache.org> wrote:
>
>> Aneela, thanks
>>
>> Madhan, we should have both the jars in our plugin impl folder and make
>> sure it works.
>>
>> Aneela, if you don't mind, can you do one experiment? Can you revert your
>> change? Have the old httpcore in the hbase lib folder, but add the 4.2.5 in
>> the ranger-impl folder (check under lib folder) and try it out?
>>
>> If it works, then we don't have to manually update the hbase libraries.
>>
>> Thanks
>>
>> Bosco
>>
>>
>> _____________________________
>> From: Aneela Saleem <an...@platalytics.com>
>> Sent: Sunday, December 6, 2015 5:09 AM
>> Subject: Re: Hbase policy issue
>> To: <us...@ranger.incubator.apache.org>
>>
>>
>>
>> The issue was due to incompatible http-core version. Hbase by default has
>> old version oh http-core. Replace old one with
>> httpcore-4.4.2.jar that is compatible with httpclient-4.2.5.jar.
>>
>> On Sat, Dec 5, 2015 at 10:43 PM, Aneela Saleem <an...@platalytics.com>
>> wrote:
>>
>>> Hi Madhan!
>>>
>>> In my Hadoop directory, The command *find . -name
>>> "ranger-plugins-common*.jar" | xargs ls –l * produces following output
>>> ./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar
>>>
>>>
>>>
>>> And above authorization issue is resolved now.
>>>
>>> Thanks
>>>
>>>
>>>
>>> On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Aneela,
>>>>
>>>> Was the earlier issue with HBase authorization resolved now? This
>>>> exception should not impact the authorization.
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>> From: Aneela Saleem <an...@platalytics.com>
>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Date: Saturday, December 5, 2015 at 9:28 AM
>>>> To: " user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Re: Hbase policy issue
>>>>
>>>> Hi Madhan!
>>>>
>>>> I reinstallled hbase plugin and checked the logs. Now the exception
>>>> changed to following
>>>>
>>>> 2015-12-05 22:25:42,251 FATAL
>>>> [org.apache.ranger.audit.queue.AuditBatchQueue0]
>>>> destination.SolrAuditDestination: Can't connect to Solr server. URL=[
>>>> http://192.168.23.126:6083/solr/ranger_audits]
>>>> java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
>>>>         at
>>>> org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
>>>>
>>>>         at
>>>> org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
>>>>
>>>>         at
>>>> org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
>>>>
>>>>         at
>>>> org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
>>>>
>>>>         at
>>>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
>>>>
>>>>         at
>>>> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
>>>>
>>>>         at
>>>> org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
>>>>
>>>>         at
>>>> org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
>>>>
>>>>         at
>>>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
>>>>
>>>>         at
>>>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
>>>>
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:356)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>>>>
>>>>         at
>>>> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
>>>>         at java.lang.Thread.run(Thread.java:745)
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <hafizmujadid00@gmail.com
>>>> > wrote:
>>>>
>>>>> hi Madhan!
>>>>>
>>>>> I have compiled ranger master branch code. Both the hbase and
>>>>> hbase-admin have same versions. I have hbase version 1.1.2.
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Aneela,
>>>>>>
>>>>>> It looks like the Ranger HBase plugin used in this environment is
>>>>>> from an earlier version than the Ranger Admin. Can you please send the
>>>>>> version details to troubleshoot this further?
>>>>>>
>>>>>> While we look at fixing the issue in the code, you should be able to
>>>>>> continue by using Ranger HBase plugin from the same version as Ranger
>>>>>> Admin.
>>>>>>
>>>>>> Thanks,
>>>>>> Madhan
>>>>>>
>>>>>> From: Aneela Saleem < aneela@platalytics.com>
>>>>>> Reply-To: " user@ranger.incubator.apache.org" <
>>>>>> user@ranger.incubator.apache.org>
>>>>>> Date: Saturday, December 5, 2015 at 7:43 AM
>>>>>> To: " user@ranger.incubator.apache.org" <
>>>>>> user@ranger.incubator.apache.org>
>>>>>> Subject: Hbase policy issue
>>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> I have made a policy on hbase table *emp *with column family *f1 *
>>>>>> and column *name.*
>>>>>> I gave mike user the permission of read and  roger have all
>>>>>> permissions.
>>>>>>
>>>>>> But when i scan table using user mike or roger even. It does not
>>>>>> allow
>>>>>> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>>>>> Insufficient permissions for user ‘roger',action: scannerOpen,
>>>>>> tableName:emp, family:f1*
>>>>>>
>>>>>> No audits are being shown and hbase logs have following exception
>>>>>>
>>>>>> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
>>>>>> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
>>>>>> continue to use last known version of policies (-1)
>>>>>> com.sun.jersey.api.client.ClientHandlerException:
>>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>>>>> field "isEnabled" (Class
>>>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>>>> as ignorable
>>>>>>  at [Source:
>>>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>>>> line: 1, column: 938] (through reference chain:
>>>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>>>
>>>>>>         at
>>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>>>>>>         at
>>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>>>>>>         at
>>>>>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>>>>>>
>>>>>>         at
>>>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>>>>>>
>>>>>>         at
>>>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>>>>>>
>>>>>>         at
>>>>>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
>>>>>> Caused by:
>>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>>>>> field "isEnabled" (Class
>>>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>>>> as ignorable
>>>>>>  at [Source:
>>>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>>>> line: 1, column: 938] (through reference chain:
>>>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>>
>>>>>>         at
>>>>>> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>>>>>>         at
>>>>>> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>>>>>>         at
>>>>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>>>>>>
>>>>>>         at
>>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>>>>>>         ... 5 more
>>>>>> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache
>>>>>> file does not exist or not readble
>>>>>> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards: HAFIZ MUJADID
>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>

Re: Hbase policy issue

Posted by Don Bosco Durai <bo...@apache.org>.
Yes, thanks for verifying it. We should have http-core in the plugin folder.

Would you be able to do the change and upload the patch for review? One of the commuters can commit the code behalf of you.

We might have to update src/main/assembly/hbase-agent.xml. Can you also create a JIRA for this?

Thanks

Bosco


From:  Aneela Saleem <an...@platalytics.com>
Reply-To:  <us...@ranger.incubator.apache.org>
Date:  Sunday, December 6, 2015 at 12:13 PM
To:  <us...@ranger.incubator.apache.org>
Subject:  Re: Hbase policy issue

Bosco!

I have Reverted changes, place httpcore-4.1.3.jar in hbase/lib again. Place httpcore-4.2.5.jar in ranger-hbase-plugin-impl folder. And now it's also working.
Bosco! there is no http-core by default in hbase-plugin and http-client in hdfs plugin. I think it should be there by default.

Thanks

On Sun, Dec 6, 2015 at 11:59 PM, Don Bosco Durai <bo...@apache.org> wrote:
 
Aneela, thanks

Madhan, we should have both the jars in our plugin impl folder and make sure it works.

Aneela, if you don't mind, can you do one experiment? Can you revert your change? Have the old httpcore in the hbase lib folder, but add the 4.2.5 in the ranger-impl folder (check under lib folder) and try it out?

If it works, then we don't have to manually update the hbase libraries. 

Thanks

Bosco


 
_____________________________
From: Aneela Saleem <an...@platalytics.com>
Sent: Sunday, December 6, 2015 5:09 AM
Subject: Re: Hbase policy issue
To:  <us...@ranger.incubator.apache.org>



    
   The issue was due to incompatible http-core version. Hbase by default has old version oh http-core. Replace old one with    
httpcore-4.4.2.jar that is compatible with httpclient-4.2.5.jar.  
  
   
   
    On Sat, Dec 5, 2015 at 10:43 PM, Aneela Saleem     <an...@platalytics.com> wrote:    
    
     
      Hi Madhan!      
       
      
      
       In my Hadoop directory, The command        find . -name "ranger-plugins-common*.jar" | xargs ls –l  produces following output      
      
       ./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar       
      
      
       
      
      
       
      
      
       And above authorization issue is resolved now.      
      
       
      
      
       Thanks      
      
       
      
      
       
      
     
     
      
       
        
        
         On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj          <ma...@apache.org> wrote:         
         
          
           
            
             
              Aneela,             
             
              
             
             
              Was the earlier issue with HBase authorization resolved now? This exception should not impact the authorization.             
             
              
             
             
              Thanks,             
             
              Madhan             
             
              
               
              
             
            
           
           
            
           
                       
             From:  Aneela Saleem <an...@platalytics.com>
Reply-To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
             Date:  Saturday, December 5, 2015 at 9:28 AM             
             To:  "             user@ranger.incubator.apache.org" <             user@ranger.incubator.apache.org>             
             Subject:  Re: Hbase policy issue             
            
            
             
              
               
              
              
               
                
                 Hi Madhan!                  
                  
                 
                 
                  I reinstallled hbase plugin and checked the logs. Now the exception changed to following                 
                 
                  
                 
                 
                  
                   2015-12-05 22:25:42,251 FATAL [org.apache.ranger.audit.queue.AuditBatchQueue0] destination.SolrAuditDestination: Can't connect to Solr server. URL=[                   http://192.168.23.126:6083/solr/ranger_audits]                  
                  
                   java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET                  
                  
                           at org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)                  
                  
                           at org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)                  
                  
                           at org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)                  
                  
                           at org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)                  
                  
                           at org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)                  
                  
                           at org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)                  
                  
                           at org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)                  
                  
                           at org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)                  
                  
                           at org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)                  
                  
                           at org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)                  
                  
                           at java.security.AccessController.doPrivileged(Native Method)                  
                  
                           at javax.security.auth.Subject.doAs(Subject.java:356)                  
                  
                           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)                  
                  
                           at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)                  
                  
                           at java.lang.Thread.run(Thread.java:745)                  
                 
                 
                  
                 
                 
                  
                 
                 
                  
                 
                
                
                 
                 
                  On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid                    <ha...@gmail.com> wrote:                  
                  
                   
                    hi Madhan!                     
                     
                    
                    
                     I have compiled ranger master branch code. Both the hbase and hbase-admin have same versions. I have hbase version 1.1.2.                    
                    
                     
                    
                    
                     Thanks                    
                   
                   
                    
                     
                      
                      
                       On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj                         <ma...@apache.org> wrote:                       
                       
                        
                         
                          
                           Aneela,                          
                          
                           
                          
                          
                           It looks like the Ranger HBase plugin used in this environment is from an earlier version than the Ranger Admin. Can you please send the version details to troubleshoot this further?                          
                          
                           
                          
                          
                           While we look at fixing the issue in the code, you should be able to continue by using Ranger HBase plugin from the same version as Ranger Admin.                          
                          
                           
                          
                          
                           Thanks,                          
                          
                           Madhan                          
                          
                           
                            
                           
                          
                         
                         
                          
                         
                                                   
                           From: Aneela Saleem <                           aneela@platalytics.com>                           
                           Reply-To: "                           user@ranger.incubator.apache.org" <                           user@ranger.incubator.apache.org>                           
                           Date: Saturday, December 5, 2015 at 7:43 AM                           
                           To: "                           user@ranger.incubator.apache.org" <                           user@ranger.incubator.apache.org>                           
                           Subject: Hbase policy issue                           
                          
                          
                           
                            
                             
                            
                            
                             
                              
                               Hi!                                
                                
                               
                               
                                I have made a policy on hbase table                                 emp with column family                                 f1  and column                                 name.                               
                               
                                I gave mike user the permission of read and  roger have all permissions.                               
                               
                                
                               
                               
                                But when i scan table using user mike or roger even. It does not allow                               
                               
                                ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘roger',action: scannerOpen, tableName:emp, family:f1                                
                               
                               
                                
                               
                               
                                No audits are being shown and hbase logs have following exception                               
                               
                                
                               
                               
                                
                                 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher: PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will continue to use last known version of policies (-1)                                
                                
                                 com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable                                
                                
                                  at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])                                
                                
                                         at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)                                
                                
                                         at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)                                
                                
                                         at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)                                
                                
                                         at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)                                
                                
                                         at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)                                
                                
                                         at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)                                
                                
                                 Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable                                
                                
                                  at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])                                
                                
                                         at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)                                
                                
                                         at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)                                
                                
                                         at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)                                
                                
                                         at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)                                
                                
                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)                                
                                
                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)                                
                                
                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)                                
                                
                                         at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)                                
                                
                                         at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)                                
                                
                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)                                
                                
                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)                                
                                
                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)                                
                                
                                         at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)                                
                                
                                         at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)                                
                                
                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)                                
                                
                                         at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)                                
                                
                                         at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)                                
                                
                                         at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)                                
                                
                                         at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)                                
                                
                                         ... 5 more                                
                                
                                 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file does not exist or not readble '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'                                
                               
                               
                                
                               
                               
                                
                               
                               
                                
                               
                               
                                Thanks                               
                              
                             
                            
                           
                          
                        
                       
                      
                      
                      
                      
                       
                      
                     
                    
                    -- 
                      
                       Regards:                         HAFIZ MUJADID                       
                      
                   
                  
                 
                 
                
               
              
             
            
          
          
        
        
       
       
     
    
   
   
  
  

 



Re: Hbase policy issue

Posted by Aneela Saleem <an...@platalytics.com>.
Bosco!

I have Reverted changes, place httpcore-4.1.3.jar in hbase/lib again.
Place httpcore-4.2.5.jar in ranger-hbase-plugin-impl folder. And now it's
also working.
Bosco! there is no http-core by default in hbase-plugin and http-client in
hdfs plugin. I think it should be there by default.

Thanks

On Sun, Dec 6, 2015 at 11:59 PM, Don Bosco Durai <bo...@apache.org> wrote:

> Aneela, thanks
>
> Madhan, we should have both the jars in our plugin impl folder and make
> sure it works.
>
> Aneela, if you don't mind, can you do one experiment? Can you revert your
> change? Have the old httpcore in the hbase lib folder, but add the 4.2.5 in
> the ranger-impl folder (check under lib folder) and try it out?
>
> If it works, then we don't have to manually update the hbase libraries.
>
> Thanks
>
> Bosco
>
>
> _____________________________
> From: Aneela Saleem <an...@platalytics.com>
> Sent: Sunday, December 6, 2015 5:09 AM
> Subject: Re: Hbase policy issue
> To: <us...@ranger.incubator.apache.org>
>
>
>
> The issue was due to incompatible http-core version. Hbase by default has
> old version oh http-core. Replace old one with
> httpcore-4.4.2.jar that is compatible with httpclient-4.2.5.jar.
>
> On Sat, Dec 5, 2015 at 10:43 PM, Aneela Saleem <an...@platalytics.com>
> wrote:
>
>> Hi Madhan!
>>
>> In my Hadoop directory, The command *find . -name
>> "ranger-plugins-common*.jar" | xargs ls –l * produces following output
>> ./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar
>>
>>
>>
>> And above authorization issue is resolved now.
>>
>> Thanks
>>
>>
>>
>> On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Aneela,
>>>
>>> Was the earlier issue with HBase authorization resolved now? This
>>> exception should not impact the authorization.
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Aneela Saleem <an...@platalytics.com>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Saturday, December 5, 2015 at 9:28 AM
>>> To: " user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Subject: Re: Hbase policy issue
>>>
>>> Hi Madhan!
>>>
>>> I reinstallled hbase plugin and checked the logs. Now the exception
>>> changed to following
>>>
>>> 2015-12-05 22:25:42,251 FATAL
>>> [org.apache.ranger.audit.queue.AuditBatchQueue0]
>>> destination.SolrAuditDestination: Can't connect to Solr server. URL=[
>>> http://192.168.23.126:6083/solr/ranger_audits]
>>> java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
>>>         at
>>> org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
>>>
>>>         at
>>> org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
>>>
>>>         at
>>> org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
>>>
>>>         at
>>> org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
>>>
>>>         at
>>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
>>>
>>>         at
>>> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
>>>
>>>         at
>>> org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
>>>
>>>         at
>>> org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
>>>
>>>         at
>>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
>>>
>>>         at
>>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
>>>
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:356)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>>>
>>>         at
>>> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
>>>         at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>>
>>>
>>> On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <ha...@gmail.com>
>>> wrote:
>>>
>>>> hi Madhan!
>>>>
>>>> I have compiled ranger master branch code. Both the hbase and
>>>> hbase-admin have same versions. I have hbase version 1.1.2.
>>>>
>>>> Thanks
>>>>
>>>> On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Aneela,
>>>>>
>>>>> It looks like the Ranger HBase plugin used in this environment is from
>>>>> an earlier version than the Ranger Admin. Can you please send the version
>>>>> details to troubleshoot this further?
>>>>>
>>>>> While we look at fixing the issue in the code, you should be able to
>>>>> continue by using Ranger HBase plugin from the same version as Ranger
>>>>> Admin.
>>>>>
>>>>> Thanks,
>>>>> Madhan
>>>>>
>>>>> From: Aneela Saleem < aneela@platalytics.com>
>>>>> Reply-To: " user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Date: Saturday, December 5, 2015 at 7:43 AM
>>>>> To: " user@ranger.incubator.apache.org" <
>>>>> user@ranger.incubator.apache.org>
>>>>> Subject: Hbase policy issue
>>>>>
>>>>> Hi!
>>>>>
>>>>> I have made a policy on hbase table *emp *with column family *f1 *
>>>>> and column *name.*
>>>>> I gave mike user the permission of read and  roger have all
>>>>> permissions.
>>>>>
>>>>> But when i scan table using user mike or roger even. It does not allow
>>>>> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>>>> Insufficient permissions for user ‘roger',action: scannerOpen,
>>>>> tableName:emp, family:f1*
>>>>>
>>>>> No audits are being shown and hbase logs have following exception
>>>>>
>>>>> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
>>>>> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
>>>>> continue to use last known version of policies (-1)
>>>>> com.sun.jersey.api.client.ClientHandlerException:
>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>>>> field "isEnabled" (Class
>>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>>> as ignorable
>>>>>  at [Source:
>>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>>> line: 1, column: 938] (through reference chain:
>>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>>
>>>>>         at
>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>>>>>         at
>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>>>>>         at
>>>>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>>>>>
>>>>>         at
>>>>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
>>>>> Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
>>>>> Unrecognized field "isEnabled" (Class
>>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>>> as ignorable
>>>>>  at [Source:
>>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>>> line: 1, column: 938] (through reference chain:
>>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>>
>>>>>         at
>>>>> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>>>>>         at
>>>>> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>>>>>         at
>>>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>>>>>
>>>>>         at
>>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>>>>>         ... 5 more
>>>>> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache
>>>>> file does not exist or not readble
>>>>> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>>>>>
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards: HAFIZ MUJADID
>>>>
>>>
>>>
>>
>
>
>

Re: Hbase policy issue

Posted by Don Bosco Durai <bo...@apache.org>.
Aneela, thanks
Madhan, we should have both the jars in our plugin impl folder and make sure it works.
Aneela, if you don't mind, can you do one experiment? Can you revert your change? Have the old httpcore in the hbase lib folder, but add the 4.2.5 in the ranger-impl folder (check under lib folder) and try it out?
If it works, then we don't have to manually update the hbase libraries. 
Thanks
Bosco



    _____________________________
From: Aneela Saleem <an...@platalytics.com>
Sent: Sunday, December 6, 2015 5:09 AM
Subject: Re: Hbase policy issue
To:  <us...@ranger.incubator.apache.org>


       The issue was due to incompatible http-core version. Hbase by default has old version oh http-core. Replace old one with    
httpcore-4.4.2.jar that is compatible with httpclient-4.2.5.jar.       
       On Sat, Dec 5, 2015 at 10:43 PM, Aneela Saleem     <an...@platalytics.com> wrote:    
               Hi Madhan!             
                   In my Hadoop directory, The command        find . -name "ranger-plugins-common*.jar" | xargs ls –l  produces following output                   ./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar       
                   
                   
                   And above authorization issue is resolved now.                   
                   Thanks                   
                   
                                     
                 On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj          <ma...@apache.org> wrote:         
                                                                     Aneela,                                        
                                        Was the earlier issue with HBase authorization resolved now? This exception should not impact the authorization.                                        
                                        Thanks,                                        Madhan                                                                                                                                
                                               From:  Aneela Saleem <an...@platalytics.com>
Reply-To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
             Date:  Saturday, December 5, 2015 at 9:28 AM             
             To:  "             user@ranger.incubator.apache.org" <             user@ranger.incubator.apache.org>             
             Subject:  Re: Hbase policy issue             
                                                                  
                                                                            Hi Madhan!                                    
                                                    I reinstallled hbase plugin and checked the logs. Now the exception changed to following                                                    
                                                                       2015-12-05 22:25:42,251 FATAL [org.apache.ranger.audit.queue.AuditBatchQueue0] destination.SolrAuditDestination: Can't connect to Solr server. URL=[                   http://192.168.23.126:6083/solr/ranger_audits]                                                       java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET                                                               at org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)                                                               at org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)                                                               at org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)                                                               at org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)                                                               at org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)                                                               at org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)                                                               at org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)                                                               at org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)                                                               at org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)                                                               at org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)                                                               at java.security.AccessController.doPrivileged(Native Method)                                                               at javax.security.auth.Subject.doAs(Subject.java:356)                                                               at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)                                                               at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)                                                               at java.lang.Thread.run(Thread.java:745)                                                                      
                                                    
                                                    
                                                                  
                                   On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid                    <ha...@gmail.com> wrote:                  
                                                         hi Madhan!                                          
                                                             I have compiled ranger master branch code. Both the hbase and hbase-admin have same versions. I have hbase version 1.1.2.                                                             
                                                             Thanks                                                                                                                         
                                             On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj                         <ma...@apache.org> wrote:                       
                                                                                                                             Aneela,                                                                               
                                                                               It looks like the Ranger HBase plugin used in this environment is from an earlier version than the Ranger Admin. Can you please send the version details to troubleshoot this further?                                                                               
                                                                               While we look at fixing the issue in the code, you should be able to continue by using Ranger HBase plugin from the same version as Ranger Admin.                                                                               
                                                                               Thanks,                                                                               Madhan                                                                                                                                                                                                                                            
                                                                                                       From: Aneela Saleem <                           aneela@platalytics.com>                           
                           Reply-To: "                           user@ranger.incubator.apache.org" <                           user@ranger.incubator.apache.org>                           
                           Date: Saturday, December 5, 2015 at 7:43 AM                           
                           To: "                           user@ranger.incubator.apache.org" <                           user@ranger.incubator.apache.org>                           
                           Subject: Hbase policy issue                           
                                                                                                                                        
                                                                                                                                                  Hi!                                                                
                                                                                              I have made a policy on hbase table                                 emp with column family                                 f1  and column                                 name.                                                                                              I gave mike user the permission of read and  roger have all permissions.                                                                                              
                                                                                              But when i scan table using user mike or roger even. It does not allow                                                                                              ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘roger',action: scannerOpen, tableName:emp, family:f1                                
                                                                                              
                                                                                              No audits are being shown and hbase logs have following exception                                                                                              
                                                                                                                               2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher: PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will continue to use last known version of policies (-1)                                                                                                 com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable                                                                                                  at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])                                                                                                         at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)                                                                                                         at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)                                                                                                         at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)                                                                                                         at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)                                                                                                         at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)                                                                                                         at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)                                                                                                 Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable                                                                                                  at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])                                                                                                         at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)                                                                                                         at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)                                                                                                         at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)                                                                                                         at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)                                                                                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)                                                                                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)                                                                                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)                                                                                                         at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)                                                                                                         at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)                                                                                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)                                                                                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)                                                                                                         at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)                                                                                                         at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)                                                                                                         at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)                                                                                                         at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)                                                                                                         at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)                                                                                                         at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)                                                                                                         at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)                                                                                                         at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)                                                                                                         ... 5 more                                                                                                 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file does not exist or not readble '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'                                                                                                                              
                                                                                              
                                                                                              
                                                                                              Thanks                                                                                                                                                                                                                                                                      
                      
                                             
                                                                                   -- 
                                             Regards:                         HAFIZ MUJADID                       
                                                                                             
                                                                                                          
                             
    


  

Re: Hbase policy issue

Posted by Aneela Saleem <an...@platalytics.com>.
The issue was due to incompatible http-core version. Hbase by default has
old version oh http-core. Replace old one with
httpcore-4.4.2.jar that is compatible with httpclient-4.2.5.jar.

On Sat, Dec 5, 2015 at 10:43 PM, Aneela Saleem <an...@platalytics.com>
wrote:

> Hi Madhan!
>
> In my Hadoop directory, The command *find . -name
> "ranger-plugins-common*.jar" | xargs ls –l * produces following output
>
> ./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar
>
>
> And above authorization issue is resolved now.
>
> Thanks
>
>
>
> On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Aneela,
>>
>> Was the earlier issue with HBase authorization resolved now? This
>> exception should not impact the authorization.
>>
>> Thanks,
>> Madhan
>>
>> From: Aneela Saleem <an...@platalytics.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Saturday, December 5, 2015 at 9:28 AM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: Hbase policy issue
>>
>> Hi Madhan!
>>
>> I reinstallled hbase plugin and checked the logs. Now the exception
>> changed to following
>>
>> 2015-12-05 22:25:42,251 FATAL
>> [org.apache.ranger.audit.queue.AuditBatchQueue0]
>> destination.SolrAuditDestination: Can't connect to Solr server. URL=[
>> http://192.168.23.126:6083/solr/ranger_audits]
>> java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
>>         at
>> org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
>>         at
>> org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
>>         at
>> org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
>>         at
>> org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
>>         at
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
>>         at
>> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
>>         at
>> org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
>>         at
>> org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
>>         at
>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
>>         at
>> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:356)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>>         at
>> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
>>         at java.lang.Thread.run(Thread.java:745)
>>
>>
>>
>>
>> On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <ha...@gmail.com>
>> wrote:
>>
>>> hi Madhan!
>>>
>>> I have compiled ranger master branch code. Both the hbase and
>>> hbase-admin have same versions. I have hbase version 1.1.2.
>>>
>>> Thanks
>>>
>>> On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Aneela,
>>>>
>>>> It looks like the Ranger HBase plugin used in this environment is from
>>>> an earlier version than the Ranger Admin. Can you please send the version
>>>> details to troubleshoot this further?
>>>>
>>>> While we look at fixing the issue in the code, you should be able to
>>>> continue by using Ranger HBase plugin from the same version as Ranger Admin.
>>>>
>>>> Thanks,
>>>> Madhan
>>>>
>>>> From: Aneela Saleem <an...@platalytics.com>
>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Date: Saturday, December 5, 2015 at 7:43 AM
>>>> To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Hbase policy issue
>>>>
>>>> Hi!
>>>>
>>>> I have made a policy on hbase table *emp *with column family *f1 * and
>>>> column *name.*
>>>> I gave mike user the permission of read and  roger have all permissions.
>>>>
>>>> But when i scan table using user mike or roger even. It does not allow
>>>> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>>> Insufficient permissions for user ‘roger',action: scannerOpen,
>>>> tableName:emp, family:f1*
>>>>
>>>> No audits are being shown and hbase logs have following exception
>>>>
>>>> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
>>>> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
>>>> continue to use last known version of policies (-1)
>>>> com.sun.jersey.api.client.ClientHandlerException:
>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>>> field "isEnabled" (Class
>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>> as ignorable
>>>>  at [Source:
>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>> line: 1, column: 938] (through reference chain:
>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>         at
>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>>>>         at
>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>>>>         at
>>>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>>>>         at
>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>>>>         at
>>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>>>>         at
>>>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
>>>> Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
>>>> Unrecognized field "isEnabled" (Class
>>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>>> as ignorable
>>>>  at [Source:
>>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>>> line: 1, column: 938] (through reference chain:
>>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>>         at
>>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>>>>         at
>>>> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>         at
>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>         at
>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>>         at
>>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>>         at
>>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>>         at
>>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>>         at
>>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>>         at
>>>> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>>>>         at
>>>> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>>>>         at
>>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>>>>         at
>>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>>>>         ... 5 more
>>>> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache
>>>> file does not exist or not readble
>>>> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>>
>>> --
>>> Regards: HAFIZ MUJADID
>>>
>>
>>
>

Re: Hbase policy issue

Posted by Aneela Saleem <an...@platalytics.com>.
Hi Madhan!

In my Hadoop directory, The command *find . -name
"ranger-plugins-common*.jar" | xargs ls –l * produces following output
./share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl/ranger-plugins-common-0.5.0.jar


And above authorization issue is resolved now.

Thanks



On Sat, Dec 5, 2015 at 10:33 PM, Madhan Neethiraj <ma...@apache.org> wrote:

> Aneela,
>
> Was the earlier issue with HBase authorization resolved now? This
> exception should not impact the authorization.
>
> Thanks,
> Madhan
>
> From: Aneela Saleem <an...@platalytics.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Saturday, December 5, 2015 at 9:28 AM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: Hbase policy issue
>
> Hi Madhan!
>
> I reinstallled hbase plugin and checked the logs. Now the exception
> changed to following
>
> 2015-12-05 22:25:42,251 FATAL
> [org.apache.ranger.audit.queue.AuditBatchQueue0]
> destination.SolrAuditDestination: Can't connect to Solr server. URL=[
> http://192.168.23.126:6083/solr/ranger_audits]
> java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
>         at
> org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
>         at
> org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
>         at
> org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
>         at
> org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
>         at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
>         at
> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
>         at
> org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
>         at
> org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
>         at
> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
>         at
> org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:356)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>         at
> org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <ha...@gmail.com>
> wrote:
>
>> hi Madhan!
>>
>> I have compiled ranger master branch code. Both the hbase and hbase-admin
>> have same versions. I have hbase version 1.1.2.
>>
>> Thanks
>>
>> On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Aneela,
>>>
>>> It looks like the Ranger HBase plugin used in this environment is from
>>> an earlier version than the Ranger Admin. Can you please send the version
>>> details to troubleshoot this further?
>>>
>>> While we look at fixing the issue in the code, you should be able to
>>> continue by using Ranger HBase plugin from the same version as Ranger Admin.
>>>
>>> Thanks,
>>> Madhan
>>>
>>> From: Aneela Saleem <an...@platalytics.com>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Saturday, December 5, 2015 at 7:43 AM
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Hbase policy issue
>>>
>>> Hi!
>>>
>>> I have made a policy on hbase table *emp *with column family *f1 * and
>>> column *name.*
>>> I gave mike user the permission of read and  roger have all permissions.
>>>
>>> But when i scan table using user mike or roger even. It does not allow
>>> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>>> Insufficient permissions for user ‘roger',action: scannerOpen,
>>> tableName:emp, family:f1*
>>>
>>> No audits are being shown and hbase logs have following exception
>>>
>>> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
>>> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
>>> continue to use last known version of policies (-1)
>>> com.sun.jersey.api.client.ClientHandlerException:
>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>>> field "isEnabled" (Class
>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>> as ignorable
>>>  at [Source:
>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>> line: 1, column: 938] (through reference chain:
>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>         at
>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>>>         at
>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>>>         at
>>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>>>         at
>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>>>         at
>>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>>>         at
>>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
>>> Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
>>> Unrecognized field "isEnabled" (Class
>>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>>> as ignorable
>>>  at [Source:
>>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>>> line: 1, column: 938] (through reference chain:
>>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>>         at
>>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>>>         at
>>> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>>>         at
>>> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>>>         at
>>> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>         at
>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>         at
>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>         at
>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>         at
>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>         at
>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>         at
>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>>         at
>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>>         at
>>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>>         at
>>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>>         at
>>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>>         at
>>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>>         at
>>> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>>>         at
>>> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>>>         at
>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>>>         at
>>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>>>         ... 5 more
>>> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache
>>> file does not exist or not readble
>>> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>>>
>>>
>>>
>>> Thanks
>>>
>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>

Re: Hbase policy issue

Posted by Madhan Neethiraj <ma...@apache.org>.
Aneela,

Was the earlier issue with HBase authorization resolved now? This exception should not impact the authorization.

Thanks,
Madhan

From:  Aneela Saleem <an...@platalytics.com>
Reply-To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date:  Saturday, December 5, 2015 at 9:28 AM
To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject:  Re: Hbase policy issue

Hi Madhan! 

I reinstallled hbase plugin and checked the logs. Now the exception changed to following

2015-12-05 22:25:42,251 FATAL [org.apache.ranger.audit.queue.AuditBatchQueue0] destination.SolrAuditDestination: Can't connect to Solr server. URL=[http://192.168.23.126:6083/solr/ranger_audits]
java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
        at org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
        at org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
        at org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
        at org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
        at org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
        at org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
        at org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
        at org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
        at org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
        at org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
        at java.lang.Thread.run(Thread.java:745)




On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <ha...@gmail.com> wrote:
hi Madhan! 

I have compiled ranger master branch code. Both the hbase and hbase-admin have same versions. I have hbase version 1.1.2.

Thanks

On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Aneela,

It looks like the Ranger HBase plugin used in this environment is from an earlier version than the Ranger Admin. Can you please send the version details to troubleshoot this further?

While we look at fixing the issue in the code, you should be able to continue by using Ranger HBase plugin from the same version as Ranger Admin.

Thanks,
Madhan

From: Aneela Saleem <an...@platalytics.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Saturday, December 5, 2015 at 7:43 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Hbase policy issue

Hi! 

I have made a policy on hbase table emp with column family f1 and column name.
I gave mike user the permission of read and  roger have all permissions.

But when i scan table using user mike or roger even. It does not allow
ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘roger',action: scannerOpen, tableName:emp, family:f1

No audits are being shown and hbase logs have following exception

2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher: PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will continue to use last known version of policies (-1)
com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
        at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
        at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
        at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
        at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
        at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
        at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
        at org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
        ... 5 more
2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file does not exist or not readble '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'



Thanks



-- 
Regards: HAFIZ MUJADID



Re: Hbase policy issue

Posted by Aneela Saleem <an...@platalytics.com>.
Hi Madhan!

I reinstallled hbase plugin and checked the logs. Now the exception changed
to following

2015-12-05 22:25:42,251 FATAL
[org.apache.ranger.audit.queue.AuditBatchQueue0]
destination.SolrAuditDestination: Can't connect to Solr server. URL=[
http://192.168.23.126:6083/solr/ranger_audits]
java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET
        at
org.apache.http.impl.client.DefaultHttpClient.setDefaultHttpParams(DefaultHttpClient.java:175)
        at
org.apache.http.impl.client.DefaultHttpClient.createHttpParams(DefaultHttpClient.java:158)
        at
org.apache.http.impl.client.AbstractHttpClient.getParams(AbstractHttpClient.java:448)
        at
org.apache.solr.client.solrj.impl.HttpClientUtil.setConnectionTimeout(HttpClientUtil.java:267)
        at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.setConnectionTimeout(LBHttpSolrClient.java:460)
        at
org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:116)
        at
org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:138)
        at
org.apache.ranger.audit.queue.AuditBatchQueue.runDoAs(AuditBatchQueue.java:320)
        at
org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:217)
        at
org.apache.ranger.audit.queue.AuditBatchQueue$1.run(AuditBatchQueue.java:215)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
        at
org.apache.ranger.audit.queue.AuditBatchQueue.run(AuditBatchQueue.java:223)
        at java.lang.Thread.run(Thread.java:745)




On Sat, Dec 5, 2015 at 9:58 PM, Hafiz Mujadid <ha...@gmail.com>
wrote:

> hi Madhan!
>
> I have compiled ranger master branch code. Both the hbase and hbase-admin
> have same versions. I have hbase version 1.1.2.
>
> Thanks
>
> On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Aneela,
>>
>> It looks like the Ranger HBase plugin used in this environment is from an
>> earlier version than the Ranger Admin. Can you please send the version
>> details to troubleshoot this further?
>>
>> While we look at fixing the issue in the code, you should be able to
>> continue by using Ranger HBase plugin from the same version as Ranger Admin.
>>
>> Thanks,
>> Madhan
>>
>> From: Aneela Saleem <an...@platalytics.com>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Saturday, December 5, 2015 at 7:43 AM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Hbase policy issue
>>
>> Hi!
>>
>> I have made a policy on hbase table *emp *with column family *f1 * and
>> column *name.*
>> I gave mike user the permission of read and  roger have all permissions.
>>
>> But when i scan table using user mike or roger even. It does not allow
>> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
>> Insufficient permissions for user ‘roger',action: scannerOpen,
>> tableName:emp, family:f1*
>>
>> No audits are being shown and hbase logs have following exception
>>
>> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
>> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
>> continue to use last known version of policies (-1)
>> com.sun.jersey.api.client.ClientHandlerException:
>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
>> field "isEnabled" (Class
>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>> as ignorable
>>  at [Source:
>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>> line: 1, column: 938] (through reference chain:
>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>         at
>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>>         at
>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>>         at
>> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>>         at
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>>         at
>> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>>         at
>> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
>> Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
>> Unrecognized field "isEnabled" (Class
>> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
>> as ignorable
>>  at [Source:
>> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
>> line: 1, column: 938] (through reference chain:
>> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>>         at
>> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>>         at
>> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>>         at
>> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>>         at
>> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>         at
>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>         at
>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>         at
>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>         at
>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>         at
>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>         at
>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>>         at
>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>>         at
>> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>>         at
>> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>>         at
>> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>>         at
>> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>>         at
>> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>>         at
>> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>>         at
>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>>         at
>> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>>         ... 5 more
>> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache
>> file does not exist or not readble
>> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>>
>>
>>
>> Thanks
>>
>
>
>
> --
> Regards: HAFIZ MUJADID
>

Re: Hbase policy issue

Posted by Madhan Neethiraj <ma...@apache.org>.
Hafiz,

Can you please check if your environment has same ranger-plugins-common library across Ranger admin and plugins? Can you run the following command from your Hadoop installation root directory and send the output?

find . -name "ranger-plugins-common*.jar" | xargs ls –l

Thanks,
Madhan

From:  Hafiz Mujadid <ha...@gmail.com>
Reply-To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date:  Saturday, December 5, 2015 at 8:58 AM
To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject:  Re: Hbase policy issue

hi Madhan! 

I have compiled ranger master branch code. Both the hbase and hbase-admin have same versions. I have hbase version 1.1.2.

Thanks

On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org> wrote:
Aneela,

It looks like the Ranger HBase plugin used in this environment is from an earlier version than the Ranger Admin. Can you please send the version details to troubleshoot this further?

While we look at fixing the issue in the code, you should be able to continue by using Ranger HBase plugin from the same version as Ranger Admin.

Thanks,
Madhan

From: Aneela Saleem <an...@platalytics.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Saturday, December 5, 2015 at 7:43 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Hbase policy issue

Hi! 

I have made a policy on hbase table emp with column family f1 and column name.
I gave mike user the permission of read and  roger have all permissions.

But when i scan table using user mike or roger even. It does not allow
ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘roger',action: scannerOpen, tableName:emp, family:f1

No audits are being shown and hbase logs have following exception

2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher: PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will continue to use last known version of policies (-1)
com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
        at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
        at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
        at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
        at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
        at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
        at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
        at org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
        ... 5 more
2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file does not exist or not readble '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'



Thanks



-- 
Regards: HAFIZ MUJADID


Re: Hbase policy issue

Posted by Hafiz Mujadid <ha...@gmail.com>.
hi Madhan!

I have compiled ranger master branch code. Both the hbase and hbase-admin
have same versions. I have hbase version 1.1.2.

Thanks

On Sat, Dec 5, 2015 at 9:20 PM, Madhan Neethiraj <ma...@apache.org> wrote:

> Aneela,
>
> It looks like the Ranger HBase plugin used in this environment is from an
> earlier version than the Ranger Admin. Can you please send the version
> details to troubleshoot this further?
>
> While we look at fixing the issue in the code, you should be able to
> continue by using Ranger HBase plugin from the same version as Ranger Admin.
>
> Thanks,
> Madhan
>
> From: Aneela Saleem <an...@platalytics.com>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Saturday, December 5, 2015 at 7:43 AM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Hbase policy issue
>
> Hi!
>
> I have made a policy on hbase table *emp *with column family *f1 * and
> column *name.*
> I gave mike user the permission of read and  roger have all permissions.
>
> But when i scan table using user mike or roger even. It does not allow
> *ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
> Insufficient permissions for user ‘roger',action: scannerOpen,
> tableName:emp, family:f1*
>
> No audits are being shown and hbase logs have following exception
>
> 2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher:
> PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will
> continue to use last known version of policies (-1)
> com.sun.jersey.api.client.ClientHandlerException:
> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
> field "isEnabled" (Class
> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
> as ignorable
>  at [Source:
> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
> line: 1, column: 938] (through reference chain:
> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>         at
> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
>         at
> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
>         at
> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
>         at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
>         at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
>         at
> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
> Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException:
> Unrecognized field "isEnabled" (Class
> org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked
> as ignorable
>  at [Source:
> sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668;
> line: 1, column: 938] (through reference chain:
> org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
>         at
> org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
>         at
> org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
>         at
> org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
>         at
> org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>         at
> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>         at
> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>         at
> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>         at
> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>         at
> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>         at
> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
>         at
> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
>         at
> org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
>         at
> org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
>         at
> org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
>         at
> org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
>         at
> org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
>         at
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
>         at
> org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
>         at
> com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
>         ... 5 more
> 2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file
> does not exist or not readble
> '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'
>
>
>
> Thanks
>



-- 
Regards: HAFIZ MUJADID

Re: Hbase policy issue

Posted by Madhan Neethiraj <ma...@apache.org>.
Aneela,

It looks like the Ranger HBase plugin used in this environment is from an earlier version than the Ranger Admin. Can you please send the version details to troubleshoot this further?

While we look at fixing the issue in the code, you should be able to continue by using Ranger HBase plugin from the same version as Ranger Admin.

Thanks,
Madhan

From:  Aneela Saleem <an...@platalytics.com>
Reply-To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date:  Saturday, December 5, 2015 at 7:43 AM
To:  "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject:  Hbase policy issue

Hi! 

I have made a policy on hbase table emp with column family f1 and column name.
I gave mike user the permission of read and  roger have all permissions.

But when i scan table using user mike or roger even. It does not allow
ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘roger',action: scannerOpen, tableName:emp, family:f1

No audits are being shown and hbase logs have following exception

2015-12-05 20:42:32,976 ERROR [Thread-75] util.PolicyRefresher: PolicyRefresher(serviceName=hbasedev): failed to refresh policies. Will continue to use last known version of policies (-1)
com.sun.jersey.api.client.ClientHandlerException: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:563)
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506)
        at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:76)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
        at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "isEnabled" (Class org.apache.ranger.plugin.model.RangerPolicy$RangerPolicyItem), not marked as ignorable
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@28826668; line: 1, column: 938] (through reference chain: org.apache.ranger.plugin.util.ServicePolicies["policies"]->org.apache.ranger.plugin.model.RangerPolicy["policyItems"]->org.apache.ranger.plugin.model.RangerPolicyItem["isEnabled"])
        at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
        at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)
        at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:673)
        at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:659)
        at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1365)
        at org.codehaus.jackson.map.deser.BeanDeserializer._handleUnknown(BeanDeserializer.java:725)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:703)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$FieldProperty.deserializeAndSet(SettableBeanProperty.java:579)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.ObjectMapper._readValue(ObjectMapper.java:2704)
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1315)
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:419)
        at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:553)
        ... 5 more
2015-12-05 20:42:32,977 WARN  [Thread-75] util.PolicyRefresher: cache file does not exist or not readble '/etc/ranger/hbasedev/policycache/hbaseRegional_hbasedev.json'



Thanks