You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Hanish Bansal <ha...@gmail.com> on 2014/12/09 17:26:04 UTC
Not able to perform hdfs operation after enabling hdfs plugin of ranger
Hi All,
I am installing ranger on a standalone machine. I have installed
ranger-0.4.0-admin successfully.
Also i have enabled hdfs plugin, that is installed successfully.
When I try to do any hdfs operation like cat a file through command line or
browse file system through web-ui (<ip:50070>), I am seeing only below
class name there:
com.xasecure.audit.model.AuditEventBase
It is not giving anything other.
Request you to help to resolve this issue.
--
*Thanks & Regards*
*Hanish Bansal*
Re: Not able to perform hdfs operation after enabling hdfs plugin of ranger
Posted by Selvamohan Neethiraj <sn...@apache.org>.
Thanks Hanish for the feedback on the library settings .... We will document these in the Wiki soon ...
The parameter "xasecure.add-hadoop-authorization" will allow Ranger Authorization to fall back to HDFS authorization ONLY IF RANGER could not provide you with requested access on the requested resource based on the RANGER policies. Unless you have all access control policies defined in RANGER, you should leave this parameter as true.
Since RANGER installation by default enable audit for all access and does not provide any default access to any user, you are seeing "permission denied" error for all access.
To test RANGER authorization/audit, you can define a policy with resource name as "/" and recursive = TRUE and grant all access to your desired user-id and then, see if the hdfs access is being granted for the specific user-id.
Thanks,
Selva-
> On Dec 10, 2014, at 9:04 AM, Hanish Bansal <ha...@gmail.com> wrote:
>
> I resolved this issue. Hadoop was not picking up libraries from <hadoop-home>/lib folder. Hdfs plugin enable script copies the script in <hadoop-home>/lib.
>
> So i manually copy all libraries from <hadoop-home>/lib to <hadoop-home>/share/hadoop/hdfs/lib directory.
>
> cp <hadoop-home>/lib/* <hadoop-home>/share/hadoop/hdfs/lib/
>
> After this, class name problem is resolved and everything working fine as expected in hadoop-2.6.0.
>
> Additionally i disabled hadoop authorization by changing below property in etc/hadoop/xasecure-hdfs-security.xml to make work xasecure(ranger) authorization.
> <property>
> <name>xasecure.add-hadoop-authorization</name>
> <value>false</value>
> </property>
>
> This thing is missing in ranger installation guide.
>
> After changes everything working fine as expected in hadoop-2.6.0. But in hadoop-2.2.0 authorization not working.
>
> For every operation on hdfs it is giving permission denied error in hadoop-2.2.0.
>
>
>
>
> On Wed, Dec 10, 2014 at 3:34 PM, Hanish Bansal <hanish.bansal.agarwal@gmail.com <ma...@gmail.com>> wrote:
> Tried with Hadoop 2.6.0 also. Getting same error.
>
> Please find snapshot as:
>
> <rsz_screenshot-3.jpg>
>
> Please let me know if anyone knows about the solution for this.
>
>
> On Wed, Dec 10, 2014 at 3:08 PM, Hanish Bansal <hanish.bansal.agarwal@gmail.com <ma...@gmail.com>> wrote:
> We are not planning for any upgrade for now.
>
> I am seeing this message on browser when i am browsing file system through web-ui (<ip:50070>)
> Also on command line when i am trying to read a file using cat command.
>
> On Browser:
> com/xasecure/audit/model/AuditEventBase
>
> On command line:
>
> [hanish@domain-d962centos hadoop-2.2.0]$ bin/hadoop fs -cat /customer_data
> cat: com/xasecure/audit/model/AuditEventBase
>
>
>
> On Wed, Dec 10, 2014 at 12:51 PM, Don Bosco Durai <bosco@apache.org <ma...@apache.org>> wrote:
> Hanish, 0.4.0 is compatible with Hadoop 2.6 and Hive 0.14
>
> The XASecure code before it was released in Apache as Ranger is compatible with older versions of Hadoop. But since they were using LGPL licenses, we had to remediate the code base as per Apache guidelines.
>
> Let me know your upgrade path. If you are not going to upgrade anytime soon, then let me see if there any options for you to try it out.
>
> Also, your error below seems to odd. HDFS plugin should have worked with previous version. Where are you seeing the message “com.xasecure.audit.model.AuditEventBase” ?
>
> Thanks
>
> Bosco
>
>> On Dec 9, 2014, at 10:10 PM, Hanish Bansal <hanish.bansal.agarwal@gmail.com <ma...@gmail.com>> wrote:
>>
>> I am using hadoop version- 2.2.0 and hive version- 0.13.1
>>
>> Are these versions compatible?
>>
>> On Tue, Dec 9, 2014 at 9:56 PM, Hanish Bansal <hanish.bansal.agarwal@gmail.com <ma...@gmail.com>> wrote:
>> Hi All,
>>
>> I am installing ranger on a standalone machine. I have installed ranger-0.4.0-admin successfully.
>>
>> Also i have enabled hdfs plugin, that is installed successfully.
>> When I try to do any hdfs operation like cat a file through command line or browse file system through web-ui (<ip:50070>), I am seeing only below class name there:
>>
>> com.xasecure.audit.model.AuditEventBase
>>
>> It is not giving anything other.
>>
>> Request you to help to resolve this issue.
>>
>> --
>> Thanks & Regards
>> Hanish Bansal
>>
>>
>>
>> --
>> Thanks & Regards
>> Hanish Bansal
>
>
>
>
> --
> Thanks & Regards
> Hanish Bansal
>
>
>
> --
> Thanks & Regards
> Hanish Bansal
>
>
>
> --
> Thanks & Regards
> Hanish Bansal
Re: Not able to perform hdfs operation after enabling hdfs plugin of ranger
Posted by Hanish Bansal <ha...@gmail.com>.
I resolved this issue. Hadoop was not picking up libraries from
<hadoop-home>/lib folder. Hdfs plugin enable script copies the script in
<hadoop-home>/lib.
So i manually copy all libraries from <hadoop-home>/lib to
<hadoop-home>/share/hadoop/hdfs/lib
directory.
cp <hadoop-home>/lib/* <hadoop-home>/share/hadoop/hdfs/lib/
After this, class name problem is resolved and everything working fine as
expected in hadoop-2.6.0.
Additionally i disabled hadoop authorization by changing below
property in etc/hadoop/xasecure-hdfs-security.xml
to make work xasecure(ranger) authorization.
<property>
<name>xasecure.add-hadoop-authorization</name>
<value>false</value>
</property>
This thing is missing in ranger installation guide.
After changes everything working fine as expected in hadoop-2.6.0. But in
hadoop-2.2.0 authorization not working.
*For every operation on hdfs it is giving permission denied error in
hadoop-2.2.0.*
On Wed, Dec 10, 2014 at 3:34 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:
> Tried with Hadoop 2.6.0 also. Getting same error.
>
> Please find snapshot as:
>
>
>
> Please let me know if anyone knows about the solution for this.
>
>
> On Wed, Dec 10, 2014 at 3:08 PM, Hanish Bansal <
> hanish.bansal.agarwal@gmail.com> wrote:
>
>> We are not planning for any upgrade for now.
>>
>> I am seeing this message on browser when i am browsing file system
>> through web-ui (<ip:50070>)
>> Also on command line when i am trying to read a file using cat command.
>>
>> On Browser:
>> com/xasecure/audit/model/AuditEventBase
>>
>> On command line:
>>
>> [hanish@domain-d962centos hadoop-2.2.0]$ bin/hadoop fs -cat
>> /customer_data
>> cat: com/xasecure/audit/model/AuditEventBase
>>
>>
>> On Wed, Dec 10, 2014 at 12:51 PM, Don Bosco Durai <bo...@apache.org>
>> wrote:
>>
>>> Hanish, 0.4.0 is compatible with Hadoop 2.6 and Hive 0.14
>>>
>>> The XASecure code before it was released in Apache as Ranger is
>>> compatible with older versions of Hadoop. But since they were using LGPL
>>> licenses, we had to remediate the code base as per Apache guidelines.
>>>
>>> Let me know your upgrade path. If you are not going to upgrade anytime
>>> soon, then let me see if there any options for you to try it out.
>>>
>>> Also, your error below seems to odd. HDFS plugin should have worked with
>>> previous version. Where are you seeing the message
>>> “com.xasecure.audit.model.AuditEventBase” ?
>>>
>>> Thanks
>>>
>>> Bosco
>>>
>>> On Dec 9, 2014, at 10:10 PM, Hanish Bansal <
>>> hanish.bansal.agarwal@gmail.com> wrote:
>>>
>>> I am using hadoop version- 2.2.0 and hive version- 0.13.1
>>>
>>> Are these versions compatible?
>>>
>>> On Tue, Dec 9, 2014 at 9:56 PM, Hanish Bansal <
>>> hanish.bansal.agarwal@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I am installing ranger on a standalone machine. I have installed
>>>> ranger-0.4.0-admin successfully.
>>>>
>>>> Also i have enabled hdfs plugin, that is installed successfully.
>>>> When I try to do any hdfs operation like cat a file through command
>>>> line or browse file system through web-ui (<ip:50070>), I am seeing only
>>>> below class name there:
>>>>
>>>> com.xasecure.audit.model.AuditEventBase
>>>>
>>>> It is not giving anything other.
>>>>
>>>> Request you to help to resolve this issue.
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>> *Hanish Bansal*
>>>>
>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>> *Hanish Bansal*
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>> *Hanish Bansal*
>>
>
>
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>
--
*Thanks & Regards*
*Hanish Bansal*
Re: Not able to perform hdfs operation after enabling hdfs plugin of ranger
Posted by Hanish Bansal <ha...@gmail.com>.
Tried with Hadoop 2.6.0 also. Getting same error.
Please find snapshot as:
Please let me know if anyone knows about the solution for this.
On Wed, Dec 10, 2014 at 3:08 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:
> We are not planning for any upgrade for now.
>
> I am seeing this message on browser when i am browsing file system through
> web-ui (<ip:50070>)
> Also on command line when i am trying to read a file using cat command.
>
> On Browser:
> com/xasecure/audit/model/AuditEventBase
>
> On command line:
>
> [hanish@domain-d962centos hadoop-2.2.0]$ bin/hadoop fs -cat
> /customer_data
> cat: com/xasecure/audit/model/AuditEventBase
>
>
> On Wed, Dec 10, 2014 at 12:51 PM, Don Bosco Durai <bo...@apache.org>
> wrote:
>
>> Hanish, 0.4.0 is compatible with Hadoop 2.6 and Hive 0.14
>>
>> The XASecure code before it was released in Apache as Ranger is
>> compatible with older versions of Hadoop. But since they were using LGPL
>> licenses, we had to remediate the code base as per Apache guidelines.
>>
>> Let me know your upgrade path. If you are not going to upgrade anytime
>> soon, then let me see if there any options for you to try it out.
>>
>> Also, your error below seems to odd. HDFS plugin should have worked with
>> previous version. Where are you seeing the message
>> “com.xasecure.audit.model.AuditEventBase” ?
>>
>> Thanks
>>
>> Bosco
>>
>> On Dec 9, 2014, at 10:10 PM, Hanish Bansal <
>> hanish.bansal.agarwal@gmail.com> wrote:
>>
>> I am using hadoop version- 2.2.0 and hive version- 0.13.1
>>
>> Are these versions compatible?
>>
>> On Tue, Dec 9, 2014 at 9:56 PM, Hanish Bansal <
>> hanish.bansal.agarwal@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I am installing ranger on a standalone machine. I have installed
>>> ranger-0.4.0-admin successfully.
>>>
>>> Also i have enabled hdfs plugin, that is installed successfully.
>>> When I try to do any hdfs operation like cat a file through command line
>>> or browse file system through web-ui (<ip:50070>), I am seeing only below
>>> class name there:
>>>
>>> com.xasecure.audit.model.AuditEventBase
>>>
>>> It is not giving anything other.
>>>
>>> Request you to help to resolve this issue.
>>>
>>> --
>>> *Thanks & Regards*
>>> *Hanish Bansal*
>>>
>>
>>
>>
>> --
>> *Thanks & Regards*
>> *Hanish Bansal*
>>
>>
>>
>
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>
--
*Thanks & Regards*
*Hanish Bansal*
Re: Not able to perform hdfs operation after enabling hdfs plugin of ranger
Posted by Hanish Bansal <ha...@gmail.com>.
We are not planning for any upgrade for now.
I am seeing this message on browser when i am browsing file system through
web-ui (<ip:50070>)
Also on command line when i am trying to read a file using cat command.
On Browser:
com/xasecure/audit/model/AuditEventBase
On command line:
[hanish@domain-d962centos hadoop-2.2.0]$ bin/hadoop fs -cat /customer_data
cat: com/xasecure/audit/model/AuditEventBase
On Wed, Dec 10, 2014 at 12:51 PM, Don Bosco Durai <bo...@apache.org> wrote:
> Hanish, 0.4.0 is compatible with Hadoop 2.6 and Hive 0.14
>
> The XASecure code before it was released in Apache as Ranger is compatible
> with older versions of Hadoop. But since they were using LGPL licenses, we
> had to remediate the code base as per Apache guidelines.
>
> Let me know your upgrade path. If you are not going to upgrade anytime
> soon, then let me see if there any options for you to try it out.
>
> Also, your error below seems to odd. HDFS plugin should have worked with
> previous version. Where are you seeing the message
> “com.xasecure.audit.model.AuditEventBase” ?
>
> Thanks
>
> Bosco
>
> On Dec 9, 2014, at 10:10 PM, Hanish Bansal <
> hanish.bansal.agarwal@gmail.com> wrote:
>
> I am using hadoop version- 2.2.0 and hive version- 0.13.1
>
> Are these versions compatible?
>
> On Tue, Dec 9, 2014 at 9:56 PM, Hanish Bansal <
> hanish.bansal.agarwal@gmail.com> wrote:
>
>> Hi All,
>>
>> I am installing ranger on a standalone machine. I have installed
>> ranger-0.4.0-admin successfully.
>>
>> Also i have enabled hdfs plugin, that is installed successfully.
>> When I try to do any hdfs operation like cat a file through command line
>> or browse file system through web-ui (<ip:50070>), I am seeing only below
>> class name there:
>>
>> com.xasecure.audit.model.AuditEventBase
>>
>> It is not giving anything other.
>>
>> Request you to help to resolve this issue.
>>
>> --
>> *Thanks & Regards*
>> *Hanish Bansal*
>>
>
>
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>
>
>
--
*Thanks & Regards*
*Hanish Bansal*
Re: Not able to perform hdfs operation after enabling hdfs plugin of ranger
Posted by Don Bosco Durai <bo...@apache.org>.
Hanish, 0.4.0 is compatible with Hadoop 2.6 and Hive 0.14
The XASecure code before it was released in Apache as Ranger is compatible with older versions of Hadoop. But since they were using LGPL licenses, we had to remediate the code base as per Apache guidelines.
Let me know your upgrade path. If you are not going to upgrade anytime soon, then let me see if there any options for you to try it out.
Also, your error below seems to odd. HDFS plugin should have worked with previous version. Where are you seeing the message “com.xasecure.audit.model.AuditEventBase” ?
Thanks
Bosco
> On Dec 9, 2014, at 10:10 PM, Hanish Bansal <ha...@gmail.com> wrote:
>
> I am using hadoop version- 2.2.0 and hive version- 0.13.1
>
> Are these versions compatible?
>
> On Tue, Dec 9, 2014 at 9:56 PM, Hanish Bansal <hanish.bansal.agarwal@gmail.com <ma...@gmail.com>> wrote:
> Hi All,
>
> I am installing ranger on a standalone machine. I have installed ranger-0.4.0-admin successfully.
>
> Also i have enabled hdfs plugin, that is installed successfully.
> When I try to do any hdfs operation like cat a file through command line or browse file system through web-ui (<ip:50070>), I am seeing only below class name there:
>
> com.xasecure.audit.model.AuditEventBase
>
> It is not giving anything other.
>
> Request you to help to resolve this issue.
>
> --
> Thanks & Regards
> Hanish Bansal
>
>
>
> --
> Thanks & Regards
> Hanish Bansal
Re: Not able to perform hdfs operation after enabling hdfs plugin of ranger
Posted by Hanish Bansal <ha...@gmail.com>.
I am using hadoop version- 2.2.0 and hive version- 0.13.1
Are these versions compatible?
On Tue, Dec 9, 2014 at 9:56 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:
> Hi All,
>
> I am installing ranger on a standalone machine. I have installed
> ranger-0.4.0-admin successfully.
>
> Also i have enabled hdfs plugin, that is installed successfully.
> When I try to do any hdfs operation like cat a file through command line
> or browse file system through web-ui (<ip:50070>), I am seeing only below
> class name there:
>
> com.xasecure.audit.model.AuditEventBase
>
> It is not giving anything other.
>
> Request you to help to resolve this issue.
>
> --
> *Thanks & Regards*
> *Hanish Bansal*
>
--
*Thanks & Regards*
*Hanish Bansal*