You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ranger.apache.org by wenxing zheng <we...@gmail.com> on 2017/06/21 08:23:47 UTC

Ranger HBase plugin doesn't work

Dear all,

I am using the Ranger 0.7.1 against my HBASE 1.2.4. From the log files, I
can see HBase got the policy file correctly. But when I executed the "scan"
command from the "hbase shell", we got "Insufficient permissions".

>
> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
> Insufficient permissions for user 'hdfs' (table=users, action=READ)


Appreciated for any hints on how to determine the root cause
Regards, Wenxing

Re: Re: Which file config policy.download.auth.users ?

Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.

Hi,
Now i solved this problem and share it. 
Problem:
   Hdfs fail to download policy from range admin in kerberos env.   In the namenode log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Solved :
   In ranger admin node,  add the parameter --hadoop.security.authentication   in ranger-admin-site.xml . 

 The config in my env as below: 

[root@admin141 conf]# pwd
/usr/local/ranger-0.7.0-admin/conf
[root@admin141 conf]# cat ranger-admin-site.xml 
<configuration>
    ... .... 
    <property>
        <name>hadoop.security.authorization</name>
        <value>true</value>
    </property>

    <property>
        <name>hadoop.security.authentication</name>
        <value>kerberos</value>
    </property>

</configuration>


Reason:
  When range admin receive the request message from hdfs, the function of RangerKRBAuthenticationFilter class will be invoked:

public void doFilter(ServletRequest request, ServletResponse response,
FilterChain filterChain) throws IOException, ServletException {
String authtype = PropertiesUtil.getProperty(RANGER_AUTH_TYPE);
HttpServletRequest httpRequest = (HttpServletRequest)request;

if(isSpnegoEnable(authtype)){   ----------------  if the authtype is not kerberos, the else step will be run.
... ... 
}else{
filterChain.doFilter(request, response);  -- At here ,the   RangerAuthenticationEntryPoint::commence() will be called. and the hdfs will receive  401 
}
}

And check the function  -- isSpnegoEnable(),   the system will get the value of  the parameter -- hadoop.security.authentication .  In my env, this parameter is not configured, so hdfs fail to download policy .  After i added this parameter, it is OK.



Thanks & Regards




luochong@gdbigdata.com
 
From: luochong@gdbigdata.com
Date: 2017-07-17 11:23
To: user
Subject: Re: Which file config policy.download.auth.users ?
Hi Ramesh,

Thanks for your reply.  I config this parameter in ranger UI, but the error is still exist.  I think i should not find the real reason.

I  read the code and find the error part .   It is in the   RangerAuthenticationEntryPoint::commence()  ( Ranger version : 0.7.0 )

public void commence(HttpServletRequest request,
HttpServletResponse response, AuthenticationException authException)
throws IOException, ServletException {
String ajaxRequestHeader = request.getHeader("X-Requested-With");
  .... .... 
if ("XMLHttpRequest".equals(ajaxRequestHeader)) {  
... .... 
return;
} else {
try {
logger.info("KKKK --- In commence.... ajaxRequestHeader = " + ajaxRequestHeader);  ///  Add one log info.    the ajaxRequestHeader is null. so the ranger admin reply 401.
VXResponse vXResponse = new VXResponse();

vXResponse.setStatusCode(HttpServletResponse.SC_UNAUTHORIZED);
vXResponse.setMsgDesc("Authentication Failed");
                                ... .... 
} 
}

 With tcpdump,  this parameter is not exist in tcp message.  But in the hdfs plugin file, this parameter could not be configed.   


Thanks & Regards
Luochong



luochong@gdbigdata.com
 
From: Ramesh Mani
Date: 2017-07-16 03:38
To: user@ranger.apache.org
Subject: Re: Which file config policy.download.auth.users ?
Luochong,

You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config  you will find “ Add new configuration”. Add these properties there.

Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng

Thanks,
Ramesh


From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>
Subject: Which file config policy.download.auth.users ?

Hi
Env: 
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.  
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html 
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users   

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file? 


Thank & Regards



luochong@gdbigdata.com

Re: Which file config policy.download.auth.users ?

Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.

Hi Ramesh,

Thanks for your reply.  I config this parameter in ranger UI, but the error is still exist.  I think i should not find the real reason.

I  read the code and find the error part .   It is in the   RangerAuthenticationEntryPoint::commence()  ( Ranger version : 0.7.0 )

public void commence(HttpServletRequest request,
HttpServletResponse response, AuthenticationException authException)
throws IOException, ServletException {
String ajaxRequestHeader = request.getHeader("X-Requested-With");
  .... .... 
if ("XMLHttpRequest".equals(ajaxRequestHeader)) {  
... .... 
return;
} else {
try {
logger.info("KKKK --- In commence.... ajaxRequestHeader = " + ajaxRequestHeader);  ///  Add one log info.    the ajaxRequestHeader is null. so the ranger admin reply 401.
VXResponse vXResponse = new VXResponse();

vXResponse.setStatusCode(HttpServletResponse.SC_UNAUTHORIZED);
vXResponse.setMsgDesc("Authentication Failed");
                                ... .... 
} 
}

 With tcpdump,  this parameter is not exist in tcp message.  But in the hdfs plugin file, this parameter could not be configed.   


Thanks & Regards
Luochong



luochong@gdbigdata.com
 
From: Ramesh Mani
Date: 2017-07-16 03:38
To: user@ranger.apache.org
Subject: Re: Which file config policy.download.auth.users ?
Luochong,

You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config  you will find “ Add new configuration”. Add these properties there.

Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng

Thanks,
Ramesh


From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>
Subject: Which file config policy.download.auth.users ?

Hi
Env: 
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.  
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html 
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users   

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file? 


Thank & Regards



luochong@gdbigdata.com

Re: Which file config policy.download.auth.users ?

Posted by Ramesh Mani <rm...@hortonworks.com>.

Luochong,

You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config  you will find “ Add new configuration”. Add these properties there.

Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng

Thanks,
Ramesh


From: "luochong@gdbigdata.com<ma...@gdbigdata.com>" <lu...@gdbigdata.com>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>>
Subject: Which file config policy.download.auth.users ?

Hi
Env:
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM<ma...@EXAMPLE.COM> (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html
In this file, there is one step:

For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:

policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)

So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file?


Thank & Regards

________________________________
luochong@gdbigdata.com<ma...@gdbigdata.com>

Which file config policy.download.auth.users ?

Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.

Hi
Env: 
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.  
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html 
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users   

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file? 


Thank & Regards



luochong@gdbigdata.com

Re: Two question about ranger usersync

Posted by Bhavik Patel <bh...@gmail.com>.

Hi luochong,

Regarding, *Is there another way to sync user info automatically?*
Ranger-Usersync process syncs the added users in particular time interval
which is in minutes; By default it is set to 5 minutes.The property name is
"SYNC_INTERVAL". So it will sync newly added user after that interval or
you have restart ranger-usersync.
You can update this value in install.properties file and run the setup.sh
script to update the value in the Usersync process, after this you will
have to restart ranger-usersync process.

Regarding,
* How to remove the user info of ranger?*    To delete the user from Ranger
database, you will have to delete the user from Ranger UI manually, as the
Ranger-Usersync process will only add users to Ranger Database.

Regard,
Bhavik Patel
+91-7208744109

On Thu, Jun 22, 2017 at 1:16 PM, luochong@gdbigdata.com <
luochong@gdbigdata.com> wrote:

> Hi,
> Now  I  sync the users  of  Linux OS to Ranger. But i meet two questions.
>
> Ranger Version :  0.7.0
>
> 1) Ranger could not sync OS user info .
>  Steps:
>      1) Excecute the command :    ranger-usersync start
>      2)  then create one user in linux :      useradd -d /usr/unix02 -m
> unix02
>      3) Check the ranger database.
>  But I could not find this user -- unix02  in ranger database.   Only
>  restart the ranger-usersync ,  this record can be found.
> In the Internet, there is one record -- https://community.
> hortonworks.com/questions/21170/is-there-a-way-to-force-
> ranger-user-sync.html.   In this, they suggested execute the command  UnixUserGroupBuilder
> manually.
>
> My Question:   Is there another way to sync user info automaticly?
>
> 2) Ranger  could not remove the user
>    Steps:
>      1)  add the user in OS system
>      2) Ranger sync this user to ranger database after restart
> ranger-usersync
>      3) delete this user  from OS system
>      4) restart the ranger-usersync.
>
> This deleted user is still in Ranger database,
>
> My question:   How to remove the user info of ranger?
>
>
> Thanks & Regards
>
> ------------------------------
> luochong@gdbigdata.com
>
>

Two question about ranger usersync

Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.

Hi, 
Now  I  sync the users  of  Linux OS to Ranger. But i meet two questions.

Ranger Version :  0.7.0

1) Ranger could not sync OS user info . 
 Steps:
     1) Excecute the command :    ranger-usersync start
     2)  then create one user in linux :      useradd -d /usr/unix02 -m unix02
     3) Check the ranger database. 
 But I could not find this user -- unix02  in ranger database.   Only  restart the ranger-usersync ,  this record can be found.   
In the Internet, there is one record -- https://community.hortonworks.com/questions/21170/is-there-a-way-to-force-ranger-user-sync.html.   In this, they suggested execute the command  UnixUserGroupBuilder manually. 

My Question:   Is there another way to sync user info automaticly?

2) Ranger  could not remove the user
   Steps:
     1)  add the user in OS system
     2) Ranger sync this user to ranger database after restart ranger-usersync
     3) delete this user  from OS system 
     4) restart the ranger-usersync.   

This deleted user is still in Ranger database,

My question:   How to remove the user info of ranger? 


Thanks & Regards



luochong@gdbigdata.com

Re: Ranger HBase plugin doesn't work

Posted by Madhan Neethiraj <ma...@apache.org>.

To troubleshoot further, consider setting log level to debug for the following packages and trace the policy evaluation details in HBase log files:

    org.apache.ranger.plugin.policyevaluator

    org.apache.ranger.plugin.policyengine

 

Madhan

 

 

From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Wednesday, June 21, 2017 at 11:17 PM
To: user <us...@ranger.apache.org>
Subject: Re: Ranger HBase plugin doesn't work

 

Hi

Maybe I did not know your question clearly.   you login the hbase system with the user -- hdfs,  , the system will check the access when you execute the command.  And it found there is no right of hdfs, it throw exception. 

 

It is corret ,  i think.  Maybe  you login with another user, it will be ok. 

 

Regards

 

luochong@gdbigdata.com

 

From: wenxing zheng

Date: 2017-06-21 16:23

To: user

Subject: Ranger HBase plugin doesn't work

Dear all,

 

I am using the Ranger 0.7.1 against my HBASE 1.2.4. From the log files, I can see HBase got the policy file correctly. But when I executed the "scan" command from the "hbase shell", we got "Insufficient permissions".


ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'hdfs' (table=users, action=READ)

 

Appreciated for any hints on how to determine the root cause

Regards, Wenxing