You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by "luochong@gdbigdata.com" <lu...@gdbigdata.com> on 2017/07/15 07:24:15 UTC

Which file config policy.download.auth.users ?

Hi
Env: 
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.  
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html 
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users   

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file? 


Thank & Regards



luochong@gdbigdata.com

Re: Re: Which file config policy.download.auth.users ?

Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.
Hi,
Now i solved this problem and share it. 
Problem:
   Hdfs fail to download policy from range admin in kerberos env.   In the namenode log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Solved :
   In ranger admin node,  add the parameter --hadoop.security.authentication   in ranger-admin-site.xml . 

 The config in my env as below: 

[root@admin141 conf]# pwd
/usr/local/ranger-0.7.0-admin/conf
[root@admin141 conf]# cat ranger-admin-site.xml 
<configuration>
    ... .... 
    <property>
        <name>hadoop.security.authorization</name>
        <value>true</value>
    </property>

    <property>
        <name>hadoop.security.authentication</name>
        <value>kerberos</value>
    </property>

</configuration>


Reason:
  When range admin receive the request message from hdfs, the function of RangerKRBAuthenticationFilter class will be invoked:

public void doFilter(ServletRequest request, ServletResponse response,
FilterChain filterChain) throws IOException, ServletException {
String authtype = PropertiesUtil.getProperty(RANGER_AUTH_TYPE);
HttpServletRequest httpRequest = (HttpServletRequest)request;

if(isSpnegoEnable(authtype)){   ----------------  if the authtype is not kerberos, the else step will be run.
... ... 
}else{
filterChain.doFilter(request, response);  -- At here ,the   RangerAuthenticationEntryPoint::commence() will be called. and the hdfs will receive  401 
}
}

And check the function  -- isSpnegoEnable(),   the system will get the value of  the parameter -- hadoop.security.authentication .  In my env, this parameter is not configured, so hdfs fail to download policy .  After i added this parameter, it is OK.



Thanks & Regards




luochong@gdbigdata.com
 
From: luochong@gdbigdata.com
Date: 2017-07-17 11:23
To: user
Subject: Re: Which file config policy.download.auth.users ?
Hi Ramesh,

Thanks for your reply.  I config this parameter in ranger UI, but the error is still exist.  I think i should not find the real reason.

I  read the code and find the error part .   It is in the   RangerAuthenticationEntryPoint::commence()  ( Ranger version : 0.7.0 )

public void commence(HttpServletRequest request,
HttpServletResponse response, AuthenticationException authException)
throws IOException, ServletException {
String ajaxRequestHeader = request.getHeader("X-Requested-With");
  .... .... 
if ("XMLHttpRequest".equals(ajaxRequestHeader)) {  
... .... 
return;
} else {
try {
logger.info("KKKK --- In commence.... ajaxRequestHeader = " + ajaxRequestHeader);  ///  Add one log info.    the ajaxRequestHeader is null. so the ranger admin reply 401.
VXResponse vXResponse = new VXResponse();

vXResponse.setStatusCode(HttpServletResponse.SC_UNAUTHORIZED);
vXResponse.setMsgDesc("Authentication Failed");
                                ... .... 
} 
}

 With tcpdump,  this parameter is not exist in tcp message.  But in the hdfs plugin file, this parameter could not be configed.   


Thanks & Regards
Luochong



luochong@gdbigdata.com
 
From: Ramesh Mani
Date: 2017-07-16 03:38
To: user@ranger.apache.org
Subject: Re: Which file config policy.download.auth.users ?
Luochong,

You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config  you will find “ Add new configuration”. Add these properties there.

Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng

Thanks,
Ramesh


From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>
Subject: Which file config policy.download.auth.users ?

Hi
Env: 
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.  
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html 
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users   

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file? 


Thank & Regards



luochong@gdbigdata.com

Re: Which file config policy.download.auth.users ?

Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.
Hi Ramesh,

Thanks for your reply.  I config this parameter in ranger UI, but the error is still exist.  I think i should not find the real reason.

I  read the code and find the error part .   It is in the   RangerAuthenticationEntryPoint::commence()  ( Ranger version : 0.7.0 )

public void commence(HttpServletRequest request,
HttpServletResponse response, AuthenticationException authException)
throws IOException, ServletException {
String ajaxRequestHeader = request.getHeader("X-Requested-With");
  .... .... 
if ("XMLHttpRequest".equals(ajaxRequestHeader)) {  
... .... 
return;
} else {
try {
logger.info("KKKK --- In commence.... ajaxRequestHeader = " + ajaxRequestHeader);  ///  Add one log info.    the ajaxRequestHeader is null. so the ranger admin reply 401.
VXResponse vXResponse = new VXResponse();

vXResponse.setStatusCode(HttpServletResponse.SC_UNAUTHORIZED);
vXResponse.setMsgDesc("Authentication Failed");
                                ... .... 
} 
}

 With tcpdump,  this parameter is not exist in tcp message.  But in the hdfs plugin file, this parameter could not be configed.   


Thanks & Regards
Luochong



luochong@gdbigdata.com
 
From: Ramesh Mani
Date: 2017-07-16 03:38
To: user@ranger.apache.org
Subject: Re: Which file config policy.download.auth.users ?
Luochong,

You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config  you will find “ Add new configuration”. Add these properties there.

Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng

Thanks,
Ramesh


From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>
Subject: Which file config policy.download.auth.users ?

Hi
Env: 
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.  
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html 
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users   

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file? 


Thank & Regards



luochong@gdbigdata.com

Re: Which file config policy.download.auth.users ?

Posted by Ramesh Mani <rm...@hortonworks.com>.
Luochong,

You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config  you will find “ Add new configuration”. Add these properties there.

Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng

Thanks,
Ramesh


From: "luochong@gdbigdata.com<ma...@gdbigdata.com>" <lu...@gdbigdata.com>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>>
Subject: Which file config policy.download.auth.users ?

Hi
Env:
Ranger version : 0.7.0
Hdfs : 2.7.0   with kerberos

After I installed hdfs\ ranger\ kerberos  manually,  i found the hdfs fail to download policy from Ranger.
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM<ma...@EXAMPLE.COM> (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null

Read this document:  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html
In this file, there is one step:

For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:

policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)

So i think it is the reason that hdfs fail to download policy.   I did not config this parameter  policy.download.auth.users

But i did not know how to config this parameter -- policy.download.auth.users   .   In the internet, this parameter should config in custom repo file.  But i did know the path the custom repo file?


Thank & Regards

________________________________
luochong@gdbigdata.com<ma...@gdbigdata.com>