You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by "luochong@gdbigdata.com" <lu...@gdbigdata.com> on 2017/06/22 07:46:51 UTC
Two question about ranger usersync
Hi,
Now I sync the users of Linux OS to Ranger. But i meet two questions.
Ranger Version : 0.7.0
1) Ranger could not sync OS user info .
Steps:
1) Excecute the command : ranger-usersync start
2) then create one user in linux : useradd -d /usr/unix02 -m unix02
3) Check the ranger database.
But I could not find this user -- unix02 in ranger database. Only restart the ranger-usersync , this record can be found.
In the Internet, there is one record -- https://community.hortonworks.com/questions/21170/is-there-a-way-to-force-ranger-user-sync.html. In this, they suggested execute the command UnixUserGroupBuilder manually.
My Question: Is there another way to sync user info automaticly?
2) Ranger could not remove the user
Steps:
1) add the user in OS system
2) Ranger sync this user to ranger database after restart ranger-usersync
3) delete this user from OS system
4) restart the ranger-usersync.
This deleted user is still in Ranger database,
My question: How to remove the user info of ranger?
Thanks & Regards
luochong@gdbigdata.com
Re: Re: Which file config policy.download.auth.users ?
Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.
Hi,
Now i solved this problem and share it.
Problem:
Hdfs fail to download policy from range admin in kerberos env. In the namenode log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null
Solved :
In ranger admin node, add the parameter --hadoop.security.authentication in ranger-admin-site.xml .
The config in my env as below:
[root@admin141 conf]# pwd
/usr/local/ranger-0.7.0-admin/conf
[root@admin141 conf]# cat ranger-admin-site.xml
<configuration>
... ....
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
</configuration>
Reason:
When range admin receive the request message from hdfs, the function of RangerKRBAuthenticationFilter class will be invoked:
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain filterChain) throws IOException, ServletException {
String authtype = PropertiesUtil.getProperty(RANGER_AUTH_TYPE);
HttpServletRequest httpRequest = (HttpServletRequest)request;
if(isSpnegoEnable(authtype)){ ---------------- if the authtype is not kerberos, the else step will be run.
... ...
}else{
filterChain.doFilter(request, response); -- At here ,the RangerAuthenticationEntryPoint::commence() will be called. and the hdfs will receive 401
}
}
And check the function -- isSpnegoEnable(), the system will get the value of the parameter -- hadoop.security.authentication . In my env, this parameter is not configured, so hdfs fail to download policy . After i added this parameter, it is OK.
Thanks & Regards
luochong@gdbigdata.com
From: luochong@gdbigdata.com
Date: 2017-07-17 11:23
To: user
Subject: Re: Which file config policy.download.auth.users ?
Hi Ramesh,
Thanks for your reply. I config this parameter in ranger UI, but the error is still exist. I think i should not find the real reason.
I read the code and find the error part . It is in the RangerAuthenticationEntryPoint::commence() ( Ranger version : 0.7.0 )
public void commence(HttpServletRequest request,
HttpServletResponse response, AuthenticationException authException)
throws IOException, ServletException {
String ajaxRequestHeader = request.getHeader("X-Requested-With");
.... ....
if ("XMLHttpRequest".equals(ajaxRequestHeader)) {
... ....
return;
} else {
try {
logger.info("KKKK --- In commence.... ajaxRequestHeader = " + ajaxRequestHeader); /// Add one log info. the ajaxRequestHeader is null. so the ranger admin reply 401.
VXResponse vXResponse = new VXResponse();
vXResponse.setStatusCode(HttpServletResponse.SC_UNAUTHORIZED);
vXResponse.setMsgDesc("Authentication Failed");
... ....
}
}
With tcpdump, this parameter is not exist in tcp message. But in the hdfs plugin file, this parameter could not be configed.
Thanks & Regards
Luochong
luochong@gdbigdata.com
From: Ramesh Mani
Date: 2017-07-16 03:38
To: user@ranger.apache.org
Subject: Re: Which file config policy.download.auth.users ?
Luochong,
You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config you will find “ Add new configuration”. Add these properties there.
Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng
Thanks,
Ramesh
From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>
Subject: Which file config policy.download.auth.users ?
Hi
Env:
Ranger version : 0.7.0
Hdfs : 2.7.0 with kerberos
After I installed hdfs\ ranger\ kerberos manually, i found the hdfs fail to download policy from Ranger.
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null
Read this document: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy. I did not config this parameter policy.download.auth.users
But i did not know how to config this parameter -- policy.download.auth.users . In the internet, this parameter should config in custom repo file. But i did know the path the custom repo file?
Thank & Regards
luochong@gdbigdata.com
Re: Which file config policy.download.auth.users ?
Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.
Hi Ramesh,
Thanks for your reply. I config this parameter in ranger UI, but the error is still exist. I think i should not find the real reason.
I read the code and find the error part . It is in the RangerAuthenticationEntryPoint::commence() ( Ranger version : 0.7.0 )
public void commence(HttpServletRequest request,
HttpServletResponse response, AuthenticationException authException)
throws IOException, ServletException {
String ajaxRequestHeader = request.getHeader("X-Requested-With");
.... ....
if ("XMLHttpRequest".equals(ajaxRequestHeader)) {
... ....
return;
} else {
try {
logger.info("KKKK --- In commence.... ajaxRequestHeader = " + ajaxRequestHeader); /// Add one log info. the ajaxRequestHeader is null. so the ranger admin reply 401.
VXResponse vXResponse = new VXResponse();
vXResponse.setStatusCode(HttpServletResponse.SC_UNAUTHORIZED);
vXResponse.setMsgDesc("Authentication Failed");
... ....
}
}
With tcpdump, this parameter is not exist in tcp message. But in the hdfs plugin file, this parameter could not be configed.
Thanks & Regards
Luochong
luochong@gdbigdata.com
From: Ramesh Mani
Date: 2017-07-16 03:38
To: user@ranger.apache.org
Subject: Re: Which file config policy.download.auth.users ?
Luochong,
You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config you will find “ Add new configuration”. Add these properties there.
Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng
Thanks,
Ramesh
From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>
Subject: Which file config policy.download.auth.users ?
Hi
Env:
Ranger version : 0.7.0
Hdfs : 2.7.0 with kerberos
After I installed hdfs\ ranger\ kerberos manually, i found the hdfs fail to download policy from Ranger.
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null
Read this document: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy. I did not config this parameter policy.download.auth.users
But i did not know how to config this parameter -- policy.download.auth.users . In the internet, this parameter should config in custom repo file. But i did know the path the custom repo file?
Thank & Regards
luochong@gdbigdata.com
Re: Which file config policy.download.auth.users ?
Posted by Ramesh Mani <rm...@hortonworks.com>.
Luochong,
You can update this in the Ranger UI.
Open the HDFS service you created in ranger UI and in the config you will find “ Add new configuration”. Add these properties there.
Refer this for Screen shots, https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide?preview=/https%3A%2F%2Flh3.googleusercontent.com%2FFH8RmMq1pIX8w-_L3jqGMt9RtvqLjUH4Ywf68wMapfPWxytFdK8fIVfU7QDelFqC-6vBIqIONkIujEE7OPql-FQgeFmsW3wZSLQiRn5TQGVJWJ2EpevB36gBtUmATNTD1i5_gng
Thanks,
Ramesh
From: "luochong@gdbigdata.com<ma...@gdbigdata.com>" <lu...@gdbigdata.com>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Saturday, July 15, 2017 at 12:24 AM
To: user <us...@ranger.apache.org>>
Subject: Which file config policy.download.auth.users ?
Hi
Env:
Ranger version : 0.7.0
Hdfs : 2.7.0 with kerberos
After I installed hdfs\ ranger\ kerberos manually, i found the hdfs fail to download policy from Ranger.
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM<ma...@EXAMPLE.COM> (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null
Read this document: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy. I did not config this parameter policy.download.auth.users
But i did not know how to config this parameter -- policy.download.auth.users . In the internet, this parameter should config in custom repo file. But i did know the path the custom repo file?
Thank & Regards
________________________________
luochong@gdbigdata.com<ma...@gdbigdata.com>
Which file config policy.download.auth.users ?
Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.
Hi
Env:
Ranger version : 0.7.0
Hdfs : 2.7.0 with kerberos
After I installed hdfs\ ranger\ kerberos manually, i found the hdfs fail to download policy from Ranger.
In the namenode Log:
2017-07-15 07:28:37,556 WARN org.apache.ranger.admin.client.RangerAdminRESTClient: Error getting policies. secureMode=true, user=nn/admin141.example.com@EXAMPLE.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication Failed"}, serviceName=hadooopdev. ret = null
Read this document: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_command-line-upgrade/content/upgrade-ranger_23.html
In this file, there is one step:
For Download Policy to be successful, use the Ranger UI to update the service configuration with the following custom properties for each supported component:
policy.download.auth.users=<Component service user>
tag.download.auth.users=<Component service user>(if tag download)
So i think it is the reason that hdfs fail to download policy. I did not config this parameter policy.download.auth.users
But i did not know how to config this parameter -- policy.download.auth.users . In the internet, this parameter should config in custom repo file. But i did know the path the custom repo file?
Thank & Regards
luochong@gdbigdata.com
Re: Two question about ranger usersync
Posted by Bhavik Patel <bh...@gmail.com>.
Hi luochong,
Regarding, *Is there another way to sync user info automatically?*
Ranger-Usersync process syncs the added users in particular time interval
which is in minutes; By default it is set to 5 minutes.The property name is
"SYNC_INTERVAL". So it will sync newly added user after that interval or
you have restart ranger-usersync.
You can update this value in install.properties file and run the setup.sh
script to update the value in the Usersync process, after this you will
have to restart ranger-usersync process.
Regarding,
* How to remove the user info of ranger?* To delete the user from Ranger
database, you will have to delete the user from Ranger UI manually, as the
Ranger-Usersync process will only add users to Ranger Database.
Regard,
Bhavik Patel
+91-7208744109
On Thu, Jun 22, 2017 at 1:16 PM, luochong@gdbigdata.com <
luochong@gdbigdata.com> wrote:
> Hi,
> Now I sync the users of Linux OS to Ranger. But i meet two questions.
>
> Ranger Version : 0.7.0
>
> 1) Ranger could not sync OS user info .
> Steps:
> 1) Excecute the command : ranger-usersync start
> 2) then create one user in linux : useradd -d /usr/unix02 -m
> unix02
> 3) Check the ranger database.
> But I could not find this user -- unix02 in ranger database. Only
> restart the ranger-usersync , this record can be found.
> In the Internet, there is one record -- https://community.
> hortonworks.com/questions/21170/is-there-a-way-to-force-
> ranger-user-sync.html. In this, they suggested execute the command UnixUserGroupBuilder
> manually.
>
> My Question: Is there another way to sync user info automaticly?
>
> 2) Ranger could not remove the user
> Steps:
> 1) add the user in OS system
> 2) Ranger sync this user to ranger database after restart
> ranger-usersync
> 3) delete this user from OS system
> 4) restart the ranger-usersync.
>
> This deleted user is still in Ranger database,
>
> My question: How to remove the user info of ranger?
>
>
> Thanks & Regards
>
> ------------------------------
> luochong@gdbigdata.com
>
>