You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Deepak Mahato (Jira)" <ji...@apache.org> on 2021/06/04 12:34:00 UTC
[jira] [Commented] (RANGER-3099) Ranger hdfs policies not syncing
automatically
[ https://issues.apache.org/jira/browse/RANGER-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357334#comment-17357334 ]
Deepak Mahato commented on RANGER-3099:
---------------------------------------
[~jasonmadam] [~anoopkumarkm]
is the issue resolved for you? I am also trying to integrate Apache Ranger 2.1 with EMR 5.33 I am able to sync the policies for hive but getting errors for s3. Below is the error message I am getting. FYI- I have changed the secureMode=true in the config file but still getting the error.
Below is the log from emrsecretagent.log
{code:java}
2021-06-04 12:19:31,751 WARN Thread-4: Error getting policies. secureMode=false, user=emrsecretagent (auth:SIMPLE), response={"httpStatusCode":400,"statusCode":0}, serviceName=amazonemrs3
2021-06-04 12:19:31,751 WARN Thread-4: cache file does not exist or not readable '/emr/secretagent/ranger_policy_cache/emrS3RangerPlugin_amazonemrs3.json'
{code}
below is the rangeradmin.log
{code:java}
2021-06-04 12:30:04,817 [https-jsse-nio-6182-exec-1] DEBUG org.apache.ranger.biz.AssetMgr (AssetMgr.java:838) - Creating RangerPluginInfo record for service-version
2021-06-04 12:30:04,819 [https-jsse-nio-6182-exec-1] DEBUG org.apache.ranger.biz.AssetMgr (AssetMgr.java:790) - <== createOrUpdatePluginInfo(pluginInfo = RangerPluginInfo={id={null} createTime={null} updateTime={null} serviceName={amazonemrs3} serviceType={null} serviceTypeDisplayName{null} serviceDisplayName={null} hostName={ip-10-224-218-88} appType={emrS3RangerPlugin} ipAddress={10.224.218.88} info={{pluginCapabilities=0, policyDownloadTime=1622809804815, adminCapabilities=fff, policyActiveVersion=-1, policyActivationTime=0, policyDownloadedVersion=null}} }, isPolicyDownloadRequest = true, httpCode = 400)
2021-06-04 12:30:04,819 [https-jsse-nio-6182-exec-1] INFO org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:345) - Request failed. loginId=null, logMessage=Unauthenticated access not allowed
javax.ws.rs.WebApplicationException
at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:337){code}
Not sure what I am missing but any input will be of great help.
> Ranger hdfs policies not syncing automatically
> ----------------------------------------------
>
> Key: RANGER-3099
> URL: https://issues.apache.org/jira/browse/RANGER-3099
> Project: Ranger
> Issue Type: Bug
> Components: plugins, Ranger
> Affects Versions: 2.1.0
> Environment: AWS EMR, WIndows AD
> Reporter: Anoop Kumar K M
> Priority: Blocker
>
> Hi,
> We are trying to implement Ranger 2 .1.0 on top of AWS EMR 6.1.0.
> EMR 6.1.0 has hadoop 3. The cluster is Kerberos enabled.
> I have installed ranger in a separate ec2 machine and able to install hdfs plugin in EMR.
> But the problem is that for policies to be applied, both ranger server and hdfs namenode should be restarted . After I restart both the policies becomes effective
> Ranger admin logs shows below error.
> ==========
> 2020-11-30 10:57:42,397 [http-bio-6080-exec-9] INFO org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:345) - Request failed. loginId=null, logMessage=Unauthenticated access not allowed javax.ws.rs.WebApplicationException at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:337) =========
>
> Namenode logs show below error.
> ==========
>
> 2020-12-02 13:32:53,863 ERROR org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Error getting Roles; service not found. secureMode=false, user=hdfs/ip-10-98-84-189.eu-west-1.compute.internal@EU-WEST-1.COMPUTE.INTERNAL (auth:KERBEROS), response=404, serviceName=hadoopdev, lastKnownRoleVersion=-1, lastActivationTimeInMillis=1606746562885
>
> 2020-12-02 13:32:53,863 WARN org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Received 404 error code with body:[null], Ignoring
> 2020-12-02 13:32:53,863 INFO org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Skip Securetrue
> 2020-12-02 13:32:53,869 WARN org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Error getting policies. secureMode=false, user=hdfs/ip-10-98-84-189.eu-west-1.compute.internal@EU-WEST-1.COMPUTE.INTERNAL (auth:KERBEROS), response=\{"httpStatusCode":400,"statusCode":0}, serviceName=hadoopdev
> ==========
>
> Under kerberos config in install.properties of ranger I have the below settings
>
> --------------Kerberos Config -----------------
> spnego_principal=HTTP/ip-10-98-84-189.eu-west-1.compute.internal@EU-WEST-1.COMPUTE.INTERNAL
> spnego_keytab=/etc/security/keytabs/spnego.keytab
> token_valid=30
> cookie_domain=ip-10-98-84-189.eu-west-1.compute.internal
> cookie_path=/
> admin_principal=rangeradmin/ip-10-98-84-189.eu-west-1.compute.internal@EU-WEST-1.COMPUTE.INTERNAL
> admin_keytab=/etc/security/keytabs/rangeradmin.keytab
> lookup_principal=rangerlookup/ip-10-98-84-189.eu-west-1.compute.internal@EU-WEST-1.COMPUTE.INTERNAL
> lookup_keytab=/etc/security/keytabs/rangerlookup.keytab
> hadoop_conf=/etc/hadoop/conf
>
> In the ranger console for the service config I have given below property
>
> [policy.download.auth.users = hdfs@EU-WEST-1.COMPUTE.INTERNAL|mailto:policy.download.auth.users=hdfs@EU-WEST-1.COMPUTE.INTERNAL]
>
> Not sure what I am missing. Any input in this will be a great help
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)