You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Lawrence Weikum <lw...@pandora.com> on 2016/05/06 19:20:36 UTC
Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Don Bosco Durai <bo...@apache.org>.
From your previous email, it seems you are getting audit records. So did your installation go through?
Thanks
Bosco
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Friday, May 13, 2016 at 8:14 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
That is correct. Kafka 0.9.0.1
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Don Bosco Durai <bo...@apache.org>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 13, 2016 at 1:43 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Ok, manual install might be a bit tricky. I assume, you are using 0.9 version of Kafka.
Thanks
Bosco
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, May 9, 2016 at 7:25 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Manual install
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Don Bosco Durai <bo...@apache.org>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 5:16 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Lawerence, also are you using Ambari or manual install?
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 2:04 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Lawerence,
Were are the ranger-kafka-plugin-impl folder and ranger-plugin-classloader*jar/ ranger-kafka-plugin-shim*jar are locate ? We expect them to be in the same folder when is kafka “libs” folder.
If ranger-kafka-plugin-impl is symlink can you please make it as local one and try?
Thanks
Ramesh
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 1:00 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Lawrence Weikum <lw...@pandora.com>.
That is correct. Kafka 0.9.0.1
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Don Bosco Durai <bo...@apache.org>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 13, 2016 at 1:43 AM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Ok, manual install might be a bit tricky. I assume, you are using 0.9 version of Kafka.
Thanks
Bosco
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Monday, May 9, 2016 at 7:25 AM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Manual install
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com<ma...@pandora.com>
From: Don Bosco Durai <bo...@apache.org>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 5:16 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Lawerence, also are you using Ambari or manual install?
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 2:04 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Lawerence,
Were are the ranger-kafka-plugin-impl folder and ranger-plugin-classloader*jar/ ranger-kafka-plugin-shim*jar are locate ? We expect them to be in the same folder when is kafka “libs” folder.
If ranger-kafka-plugin-impl is symlink can you please make it as local one and try?
Thanks
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:00 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com<ma...@pandora.com>
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Don Bosco Durai <bo...@apache.org>.
Ok, manual install might be a bit tricky. I assume, you are using 0.9 version of Kafka.
Thanks
Bosco
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, May 9, 2016 at 7:25 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Manual install
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Don Bosco Durai <bo...@apache.org>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 5:16 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Lawerence, also are you using Ambari or manual install?
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 2:04 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Lawerence,
Were are the ranger-kafka-plugin-impl folder and ranger-plugin-classloader*jar/ ranger-kafka-plugin-shim*jar are locate ? We expect them to be in the same folder when is kafka “libs” folder.
If ranger-kafka-plugin-impl is symlink can you please make it as local one and try?
Thanks
Ramesh
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 1:00 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Lawrence Weikum <lw...@pandora.com>.
Manual install
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Don Bosco Durai <bo...@apache.org>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 5:16 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Lawerence, also are you using Ambari or manual install?
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 2:04 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Lawerence,
Were are the ranger-kafka-plugin-impl folder and ranger-plugin-classloader*jar/ ranger-kafka-plugin-shim*jar are locate ? We expect them to be in the same folder when is kafka “libs” folder.
If ranger-kafka-plugin-impl is symlink can you please make it as local one and try?
Thanks
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:00 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com<ma...@pandora.com>
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Don Bosco Durai <bo...@apache.org>.
Lawerence, also are you using Ambari or manual install?
Thanks
Bosco
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 2:04 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Lawerence,
Were are the ranger-kafka-plugin-impl folder and ranger-plugin-classloader*jar/ ranger-kafka-plugin-shim*jar are locate ? We expect them to be in the same folder when is kafka “libs” folder.
If ranger-kafka-plugin-impl is symlink can you please make it as local one and try?
Thanks
Ramesh
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 1:00 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Ramesh Mani <rm...@hortonworks.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Ramesh Mani <rm...@hortonworks.com>.
Lawerence,
Were are the ranger-kafka-plugin-impl folder and ranger-plugin-classloader*jar/ ranger-kafka-plugin-shim*jar are locate ? We expect them to be in the same folder when is kafka "libs" folder.
If ranger-kafka-plugin-impl is symlink can you please make it as local one and try?
Thanks
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:00 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com<ma...@pandora.com>
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Lawrence Weikum <lw...@pandora.com>.
Even with full permissions, still no luck.
It seems that the RangerConfiguration.getFileLocation() always returns a null value for any ranger-related configurations.
Lawrence Weikum | Software Engineer | Pandora
1426 Pearl Street, Suite 100, Boulder CO 80302
m 720.203.1578 | lweikum@pandora.com
From: Ramesh Mani <rm...@hortonworks.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 1:29 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Re: Loading config error
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence
Re: Loading config error
Posted by Ramesh Mani <rm...@hortonworks.com>.
From the log what I might be happening is for some reason the ranger*.xml files in the config folders not available, please check the permission on the files and the folders for this.
Also you can check in the xml files the values are correct.
Thanks,
Ramesh
From: Lawrence Weikum <lw...@pandora.com>>
Reply-To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Date: Friday, May 6, 2016 at 12:20 PM
To: "user@ranger.incubator.apache.org<ma...@ranger.incubator.apache.org>" <us...@ranger.incubator.apache.org>>
Subject: Loading config error
Hello,
I'm having a bit of trouble getting the Kafka Ranger 0.5 pluigin to load the configuration files. My guess is that I have the config files in the wrong place.
I know normally, Ranger and Kafka are supposed to be installed natively on the machine, but we have a use case where they must be used with relative paths.
Currently, the jars are linked correctly and I can see in the logs that the plugin is starting. I have the xml config files in Kafka's config directory.
With Log4J, I see:
[2016-05-06 13:03:01,737] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:73)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:205)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:175)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:154)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2016-05-06 11:23:31,056] DEBUG ==> addResourceIfReadable(ranger-kafka-security.xml) (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2016-05-06 11:23:31,056] DEBUG ==> RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingChildClassLoader(): calling childClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResourcesUsingComponentClassLoader(): sun.misc.CompoundEnumeration@47c81abf (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,056] DEBUG <== RangerPluginClassLoader.findResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG ==> RangerPluginClassLoader.findResource(/ranger-kafka-security.xml) (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): calling componentClassLoader.getResources() (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== RangerPluginClassLoader.findResource(/ranger-kafka-security.xml): null (org.apache.ranger.plugin.classloader.RangerPluginClassLoader)
[2016-05-06 11:23:31,058] DEBUG <== addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
Do I have these configs in the wrong location, or is there something else I'm missing?
Thank you for your help!
Lawrence