You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Matteo Alessandroni <ma...@tirasa.net> on 2019/01/28 12:30:52 UTC
Extend Ranger specific service logic
Hi,
I'm trying to build a project to extend the "RangerServiceHdfs" class
because I want to override the "getDefaultRangerPolicies()" method [1]:
public class CustomRangerServiceHdfs extends RangerServiceHdfs {
...
This way I would like to use the default HDFS Ranger plugin on my Hadoop
instance but use my custom class as HDFS Ranger service "implClass".
To do that I have created a maven project to produce a .jar file, I have
added:
<parent>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger</artifactId>
<version>1.2.0</version>
<relativePath>..</relativePath>
</parent>
in the "pom.xml" and the following dependencies:
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>${commons.lang.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
</dependency>
<dependency>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-plugins-common</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-plugins-audit</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-plugins-cred</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.ranger</groupId>
<artifactId>ranger-hdfs-plugin</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>${commons.configuration.version}</version>
</dependency>
then, I have built it and copied to
"/opt/ranger-1.2.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs_custom".
Then I have registered my custom service definition by running:
curl -u admin:admin -X POST -H "Accept: application/json" -H
"Content-Type: application/json" -d @ranger-servicedef-hdfs_custom.json
http://localhost:6080/service/plugins/definitions
where "ranger-servicedef-hdfs_custom.json" contains:
{
"name": "hdfs_custom",
"implClass": "org.apache.ranger.services.hdfs.CustomRangerServiceHdfs",
...
At last I have restarted the Ranger Admin service,
The problem is that when I try to add a new service to that service
definition I get the followin error:
{"statusCode":1,"msgDesc":"service does not exist - name=null"}
(it also happens when adding it from Ranger Admin Console).
This is the first part of the stack trace:
ERROR org.apache.ranger.rest.ServiceREST (ServiceREST.java:672) -
createService(RangerService={id={null} guid={null} isEnabled={true}
createdBy={null} updatedBy={null} createTime={null} updateTime={null}
version={1} name={HDFSTest} type={hdfs_custom} description={null}
tagService={null} configs={password={admin}
hadoop.security.authentication={simple}
hadoop.rpc.protection={authentication}
fs.default.name={hdfs://hadoop-namenode:8020}
hadoop.security.authorization={false} username={admin} }
policyVersion={null} policyUpdateTime={null} tagVersion={1}
tagUpdateTime={null} }) failed
java.lang.Exception: service does not exist - name=null
at
org.apache.ranger.biz.ServiceDBStore.createPolicy(ServiceDBStore.java:1863)
at
org.apache.ranger.biz.ServiceDBStore.createDefaultPolicies(ServiceDBStore.java:2690)
at
org.apache.ranger.biz.ServiceDBStore.createService(ServiceDBStore.java:1500)
at
org.apache.ranger.rest.ServiceREST.createService(ServiceREST.java:668)
at
org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
...
I need to add that service definition and also a related service called
"HDFSTest" because it will be used as "REPOSITORY_NAME" in the
"install.properties" file of the Ranger HDFS Plugin (the default one) I
have installed on my Hadoop instance.
Could you please help me with this error?
Thanks
Best regards,
Matteo
[1]
https://github.com/apache/ranger/blob/master/hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/RangerServiceHdfs.java#L108
--
Dott. Matteo Alessandroni
Software Engineer @ Tirasa S.r.l.
Viale Vittoria Colonna, 97 - 65127 Pescara
Tel +39 0859116307 / FAX +39 0859111173
http://www.tirasa.net
Apache Syncope PMC Member
http://people.apache.org/phonebook.html?uid=skylark17
<http://people.apache.org/phonebook.html?uid=skylark17>
Tirasa S.r.l. <http://www.tirasa.net>
Re: Extend Ranger specific service logic
Posted by Matteo Alessandroni <ma...@tirasa.net>.
Hi,
just solved the issue. It was a problem on the custom logic I had added
in the "getDefaultRangerPolicies()" method.
Just fixed it and now it works!
Thanks anyway!
Regards,
Matteo
On 28/01/19 13:30, Matteo Alessandroni wrote:
> Hi,
>
> I'm trying to build a project to extend the "RangerServiceHdfs" class
> because I want to override the "getDefaultRangerPolicies()" method [1]:
>
> public class CustomRangerServiceHdfs extends RangerServiceHdfs {
> ...
>
> This way I would like to use the default HDFS Ranger plugin on my
> Hadoop instance but use my custom class as HDFS Ranger service
> "implClass".
>
> To do that I have created a maven project to produce a .jar file, I
> have added:
>
> <parent>
> <groupId>org.apache.ranger</groupId>
> <artifactId>ranger</artifactId>
> <version>1.2.0</version>
> <relativePath>..</relativePath>
> </parent>
>
> in the "pom.xml" and the following dependencies:
>
> <dependency>
> <groupId>commons-lang</groupId>
> <artifactId>commons-lang</artifactId>
> <version>${commons.lang.version}</version>
> </dependency>
> <dependency>
> <groupId>com.google.code.gson</groupId>
> <artifactId>gson</artifactId>
> </dependency>
>
> <dependency>
> <groupId>org.apache.ranger</groupId>
> <artifactId>ranger-plugins-common</artifactId>
> <version>${project.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.ranger</groupId>
> <artifactId>ranger-plugins-audit</artifactId>
> <version>${project.version}</version>
> </dependency>
> <dependency>
> <groupId>org.apache.ranger</groupId>
> <artifactId>ranger-plugins-cred</artifactId>
> <version>${project.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-common</artifactId>
> <version>${hadoop.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.ranger</groupId>
> <artifactId>ranger-hdfs-plugin</artifactId>
> <version>${project.version}</version>
> </dependency>
>
> <dependency>
> <groupId>commons-configuration</groupId>
> <artifactId>commons-configuration</artifactId>
> <version>${commons.configuration.version}</version>
> </dependency>
>
> then, I have built it and copied to
> "/opt/ranger-1.2.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs_custom".
> Then I have registered my custom service definition by running:
>
> curl -u admin:admin -X POST -H "Accept: application/json" -H
> "Content-Type: application/json" -d
> @ranger-servicedef-hdfs_custom.json
> http://localhost:6080/service/plugins/definitions
>
> where "ranger-servicedef-hdfs_custom.json" contains:
>
> {
> "name": "hdfs_custom",
> "implClass": "org.apache.ranger.services.hdfs.CustomRangerServiceHdfs",
> ...
>
> At last I have restarted the Ranger Admin service,
> The problem is that when I try to add a new service to that service
> definition I get the followin error:
>
> {"statusCode":1,"msgDesc":"service does not exist - name=null"}
>
> (it also happens when adding it from Ranger Admin Console).
> This is the first part of the stack trace:
>
> ERROR org.apache.ranger.rest.ServiceREST (ServiceREST.java:672) -
> createService(RangerService={id={null} guid={null} isEnabled={true}
> createdBy={null} updatedBy={null} createTime={null} updateTime={null}
> version={1} name={HDFSTest} type={hdfs_custom} description={null}
> tagService={null} configs={password={admin}
> hadoop.security.authentication={simple}
> hadoop.rpc.protection={authentication}
> fs.default.name={hdfs://hadoop-namenode:8020}
> hadoop.security.authorization={false} username={admin} }
> policyVersion={null} policyUpdateTime={null} tagVersion={1}
> tagUpdateTime={null} }) failed
> java.lang.Exception: service does not exist - name=null
> at
> org.apache.ranger.biz.ServiceDBStore.createPolicy(ServiceDBStore.java:1863)
> at
> org.apache.ranger.biz.ServiceDBStore.createDefaultPolicies(ServiceDBStore.java:2690)
> at
> org.apache.ranger.biz.ServiceDBStore.createService(ServiceDBStore.java:1500)
> at
> org.apache.ranger.rest.ServiceREST.createService(ServiceREST.java:668)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ...
>
> I need to add that service definition and also a related service
> called "HDFSTest" because it will be used as "REPOSITORY_NAME" in the
> "install.properties" file of the Ranger HDFS Plugin (the default one)
> I have installed on my Hadoop instance.
>
> Could you please help me with this error?
>
> Thanks
> Best regards,
> Matteo
>
>
> [1]
> https://github.com/apache/ranger/blob/master/hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/RangerServiceHdfs.java#L108
>
> --
>
> Dott. Matteo Alessandroni
>
> Software Engineer @ Tirasa S.r.l.
>
> Viale Vittoria Colonna, 97 - 65127 Pescara
> Tel +39 0859116307 / FAX +39 0859111173
>
> http://www.tirasa.net
>
> Apache Syncope PMC Member
> http://people.apache.org/phonebook.html?uid=skylark17
> <http://people.apache.org/phonebook.html?uid=skylark17>
>
> Tirasa S.r.l. <http://www.tirasa.net>
--
Dott. Matteo Alessandroni
Software Engineer @ Tirasa S.r.l.
Viale Vittoria Colonna, 97 - 65127 Pescara
Tel +39 0859116307 / FAX +39 0859111173
http://www.tirasa.net
Apache Syncope PMC Member
http://people.apache.org/phonebook.html?uid=skylark17
<http://people.apache.org/phonebook.html?uid=skylark17>
Tirasa S.r.l. <http://www.tirasa.net>