You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Matteo Alessandroni <ma...@tirasa.net> on 2019/01/28 16:17:37 UTC
Periodically refresh a service definition
Hi,
I have created a custom service definition that extends
"RangerServiceHdfs" and overridden the "getDefaultRangerPolicies()"
method so that every time the service definition is registered to Ranger
Admin a list of "RangerPolicy" is taken from an external REST service
and added to Ranger.
Would it be possible to periodically refresh a service definition? I
mean like automatically delete and re-register it?
Thanks!
Best regards,
Matteo
--
Dott. Matteo Alessandroni
Software Engineer @ Tirasa S.r.l.
Viale Vittoria Colonna, 97 - 65127 Pescara
Tel +39 0859116307 / FAX +39 0859111173
http://www.tirasa.net
Apache Syncope PMC Member
http://people.apache.org/phonebook.html?uid=skylark17
<http://people.apache.org/phonebook.html?uid=skylark17>
Tirasa S.r.l. <http://www.tirasa.net>
Re: Periodically refresh a service definition
Posted by Matteo Alessandroni <ma...@tirasa.net>.
On 29/01/19 12:10, Zs. wrote:
>
>
> On Tue, Jan 29, 2019 at 9:26 AM Matteo Alessandroni
> <matteo.alessandroni@tirasa.net
> <ma...@tirasa.net>> wrote:
>
> Hi Zsombor,
>
> On 29/01/19 00:09, Zs. wrote:
>> Hi,
>>
>> The getDefaultRangerPolicies is get called only when a new
>> service is created - so no need to re-register the definition,
>> just re-create the service, and your service will be called.
>
> yes thanks, but the service is created in the
> "getDefaultRangerPolicies()" logic when I register the service
> definition.
> Anyway, I could change this logic but then is there a way to
> configure Ranger to periodically refresh the service?
>
>
>
> The RangerService.getDefaultRangerPolicies gets called from
> ServiceDBStore.createDefaultPolicies, which gets called from
> ServiceDBStore.createService, not from ServiceDBStore.createService*Def.*
> The Ranger plugins periodically connect to the Admin webapp, to fetch
> the latest list of policies, not the other way around.
yes thanks I'm aware about that, but actually when I call:
curl -u admin:admin -X POST -H "Accept: application/json" -H
"Content-Type: application/json" -d @ranger-servicedef-hdfs_custom.json
http://localhost:6080/service/public/v2/api/*servicedef*
I see the logic in "getDefaultRangerPolicies()" is executed.
> It's unclear what you want to achieve. From your description, I
> thought, that you have an external service, which generates policies,
> what you would like to to apply to your HDFS cluster.
> If it's the case, then the simplest solution would be for your setup,
> is to push the newly generated policies through the REST interface to
> Ranger Admin.
Yes I have an external service like that and your solution is a good one
and it actually clear my doubts.
Anyway, just FYI, at the beginning I was trying to find a solution that
would have been pluggable in the Ranger plugins (now I'm working with
HDFS but I'll on other services too).
So a solution that does not require building any standalone application,
something like what I tried to do that is extending the HDFS Ranger
plugin by extending "RangerHdfsPlugin" [1] and override the
"isAccessAllowed()" method in order to add there the policies coming
from the external service [2].
This way this logic would have been plugged on the Ranger HDFS plugin.
But I had problems in extending the HDFS plugin according to the code
structure of the HDFS plugin. Also I could not understand what kind of
Java project I need to build in order to extend an existing Ranger
plugin (if anyone could advise on this it would be nice!),
I just found out how to install a new service definition in Ranger (by
adding my .jar plugin in e.g.
"/opt/ranger-1.2.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs"
and then register the service def via REST).
I actually would still prefer a solution like that, is there anything I
still can do to obtain that?
Regards,
Matteo
[1]
https://github.com/apache/ranger/blob/ranger-1.2/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java#L759
[2]
https://github.com/apache/ranger/blob/ranger-1.2/agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java#L337-L347
>
> Regards,
> Zsombor
>
>
>
>
>
>> However, why don't you just push the new policies from your
>> external services to Ranger admin?
>
> Well what is your idea to do that? You mean e.g. creating a
> standalone application that use Ranger REST API to create / update
> a service?
>
> Thanks.
> Regards,
> Matteo
>
>>
>> Regards,
>> Zsombor
>>
>>
>> On Mon, Jan 28, 2019 at 5:17 PM Matteo Alessandroni
>> <matteo.alessandroni@tirasa.net
>> <ma...@tirasa.net>> wrote:
>>
>> Hi,
>>
>> I have created a custom service definition that extends
>> "RangerServiceHdfs" and overridden the
>> "getDefaultRangerPolicies()" method so that every time the
>> service definition is registered to Ranger Admin a list of
>> "RangerPolicy" is taken from an external REST service and
>> added to Ranger.
>>
>> Would it be possible to periodically refresh a service
>> definition? I mean like automatically delete and re-register it?
>>
>> Thanks!
>> Best regards,
>> Matteo
>>
>>
>> --
>>
>> Dott. Matteo Alessandroni
>>
>> Software Engineer @ Tirasa S.r.l.
>>
>> Viale Vittoria Colonna, 97 - 65127 Pescara
>> Tel +39 0859116307 / FAX +39 0859111173
>>
>> http://www.tirasa.net
>>
>> Apache Syncope PMC Member
>> http://people.apache.org/phonebook.html?uid=skylark17
>> <http://people.apache.org/phonebook.html?uid=skylark17>
>>
>> Tirasa S.r.l. <http://www.tirasa.net>
>>
>
> --
>
> Dott. Matteo Alessandroni
>
> Software Engineer @ Tirasa S.r.l.
>
> Viale Vittoria Colonna, 97 - 65127 Pescara
> Tel +39 0859116307 / FAX +39 0859111173
>
> http://www.tirasa.net
>
> Apache Syncope PMC Member
> http://people.apache.org/phonebook.html?uid=skylark17
> <http://people.apache.org/phonebook.html?uid=skylark17>
>
> Tirasa S.r.l. <http://www.tirasa.net>
>
--
Dott. Matteo Alessandroni
Software Engineer @ Tirasa S.r.l.
Viale Vittoria Colonna, 97 - 65127 Pescara
Tel +39 0859116307 / FAX +39 0859111173
http://www.tirasa.net
Apache Syncope PMC Member
http://people.apache.org/phonebook.html?uid=skylark17
<http://people.apache.org/phonebook.html?uid=skylark17>
Tirasa S.r.l. <http://www.tirasa.net>
Re: Periodically refresh a service definition
Posted by "Zs." <gz...@gmail.com>.
On Tue, Jan 29, 2019 at 9:26 AM Matteo Alessandroni <
matteo.alessandroni@tirasa.net> wrote:
> Hi Zsombor,
>
> On 29/01/19 00:09, Zs. wrote:
>
> Hi,
>
> The getDefaultRangerPolicies is get called only when a new service is
> created - so no need to re-register the definition, just re-create the
> service, and your service will be called.
>
>
> yes thanks, but the service is created in the "getDefaultRangerPolicies()"
> logic when I register the service definition.
> Anyway, I could change this logic but then is there a way to configure
> Ranger to periodically refresh the service?
>
>
The RangerService.getDefaultRangerPolicies gets called from
ServiceDBStore.createDefaultPolicies, which gets called from
ServiceDBStore.createService, not from ServiceDBStore.createService*Def.*
The Ranger plugins periodically connect to the Admin webapp, to fetch the
latest list of policies, not the other way around.
It's unclear what you want to achieve. From your description, I thought,
that you have an external service, which generates policies, what you would
like to to apply to your HDFS cluster.
If it's the case, then the simplest solution would be for your setup, is to
push the newly generated policies through the REST interface to Ranger
Admin.
Regards,
Zsombor
> However, why don't you just push the new policies from your external
> services to Ranger admin?
>
>
> Well what is your idea to do that? You mean e.g. creating a standalone
> application that use Ranger REST API to create / update a service?
>
> Thanks.
> Regards,
> Matteo
>
>
> Regards,
> Zsombor
>
>
> On Mon, Jan 28, 2019 at 5:17 PM Matteo Alessandroni <
> matteo.alessandroni@tirasa.net> wrote:
>
>> Hi,
>>
>> I have created a custom service definition that extends
>> "RangerServiceHdfs" and overridden the "getDefaultRangerPolicies()" method
>> so that every time the service definition is registered to Ranger Admin a
>> list of "RangerPolicy" is taken from an external REST service and added to
>> Ranger.
>>
>> Would it be possible to periodically refresh a service definition? I mean
>> like automatically delete and re-register it?
>>
>> Thanks!
>> Best regards,
>> Matteo
>>
>>
>> --
>>
>> Dott. Matteo Alessandroni
>>
>> Software Engineer @ Tirasa S.r.l.
>>
>> Viale Vittoria Colonna, 97 - 65127 Pescara
>> Tel +39 0859116307 / FAX +39 0859111173
>> http://www.tirasa.net
>>
>> Apache Syncope PMC Member
>> http://people.apache.org/phonebook.html?uid=skylark17
>> [image: Tirasa S.r.l.] <http://www.tirasa.net>
>>
>
> --
>
> Dott. Matteo Alessandroni
>
> Software Engineer @ Tirasa S.r.l.
>
> Viale Vittoria Colonna, 97 - 65127 Pescara
> Tel +39 0859116307 / FAX +39 0859111173
> http://www.tirasa.net
>
> Apache Syncope PMC Member
> http://people.apache.org/phonebook.html?uid=skylark17
> [image: Tirasa S.r.l.] <http://www.tirasa.net>
>
Re: Periodically refresh a service definition
Posted by Matteo Alessandroni <ma...@tirasa.net>.
Hi Zsombor,
On 29/01/19 00:09, Zs. wrote:
> Hi,
>
> The getDefaultRangerPolicies is get called only when a new service is
> created - so no need to re-register the definition, just re-create the
> service, and your service will be called.
yes thanks, but the service is created in the
"getDefaultRangerPolicies()" logic when I register the service definition.
Anyway, I could change this logic but then is there a way to configure
Ranger to periodically refresh the service?
> However, why don't you just push the new policies from your external
> services to Ranger admin?
Well what is your idea to do that? You mean e.g. creating a standalone
application that use Ranger REST API to create / update a service?
Thanks.
Regards,
Matteo
>
> Regards,
> Zsombor
>
>
> On Mon, Jan 28, 2019 at 5:17 PM Matteo Alessandroni
> <matteo.alessandroni@tirasa.net
> <ma...@tirasa.net>> wrote:
>
> Hi,
>
> I have created a custom service definition that extends
> "RangerServiceHdfs" and overridden the
> "getDefaultRangerPolicies()" method so that every time the service
> definition is registered to Ranger Admin a list of "RangerPolicy"
> is taken from an external REST service and added to Ranger.
>
> Would it be possible to periodically refresh a service definition?
> I mean like automatically delete and re-register it?
>
> Thanks!
> Best regards,
> Matteo
>
>
> --
>
> Dott. Matteo Alessandroni
>
> Software Engineer @ Tirasa S.r.l.
>
> Viale Vittoria Colonna, 97 - 65127 Pescara
> Tel +39 0859116307 / FAX +39 0859111173
>
> http://www.tirasa.net
>
> Apache Syncope PMC Member
> http://people.apache.org/phonebook.html?uid=skylark17
> <http://people.apache.org/phonebook.html?uid=skylark17>
>
> Tirasa S.r.l. <http://www.tirasa.net>
>
--
Dott. Matteo Alessandroni
Software Engineer @ Tirasa S.r.l.
Viale Vittoria Colonna, 97 - 65127 Pescara
Tel +39 0859116307 / FAX +39 0859111173
http://www.tirasa.net
Apache Syncope PMC Member
http://people.apache.org/phonebook.html?uid=skylark17
<http://people.apache.org/phonebook.html?uid=skylark17>
Tirasa S.r.l. <http://www.tirasa.net>
Re: Periodically refresh a service definition
Posted by "Zs." <gz...@gmail.com>.
Hi,
The getDefaultRangerPolicies is get called only when a new service is
created - so no need to re-register the definition, just re-create the
service, and your service will be called.
However, why don't you just push the new policies from your external
services to Ranger admin?
Regards,
Zsombor
On Mon, Jan 28, 2019 at 5:17 PM Matteo Alessandroni <
matteo.alessandroni@tirasa.net> wrote:
> Hi,
>
> I have created a custom service definition that extends
> "RangerServiceHdfs" and overridden the "getDefaultRangerPolicies()" method
> so that every time the service definition is registered to Ranger Admin a
> list of "RangerPolicy" is taken from an external REST service and added to
> Ranger.
>
> Would it be possible to periodically refresh a service definition? I mean
> like automatically delete and re-register it?
>
> Thanks!
> Best regards,
> Matteo
>
>
> --
>
> Dott. Matteo Alessandroni
>
> Software Engineer @ Tirasa S.r.l.
>
> Viale Vittoria Colonna, 97 - 65127 Pescara
> Tel +39 0859116307 / FAX +39 0859111173
> http://www.tirasa.net
>
> Apache Syncope PMC Member
> http://people.apache.org/phonebook.html?uid=skylark17
> [image: Tirasa S.r.l.] <http://www.tirasa.net>
>