You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by rohit sinha <ta...@gmail.com> on 2016/04/22 00:03:12 UTC
TagSync without Atlas
Hello,
I saw this Jira: https://issues.apache.org/jira/browse/RANGER-942
It looks like Ranger supports TagSync without Atlas. Is it possible to use
Kafka as a source or HBase tables ? Can someone point me to some related
documentation on this.
Thanks,
Rohit Sinha
Re: TagSync without Atlas
Posted by rohit sinha <ta...@gmail.com>.
Hello Abhay,
Thanks for your response. I have opened a jira for this feature here:
https://issues.apache.org/jira/browse/RANGER-979
I will really appreciate if you can add/point out any details which are
missing from the issue.
I will look into the details you have provided for the approach.
Thanks,
Rohit Sinha
On Thu, Apr 28, 2016 at 3:37 PM, Abhay Kulkarni <ak...@hortonworks.com>
wrote:
> Rohit,
>
> Here are the details on adding a tag-source implementation to
> Ranger TagSync.
>
> 1. Extend AbstractTagSource class to implement your tag-source.
> Please refer to following existing implementations for further
> details:
> - AtlasTagSource: processes notifications from Atlas
> - FileTagSource: processes tags specified in a file
> - AtlasRESTTagSource: periodically pulls tags from Atlas
>
> 2. Configure Ranger TagSync to use your tag-source implementation,
> let¹s call it ³mytagsource", by adding the following properties
> to ranger-tagsync-site.xml
> ranger.tagsync.source.mytagsource=true
> ranger.tagsync.source.mytagsource.class=<tag-source-class-name>
>
> 3. Make the tag-source implementation class and its dependent
> libraries, if any, available to Ranger TagSync, by copying them
> to <install_dir>/lib directory.
>
> 4. Restart Ranger TagSync.
>
> Here is some more detail on Ranger TagSync startup and how it uses
> tag-sources:
>
>
> - During startup, Ranger Tagsync process instantiates all tag-source
> Implementations
> - on each tag-source object
> - initialize() is called with properties in ranger-tagsync-site.xml
> - start() is called on each tag-source. Tag source implementation
> should quickly return after starting the source perhaps in a
> separate thread. When tags need to be sent to ranger-admin, it
> needs
> to create ServiceTags object and call updateSink() with it.
> - A sample ServiceTags, in JSON, is provided in
> <install_dir>/conf.dist/etc/ranger/data/tags.json.
> In particular, each entry in ServiceTags.resourceToTagIds element
> maps id of a ServiceResource specified in ServiceTags.serviceResources
> to an key of a tag specified in the ServiceTags.tags map.
>
>
> Hope this helps.
>
> Thanks,
> Abhay
>
>
> On 4/21/16, 3:27 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Thanks for the prompt reply Madhan.
> >
> >I will do that. Since, Ranger does work with file I am guessing the source
> >does not push the tags to Ranger and Ranger polls/pulls the tags
> >periodically. Is there any documentation on how to add TagSync sources or
> >any other relevant information which can be helpful in understanding the
> >architecture with the possibility of adding a new one.
> >
> >Thanks.
> >
> >Thanks,
> >Rohit Sinha
> >
> >
> >On Thu, Apr 21, 2016 at 3:16 PM, Madhan Neethiraj <ma...@apache.org>
> >wrote:
> >
> >> Rohit,
> >>
> >> Ranger TagSync supports the following as source for tags:
> >> - Apache Atlas
> >> - File
> >>
> >> If you are looking for support for other sources, like Kafka or HBase
> >> tables, can you please file a JIRA with details of the integration? It
> >>will
> >> be helpful to prioritize this enhancement.
> >>
> >> Thanks,
> >> Madhan
> >>
> >>
> >>
> >>
> >> On 4/21/16, 3:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
> >>
> >> >Hello,
> >> >I saw this Jira: https://issues.apache.org/jira/browse/RANGER-942
> >> >
> >> >It looks like Ranger supports TagSync without Atlas. Is it possible to
> >>use
> >> >Kafka as a source or HBase tables ? Can someone point me to some
> >>related
> >> >documentation on this.
> >> >
> >> >
> >> >Thanks,
> >> >Rohit Sinha
> >>
> >>
>
>
Re: TagSync without Atlas
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
Rohit,
Here are the details on adding a tag-source implementation to
Ranger TagSync.
1. Extend AbstractTagSource class to implement your tag-source.
Please refer to following existing implementations for further
details:
- AtlasTagSource: processes notifications from Atlas
- FileTagSource: processes tags specified in a file
- AtlasRESTTagSource: periodically pulls tags from Atlas
2. Configure Ranger TagSync to use your tag-source implementation,
let¹s call it ³mytagsource", by adding the following properties
to ranger-tagsync-site.xml
ranger.tagsync.source.mytagsource=true
ranger.tagsync.source.mytagsource.class=<tag-source-class-name>
3. Make the tag-source implementation class and its dependent
libraries, if any, available to Ranger TagSync, by copying them
to <install_dir>/lib directory.
4. Restart Ranger TagSync.
Here is some more detail on Ranger TagSync startup and how it uses
tag-sources:
- During startup, Ranger Tagsync process instantiates all tag-source
Implementations
- on each tag-source object
- initialize() is called with properties in ranger-tagsync-site.xml
- start() is called on each tag-source. Tag source implementation
should quickly return after starting the source perhaps in a
separate thread. When tags need to be sent to ranger-admin, it
needs
to create ServiceTags object and call updateSink() with it.
- A sample ServiceTags, in JSON, is provided in
<install_dir>/conf.dist/etc/ranger/data/tags.json.
In particular, each entry in ServiceTags.resourceToTagIds element
maps id of a ServiceResource specified in ServiceTags.serviceResources
to an key of a tag specified in the ServiceTags.tags map.
Hope this helps.
Thanks,
Abhay
On 4/21/16, 3:27 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Thanks for the prompt reply Madhan.
>
>I will do that. Since, Ranger does work with file I am guessing the source
>does not push the tags to Ranger and Ranger polls/pulls the tags
>periodically. Is there any documentation on how to add TagSync sources or
>any other relevant information which can be helpful in understanding the
>architecture with the possibility of adding a new one.
>
>Thanks.
>
>Thanks,
>Rohit Sinha
>
>
>On Thu, Apr 21, 2016 at 3:16 PM, Madhan Neethiraj <ma...@apache.org>
>wrote:
>
>> Rohit,
>>
>> Ranger TagSync supports the following as source for tags:
>> - Apache Atlas
>> - File
>>
>> If you are looking for support for other sources, like Kafka or HBase
>> tables, can you please file a JIRA with details of the integration? It
>>will
>> be helpful to prioritize this enhancement.
>>
>> Thanks,
>> Madhan
>>
>>
>>
>>
>> On 4/21/16, 3:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
>>
>> >Hello,
>> >I saw this Jira: https://issues.apache.org/jira/browse/RANGER-942
>> >
>> >It looks like Ranger supports TagSync without Atlas. Is it possible to
>>use
>> >Kafka as a source or HBase tables ? Can someone point me to some
>>related
>> >documentation on this.
>> >
>> >
>> >Thanks,
>> >Rohit Sinha
>>
>>
Re: TagSync without Atlas
Posted by rohit sinha <ta...@gmail.com>.
Thanks for the prompt reply Madhan.
I will do that. Since, Ranger does work with file I am guessing the source
does not push the tags to Ranger and Ranger polls/pulls the tags
periodically. Is there any documentation on how to add TagSync sources or
any other relevant information which can be helpful in understanding the
architecture with the possibility of adding a new one.
Thanks.
Thanks,
Rohit Sinha
On Thu, Apr 21, 2016 at 3:16 PM, Madhan Neethiraj <ma...@apache.org> wrote:
> Rohit,
>
> Ranger TagSync supports the following as source for tags:
> - Apache Atlas
> - File
>
> If you are looking for support for other sources, like Kafka or HBase
> tables, can you please file a JIRA with details of the integration? It will
> be helpful to prioritize this enhancement.
>
> Thanks,
> Madhan
>
>
>
>
> On 4/21/16, 3:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
>
> >Hello,
> >I saw this Jira: https://issues.apache.org/jira/browse/RANGER-942
> >
> >It looks like Ranger supports TagSync without Atlas. Is it possible to use
> >Kafka as a source or HBase tables ? Can someone point me to some related
> >documentation on this.
> >
> >
> >Thanks,
> >Rohit Sinha
>
>
Re: TagSync without Atlas
Posted by Madhan Neethiraj <ma...@apache.org>.
Rohit,
Ranger TagSync supports the following as source for tags:
- Apache Atlas
- File
If you are looking for support for other sources, like Kafka or HBase tables, can you please file a JIRA with details of the integration? It will be helpful to prioritize this enhancement.
Thanks,
Madhan
On 4/21/16, 3:03 PM, "rohit sinha" <ta...@gmail.com> wrote:
>Hello,
>I saw this Jira: https://issues.apache.org/jira/browse/RANGER-942
>
>It looks like Ranger supports TagSync without Atlas. Is it possible to use
>Kafka as a source or HBase tables ? Can someone point me to some related
>documentation on this.
>
>
>Thanks,
>Rohit Sinha