You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2016/07/07 07:03:05 UTC
Review Request 49743: tagsync should attempt re-initializing failed
tag sources periodically
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49743/
-----------------------------------------------------------
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Bugs: RANGER-1079
https://issues.apache.org/jira/browse/RANGER-1079
Repository: ranger
Description
-------
tag sync process, during startup, initializes tag sources only once and ignores those which failed initialization. This causes a tag source to be ignored even if its initialization failed due to a transient error. Tag sync needs to be made more robust by periodically attempting re-initialization of failed tag sources and recover from transient errors during initialization.
Diffs
-----
tagsync/src/main/java/org/apache/ranger/tagsync/model/AbstractTagSource.java d46170a
tagsync/src/main/java/org/apache/ranger/tagsync/model/TagSource.java 5ef6c57
tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java b1b17cd
Diff: https://reviews.apache.org/r/49743/diff/
Testing
-------
Tested in a single node cluster with tag source as 'Atlas' and with zookeeper stopped. Tag-source initialization kept failing until zookeeper was started. Then tag-synchronizer started receiving tag events from Atlas.
Thanks,
Abhay Kulkarni
Re: Review Request 49743: tagsync should attempt re-initializing
failed tag sources periodically
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49743/#review141186
-----------------------------------------------------------
Fix it, then Ship it!
tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java (line 41)
<https://reviews.apache.org/r/49743/#comment206608>
Consider reading this value from a property.
tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java (line 305)
<https://reviews.apache.org/r/49743/#comment206607>
instead of removing from failedTagSources here, wouldn't remove at #289 be simpler? That also would eliminate the need for successfullyReinitialized list. Please review.
- Madhan Neethiraj
On July 7, 2016, 7:03 a.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49743/
> -----------------------------------------------------------
>
> (Updated July 7, 2016, 7:03 a.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1079
> https://issues.apache.org/jira/browse/RANGER-1079
>
>
> Repository: ranger
>
>
> Description
> -------
>
> tag sync process, during startup, initializes tag sources only once and ignores those which failed initialization. This causes a tag source to be ignored even if its initialization failed due to a transient error. Tag sync needs to be made more robust by periodically attempting re-initialization of failed tag sources and recover from transient errors during initialization.
>
>
> Diffs
> -----
>
> tagsync/src/main/java/org/apache/ranger/tagsync/model/AbstractTagSource.java d46170a
> tagsync/src/main/java/org/apache/ranger/tagsync/model/TagSource.java 5ef6c57
> tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java b1b17cd
>
> Diff: https://reviews.apache.org/r/49743/diff/
>
>
> Testing
> -------
>
> Tested in a single node cluster with tag source as 'Atlas' and with zookeeper stopped. Tag-source initialization kept failing until zookeeper was started. Then tag-synchronizer started receiving tag events from Atlas.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 49743: tagsync should attempt re-initializing
failed tag sources periodically
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49743/#review141245
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On July 8, 2016, 4:05 a.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49743/
> -----------------------------------------------------------
>
> (Updated July 8, 2016, 4:05 a.m.)
>
>
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1079
> https://issues.apache.org/jira/browse/RANGER-1079
>
>
> Repository: ranger
>
>
> Description
> -------
>
> tag sync process, during startup, initializes tag sources only once and ignores those which failed initialization. This causes a tag source to be ignored even if its initialization failed due to a transient error. Tag sync needs to be made more robust by periodically attempting re-initialization of failed tag sources and recover from transient errors during initialization.
>
>
> Diffs
> -----
>
> tagsync/src/main/java/org/apache/ranger/tagsync/model/AbstractTagSource.java d46170a
> tagsync/src/main/java/org/apache/ranger/tagsync/model/TagSource.java 5ef6c57
> tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncConfig.java c52e0d2
> tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java b1b17cd
>
> Diff: https://reviews.apache.org/r/49743/diff/
>
>
> Testing
> -------
>
> Tested in a single node cluster with tag source as 'Atlas' and with zookeeper stopped. Tag-source initialization kept failing until zookeeper was started. Then tag-synchronizer started receiving tag events from Atlas.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 49743: tagsync should attempt re-initializing
failed tag sources periodically
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49743/
-----------------------------------------------------------
(Updated July 8, 2016, 4:05 a.m.)
Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
Changes
-------
Addressed review comments
Bugs: RANGER-1079
https://issues.apache.org/jira/browse/RANGER-1079
Repository: ranger
Description
-------
tag sync process, during startup, initializes tag sources only once and ignores those which failed initialization. This causes a tag source to be ignored even if its initialization failed due to a transient error. Tag sync needs to be made more robust by periodically attempting re-initialization of failed tag sources and recover from transient errors during initialization.
Diffs (updated)
-----
tagsync/src/main/java/org/apache/ranger/tagsync/model/AbstractTagSource.java d46170a
tagsync/src/main/java/org/apache/ranger/tagsync/model/TagSource.java 5ef6c57
tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncConfig.java c52e0d2
tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java b1b17cd
Diff: https://reviews.apache.org/r/49743/diff/
Testing
-------
Tested in a single node cluster with tag source as 'Atlas' and with zookeeper stopped. Tag-source initialization kept failing until zookeeper was started. Then tag-synchronizer started receiving tag events from Atlas.
Thanks,
Abhay Kulkarni