You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org> on 2019/02/22 18:40:00 UTC
Review Request 70043: SENTRY-2502: Sentry NN plug-in stops fetching
updates from sentry server
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/
-----------------------------------------------------------
Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
Bugs: SENTRY-2502
https://issues.apache.org/jira/browse/SENTRY-2502
Repository: sentry
Description
-------
Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
Create a table
Add a single partition to it.
Rename the table created above.
Drop the partition added above
Add the partition again.
Rename the table
There are a couple of issues observed
When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
Diffs
-----
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
Diff: https://reviews.apache.org/r/70043/diff/1/
Testing
-------
Added new test case to verify that issues that does not occur after the fix.
Thanks,
kalyan kumar kalvagadda
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org>.
> On Feb. 22, 2019, 8:58 p.m., Na Li wrote:
> > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
> > Lines 507 (patched)
> > <https://reviews.apache.org/r/70043/diff/1/?file=2126422#file2126422line509>
> >
> > should this be removed if it has no authz obj?
> >
> > Why change its type to DIR?
one entry object is created for every directoy in the path of file. Some of the directories may not be associated with any object. In those cases Entry is marked as DIR.
In this case if Entry is not associated with any object it should be marked as DIR. Previously it was marked as directory when when it was associated with one or more objects.
- kalyan kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213095
-----------------------------------------------------------
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by Na Li via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213095
-----------------------------------------------------------
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
Lines 507 (patched)
<https://reviews.apache.org/r/70043/#comment298962>
should this be removed if it has no authz obj?
Why change its type to DIR?
- Na Li
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org>.
> On Feb. 22, 2019, 8:46 p.m., Na Li wrote:
> > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
> > Line 689 (original), 694 (patched)
> > <https://reviews.apache.org/r/70043/diff/1/?file=2126422#file2126422line696>
> >
> > Kalyan, do you mean "If we use the updated Entry to find/delete it from the set it will NOT be found/delted." instead of "If we use the updated Entry to find/delete it from the set it will be found/delted."?
> >
> > Your change has performance penalty. Do you have an idea how big the performance degradation could it be? and will we be able to accept that?
If we use the updated Entry to find/delete it from the set it will NOT be found/delted."
Yes, there will be performence impact when the collection of Entries is tool authzObjToEntries when the updates from sentry are processed.
- kalyan kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213094
-----------------------------------------------------------
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by Na Li via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213094
-----------------------------------------------------------
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
Line 689 (original), 694 (patched)
<https://reviews.apache.org/r/70043/#comment298961>
Kalyan, do you mean "If we use the updated Entry to find/delete it from the set it will NOT be found/delted." instead of "If we use the updated Entry to find/delete it from the set it will be found/delted."?
Your change has performance penalty. Do you have an idea how big the performance degradation could it be? and will we be able to accept that?
- Na Li
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by Haley Reeve via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213091
-----------------------------------------------------------
Ship it!
Ship It!
- Haley Reeve
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by Na Li via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213102
-----------------------------------------------------------
Ship it!
Ship It!
- Na Li
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org>.
> On Feb. 22, 2019, 6:48 p.m., Haley Reeve wrote:
> > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
> > Line 689 (original), 694 (patched)
> > <https://reviews.apache.org/r/70043/diff/1/?file=2126422#file2126422line696>
> >
> > Why the change from sets to lists? Was there some kind of duplicate entries that were getting lost when using a set? (Just asking for my own curiosity since I'm not super familiar with this code.)
We were using HashSet before. When a element is inserted into hashset there is hash value calculated and stored for that element. This hash value is used for future lookups.
Entry's that were inserted into hashset are updated later over time. There are no reference older Entry. If we use the updated Entry to find/delete it from the set it will be found/delted. Reason for that is that hash value of Entry which is updated does not match hash value calculated when it was inserted. As a result, code we had was never able to remove the entries from authzObjToEntries. That is the reason i have changed it to a list.
- kalyan kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213088
-----------------------------------------------------------
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by Haley Reeve via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/#review213088
-----------------------------------------------------------
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
Line 689 (original), 694 (patched)
<https://reviews.apache.org/r/70043/#comment298958>
Why the change from sets to lists? Was there some kind of duplicate entries that were getting lost when using a set? (Just asking for my own curiosity since I'm not super familiar with this code.)
- Haley Reeve
On Feb. 22, 2019, 6:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70043/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2019, 6:40 p.m.)
>
>
> Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
>
>
> Bugs: SENTRY-2502
> https://issues.apache.org/jira/browse/SENTRY-2502
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
>
> Create a table
> Add a single partition to it.
> Rename the table created above.
> Drop the partition added above
> Add the partition again.
> Rename the table
>
> There are a couple of issues observed
>
> When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
> When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
> These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java 8a7c8a2901af18b375f8637572d9b3ddee0ded6c
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
>
>
> Diff: https://reviews.apache.org/r/70043/diff/1/
>
>
> Testing
> -------
>
> Added new test case to verify that issues that does not occur after the fix.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 70043: SENTRY-2502: Sentry NN plug-in stops
fetching updates from sentry server
Posted by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70043/
-----------------------------------------------------------
(Updated Feb. 23, 2019, 5:28 a.m.)
Review request for sentry, Arjun Mishra, Haley Reeve, and Na Li.
Bugs: SENTRY-2502
https://issues.apache.org/jira/browse/SENTRY-2502
Repository: sentry
Description
-------
Sentry plug-in in name node is stopping to fetch updates from sentry server when below sequence of events occurs.
Create a table
Add a single partition to it.
Rename the table created above.
Drop the partition added above
Add the partition again.
Rename the table
There are a couple of issues observed
When the table is renamed the Entry object for the table that is renamed is wrongly update. Type of Entry is changed to "DIR".
When paths have removed the entries with empty paths are not removed from the collection of Entries for an object.
These two issues can manifest in multiple problems. Issue reported in this Jira is one such problem.
Diffs (updated)
-----
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 1dfa1d9aafbd482903e4bb935509a76768e02b58
sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java 9fe22ef8cc1533b28462e2b8749e3ba0e7b56582
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 574bc4b0432824abc81fe3c0dd30d1fe01f012e5
Diff: https://reviews.apache.org/r/70043/diff/2/
Changes: https://reviews.apache.org/r/70043/diff/1-2/
Testing
-------
Added new test case to verify that issues that does not occur after the fix.
Thanks,
kalyan kumar kalvagadda