You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Andrew Luo <an...@outlook.com> on 2020/02/16 22:48:46 UTC
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/
-----------------------------------------------------------
(Updated Feb. 16, 2020, 10:48 p.m.)
Review request for ranger.
Summary (updated)
-----------------
RANGER-2732: Batch lookup role, group, and user IDs during policy creation
Bugs: RANGER-2732
https://issues.apache.org/jira/browse/RANGER-2732
Repository: ranger
Description
-------
Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
Diffs
-----
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
Diff: https://reviews.apache.org/r/72142/diff/1/
Testing
-------
Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
Thanks,
Andrew Luo
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/#review219641
-----------------------------------------------------------
Ship it!
Ship It!
- Pradeep Agrawal
On Feb. 17, 2020, 7:56 p.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72142/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2020, 7:56 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2732
> https://issues.apache.org/jira/browse/RANGER-2732
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
> security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
> security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
> security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
> security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
>
>
> Diff: https://reviews.apache.org/r/72142/diff/2/
>
>
> Testing
> -------
>
> Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/#review219642
-----------------------------------------------------------
Can you rebase your patch on latest Ranger. Patch is not applying on latest Ranger.
- Pradeep Agrawal
On Feb. 17, 2020, 7:56 p.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72142/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2020, 7:56 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2732
> https://issues.apache.org/jira/browse/RANGER-2732
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
> security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
> security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
> security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
> security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
>
>
> Diff: https://reviews.apache.org/r/72142/diff/2/
>
>
> Testing
> -------
>
> Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
Posted by Andrew Luo <an...@outlook.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/
-----------------------------------------------------------
(Updated Feb. 17, 2020, 7:56 p.m.)
Review request for ranger.
Changes
-------
Addressed review comments
Bugs: RANGER-2732
https://issues.apache.org/jira/browse/RANGER-2732
Repository: ranger
Description
-------
Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
Diffs (updated)
-----
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
Diff: https://reviews.apache.org/r/72142/diff/2/
Changes: https://reviews.apache.org/r/72142/diff/1-2/
Testing
-------
Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
Thanks,
Andrew Luo
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
Posted by Pradeep Agrawal <pr...@gmail.com>.
> On Feb. 17, 2020, 1:31 a.m., Pradeep Agrawal wrote:
> > security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java
> > Lines 77 (patched)
> > <https://reviews.apache.org/r/72142/diff/1/?file=2211449#file2211449line78>
> >
> > if possible avoid multiple return statements from the same method.
>
> Andrew Luo wrote:
> I just followed the structure of the existing findByGroupName/findByRoleName/findByUserName functions. I can change it if you believe this is the right way, let me know what you prefer.
yes, let's try to follow this practice in new code, will changes in the previous code also whenever we get opportunity to touch them.
- Pradeep
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/#review219598
-----------------------------------------------------------
On Feb. 16, 2020, 10:48 p.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72142/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2020, 10:48 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2732
> https://issues.apache.org/jira/browse/RANGER-2732
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
> security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
> security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
> security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
> security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
>
>
> Diff: https://reviews.apache.org/r/72142/diff/1/
>
>
> Testing
> -------
>
> Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
Posted by Andrew Luo <an...@outlook.com>.
> On Feb. 17, 2020, 1:31 a.m., Pradeep Agrawal wrote:
> > security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java
> > Lines 76 (patched)
> > <https://reviews.apache.org/r/72142/diff/1/?file=2211449#file2211449line77>
> >
> > Try to use CollectionUtils.isEmpty(groupNames) here from package org.apache.commons.collections;
> >
> > update the same in other places also.
Thanks, will fix.
> On Feb. 17, 2020, 1:31 a.m., Pradeep Agrawal wrote:
> > security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java
> > Lines 77 (patched)
> > <https://reviews.apache.org/r/72142/diff/1/?file=2211449#file2211449line78>
> >
> > if possible avoid multiple return statements from the same method.
I just followed the structure of the existing findByGroupName/findByRoleName/findByUserName functions. I can change it if you believe this is the right way, let me know what you prefer.
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/#review219598
-----------------------------------------------------------
On Feb. 16, 2020, 10:48 p.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72142/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2020, 10:48 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2732
> https://issues.apache.org/jira/browse/RANGER-2732
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
> security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
> security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
> security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
> security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
>
>
> Diff: https://reviews.apache.org/r/72142/diff/1/
>
>
> Testing
> -------
>
> Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72142: RANGER-2732: Batch lookup role, group,
and user IDs during policy creation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72142/#review219598
-----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java
Lines 76 (patched)
<https://reviews.apache.org/r/72142/#comment307778>
Try to use CollectionUtils.isEmpty(groupNames) here from package org.apache.commons.collections;
update the same in other places also.
security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java
Lines 77 (patched)
<https://reviews.apache.org/r/72142/#comment307779>
if possible avoid multiple return statements from the same method.
- Pradeep Agrawal
On Feb. 16, 2020, 10:48 p.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72142/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2020, 10:48 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2732
> https://issues.apache.org/jira/browse/RANGER-2732
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Getting the ID for each role/group/user one-by-one is slow, especially for large policies with many roles/groups/users. Batching significantly improves performance.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 318f9f505
> security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 1bd59f8d2
> security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 8528652fc
> security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java cea90c165
> security-admin/src/main/resources/META-INF/jpa_named_queries.xml f23bf2e65
>
>
> Diff: https://reviews.apache.org/r/72142/diff/1/
>
>
> Testing
> -------
>
> Tested end-to-end. Verified that missing users are created, and existing users are correctly added to policies.
>
>
> Thanks,
>
> Andrew Luo
>
>