You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Andrew Luo <an...@outlook.com> on 2020/01/28 07:32:33 UTC
Review Request 72051: RANGER-2713: Remove audit fields from
XXPolicyRef objects
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72051/
-----------------------------------------------------------
Review request for ranger.
Bugs: RANGER-2713
https://issues.apache.org/jira/browse/RANGER-2713
Repository: ranger
Description
-------
XXPolicyRef objects have fields such as create time, update time, added by user ID, updated by user ID, but there fields are entirely useless since they are all copied from the XXPolicy object. In addition, while improving performance for creation of policies with large numbers of users, we discovered that a lot of time was being spent in JPA converting these Date objects especially. After removing these fields we saw a significant performance improvement (a secondary benefit is less database space usage).
Diffs
-----
security-admin/db/mysql/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/oracle/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/postgres/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/sqlanywhere/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/sqlserver/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 85e6aebc1
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java 6af8f99f4
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 4f4409d6a
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 32a1b9f24
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 115064621
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 7aee502e0
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 8dfb92833
security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 69c8a4cb6
Diff: https://reviews.apache.org/r/72051/diff/1/
Testing
-------
Tested locally on Postgres and MySQL. Tested syntax of SQL on all 5 databases.
Thanks,
Andrew Luo
Re: Review Request 72051: RANGER-2713: Remove audit fields from
XXPolicyRef objects
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72051/#review219411
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Jan. 28, 2020, 7:32 a.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72051/
> -----------------------------------------------------------
>
> (Updated Jan. 28, 2020, 7:32 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
>
>
> Repository: ranger
>
>
> Description
> -------
>
> XXPolicyRef objects have fields such as create time, update time, added by user ID, updated by user ID, but there fields are entirely useless since they are all copied from the XXPolicy object. In addition, while improving performance for creation of policies with large numbers of users, we discovered that a lot of time was being spent in JPA converting these Date objects especially. After removing these fields we saw a significant performance improvement (a secondary benefit is less database space usage).
>
>
> Diffs
> -----
>
> security-admin/db/mysql/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/oracle/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/postgres/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/sqlanywhere/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/sqlserver/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 85e6aebc1
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java 6af8f99f4
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 4f4409d6a
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 32a1b9f24
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 115064621
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 7aee502e0
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 8dfb92833
> security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 69c8a4cb6
>
>
> Diff: https://reviews.apache.org/r/72051/diff/1/
>
>
> Testing
> -------
>
> Tested locally on Postgres and MySQL. Tested syntax of SQL on all 5 databases.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72051: RANGER-2713: Remove audit fields from
XXPolicyRef objects
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72051/#review219412
-----------------------------------------------------------
Please update XXPolicyRefDataMaskType as well, to remove audit fields (XXDBBase).
- Madhan Neethiraj
On Jan. 28, 2020, 7:32 a.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72051/
> -----------------------------------------------------------
>
> (Updated Jan. 28, 2020, 7:32 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
>
>
> Repository: ranger
>
>
> Description
> -------
>
> XXPolicyRef objects have fields such as create time, update time, added by user ID, updated by user ID, but there fields are entirely useless since they are all copied from the XXPolicy object. In addition, while improving performance for creation of policies with large numbers of users, we discovered that a lot of time was being spent in JPA converting these Date objects especially. After removing these fields we saw a significant performance improvement (a secondary benefit is less database space usage).
>
>
> Diffs
> -----
>
> security-admin/db/mysql/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/oracle/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/postgres/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/sqlanywhere/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/sqlserver/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 85e6aebc1
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java 6af8f99f4
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 4f4409d6a
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 32a1b9f24
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 115064621
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 7aee502e0
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 8dfb92833
> security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 69c8a4cb6
>
>
> Diff: https://reviews.apache.org/r/72051/diff/1/
>
>
> Testing
> -------
>
> Tested locally on Postgres and MySQL. Tested syntax of SQL on all 5 databases.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72051: RANGER-2713: Remove audit fields from
XXPolicyRef objects
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72051/#review219505
-----------------------------------------------------------
Ship it!
Ship It!
- Pradeep Agrawal
On Jan. 31, 2020, 5:39 a.m., Andrew Luo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72051/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2020, 5:39 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
>
>
> Repository: ranger
>
>
> Description
> -------
>
> XXPolicyRef objects have fields such as create time, update time, added by user ID, updated by user ID, but there fields are entirely useless since they are all copied from the XXPolicy object. In addition, while improving performance for creation of policies with large numbers of users, we discovered that a lot of time was being spent in JPA converting these Date objects especially. After removing these fields we saw a significant performance improvement (a secondary benefit is less database space usage).
>
>
> Diffs
> -----
>
> security-admin/db/mysql/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/oracle/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/postgres/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/sqlanywhere/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/db/sqlserver/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 85e6aebc1
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java 6af8f99f4
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 4f4409d6a
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java cb926740e
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 32a1b9f24
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 115064621
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 7aee502e0
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 8dfb92833
> security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 69c8a4cb6
>
>
> Diff: https://reviews.apache.org/r/72051/diff/2/
>
>
> Testing
> -------
>
> Tested locally on Postgres and MySQL. Tested syntax of SQL on all 5 databases.
>
>
> Thanks,
>
> Andrew Luo
>
>
Re: Review Request 72051: RANGER-2713: Remove audit fields from
XXPolicyRef objects
Posted by Andrew Luo <an...@outlook.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72051/
-----------------------------------------------------------
(Updated Jan. 31, 2020, 5:39 a.m.)
Review request for ranger.
Bugs: RANGER-2713
https://issues.apache.org/jira/browse/RANGER-2713
Repository: ranger
Description
-------
XXPolicyRef objects have fields such as create time, update time, added by user ID, updated by user ID, but there fields are entirely useless since they are all copied from the XXPolicy object. In addition, while improving performance for creation of policies with large numbers of users, we discovered that a lot of time was being spent in JPA converting these Date objects especially. After removing these fields we saw a significant performance improvement (a secondary benefit is less database space usage).
Diffs (updated)
-----
security-admin/db/mysql/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/oracle/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/postgres/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/sqlanywhere/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/db/sqlserver/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 85e6aebc1
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java 6af8f99f4
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 4f4409d6a
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java cb926740e
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 32a1b9f24
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 115064621
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 7aee502e0
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 8dfb92833
security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 69c8a4cb6
Diff: https://reviews.apache.org/r/72051/diff/2/
Changes: https://reviews.apache.org/r/72051/diff/1-2/
Testing
-------
Tested locally on Postgres and MySQL. Tested syntax of SQL on all 5 databases.
Thanks,
Andrew Luo