You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2018/10/18 19:28:03 UTC

Review Request 69072: RANGER-2203 : Review and update database schema for ranger policies to minimize database queries/updates, RANGER-2219: De-normalize schema for storing tags and related objects

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69072/
-----------------------------------------------------------

Review request for ranger, Madhan Neethiraj, Ramesh Mani, and Velmurugan Periasamy.


Summary (updated)
-----------------

RANGER-2203 : Review and update database schema for ranger policies to minimize database queries/updates, RANGER-2219: De-normalize schema for storing tags and related objects


Bugs: RANGER-2203 and RANGER-2219
    https://issues.apache.org/jira/browse/RANGER-2203
    https://issues.apache.org/jira/browse/RANGER-2219


Repository: ranger


Description (updated)
-------

Currently, ranger policies are fully normalized and stored in a multiple Relational database tables. There is a performance overhead incurred when retrieving a ranger policy, as multiple database accesses are required to fully reconstruct it. This is significant when there are large ranger policies (that is, the number of resources addressed by the policy is large), and/or when there is a large number of ranger policies in an installation.

This Jira tracks alternate design of database schema, where a policy is stored in a de-normalized way, in its entirely, in one database table (preferably as a JSON string).

Currently, tag-definitions, tags and service-resources are stored in database using a normalized form. When constructing resource->tag mappings, this schema design may lead to a large number of database accesses, thereby causing a major performance bottleneck when the number of resource->tag associations is large.

Denormalized schema will reduce the number of database accesses, and improve overall performance significantly.


Diffs (updated)
-----

  agents-common/pom.xml 09a32fa8b 
  agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java bc4a8b523 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerServiceDef.java adafb9985 
  agents-common/src/main/java/org/apache/ranger/plugin/store/TagStore.java 5918b1292 
  embeddedwebserver/scripts/ranger-admin-services.sh b8ca6c7d5 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 32cf6db3b 
  security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/mysql/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 2e577f3bd 
  security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/oracle/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql bad32efbc 
  security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/postgres/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 9482992e7 
  security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/sqlanywhere/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 85f328556 
  security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/sqlserver/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/scripts/db_setup.py 02701c726 
  security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java c26f0a576 
  security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 7875bc2a8 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 6c699cad9 
  security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java d29df930c 
  security-admin/src/main/java/org/apache/ranger/common/RangerServicePoliciesCache.java 0d5689a03 
  security-admin/src/main/java/org/apache/ranger/db/RangerDaoManagerBase.java 5dffc0ef2 
  security-admin/src/main/java/org/apache/ranger/db/XXAccessTypeDefDao.java e04280bc8 
  security-admin/src/main/java/org/apache/ranger/db/XXDataMaskTypeDefDao.java 3dd4376e4 
  security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 9a87b4cfd 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyConditionDefDao.java a12140aba 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemAccessDao.java 9be38decd 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemConditionDao.java 9b11545a0 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemDataMaskInfoDao.java 67c7e9961 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemGroupPermDao.java a6fd8c65b 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemRowFilterInfoDao.java 01a36a50d 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemUserPermDao.java 667265485 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefAccessTypeDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefDataMaskTypeDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefResourceDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceDao.java a7157deb4 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceMapDao.java 55d8c50e4 
  security-admin/src/main/java/org/apache/ranger/db/XXResourceDefDao.java b2e311fef 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceDao.java ee0e40043 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementDao.java c9a1c2132 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementValueDao.java 364af6d0f 
  security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDao.java 40c3a887b 
  security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDefDao.java 129f3c13c 
  security-admin/src/main/java/org/apache/ranger/db/XXTagDao.java e14f836e7 
  security-admin/src/main/java/org/apache/ranger/db/XXTagDefDao.java c0dd88371 
  security-admin/src/main/java/org/apache/ranger/db/XXTagResourceMapDao.java f9e041a3f 
  security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 58bf4d891 
  security-admin/src/main/java/org/apache/ranger/entity/XXDBBase.java 8405eb3e4 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyBase.java 584a103ff 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXServiceResource.java 961627a3c 
  security-admin/src/main/java/org/apache/ranger/entity/XXTag.java d26a0b079 
  security-admin/src/main/java/org/apache/ranger/entity/XXTagDef.java 818908ba8 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 97b8d7192 
  security-admin/src/main/java/org/apache/ranger/service/RangerAuditFields.java e6a519a17 
  security-admin/src/main/java/org/apache/ranger/service/RangerPolicyServiceBase.java 8d42165f7 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceDefServiceBase.java 5cbe47ad3 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceService.java d7256802a 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceServiceBase.java 6af682a81 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagDefService.java 82eb252e6 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagDefServiceBase.java 408358c48 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagService.java 28b9115fa 
  security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java c96878d21 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml cdf6ba655 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java c9db90a0e 
  security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 15344ea30 
  security-admin/src/test/java/org/apache/ranger/service/TestRangerTagDefService.java 8f19ffd78 
  security-admin/src/test/java/org/apache/ranger/service/TestRangerTagService.java 1175989e2 


Diff: https://reviews.apache.org/r/69072/diff/1/


Testing (updated)
-------

Tested with local VM for migration, policy CRUD, policy browsing and policy enforcement in Hive plugin.


Thanks,

Abhay Kulkarni


Re: Review Request 69072: RANGER-2203 : Review and update database schema for ranger policies to minimize database queries/updates, RANGER-2219: De-normalize schema for storing tags and related objects

Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69072/
-----------------------------------------------------------

(Updated Oct. 18, 2018, 10:01 p.m.)


Review request for ranger, Madhan Neethiraj, Ramesh Mani, and Velmurugan Periasamy.


Changes
-------

Addressed review comments on a related patch for ranger-0.7 (https://reviews.apache.org/r/68881/)


Bugs: RANGER-2203 and RANGER-2219
    https://issues.apache.org/jira/browse/RANGER-2203
    https://issues.apache.org/jira/browse/RANGER-2219


Repository: ranger


Description
-------

Currently, ranger policies are fully normalized and stored in a multiple Relational database tables. There is a performance overhead incurred when retrieving a ranger policy, as multiple database accesses are required to fully reconstruct it. This is significant when there are large ranger policies (that is, the number of resources addressed by the policy is large), and/or when there is a large number of ranger policies in an installation.

This Jira tracks alternate design of database schema, where a policy is stored in a de-normalized way, in its entirely, in one database table (preferably as a JSON string).

Currently, tag-definitions, tags and service-resources are stored in database using a normalized form. When constructing resource->tag mappings, this schema design may lead to a large number of database accesses, thereby causing a major performance bottleneck when the number of resource->tag associations is large.

Denormalized schema will reduce the number of database accesses, and improve overall performance significantly.


Diffs (updated)
-----

  agents-common/pom.xml 09a32fa8b 
  agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java bc4a8b523 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerServiceDef.java adafb9985 
  agents-common/src/main/java/org/apache/ranger/plugin/store/TagStore.java 5918b1292 
  embeddedwebserver/scripts/ranger-admin-services.sh b8ca6c7d5 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 32cf6db3b 
  security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/mysql/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 2e577f3bd 
  security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/oracle/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql bad32efbc 
  security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/postgres/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 9482992e7 
  security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/sqlanywhere/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 85f328556 
  security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
  security-admin/db/sqlserver/patches/036-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/scripts/db_setup.py 02701c726 
  security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java c26f0a576 
  security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 7875bc2a8 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 6c699cad9 
  security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java d29df930c 
  security-admin/src/main/java/org/apache/ranger/common/RangerServicePoliciesCache.java 0d5689a03 
  security-admin/src/main/java/org/apache/ranger/db/RangerDaoManagerBase.java 5dffc0ef2 
  security-admin/src/main/java/org/apache/ranger/db/XXAccessTypeDefDao.java e04280bc8 
  security-admin/src/main/java/org/apache/ranger/db/XXDataMaskTypeDefDao.java 3dd4376e4 
  security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 9a87b4cfd 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyConditionDefDao.java a12140aba 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemAccessDao.java 9be38decd 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemConditionDao.java 9b11545a0 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemDataMaskInfoDao.java 67c7e9961 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemGroupPermDao.java a6fd8c65b 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemRowFilterInfoDao.java 01a36a50d 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemUserPermDao.java 667265485 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefAccessTypeDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefDataMaskTypeDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefResourceDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceDao.java a7157deb4 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceMapDao.java 55d8c50e4 
  security-admin/src/main/java/org/apache/ranger/db/XXResourceDefDao.java b2e311fef 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceDao.java ee0e40043 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementDao.java c9a1c2132 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementValueDao.java 364af6d0f 
  security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDao.java 40c3a887b 
  security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDefDao.java 129f3c13c 
  security-admin/src/main/java/org/apache/ranger/db/XXTagDao.java e14f836e7 
  security-admin/src/main/java/org/apache/ranger/db/XXTagDefDao.java c0dd88371 
  security-admin/src/main/java/org/apache/ranger/db/XXTagResourceMapDao.java f9e041a3f 
  security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 58bf4d891 
  security-admin/src/main/java/org/apache/ranger/entity/XXDBBase.java 8405eb3e4 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyBase.java 584a103ff 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/entity/XXServiceResource.java 961627a3c 
  security-admin/src/main/java/org/apache/ranger/entity/XXTag.java d26a0b079 
  security-admin/src/main/java/org/apache/ranger/entity/XXTagDef.java 818908ba8 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 97b8d7192 
  security-admin/src/main/java/org/apache/ranger/service/RangerAuditFields.java e6a519a17 
  security-admin/src/main/java/org/apache/ranger/service/RangerPolicyServiceBase.java 8d42165f7 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceDefServiceBase.java 5cbe47ad3 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceService.java d7256802a 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceServiceBase.java 6af682a81 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagDefService.java 82eb252e6 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagDefServiceBase.java 408358c48 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagService.java 28b9115fa 
  security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java c96878d21 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml cdf6ba655 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java c9db90a0e 
  security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 15344ea30 
  security-admin/src/test/java/org/apache/ranger/service/TestRangerTagDefService.java 8f19ffd78 
  security-admin/src/test/java/org/apache/ranger/service/TestRangerTagService.java 1175989e2 


Diff: https://reviews.apache.org/r/69072/diff/2/

Changes: https://reviews.apache.org/r/69072/diff/1-2/


Testing
-------

Tested with local VM for migration, policy CRUD, policy browsing and policy enforcement in Hive plugin.


Thanks,

Abhay Kulkarni


Re: Review Request 69072: RANGER-2203 : Review and update database schema for ranger policies to minimize database queries/updates, RANGER-2219: De-normalize schema for storing tags and related objects

Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69072/#review209758
-----------------------------------------------------------


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 18, 2018, 7:27 p.m., Abhay Kulkarni wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69072/
> -----------------------------------------------------------
> 
> (Updated Oct. 18, 2018, 7:27 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2203 and RANGER-2219
>     https://issues.apache.org/jira/browse/RANGER-2203
>     https://issues.apache.org/jira/browse/RANGER-2219
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Currently, ranger policies are fully normalized and stored in a multiple Relational database tables. There is a performance overhead incurred when retrieving a ranger policy, as multiple database accesses are required to fully reconstruct it. This is significant when there are large ranger policies (that is, the number of resources addressed by the policy is large), and/or when there is a large number of ranger policies in an installation.
> 
> This Jira tracks alternate design of database schema, where a policy is stored in a de-normalized way, in its entirely, in one database table (preferably as a JSON string).
> 
> Currently, tag-definitions, tags and service-resources are stored in database using a normalized form. When constructing resource->tag mappings, this schema design may lead to a large number of database accesses, thereby causing a major performance bottleneck when the number of resource->tag associations is large.
> 
> Denormalized schema will reduce the number of database accesses, and improve overall performance significantly.
> 
> 
> Diffs
> -----
> 
>   agents-common/pom.xml 09a32fa8b 
>   agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java bc4a8b523 
>   agents-common/src/main/java/org/apache/ranger/plugin/model/RangerServiceDef.java adafb9985 
>   agents-common/src/main/java/org/apache/ranger/plugin/store/TagStore.java 5918b1292 
>   embeddedwebserver/scripts/ranger-admin-services.sh b8ca6c7d5 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 32cf6db3b 
>   security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
>   security-admin/db/mysql/patches/036-denormalize-tag-tables.sql PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 2e577f3bd 
>   security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
>   security-admin/db/oracle/patches/036-denormalize-tag-tables.sql PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql bad32efbc 
>   security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
>   security-admin/db/postgres/patches/036-denormalize-tag-tables.sql PRE-CREATION 
>   security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 9482992e7 
>   security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
>   security-admin/db/sqlanywhere/patches/036-denormalize-tag-tables.sql PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 85f328556 
>   security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql PRE-CREATION 
>   security-admin/db/sqlserver/patches/036-denormalize-tag-tables.sql PRE-CREATION 
>   security-admin/scripts/db_setup.py 02701c726 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java c26f0a576 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 7875bc2a8 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 6c699cad9 
>   security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java d29df930c 
>   security-admin/src/main/java/org/apache/ranger/common/RangerServicePoliciesCache.java 0d5689a03 
>   security-admin/src/main/java/org/apache/ranger/db/RangerDaoManagerBase.java 5dffc0ef2 
>   security-admin/src/main/java/org/apache/ranger/db/XXAccessTypeDefDao.java e04280bc8 
>   security-admin/src/main/java/org/apache/ranger/db/XXDataMaskTypeDefDao.java 3dd4376e4 
>   security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 9a87b4cfd 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyConditionDefDao.java a12140aba 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemAccessDao.java 9be38decd 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemConditionDao.java 9b11545a0 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemDataMaskInfoDao.java 67c7e9961 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemGroupPermDao.java a6fd8c65b 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemRowFilterInfoDao.java 01a36a50d 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemUserPermDao.java 667265485 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefAccessTypeDao.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefDataMaskTypeDao.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefResourceDao.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceDao.java a7157deb4 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceMapDao.java 55d8c50e4 
>   security-admin/src/main/java/org/apache/ranger/db/XXResourceDefDao.java b2e311fef 
>   security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceDao.java ee0e40043 
>   security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementDao.java c9a1c2132 
>   security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementValueDao.java 364af6d0f 
>   security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDao.java 40c3a887b 
>   security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDefDao.java 129f3c13c 
>   security-admin/src/main/java/org/apache/ranger/db/XXTagDao.java e14f836e7 
>   security-admin/src/main/java/org/apache/ranger/db/XXTagDefDao.java c0dd88371 
>   security-admin/src/main/java/org/apache/ranger/db/XXTagResourceMapDao.java f9e041a3f 
>   security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 58bf4d891 
>   security-admin/src/main/java/org/apache/ranger/entity/XXDBBase.java 8405eb3e4 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyBase.java 584a103ff 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/entity/XXServiceResource.java 961627a3c 
>   security-admin/src/main/java/org/apache/ranger/entity/XXTag.java d26a0b079 
>   security-admin/src/main/java/org/apache/ranger/entity/XXTagDef.java 818908ba8 
>   security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 97b8d7192 
>   security-admin/src/main/java/org/apache/ranger/service/RangerAuditFields.java e6a519a17 
>   security-admin/src/main/java/org/apache/ranger/service/RangerPolicyServiceBase.java 8d42165f7 
>   security-admin/src/main/java/org/apache/ranger/service/RangerServiceDefServiceBase.java 5cbe47ad3 
>   security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceService.java d7256802a 
>   security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceServiceBase.java 6af682a81 
>   security-admin/src/main/java/org/apache/ranger/service/RangerTagDefService.java 82eb252e6 
>   security-admin/src/main/java/org/apache/ranger/service/RangerTagDefServiceBase.java 408358c48 
>   security-admin/src/main/java/org/apache/ranger/service/RangerTagService.java 28b9115fa 
>   security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java c96878d21 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml cdf6ba655 
>   security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java c9db90a0e 
>   security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 15344ea30 
>   security-admin/src/test/java/org/apache/ranger/service/TestRangerTagDefService.java 8f19ffd78 
>   security-admin/src/test/java/org/apache/ranger/service/TestRangerTagService.java 1175989e2 
> 
> 
> Diff: https://reviews.apache.org/r/69072/diff/1/
> 
> 
> Testing
> -------
> 
> Tested with local VM for migration, policy CRUD, policy browsing and policy enforcement in Hive plugin.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>