You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Pradeep Agrawal <pr...@gmail.com> on 2018/10/22 09:04:46 UTC

Review Request 69111: RANGER-2260: Atlas servicedef version change patch should update atlas access type def for tag def also.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69111/
-----------------------------------------------------------

Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-2260
    https://issues.apache.org/jira/browse/RANGER-2260


Repository: ranger


Description
-------

**Problem Statement:** Currently atlas ranger service def name change patch J10013 does not rename its access types of the tag service def.

**Proposed Solution:** Proposed solution includes changes in java patch J10013 which shall rename the tag access types and implied grants also if atlas service def is renamed.

If java patch J10013 is already executed then sql patch 035 execution will make sure that tag service def access types are renamed as per new atlas servicedef name.


Diffs
-----

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 70447fa2d 
  security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql ee82ae3eb 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8b513073d 
  security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 11b41720e 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a12391191 
  security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql cb87cd0b8 
  security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 142302a80 
  security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 24d072f83 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 75c8fafeb 
  security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql ebf44ac83 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java 1b315cd76 


Diff: https://reviews.apache.org/r/69111/diff/1/


Testing
-------

**Steps Performed (without patch):**
1. After mvn Build; untar the Ranger-0.7 module and updated install.properties for MySQL DB flavor.
2. Called setup.sh to install Ranger-admin.
3. Started Ranger-admin.
4. Created atlas and tag based policies in ranger-admin
5. Stopped ranger-admin
6. Changed branch to ranger-1.0 and started mvn Build; after getting build untar the Ranger-1.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
7. Called setup.sh to install Ranger.
8. Started Ranger.
9. atlas service def was renamed to atlasdev.v1 and was able to view atlas and tag based policies in ranger-admin
10. Stopped ranger-admin
11. Changed branch to ranger-2.0 and started mvn Build; after getting build untar the Ranger-2.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
12. Removed 'DB_PATCHES' and 'JAVA_PATCHES' entry from x_db_version_h table to enable execution of new SQL patches.
13. Called setup.sh to install Ranger.

**Expected Behavior:**
1. Ranger installation should finish successfully and patch 035 should get applied successfully.
2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries should rename to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
3. Java patch 19 and 20 starting should migrate policies to new schema and policies should appear in policy listing page after starting the ranger-admin.
4. New policies creation should work and policy should get save in new policy schema.

** Actual Behavior: **
1. Ranger installation finished successfully and patch 035 was applied successfully. all other new sql and java patches was applied successfully.
2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries were renamed to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
3. Policy migration completed successfully and after starting ranger-admin policies was appearing in policy listing page.
4. Was able to create new policies in the the new atlas service def.

**Note:**
1. Patch has been tested on all supported DB Flavors.
2. Have not done load testing.
3. Have tested the patch on ranger-2.0 (installed ranger from pre RANGER-2203 commit and upgraded to latest master code build having change of this patch. This steps is tested for MySQL db flavors only.


Thanks,

Pradeep Agrawal


Re: Review Request 69111: RANGER-2260: Atlas servicedef version change patch should update atlas access type def for tag def also.

Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69111/#review209843
-----------------------------------------------------------


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 22, 2018, 9:04 a.m., Pradeep Agrawal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69111/
> -----------------------------------------------------------
> 
> (Updated Oct. 22, 2018, 9:04 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2260
>     https://issues.apache.org/jira/browse/RANGER-2260
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> **Problem Statement:** Currently atlas ranger service def name change patch J10013 does not rename its access types of the tag service def.
> 
> **Proposed Solution:** Proposed solution includes changes in java patch J10013 which shall rename the tag access types and implied grants also if atlas service def is renamed.
> 
> If java patch J10013 is already executed then sql patch 035 execution will make sure that tag service def access types are renamed as per new atlas servicedef name.
> 
> 
> Diffs
> -----
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 70447fa2d 
>   security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql ee82ae3eb 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8b513073d 
>   security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 11b41720e 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a12391191 
>   security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql cb87cd0b8 
>   security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 142302a80 
>   security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 24d072f83 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 75c8fafeb 
>   security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql ebf44ac83 
>   security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java 1b315cd76 
> 
> 
> Diff: https://reviews.apache.org/r/69111/diff/1/
> 
> 
> Testing
> -------
> 
> **Steps Performed (without patch):**
> 1. After mvn Build; untar the Ranger-0.7 module and updated install.properties for MySQL DB flavor.
> 2. Called setup.sh to install Ranger-admin.
> 3. Started Ranger-admin.
> 4. Created atlas and tag based policies in ranger-admin
> 5. Stopped ranger-admin
> 6. Changed branch to ranger-1.0 and started mvn Build; after getting build untar the Ranger-1.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
> 7. Called setup.sh to install Ranger.
> 8. Started Ranger.
> 9. atlas service def was renamed to atlasdev.v1 and was able to view atlas and tag based policies in ranger-admin
> 10. Stopped ranger-admin
> 11. Changed branch to ranger-2.0 and started mvn Build; after getting build untar the Ranger-2.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
> 12. Removed 'DB_PATCHES' and 'JAVA_PATCHES' entry from x_db_version_h table to enable execution of new SQL patches.
> 13. Called setup.sh to install Ranger.
> 
> **Expected Behavior:**
> 1. Ranger installation should finish successfully and patch 035 should get applied successfully.
> 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries should rename to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
> 3. Java patch 19 and 20 starting should migrate policies to new schema and policies should appear in policy listing page after starting the ranger-admin.
> 4. New policies creation should work and policy should get save in new policy schema.
> 
> ** Actual Behavior: **
> 1. Ranger installation finished successfully and patch 035 was applied successfully. all other new sql and java patches was applied successfully.
> 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries were renamed to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
> 3. Policy migration completed successfully and after starting ranger-admin policies was appearing in policy listing page.
> 4. Was able to create new policies in the the new atlas service def.
> 
> **Note:**
> 1. Patch has been tested on all supported DB Flavors.
> 2. Have not done load testing.
> 3. Have tested the patch on ranger-2.0 (installed ranger from pre RANGER-2203 commit and upgraded to latest master code build having change of this patch. This steps is tested for MySQL db flavors only.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>


Re: Review Request 69111: RANGER-2260: Atlas servicedef version change patch should update atlas access type def for tag def also.

Posted by Velmurugan Periasamy <vp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69111/#review209849
-----------------------------------------------------------


Fix it, then Ship it!




Fix the PMD violation.


security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 230 (patched)
<https://reviews.apache.org/r/69111/#comment294456>

    Could you please addresss PMD violation?
    
    ```
    [INFO] --- maven-pmd-plugin:3.7:check (default) @ security-admin-web ---
    [INFO] PMD Failure: org.apache.ranger.patch.PatchForAtlasServiceDefUpdate_J10013:230 Rule:UnusedFormalParameter Priority:3 Avoid unused method parameters such as 'prefix'..
    ```


- Velmurugan Periasamy


On Oct. 22, 2018, 9:04 a.m., Pradeep Agrawal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69111/
> -----------------------------------------------------------
> 
> (Updated Oct. 22, 2018, 9:04 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2260
>     https://issues.apache.org/jira/browse/RANGER-2260
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> **Problem Statement:** Currently atlas ranger service def name change patch J10013 does not rename its access types of the tag service def.
> 
> **Proposed Solution:** Proposed solution includes changes in java patch J10013 which shall rename the tag access types and implied grants also if atlas service def is renamed.
> 
> If java patch J10013 is already executed then sql patch 035 execution will make sure that tag service def access types are renamed as per new atlas servicedef name.
> 
> 
> Diffs
> -----
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 70447fa2d 
>   security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql ee82ae3eb 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8b513073d 
>   security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 11b41720e 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a12391191 
>   security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql cb87cd0b8 
>   security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 142302a80 
>   security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 24d072f83 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 75c8fafeb 
>   security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql ebf44ac83 
>   security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java 1b315cd76 
> 
> 
> Diff: https://reviews.apache.org/r/69111/diff/1/
> 
> 
> Testing
> -------
> 
> **Steps Performed (without patch):**
> 1. After mvn Build; untar the Ranger-0.7 module and updated install.properties for MySQL DB flavor.
> 2. Called setup.sh to install Ranger-admin.
> 3. Started Ranger-admin.
> 4. Created atlas and tag based policies in ranger-admin
> 5. Stopped ranger-admin
> 6. Changed branch to ranger-1.0 and started mvn Build; after getting build untar the Ranger-1.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
> 7. Called setup.sh to install Ranger.
> 8. Started Ranger.
> 9. atlas service def was renamed to atlasdev.v1 and was able to view atlas and tag based policies in ranger-admin
> 10. Stopped ranger-admin
> 11. Changed branch to ranger-2.0 and started mvn Build; after getting build untar the Ranger-2.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
> 12. Removed 'DB_PATCHES' and 'JAVA_PATCHES' entry from x_db_version_h table to enable execution of new SQL patches.
> 13. Called setup.sh to install Ranger.
> 
> **Expected Behavior:**
> 1. Ranger installation should finish successfully and patch 035 should get applied successfully.
> 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries should rename to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
> 3. Java patch 19 and 20 starting should migrate policies to new schema and policies should appear in policy listing page after starting the ranger-admin.
> 4. New policies creation should work and policy should get save in new policy schema.
> 
> ** Actual Behavior: **
> 1. Ranger installation finished successfully and patch 035 was applied successfully. all other new sql and java patches was applied successfully.
> 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries were renamed to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
> 3. Policy migration completed successfully and after starting ranger-admin policies was appearing in policy listing page.
> 4. Was able to create new policies in the the new atlas service def.
> 
> **Note:**
> 1. Patch has been tested on all supported DB Flavors.
> 2. Have not done load testing.
> 3. Have tested the patch on ranger-2.0 (installed ranger from pre RANGER-2203 commit and upgraded to latest master code build having change of this patch. This steps is tested for MySQL db flavors only.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>


Re: Review Request 69111: RANGER-2260: Atlas servicedef version change patch should update atlas access type def for tag def also.

Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69111/
-----------------------------------------------------------

(Updated Oct. 22, 2018, 2:02 p.m.)


Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy.


Changes
-------

Fixed PMD violation


Bugs: RANGER-2260
    https://issues.apache.org/jira/browse/RANGER-2260


Repository: ranger


Description
-------

**Problem Statement:** Currently atlas ranger service def name change patch J10013 does not rename its access types of the tag service def.

**Proposed Solution:** Proposed solution includes changes in java patch J10013 which shall rename the tag access types and implied grants also if atlas service def is renamed.

If java patch J10013 is already executed then sql patch 035 execution will make sure that tag service def access types are renamed as per new atlas servicedef name.


Diffs (updated)
-----

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 70447fa2d 
  security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql ee82ae3eb 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8b513073d 
  security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 11b41720e 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a12391191 
  security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql cb87cd0b8 
  security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 142302a80 
  security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 24d072f83 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 75c8fafeb 
  security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql ebf44ac83 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java 1b315cd76 


Diff: https://reviews.apache.org/r/69111/diff/2/

Changes: https://reviews.apache.org/r/69111/diff/1-2/


Testing
-------

**Steps Performed (without patch):**
1. After mvn Build; untar the Ranger-0.7 module and updated install.properties for MySQL DB flavor.
2. Called setup.sh to install Ranger-admin.
3. Started Ranger-admin.
4. Created atlas and tag based policies in ranger-admin
5. Stopped ranger-admin
6. Changed branch to ranger-1.0 and started mvn Build; after getting build untar the Ranger-1.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
7. Called setup.sh to install Ranger.
8. Started Ranger.
9. atlas service def was renamed to atlasdev.v1 and was able to view atlas and tag based policies in ranger-admin
10. Stopped ranger-admin
11. Changed branch to ranger-2.0 and started mvn Build; after getting build untar the Ranger-2.0 module and updated install.properties  for MySQL DB flavor with the config provided in step-1.
12. Removed 'DB_PATCHES' and 'JAVA_PATCHES' entry from x_db_version_h table to enable execution of new SQL patches.
13. Called setup.sh to install Ranger.

**Expected Behavior:**
1. Ranger installation should finish successfully and patch 035 should get applied successfully.
2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries should rename to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
3. Java patch 19 and 20 starting should migrate policies to new schema and policies should appear in policy listing page after starting the ranger-admin.
4. New policies creation should work and policy should get save in new policy schema.

** Actual Behavior: **
1. Ranger installation finished successfully and patch 035 was applied successfully. all other new sql and java patches was applied successfully.
2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries were renamed to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries.
3. Policy migration completed successfully and after starting ranger-admin policies was appearing in policy listing page.
4. Was able to create new policies in the the new atlas service def.

**Note:**
1. Patch has been tested on all supported DB Flavors.
2. Have not done load testing.
3. Have tested the patch on ranger-2.0 (installed ranger from pre RANGER-2203 commit and upgraded to latest master code build having change of this patch. This steps is tested for MySQL db flavors only.


Thanks,

Pradeep Agrawal