You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Mahesh Bandal <ma...@gmail.com> on 2020/03/30 15:49:51 UTC
Re: Review Request 72252: RANGER-2764 : Role API : Duplicate key value
violates unique constraint "x_ranger_global_state_uk_state_name"
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72252/
-----------------------------------------------------------
(Updated March 30, 2020, 3:49 p.m.)
Review request for ranger, Ankita Sinha, Gautam Borad, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Bugs: RANGER-2764
https://issues.apache.org/jira/browse/RANGER-2764
Repository: ranger
Description
-------
The issue came when Role API was called in multi thread and two separate threads were trying to INSERT same roletype, which has unique constraint. ie 'state_name' column in 'x_ranger_global_state' table. To handle such scenarios, loading 'x_ranger_global_state' table with initial version values during initial setup only.
Also OptimisticLockException occurs for RoleCreation and RoleDeletion when two threads are trying to update version column of x_ranger_global_state table. To solve this proble I have used RangerTransactionSynchronizationAdapter class for updating version in synchonised way.
Diffs
-----
security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2d93fc429
security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 38e546a56
security-admin/db/oracle/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 7c2ada55e
security-admin/db/postgres/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 7b0509cc0
security-admin/db/sqlanywhere/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 25b3bed34
security-admin/db/sqlserver/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java c4a32e46e
security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java b979459e8
Diff: https://reviews.apache.org/r/72252/diff/5/
Testing
-------
1. Fresh(Initial) Setup : Created new environment for ranger master. So now patch execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
2. Upgrade ranger : Created new environment for ranger master branch and ranger-2.0 branch. Run setup.sh file for ranger-2.0. Initially no values in 'x_ranger_global_state' table. Now setting up master branch with upgraded patch and run setup.sh on master branch where db points to same schema as ranger-2.0. db file '046-insert-statename-in-x-ranger-global-state.sql' execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
3. Created/Deleted 300 roles simultaneously. Roles are successfully created but version update fails at some point with OptimisticLockException. Handled exception and success response retured.
After applying patch; created/deleted multiple roles. 'state_name' column updates successfully.
Thanks,
Mahesh Bandal
Re: Review Request 72252: RANGER-2764 : Role API : Duplicate key value
violates unique constraint "x_ranger_global_state_uk_state_name"
Posted by Mehul Parikh <xs...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72252/#review220151
-----------------------------------------------------------
Ship it!
Ship It!
- Mehul Parikh
On March 31, 2020, 7:25 a.m., Mahesh Bandal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72252/
> -----------------------------------------------------------
>
> (Updated March 31, 2020, 7:25 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Gautam Borad, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2764
> https://issues.apache.org/jira/browse/RANGER-2764
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The issue came when Role API was called in multi thread and two separate threads were trying to INSERT same roletype, which has unique constraint. ie 'state_name' column in 'x_ranger_global_state' table. To handle such scenarios, loading 'x_ranger_global_state' table with initial version values during initial setup only.
> Also OptimisticLockException occurs for RoleCreation and RoleDeletion when two threads are trying to update version column of x_ranger_global_state table. To solve this proble I have used RangerTransactionSynchronizationAdapter class for updating version in synchonised way.
>
>
> Diffs
> -----
>
> security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2d93fc429
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 38e546a56
> security-admin/db/oracle/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 7c2ada55e
> security-admin/db/postgres/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 7b0509cc0
> security-admin/db/sqlanywhere/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 25b3bed34
> security-admin/db/sqlserver/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java c4a32e46e
> security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java b979459e8
>
>
> Diff: https://reviews.apache.org/r/72252/diff/6/
>
>
> Testing
> -------
>
> 1. Fresh(Initial) Setup : Created new environment for ranger master. So now patch execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
>
> 2. Upgrade ranger : Created new environment for ranger master branch and ranger-2.0 branch. Run setup.sh file for ranger-2.0. Initially no values in 'x_ranger_global_state' table. Now setting up master branch with upgraded patch and run setup.sh on master branch where db points to same schema as ranger-2.0. db file '046-insert-statename-in-x-ranger-global-state.sql' execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
> 3. Created/Deleted 300 roles simultaneously. Roles are successfully created but version update fails at some point with OptimisticLockException. Handled exception and success response retured.
>
> After applying patch; created/deleted multiple roles. 'state_name' column updates successfully.
>
>
> Thanks,
>
> Mahesh Bandal
>
>
Re: Review Request 72252: RANGER-2764 : Role API : Duplicate key value
violates unique constraint "x_ranger_global_state_uk_state_name"
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72252/#review220150
-----------------------------------------------------------
Ship it!
Ship It!
- Abhay Kulkarni
On March 31, 2020, 7:25 a.m., Mahesh Bandal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72252/
> -----------------------------------------------------------
>
> (Updated March 31, 2020, 7:25 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Gautam Borad, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2764
> https://issues.apache.org/jira/browse/RANGER-2764
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The issue came when Role API was called in multi thread and two separate threads were trying to INSERT same roletype, which has unique constraint. ie 'state_name' column in 'x_ranger_global_state' table. To handle such scenarios, loading 'x_ranger_global_state' table with initial version values during initial setup only.
> Also OptimisticLockException occurs for RoleCreation and RoleDeletion when two threads are trying to update version column of x_ranger_global_state table. To solve this proble I have used RangerTransactionSynchronizationAdapter class for updating version in synchonised way.
>
>
> Diffs
> -----
>
> security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2d93fc429
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 38e546a56
> security-admin/db/oracle/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 7c2ada55e
> security-admin/db/postgres/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 7b0509cc0
> security-admin/db/sqlanywhere/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 25b3bed34
> security-admin/db/sqlserver/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java c4a32e46e
> security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java b979459e8
>
>
> Diff: https://reviews.apache.org/r/72252/diff/6/
>
>
> Testing
> -------
>
> 1. Fresh(Initial) Setup : Created new environment for ranger master. So now patch execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
>
> 2. Upgrade ranger : Created new environment for ranger master branch and ranger-2.0 branch. Run setup.sh file for ranger-2.0. Initially no values in 'x_ranger_global_state' table. Now setting up master branch with upgraded patch and run setup.sh on master branch where db points to same schema as ranger-2.0. db file '046-insert-statename-in-x-ranger-global-state.sql' execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
> 3. Created/Deleted 300 roles simultaneously. Roles are successfully created but version update fails at some point with OptimisticLockException. Handled exception and success response retured.
>
> After applying patch; created/deleted multiple roles. 'state_name' column updates successfully.
>
>
> Thanks,
>
> Mahesh Bandal
>
>
Re: Review Request 72252: RANGER-2764 : Role API : Duplicate key value
violates unique constraint "x_ranger_global_state_uk_state_name"
Posted by Mahesh Bandal <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72252/
-----------------------------------------------------------
(Updated March 31, 2020, 7:25 a.m.)
Review request for ranger, Ankita Sinha, Gautam Borad, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Changes
-------
Changed logger level to warn for OptimisticLockException
Bugs: RANGER-2764
https://issues.apache.org/jira/browse/RANGER-2764
Repository: ranger
Description
-------
The issue came when Role API was called in multi thread and two separate threads were trying to INSERT same roletype, which has unique constraint. ie 'state_name' column in 'x_ranger_global_state' table. To handle such scenarios, loading 'x_ranger_global_state' table with initial version values during initial setup only.
Also OptimisticLockException occurs for RoleCreation and RoleDeletion when two threads are trying to update version column of x_ranger_global_state table. To solve this proble I have used RangerTransactionSynchronizationAdapter class for updating version in synchonised way.
Diffs (updated)
-----
security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2d93fc429
security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 38e546a56
security-admin/db/oracle/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 7c2ada55e
security-admin/db/postgres/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 7b0509cc0
security-admin/db/sqlanywhere/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 25b3bed34
security-admin/db/sqlserver/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java c4a32e46e
security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java b979459e8
Diff: https://reviews.apache.org/r/72252/diff/6/
Changes: https://reviews.apache.org/r/72252/diff/5-6/
Testing
-------
1. Fresh(Initial) Setup : Created new environment for ranger master. So now patch execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
2. Upgrade ranger : Created new environment for ranger master branch and ranger-2.0 branch. Run setup.sh file for ranger-2.0. Initially no values in 'x_ranger_global_state' table. Now setting up master branch with upgraded patch and run setup.sh on master branch where db points to same schema as ranger-2.0. db file '046-insert-statename-in-x-ranger-global-state.sql' execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
3. Created/Deleted 300 roles simultaneously. Roles are successfully created but version update fails at some point with OptimisticLockException. Handled exception and success response retured.
After applying patch; created/deleted multiple roles. 'state_name' column updates successfully.
Thanks,
Mahesh Bandal
Re: Review Request 72252: RANGER-2764 : Role API : Duplicate key value
violates unique constraint "x_ranger_global_state_uk_state_name"
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72252/#review220119
-----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java
Lines 84 (patched)
<https://reviews.apache.org/r/72252/#comment308446>
Please consider setting logging level to "warn", as we dont expect users to take any action on it. Also, please change the message to indicate that the processing will be continued.
security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java
Line 82 (original), 86 (patched)
<https://reviews.apache.org/r/72252/#comment308448>
Please change the logging level to warn. Also, do not print exception as it will clutter up output log without adding much help to debug the issue. Include "continuing" to the log message.
- Abhay Kulkarni
On March 30, 2020, 3:49 p.m., Mahesh Bandal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72252/
> -----------------------------------------------------------
>
> (Updated March 30, 2020, 3:49 p.m.)
>
>
> Review request for ranger, Ankita Sinha, Gautam Borad, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2764
> https://issues.apache.org/jira/browse/RANGER-2764
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The issue came when Role API was called in multi thread and two separate threads were trying to INSERT same roletype, which has unique constraint. ie 'state_name' column in 'x_ranger_global_state' table. To handle such scenarios, loading 'x_ranger_global_state' table with initial version values during initial setup only.
> Also OptimisticLockException occurs for RoleCreation and RoleDeletion when two threads are trying to update version column of x_ranger_global_state table. To solve this proble I have used RangerTransactionSynchronizationAdapter class for updating version in synchonised way.
>
>
> Diffs
> -----
>
> security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2d93fc429
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 38e546a56
> security-admin/db/oracle/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 7c2ada55e
> security-admin/db/postgres/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 7b0509cc0
> security-admin/db/sqlanywhere/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 25b3bed34
> security-admin/db/sqlserver/patches/046-insert-statename-in-x-ranger-global-state.sql PRE-CREATION
> security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java c4a32e46e
> security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java b979459e8
>
>
> Diff: https://reviews.apache.org/r/72252/diff/5/
>
>
> Testing
> -------
>
> 1. Fresh(Initial) Setup : Created new environment for ranger master. So now patch execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
>
> 2. Upgrade ranger : Created new environment for ranger master branch and ranger-2.0 branch. Run setup.sh file for ranger-2.0. Initially no values in 'x_ranger_global_state' table. Now setting up master branch with upgraded patch and run setup.sh on master branch where db points to same schema as ranger-2.0. db file '046-insert-statename-in-x-ranger-global-state.sql' execute successfully and 'x_ranger_global_state' table have intial values with respected 'staste_name' columns with default 'version=1'.
> 3. Created/Deleted 300 roles simultaneously. Roles are successfully created but version update fails at some point with OptimisticLockException. Handled exception and success response retured.
>
> After applying patch; created/deleted multiple roles. 'state_name' column updates successfully.
>
>
> Thanks,
>
> Mahesh Bandal
>
>