You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Pradeep Agrawal <pr...@gmail.com> on 2022/01/06 18:08:48 UTC
Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/
-----------------------------------------------------------
Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Bugs: RANGER-3576
https://issues.apache.org/jira/browse/RANGER-3576
Repository: ranger
Description
-------
**Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
**Proposed solution:** x_user table user's creation request also can be done via a separate transaction mechanism and overall process can be guarded and retried if the request fails.
Diffs
-----
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
Diff: https://reviews.apache.org/r/73790/diff/1/
Testing
-------
Pending.
Note: Its difficult to reproduce the issue as its intermittent, so unable to figure out that the approach is resolving the issue or not. will retry the patch and update the details here.
Thanks,
Pradeep Agrawal
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Pradeep Agrawal <pr...@gmail.com>.
> On Jan. 6, 2022, 6:54 p.m., Abhay Kulkarni wrote:
> > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
> > Line 2574 (original), 2577 (patched)
> > <https://reviews.apache.org/r/73790/diff/1/?file=2257979#file2257979line2579>
> >
> > Should this condition be as follows? Please review.
> >
> > if (xXPortalUser != null && xxUser == null) {
Yes, we can consider this. I don't see any problem with this. Thanks for reviewing the patch.
- Pradeep
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/#review223939
-----------------------------------------------------------
On Jan. 6, 2022, 6:08 p.m., Pradeep Agrawal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73790/
> -----------------------------------------------------------
>
> (Updated Jan. 6, 2022, 6:08 p.m.)
>
>
> Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3576
> https://issues.apache.org/jira/browse/RANGER-3576
>
>
> Repository: ranger
>
>
> Description
> -------
>
> **Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
> It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
>
> **Proposed solution:** x_user table user's creation request also can be done via a separate transaction mechanism and overall process can be guarded and retried if the request fails.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
>
>
> Diff: https://reviews.apache.org/r/73790/diff/1/
>
>
> Testing
> -------
>
> Pending.
>
> Note: Its difficult to reproduce the issue as its intermittent, so unable to figure out that the approach is resolving the issue or not. will retry the patch and update the details here.
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/#review223939
-----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
Line 2574 (original), 2577 (patched)
<https://reviews.apache.org/r/73790/#comment312982>
Should this condition be as follows? Please review.
if (xXPortalUser != null && xxUser == null) {
- Abhay Kulkarni
On Jan. 6, 2022, 6:08 p.m., Pradeep Agrawal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73790/
> -----------------------------------------------------------
>
> (Updated Jan. 6, 2022, 6:08 p.m.)
>
>
> Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3576
> https://issues.apache.org/jira/browse/RANGER-3576
>
>
> Repository: ranger
>
>
> Description
> -------
>
> **Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
> It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
>
> **Proposed solution:** x_user table user's creation request also can be done via a separate transaction mechanism and overall process can be guarded and retried if the request fails.
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
>
>
> Diff: https://reviews.apache.org/r/73790/diff/1/
>
>
> Testing
> -------
>
> Pending.
>
> Note: Its difficult to reproduce the issue as its intermittent, so unable to figure out that the approach is resolving the issue or not. will retry the patch and update the details here.
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/#review223951
-----------------------------------------------------------
Ship it!
Ship It!
- Abhay Kulkarni
On Jan. 11, 2022, 6:30 p.m., Pradeep Agrawal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73790/
> -----------------------------------------------------------
>
> (Updated Jan. 11, 2022, 6:30 p.m.)
>
>
> Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3576
> https://issues.apache.org/jira/browse/RANGER-3576
>
>
> Repository: ranger
>
>
> Description
> -------
>
> **Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
> It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
>
> **Proposed solution:** Considered the approach from RANGER-3578 (RR https://reviews.apache.org/r/73792/)
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
> security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 945bba48d
> ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/GroupUserInfo.java ffb6625f5
>
>
> Diff: https://reviews.apache.org/r/73790/diff/6/
>
>
> Testing
> -------
>
> 1) Tested service and policy creation manually and its creating the non existence user successfully.
> 2) Tested stability-tests script with 1000 iteration and 5 clients(parallel requests), script run successfully, however during the run saw error messages like below frequently but it should be okay i think. Please review and provide suggestions.
>
> 2022-01-10 09:02:17,210 [http-nio-6080-exec-9] ERROR org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator (PolicyRefUpdater.java:400) - serviceConfigUser:[test-ra-user-4999] creation failed
> 2022-01-10 09:02:17,211 [http-nio-6080-exec-9] ERROR org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1 (RangerTransactionSynchronizationAdapter.java:188) - Failed to execute runnable org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator@1e13247a
> java.lang.RuntimeException: Failed to associate USER:test-ra-user-4999 with policy id:[14997]
> at org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator.run(PolicyRefUpdater.java:301)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1.doInTransaction(RangerTransactionSynchronizationAdapter.java:178)
> at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.runRunnables(RangerTransactionSynchronizationAdapter.java:171)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.afterCompletion(RangerTransactionSynchronizationAdapter.java:131)
> at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:172)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:977)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:952)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:785)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
> at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
> at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
> at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
> at org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$1df651f2.createPolicy(<generated>)
> at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
> at org.apache.ranger.security.web.filter.RangerSecurityContextFormationFilter.doFilter(RangerSecurityContextFormationFilter.java:142)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(RangerCSRFPreventionFilter.java:226)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.handleHttpInteraction(RangerCSRFPreventionFilter.java:171)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.doFilter(RangerCSRFPreventionFilter.java:181)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerKRBAuthenticationFilter.doFilter(RangerKRBAuthenticationFilter.java:426)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerSSOAuthenticationFilter.doFilter(RangerSSOAuthenticationFilter.java:257)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:178)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
> at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
> at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
> at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
> at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
> at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1650)
> at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
> at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
> at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748)
> 2022-01-10 09:02:17,217 [http-nio-6080-exec-4] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-5000
> 2022-01-10 09:02:17,218 [http-nio-6080-exec-9] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-4999
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/
-----------------------------------------------------------
(Updated Jan. 11, 2022, 6:30 p.m.)
Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Changes
-------
updated error messages
Bugs: RANGER-3576
https://issues.apache.org/jira/browse/RANGER-3576
Repository: ranger
Description
-------
**Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
**Proposed solution:** Considered the approach from RANGER-3578 (RR https://reviews.apache.org/r/73792/)
Diffs (updated)
-----
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 945bba48d
ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/GroupUserInfo.java ffb6625f5
Diff: https://reviews.apache.org/r/73790/diff/6/
Changes: https://reviews.apache.org/r/73790/diff/5-6/
Testing
-------
1) Tested service and policy creation manually and its creating the non existence user successfully.
2) Tested stability-tests script with 1000 iteration and 5 clients(parallel requests), script run successfully, however during the run saw error messages like below frequently but it should be okay i think. Please review and provide suggestions.
2022-01-10 09:02:17,210 [http-nio-6080-exec-9] ERROR org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator (PolicyRefUpdater.java:400) - serviceConfigUser:[test-ra-user-4999] creation failed
2022-01-10 09:02:17,211 [http-nio-6080-exec-9] ERROR org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1 (RangerTransactionSynchronizationAdapter.java:188) - Failed to execute runnable org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator@1e13247a
java.lang.RuntimeException: Failed to associate USER:test-ra-user-4999 with policy id:[14997]
at org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator.run(PolicyRefUpdater.java:301)
at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1.doInTransaction(RangerTransactionSynchronizationAdapter.java:178)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.runRunnables(RangerTransactionSynchronizationAdapter.java:171)
at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.afterCompletion(RangerTransactionSynchronizationAdapter.java:131)
at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:172)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:977)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:952)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:785)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$1df651f2.createPolicy(<generated>)
at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
at org.apache.ranger.security.web.filter.RangerSecurityContextFormationFilter.doFilter(RangerSecurityContextFormationFilter.java:142)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(RangerCSRFPreventionFilter.java:226)
at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.handleHttpInteraction(RangerCSRFPreventionFilter.java:171)
at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.doFilter(RangerCSRFPreventionFilter.java:181)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.ranger.security.web.filter.RangerKRBAuthenticationFilter.doFilter(RangerKRBAuthenticationFilter.java:426)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.ranger.security.web.filter.RangerSSOAuthenticationFilter.doFilter(RangerSSOAuthenticationFilter.java:257)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:178)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1650)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2022-01-10 09:02:17,217 [http-nio-6080-exec-4] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-5000
2022-01-10 09:02:17,218 [http-nio-6080-exec-9] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-4999
Thanks,
Pradeep Agrawal
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/#review223948
-----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
Lines 3216 (patched)
<https://reviews.apache.org/r/73790/#comment312985>
Should be okay if trx logs are not created (in case of failure) ?
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
Lines 3225 (patched)
<https://reviews.apache.org/r/73790/#comment312986>
Should be okay if permissions are not assigned to default modules (in case of failure) ?
- Pradeep Agrawal
On Jan. 11, 2022, 8:15 a.m., Pradeep Agrawal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73790/
> -----------------------------------------------------------
>
> (Updated Jan. 11, 2022, 8:15 a.m.)
>
>
> Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3576
> https://issues.apache.org/jira/browse/RANGER-3576
>
>
> Repository: ranger
>
>
> Description
> -------
>
> **Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
> It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
>
> **Proposed solution:** Considered the approach from RANGER-3578 (RR https://reviews.apache.org/r/73792/)
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
> security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 945bba48d
> ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/GroupUserInfo.java ffb6625f5
>
>
> Diff: https://reviews.apache.org/r/73790/diff/5/
>
>
> Testing
> -------
>
> 1) Tested service and policy creation manually and its creating the non existence user successfully.
> 2) Tested stability-tests script with 1000 iteration and 5 clients(parallel requests), script run successfully, however during the run saw error messages like below frequently but it should be okay i think. Please review and provide suggestions.
>
> 2022-01-10 09:02:17,210 [http-nio-6080-exec-9] ERROR org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator (PolicyRefUpdater.java:400) - serviceConfigUser:[test-ra-user-4999] creation failed
> 2022-01-10 09:02:17,211 [http-nio-6080-exec-9] ERROR org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1 (RangerTransactionSynchronizationAdapter.java:188) - Failed to execute runnable org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator@1e13247a
> java.lang.RuntimeException: Failed to associate USER:test-ra-user-4999 with policy id:[14997]
> at org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator.run(PolicyRefUpdater.java:301)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1.doInTransaction(RangerTransactionSynchronizationAdapter.java:178)
> at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.runRunnables(RangerTransactionSynchronizationAdapter.java:171)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.afterCompletion(RangerTransactionSynchronizationAdapter.java:131)
> at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:172)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:977)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:952)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:785)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
> at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
> at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
> at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
> at org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$1df651f2.createPolicy(<generated>)
> at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
> at org.apache.ranger.security.web.filter.RangerSecurityContextFormationFilter.doFilter(RangerSecurityContextFormationFilter.java:142)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(RangerCSRFPreventionFilter.java:226)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.handleHttpInteraction(RangerCSRFPreventionFilter.java:171)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.doFilter(RangerCSRFPreventionFilter.java:181)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerKRBAuthenticationFilter.doFilter(RangerKRBAuthenticationFilter.java:426)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerSSOAuthenticationFilter.doFilter(RangerSSOAuthenticationFilter.java:257)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:178)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
> at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
> at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
> at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
> at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
> at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1650)
> at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
> at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
> at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748)
> 2022-01-10 09:02:17,217 [http-nio-6080-exec-4] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-5000
> 2022-01-10 09:02:17,218 [http-nio-6080-exec-9] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-4999
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/
-----------------------------------------------------------
(Updated Jan. 11, 2022, 8:15 a.m.)
Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Changes
-------
Addressed review comment
Bugs: RANGER-3576
https://issues.apache.org/jira/browse/RANGER-3576
Repository: ranger
Description
-------
**Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
**Proposed solution:** Considered the approach from RANGER-3578 (RR https://reviews.apache.org/r/73792/)
Diffs (updated)
-----
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 945bba48d
ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/GroupUserInfo.java ffb6625f5
Diff: https://reviews.apache.org/r/73790/diff/5/
Changes: https://reviews.apache.org/r/73790/diff/4-5/
Testing
-------
1) Tested service and policy creation manually and its creating the non existence user successfully.
2) Tested stability-tests script with 1000 iteration and 5 clients(parallel requests), script run successfully, however during the run saw error messages like below frequently but it should be okay i think. Please review and provide suggestions.
2022-01-10 09:02:17,210 [http-nio-6080-exec-9] ERROR org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator (PolicyRefUpdater.java:400) - serviceConfigUser:[test-ra-user-4999] creation failed
2022-01-10 09:02:17,211 [http-nio-6080-exec-9] ERROR org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1 (RangerTransactionSynchronizationAdapter.java:188) - Failed to execute runnable org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator@1e13247a
java.lang.RuntimeException: Failed to associate USER:test-ra-user-4999 with policy id:[14997]
at org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator.run(PolicyRefUpdater.java:301)
at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1.doInTransaction(RangerTransactionSynchronizationAdapter.java:178)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.runRunnables(RangerTransactionSynchronizationAdapter.java:171)
at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.afterCompletion(RangerTransactionSynchronizationAdapter.java:131)
at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:172)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:977)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:952)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:785)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$1df651f2.createPolicy(<generated>)
at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
at org.apache.ranger.security.web.filter.RangerSecurityContextFormationFilter.doFilter(RangerSecurityContextFormationFilter.java:142)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(RangerCSRFPreventionFilter.java:226)
at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.handleHttpInteraction(RangerCSRFPreventionFilter.java:171)
at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.doFilter(RangerCSRFPreventionFilter.java:181)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.ranger.security.web.filter.RangerKRBAuthenticationFilter.doFilter(RangerKRBAuthenticationFilter.java:426)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.ranger.security.web.filter.RangerSSOAuthenticationFilter.doFilter(RangerSSOAuthenticationFilter.java:257)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:178)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1650)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2022-01-10 09:02:17,217 [http-nio-6080-exec-4] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-5000
2022-01-10 09:02:17,218 [http-nio-6080-exec-9] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-4999
Thanks,
Pradeep Agrawal
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/#review223946
-----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
Lines 3184 (patched)
<https://reviews.apache.org/r/73790/#comment312984>
It appears that there are many try .. catch blocks in the run() method. If all exceptions are caught and not rethrown, then how would the RangerTransactionSynchronizationAdapter.runRunnables() method retry the exception cases?
In general, the task scheduled to run after commit should throw exception when it cannot complete the work.
- Abhay Kulkarni
On Jan. 7, 2022, 6:23 a.m., Pradeep Agrawal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73790/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2022, 6:23 a.m.)
>
>
> Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3576
> https://issues.apache.org/jira/browse/RANGER-3576
>
>
> Repository: ranger
>
>
> Description
> -------
>
> **Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
> It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
>
> **Proposed solution:** Considered the approach from RANGER-3578 (RR https://reviews.apache.org/r/73792/)
>
>
> Diffs
> -----
>
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
> security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 945bba48d
> ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/GroupUserInfo.java ffb6625f5
>
>
> Diff: https://reviews.apache.org/r/73790/diff/4/
>
>
> Testing
> -------
>
> 1) Tested service and policy creation manually and its creating the non existence user successfully.
> 2) Tested stability-tests script with 1000 iteration and 5 clients(parallel requests), script run successfully, however during the run saw error messages like below frequently but it should be okay i think. Please review and provide suggestions.
>
> 2022-01-10 09:02:17,210 [http-nio-6080-exec-9] ERROR org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator (PolicyRefUpdater.java:400) - serviceConfigUser:[test-ra-user-4999] creation failed
> 2022-01-10 09:02:17,211 [http-nio-6080-exec-9] ERROR org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1 (RangerTransactionSynchronizationAdapter.java:188) - Failed to execute runnable org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator@1e13247a
> java.lang.RuntimeException: Failed to associate USER:test-ra-user-4999 with policy id:[14997]
> at org.apache.ranger.biz.PolicyRefUpdater$PolicyPrincipalAssociator.run(PolicyRefUpdater.java:301)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter$1.doInTransaction(RangerTransactionSynchronizationAdapter.java:178)
> at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.runRunnables(RangerTransactionSynchronizationAdapter.java:171)
> at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.afterCompletion(RangerTransactionSynchronizationAdapter.java:131)
> at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:172)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:977)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:952)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:785)
> at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
> at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
> at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
> at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
> at org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$1df651f2.createPolicy(<generated>)
> at sun.reflect.GeneratedMethodAccessor279.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
> at org.apache.ranger.security.web.filter.RangerSecurityContextFormationFilter.doFilter(RangerSecurityContextFormationFilter.java:142)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter$ServletFilterHttpInteraction.proceed(RangerCSRFPreventionFilter.java:226)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.handleHttpInteraction(RangerCSRFPreventionFilter.java:171)
> at org.apache.ranger.security.web.filter.RangerCSRFPreventionFilter.doFilter(RangerCSRFPreventionFilter.java:181)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerKRBAuthenticationFilter.doFilter(RangerKRBAuthenticationFilter.java:426)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.apache.ranger.security.web.filter.RangerSSOAuthenticationFilter.doFilter(RangerSSOAuthenticationFilter.java:257)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:178)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
> at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
> at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
> at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364)
> at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:624)
> at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
> at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1650)
> at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
> at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
> at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748)
> 2022-01-10 09:02:17,217 [http-nio-6080-exec-4] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-5000
> 2022-01-10 09:02:17,218 [http-nio-6080-exec-9] INFO org.apache.ranger.biz.XUserMgr$ExternalUserCreator (XUserMgr.java:3209) - User created: test-ra-user-4999
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Re: Review Request 73790: RANGER-3576: service creation is failing intermittently due to DB unique key constraint violation
Posted by Pradeep Agrawal <pr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73790/
-----------------------------------------------------------
(Updated Jan. 7, 2022, 6:23 a.m.)
Review request for ranger, Dineshkumar Yadav, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
Changes
-------
Address review comments.
Bugs: RANGER-3576
https://issues.apache.org/jira/browse/RANGER-3576
Repository: ranger
Description
-------
**Problem Statement:** Sometime parallel service creation request may fail due to DB unique key constraint on x_user table as first request to create a user may succeed for the same user while other requests might fail to create it as they dont have information about user creation event by another request.
It seems issue has been handled for x_portal_user table because the user creation process happens through a separate transaction mechanism, while its failing for x_user request as that is not done in a separate transaction. When x_user creation request fails from db end then the issue is not handled by jpa/eclipselink/spring and it seems overall transaction is aborted/roledback, which causes the failure of service creation.
**Proposed solution:** x_user table user's creation request also can be done via a separate transaction mechanism and overall process can be guarded and retried if the request fails.
Diffs (updated)
-----
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e
Diff: https://reviews.apache.org/r/73790/diff/2/
Changes: https://reviews.apache.org/r/73790/diff/1-2/
Testing
-------
Pending.
Note: Its difficult to reproduce the issue as its intermittent, so unable to figure out that the approach is resolving the issue or not. will retry the patch and update the details here.
Thanks,
Pradeep Agrawal