[jira] [Comment Edited] (ZOOKEEPER-3986) ConcurrentModificationException in PrepRequestProcessor.checkACL

["Jonathan Halterman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221933#comment-17221933 ] 

Jonathan Halterman edited comment on ZOOKEEPER-3986 at 10/28/20, 3:24 AM:
--------------------------------------------------------------------------

Since this was discovered against 3.5, checking master, it looks like this may have been fixed indirectly by Jordan: [https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/ServerCnxn.java#L63]

 

I am not sure if there's a good way to reproduce this exact issue, but I suspect switching to a ConcurrentHashMap there would avoid the ConcurrentModificationException.


was (Author: jhalterman):
Checking master, it looks like this may have been fixed indirectly by Jordan: [https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/ServerCnxn.java#L63]

 

I am not sure if there's a good way to reproduce this exact issue, but I suspect switching to a ConcurrentHashMap there would avoid the ConcurrentModificationException.

> ConcurrentModificationException in PrepRequestProcessor.checkACL
> ----------------------------------------------------------------
>
>                 Key: ZOOKEEPER-3986
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3986
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: server
>    Affects Versions: 3.5.5, 3.7.0, 3.6.2
>            Reporter: Jonathan Halterman
>            Priority: Major
>             Fix For: 3.7.0
>
>
> We have some Zookeeper clusters under pretty heavy load that are occasionally throwing ConcurrentModificationException during ACL checking:
> {noformat}
>  
> ERROR [CommitProcWorkThread-17:FinalRequestProcessor@439] - Failed to process sessionid:0xa006cd06b181510 type:getChildren2 cxid:0x4dae zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a
> java.util.ConcurrentModificationException
>  at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
>  at java.util.ArrayList$Itr.next(ArrayList.java:859)
>  at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1044)
>  at org.apache.zookeeper.server.PrepRequestProcessor.checkACL(PrepRequestProcessor.java:321)
>  at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:383)
>  at org.apache.zookeeper.server.quorum.CommitProcessor$CommitWorkRequest.doWork(CommitProcessor.java:297)
>  at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:162)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
> {noformat}
> This appears to be caused by auths changing while we're processing a request, such as here:
> [https://github.com/apache/zookeeper/blob/branch-3.5/zookeeper-server/src/main/java/org/apache/zookeeper/server/ServerCnxn.java#L101-L110]
> The result of this appears to be that occasionally requests fail unexpectedly.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)