[jira] Updated: (CASSANDRA-392) Deadlock with SelectorManager.doProcess and TcpConnection.write

["Sammy Yu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sammy Yu updated CASSANDRA-392:
-------------------------------

    Attachment: 0001-CASSANDRA-392-Moved-turnOnInterestOps-outside-of-syn.patch

For TcpConnection.write(Message) moved turnOnInterestOps outside of synchronized block so that call only depends on having the SelectionKey monitor.



> Deadlock with SelectorManager.doProcess and TcpConnection.write
> ---------------------------------------------------------------
>
>                 Key: CASSANDRA-392
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-392
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.4
>            Reporter: Sammy Yu
>            Assignee: Sammy Yu
>             Fix For: 0.4
>
>         Attachments: 0001-CASSANDRA-392-Moved-turnOnInterestOps-outside-of-syn.patch
>
>
> We ran into a deadlock last night:
> Name: MESSAGE-SERIALIZER-POOL:2
> State: BLOCKED on sun.nio.ch.SelectionKeyImpl@2e257f1b owned by: TCP Selector Manager
> Total blocked: 1  Total waited: 1
> Stack trace: 
> org.apache.cassandra.net.SelectionKeyHandler.turnOnInterestOps(SelectionKeyHandler.java:73)
> org.apache.cassandra.net.TcpConnection.write(TcpConnection.java:186)
>    - locked org.apache.cassandra.net.TcpConnection@5ab9f791
> org.apache.cassandra.net.MessageSerializationTask.run(MessageSerializationTask.java:67)
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> java.lang.Thread.run(Thread.java:619)
> Name: TCP Selector Manager
> State: BLOCKED on org.apache.cassandra.net.TcpConnection@5ab9f791 owned by: MESSAGE-SERIALIZER-POOL:2
> Total blocked: 2  Total waited: 0
> Stack trace: 
> org.apache.cassandra.net.TcpConnection.connect(TcpConnection.java:360)
> org.apache.cassandra.net.SelectorManager.doProcess(SelectorManager.java:131)
>    - locked sun.nio.ch.SelectionKeyImpl@2e257f1b
> org.apache.cassandra.net.SelectorManager.run(SelectorManager.java:98)
> The SelectionManager.doProcess acquires a monitor on the SelectionKey and then calls methods such as TcpConnection.connect(SelectionKey key) which obtains a monitor for the TcpConnection object itself.  Another task eg: MessageSerializationTask can come along and call write(Message message) which obtains a monitor for the TCPConnection first and then on calls to turnOnInterestOps tries to obtain the monitor for the SelectionKey which causes the deadlock.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.