[jira] Updated: (CASSANDRA-220) TCP writes get stuck

["Jun Rao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao updated CASSANDRA-220:
------------------------------

    Attachment: issue220.patchv1

Attache a patch to trunk by expanding the synchronization scope in TcpConnection.doPendingWrites().

We need to patch to 0.3 branch too.

Also, we need to revisit the code in TcpConnection. Under heavy writes, is it better to keep accumulating writes until we run out of memory, or to put a cap on the write buffer and simply block and possibly timeout the writer.

> TCP writes get stuck
> --------------------
>
>                 Key: CASSANDRA-220
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-220
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: trunk
>            Reporter: Jun Rao
>            Assignee: Jun Rao
>             Fix For: trunk
>
>         Attachments: issue220.patchv1
>
>
> In our test cluster, I observed that on some nodes, TCP writes get accumulated in TcpConnection.pendingWrites. However, the selector never gets a ready to write event. As a result, all writes get stuck. This is because write(message) and doPendingWrites() are not fully synchronized.  This following situation can happen:
> 1. write(message) adds stuff to pendingWrites.
> 2. a ready to write event happens; in doPendingWrites() buffered requests are written to socket
> 3. another write request happens, in write(message), the test for pendingWrites.isEmpty() is false
> 4. doPendingWrites() finishes writing all buffered request to socket
> 5. in write(message), the new request is added to pendingWrites
> Now, ready to write events will never happen again and all write requests get stuck in pendingWrites.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.