[jira] [Commented] (DIRMINA-1039) Response messages queue up on the server side waiting to be written to socket, while the server continues to read more request messages, causing out of heap memory

["Maria Petridean (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DIRMINA-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364348#comment-15364348 ] 

Maria Petridean commented on DIRMINA-1039:
------------------------------------------

This issue was not caused by a slow-reading client. The client is able to both read and write messages fast. This issue addresses a problem on the server side, after the changes in mina 2. With mina 1, testing the server in the exact same conditions behaves correctly.

When writing responses, if we assume the response is small in size and it is followed by a (empty message) marker in the queue (which is part of mina code). When writing, in AbstractPollingIoProcessor flushNow method, there is a check that (localWrittenBytes == 0), which holds true for the marker-WriteRequests. The comment in the code at this point, suggests that this is interpreted as kernel buffer being full, which is false. This check is true because we just processed a message marker. The return statement in this case will cause exiting the write-loop and starting to read messages again. It seems to me that this check here should be (localWrittenBytes == 0 && isNotAnEmptyMessage). And also add another check below: if ((localWrittenBytes == 0 && isAnEmptyMessage), then simply set the session as interested in write and continue the write cycle. The loop will then exit the write cycle when will be the case (when the kernel buffer is full, or when the max written bytes is reached, etc).
Otherwise, by design, mina allows reading multiple messages during one read-phase (which depends on the buffer size, message size) and then only allows writing one response during the write-phase and return to read-phase. It seems to me this can be fixed (I'm sure my suggestion above is not the only solution). I understand that this will not fix also the problem you [~elecharny] mentioned - which has to be handled at application implementation level. 

I am not sure the title of this issue is the best to describe this issue, this is why i tried to detail in this comment the problem and the possible fix. What do you think?

> Response messages queue up on the server side waiting to be written to socket, while the server continues to read more request messages, causing out of heap memory
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DIRMINA-1039
>                 URL: https://issues.apache.org/jira/browse/DIRMINA-1039
>             Project: MINA
>          Issue Type: Bug
>          Components: Core
>            Reporter: Maria Petridean
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> One case in which this bug reproduces is by using a client which generates a heavy request-load. The mina thread which processes both reads and writes - exits the write cycle after processing every empty marker (the WriteRequest which wraps an empty buffer, acting as a message marker). This will result in the thread resuming the read cycle, hence reading more client request messages. After a few minutes, the number of read messages is much larger than the number of written response messages, even though the responses are waiting in the queue, ready to be written to socket.
> To solve this, the sever shouldn't exit the write cycle after processing every marker WriteRequest. This way the ratio between the read and written messages will be more balanced; this will avoid the heap memory getting full and causing server degradation.
> Also, an improvement can be considered here to avoid using the same single thread for both reads and writes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)