You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Bob Dionne (JIRA)" <ji...@apache.org> on 2012/10/31 17:25:13 UTC
[jira] [Created] (COUCHDB-1583) improve the flow control in ibrowse
streams
Bob Dionne created COUCHDB-1583:
-----------------------------------
Summary: improve the flow control in ibrowse streams
Key: COUCHDB-1583
URL: https://issues.apache.org/jira/browse/COUCHDB-1583
Project: CouchDB
Issue Type: Improvement
Components: Replication
Reporter: Bob Dionne
Assignee: Adam Kocoloski
Fix For: 1.3
The flow control in ibrowse's async response streams is tricky. We call
stream_next to pull more data off the socket, but it seems that ibrowse
will sometimes split that data into multiple messages. If we call
stream_next for each message we process we end up with an overflowing
mailbox.
This commit[1] changes the consumer so that it clears out the mailbox before
calling stream_next.
[1] 8ccf696f833a0f0a453d
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1583) improve the flow control in
ibrowse streams
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski resolved COUCHDB-1583.
-------------------------------------
Resolution: Fixed
> improve the flow control in ibrowse streams
> -------------------------------------------
>
> Key: COUCHDB-1583
> URL: https://issues.apache.org/jira/browse/COUCHDB-1583
> Project: CouchDB
> Issue Type: Improvement
> Components: Replication
> Reporter: Bob Dionne
> Assignee: Adam Kocoloski
> Fix For: 1.3
>
>
> The flow control in ibrowse's async response streams is tricky. We call
> stream_next to pull more data off the socket, but it seems that ibrowse
> will sometimes split that data into multiple messages. If we call
> stream_next for each message we process we end up with an overflowing
> mailbox.
> This commit[1] changes the consumer so that it clears out the mailbox before
> calling stream_next.
> [1] 8ccf696f833a0f0a453d
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1583) improve the flow control in
ibrowse streams
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488134#comment-13488134 ]
Adam Kocoloski commented on COUCHDB-1583:
-----------------------------------------
Thanks for opening this ticket Bob. Sorry for not firing off a topic branch sooner. When we started using the new replicator internally at Cloudant we noticed that long-running replications would ultimately consume a ton of memory on the node mediating the replication and crash. I tracked it down to the mailbox for the process consuming the _changes feed over HTTP, and from there to a misuse of ibrowse's stream_next API. Admittedly the API is super-confusing; it should have a big read warning box telling you that you may receive multiple messages for each invocation of the function.
The patch is working well for us, and given that the commit landed directly on master I'm going to go ahead and resolve this one.
> improve the flow control in ibrowse streams
> -------------------------------------------
>
> Key: COUCHDB-1583
> URL: https://issues.apache.org/jira/browse/COUCHDB-1583
> Project: CouchDB
> Issue Type: Improvement
> Components: Replication
> Reporter: Bob Dionne
> Assignee: Adam Kocoloski
> Fix For: 1.3
>
>
> The flow control in ibrowse's async response streams is tricky. We call
> stream_next to pull more data off the socket, but it seems that ibrowse
> will sometimes split that data into multiple messages. If we call
> stream_next for each message we process we end up with an overflowing
> mailbox.
> This commit[1] changes the consumer so that it clears out the mailbox before
> calling stream_next.
> [1] 8ccf696f833a0f0a453d
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira