[jira] [Commented] (COUCHDB-2724) Batch rows in streaming responses to improve throughput

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/COUCHDB-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597000#comment-14597000 ] 

ASF subversion and git services commented on COUCHDB-2724:
----------------------------------------------------------

Commit f45c8b2c4262c682000b5e2ac50d16c9cade3ec6 in couchdb-chttpd's branch refs/heads/2724-chunked-buffering from [~kocolosk]
[ https://git-wip-us.apache.org/repos/asf?p=couchdb-chttpd.git;h=f45c8b2 ]

Use a record for changes_callback accumulator

This change allows us to evolve the accumulator in a less-brittle way
and sets the stage for new data to be held in the accumulator to address
COUCHDB-2724.

In the course of this change I also switched the feed labels from strings
to atoms (they're only used for pattern matching in the accumulator, and
multiple matches are executed for every row in the feed, so it seemed
silly to be using Erlang lists for that comparison), and I explicitly
indicated when we start a chunked response instead of guessing it
heuristically based on other contents in the accumulator.

COUCHDB-2724


> Batch rows in streaming responses to improve throughput
> -------------------------------------------------------
>
>                 Key: COUCHDB-2724
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-2724
>             Project: CouchDB
>          Issue Type: Improvement
>      Security Level: public(Regular issues) 
>          Components: Database Core, HTTP Interface
>            Reporter: Adam Kocoloski
>            Assignee: Adam Kocoloski
>
> [~tonysun83] showed me some profiling of the {{_changes}} feed which indicated that the coordinator process was spending about 1/3 of its time executing inside {{send_delayed_chunk}}. We can reduce the number of invocations of this function by buffering individual rows until we reach a (configurable) threshold for sending the data out the wire.
> We'll of course want to be careful about continuous feeds; if we're in the "slow drip" portion of the feed we'll obviously want to emit right away instead of adding latency unnecessarily.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)