You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jason Smith <jh...@couch.io> on 2010/08/19 23:27:36 UTC
No heartbeat during filtered replication
Not sure if this is a bug yet so I'm running it by the list.
I believe couchdb is sometimes not sending a heartbeat during _changes. For
/_changes?feed=continuous&filter=ddoc/a_filter&heartbeat=1000&since=0
If the database has millions of documents but the filter only returns
a very small number of them, considerable time will pass between
change updates, possibly well longer than the requested heartbeat. It
is not easy to tell whether the server or network link is down, or
whether there is hardcore filtering action.
I glanced at couch_changes.erl and keep_sending_changes/7 seems to try
to send the entire change set to date, and only then start sending
heartbeats after a timeout. Is this a correct assessment? If so I'll
put it in JIRA.
It would be nice for an AJAX client to simply specify a heartbeat and
then setTimeout(die, heartbeat*1.1) or whatever. If the timeout fires
before data arrives, that situation can be handled.
Thanks!
--
Jason Smith
Couchio Hosting
Re: No heartbeat during filtered replication
Posted by Sebastian Cohnen <se...@googlemail.com>.
could this be related to the replication issues I was having? it was a quite a big database and a slow connection (5 mbit)...
Am 19.08.2010 um 23:27 schrieb Jason Smith <jh...@couch.io>:
> Not sure if this is a bug yet so I'm running it by the list.
>
> I believe couchdb is sometimes not sending a heartbeat during _changes. For
>
> /_changes?feed=continuous&filter=ddoc/a_filter&heartbeat=1000&since=0
>
> If the database has millions of documents but the filter only returns
> a very small number of them, considerable time will pass between
> change updates, possibly well longer than the requested heartbeat. It
> is not easy to tell whether the server or network link is down, or
> whether there is hardcore filtering action.
>
> I glanced at couch_changes.erl and keep_sending_changes/7 seems to try
> to send the entire change set to date, and only then start sending
> heartbeats after a timeout. Is this a correct assessment? If so I'll
> put it in JIRA.
> It would be nice for an AJAX client to simply specify a heartbeat and
> then setTimeout(die, heartbeat*1.1) or whatever. If the timeout fires
> before data arrives, that situation can be handled.
>
> Thanks!
>
> --
> Jason Smith
> Couchio Hosting
Re: No heartbeat during filtered replication
Posted by Benoit Chesneau <bc...@gmail.com>.
On Thu, Aug 19, 2010 at 11:27 PM, Jason Smith <jh...@couch.io> wrote:
> Not sure if this is a bug yet so I'm running it by the list.
>
> I believe couchdb is sometimes not sending a heartbeat during _changes. For
>
> /_changes?feed=continuous&filter=ddoc/a_filter&heartbeat=1000&since=0
>
> If the database has millions of documents but the filter only returns
> a very small number of them, considerable time will pass between
> change updates, possibly well longer than the requested heartbeat. It
> is not easy to tell whether the server or network link is down, or
> whether there is hardcore filtering action.
>
> I glanced at couch_changes.erl and keep_sending_changes/7 seems to try
> to send the entire change set to date, and only then start sending
> heartbeats after a timeout. Is this a correct assessment? If so I'll
> put it in JIRA.
> It would be nice for an AJAX client to simply specify a heartbeat and
> then setTimeout(die, heartbeat*1.1) or whatever. If the timeout fires
> before data arrives, that situation can be handled.
>
> Thanks!
>
> --
> Jason Smith
> Couchio Hosting
>
heartbeat=true ?
Re: No heartbeat during filtered replication
Posted by Matthew Sinclair-Day <ms...@gmail.com>.
Just wanted to say I am seeing similar problems when couch is
under constant or moderate bursts of load. It seems the change
filter cannot keep up and the heartbeat is not sent. In my
system, on the client side of the change listener, the silence
is interpreted as a dead socket/read timeout causing the client
to drop and reestablish the listener feed. Based on the
documentation, I would expect a heartbeat to be sent even in
situations when data were still being processed through the
filter pipeline.
Matt
On 8/20/10 at 5:27 PM, jhs@couch.io (Jason Smith) wrote:
>Not sure if this is a bug yet so I'm running it by the list.
>
>I believe couchdb is sometimes not sending a heartbeat during _changes. For
>
> /_changes?feed=continuous&filter=ddoc/a_filter&heartbeat=1000&since=0
>
>If the database has millions of documents but the filter only returns
>a very small number of them, considerable time will pass between
>change updates, possibly well longer than the requested heartbeat. It
>is not easy to tell whether the server or network link is down, or
>whether there is hardcore filtering action.
>
>I glanced at couch_changes.erl and keep_sending_changes/7 seems to try
>to send the entire change set to date, and only then start sending
>heartbeats after a timeout. Is this a correct assessment? If so I'll
>put it in JIRA.
>It would be nice for an AJAX client to simply specify a heartbeat and
>then setTimeout(die, heartbeat*1.1) or whatever. If the timeout fires
>before data arrives, that situation can be handled.
>
>Thanks!
>
>--
>Jason Smith
>Couchio Hosting