You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Bob Dionne (JIRA)" <ji...@apache.org> on 2011/09/19 20:53:09 UTC
[jira] [Created] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
heartbeats skipped when continuous changes feed filter function produces no results
-----------------------------------------------------------------------------------
Key: COUCHDB-1289
URL: https://issues.apache.org/jira/browse/COUCHDB-1289
Project: CouchDB
Issue Type: Bug
Components: Database Core
Reporter: Bob Dionne
Priority: Minor
if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Attachment: (was: 0002-Failing-etap-for-heartbeats-skipped.patch)
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157473#comment-13157473 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Thanks for the review Filipe. I pushed your changes on the branch[1]. I'll add an additional etap for this case and rework the two patches when I return later this morning. It's nice to get the maybe_timeout test into changes_enumerator and out of the keep_sending_changes loop, as it makes both the continuous and normal cases the same.
[1] https://github.com/bdionne/couchdb/commit/72cd7a5a214c0f1d810d7c9d8890645
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne resolved COUCHDB-1289.
---------------------------------
Resolution: Fixed
thanks Filipe
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Labels: patch
> Fix For: 1.2
>
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157615#comment-13157615 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Add etap test for this last case[1]
[1] https://github.com/bdionne/couchdb/compare/master...1289-heartbeats-skipped2
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158376#comment-13158376 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob, just caught another case where the default TimeoutFun will be called (which stops the streaming immediately):
>From 7160021512161e3fa8f83165adc1a64285da6518 Mon Sep 17 00:00:00 2001
From: Filipe David Borba Manana <fd...@apache.org>
Date: Mon, 28 Nov 2011 12:30:50 +0000
Subject: [PATCH 2/2] Make reset_heartbeat/0 a no-op
If no heartbeat option was given, the couch_changes heartbeat
function should really be a no-op, that is, to not set the
last_changes_heartbeat to the current time, otherwise the
default TimeoutFun (when no heartbeat option is given) might
be called which causes the changes feed to stop immediately
before folding the whole seq tree.
---
src/couchdb/couch_changes.erl | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/src/couchdb/couch_changes.erl b/src/couchdb/couch_changes.erl
index f124567..72ee346 100644
--- a/src/couchdb/couch_changes.erl
+++ b/src/couchdb/couch_changes.erl
@@ -531,7 +531,12 @@ get_rest_db_updated(UserAcc) ->
end.
reset_heartbeat() ->
- put(last_changes_heartbeat,now()).
+ case get(last_changes_heartbeat) of
+ undefined ->
+ ok;
+ _ ->
+ put(last_changes_heartbeat, now())
+ end.
maybe_heartbeat(Timeout, TimeoutFun, Acc) ->
Before = get(last_changes_heartbeat),
--
1.7.4.4
Do you agree?
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Labels: patch
> Fix For: 1.2
>
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158032#comment-13158032 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Yep, so before we never called TimeoutFun for feed=normal, or better said, when folding the seq tree.
I'm finding the existence of both a timeout and heartbeat quite confusing.
But it makes sense to send heartbeats while folding the seq tree (if the client specified an heartbeat). As you said before as well, renaming "reset_timeout" to "reset_heartbeat" is saner.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108065#comment-13108065 ]
Paul Joseph Davis commented on COUCHDB-1289:
--------------------------------------------
There are two aspects to this that need to be considered. We've fixed both of these in BigCouch but I'm not sure if we've the first one.
The first one is that if you have a large range of documents (ordered by update seq) that fail to pass a filter, there will be no heartbeat sent. Basically, the changes feed thinks its making process all hunky dory but nothing is being sent, and we never checked that its taking us too long to filter all these docs.
The second version of this is slightly more insidious. Basically, the timeouts we had in BigCouch were receive timeouts in two places (no db updates, and while processing in case everything is filtered). What can end up happening is that updates to the db can end up trickling in that all fail to make it through the filter function. This means that the changes loop thinks its working again but is not hitting either timeout clause to send the heartbeat.
The clustered calls in BigCouch are different enough that the patches aren't exactly applicable, but reading the code, CouchDB is at least susceptible to the second version and possible the first as well.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158020#comment-13158020 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob, looks good.
I was testing this against a very large database (http://fdmanana.iriscouch.com/many_docs) by querying _changes without an heartbeat parameter and with a filter that returned false for the first half of the changes of the db.
It turns out that about 1 minute after the feed terminated without folding the entire seq tree (last_seq in the response was not the current db's seq number).
So what happens is that when no heartbeat is given, a default TimeoutFun which returns {stop, Acc} (always) is used:
https://github.com/fdmanana/couchdb/blob/c574d29bb92f2b61688b52e2e03548b05cc4a163/src/couchdb/couch_changes.erl#L239
So we have 2 distinct concepts in the changes feed: heartbeat and timeout.
I think renaming "maybe_timeout" to "maybe_heartbeat" makes it more clear, and call the function only if no heartbeat was specified. I made quick code change to test this:
https://github.com/fdmanana/couchdb/commit/c574d29bb92f2b61688b52e2e03548b05cc4a163
Makes sense?
thanks :)
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158338#comment-13158338 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob, +1.
You did most of the hard work, no need to set me as the author for one of them.
Lets push it to master and 1.2.x?
Thanks :)
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155283#comment-13155283 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Filipe,
I've seen these guidelines. I think this code adheres to them. If it doesn't I'll fix it. The "true" clause is aligned with the initial "Done" boolean. I think this is acceptable. I know you and Adam think these should be under the "If", similar to the "case", but there is many places in the couchdb code where this is not true. I'll put it under the if since the's the way it is elsewhere in this same file.
How about for case statements? Should the clauses line up under the "case"?
Cheers,
Bob
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155370#comment-13155370 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
cool, I'll fix it.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155261#comment-13155261 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Filipe, thanks for the comments:
3. It took me a bit to figure out how this test was working at all, so I knew I could probably simplify it further. This is better, I always forget to use some state on the heap for these cases.
2. I'll revisit this. I think you're saying I've missed some cases
1. Meh, this one is getting old. This indenting looks good to me. People claim to not care too much yet keep bringing it up. The problem I have is I like to edit using modes, not manually (which from what I can tell is how most people edit couchdb source). The other problem I have is that the only spec I know for formatting erlang is what's in the erlang.el mode. There's a claim about a couchdb style but this seems to vary with each committer. So I'm increasingly reluctant to "fix" formatting when no one seems to be able to specify the rules.
Best,
Bob
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Fix Version/s: 1.2
Labels: patch (was: )
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Labels: patch
> Fix For: 1.2
>
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158378#comment-13158378 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
(Jira messed up with the patch indentation)
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Labels: patch
> Fix For: 1.2
>
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Attachment: (was: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch)
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155293#comment-13155293 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob, the thing about the "if" is that the "true" clause is not a column multiple of 4 (it's on column 11), which violates the soft convention we have.
As for the cases, some indent the clauses to the "case" keyword, others indent it one more level (4 spaces). I think either is readable and don't care much about it unless it starts to be unreadable (not on 4 space multiples, of a big mix of both styles in the same module/region).
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155469#comment-13155469 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Filipe,
Sorry to be dense about this, but what you describe in #2 is precisely what is happening. `maybe_timeout` takes a boolean `Sent` which comes from the changes_sent field in the Accumulator. It's set depending on whether or not the filter function is successful. `maybe_timeout' then uses this to either reset the timer in the heap to now() or not depending on the timeout.
Is is easy enough for you to extend the failing test to demo this missing case?
Thanks,
Bob
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108477#comment-13108477 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Good finding Bob.
What about using a simple function that is called before processing each changes row and uses the process dictionary to reduce code size/complexity, along the lines:
maybe_timeout(TimeoutFun) ->
Now = now(),
Before = case get(changes_timeout) of
undefined ->
Now;
OldNow ->
OldNow
end,
case timer:diff(Now, Before) >= Timeout of
true ->
Acc2 = TimeoutFun(get(changes_user_acc))
put(changes_timeout(Now)),
put(changes_user_acc, Acc2);
false ->
ok
end.
Probably it misses some cases, i haven't thought about this issue before.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157464#comment-13157464 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Looks good Bob, only a small regression compared to the previous versions, which is:
The last time a row was sent is never reset to "now()" once the filter function returns true. Basically the maybe_timeout(true, ...) function clause is never used in this new version.
This could cause unnecessary timeouts when the filter functions returns false several times in a row, then returns true (1 or more times in a row) and then starts returning false again. Here's what I mean:
https://github.com/fdmanana/couchdb/commit/31c80e34875932969716791b4b8adf374d240821
A test for this case would be awesome :)
Thanks for working on this, this has been affecting many users for a long time (when doing filtered replications).
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Filipe Manana (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-1289:
-----------------------------------
Attachment: 0002-Make-reset_heartbeat-0-a-no-op.patch
Added the patch as an attachment instead
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Labels: patch
> Fix For: 1.2
>
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch, 0002-Make-reset_heartbeat-0-a-no-op.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109147#comment-13109147 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Good idea Filipe, much as I dislike using the heap, this does make the code simpler[1]. Still need the boolean `changes_sent` in the `changes_acc` to know something actually got sent, but it's better
[1] https://github.com/bdionne/couchdb/commit/8b2bf85fcade
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne reassigned COUCHDB-1289:
-----------------------------------
Assignee: Bob Dionne
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155811#comment-13155811 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob, np.
Looking at the whole context, not limited to see only the diff, I think you're right. The patch addresses the case I mentioned before, since it each tree fold has changes_sent set to false for initial accumulator and the changes_enumerator callback sets it to true once a row is accepted by the filter. However this is set only for the "continuous" clause of changes_enumerator. The other clause should set changes_set to true according to the same condition.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108432#comment-13108432 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
The issue in couchdb is that the timeout function is called in wait_db_updated and never triggered if an update occurs, which is problematic when the update leads to no changes sent. I've tried to find a minimally disruptive fix, but this is the best[1] I can do so far modulo refactoring. I think a better solution that removes the timeout function from wait_db_updated will be the way to go but this will be a more extensive refactoring. As an aside I think this line is also a bug[2] and should perhaps read like the other version of changes_enumerator
[1] https://github.com/bdionne/couchdb/commit/a4d26f9b0c82c0423521cbe
[2] https://github.com/bdionne/couchdb/blob/a4d26f9b0c82c0423521cbe5a7d5f17354aa4b9a/src/couchdb/couch_changes.erl#L363
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Attachment: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
This patch against current master fixes the heartbeat issue
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Attachment: 0002-Failing-etap-for-heartbeats-skipped.patch
0001-Ensure-heartbeats-are-not-skipped.patch
second draft of patch. Filipe, this handles both cases now and is refactored to be a bit simpler.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Attachment: 0002-Failing-etap-for-heartbeats-skipped.patch
0001-Ensure-heartbeats-are-not-skipped.patch
Filipe,
I pulled your last commit, fixed those nits, rebased, squashed and formatted two new patches. I made you author of one, since it doesn't look like git supports multiple authors. Thanks, this one was gnarly, I'm hopeful we got all the edges,
Bob
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156641#comment-13156641 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Filipe,
Ah, thanks, it looks like heartbeat was never supported in the feed="normal" case. This next draft[1] handles this case also. I cherry-picked your cleaned up etap to test with and it all passes. There is still a bug to fix and looking at get_changes_timeout I see there are other cases missing, .eg. heartbeat might be undefined and timeout infinity and I'm not dealing with {stop, UsrAcc}. I've added the Timeout and TimeoutFun to the changes_acc record to try and reduce the number of args to send_changes.
Anyway, it needs more simplifying and refactoring. I'll upload a patch when there's something for you to review.
Cheers,
Bob
[1] https://github.com/bdionne/couchdb/compare/master...1289-heartbeats-skipped2
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155273#comment-13155273 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob, for the indentation we have those rules in the wiki: http://wiki.apache.org/couchdb/Coding_Standards.
For e.g. the "true" at: https://github.com/bdionne/couchdb/compare/master...1289-heartbeats-skipped2#L0R401
Is not aligned with corresponding "if" (line 398) neither it is on a column multiple of 4 spaces.
Yep, for 2) it's an extra scenario. Probably after passing a row to the filter function, if the filter returns "false" and the time difference between "now()" and the time the last row was sent is greater than the threshold, it would call the callback with "timeout" (so it sends the newline to http clients). If the filter returns "true", set that "last row sent time" to "now()". The only case where this wouldn't work is if the filter function takes "heartbeat" seconds to evaluate a changes row (possibly very unlikely).
This can probably be addressed separately, I have no objection about that.
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154769#comment-13154769 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
Added a failing etap test for this[1]
[1] https://github.com/bdionne/couchdb/compare/master...1289-heartbeats-skipped2
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1289) heartbeats skipped when continuous
changes feed filter function produces no results
Posted by "Bob Dionne (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne updated COUCHDB-1289:
--------------------------------
Attachment: (was: 0001-Ensure-heartbeats-are-not-skipped.patch)
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155246#comment-13155246 ]
Filipe Manana commented on COUCHDB-1289:
----------------------------------------
Bob looks good.
Just have a few comments:
1) The indentation is a bit messed up, like the "if" expression "true" clause and its inner "cause" expression: https://github.com/bdionne/couchdb/compare/master...1289-heartbeats-skipped2#L0R398
2) This issue doesn't happen exclusively once the code finishes folding the seq tree and starts listening for db update events. Lets say we have a heartbeat timeout of 10 seconds, a seq tree with millions of entries, a filter functions which returns false for many consecutive entries. It's quite possible that before folding the entire seq tree snapshot and sending the first changes row which passes the filter (if any) the heartbeat timeout is exceeded. Makes sense?
3) The test can have way less duplicated code (spawn_consumer_heartbeat is basically a copy-paste of spawn_consumer). I made a few changes to it in my repo: https://github.com/fdmanana/couchdb/commit/2af5d56b909449569d60573f8118d7187e9334ca
thanks for working on this one
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped-in-continuous-chan.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1289) heartbeats skipped when
continuous changes feed filter function produces no results
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158030#comment-13158030 ]
Bob Dionne commented on COUCHDB-1289:
-------------------------------------
I think this is correct, I certainly agree with the name change. So in these "feed=normal" cases that are long running you won't timeout at all?
> heartbeats skipped when continuous changes feed filter function produces no results
> -----------------------------------------------------------------------------------
>
> Key: COUCHDB-1289
> URL: https://issues.apache.org/jira/browse/COUCHDB-1289
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Reporter: Bob Dionne
> Assignee: Bob Dionne
> Priority: Minor
> Attachments: 0001-Ensure-heartbeats-are-not-skipped.patch, 0002-Failing-etap-for-heartbeats-skipped.patch
>
>
> if the changes feed has a filter function that produces no results, db_updated messages will still be sent and the heartbeat timeout will never be reached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira