You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Elliot Murphy (JIRA)" <ji...@apache.org> on 2009/03/16 04:25:50 UTC
[jira] Created: (COUCHDB-290) Include sequence number in update
notifications
Include sequence number in update notifications
-----------------------------------------------
Key: COUCHDB-290
URL: https://issues.apache.org/jira/browse/COUCHDB-290
Project: CouchDB
Issue Type: Improvement
Affects Versions: 0.9
Reporter: Elliot Murphy
Priority: Minor
Fix For: 0.9
Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682315#action_12682315 ]
Robert Newson commented on COUCHDB-290:
---------------------------------------
@Jan I could, but I'm not going to. :) I'm not going to assume there's a time when "no more new docs come in". Without some end condition, no new updates will become searchable. Again, the reason I fetch the current update_seq is to prevent a runaway while loop in my code, I could, instead, simply flush my changes to the index every N minutes or N changes (whichever occurs first). Since there's interest in that model anyway, I guess it's moot.
For now, I'm happy with the way I handle update notifications, receiving the new update_seq in the notification would just be tidier for my case.
As Damien mentioned on the dev list, update notification might be obsolete soon anyway, so we'll all need to change to a different solution anyway.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "eric casteleijn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694976#action_12694976 ]
eric casteleijn commented on COUCHDB-290:
-----------------------------------------
Now that 0.9 has landed, is there any chance this patch could land? I found an issue in our specific use case which make it more that just optimization:
we log sequence numbers with timestamps to a file, and getting the sequence number from couch with a request from the update trigger frequently returns a higher sequence number than one would expect, since another update has already happened before the update trigger gets back the sequence number. This can be coded around, but it makes it *very* hard to write deterministic tests for the update triggers.
If there is anything I or Elliot can do to help this patch along, we'd be glad to do it.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "eric casteleijn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682301#action_12682301 ]
eric casteleijn commented on COUCHDB-290:
-----------------------------------------
The problem it solves is that for many update triggers, it is important to find out what has changed in the update, which unfortunately means logging sequence numbers outside of couchdb, so that an all_documents_by_seq query?startkey=old_seq can be run to find out which documents have changed. I think this is a pretty common use case.
We can of course let each update trigger call fire off a request to the db to get its current seq_no, but that's unfortunate since the information could easily be provided to the update trigger, and by the time the request has returned, the sequence number might have changed again.
A list of changed document ids would be even better, but if this is not possible, we will have to do requests for them.
Bulk updates aside, this would always be a single document id though, or am I wrong? If there is a way to pass that document id *or* "_bulk" or something, that would solve it for all non-nulk updates at least. I realize that would probably be a more involved patch.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682307#action_12682307 ]
Robert Newson commented on COUCHDB-290:
---------------------------------------
@Jan no, I do not use endkey. I specify startkey and limit only (I already know startkey and limit is just to make the couchdb call a constant cost). If the update notification contained the new update_seq I would not have to find it with a call to http://localhost:5984/<dbname>. I use the new update_seq as the escape condition for my loop. That is, my code attempts to catch up from where I left off to where couchdb is at the time of the notification, neither more nor less.
Having the new sequence in the update notification would eliminate the http://localhost:5984/<dbname> call. It's not a big deal for me, couchdb-lucene works fine without it, but I don't see any downside, and some modest upside, to the suggestion.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lehnardt updated COUCHDB-290:
---------------------------------
Fix Version/s: (was: 0.9)
0.10
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682216#action_12682216 ]
Paul Joseph Davis commented on COUCHDB-290:
-------------------------------------------
Patch looks simple enough. I'd rename seq to update_seq to maintain consistency. I'm uncertain about adding the list of affected document ids. I'd have to go back and look at the code paths again to see what's what. If adding the list included making a hit all the way down to the update sequence btree I'd force the overhead to the client instead of including it in every update notification. Minus the s/seq/update_seq/ I'd be good applying this as is.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682299#action_12682299 ]
Adam Kocoloski commented on COUCHDB-290:
----------------------------------------
The last time I used an update notification process it only received one notification per _bulk_docs. Including the update_seq would allow a process to do something (e.g. rebuild views) for every N updates to the DB, regardless of whether the update occurred in a _bulk_docs or in a standalone request.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "eric casteleijn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695503#action_12695503 ]
eric casteleijn commented on COUCHDB-290:
-----------------------------------------
That is excellent news! If there is any way we can assist with that work (testing, discussion/feedback, documenting on wiki, or coding) please let us know. If this is in the near future, I guess we can live with patching our couch instances for the duration.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lehnardt updated COUCHDB-290:
---------------------------------
Priority: Blocker (was: Minor)
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Blocker
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Elliot Murphy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682751#action_12682751 ]
Elliot Murphy commented on COUCHDB-290:
---------------------------------------
Wow, thanks for all the feedback! From reading all the discussion, it sounded like the attached patch is useful to some people, hurts nothing, and might get thrown away when the entire update notification system changed to use comet. Will the patch be applied then, or rejected? I've made the name change as requested, let me know if I should change anything else. Hugs, -elliot
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682261#action_12682261 ]
Robert Newson commented on COUCHDB-290:
---------------------------------------
+1 for including update_seq, -1 for including affected doc id's (potentially huge).
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Elliot Murphy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Elliot Murphy updated COUCHDB-290:
----------------------------------
Attachment: couchdb-sequences.patch
Renamed seq to update_seq as requested, and regenerated patch.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Damien Katz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695494#action_12695494 ]
Damien Katz commented on COUCHDB-290:
-------------------------------------
I'm actually working on this HTTP functionality right now. Then not only child processes, but remote processes will be able to easily register for notifications over HTTP, and we can present a richer interface.
Once in place the std io notifications would be removed completely and we could then use std io for logging and error reporting of the child process.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695258#action_12695258 ]
Chris Anderson commented on COUCHDB-290:
----------------------------------------
reviewing the patch. +1 on applying it.
can someone please apply this? I'm sorry to ask but the bandwidth where I am is unacceptable. svn commit fail. thanks!
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Damien Katz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695304#action_12695304 ]
Damien Katz commented on COUCHDB-290:
-------------------------------------
-1 on the patch. This functionality should be at the HTTP level,
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682302#action_12682302 ]
Robert Newson commented on COUCHDB-290:
---------------------------------------
for me (that is, couchdb-lucene) it would for verification and would save one call per update. When I receive an update notification I GET /<dbname> to find the latest update_seq, I then call _all_docs_by_seq with a startkey that I track independently and a configurable limit that defaults to 1000. I loop until I've processed up to the update_seq I got in the last call.
If the update_seq were provided in the notification, I could simplify the code. I would think every update trigger must do something much the same, it seems reasonable to add it the update notification.
If, as eric suggests, the before and after value was supplied, that might further simplify the code.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695556#action_12695556 ]
Chris Anderson commented on COUCHDB-290:
----------------------------------------
Very exciting. If you don't mind working in a branch or on git, I bet you'll get a lot of help along the way.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Elliot Murphy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Elliot Murphy updated COUCHDB-290:
----------------------------------
Attachment: couchdb-sequences.patch
This is the patch that adds the sequence number to the update notification. Output looks like this:
{"type":"updated","db":"replication_notification_test","seq":1}
The "seq" : 1 part is what is added by this patch.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682287#action_12682287 ]
Jan Lehnardt commented on COUCHDB-290:
--------------------------------------
What problem does including the update_seq solve?
-1 on adding doc ids, this can not be solved in a sensible way.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (COUCHDB-290) Include sequence number in update notifications
Posted by Damien Katz <da...@apache.org>.
I agree with Jan.
I just realized, I think the planned COMET interfaces for near-
realtime replication will obsolete update the notification process.
That should give us all the flexibility we want here.
-Damien
On Mar 16, 2009, at 8:58 AM, Jan Lehnardt (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682305
> #action_12682305 ]
>
> Jan Lehnardt commented on COUCHDB-290:
> --------------------------------------
>
> @Adam Good one.
>
> @Eric The default use case is that you track the last seq_nr you
> queried on. The only special case is the first request where you
> don't have a seq_nr and get everything from your DB. There is
> nothing "unfortunate" about that, it's the API. I don't see how
> including the current seq_nr in the notification would help at all
> in this case. It would help if the result of _all_docs_by_seq would
> include the then latest seq_nr so the process can store it and I
> believe it does.
>
> Adding a list of documents comes with more problems:
>
> 1) In a bulk operation, you might get _a lot_ of ids and you'd need
> to take care of that.
> 2) The API is meant to be asynchronous. CouchDB does not have to
> track which documents get sent to the notifier process in case of an
> error. The process would just restart with its last know sequence
> number. It significantly simplifies on the CouchDB end which is good
> for reliability. I think the extra step to go for the process is not
> too bad.
>
> @Robert so you query _all_docs_by_seq with a startkey and endkey?
> Why not just drop the endkey?
>
>
>> Include sequence number in update notifications
>> -----------------------------------------------
>>
>> Key: COUCHDB-290
>> URL: https://issues.apache.org/jira/browse/COUCHDB-290
>> Project: CouchDB
>> Issue Type: Improvement
>> Affects Versions: 0.9
>> Reporter: Elliot Murphy
>> Priority: Minor
>> Fix For: 0.9
>>
>> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>>
>>
>> Hi! There's been requests to include the sequence number when
>> sending an update notification. Thanks to the guidance from davisp
>> on #couchdb on March 13th, I've been able to put together a little
>> patch that does just that. In the future I'm interested in doing
>> the same for the create notification, and perhaps extending create/
>> delete/update notifications to include a list of affected doc IDs.
>> For now though, just this simple patch.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682305#action_12682305 ]
Jan Lehnardt commented on COUCHDB-290:
--------------------------------------
@Adam Good one.
@Eric The default use case is that you track the last seq_nr you queried on. The only special case is the first request where you don't have a seq_nr and get everything from your DB. There is nothing "unfortunate" about that, it's the API. I don't see how including the current seq_nr in the notification would help at all in this case. It would help if the result of _all_docs_by_seq would include the then latest seq_nr so the process can store it and I believe it does.
Adding a list of documents comes with more problems:
1) In a bulk operation, you might get _a lot_ of ids and you'd need to take care of that.
2) The API is meant to be asynchronous. CouchDB does not have to track which documents get sent to the notifier process in case of an error. The process would just restart with its last know sequence number. It significantly simplifies on the CouchDB end which is good for reliability. I think the extra step to go for the process is not too bad.
@Robert so you query _all_docs_by_seq with a startkey and endkey? Why not just drop the endkey?
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lehnardt closed COUCHDB-290.
--------------------------------
Resolution: Fixed
update_seq numbers are included in the new _changes feed. DbUpdateNotifiers are deprecated now. and we won't include the document data in the notification.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Blocker
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682310#action_12682310 ]
Jan Lehnardt commented on COUCHDB-290:
--------------------------------------
@Robert Can you not simplify by not ending your loop based on the "current" seq_nr, but keep going until no more new docs come in and then stop and wait for the next update notification to kick off? You can then ignore all update notifications that come in while you are looping.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "eric casteleijn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682313#action_12682313 ]
eric casteleijn commented on COUCHDB-290:
-----------------------------------------
@Jan ah, yes: remembering the last sequence number you got in your last query would solve the problem for some use cases, unfortunately not for ours: we want to track sequence numbers and log them with a timestamp, i.e. we don't want to query the couchdb at all at that particular moment, but possibly at some later date for everything updated between seq_n and seq_m, based on their timestamps.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.9
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-290) Include sequence number in update
notifications
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695490#action_12695490 ]
Chris Anderson commented on COUCHDB-290:
----------------------------------------
My general thought processes had been that since the db update notifier is basically deprecated, it wouldn't hurt to start giving it a few bells and whistles, in preparation for the making the same info available over the Comet interface.
Damien, I take it that you're not against making the update-seq available, you'd just rather see the db-update-notification-process go away altogether?
If now is the time to start implementing the Comet hook I guess we should start a thread about the implementation. Eric, Elliot - I'm with Damien on thinking this should go into the HTTP layer. I may be a little more cavalier about playing around with the old implementation, instead of moving forward on Comet. Damien may have a point though, that enhancing a deprecated API gives the wrong expectations.
> Include sequence number in update notifications
> -----------------------------------------------
>
> Key: COUCHDB-290
> URL: https://issues.apache.org/jira/browse/COUCHDB-290
> Project: CouchDB
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Elliot Murphy
> Priority: Minor
> Fix For: 0.10
>
> Attachments: couchdb-sequences.patch, couchdb-sequences.patch
>
>
> Hi! There's been requests to include the sequence number when sending an update notification. Thanks to the guidance from davisp on #couchdb on March 13th, I've been able to put together a little patch that does just that. In the future I'm interested in doing the same for the create notification, and perhaps extending create/delete/update notifications to include a list of affected doc IDs.
> For now though, just this simple patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.