You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Jens Alfke <je...@mooseyard.com> on 2011/06/22 19:17:03 UTC
How to identify new docs in the _changes feed?
By watching the _changes feed I can detect when a document is deleted (via the “deleted” key), but is there a way to tell when a new document is created? There doesn’t seem to be a corresponding extra key in the notification in this case.
I am tempted to look for a prefix of “1-“ in the revision, but I’m pretty sure that this is an implementation detail and revision IDs should be treated as purely opaque cookies, amiright?
The cheapest alternative I can think of is to do a HEAD on the _all_docs view after I get a new notification from _changes, and see if its ETag has changed.
—Jens
Re: How to identify new docs in the _changes feed?
Posted by Mark Hahn <ma...@boutiquing.com>.
> What if revision 2-* replicates in?
I assume you mean: What if revision 2-* replicates in and there is no 1-*?
On Wed, Jun 22, 2011 at 1:26 PM, Randall Leeds <ra...@gmail.com>wrote:
> On Wed, Jun 22, 2011 at 10:17, Jens Alfke <je...@mooseyard.com> wrote:
> > By watching the _changes feed I can detect when a document is deleted
> (via the “deleted” key), but is there a way to tell when a new document is
> created? There doesn’t seem to be a corresponding extra key in the
> notification in this case.
> >
> > I am tempted to look for a prefix of “1-“ in the revision, but I’m pretty
> sure that this is an implementation detail and revision IDs should be
> treated as purely opaque cookies, amiright?
>
> Youisright. But it would work. ;)
>
> >
> > The cheapest alternative I can think of is to do a HEAD on the _all_docs
> view after I get a new notification from _changes, and see if its ETag has
> changed.
> >
> > —Jens
>
> Careful, this is not cheap (yet):
> https://issues.apache.org/jira/browse/COUCHDB-941
> (The workaround is be sure to specify ?limit=1 or some similarly
> restrictive query)
>
> Why do you need to know the difference and what defines "created" for you?
> Would a document that replicated from elsewhere count as "created" or
> only if it's the first revision?
> What if revision 2-* replicates in?
>
--
Mark Hahn
Website Manager
mark@boutiquing.com
949-342-4246
Re: How to identify new docs in the _changes feed?
Posted by Jens Alfke <je...@mooseyard.com>.
On Jun 23, 2011, at 2:41 PM, Randall Leeds wrote:
> Would it be a bad idea for some reason to just send every change via
> NSNotifications and let the client/app deal with determining if it's
> "new" (if they care )?
> This sounds more "generic" to me.
That’s what I’ll probably do. But since there’s already an “is deleted” property of the notification, I wanted to see if I could have a corresponding “is new” property. Thanks anyway.
—Jens
Re: How to identify new docs in the _changes feed?
Posted by Randall Leeds <ra...@gmail.com>.
On Thu, Jun 23, 2011 at 11:13, Jens Alfke <je...@mooseyard.com> wrote:
>
> On Jun 22, 2011, at 6:31 PM, Jason Smith wrote:
>
>> _changes is a false prophet.
>> The best bang-for-buck with the _changes feed is cache invalidation: A
>> change is a *hint* that you might want to check something out, but it
>> is (IMHO) not generally the primary trigger to execute some code.
>
> Could you explain that in more detail? In particular, what is it that makes _changes unreliable? Could something be changed [sic] to make it more reliable/useful?
>
>> In your case: Require all new docs to have a field "is_new: true".
>> That is very easy to confirm in the validator:
>
> I’m writing a generic framework, not a specific app. I can’t make any assumptions about the document schema, that’s up to the application. This specific task is just to map the _changes feed to Cocoa NSNotifications so the client app can act on them.
Would it be a bad idea for some reason to just send every change via
NSNotifications and let the client/app deal with determining if it's
"new" (if they care )?
This sounds more "generic" to me.
Re: How to identify new docs in the _changes feed?
Posted by Jens Alfke <je...@mooseyard.com>.
On Jun 22, 2011, at 6:31 PM, Jason Smith wrote:
> _changes is a false prophet.
> The best bang-for-buck with the _changes feed is cache invalidation: A
> change is a *hint* that you might want to check something out, but it
> is (IMHO) not generally the primary trigger to execute some code.
Could you explain that in more detail? In particular, what is it that makes _changes unreliable? Could something be changed [sic] to make it more reliable/useful?
> In your case: Require all new docs to have a field "is_new: true".
> That is very easy to confirm in the validator:
I’m writing a generic framework, not a specific app. I can’t make any assumptions about the document schema, that’s up to the application. This specific task is just to map the _changes feed to Cocoa NSNotifications so the client app can act on them.
—Jens
Re: How to identify new docs in the _changes feed?
Posted by Jason Smith <jh...@iriscouch.com>.
On Thu, Jun 23, 2011 at 3:26 AM, Randall Leeds <ra...@gmail.com> wrote:
> Why do you need to know the difference and what defines "created" for you?
> Would a document that replicated from elsewhere count as "created" or
> only if it's the first revision?
> What if revision 2-* replicates in?
_changes is a false prophet.
The best bang-for-buck with the _changes feed is cache invalidation: A
change is a *hint* that you might want to check something out, but it
is (IMHO) not generally the primary trigger to execute some code.
I have found success pushing intelligence to the client, combined with
a validate_doc_update() function that confirms that intelligence.
In your case: Require all new docs to have a field "is_new: true".
That is very easy to confirm in the validator:
if(oldDoc && newDoc.is_new)
throw {forbidden: "Updates must REMOVE is_new"};
if(!oldDoc && !newDoc.is_new)
throw {forbidden: "Creates must SET is_new"}
It is also very easy to see in the _changes feed
if(change.doc.is_new)
do_stuff_with_new_doc(change.doc);
This workflow is compatible with replication, since the peers will
(presumably) enforce the same policy. (If you are allowing them to
update your db, presumably you trust them.)
--
Iris Couch
Re: How to identify new docs in the _changes feed?
Posted by Randall Leeds <ra...@gmail.com>.
On Wed, Jun 22, 2011 at 10:17, Jens Alfke <je...@mooseyard.com> wrote:
> By watching the _changes feed I can detect when a document is deleted (via the “deleted” key), but is there a way to tell when a new document is created? There doesn’t seem to be a corresponding extra key in the notification in this case.
>
> I am tempted to look for a prefix of “1-“ in the revision, but I’m pretty sure that this is an implementation detail and revision IDs should be treated as purely opaque cookies, amiright?
Youisright. But it would work. ;)
>
> The cheapest alternative I can think of is to do a HEAD on the _all_docs view after I get a new notification from _changes, and see if its ETag has changed.
>
> —Jens
Careful, this is not cheap (yet):
https://issues.apache.org/jira/browse/COUCHDB-941
(The workaround is be sure to specify ?limit=1 or some similarly
restrictive query)
Why do you need to know the difference and what defines "created" for you?
Would a document that replicated from elsewhere count as "created" or
only if it's the first revision?
What if revision 2-* replicates in?
Re: How to identify new docs in the _changes feed?
Posted by Jens Alfke <je...@mooseyard.com>.
On Jun 22, 2011, at 11:08 AM, kowsik wrote:
> I typically have created_at and updated_at (monkey patch couchrest's
> save method, assuming you are using ruby) in all of my documents which
> makes it easy to know this difference.
I’m writing a generic CouchDB framework (for Objective-C/Cocoa) so I can’t make any assumptions about the schema.
—Jens
Re: How to identify new docs in the _changes feed?
Posted by kowsik <ko...@gmail.com>.
On Wed, Jun 22, 2011 at 10:17 AM, Jens Alfke <je...@mooseyard.com> wrote:
> By watching the _changes feed I can detect when a document is deleted (via the “deleted” key), but is there a way to tell when a new document is created? There doesn’t seem to be a corresponding extra key in the notification in this case.
>
> I am tempted to look for a prefix of “1-“ in the revision, but I’m pretty sure that this is an implementation detail and revision IDs should be treated as purely opaque cookies, amiright?
>
> The cheapest alternative I can think of is to do a HEAD on the _all_docs view after I get a new notification from _changes, and see if its ETag has changed.
I typically have created_at and updated_at (monkey patch couchrest's
save method, assuming you are using ruby) in all of my documents which
makes it easy to know this difference.
K.
---
http://blitz.io
http://twitter.com/pcapr
http://labs.mudynamics.com