You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Thomas Vander Stichele <sv...@gmail.com> on 2011/04/07 19:43:53 UTC
change notifications question
Hi everyone,
I am adding change notification support to paisley (twisted library for
couchdb) and am writing a simple client that sends GUI notifications for
changed documents.
While doing so, I was wondering if anyone ever considered having change
notifications not only notify the new rev id, but also the old one ?
In my use case, after receiving a notification of a change, I now have
to request the new version, then request what I think is the previous
version (which I will have to do by asking for a list of revision id's -
since I don't know the previous revision) and then figuring out what was
the last one that wasn't mentioned in the change notification.
These are three separate requests to be able to figure out changes in a
way to present them to the user (for example - was the task
added/completed/deleted ?)
Any thoughts?
Thomas
Re: change notifications question
Posted by Robert Newson <ro...@gmail.com>.
It's essential as it's the only way to reclaim disk space. Without it,
all your database files will grow with every addition, update and
deletion (and so will your views).
It would be better if your documents modelled the change itself,
rather than attempting to derive a diff from two revisions of a
document. CouchDB does not show you the previous revision as it should
be regarded as unavailable.
B.
On 9 April 2011 10:59, Thomas Vander Stichele <th...@apestaart.org> wrote:
> On Sat, 2011-04-09 at 10:42 +0100, Robert Newson wrote:
>> CouchDB does not provide revisioning or history features to the end
>> user at all. This is a common mistake.
>>
>> The use of _rev and MVCC is purely a mechanism to enable sane
>> replication between servers where writes are permitted everywhere.
>> CouchDB's replication will only ensure that the current version* of
>> any document is replicated, earlier revisions are *not* replicated.
>
> I'm not interested in replicating earlier revisions.
>
> I'm interested in receiving the change notification (either because of
> replication or because of a local update) and showing what changed.
>
> I'm pretty sure that:
> - in the case of a local update, the previous revision is still there at
> the moment the change notification happens
> - in the case of a replication update, the previous local revision is
> still locally available too.
>
> Those are the changes I'm interested in showing, and hence I see no
> reason why couchdb couldn't inform both the new revid and the revid just
> before the change.
>
>> Earlier revisions are also removed on compaction, which is an
>> essential maintenance operation.
>
> Essential how ? It's not going to stop working if you don't compact,
> just slow down. Depends on your usage model. And in any case,
> compaction is triggered, so I would be completely ok with once in a
> while not being able to get an older revision and not correctly show
> changes because I triggered compaction. 99% of the time this wouldn't
> be the case.
>
> Thomas
>
> --
>
> --
> you own me
> there's nothing you can do
> you own me
> lucky you
> --
> Flumotion - the only way to stream!
> http://www.flumotion.net/
>
>
>
Re: change notifications question
Posted by Thomas Vander Stichele <th...@apestaart.org>.
On Sat, 2011-04-09 at 10:42 +0100, Robert Newson wrote:
> CouchDB does not provide revisioning or history features to the end
> user at all. This is a common mistake.
>
> The use of _rev and MVCC is purely a mechanism to enable sane
> replication between servers where writes are permitted everywhere.
> CouchDB's replication will only ensure that the current version* of
> any document is replicated, earlier revisions are *not* replicated.
I'm not interested in replicating earlier revisions.
I'm interested in receiving the change notification (either because of
replication or because of a local update) and showing what changed.
I'm pretty sure that:
- in the case of a local update, the previous revision is still there at
the moment the change notification happens
- in the case of a replication update, the previous local revision is
still locally available too.
Those are the changes I'm interested in showing, and hence I see no
reason why couchdb couldn't inform both the new revid and the revid just
before the change.
> Earlier revisions are also removed on compaction, which is an
> essential maintenance operation.
Essential how ? It's not going to stop working if you don't compact,
just slow down. Depends on your usage model. And in any case,
compaction is triggered, so I would be completely ok with once in a
while not being able to get an older revision and not correctly show
changes because I triggered compaction. 99% of the time this wouldn't
be the case.
Thomas
--
--
you own me
there's nothing you can do
you own me
lucky you
--
Flumotion - the only way to stream!
http://www.flumotion.net/
Re: change notifications question
Posted by Robert Newson <ro...@gmail.com>.
CouchDB does not provide revisioning or history features to the end
user at all. This is a common mistake.
The use of _rev and MVCC is purely a mechanism to enable sane
replication between servers where writes are permitted everywhere.
CouchDB's replication will only ensure that the current version* of
any document is replicated, earlier revisions are *not* replicated.
Earlier revisions are also removed on compaction, which is an
essential maintenance operation.
B.
* Strictly, CouchDB replicates all leaf revisions of all documents,
the difference is only noticeable if a document has conflicts (all
conflicting versions will be replicated).
On 9 April 2011 09:45, Thomas Vander Stichele <sv...@gmail.com> wrote:
> On Thu, 2011-04-07 at 11:19 -0700, Mark Hahn wrote:
>> You are not even guaranteed the old version will be there.
>
> I would be ok with that, it doesn't seem to be a big problem in
> practice. (Although it's strange to me that all the infrastructure is
> there to have document revisions, but we have no control over how much
> history we want couchdb to keep. Seems like a relatively
> straightforward next step).
>
>> You need
>> to build history into your app.
>
> Can you be a bit more explicit about what you mean?
>
> It sounds like you say I should re-write CouchDB's revisioning/history
> approach in my app just to get around the fact that couchdb has no
> controls for history. I might as well build it on top of anything else
> than couchdb no ? :)
>
> Thanks
> Thomas
>
>
>
RE: change notifications question
Posted by Nils Breunese <N....@vpro.nl>.
This blog post suggests storing versions as attachments:
Simple Document Versioning with CouchDB
http://blog.couchbase.com/simple-document-versioning-with-couchdb
Nils.
________________________________________
Van: Mark Hahn [mark@boutiquing.com]
Verzonden: zaterdag 9 april 2011 20:50
Aan: Thomas Vander Stichele
CC: user@couchdb.apache.org
Onderwerp: Re: change notifications question
> I might as well build it on top of anything else than couchdb no ? :)
If the only feature you care about in your db is versioning then you are
correct, all non-versioning dbs are equal. I personally love using couch
for the many reasons discussed. I find it so much less painful than a
structured db.
If you are willing to build the versioning in your app then there is a
simple solution, IMHO. Save new versions as new docs, not revisions. Couch
doesn't utilize differencing of versions so there is no performance hit.
Keep your own ID that is separate from the couch ID. You can create a view
that links to the previous "version" and you can pull both docs in one
query. Replication will work the same. DB size growth will be the same.
You will need to do your own deleting at your leisure but you will have
full control with your own algorithm.
The coding of this would be trivial compared to changing the erlang
implementation of couch and you wouldn't be risking losing support by
forking. I think I could do it in a day or so. It would be fun.
------------------------------------------------------------------------
VPRO www.vpro.nl
------------------------------------------------------------------------
Re: change notifications question
Posted by Mark Hahn <ma...@boutiquing.com>.
> I might as well build it on top of anything else than couchdb no ? :)
If the only feature you care about in your db is versioning then you are
correct, all non-versioning dbs are equal. I personally love using couch
for the many reasons discussed. I find it so much less painful than a
structured db.
If you are willing to build the versioning in your app then there is a
simple solution, IMHO. Save new versions as new docs, not revisions. Couch
doesn't utilize differencing of versions so there is no performance hit.
Keep your own ID that is separate from the couch ID. You can create a view
that links to the previous "version" and you can pull both docs in one
query. Replication will work the same. DB size growth will be the same.
You will need to do your own deleting at your leisure but you will have
full control with your own algorithm.
The coding of this would be trivial compared to changing the erlang
implementation of couch and you wouldn't be risking losing support by
forking. I think I could do it in a day or so. It would be fun.
On Sat, Apr 9, 2011 at 1:45 AM, Thomas Vander Stichele
<sv...@gmail.com>wrote:
> On Thu, 2011-04-07 at 11:19 -0700, Mark Hahn wrote:
> > You are not even guaranteed the old version will be there.
>
> I would be ok with that, it doesn't seem to be a big problem in
> practice. (Although it's strange to me that all the infrastructure is
> there to have document revisions, but we have no control over how much
> history we want couchdb to keep. Seems like a relatively
> straightforward next step).
>
> > You need
> > to build history into your app.
>
> Can you be a bit more explicit about what you mean?
>
> It sounds like you say I should re-write CouchDB's revisioning/history
> approach in my app just to get around the fact that couchdb has no
> controls for history. I might as well build it on top of anything else
> than couchdb no ? :)
>
> Thanks
> Thomas
>
>
>
--
Mark Hahn
Website Manager
mark@boutiquing.com
949-229-1012
Re: change notifications question
Posted by Thomas Vander Stichele <sv...@gmail.com>.
On Thu, 2011-04-07 at 11:19 -0700, Mark Hahn wrote:
> You are not even guaranteed the old version will be there.
I would be ok with that, it doesn't seem to be a big problem in
practice. (Although it's strange to me that all the infrastructure is
there to have document revisions, but we have no control over how much
history we want couchdb to keep. Seems like a relatively
straightforward next step).
> You need
> to build history into your app.
Can you be a bit more explicit about what you mean?
It sounds like you say I should re-write CouchDB's revisioning/history
approach in my app just to get around the fact that couchdb has no
controls for history. I might as well build it on top of anything else
than couchdb no ? :)
Thanks
Thomas
Re: change notifications question
Posted by Mark Hahn <ma...@boutiquing.com>.
You are not even guaranteed the old version will be there. You need
to build history into your app.
On Thu, Apr 7, 2011 at 10:43 AM, Thomas Vander Stichele
<sv...@gmail.com> wrote:
> Hi everyone,
>
> I am adding change notification support to paisley (twisted library for
> couchdb) and am writing a simple client that sends GUI notifications for
> changed documents.
>
> While doing so, I was wondering if anyone ever considered having change
> notifications not only notify the new rev id, but also the old one ?
>
> In my use case, after receiving a notification of a change, I now have
> to request the new version, then request what I think is the previous
> version (which I will have to do by asking for a list of revision id's -
> since I don't know the previous revision) and then figuring out what was
> the last one that wasn't mentioned in the change notification.
>
> These are three separate requests to be able to figure out changes in a
> way to present them to the user (for example - was the task
> added/completed/deleted ?)
>
> Any thoughts?
>
> Thomas
>
>
--
Mark Hahn
Website Manager
mark@boutiquing.com
949-229-1012
Re: change notifications question
Posted by Thomas Vander Stichele <sv...@gmail.com>.
On Fri, 2011-04-08 at 10:17 +1200, Dave Cottlehuber wrote:
> On 8 April 2011 05:43, Thomas Vander Stichele <sv...@gmail.com> wrote:
> > Hi everyone,
> >
> > I am adding change notification support to paisley (twisted library for
> > couchdb) and am writing a simple client that sends GUI notifications for
> > changed documents.
> >
> > While doing so, I was wondering if anyone ever considered having change
> > notifications not only notify the new rev id, but also the old one ?
> >
> > In my use case, after receiving a notification of a change, I now have
> > to request the new version, then request what I think is the previous
> > version (which I will have to do by asking for a list of revision id's -
> > since I don't know the previous revision) and then figuring out what was
> > the last one that wasn't mentioned in the change notification.
> >
> > These are three separate requests to be able to figure out changes in a
> > way to present them to the user (for example - was the task
> > added/completed/deleted ?)
> >
> > Any thoughts?
> >
> > Thomas
>
> Thomas,
>
> From an app perspective, I would be looking to see if the documents
> can be refactored to store the change as a separate doc, and use
> ?include_docs=true to pull the the prior version if required. This
> means one call to couchdb from the changes feed has the doc, the
> previous doc, and rev if you wish to push changes.
>
> Does this simplify what you're trying to do in paisley itself?
Well...
Paisley is a library for CouchDB. As such I don't think it should rely
on a specific structure of the documents in the database. (Compare to
desktopcouch, where I think they make the same mistake - the nice
features of desktopcouch should be orthogonal to their desire of
enforcing a document structure).
Second, the 'couchdb way' pushes towards storing related data inside one
document, or more denormalized. Currently best practices show how you
can do the equivalent of a single join, but that's it when it comes to
related data. Your approach would mean that you'd get the current
'version' of something from a view instead of a simple get, and you
wouldn't be able to 'JOIN' to another type. So the approach is very
limiting.
That is why I'm suggesting that it should be in fact couchdb providing
the necessary primitive for an app to figure out changes; ie, instead of
making the app figure out the revision of the document that was changed
before the change was made, simply pass it as an extra option in the
change notification.
Maybe I'm making the wrong point, or maybe I'm mistaken about the
couchdb way ? Feel free to tell me so.
Thomas
Re: change notifications question
Posted by Dave Cottlehuber <da...@muse.net.nz>.
On 8 April 2011 05:43, Thomas Vander Stichele <sv...@gmail.com> wrote:
> Hi everyone,
>
> I am adding change notification support to paisley (twisted library for
> couchdb) and am writing a simple client that sends GUI notifications for
> changed documents.
>
> While doing so, I was wondering if anyone ever considered having change
> notifications not only notify the new rev id, but also the old one ?
>
> In my use case, after receiving a notification of a change, I now have
> to request the new version, then request what I think is the previous
> version (which I will have to do by asking for a list of revision id's -
> since I don't know the previous revision) and then figuring out what was
> the last one that wasn't mentioned in the change notification.
>
> These are three separate requests to be able to figure out changes in a
> way to present them to the user (for example - was the task
> added/completed/deleted ?)
>
> Any thoughts?
>
> Thomas
Thomas,
>From an app perspective, I would be looking to see if the documents
can be refactored to store the change as a separate doc, and use
?include_docs=true to pull the the prior version if required. This
means one call to couchdb from the changes feed has the doc, the
previous doc, and rev if you wish to push changes.
Does this simplify what you're trying to do in paisley itself?
A+
Dave