suggestion for improvement

[Mark Hahn <ma...@hahnca.com>]

Was there a suggestion in the voting list to tell the user what has changed
when a change notification or conflict is issued?  My app burns a lot of
resources because of this problem.

Re: suggestion for improvement

[Mark Hahn <ma...@hahnca.com>]

I think I understand my options.  I'll come back here when I think I have a
solution.

Thanks everyone.

On Thu, Apr 19, 2012 at 4:05 PM, Dominic Tarr <do...@gmail.com>wrote:

> you would need to respond to each item in the feed by making a request back
> to couch
> (since merely including the document is not sufficient, in your case)
> and then combining that into your own augmented changes feed, perhaps.
>
> On Fri, Apr 20, 2012 at 11:01 AM, Dominic Tarr <dominic.tarr@gmail.com
> >wrote:
>
> > I'm think mark wants to know what has changed with the last update,
> > rather than to keep every version around forever.
> >
> >
> > On Fri, Apr 20, 2012 at 10:10 AM, Robert Newson <rnewson@apache.org
> >wrote:
> >
> >> You can fetch previous revisions with ?rev=<rev> until compaction
> >> sweeps them away, but it's considered bad practice (your administrator
> >> should be free to compact whenever needed).
> >>
> >> I don't think it's CouchDB's job to keep all the old versions of
> >> documents forever, this should be solved at the application layer, it
> >> doesn't seem particularly difficult either. If you care about the
> >> changes a doc goes through, you should record them in the doc itself.
> >> Whenever you update a document, add a new entry to a "history" array,
> >> for example. We have this for replication checkpoints already.
> >>
> >> B.
> >>
> >> On 19 April 2012 22:33, Mark Hahn <ma...@hahnca.com> wrote:
> >> > Thanks, I had forgotten about open_revs.  But open_revs only applies
> to
> >> > conflicts, right?
> >> >
> >> > How do I find out what changed in a change feed?  In other words, if I
> >> get
> >> > a change feed is there a way to access the previous version to find
> out
> >> > what changed?
> >>
> >
> >
>

Re: suggestion for improvement

[Dominic Tarr <do...@gmail.com>]

you would need to respond to each item in the feed by making a request back
to couch
(since merely including the document is not sufficient, in your case)
and then combining that into your own augmented changes feed, perhaps.

On Fri, Apr 20, 2012 at 11:01 AM, Dominic Tarr <do...@gmail.com>wrote:

> I'm think mark wants to know what has changed with the last update,
> rather than to keep every version around forever.
>
>
> On Fri, Apr 20, 2012 at 10:10 AM, Robert Newson <rn...@apache.org>wrote:
>
>> You can fetch previous revisions with ?rev=<rev> until compaction
>> sweeps them away, but it's considered bad practice (your administrator
>> should be free to compact whenever needed).
>>
>> I don't think it's CouchDB's job to keep all the old versions of
>> documents forever, this should be solved at the application layer, it
>> doesn't seem particularly difficult either. If you care about the
>> changes a doc goes through, you should record them in the doc itself.
>> Whenever you update a document, add a new entry to a "history" array,
>> for example. We have this for replication checkpoints already.
>>
>> B.
>>
>> On 19 April 2012 22:33, Mark Hahn <ma...@hahnca.com> wrote:
>> > Thanks, I had forgotten about open_revs.  But open_revs only applies to
>> > conflicts, right?
>> >
>> > How do I find out what changed in a change feed?  In other words, if I
>> get
>> > a change feed is there a way to access the previous version to find out
>> > what changed?
>>
>
>

Re: suggestion for improvement

[Dominic Tarr <do...@gmail.com>]

I'm think mark wants to know what has changed with the last update,
rather than to keep every version around forever.

On Fri, Apr 20, 2012 at 10:10 AM, Robert Newson <rn...@apache.org> wrote:

> You can fetch previous revisions with ?rev=<rev> until compaction
> sweeps them away, but it's considered bad practice (your administrator
> should be free to compact whenever needed).
>
> I don't think it's CouchDB's job to keep all the old versions of
> documents forever, this should be solved at the application layer, it
> doesn't seem particularly difficult either. If you care about the
> changes a doc goes through, you should record them in the doc itself.
> Whenever you update a document, add a new entry to a "history" array,
> for example. We have this for replication checkpoints already.
>
> B.
>
> On 19 April 2012 22:33, Mark Hahn <ma...@hahnca.com> wrote:
> > Thanks, I had forgotten about open_revs.  But open_revs only applies to
> > conflicts, right?
> >
> > How do I find out what changed in a change feed?  In other words, if I
> get
> > a change feed is there a way to access the previous version to find out
> > what changed?
>

Re: suggestion for improvement

[Robert Newson <rn...@apache.org>]

You can fetch previous revisions with ?rev=<rev> until compaction
sweeps them away, but it's considered bad practice (your administrator
should be free to compact whenever needed).

I don't think it's CouchDB's job to keep all the old versions of
documents forever, this should be solved at the application layer, it
doesn't seem particularly difficult either. If you care about the
changes a doc goes through, you should record them in the doc itself.
Whenever you update a document, add a new entry to a "history" array,
for example. We have this for replication checkpoints already.

B.

On 19 April 2012 22:33, Mark Hahn <ma...@hahnca.com> wrote:
> Thanks, I had forgotten about open_revs.  But open_revs only applies to
> conflicts, right?
>
> How do I find out what changed in a change feed?  In other words, if I get
> a change feed is there a way to access the previous version to find out
> what changed?

Re: suggestion for improvement

[Mark Hahn <ma...@hahnca.com>]

Thanks, I had forgotten about open_revs.  But open_revs only applies to
conflicts, right?

How do I find out what changed in a change feed?  In other words, if I get
a change feed is there a way to access the previous version to find out
what changed?

Re: suggestion for improvement

[Robert Newson <rn...@apache.org>]

"It's just a shame that couch says "here's a conflict" and then gives you no
way to find out what the nature of the conflict is. All couch needs to do
is make both versions available."

CouchDB already does this! Call GET with ?open_revs=all to get all
open revisions.

CouchDB does not know your applications semantics, so it cannot tell
you the "nature" of the conflict. It can, and does, allow you to fetch
all conflicts and make that determination yourself.

B.

On 19 April 2012 21:57, Mark Hahn <ma...@hahnca.com> wrote:
>>  think that managing versions/diff of a document is out of the scope of
> couchDB
>
> I don't see how making different versions of a doc available is out of
> couch's realm of responsibility.  It's job is to present docs.  Letting us
> get to another version isn't rocket science.
>
> It's just a shame that couch says "here's a conflict" and then gives you no
> way to find out what the nature of the conflict is. All couch needs to do
> is make both versions available.   I'm sure that could be done and would be
> appropriate to include.
>
> General doc change feeds could also make the old and new docs available.
>  Just give us some way to access any and all docs with one lower rev
> number.
>
> *I know how all of this could be easily fixed*
>
> Just give us a query by doc id and rev number (just the rev count at the
> beginning of the rev value).  Then on a conflict we could query all
> matching revs that conflicted.  On a change feed we could query rev counts
> one less.
>
> You can say this doesn't belong in couch.

Re: suggestion for improvement

[Mark Hahn <ma...@hahnca.com>]

>  think that managing versions/diff of a document is out of the scope of
couchDB

I don't see how making different versions of a doc available is out of
couch's realm of responsibility.  It's job is to present docs.  Letting us
get to another version isn't rocket science.

It's just a shame that couch says "here's a conflict" and then gives you no
way to find out what the nature of the conflict is. All couch needs to do
is make both versions available.   I'm sure that could be done and would be
appropriate to include.

General doc change feeds could also make the old and new docs available.
 Just give us some way to access any and all docs with one lower rev
number.

*I know how all of this could be easily fixed*

Just give us a query by doc id and rev number (just the rev count at the
beginning of the rev value).  Then on a conflict we could query all
matching revs that conflicted.  On a change feed we could query rev counts
one less.

You can say this doesn't belong in couch.

Re: suggestion for improvement

[Matthieu Rakotojaona <ma...@gmail.com>]

I think that managing versions/diff of a document is out of the scope
of couchDB (although one could argue about the versioning). If I were
you, I would store each new revision with a new UUID, link all these
revisions with some common id (the name of a post, ...)  and use it to
retrieve the diff.

You could use a view that emits [common_id,revision_for_this_document]
as a key to build an index of all the revs of a document, and retrieve
the diffs by using a _list function that queries the previous view
with keys like [common_id_of_interest,{}] to get all the revisions and
computes the diff. The advantage of doing it in a _list is that it is
done in the database, aka server-side.

In fact this is an interesting problem; I would be glad to know how
you end up solving it.

-- 
Matthieu RAKOTOJAONA

Re: suggestion for improvement

[Dominic Tarr <do...@gmail.com>]

I'm working on some projects that may be of use to you.

I'm already down these rabbit holes!

I have https://github.com/dominictarr/xdiff which diffs and patches json.
it can also do a 3-way merge, which is used to reconcile concurrent changes.

Also, I have https://github.com/dominictarr/snob which manages series of
changes.
it's the same architecture as git. but allows you to plug in a difftool
(that implements diff, diff3, and patch)

since you have already architected your application, xdiff may be more
useful to you.
either way, i've been working on these mainly because they are interesting
problems,
and would be very interested to see how they apply in practice.

is the browser also sending back changes? or is it only going one way?

On Thu, Apr 19, 2012 at 2:56 PM, Mark Hahn <ma...@hahnca.com> wrote:

> I'm sending changed docs from node to the browser.  Then the browser
> compares the doc to the dom.   If i only sent doc changes there would be
> less net traffic and several many of changes wouldn't need to be sent at
> all.
>
> The particular situation that caused me to post this question is a case
> where only about 1% of the changes are needed.  It bugs me to have so much
> unneeded traffic.
>
> I could cache all the data sent to the client to compare in the server.
>  I've also thought of writing a summary of what is changed on writes along
> with the document.  I wanted to ask here before I went down any of these
> rabbit holes.
>
> On Wed, Apr 18, 2012 at 6:50 PM, Dominic Tarr <dominic.tarr@gmail.com
> >wrote:
>
> > couch just doesn't track things at that level, since versioning is done
> at
> > the document level.
> >
> > how are you doing handling this at the moment?
> >
> > are you using a library to do a diff of the conflicted versions or
> > something like that?
> >
> > On Thu, Apr 19, 2012 at 8:28 AM, Mark Hahn <ma...@hahnca.com> wrote:
> >
> > > Was there a suggestion in the voting list to tell the user what has
> > changed
> > > when a change notification or conflict is issued?  My app burns a lot
> of
> > > resources because of this problem.
> > >
> >
>

Re: suggestion for improvement

[Mark Hahn <ma...@hahnca.com>]

I'm sending changed docs from node to the browser.  Then the browser
compares the doc to the dom.   If i only sent doc changes there would be
less net traffic and several many of changes wouldn't need to be sent at
all.

The particular situation that caused me to post this question is a case
where only about 1% of the changes are needed.  It bugs me to have so much
unneeded traffic.

I could cache all the data sent to the client to compare in the server.
 I've also thought of writing a summary of what is changed on writes along
with the document.  I wanted to ask here before I went down any of these
rabbit holes.

On Wed, Apr 18, 2012 at 6:50 PM, Dominic Tarr <do...@gmail.com>wrote:

> couch just doesn't track things at that level, since versioning is done at
> the document level.
>
> how are you doing handling this at the moment?
>
> are you using a library to do a diff of the conflicted versions or
> something like that?
>
> On Thu, Apr 19, 2012 at 8:28 AM, Mark Hahn <ma...@hahnca.com> wrote:
>
> > Was there a suggestion in the voting list to tell the user what has
> changed
> > when a change notification or conflict is issued?  My app burns a lot of
> > resources because of this problem.
> >
>

Re: suggestion for improvement

[Dominic Tarr <do...@gmail.com>]

couch just doesn't track things at that level, since versioning is done at
the document level.

how are you doing handling this at the moment?

are you using a library to do a diff of the conflicted versions or
something like that?

On Thu, Apr 19, 2012 at 8:28 AM, Mark Hahn <ma...@hahnca.com> wrote:

> Was there a suggestion in the voting list to tell the user what has changed
> when a change notification or conflict is issued?  My app burns a lot of
> resources because of this problem.
>