You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Daniel Gonzalez <go...@gonvaled.com> on 2014/04/08 13:00:58 UTC
Is the revision field deterministic?
Hi all,
I have just seen a very intersting issue which I do not fullly understand.
I have serverA and serverB, with the same replicated database db1.
Sometimes replication is just turned off. Currently it is off.
I have taken a look at a bunch of documents.and I have found a couple of
them where the revision field is exactly the same, but the documents
content is different. Since the content is different, that means those
documents have not been replicated, but editted indepently in serverA and
serverB.
How could happen then that the revision field is the same? The only
explanation that I can find is that the rev is derived from the document
id, deterministically (together with the serial counter indicating the
number of revision). But why would the revision be generated like that? I
was under the impression that the revision was a random number (or "random
enough"), so that cases like this (independent edition of a document) could
be easily detected.
So my question is: how is the revision field generated and, more
importantly, makes my description of what is happening sense, or is there
another explanation of why the rev field is the same? (I hope I am wrong!)
Thanks and regards,
Daniel Gonzalez
Re: Is the revision field deterministic?
Posted by Jan Lehnardt <ja...@apache.org>.
On 09 Apr 2014, at 09:24 , Robert Samuel Newson <rn...@apache.org> wrote:
>
> It’s not necessary for all things-that-work-with-couchdb to implement this particular optimization (and that’s all it is). Reducing the occasions when _rev is different for two documents in the exact same state is great but there’s no harm if it’s not achieved.
>
> I think it would be a mistake to define the _rev calculation as its current implementation.
+1
At the same time, it might be worth considering a portable way to make this optimisation and change it in CouchDB, so other implementations have a chance of opting into it.
Best
Jan
--
>
> B.
>
> On 9 Apr 2014, at 01:26, Dale Harvey <da...@arandomurl.com> wrote:
>
>> Sorry, I meant in the case in which the same document was posted to
>> different servers that replicated
>>
>>
>> On 9 April 2014 01:09, Mark Hahn <ma...@hahnca.com> wrote:
>>
>>>> by not conflicting the users loses information
>>>
>>> By not conflicting, do you mean two docs with different content having same
>>> rev? How is this possible? I thought the original poster on this thread
>>> said he was wrong about that.
>>>
>>>
>>>
>>>
>>> On Tue, Apr 8, 2014 at 4:56 PM, Dale Harvey <da...@arandomurl.com> wrote:
>>>
>>>> Yup I almost mentioned PouchDB when I seen this thread come up, from an
>>>> external implementors perspective it isnt great that CouchDB uses the
>>>> internal erlang binary term format to create revisions, its possible to
>>>> implement it compatibly but decided against it due to extra deps +
>>>> complexity
>>>>
>>>> From an API perspective I believe its a leaky abstraction, I havent
>>> tested
>>>> this but it seems like if the binary term format changes between erlang
>>>> versions then the same code can have different behaviour, similiarly
>>>> possible between couch versions.
>>>>
>>>> Also I just dont think its a great idea, by not conflicting the users
>>> loses
>>>> information, its easy for audit software to force conflicts by adding the
>>>> target id to the documents, but yeh it still makes me feel weird.
>>>>
>>>> And yup while unfortunately named, Couchbase Lite is very much along the
>>>> same lines as PouchDB, with a different target platform. (we even stole
>>> the
>>>> same replication optimisations :))
>>>>
>>>> Cheers
>>>> Dale
>>>>
>>>>
>>>>
>>>>
>>>> On 9 April 2014 00:36, Jens Alfke <je...@couchbase.com> wrote:
>>>>
>>>>>
>>>>> On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
>>>>>
>>>>>> Jens, is that was another uncanny attempt to adv. unrelated product
>>>>>> without providing relevant information? These actions only confuses
>>>>>> people forcing them thinking that Couchbase and CouchDB are the
>>>>>> similar products while they don't.
>>>>>
>>>>> Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the
>>>> exact
>>>>> same data model, a 95% similar REST API*, and can replicate with each
>>>>> other. (The implementations are entirely different, though, because
>>>>> Couchbase Lite is meant to be embedded in mobile apps.)
>>>>>
>>>>> Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no
>>>> one
>>>>> complains about PouchDB being mentioned here.
>>>>>
>>>>> Couchbase *Server* is a different beast entirely. It was wrong of me to
>>>>> discuss it here last week; apologies again.
>>>>>
>>>>> I know the names are confusing. Couchbase-the-company was named back
>>> when
>>>>> it was very much about CouchDB, which then changed for complicated
>>>>> technical reasons. Couchbase Lite was originally called TouchDB, but
>>> was
>>>>> renamed for branding/marketing reasons outside my control.
>>>>>
>>>>> --Jens
>>>>>
>>>>> * CBL is missing some of the admin features, like user accounts and
>>>>> _restart, that don't make sense for an embedded database; and it adds a
>>>>> couple of replication optimizations that I've been discussing on the
>>>>> 'replication' list here.
>>>>
>>>
>
Re: Is the revision field deterministic?
Posted by Robert Samuel Newson <rn...@apache.org>.
It’s not necessary for all things-that-work-with-couchdb to implement this particular optimization (and that’s all it is). Reducing the occasions when _rev is different for two documents in the exact same state is great but there’s no harm if it’s not achieved.
I think it would be a mistake to define the _rev calculation as its current implementation.
B.
On 9 Apr 2014, at 01:26, Dale Harvey <da...@arandomurl.com> wrote:
> Sorry, I meant in the case in which the same document was posted to
> different servers that replicated
>
>
> On 9 April 2014 01:09, Mark Hahn <ma...@hahnca.com> wrote:
>
>>> by not conflicting the users loses information
>>
>> By not conflicting, do you mean two docs with different content having same
>> rev? How is this possible? I thought the original poster on this thread
>> said he was wrong about that.
>>
>>
>>
>>
>> On Tue, Apr 8, 2014 at 4:56 PM, Dale Harvey <da...@arandomurl.com> wrote:
>>
>>> Yup I almost mentioned PouchDB when I seen this thread come up, from an
>>> external implementors perspective it isnt great that CouchDB uses the
>>> internal erlang binary term format to create revisions, its possible to
>>> implement it compatibly but decided against it due to extra deps +
>>> complexity
>>>
>>> From an API perspective I believe its a leaky abstraction, I havent
>> tested
>>> this but it seems like if the binary term format changes between erlang
>>> versions then the same code can have different behaviour, similiarly
>>> possible between couch versions.
>>>
>>> Also I just dont think its a great idea, by not conflicting the users
>> loses
>>> information, its easy for audit software to force conflicts by adding the
>>> target id to the documents, but yeh it still makes me feel weird.
>>>
>>> And yup while unfortunately named, Couchbase Lite is very much along the
>>> same lines as PouchDB, with a different target platform. (we even stole
>> the
>>> same replication optimisations :))
>>>
>>> Cheers
>>> Dale
>>>
>>>
>>>
>>>
>>> On 9 April 2014 00:36, Jens Alfke <je...@couchbase.com> wrote:
>>>
>>>>
>>>> On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
>>>>
>>>>> Jens, is that was another uncanny attempt to adv. unrelated product
>>>>> without providing relevant information? These actions only confuses
>>>>> people forcing them thinking that Couchbase and CouchDB are the
>>>>> similar products while they don't.
>>>>
>>>> Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the
>>> exact
>>>> same data model, a 95% similar REST API*, and can replicate with each
>>>> other. (The implementations are entirely different, though, because
>>>> Couchbase Lite is meant to be embedded in mobile apps.)
>>>>
>>>> Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no
>>> one
>>>> complains about PouchDB being mentioned here.
>>>>
>>>> Couchbase *Server* is a different beast entirely. It was wrong of me to
>>>> discuss it here last week; apologies again.
>>>>
>>>> I know the names are confusing. Couchbase-the-company was named back
>> when
>>>> it was very much about CouchDB, which then changed for complicated
>>>> technical reasons. Couchbase Lite was originally called TouchDB, but
>> was
>>>> renamed for branding/marketing reasons outside my control.
>>>>
>>>> --Jens
>>>>
>>>> * CBL is missing some of the admin features, like user accounts and
>>>> _restart, that don't make sense for an embedded database; and it adds a
>>>> couple of replication optimizations that I've been discussing on the
>>>> 'replication' list here.
>>>
>>
Re: Is the revision field deterministic?
Posted by Dale Harvey <da...@arandomurl.com>.
Sorry, I meant in the case in which the same document was posted to
different servers that replicated
On 9 April 2014 01:09, Mark Hahn <ma...@hahnca.com> wrote:
> > by not conflicting the users loses information
>
> By not conflicting, do you mean two docs with different content having same
> rev? How is this possible? I thought the original poster on this thread
> said he was wrong about that.
>
>
>
>
> On Tue, Apr 8, 2014 at 4:56 PM, Dale Harvey <da...@arandomurl.com> wrote:
>
> > Yup I almost mentioned PouchDB when I seen this thread come up, from an
> > external implementors perspective it isnt great that CouchDB uses the
> > internal erlang binary term format to create revisions, its possible to
> > implement it compatibly but decided against it due to extra deps +
> > complexity
> >
> > From an API perspective I believe its a leaky abstraction, I havent
> tested
> > this but it seems like if the binary term format changes between erlang
> > versions then the same code can have different behaviour, similiarly
> > possible between couch versions.
> >
> > Also I just dont think its a great idea, by not conflicting the users
> loses
> > information, its easy for audit software to force conflicts by adding the
> > target id to the documents, but yeh it still makes me feel weird.
> >
> > And yup while unfortunately named, Couchbase Lite is very much along the
> > same lines as PouchDB, with a different target platform. (we even stole
> the
> > same replication optimisations :))
> >
> > Cheers
> > Dale
> >
> >
> >
> >
> > On 9 April 2014 00:36, Jens Alfke <je...@couchbase.com> wrote:
> >
> > >
> > > On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
> > >
> > > > Jens, is that was another uncanny attempt to adv. unrelated product
> > > > without providing relevant information? These actions only confuses
> > > > people forcing them thinking that Couchbase and CouchDB are the
> > > > similar products while they don't.
> > >
> > > Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the
> > exact
> > > same data model, a 95% similar REST API*, and can replicate with each
> > > other. (The implementations are entirely different, though, because
> > > Couchbase Lite is meant to be embedded in mobile apps.)
> > >
> > > Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no
> > one
> > > complains about PouchDB being mentioned here.
> > >
> > > Couchbase *Server* is a different beast entirely. It was wrong of me to
> > > discuss it here last week; apologies again.
> > >
> > > I know the names are confusing. Couchbase-the-company was named back
> when
> > > it was very much about CouchDB, which then changed for complicated
> > > technical reasons. Couchbase Lite was originally called TouchDB, but
> was
> > > renamed for branding/marketing reasons outside my control.
> > >
> > > --Jens
> > >
> > > * CBL is missing some of the admin features, like user accounts and
> > > _restart, that don't make sense for an embedded database; and it adds a
> > > couple of replication optimizations that I've been discussing on the
> > > 'replication' list here.
> >
>
Re: Is the revision field deterministic?
Posted by Mark Hahn <ma...@hahnca.com>.
> by not conflicting the users loses information
By not conflicting, do you mean two docs with different content having same
rev? How is this possible? I thought the original poster on this thread
said he was wrong about that.
On Tue, Apr 8, 2014 at 4:56 PM, Dale Harvey <da...@arandomurl.com> wrote:
> Yup I almost mentioned PouchDB when I seen this thread come up, from an
> external implementors perspective it isnt great that CouchDB uses the
> internal erlang binary term format to create revisions, its possible to
> implement it compatibly but decided against it due to extra deps +
> complexity
>
> From an API perspective I believe its a leaky abstraction, I havent tested
> this but it seems like if the binary term format changes between erlang
> versions then the same code can have different behaviour, similiarly
> possible between couch versions.
>
> Also I just dont think its a great idea, by not conflicting the users loses
> information, its easy for audit software to force conflicts by adding the
> target id to the documents, but yeh it still makes me feel weird.
>
> And yup while unfortunately named, Couchbase Lite is very much along the
> same lines as PouchDB, with a different target platform. (we even stole the
> same replication optimisations :))
>
> Cheers
> Dale
>
>
>
>
> On 9 April 2014 00:36, Jens Alfke <je...@couchbase.com> wrote:
>
> >
> > On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
> >
> > > Jens, is that was another uncanny attempt to adv. unrelated product
> > > without providing relevant information? These actions only confuses
> > > people forcing them thinking that Couchbase and CouchDB are the
> > > similar products while they don't.
> >
> > Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the
> exact
> > same data model, a 95% similar REST API*, and can replicate with each
> > other. (The implementations are entirely different, though, because
> > Couchbase Lite is meant to be embedded in mobile apps.)
> >
> > Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no
> one
> > complains about PouchDB being mentioned here.
> >
> > Couchbase *Server* is a different beast entirely. It was wrong of me to
> > discuss it here last week; apologies again.
> >
> > I know the names are confusing. Couchbase-the-company was named back when
> > it was very much about CouchDB, which then changed for complicated
> > technical reasons. Couchbase Lite was originally called TouchDB, but was
> > renamed for branding/marketing reasons outside my control.
> >
> > --Jens
> >
> > * CBL is missing some of the admin features, like user accounts and
> > _restart, that don't make sense for an embedded database; and it adds a
> > couple of replication optimizations that I've been discussing on the
> > 'replication' list here.
>
Re: Is the revision field deterministic?
Posted by Dale Harvey <da...@arandomurl.com>.
Yup I almost mentioned PouchDB when I seen this thread come up, from an
external implementors perspective it isnt great that CouchDB uses the
internal erlang binary term format to create revisions, its possible to
implement it compatibly but decided against it due to extra deps +
complexity
>From an API perspective I believe its a leaky abstraction, I havent tested
this but it seems like if the binary term format changes between erlang
versions then the same code can have different behaviour, similiarly
possible between couch versions.
Also I just dont think its a great idea, by not conflicting the users loses
information, its easy for audit software to force conflicts by adding the
target id to the documents, but yeh it still makes me feel weird.
And yup while unfortunately named, Couchbase Lite is very much along the
same lines as PouchDB, with a different target platform. (we even stole the
same replication optimisations :))
Cheers
Dale
On 9 April 2014 00:36, Jens Alfke <je...@couchbase.com> wrote:
>
> On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
>
> > Jens, is that was another uncanny attempt to adv. unrelated product
> > without providing relevant information? These actions only confuses
> > people forcing them thinking that Couchbase and CouchDB are the
> > similar products while they don't.
>
> Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the exact
> same data model, a 95% similar REST API*, and can replicate with each
> other. (The implementations are entirely different, though, because
> Couchbase Lite is meant to be embedded in mobile apps.)
>
> Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no one
> complains about PouchDB being mentioned here.
>
> Couchbase *Server* is a different beast entirely. It was wrong of me to
> discuss it here last week; apologies again.
>
> I know the names are confusing. Couchbase-the-company was named back when
> it was very much about CouchDB, which then changed for complicated
> technical reasons. Couchbase Lite was originally called TouchDB, but was
> renamed for branding/marketing reasons outside my control.
>
> --Jens
>
> * CBL is missing some of the admin features, like user accounts and
> _restart, that don't make sense for an embedded database; and it adds a
> couple of replication optimizations that I've been discussing on the
> 'replication' list here.
Re: Is the revision field deterministic?
Posted by Andy Wenk <an...@apache.org>.
Jens,
On 9 April 2014 18:30, Jens Alfke <je...@couchbase.com> wrote:
>
> On Apr 9, 2014, at 3:48 AM, Alexander Shorin <kx...@gmail.com> wrote:
>
> > Hm..Naming becomes confused. While I'm aware about TouchDB,
> > personally, as user, I thought that Couchbase Lite is some kind
> > version of Couchbase product. Couchbase Lite for small business,
> > Couchbase Server for enterprise one
>
> I understand, and raised the issue when renaming came up (in early 2013.)
> But the product marketing folks’ priorities are Couchbase customers and
> potential customers, rather than the smaller set of people who already knew
> about TouchDB.
>
> > I'm sorry for my tone in this case. Would you have some time to create
> > an article in CouchDB wiki about comparison CBL with CouchDB and with
> > few words about why CBS isn't the same to bring to the end confusion
> > between Couchbase and CouchDB?
>
> Apology accepted, and that’s a great idea. Maybe there could be a
> collection of pages about the various databases in the CouchDB ecosystem,
> including PouchDB, Cloudant’s server, and Cloudant’s new mobile library
> (which is a fork of TouchDB.)
>
> Right now we’re nearing code-freeze for Couchbase Lite 1.0, so I’m a bit
> busy, but I’ll make a note to do this soon.
>
Maybe you find time for a writeup on this? That would be awesome as we do
have issues with naming in the Couch-World.
Thanks a lot and Cheers
Andy
P.S.: You are welcome to use the new cwiki at
https://cwiki.apache.org/confluence/display/COUCHDB/Databases+in+the+CouchDB+ecosystem
--
Andy Wenk
Hamburg - Germany
RockIt!
GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
https://people.apache.org/keys/committer/andywenk.asc
Re: Is the revision field deterministic?
Posted by Stanley Iriele <si...@gmail.com>.
In an odd way this relates back to the BigCouch vs couchbase server
discussion the other week in that it clears up the confusion that arises
due to similar names and what not.
On Apr 9, 2014 9:48 AM, "Jens Alfke" <je...@couchbase.com> wrote:
>
> On Apr 9, 2014, at 3:48 AM, Alexander Shorin <kx...@gmail.com> wrote:
>
> > Hm..Naming becomes confused. While I'm aware about TouchDB,
> > personally, as user, I thought that Couchbase Lite is some kind
> > version of Couchbase product. Couchbase Lite for small business,
> > Couchbase Server for enterprise one
>
> I understand, and raised the issue when renaming came up (in early 2013.)
> But the product marketing folks' priorities are Couchbase customers and
> potential customers, rather than the smaller set of people who already knew
> about TouchDB.
>
> > I'm sorry for my tone in this case. Would you have some time to create
> > an article in CouchDB wiki about comparison CBL with CouchDB and with
> > few words about why CBS isn't the same to bring to the end confusion
> > between Couchbase and CouchDB?
>
> Apology accepted, and that's a great idea. Maybe there could be a
> collection of pages about the various databases in the CouchDB ecosystem,
> including PouchDB, Cloudant's server, and Cloudant's new mobile library
> (which is a fork of TouchDB.)
>
> Right now we're nearing code-freeze for Couchbase Lite 1.0, so I'm a bit
> busy, but I'll make a note to do this soon.
>
> --Jens
Re: Is the revision field deterministic?
Posted by Jens Alfke <je...@couchbase.com>.
On Apr 9, 2014, at 3:48 AM, Alexander Shorin <kx...@gmail.com> wrote:
> Hm..Naming becomes confused. While I'm aware about TouchDB,
> personally, as user, I thought that Couchbase Lite is some kind
> version of Couchbase product. Couchbase Lite for small business,
> Couchbase Server for enterprise one
I understand, and raised the issue when renaming came up (in early 2013.) But the product marketing folks’ priorities are Couchbase customers and potential customers, rather than the smaller set of people who already knew about TouchDB.
> I'm sorry for my tone in this case. Would you have some time to create
> an article in CouchDB wiki about comparison CBL with CouchDB and with
> few words about why CBS isn't the same to bring to the end confusion
> between Couchbase and CouchDB?
Apology accepted, and that’s a great idea. Maybe there could be a collection of pages about the various databases in the CouchDB ecosystem, including PouchDB, Cloudant’s server, and Cloudant’s new mobile library (which is a fork of TouchDB.)
Right now we’re nearing code-freeze for Couchbase Lite 1.0, so I’m a bit busy, but I’ll make a note to do this soon.
—Jens
Re: Is the revision field deterministic?
Posted by Alexander Shorin <kx...@gmail.com>.
On Wed, Apr 9, 2014 at 3:36 AM, Jens Alfke <je...@couchbase.com> wrote:
> On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
>
>> Jens, is that was another uncanny attempt to adv. unrelated product
>> without providing relevant information? These actions only confuses
>> people forcing them thinking that Couchbase and CouchDB are the
>> similar products while they don't.
>
> Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the exact same data model, a 95% similar REST API*, and can replicate with each other. (The implementations are entirely different, though, because Couchbase Lite is meant to be embedded in mobile apps.)
>
> Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no one complains about PouchDB being mentioned here.
>
> Couchbase *Server* is a different beast entirely. It was wrong of me to discuss it here last week; apologies again.
>
> I know the names are confusing. Couchbase-the-company was named back when it was very much about CouchDB, which then changed for complicated technical reasons. Couchbase Lite was originally called TouchDB, but was renamed for branding/marketing reasons outside my control.
>
> —Jens
>
> * CBL is missing some of the admin features, like user accounts and _restart, that don’t make sense for an embedded database; and it adds a couple of replication optimizations that I’ve been discussing on the ‘replication’ list here.
Hm..Naming becomes confused. While I'm aware about TouchDB,
personally, as user, I thought that Couchbase Lite is some kind
version of Couchbase product. Couchbase Lite for small business,
Couchbase Server for enterprise one. That's a bit surprise to see them
different, especially when everywhere Couchbase name becomes promoted
without "edition" part.
I'm sorry for my tone in this case. Would you have some time to create
an article in CouchDB wiki about comparison CBL with CouchDB and with
few words about why CBS isn't the same to bring to the end confusion
between Couchbase and CouchDB? It would be still good call to maintain
comparison chart between CouchDB and similar, compatible and friendly
products.
Thank you for branding clarification.
--
,,,^..^,,,
Re: Is the revision field deterministic?
Posted by Jens Alfke <je...@couchbase.com>.
On Apr 8, 2014, at 8:34 AM, Alexander Shorin <kx...@gmail.com> wrote:
> Jens, is that was another uncanny attempt to adv. unrelated product
> without providing relevant information? These actions only confuses
> people forcing them thinking that Couchbase and CouchDB are the
> similar products while they don't.
Nope. Couchbase *Lite* and CouchDB _are_ very similar. They have the exact same data model, a 95% similar REST API*, and can replicate with each other. (The implementations are entirely different, though, because Couchbase Lite is meant to be embedded in mobile apps.)
Couchbase Lite is at least as similar to CouchDB as PouchDB is, and no one complains about PouchDB being mentioned here.
Couchbase *Server* is a different beast entirely. It was wrong of me to discuss it here last week; apologies again.
I know the names are confusing. Couchbase-the-company was named back when it was very much about CouchDB, which then changed for complicated technical reasons. Couchbase Lite was originally called TouchDB, but was renamed for branding/marketing reasons outside my control.
—Jens
* CBL is missing some of the admin features, like user accounts and _restart, that don’t make sense for an embedded database; and it adds a couple of replication optimizations that I’ve been discussing on the ‘replication’ list here.
Re: Is the revision field deterministic?
Posted by Alexander Shorin <kx...@gmail.com>.
On Tue, Apr 8, 2014 at 7:01 PM, Jens Alfke <je...@couchbase.com> wrote:
> On Apr 8, 2014, at 7:31 AM, Daniel Gonzalez <go...@gonvaled.com> wrote:
>
>> Ok, but does mean it is indeed related to the document content, so a
>> different content would produce a different revision number.
>
> Yes. It’s an MD5 (?) digest computed from the document JSON.
It's an MD5 digest from Erlang representation (proplist) of JSON
document, not from JSON document itself.
>> And just curious: what does "the same content mean"? Is a different
>> re-ordering of the json fields still passing the comparisson check?
Yes, order matters.
{"foo": 0, "bar": 1} would be represented internally as [{<<"foo">>,
0}, {<<"bar">>, 1}]
while
{"bar": 1, "foo": 0} would be [{<<"bar">>, 1}, {<<"foo">>, 0}]
Logically objects are the same, but different representation produces
different digest.
As for JavaScript, it's by accident preserves object's fields order
during serialization to JSON, so by the same accident it gives the
same internal representation and the same digest for CouchDB. With
Python or Ruby this trick wouldn't work unless underlying JSON library
wouldn't produce determinate output (for instance, by sorting object's
keys).
P.S.
> Couchbase Lite computes rev IDs based on a canonical encoding[2] of the JSON that will always generate the same result regardless of serialization differences. It won’t be the same result arrived at by CouchDB, though.
Jens, is that was another uncanny attempt to adv. unrelated product
without providing relevant information? These actions only confuses
people forcing them thinking that Couchbase and CouchDB are the
similar products while they don't.
--
,,,^..^,,,
Re: Is the revision field deterministic?
Posted by Jens Alfke <je...@couchbase.com>.
On Apr 8, 2014, at 7:31 AM, Daniel Gonzalez <go...@gonvaled.com> wrote:
> Ok, but does mean it is indeed related to the document content, so a
> different content would produce a different revision number.
Yes. It’s an MD5 (?) digest computed from the document JSON.
> And just curious: what does "the same content mean"? Is a different
> re-ordering of the json fields still passing the comparisson check?
I hope not[1], since the ordering of keys is undefined/insignificant in JSON and randomized in many languages/libraries (ones that implement objects/dictionaries/maps using hash tables.) In many languages simply reading JSON into memory and then serializing again can re-order the keys. Even if it doesn’t (as in Erlang), it can change the escape sequences used in strings, the number of decimal places used in non-integers, the presence of whitespace, etc.
Couchbase Lite computes rev IDs based on a canonical encoding[2] of the JSON that will always generate the same result regardless of serialization differences. It won’t be the same result arrived at by CouchDB, though.
—Jens
[1] I say “I hope not” because I don’t actually know the implementation of this in CouchDB.
[2] https://github.com/couchbase/couchbase-lite-ios/wiki/Signed-Documents#generating-canonical-digests
Re: Is the revision field deterministic?
Posted by Daniel Gonzalez <go...@gonvaled.com>.
Ok, but does mean it is indeed related to the document content, so a
different content would produce a different revision number. In my case I
was having the same rev but different content, which is what I could not
understand (as explained, it was caused by my personal libraries)
And just curious: what does "the same content mean"? Is a different
re-ordering of the json fields still passing the comparisson check?
BR,
Daniel
On Tue, Apr 8, 2014 at 2:45 PM, Robert Samuel Newson <rn...@apache.org>wrote:
>
> I'll answer anyway.
>
> Yes, it's deterministic. The same document with the same history will have
> the same _rev value. This is an optimization over the previous algorithm
> where _rev was a random UUID. This changed years ago.
>
> The advantage is that two servers receiving the same update can more
> optimally replicate. They still have to check that the target has all
> _id/_rev pairs but will usually be able to skip actually transmitting
> document and attachment content.
>
> B.
>
>
> On 8 Apr 2014, at 12:44, Daniel Gonzalez <go...@gonvaled.com> wrote:
>
> > Hi,
> >
> > I have found the cause of the difference in the content: my libraries are
> > correcting some documents when reading from couchdb (to account for
> missing
> > properties in wrongly-structured documents). Some of those corrections
> are
> > using random-generated values, and that is why I am seeing a difference
> > when reading from one or another server.
> >
> > The revision in couchdb is the same, and the content in couchdb is the
> > same, as expected.
> >
> > Sorry for spamming the list.
> >
> > BR,
> > Daniel
> >
> >
> > On Tue, Apr 8, 2014 at 1:00 PM, Daniel Gonzalez <gonvaled@gonvaled.com
> >wrote:
> >
> >> Hi all,
> >>
> >> I have just seen a very intersting issue which I do not fullly
> understand.
> >> I have serverA and serverB, with the same replicated database db1.
> >>
> >> Sometimes replication is just turned off. Currently it is off.
> >>
> >> I have taken a look at a bunch of documents.and I have found a couple of
> >> them where the revision field is exactly the same, but the documents
> >> content is different. Since the content is different, that means those
> >> documents have not been replicated, but editted indepently in serverA
> and
> >> serverB.
> >>
> >> How could happen then that the revision field is the same? The only
> >> explanation that I can find is that the rev is derived from the document
> >> id, deterministically (together with the serial counter indicating the
> >> number of revision). But why would the revision be generated like that?
> I
> >> was under the impression that the revision was a random number (or
> "random
> >> enough"), so that cases like this (independent edition of a document)
> could
> >> be easily detected.
> >>
> >> So my question is: how is the revision field generated and, more
> >> importantly, makes my description of what is happening sense, or is
> there
> >> another explanation of why the rev field is the same? (I hope I am
> wrong!)
> >>
> >> Thanks and regards,
> >> Daniel Gonzalez
> >>
>
>
Re: Is the revision field deterministic?
Posted by Robert Samuel Newson <rn...@apache.org>.
I’ll answer anyway.
Yes, it’s deterministic. The same document with the same history will have the same _rev value. This is an optimization over the previous algorithm where _rev was a random UUID. This changed years ago.
The advantage is that two servers receiving the same update can more optimally replicate. They still have to check that the target has all _id/_rev pairs but will usually be able to skip actually transmitting document and attachment content.
B.
On 8 Apr 2014, at 12:44, Daniel Gonzalez <go...@gonvaled.com> wrote:
> Hi,
>
> I have found the cause of the difference in the content: my libraries are
> correcting some documents when reading from couchdb (to account for missing
> properties in wrongly-structured documents). Some of those corrections are
> using random-generated values, and that is why I am seeing a difference
> when reading from one or another server.
>
> The revision in couchdb is the same, and the content in couchdb is the
> same, as expected.
>
> Sorry for spamming the list.
>
> BR,
> Daniel
>
>
> On Tue, Apr 8, 2014 at 1:00 PM, Daniel Gonzalez <go...@gonvaled.com>wrote:
>
>> Hi all,
>>
>> I have just seen a very intersting issue which I do not fullly understand.
>> I have serverA and serverB, with the same replicated database db1.
>>
>> Sometimes replication is just turned off. Currently it is off.
>>
>> I have taken a look at a bunch of documents.and I have found a couple of
>> them where the revision field is exactly the same, but the documents
>> content is different. Since the content is different, that means those
>> documents have not been replicated, but editted indepently in serverA and
>> serverB.
>>
>> How could happen then that the revision field is the same? The only
>> explanation that I can find is that the rev is derived from the document
>> id, deterministically (together with the serial counter indicating the
>> number of revision). But why would the revision be generated like that? I
>> was under the impression that the revision was a random number (or "random
>> enough"), so that cases like this (independent edition of a document) could
>> be easily detected.
>>
>> So my question is: how is the revision field generated and, more
>> importantly, makes my description of what is happening sense, or is there
>> another explanation of why the rev field is the same? (I hope I am wrong!)
>>
>> Thanks and regards,
>> Daniel Gonzalez
>>
Re: Is the revision field deterministic?
Posted by Daniel Gonzalez <go...@gonvaled.com>.
Hi,
I have found the cause of the difference in the content: my libraries are
correcting some documents when reading from couchdb (to account for missing
properties in wrongly-structured documents). Some of those corrections are
using random-generated values, and that is why I am seeing a difference
when reading from one or another server.
The revision in couchdb is the same, and the content in couchdb is the
same, as expected.
Sorry for spamming the list.
BR,
Daniel
On Tue, Apr 8, 2014 at 1:00 PM, Daniel Gonzalez <go...@gonvaled.com>wrote:
> Hi all,
>
> I have just seen a very intersting issue which I do not fullly understand.
> I have serverA and serverB, with the same replicated database db1.
>
> Sometimes replication is just turned off. Currently it is off.
>
> I have taken a look at a bunch of documents.and I have found a couple of
> them where the revision field is exactly the same, but the documents
> content is different. Since the content is different, that means those
> documents have not been replicated, but editted indepently in serverA and
> serverB.
>
> How could happen then that the revision field is the same? The only
> explanation that I can find is that the rev is derived from the document
> id, deterministically (together with the serial counter indicating the
> number of revision). But why would the revision be generated like that? I
> was under the impression that the revision was a random number (or "random
> enough"), so that cases like this (independent edition of a document) could
> be easily detected.
>
> So my question is: how is the revision field generated and, more
> importantly, makes my description of what is happening sense, or is there
> another explanation of why the rev field is the same? (I hope I am wrong!)
>
> Thanks and regards,
> Daniel Gonzalez
>