You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Davies, Owain" <Ow...@baesystemsdetica.com> on 2012/03/06 15:58:27 UTC
-- Revision ID Calculation -- and -- Request for edit permissions on wiki --
Hi,
I have just joined this list. I am using CouchDB to do an offline web
app. I am having to re-implement some of the logic for conflict
detection, win determination and replication. I have implemented a
dojo.store that represents that CouchDB API, and am working on the
offline versions using IndexDB and WebSQL as the persistence provider.
First off, please could somebody let me have edit permissions to the
wiki. I am finding lots of undocumented features, or features documented
in one place but not another. A prime example is _changes?style=all_docs
which has taken me some time to discover. Of course, one I knew what it
was called I found it documented in the couchdb guide. Anyhow, I
digress, if I get edit permissions I will endeavour to record my
changes.
Second, can somebody please tell me how the revision id for documents
with attachments are calculated. I have produced a JavaScript module
that consistently calculates the revision of documents without
attachments. I had to reverse engineer erlangs term_to_binary function,
which is actually quite tricky. But I do not know erlang and I do not
understand how the Atts2 term is calculated. Any help?
Kind regards,
Owain
Please consider the environment before printing this email.
This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately.
Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory.
The contents of this email may relate to dealings with other companies under the control of BAE Systems plc details of which can be found at http://www.baesystems.com/Businesses/index.htm.
Detica Limited is a BAE Systems company trading as BAE Systems Detica.
Detica Limited is registered in England and Wales under No: 1337451.
Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP, England.
RE: -- Revision ID Calculation -- and -- Request for edit permissions on wiki --
Posted by "Davies, Owain" <Ow...@baesystemsdetica.com>.
Bob,
Thanks, for the assistance. I shall not waste too much more time on this then.
Kind regards,
Owain
-----Original Message-----
From: Robert Newson [mailto:rnewson@apache.org]
Sent: 06 March 2012 16:01
To: dev@couchdb.apache.org
Subject: Re: -- Revision ID Calculation -- and -- Request for edit permissions on wiki --
It's typically the uncompressed byte stream (and un-base64 encoded), though I think there's a path where it's the compressed stream instead.
Yes, _attachments will be returned in the document body (along with other special fields like _id and _rev) but the bodies will not normally be present (they are so-called 'stubs'), though you can ask for them.
While the rev calculation is unlikely to significantly change it is not frozen, has changed in the past, is an internal detail, and not part of the published and stable API. Since it's not necessary to replicate the logic of this to build a compatible store, I urge caution.
B.
On 6 March 2012 15:43, Davies, Owain <Ow...@baesystemsdetica.com> wrote:
> Hi, It my wikiname is OwainDavies
>
> I figured out in my first version that the hash part of the rev id was
> an opaque string, but I wanted consistency of behaviour when I
> create/modify documents offline (where I calculate the revision id)
> and online (where it is submitted to the store directly and it assigns
> the revision id)?
>
> Currently my application does not use attachments but it will do in
> the future.
>
> When you say the you take an md5 of the file, what do you mean?
> A. Is it an md5 of the uncompressed byte stream
> B. An md5 of the text of the BASE64 encoded data.
> C. Something else.
>
> Also is the _attachments field included in the document body (are any
> special fields ever included in the document body?
>
> A lot questions I know, but thanks for any info?
>
>
> Owain
>
>
> -----Original Message-----
> From: Robert Newson [mailto:rnewson@apache.org]
> Sent: 06 March 2012 15:29
> To: dev@couchdb.apache.org
> Subject: Re: -- Revision ID Calculation -- and -- Request for edit
> permissions on wiki --
>
> Presumably you're reading this;
>
> new_revid(#doc{body=Body,revs={OldStart,OldRevs},
> atts=Atts,deleted=Deleted}) ->
> case [{N, T, M} || #att{name=N,type=T,md5=M} <- Atts, M =/= <<>>]
> of
> Atts2 when length(Atts) =/= length(Atts2) ->
> % We must have old style non-md5 attachments
> ?l2b(integer_to_list(couch_util:rand32()));
> Atts2 ->
> OldRev = case OldRevs of [] -> 0; [OldRev0|_] -> OldRev0 end,
> couch_util:md5(term_to_binary([Deleted, OldStart, OldRev, Body,
> Atts2]))
> end.
>
> Which, basically, means we add in the md5's of the attachments if we
> have them (and a random value if we don't).
>
> In any case, you don't need to slavishly reimplement the current _rev
> algorithm to interoperate correctly. The previous revision calculation
> was just a random value, after all.
>
> If you include your wiki username I can grant you edit access.
>
> B.
>
> P.S Who in the world would think of printing an email?
>
> On 6 March 2012 14:58, Davies, Owain
> <Ow...@baesystemsdetica.com>
> wrote:
>> Hi,
>>
>> I have just joined this list. I am using CouchDB to do an offline web
>> app. I am having to re-implement some of the logic for conflict
>> detection, win determination and replication. I have implemented a
>> dojo.store that represents that CouchDB API, and am working on the
>> offline versions using IndexDB and WebSQL as the persistence provider.
>>
>> First off, please could somebody let me have edit permissions to the
>> wiki. I am finding lots of undocumented features, or features
>> documented in one place but not another. A prime example is
>> _changes?style=all_docs which has taken me some time to discover. Of
>> course, one I knew what it was called I found it documented in the
>> couchdb guide. Anyhow, I digress, if I get edit permissions I will
>> endeavour to record my changes.
>>
>> Second, can somebody please tell me how the revision id for documents
>> with attachments are calculated. I have produced a JavaScript module
>> that consistently calculates the revision of documents without
>> attachments. I had to reverse engineer erlangs term_to_binary
>> function, which is actually quite tricky. But I do not know erlang
>> and
>
>> I do not understand how the Atts2 term is calculated. Any help?
>>
>> Kind regards,
>>
>> Owain
>>
>>
>> Please consider the environment before printing this email.
>>
>> This message should be regarded as confidential. If you have received
> this email in error please notify the sender and destroy it immediately.
>>
>> Statements of intent shall only become binding when confirmed in hard
> copy by an authorised signatory.
>>
>> The contents of this email may relate to dealings with other
>> companies
> under the control of BAE Systems plc details of which can be found at
> http://www.baesystems.com/Businesses/index.htm.
>>
>> Detica Limited is a BAE Systems company trading as BAE Systems Detica.
>> Detica Limited is registered in England and Wales under No: 1337451.
>> Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP,
> England.
>>
> Please consider the environment before printing this email.
>
> This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately.
>
> Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory.
>
> The contents of this email may relate to dealings with other companies under the control of BAE Systems plc details of which can be found at http://www.baesystems.com/Businesses/index.htm.
>
> Detica Limited is a BAE Systems company trading as BAE Systems Detica.
> Detica Limited is registered in England and Wales under No: 1337451.
> Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP, England.
>
Please consider the environment before printing this email.
This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately.
Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory.
The contents of this email may relate to dealings with other companies under the control of BAE Systems plc details of which can be found at http://www.baesystems.com/Businesses/index.htm.
Detica Limited is a BAE Systems company trading as BAE Systems Detica.
Detica Limited is registered in England and Wales under No: 1337451.
Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP, England.
Re: -- Revision ID Calculation -- and -- Request for edit permissions
on wiki --
Posted by Robert Newson <rn...@apache.org>.
It's typically the uncompressed byte stream (and un-base64 encoded),
though I think there's a path where it's the compressed stream
instead.
Yes, _attachments will be returned in the document body (along with
other special fields like _id and _rev) but the bodies will not
normally be present (they are so-called 'stubs'), though you can ask
for them.
While the rev calculation is unlikely to significantly change it is
not frozen, has changed in the past, is an internal detail, and not
part of the published and stable API. Since it's not necessary to
replicate the logic of this to build a compatible store, I urge
caution.
B.
On 6 March 2012 15:43, Davies, Owain <Ow...@baesystemsdetica.com> wrote:
> Hi, It my wikiname is OwainDavies
>
> I figured out in my first version that the hash part of the rev id was
> an opaque string, but I wanted consistency of behaviour when I
> create/modify documents offline (where I calculate the revision id) and
> online (where it is submitted to the store directly and it assigns the
> revision id)?
>
> Currently my application does not use attachments but it will do in the
> future.
>
> When you say the you take an md5 of the file, what do you mean?
> A. Is it an md5 of the uncompressed byte stream
> B. An md5 of the text of the BASE64 encoded data.
> C. Something else.
>
> Also is the _attachments field included in the document body (are any
> special fields ever included in the document body?
>
> A lot questions I know, but thanks for any info?
>
>
> Owain
>
>
> -----Original Message-----
> From: Robert Newson [mailto:rnewson@apache.org]
> Sent: 06 March 2012 15:29
> To: dev@couchdb.apache.org
> Subject: Re: -- Revision ID Calculation -- and -- Request for edit
> permissions on wiki --
>
> Presumably you're reading this;
>
> new_revid(#doc{body=Body,revs={OldStart,OldRevs},
> atts=Atts,deleted=Deleted}) ->
> case [{N, T, M} || #att{name=N,type=T,md5=M} <- Atts, M =/= <<>>] of
> Atts2 when length(Atts) =/= length(Atts2) ->
> % We must have old style non-md5 attachments
> ?l2b(integer_to_list(couch_util:rand32()));
> Atts2 ->
> OldRev = case OldRevs of [] -> 0; [OldRev0|_] -> OldRev0 end,
> couch_util:md5(term_to_binary([Deleted, OldStart, OldRev, Body,
> Atts2]))
> end.
>
> Which, basically, means we add in the md5's of the attachments if we
> have them (and a random value if we don't).
>
> In any case, you don't need to slavishly reimplement the current _rev
> algorithm to interoperate correctly. The previous revision calculation
> was just a random value, after all.
>
> If you include your wiki username I can grant you edit access.
>
> B.
>
> P.S Who in the world would think of printing an email?
>
> On 6 March 2012 14:58, Davies, Owain <Ow...@baesystemsdetica.com>
> wrote:
>> Hi,
>>
>> I have just joined this list. I am using CouchDB to do an offline web
>> app. I am having to re-implement some of the logic for conflict
>> detection, win determination and replication. I have implemented a
>> dojo.store that represents that CouchDB API, and am working on the
>> offline versions using IndexDB and WebSQL as the persistence provider.
>>
>> First off, please could somebody let me have edit permissions to the
>> wiki. I am finding lots of undocumented features, or features
>> documented in one place but not another. A prime example is
>> _changes?style=all_docs which has taken me some time to discover. Of
>> course, one I knew what it was called I found it documented in the
>> couchdb guide. Anyhow, I digress, if I get edit permissions I will
>> endeavour to record my changes.
>>
>> Second, can somebody please tell me how the revision id for documents
>> with attachments are calculated. I have produced a JavaScript module
>> that consistently calculates the revision of documents without
>> attachments. I had to reverse engineer erlangs term_to_binary
>> function, which is actually quite tricky. But I do not know erlang and
>
>> I do not understand how the Atts2 term is calculated. Any help?
>>
>> Kind regards,
>>
>> Owain
>>
>>
>> Please consider the environment before printing this email.
>>
>> This message should be regarded as confidential. If you have received
> this email in error please notify the sender and destroy it immediately.
>>
>> Statements of intent shall only become binding when confirmed in hard
> copy by an authorised signatory.
>>
>> The contents of this email may relate to dealings with other companies
> under the control of BAE Systems plc details of which can be found at
> http://www.baesystems.com/Businesses/index.htm.
>>
>> Detica Limited is a BAE Systems company trading as BAE Systems Detica.
>> Detica Limited is registered in England and Wales under No: 1337451.
>> Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP,
> England.
>>
> Please consider the environment before printing this email.
>
> This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately.
>
> Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory.
>
> The contents of this email may relate to dealings with other companies under the control of BAE Systems plc details of which can be found at http://www.baesystems.com/Businesses/index.htm.
>
> Detica Limited is a BAE Systems company trading as BAE Systems Detica.
> Detica Limited is registered in England and Wales under No: 1337451.
> Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP, England.
>
RE: -- Revision ID Calculation -- and -- Request for edit permissions on wiki --
Posted by "Davies, Owain" <Ow...@baesystemsdetica.com>.
Hi, It my wikiname is OwainDavies
I figured out in my first version that the hash part of the rev id was
an opaque string, but I wanted consistency of behaviour when I
create/modify documents offline (where I calculate the revision id) and
online (where it is submitted to the store directly and it assigns the
revision id)?
Currently my application does not use attachments but it will do in the
future.
When you say the you take an md5 of the file, what do you mean?
A. Is it an md5 of the uncompressed byte stream
B. An md5 of the text of the BASE64 encoded data.
C. Something else.
Also is the _attachments field included in the document body (are any
special fields ever included in the document body?
A lot questions I know, but thanks for any info?
Owain
-----Original Message-----
From: Robert Newson [mailto:rnewson@apache.org]
Sent: 06 March 2012 15:29
To: dev@couchdb.apache.org
Subject: Re: -- Revision ID Calculation -- and -- Request for edit
permissions on wiki --
Presumably you're reading this;
new_revid(#doc{body=Body,revs={OldStart,OldRevs},
atts=Atts,deleted=Deleted}) ->
case [{N, T, M} || #att{name=N,type=T,md5=M} <- Atts, M =/= <<>>] of
Atts2 when length(Atts) =/= length(Atts2) ->
% We must have old style non-md5 attachments
?l2b(integer_to_list(couch_util:rand32()));
Atts2 ->
OldRev = case OldRevs of [] -> 0; [OldRev0|_] -> OldRev0 end,
couch_util:md5(term_to_binary([Deleted, OldStart, OldRev, Body,
Atts2]))
end.
Which, basically, means we add in the md5's of the attachments if we
have them (and a random value if we don't).
In any case, you don't need to slavishly reimplement the current _rev
algorithm to interoperate correctly. The previous revision calculation
was just a random value, after all.
If you include your wiki username I can grant you edit access.
B.
P.S Who in the world would think of printing an email?
On 6 March 2012 14:58, Davies, Owain <Ow...@baesystemsdetica.com>
wrote:
> Hi,
>
> I have just joined this list. I am using CouchDB to do an offline web
> app. I am having to re-implement some of the logic for conflict
> detection, win determination and replication. I have implemented a
> dojo.store that represents that CouchDB API, and am working on the
> offline versions using IndexDB and WebSQL as the persistence provider.
>
> First off, please could somebody let me have edit permissions to the
> wiki. I am finding lots of undocumented features, or features
> documented in one place but not another. A prime example is
> _changes?style=all_docs which has taken me some time to discover. Of
> course, one I knew what it was called I found it documented in the
> couchdb guide. Anyhow, I digress, if I get edit permissions I will
> endeavour to record my changes.
>
> Second, can somebody please tell me how the revision id for documents
> with attachments are calculated. I have produced a JavaScript module
> that consistently calculates the revision of documents without
> attachments. I had to reverse engineer erlangs term_to_binary
> function, which is actually quite tricky. But I do not know erlang and
> I do not understand how the Atts2 term is calculated. Any help?
>
> Kind regards,
>
> Owain
>
>
> Please consider the environment before printing this email.
>
> This message should be regarded as confidential. If you have received
this email in error please notify the sender and destroy it immediately.
>
> Statements of intent shall only become binding when confirmed in hard
copy by an authorised signatory.
>
> The contents of this email may relate to dealings with other companies
under the control of BAE Systems plc details of which can be found at
http://www.baesystems.com/Businesses/index.htm.
>
> Detica Limited is a BAE Systems company trading as BAE Systems Detica.
> Detica Limited is registered in England and Wales under No: 1337451.
> Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP,
England.
>
Please consider the environment before printing this email.
This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately.
Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory.
The contents of this email may relate to dealings with other companies under the control of BAE Systems plc details of which can be found at http://www.baesystems.com/Businesses/index.htm.
Detica Limited is a BAE Systems company trading as BAE Systems Detica.
Detica Limited is registered in England and Wales under No: 1337451.
Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP, England.
Re: -- Revision ID Calculation -- and -- Request for edit permissions
on wiki --
Posted by Robert Newson <rn...@apache.org>.
Presumably you're reading this;
new_revid(#doc{body=Body,revs={OldStart,OldRevs},
atts=Atts,deleted=Deleted}) ->
case [{N, T, M} || #att{name=N,type=T,md5=M} <- Atts, M =/= <<>>] of
Atts2 when length(Atts) =/= length(Atts2) ->
% We must have old style non-md5 attachments
?l2b(integer_to_list(couch_util:rand32()));
Atts2 ->
OldRev = case OldRevs of [] -> 0; [OldRev0|_] -> OldRev0 end,
couch_util:md5(term_to_binary([Deleted, OldStart, OldRev, Body, Atts2]))
end.
Which, basically, means we add in the md5's of the attachments if we
have them (and a random value if we don't).
In any case, you don't need to slavishly reimplement the current _rev
algorithm to interoperate correctly. The previous revision calculation
was just a random value, after all.
If you include your wiki username I can grant you edit access.
B.
P.S Who in the world would think of printing an email?
On 6 March 2012 14:58, Davies, Owain <Ow...@baesystemsdetica.com> wrote:
> Hi,
>
> I have just joined this list. I am using CouchDB to do an offline web
> app. I am having to re-implement some of the logic for conflict
> detection, win determination and replication. I have implemented a
> dojo.store that represents that CouchDB API, and am working on the
> offline versions using IndexDB and WebSQL as the persistence provider.
>
> First off, please could somebody let me have edit permissions to the
> wiki. I am finding lots of undocumented features, or features documented
> in one place but not another. A prime example is _changes?style=all_docs
> which has taken me some time to discover. Of course, one I knew what it
> was called I found it documented in the couchdb guide. Anyhow, I
> digress, if I get edit permissions I will endeavour to record my
> changes.
>
> Second, can somebody please tell me how the revision id for documents
> with attachments are calculated. I have produced a JavaScript module
> that consistently calculates the revision of documents without
> attachments. I had to reverse engineer erlangs term_to_binary function,
> which is actually quite tricky. But I do not know erlang and I do not
> understand how the Atts2 term is calculated. Any help?
>
> Kind regards,
>
> Owain
>
>
> Please consider the environment before printing this email.
>
> This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately.
>
> Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory.
>
> The contents of this email may relate to dealings with other companies under the control of BAE Systems plc details of which can be found at http://www.baesystems.com/Businesses/index.htm.
>
> Detica Limited is a BAE Systems company trading as BAE Systems Detica.
> Detica Limited is registered in England and Wales under No: 1337451.
> Registered office: Surrey Research Park, Guildford, Surrey, GU2 7YP, England.
>