What's the best way to jack up the rev number?

[Ryan Ramage <ry...@gmail.com>]

I have a need to make a document revision number go up a lot. eg from
245-xxxx...    to 430-xxxxx....

My reason for is here:
http://groups.google.com/group/couchapp/browse_thread/thread/c51903ad5096fe2


What is the fastest/best way of doing this?

Ryan


-- 
Twitter: @eckoit

Re: What's the best way to jack up the rev number?

[Randall Leeds <ra...@gmail.com>]

On Mon, Aug 22, 2011 at 02:03, Jason Smith <jh...@iriscouch.com> wrote:

> On Mon, Aug 22, 2011 at 12:04 PM, Ryan Ramage <ry...@gmail.com>
> wrote:
> > How to I grant mega points for a fantastic answer? Thanks Jason.
> >
> > A summary from the link....I have a master couchapp that clients pull
> > replicate from. A refactoring of the couchapp may mess up the rev number
> for
> > the main design doc. So i need a way to go past the rev number to force
> an
> > update to the clients. Is there a better way?
>
> Maybe a few ideas?
>

Here's another nasty way to trick CouchDB, just for fun:

PUT the new version of the document with whatever _rev you want and put
?new_edits=false in the query string.
Couch will accept the new revision no matter what. Replication uses this
feature to bypass the conflict rejections and merge conflicting trees.
Choose whatever number you feel like and abuse the definition of "winning"
revision so your newest one is what shows up :).

You should still DELETE the old revisions or you'll proliferate conflicts
and that puts unnecessary burden on couch in some places.

-Randall

Re: What's the best way to jack up the rev number?

[Adam Kocoloski <ko...@apache.org>]

On Aug 22, 2011, at 5:03 AM, Jason Smith wrote:

> I think also that BigCouch changes the _rev values.

Nope, it changes the update_seq of a database, because an integer isn't capable of representing all the details of the current state of the database shards.  The document representation stays the same.  Best,

Adam

Re: What's the best way to jack up the rev number?

[Jason Smith <jh...@iriscouch.com>]

On Mon, Aug 22, 2011 at 12:04 PM, Ryan Ramage <ry...@gmail.com> wrote:
> How to I grant mega points for a fantastic answer? Thanks Jason.
>
> A summary from the link....I have a master couchapp that clients pull
> replicate from. A refactoring of the couchapp may mess up the rev number for
> the main design doc. So i need a way to go past the rev number to force an
> update to the clients. Is there a better way?

Maybe a few ideas?

## The Bad Way: Sticking your fingers in _rev

It is a tempting illusion that _rev not only keeps a nice
AUTO_INCREMENT integer for every change, but it also contains an MD5
checksum of the entire document.

I think the _rev format has changed before, and it could change again.
I think also that BigCouch changes the _rev values.

## The Good Way: Work with _rev

I think your application has a concept of "upgrading" from one
revision to another. There is staging or development code, and there
is production code. Periodically you promote development code to
production. That sounds like two Git branches and it also sounds like
two doc ids. (Or two sets of doc ids.)

You can test and refactor your code all day long, in the temporary doc
(_design/dev). But in production (_design/pro), it's just like a long
Git history. Every revision built from the one previous, to the
beginning of time.

If you want to promote _design/dev, the latest deploy is
_rev=4-abcdef. So this will be the fifth revision deployed, right?
Hey! Stop reading the "_rev" field! But yeah, probably.

COPY /db/_design/dev
Destination: _design/pro?rev=4-abcdef

{"id":"_design/pro","rev":"5-12345whatever"}

Notice that each deployed _design/pro builds from the other, so it
will naturally float out to the slaves when they replicate.

In real-life, you may have add a middle step, pushing design documents
to production servers before actually publishing them. Once you push,
how long will it take couch to build new views? The answer is,
"Christ, who knows?"

Therefore you have to copy _design/dev to _design/staging and then
push that out into the wild. Then you have to query its views until
you are satisfied that they are fresh and fast. (You can compare
"update_seq" from /db vs. "update_seq" from /db/_design/ddoc/_info).
And only then do you HTTP copy from _design/staging to _design/pro and
let that propagate out.

## The Army Way: Create conflicts. Kill the weak.

You do not need a temporary "development" document. Just rebase all you want.

To deploy, note the good version's _rev and replicate to the central
server and you will have a conflict.

GET /db/_design/app?conflicts=true

{"_id":"_design/app", // ...
"_conflicts":["whatever", "old_revisions_here", "the_rev_i_like"]
}

You should spot the revision you like in there, so you can send out
DELETEs for all the rest (or batch them in _bulk_update).

Your tool might crash and forget the good _rev before replication
completes. I would say, just push again, but if you have a central
server, you can use a timestamp trick. The central maser is your
central time authority. Just `GET /` and check the "Date" header.

You can synchronize your client clock with this clock (it doesn't have
to be super accurate, within a few seconds--sometimes I time the query
and assume server time is the timestamp value plus half of the query
duration). You don't have to set the system clock, but you *do* have
to send and query timestamps which correspond with the server's
authority clock. (The server can sync with NTP, or it could even be
completely wrong, as long as we all agree it is the authority.)

When updating the design document, add an "updated_at" field.  This
value cannot be confirmed with validate_doc_update; so that is a
drawback.

If you've forgotten which revision is to be promoted (e.g. your
software crashed), you can fetch them all and take the latest
updated_at value.

-- 
Iris Couch

Re: What's the best way to jack up the rev number?

[Robert Newson <rn...@apache.org>]

Ryan,

As others have noted, you don't need to bump the 'number on the front'
to 'force' updates. Just update the document and replicate. If the
change isn't appearing on the target then the document has conflicting
revisions and your new update was one of the losers. Delete the
revisions *you* deem the losers, and it'll pop right up.

I confess to not finding illumination in comparisons between CouchDB
and Git, any analogy there tends to falls down at exactly the point
where the differences are important.

B.

On 22 August 2011 06:24, Jens Alfke <je...@couchbase.com> wrote:
>
> On Aug 21, 2011, at 10:04 PM, Ryan Ramage wrote:
>
> A summary from the link....I have a master couchapp that clients pull
> replicate from. A refactoring of the couchapp may mess up the rev number for
> the main design doc. So i need a way to go past the rev number to force an
> update to the clients. Is there a better way?
>
> The revision IDs aren’t “numbers” (again, just like Git). They do happen to have incrementing digits at the start, before the hashcode, but I’m told that’s just a detail of the implementation and shouldn’t be relied on. In the general case there isn’t any way to put an ordering on revisions, since different peers can be creating them out of contact with each other and then merging, so the replication algorithm doesn’t pay any attention to those digits at the beginning.
>
> —Jens
>

Re: What's the best way to jack up the rev number?

[Jens Alfke <je...@couchbase.com>]

On Aug 21, 2011, at 10:04 PM, Ryan Ramage wrote:

A summary from the link....I have a master couchapp that clients pull
replicate from. A refactoring of the couchapp may mess up the rev number for
the main design doc. So i need a way to go past the rev number to force an
update to the clients. Is there a better way?

The revision IDs aren’t “numbers” (again, just like Git). They do happen to have incrementing digits at the start, before the hashcode, but I’m told that’s just a detail of the implementation and shouldn’t be relied on. In the general case there isn’t any way to put an ordering on revisions, since different peers can be creating them out of contact with each other and then merging, so the replication algorithm doesn’t pay any attention to those digits at the beginning.

—Jens

Re: What's the best way to jack up the rev number?

[Ryan Ramage <ry...@gmail.com>]

How to I grant mega points for a fantastic answer? Thanks Jason.

A summary from the link....I have a master couchapp that clients pull
replicate from. A refactoring of the couchapp may mess up the rev number for
the main design doc. So i need a way to go past the rev number to force an
update to the clients. Is there a better way?
On Aug 21, 2011 9:40 PM, "Jason Smith" <jh...@iriscouch.com> wrote:
> On Mon, Aug 22, 2011 at 8:43 AM, Ryan Ramage <ry...@gmail.com>
wrote:
>> I have a need to make a document revision number go up a lot. eg from
>> 245-xxxx...    to 430-xxxxx....
>>
>> My reason for is here:
>>
http://groups.google.com/group/couchapp/browse_thread/thread/c51903ad5096fe2
>>
>>
>> What is the fastest/best way of doing this?
>
> ## Preamble
>
> CouchDB is a lot like Git.
>
> In Git, you have huge freedom to rewrite the commit history. If you
> have special needs, you can make radical changes to commits and
> branches (which are commit histories, all aggregated together).
>
> In CouchDB, you have huge freedom to rewrite the revision history. If
> you have special needs, you can make radical changes to revision
> histories and documents (which are revision histories, all aggregated
> together).
>
> In other words, create a conflict (in Git these are called "branches")
> and demote (delete) the unwanted document revision.
>
> In other words, suppose your doc _rev is "245-abcdef".
>
> POST /db/_bulk_docs
> Content-Type: application/json
>
> { "all_or_nothing": true
> , "docs": [ {"_id", "some_doc", "this_works":true, "_rev":"430-fedcba"} ]
> }
>
> Response: [{"id":"some_doc",
"rev":"431-ab4935e188dc777d59b5fcfc2d942f29"}]
>
> **Note** the revision is not what you requested, but rather the
> subsequent revision, which makes sense.
>
> DELETE /db/some_doc?rev=245-abcdef
>
> Response: {"ok":true, "id":"some_doc",
> "rev":"2-eec205a9d413992850a6e32678485900"}
>
> P.S. Sorry, I did not read your link explaining *why* you need this. I
> am only addressing how to do it. IMO, tracking deployment versions,
> build numbers, etc. is application-level data and therefore should not
> depend in _rev. I see the temptation, though. _rev looks a lot like a
> vector clock plus checksum of the document contents.
>
> --
> Iris Couch

Re: What's the best way to jack up the rev number?

[Jason Smith <jh...@iriscouch.com>]

On Mon, Aug 22, 2011 at 8:43 AM, Ryan Ramage <ry...@gmail.com> wrote:
> I have a need to make a document revision number go up a lot. eg from
> 245-xxxx...    to 430-xxxxx....
>
> My reason for is here:
> http://groups.google.com/group/couchapp/browse_thread/thread/c51903ad5096fe2
>
>
> What is the fastest/best way of doing this?

## Preamble

CouchDB is a lot like Git.

In Git, you have huge freedom to rewrite the commit history. If you
have special needs, you can make radical changes to commits and
branches (which are commit histories, all aggregated together).

In CouchDB, you have huge freedom to rewrite the revision history. If
you have special needs, you can make radical changes to revision
histories and documents (which are revision histories, all aggregated
together).

In other words, create a conflict (in Git these are called "branches")
and demote (delete) the unwanted document revision.

In other words, suppose your doc _rev is "245-abcdef".

POST /db/_bulk_docs
Content-Type: application/json

{ "all_or_nothing": true
, "docs": [ {"_id", "some_doc", "this_works":true, "_rev":"430-fedcba"} ]
}

Response: [{"id":"some_doc", "rev":"431-ab4935e188dc777d59b5fcfc2d942f29"}]

**Note** the revision is not what you requested, but rather the
subsequent revision, which makes sense.

DELETE /db/some_doc?rev=245-abcdef

Response: {"ok":true, "id":"some_doc",
"rev":"2-eec205a9d413992850a6e32678485900"}

P.S. Sorry, I did not read your link explaining *why* you need this. I
am only addressing how to do it. IMO, tracking deployment versions,
build numbers, etc. is application-level data and therefore should not
depend in _rev. I see the temptation, though. _rev looks a lot like a
vector clock plus checksum of the document contents.

-- 
Iris Couch