You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Brian Candler <B....@pobox.com> on 2009/04/06 09:40:01 UTC
Removing PUT 409?
The following is part thought-experiment, part serious suggestion.
I propose the following: remove all concurrency control from PUT operations,
and hence also the 409 response. If you PUT a document where the _rev is not
the same as a 'head' revision, then a new conflicting version is inserted.
[1]
The reasoning is as follows:
1. Any application which relies on the 409 PUT conflict behaviour is
not going to work properly in a multi-master replication environment.
That is: it is protected against concurrent changes on the same node,
but not on a different node. This is arbitrary.
2. The same reasoning was used for getting rid of bulk non-conflicting
updates. Paraphrasing: "a grown-up CouchDB app which runs on a replicated
cluster won't be able to rely on these semantics, so removing this
capability will encourage you to write your app in a more scalable way.
You will thank us later."
3. A CouchDB app should be written so that it "treats edit conflicts as a
common state, not an exceptional one" [2]
This change will slightly increase the number of these normal conflicts,
whilst forcing the app writer to deal with them.
4. By increasing the number of conflicting versions, it is likely to
exercise more the underlying code and flush out bugs (for example, more
fully testing what happens in views when multiple conflicting versions of
a document are updated or removed)
5. It may highlight more clearly where API improvements are needed to help
applications deal with and resolve conflicts. For example:
- making it easier for applications to be aware of the existence of
conflicts (Maybe a GET without _rev should fail if there are multiple
conflicting revs, or return all of the versions)
- given that multiple concurrent clients will see conflicts, and may
attempt to resolve them at the same time, then it's likely that two
clients will independently submit exactly the same document content
after running the conflict-resolution algorithm. It could be helpful
if these were treated as a single new rev, and not two new conflicts.
Comments? I would be especially interested in hearing from core developers
who didn't want bulk non-conflicting updates, but *do* want to retain single
non-conflicting updates, as to why this is logical.
Regards,
Brian.
[1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
{"all_or_nothing":true}
[2] http://couchdb.apache.org/docs/overview.html under heading "Conflicts"
Re: Removing PUT 409?
Posted by Brian Candler <B....@pobox.com>.
On Mon, Apr 06, 2009 at 12:21:17PM -0400, Damien Katz wrote:
> The interactive conflict behavior in CouchDB is useful for completely
> eliminating persisted conflicts in a lot of replicated circumstances,
> such as master-slave replication, or in instances where different
> documents will be only edited at a single node (like documents
> pertaining to different branches offices) but replicated amongst all
> nodes.
That's fine. But why doesn't the same argument apply to the removed
_bulk_docs transaction feature?
Re: Removing PUT 409?
Posted by Damien Katz <da...@apache.org>.
You can already get the proposed behavior by using _bulk_docs POST and
all_or_nothing:true with a single document. We can easily add a flag
to single doc PUTs too.
The interactive conflict behavior in CouchDB is useful for completely
eliminating persisted conflicts in a lot of replicated circumstances,
such as master-slave replication, or in instances where different
documents will be only edited at a single node (like documents
pertaining to different branches offices) but replicated amongst all
nodes.
-Damien
On Apr 6, 2009, at 3:40 AM, Brian Candler wrote:
> The following is part thought-experiment, part serious suggestion.
>
> I propose the following: remove all concurrency control from PUT
> operations,
> and hence also the 409 response. If you PUT a document where the
> _rev is not
> the same as a 'head' revision, then a new conflicting version is
> inserted.
> [1]
>
> The reasoning is as follows:
>
> 1. Any application which relies on the 409 PUT conflict behaviour is
> not going to work properly in a multi-master replication
> environment.
> That is: it is protected against concurrent changes on the same
> node,
> but not on a different node. This is arbitrary.
>
> 2. The same reasoning was used for getting rid of bulk non-conflicting
> updates. Paraphrasing: "a grown-up CouchDB app which runs on a
> replicated
> cluster won't be able to rely on these semantics, so removing this
> capability will encourage you to write your app in a more scalable
> way.
> You will thank us later."
>
> 3. A CouchDB app should be written so that it "treats edit conflicts
> as a
> common state, not an exceptional one" [2]
>
> This change will slightly increase the number of these normal
> conflicts,
> whilst forcing the app writer to deal with them.
>
> 4. By increasing the number of conflicting versions, it is likely to
> exercise more the underlying code and flush out bugs (for example,
> more
> fully testing what happens in views when multiple conflicting
> versions of
> a document are updated or removed)
>
> 5. It may highlight more clearly where API improvements are needed
> to help
> applications deal with and resolve conflicts. For example:
>
> - making it easier for applications to be aware of the existence of
> conflicts (Maybe a GET without _rev should fail if there are
> multiple
> conflicting revs, or return all of the versions)
>
> - given that multiple concurrent clients will see conflicts, and may
> attempt to resolve them at the same time, then it's likely that
> two
> clients will independently submit exactly the same document
> content
> after running the conflict-resolution algorithm. It could be
> helpful
> if these were treated as a single new rev, and not two new
> conflicts.
>
> Comments? I would be especially interested in hearing from core
> developers
> who didn't want bulk non-conflicting updates, but *do* want to
> retain single
> non-conflicting updates, as to why this is logical.
>
> Regards,
>
> Brian.
>
> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
> {"all_or_nothing":true}
>
> [2] http://couchdb.apache.org/docs/overview.html under heading
> "Conflicts"
Re: Removing PUT 409?
Posted by Brian Candler <B....@pobox.com>.
On Mon, Apr 06, 2009 at 12:40:33PM -0700, Scott Shumaker wrote:
> Getting rid of conflict handling greatly limits the utility of CouchDB
> for real-world applications (it will certainly force us to adopt
> another technology instead). And worse, this is all for the goal of
> supporting multi-master replication, which really isn't a great
> technology solution anyway. If you want durability and scalability,
> CouchDB should really adopt the much more robust multiple write nodes
> / read nodes system (with quorum and reconciliation) in Dynamo or a
> few other distributed key/value stores.
There's a Dynamo paper here:
http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf
The write primitive is put(key, context, object) where 'context' includes
versioning info. Does put() ever fail simply because the object context is
stale? From reading this paper I get the impression that it doesn't, but I
could be wrong. [1]
I do like the idea that when there are multiple versions, the get()
primitive returns all of them, together with a context which refers to all
the versions included. A put() in this context will then automatically mark
the conflict as resolved.
I think it would be good for CouchDB to handle GET in the same way, rather
than picking a single arbitrary version. However, CouchDB also needs to deal
with multiple versions in views too. Would it be reasonable for a view to
contain data from *all* the live versions of a document, rather than just
one? Each map row is tagged with "id" already, so it could be tagged with
"id" and "rev".
Regards,
Brian.
[1] Section 6.3 says:
"Divergent versions of a data item arise in two scenarios. ...
The second is when the system is handling a large number of concurrent
writers to a single data item and multiple nodes end up coordinating the
updates concurrently."
Re: Removing PUT 409?
Posted by Scott Shumaker <ss...@gmail.com>.
Fair enough - apparently I've just gotten concerned CouchDB was
heading in a new direction w.r.t. conflicts - especially after the
latest changes made bulk_docs no longer transactional (which is
already causing issues for us in our staging environment. I'd really
like an option to have it to fail entirely if there's any conflicts).
Do you think there's any merit to to adding a new mode to to bulk_docs
to yield the old transactional behavior - except re-implement it using
hidden revisions (as I suggested earlier) to represent the in-progress
state of the transaction? This would allow the mode to work with
sharding, at the cost of being twice as expensive (requiring two
writes instead of one).
Scott
On Tue, Apr 7, 2009 at 3:51 AM, Jan Lehnardt <ja...@apache.org> wrote:
> Hi Scott,
>
> thanks for raising your concerns. I share it, but I think Brian and
> Adam are only suggesting an optional addition to the existing
> API which leaves the existing case in place. Much like bulk docs
> now has two modes, PUT can have two modes.
>
> Somthing like
>
> PUT /db/doc?rev=foo&allow_conflicts=true
> {"json":"body"}
>
> I wouldn't be opposed to add this.
>
> Cheers
> Jan
> --
>
>
> On 6 Apr 2009, at 20:40, Scott Shumaker wrote:
>
>> Just my $0.02, but I think CouchDB is moving in entirely the wrong
>> direction with conflicts in a misguided attempt to make multi-master
>> replication the 'only' way to do things.
>>
>> Very frequently, you need to attempt to resolve a conflict as soon as
>> it occurs - and you often need user interaction to help you resolve
>> the conflict. Sometimes you may need to just refresh the user to the
>> latest version, other times you may be able to choose one of the
>> versions based on some criteria, sometimes you can automatically merge
>> the two versions, and occasionally you need to ask the user what to
>> do. This just won't work if the process is happening offline, in a
>> background job.
>>
>> This isn't just true of CouchDB, but of other distributed systems like
>> Dynamo (read the paper, they talk about this exact issue. Amazon.com
>> has a "merge shopping carts" screen for this exact reason).
>>
>> Getting rid of conflict handling greatly limits the utility of CouchDB
>> for real-world applications (it will certainly force us to adopt
>> another technology instead). And worse, this is all for the goal of
>> supporting multi-master replication, which really isn't a great
>> technology solution anyway. If you want durability and scalability,
>> CouchDB should really adopt the much more robust multiple write nodes
>> / read nodes system (with quorum and reconciliation) in Dynamo or a
>> few other distributed key/value stores.
>>
>> Scott
>>
>>
>> On Mon, Apr 6, 2009 at 12:40 AM, Brian Candler <B....@pobox.com>
>> wrote:
>>>
>>> The following is part thought-experiment, part serious suggestion.
>>>
>>> I propose the following: remove all concurrency control from PUT
>>> operations,
>>> and hence also the 409 response. If you PUT a document where the _rev is
>>> not
>>> the same as a 'head' revision, then a new conflicting version is
>>> inserted.
>>> [1]
>>>
>>> The reasoning is as follows:
>>>
>>> 1. Any application which relies on the 409 PUT conflict behaviour is
>>> not going to work properly in a multi-master replication environment.
>>> That is: it is protected against concurrent changes on the same node,
>>> but not on a different node. This is arbitrary.
>>>
>>> 2. The same reasoning was used for getting rid of bulk non-conflicting
>>> updates. Paraphrasing: "a grown-up CouchDB app which runs on a
>>> replicated
>>> cluster won't be able to rely on these semantics, so removing this
>>> capability will encourage you to write your app in a more scalable way.
>>> You will thank us later."
>>>
>>> 3. A CouchDB app should be written so that it "treats edit conflicts as a
>>> common state, not an exceptional one" [2]
>>>
>>> This change will slightly increase the number of these normal conflicts,
>>> whilst forcing the app writer to deal with them.
>>>
>>> 4. By increasing the number of conflicting versions, it is likely to
>>> exercise more the underlying code and flush out bugs (for example, more
>>> fully testing what happens in views when multiple conflicting versions
>>> of
>>> a document are updated or removed)
>>>
>>> 5. It may highlight more clearly where API improvements are needed to
>>> help
>>> applications deal with and resolve conflicts. For example:
>>>
>>> - making it easier for applications to be aware of the existence of
>>> conflicts (Maybe a GET without _rev should fail if there are multiple
>>> conflicting revs, or return all of the versions)
>>>
>>> - given that multiple concurrent clients will see conflicts, and may
>>> attempt to resolve them at the same time, then it's likely that two
>>> clients will independently submit exactly the same document content
>>> after running the conflict-resolution algorithm. It could be helpful
>>> if these were treated as a single new rev, and not two new conflicts.
>>>
>>> Comments? I would be especially interested in hearing from core
>>> developers
>>> who didn't want bulk non-conflicting updates, but *do* want to retain
>>> single
>>> non-conflicting updates, as to why this is logical.
>>>
>>> Regards,
>>>
>>> Brian.
>>>
>>> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
>>> {"all_or_nothing":true}
>>>
>>> [2] http://couchdb.apache.org/docs/overview.html under heading
>>> "Conflicts"
>>>
>>
>
>
Re: Removing PUT 409?
Posted by Jan Lehnardt <ja...@apache.org>.
On 7 Apr 2009, at 12:59, Jeff Hinrichs - DM&T wrote:
> On Tue, Apr 7, 2009 at 5:51 AM, Jan Lehnardt <ja...@apache.org> wrote:
>
>> Hi Scott,
>>
>> thanks for raising your concerns. I share it, but I think Brian and
>> Adam are only suggesting an optional addition to the existing
>> API which leaves the existing case in place. Much like bulk docs
>> now has two modes, PUT can have two modes.
>>
>> Somthing like
>>
>> PUT /db/doc?rev=foo&allow_conflicts=true
>> {"json":"body"}
>
>
> If I haven't missed the point, I would favor the _rev being in the
> body of
> the json document
> PUT /db/doc?allow_conflicts=true {"_rev":"somerev","json":"body"}
This was just for example reasons, not for detailing the API :) I think
your idea makes sense, but I think we're not yet at the point of
discussing
API details :)
Cheers
Jan
--
>
>
>> I wouldn't be opposed to add this.
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On 6 Apr 2009, at 20:40, Scott Shumaker wrote:
>>
>> Just my $0.02, but I think CouchDB is moving in entirely the wrong
>>> direction with conflicts in a misguided attempt to make multi-master
>>> replication the 'only' way to do things.
>>>
>>> Very frequently, you need to attempt to resolve a conflict as soon
>>> as
>>> it occurs - and you often need user interaction to help you resolve
>>> the conflict. Sometimes you may need to just refresh the user to
>>> the
>>> latest version, other times you may be able to choose one of the
>>> versions based on some criteria, sometimes you can automatically
>>> merge
>>> the two versions, and occasionally you need to ask the user what to
>>> do. This just won't work if the process is happening offline, in a
>>> background job.
>>>
>>> This isn't just true of CouchDB, but of other distributed systems
>>> like
>>> Dynamo (read the paper, they talk about this exact issue.
>>> Amazon.com
>>> has a "merge shopping carts" screen for this exact reason).
>>>
>>> Getting rid of conflict handling greatly limits the utility of
>>> CouchDB
>>> for real-world applications (it will certainly force us to adopt
>>> another technology instead). And worse, this is all for the goal of
>>> supporting multi-master replication, which really isn't a great
>>> technology solution anyway. If you want durability and scalability,
>>> CouchDB should really adopt the much more robust multiple write
>>> nodes
>>> / read nodes system (with quorum and reconciliation) in Dynamo or a
>>> few other distributed key/value stores.
>>>
>>> Scott
>>>
>>>
>>> On Mon, Apr 6, 2009 at 12:40 AM, Brian Candler <B....@pobox.com>
>>> wrote:
>>>
>>>> The following is part thought-experiment, part serious suggestion.
>>>>
>>>> I propose the following: remove all concurrency control from PUT
>>>> operations,
>>>> and hence also the 409 response. If you PUT a document where the
>>>> _rev is
>>>> not
>>>> the same as a 'head' revision, then a new conflicting version is
>>>> inserted.
>>>> [1]
>>>>
>>>> The reasoning is as follows:
>>>>
>>>> 1. Any application which relies on the 409 PUT conflict behaviour
>>>> is
>>>> not going to work properly in a multi-master replication
>>>> environment.
>>>> That is: it is protected against concurrent changes on the same
>>>> node,
>>>> but not on a different node. This is arbitrary.
>>>>
>>>> 2. The same reasoning was used for getting rid of bulk non-
>>>> conflicting
>>>> updates. Paraphrasing: "a grown-up CouchDB app which runs on a
>>>> replicated
>>>> cluster won't be able to rely on these semantics, so removing this
>>>> capability will encourage you to write your app in a more
>>>> scalable way.
>>>> You will thank us later."
>>>>
>>>> 3. A CouchDB app should be written so that it "treats edit
>>>> conflicts as a
>>>> common state, not an exceptional one" [2]
>>>>
>>>> This change will slightly increase the number of these normal
>>>> conflicts,
>>>> whilst forcing the app writer to deal with them.
>>>>
>>>> 4. By increasing the number of conflicting versions, it is likely
>>>> to
>>>> exercise more the underlying code and flush out bugs (for
>>>> example, more
>>>> fully testing what happens in views when multiple conflicting
>>>> versions
>>>> of
>>>> a document are updated or removed)
>>>>
>>>> 5. It may highlight more clearly where API improvements are
>>>> needed to
>>>> help
>>>> applications deal with and resolve conflicts. For example:
>>>>
>>>> - making it easier for applications to be aware of the existence of
>>>> conflicts (Maybe a GET without _rev should fail if there are
>>>> multiple
>>>> conflicting revs, or return all of the versions)
>>>>
>>>> - given that multiple concurrent clients will see conflicts, and
>>>> may
>>>> attempt to resolve them at the same time, then it's likely that
>>>> two
>>>> clients will independently submit exactly the same document
>>>> content
>>>> after running the conflict-resolution algorithm. It could be
>>>> helpful
>>>> if these were treated as a single new rev, and not two new
>>>> conflicts.
>>>>
>>>> Comments? I would be especially interested in hearing from core
>>>> developers
>>>> who didn't want bulk non-conflicting updates, but *do* want to
>>>> retain
>>>> single
>>>> non-conflicting updates, as to why this is logical.
>>>>
>>>> Regards,
>>>>
>>>> Brian.
>>>>
>>>> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs
>>>> with
>>>> {"all_or_nothing":true}
>>>>
>>>> [2] http://couchdb.apache.org/docs/overview.html under heading
>>>> "Conflicts"
>>>>
>>>>
>>>
>>
Re: Removing PUT 409?
Posted by Jeff Hinrichs - DM&T <du...@gmail.com>.
On Tue, Apr 7, 2009 at 5:51 AM, Jan Lehnardt <ja...@apache.org> wrote:
> Hi Scott,
>
> thanks for raising your concerns. I share it, but I think Brian and
> Adam are only suggesting an optional addition to the existing
> API which leaves the existing case in place. Much like bulk docs
> now has two modes, PUT can have two modes.
>
> Somthing like
>
> PUT /db/doc?rev=foo&allow_conflicts=true
> {"json":"body"}
If I haven't missed the point, I would favor the _rev being in the body of
the json document
PUT /db/doc?allow_conflicts=true {"_rev":"somerev","json":"body"}
This would allow one to restore a dump without having to process each json
document (removing the _rev attribute) before submitting.
Regards,
Jeff Hinrichs
> I wouldn't be opposed to add this.
>
> Cheers
> Jan
> --
>
>
> On 6 Apr 2009, at 20:40, Scott Shumaker wrote:
>
> Just my $0.02, but I think CouchDB is moving in entirely the wrong
>> direction with conflicts in a misguided attempt to make multi-master
>> replication the 'only' way to do things.
>>
>> Very frequently, you need to attempt to resolve a conflict as soon as
>> it occurs - and you often need user interaction to help you resolve
>> the conflict. Sometimes you may need to just refresh the user to the
>> latest version, other times you may be able to choose one of the
>> versions based on some criteria, sometimes you can automatically merge
>> the two versions, and occasionally you need to ask the user what to
>> do. This just won't work if the process is happening offline, in a
>> background job.
>>
>> This isn't just true of CouchDB, but of other distributed systems like
>> Dynamo (read the paper, they talk about this exact issue. Amazon.com
>> has a "merge shopping carts" screen for this exact reason).
>>
>> Getting rid of conflict handling greatly limits the utility of CouchDB
>> for real-world applications (it will certainly force us to adopt
>> another technology instead). And worse, this is all for the goal of
>> supporting multi-master replication, which really isn't a great
>> technology solution anyway. If you want durability and scalability,
>> CouchDB should really adopt the much more robust multiple write nodes
>> / read nodes system (with quorum and reconciliation) in Dynamo or a
>> few other distributed key/value stores.
>>
>> Scott
>>
>>
>> On Mon, Apr 6, 2009 at 12:40 AM, Brian Candler <B....@pobox.com>
>> wrote:
>>
>>> The following is part thought-experiment, part serious suggestion.
>>>
>>> I propose the following: remove all concurrency control from PUT
>>> operations,
>>> and hence also the 409 response. If you PUT a document where the _rev is
>>> not
>>> the same as a 'head' revision, then a new conflicting version is
>>> inserted.
>>> [1]
>>>
>>> The reasoning is as follows:
>>>
>>> 1. Any application which relies on the 409 PUT conflict behaviour is
>>> not going to work properly in a multi-master replication environment.
>>> That is: it is protected against concurrent changes on the same node,
>>> but not on a different node. This is arbitrary.
>>>
>>> 2. The same reasoning was used for getting rid of bulk non-conflicting
>>> updates. Paraphrasing: "a grown-up CouchDB app which runs on a
>>> replicated
>>> cluster won't be able to rely on these semantics, so removing this
>>> capability will encourage you to write your app in a more scalable way.
>>> You will thank us later."
>>>
>>> 3. A CouchDB app should be written so that it "treats edit conflicts as a
>>> common state, not an exceptional one" [2]
>>>
>>> This change will slightly increase the number of these normal conflicts,
>>> whilst forcing the app writer to deal with them.
>>>
>>> 4. By increasing the number of conflicting versions, it is likely to
>>> exercise more the underlying code and flush out bugs (for example, more
>>> fully testing what happens in views when multiple conflicting versions
>>> of
>>> a document are updated or removed)
>>>
>>> 5. It may highlight more clearly where API improvements are needed to
>>> help
>>> applications deal with and resolve conflicts. For example:
>>>
>>> - making it easier for applications to be aware of the existence of
>>> conflicts (Maybe a GET without _rev should fail if there are multiple
>>> conflicting revs, or return all of the versions)
>>>
>>> - given that multiple concurrent clients will see conflicts, and may
>>> attempt to resolve them at the same time, then it's likely that two
>>> clients will independently submit exactly the same document content
>>> after running the conflict-resolution algorithm. It could be helpful
>>> if these were treated as a single new rev, and not two new conflicts.
>>>
>>> Comments? I would be especially interested in hearing from core
>>> developers
>>> who didn't want bulk non-conflicting updates, but *do* want to retain
>>> single
>>> non-conflicting updates, as to why this is logical.
>>>
>>> Regards,
>>>
>>> Brian.
>>>
>>> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
>>> {"all_or_nothing":true}
>>>
>>> [2] http://couchdb.apache.org/docs/overview.html under heading
>>> "Conflicts"
>>>
>>>
>>
>
Re: Removing PUT 409?
Posted by Jan Lehnardt <ja...@apache.org>.
Hi Scott,
thanks for raising your concerns. I share it, but I think Brian and
Adam are only suggesting an optional addition to the existing
API which leaves the existing case in place. Much like bulk docs
now has two modes, PUT can have two modes.
Somthing like
PUT /db/doc?rev=foo&allow_conflicts=true
{"json":"body"}
I wouldn't be opposed to add this.
Cheers
Jan
--
On 6 Apr 2009, at 20:40, Scott Shumaker wrote:
> Just my $0.02, but I think CouchDB is moving in entirely the wrong
> direction with conflicts in a misguided attempt to make multi-master
> replication the 'only' way to do things.
>
> Very frequently, you need to attempt to resolve a conflict as soon as
> it occurs - and you often need user interaction to help you resolve
> the conflict. Sometimes you may need to just refresh the user to the
> latest version, other times you may be able to choose one of the
> versions based on some criteria, sometimes you can automatically merge
> the two versions, and occasionally you need to ask the user what to
> do. This just won't work if the process is happening offline, in a
> background job.
>
> This isn't just true of CouchDB, but of other distributed systems like
> Dynamo (read the paper, they talk about this exact issue. Amazon.com
> has a "merge shopping carts" screen for this exact reason).
>
> Getting rid of conflict handling greatly limits the utility of CouchDB
> for real-world applications (it will certainly force us to adopt
> another technology instead). And worse, this is all for the goal of
> supporting multi-master replication, which really isn't a great
> technology solution anyway. If you want durability and scalability,
> CouchDB should really adopt the much more robust multiple write nodes
> / read nodes system (with quorum and reconciliation) in Dynamo or a
> few other distributed key/value stores.
>
> Scott
>
>
> On Mon, Apr 6, 2009 at 12:40 AM, Brian Candler <B....@pobox.com>
> wrote:
>> The following is part thought-experiment, part serious suggestion.
>>
>> I propose the following: remove all concurrency control from PUT
>> operations,
>> and hence also the 409 response. If you PUT a document where the
>> _rev is not
>> the same as a 'head' revision, then a new conflicting version is
>> inserted.
>> [1]
>>
>> The reasoning is as follows:
>>
>> 1. Any application which relies on the 409 PUT conflict behaviour is
>> not going to work properly in a multi-master replication
>> environment.
>> That is: it is protected against concurrent changes on the same
>> node,
>> but not on a different node. This is arbitrary.
>>
>> 2. The same reasoning was used for getting rid of bulk non-
>> conflicting
>> updates. Paraphrasing: "a grown-up CouchDB app which runs on a
>> replicated
>> cluster won't be able to rely on these semantics, so removing this
>> capability will encourage you to write your app in a more
>> scalable way.
>> You will thank us later."
>>
>> 3. A CouchDB app should be written so that it "treats edit
>> conflicts as a
>> common state, not an exceptional one" [2]
>>
>> This change will slightly increase the number of these normal
>> conflicts,
>> whilst forcing the app writer to deal with them.
>>
>> 4. By increasing the number of conflicting versions, it is likely to
>> exercise more the underlying code and flush out bugs (for
>> example, more
>> fully testing what happens in views when multiple conflicting
>> versions of
>> a document are updated or removed)
>>
>> 5. It may highlight more clearly where API improvements are needed
>> to help
>> applications deal with and resolve conflicts. For example:
>>
>> - making it easier for applications to be aware of the existence of
>> conflicts (Maybe a GET without _rev should fail if there are
>> multiple
>> conflicting revs, or return all of the versions)
>>
>> - given that multiple concurrent clients will see conflicts, and
>> may
>> attempt to resolve them at the same time, then it's likely that
>> two
>> clients will independently submit exactly the same document
>> content
>> after running the conflict-resolution algorithm. It could be
>> helpful
>> if these were treated as a single new rev, and not two new
>> conflicts.
>>
>> Comments? I would be especially interested in hearing from core
>> developers
>> who didn't want bulk non-conflicting updates, but *do* want to
>> retain single
>> non-conflicting updates, as to why this is logical.
>>
>> Regards,
>>
>> Brian.
>>
>> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
>> {"all_or_nothing":true}
>>
>> [2] http://couchdb.apache.org/docs/overview.html under heading
>> "Conflicts"
>>
>
Re: Removing PUT 409?
Posted by Scott Shumaker <ss...@gmail.com>.
Just my $0.02, but I think CouchDB is moving in entirely the wrong
direction with conflicts in a misguided attempt to make multi-master
replication the 'only' way to do things.
Very frequently, you need to attempt to resolve a conflict as soon as
it occurs - and you often need user interaction to help you resolve
the conflict. Sometimes you may need to just refresh the user to the
latest version, other times you may be able to choose one of the
versions based on some criteria, sometimes you can automatically merge
the two versions, and occasionally you need to ask the user what to
do. This just won't work if the process is happening offline, in a
background job.
This isn't just true of CouchDB, but of other distributed systems like
Dynamo (read the paper, they talk about this exact issue. Amazon.com
has a "merge shopping carts" screen for this exact reason).
Getting rid of conflict handling greatly limits the utility of CouchDB
for real-world applications (it will certainly force us to adopt
another technology instead). And worse, this is all for the goal of
supporting multi-master replication, which really isn't a great
technology solution anyway. If you want durability and scalability,
CouchDB should really adopt the much more robust multiple write nodes
/ read nodes system (with quorum and reconciliation) in Dynamo or a
few other distributed key/value stores.
Scott
On Mon, Apr 6, 2009 at 12:40 AM, Brian Candler <B....@pobox.com> wrote:
> The following is part thought-experiment, part serious suggestion.
>
> I propose the following: remove all concurrency control from PUT operations,
> and hence also the 409 response. If you PUT a document where the _rev is not
> the same as a 'head' revision, then a new conflicting version is inserted.
> [1]
>
> The reasoning is as follows:
>
> 1. Any application which relies on the 409 PUT conflict behaviour is
> not going to work properly in a multi-master replication environment.
> That is: it is protected against concurrent changes on the same node,
> but not on a different node. This is arbitrary.
>
> 2. The same reasoning was used for getting rid of bulk non-conflicting
> updates. Paraphrasing: "a grown-up CouchDB app which runs on a replicated
> cluster won't be able to rely on these semantics, so removing this
> capability will encourage you to write your app in a more scalable way.
> You will thank us later."
>
> 3. A CouchDB app should be written so that it "treats edit conflicts as a
> common state, not an exceptional one" [2]
>
> This change will slightly increase the number of these normal conflicts,
> whilst forcing the app writer to deal with them.
>
> 4. By increasing the number of conflicting versions, it is likely to
> exercise more the underlying code and flush out bugs (for example, more
> fully testing what happens in views when multiple conflicting versions of
> a document are updated or removed)
>
> 5. It may highlight more clearly where API improvements are needed to help
> applications deal with and resolve conflicts. For example:
>
> - making it easier for applications to be aware of the existence of
> conflicts (Maybe a GET without _rev should fail if there are multiple
> conflicting revs, or return all of the versions)
>
> - given that multiple concurrent clients will see conflicts, and may
> attempt to resolve them at the same time, then it's likely that two
> clients will independently submit exactly the same document content
> after running the conflict-resolution algorithm. It could be helpful
> if these were treated as a single new rev, and not two new conflicts.
>
> Comments? I would be especially interested in hearing from core developers
> who didn't want bulk non-conflicting updates, but *do* want to retain single
> non-conflicting updates, as to why this is logical.
>
> Regards,
>
> Brian.
>
> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
> {"all_or_nothing":true}
>
> [2] http://couchdb.apache.org/docs/overview.html under heading "Conflicts"
>
Re: Removing PUT 409?
Posted by Adam Kocoloski <ko...@apache.org>.
Hi Brian, there's something about your proposal that really appeals to
me. I wonder if there's another variant that might be useful --
insert a new conflicting version AND specify a 409 status code in the
response. The status code is a nice way to inform the client that a
conflict occurred.
--Adam
On Apr 6, 2009, at 3:40 AM, Brian Candler wrote:
> The following is part thought-experiment, part serious suggestion.
>
> I propose the following: remove all concurrency control from PUT
> operations,
> and hence also the 409 response. If you PUT a document where the
> _rev is not
> the same as a 'head' revision, then a new conflicting version is
> inserted.
> [1]
>
> The reasoning is as follows:
>
> 1. Any application which relies on the 409 PUT conflict behaviour is
> not going to work properly in a multi-master replication
> environment.
> That is: it is protected against concurrent changes on the same
> node,
> but not on a different node. This is arbitrary.
>
> 2. The same reasoning was used for getting rid of bulk non-conflicting
> updates. Paraphrasing: "a grown-up CouchDB app which runs on a
> replicated
> cluster won't be able to rely on these semantics, so removing this
> capability will encourage you to write your app in a more scalable
> way.
> You will thank us later."
>
> 3. A CouchDB app should be written so that it "treats edit conflicts
> as a
> common state, not an exceptional one" [2]
>
> This change will slightly increase the number of these normal
> conflicts,
> whilst forcing the app writer to deal with them.
>
> 4. By increasing the number of conflicting versions, it is likely to
> exercise more the underlying code and flush out bugs (for example,
> more
> fully testing what happens in views when multiple conflicting
> versions of
> a document are updated or removed)
>
> 5. It may highlight more clearly where API improvements are needed
> to help
> applications deal with and resolve conflicts. For example:
>
> - making it easier for applications to be aware of the existence of
> conflicts (Maybe a GET without _rev should fail if there are
> multiple
> conflicting revs, or return all of the versions)
>
> - given that multiple concurrent clients will see conflicts, and may
> attempt to resolve them at the same time, then it's likely that
> two
> clients will independently submit exactly the same document
> content
> after running the conflict-resolution algorithm. It could be
> helpful
> if these were treated as a single new rev, and not two new
> conflicts.
>
> Comments? I would be especially interested in hearing from core
> developers
> who didn't want bulk non-conflicting updates, but *do* want to
> retain single
> non-conflicting updates, as to why this is logical.
>
> Regards,
>
> Brian.
>
> [1] You can get this behaviour on 0.9.0 by POSTing to _bulk_docs with
> {"all_or_nothing":true}
>
> [2] http://couchdb.apache.org/docs/overview.html under heading
> "Conflicts"