Multi doc transactions

[Paul Carey <pa...@gmail.com>]

Hi all

Given that multi doc updates that succeed in spite of update conflict
seem to be the way of the future for CouchDB, I'm trying to understand
how best to ensure single node consistency.

Consider a meeting room in a virtual world - thousands of users, a
public API and any number of API clients.

The desired behaviour is simple - users book rooms for meetings and
meeting notification emails are sent to invitees.

A room can host only a single meeting at any one time. CouchDB will
help enforce this if we set the booking id to be a hash of the room
and meeting time. (For simplicity I'm assuming all meetings start at
the same time and have the same duration).

We also want to create invite docs when we create the booking. The
invites are used to send notification emails and they may be accepted
or rejected by users.

Using a bulk update that fails if any doc update conflict occurs, we
know that the booking and invites are saved safely together and that a
duplicate booking hasn't been created.

Using a bulk update that doesn't fail if any doc update conflict
occurs, we either
i) save the booking and invites, resulting in an update conflict if
the booking is a duplicate. If a conflict occurred, we delete the
invite docs. We also need to ensure that emails aren't sent to
invitees before we've had a chance to delete spurious invite docs.
ii) save the booking and if it succeeds, then save the invites. The
issue here is node failure after saving the booking, but before saving
the invites.

An extension to ii) to ensure consistency would be
1. save the booking doc with a state property set to "incomplete"
2. if the booking doc save succeeds, use the bulk update to save the
invites, and update the booking doc with state "complete".

It's not very elegant, but I think it should work. I'd love to hear
better solutions.

Paul

documents as state machines

[Rich Morin <rd...@cfcl.com>]

At 13:20 -0800 2/9/09, Chris Anderson wrote:
> There's a lot of cool things you can do when you treat
> documents as state machines.  ...

This is a nifty topic, well worth inclusion in the book, IMHO.

I have built processing networks out of scripts and YAML files,
using generated make files to keep the outputs in sync with the
inputs.  In a CouchDB-based system, daemons (ie, bots) could be
set up to move the state of the files forward.  Whether a file
gets modified (or simply used as input) depends on the design.

-r
-- 
http://www.cfcl.com/rdm            Rich Morin
http://www.cfcl.com/rdm/resume     rdm@cfcl.com
http://www.cfcl.com/rdm/weblog     +1 650-873-7841

Technical editing and writing, programming, and web development

Re: Multi doc transactions

[kowsik <ko...@gmail.com>]

On www.pcapr.net, we use couch as an event bus (besides other things)
to be picked up a Helma (Java/JavaScript) thread for sending emails
and such. For one thing, if we ever bring the site down for
maintenance, all of the "pending" events just sit in persistent
storage within couch and we can pull in all those events just by a
single view sorted by timestamp. Obviously, each event itself is a
JSON document. One advantage of this is event-rollup where you really
only want to send one email that aggregates (map/reduce) a bunch of
events into a single email to avoid spamming users.

I'm not sure if this is "best-practices", but it works like a charm
and we don't have to resort to writing to a file or using memcache for
this transient storage.

K.

On Mon, Feb 9, 2009 at 3:54 PM, Paul Carey <pa...@gmail.com> wrote:
>> There's a lot of cool things you can do when you treat documents as
>> state machines. Also, getting a view of "complete" bookings will be
>> easy.
>>
>> Docs as state machines also works well in a multi-master environment,
>> where you want to watch for "events" to be replicated to redundant
>> storage.
>
> I've been using document state to allow some operations to be
> idempotent - this has been working well. But I felt less comfortable
> using state to effectively demarcate transactions. Your reply has
> given me greater confidence with this approach. Thanks Chris.
>
> Paul
>

Re: Multi doc transactions

[Paul Carey <pa...@gmail.com>]

> There's a lot of cool things you can do when you treat documents as
> state machines. Also, getting a view of "complete" bookings will be
> easy.
>
> Docs as state machines also works well in a multi-master environment,
> where you want to watch for "events" to be replicated to redundant
> storage.

I've been using document state to allow some operations to be
idempotent - this has been working well. But I felt less comfortable
using state to effectively demarcate transactions. Your reply has
given me greater confidence with this approach. Thanks Chris.

Paul

Re: Multi doc transactions

[Chris Anderson <jc...@apache.org>]

On Mon, Feb 9, 2009 at 1:13 PM, Paul Carey <pa...@gmail.com> wrote:
> An extension to ii) to ensure consistency would be
> 1. save the booking doc with a state property set to "incomplete"
> 2. if the booking doc save succeeds, use the bulk update to save the
> invites, and update the booking doc with state "complete".
>
> It's not very elegant, but I think it should work. I'd love to hear
> better solutions.

There's a lot of cool things you can do when you treat documents as
state machines. Also, getting a view of "complete" bookings will be
easy.

Docs as state machines also works well in a multi-master environment,
where you want to watch for "events" to be replicated to redundant
storage.

-- 
Chris Anderson
http://jchris.mfdz.com