reserving resources in Couch

[Wout Mertens <wm...@cisco.com>]

Ok,

(no actual code yet, I don't have time to code right now :( )

I have a project currently using an RDBMS and I'd like to port it to  
CouchDB. One of the things I do is lock a table, choose a free  
resource from a query on a static table and the session list, assign  
the resource to a new session and unlock the table.

How would I be able to do the same thing with CouchDB given that 2  
sessions could start at the same time? I do have the advantage that  
simultaneous starters would contact the same CouchDB instance.

I was thinking of using sums: make a view that calculates the sum of  
resources. A resource record would count as +1 and an in-use record  
would be -1.

Then when you reserve a resource, you save the in-use record. After  
saving, look up the sum for the resource you reserved. If it's not  
equal to 0, then use a stable algorithm to determine who has to  
release the resource again.

Would this close the race condition? Note that no documents are  
overwritten at reservation time, each reservation doubles as the event  
log. When the session clears up, the document that represents it is  
updated to release the resource.

Does this work? Is there a better way to do it?

Thanks,

Wout.

Re: [user] Re: reserving resources in Couch

[Wout Mertens <wm...@cisco.com>]

On Feb 13, 2009, at 12:14 PM, Jan Lehnardt wrote:

> On 13 Feb 2009, at 11:59, Wout Mertens wrote:
>
>> Can I count on this always being true for a single-node CouchDB?
> Yup.
>> What about a replicating CouchDB cloud where competing instances (A  
>> and B) connect to the same CouchDB?
> Not sure how this scenario is different from the single-node CouchDB
> instance.

Me neither, is why I asked ;-) Ok so it's the same.

>> And, just out of interest, what would be a good way to do this if  
>> you have competing instances connecting to different CouchDBs in a  
>> replicating cloud? I think you'd have to make replication a part of  
>> the reservation process, right?
>
> Say you have two nodes and you reserve resource X on node 1
> with client A and resource X on node 2 with client B. X on 1 and 2
> will have different revision ids. On replication, this creates a  
> conflict.
> Automatic conflict resolution will pick one of the revisions and  
> save it
> as the lastest revision. All nodes participating in replication will  
> pick
> the same winning revision. The losing revision is stored as a previous
> revision. So you are guaranteed that only one process effectively can
> hold the lock after replication.

Ok so if that happens you need to make sure your app knows that it can  
have its lock revoked by replication. Looks like forcing per-document  
masters like Paul proposes seems the way to go (but then you have to  
be careful about node outages).

Ok, thanks for the info, I get it now!

Wout.

Re: [user] Re: reserving resources in Couch

[Jan Lehnardt <ja...@apache.org>]

On 13 Feb 2009, at 11:59, Wout Mertens wrote:

> Can I count on this always being true for a single-node CouchDB?

Yup.


> What about a replicating CouchDB cloud where competing instances (A  
> and B) connect to the same CouchDB?

Not sure how this scenario is different from the single-node CouchDB
instance.


> And, just out of interest, what would be a good way to do this if  
> you have competing instances connecting to different CouchDBs in a  
> replicating cloud? I think you'd have to make replication a part of  
> the reservation process, right?

Say you have two nodes and you reserve resource X on node 1
with client A and resource X on node 2 with client B. X on 1 and 2
will have different revision ids. On replication, this creates a  
conflict.
Automatic conflict resolution will pick one of the revisions and save it
as the lastest revision. All nodes participating in replication will  
pick
the same winning revision. The losing revision is stored as a previous
revision. So you are guaranteed that only one process effectively can
hold the lock after replication.


Cheers
Jan
--

>
>
> Wout.
>
> On Feb 12, 2009, at 8:56 PM, Troy Kruthoff wrote:
>
>> If I understand you correctly, what you need is already baked in  
>> with revision #'s.
>>
>> 1) Get a doc that is not assigned a resource
>> 2) Flag the doc as being in-use and then save it.
>> 2a)  If the save fails because of conflict, you can then verify the  
>> new rev is in use and forget about it
>> 2b)  If save is success, you know that process has secured the "in- 
>> use" lock
>>
>> -- troy
>>
>>
>>
>> On Feb 12, 2009, at 9:11 AM, Wout Mertens wrote:
>>
>>> Ok,
>>>
>>> (no actual code yet, I don't have time to code right now :( )
>>>
>>> I have a project currently using an RDBMS and I'd like to port it  
>>> to CouchDB. One of the things I do is lock a table, choose a free  
>>> resource from a query on a static table and the session list,  
>>> assign the resource to a new session and unlock the table.
>>>
>>> How would I be able to do the same thing with CouchDB given that 2  
>>> sessions could start at the same time? I do have the advantage  
>>> that simultaneous starters would contact the same CouchDB instance.
>>>
>>> I was thinking of using sums: make a view that calculates the sum  
>>> of resources. A resource record would count as +1 and an in-use  
>>> record would be -1.
>>>
>>> Then when you reserve a resource, you save the in-use record.  
>>> After saving, look up the sum for the resource you reserved. If  
>>> it's not equal to 0, then use a stable algorithm to determine who  
>>> has to release the resource again.
>>>
>>> Would this close the race condition? Note that no documents are  
>>> overwritten at reservation time, each reservation doubles as the  
>>> event log. When the session clears up, the document that  
>>> represents it is updated to release the resource.
>>>
>>> Does this work? Is there a better way to do it?
>>>
>>> Thanks,
>>>
>>> Wout.
>
>

Re: [user] Re: reserving resources in Couch

[Paul Davis <pa...@gmail.com>]

On Fri, Feb 13, 2009 at 5:59 AM, Wout Mertens <wm...@cisco.com> wrote:
> Oh wow, I completely missed that functionality :-)
>
> Actually, I think you mean that I should rewrite the resource documents,
> since they are being locked. Let's look at the sequence:
>
> CouchDB: C
> App instances: A and B
> Resource: R_0 (rev 0 unused), R_1_A (rev 1, resource reserved by A), R_1_B
> (rev 1, resource reserved by B)
>
> Time 0:
> A reads R_0 from C
> B reads R_0 from C
>
> Time 1:
> A writes R_1_A to C
> B writes R_1_B to C
>
> Time 2:
> A gets failure from C => A knows it didn't reserve R
> B gets success from C => B has the resource reserved
>
> Is that correct? Man that's easy :)
>
> Can I count on this always being true for a single-node CouchDB?
> What about a replicating CouchDB cloud where competing instances (A and B)
> connect to the same CouchDB?
>
> And, just out of interest, what would be a good way to do this if you have
> competing instances connecting to different CouchDBs in a replicating cloud?
> I think you'd have to make replication a part of the reservation process,
> right?
>
> Wout.
>

I think the traditional method would be to nominate a write node for
each document a la consistent hashing or some other scheme. Then no
matter where you read a document from, you're guaranteed to have
conflict resolution at write time,

Also, remember, that people don't expect perfection like software
engineers. If once a year, I go to book a conference room, and a few
minutes later I get an email that says, "ooops, scheduling conflict,
can you reschedule?" I wouldn't care so much. As long as it can be
fixed quickly and easily, no one much minds assuming its a low
frequency event.

Granted, that whole argument is invalid for things like the email thread. :)

HTH,
Paul Davis

> On Feb 12, 2009, at 8:56 PM, Troy Kruthoff wrote:
>
>> If I understand you correctly, what you need is already baked in with
>> revision #'s.
>>
>> 1) Get a doc that is not assigned a resource
>> 2) Flag the doc as being in-use and then save it.
>> 2a)  If the save fails because of conflict, you can then verify the new
>> rev is in use and forget about it
>> 2b)  If save is success, you know that process has secured the "in-use"
>> lock
>>
>> -- troy
>>
>>
>>
>> On Feb 12, 2009, at 9:11 AM, Wout Mertens wrote:
>>
>>> Ok,
>>>
>>> (no actual code yet, I don't have time to code right now :( )
>>>
>>> I have a project currently using an RDBMS and I'd like to port it to
>>> CouchDB. One of the things I do is lock a table, choose a free resource from
>>> a query on a static table and the session list, assign the resource to a new
>>> session and unlock the table.
>>>
>>> How would I be able to do the same thing with CouchDB given that 2
>>> sessions could start at the same time? I do have the advantage that
>>> simultaneous starters would contact the same CouchDB instance.
>>>
>>> I was thinking of using sums: make a view that calculates the sum of
>>> resources. A resource record would count as +1 and an in-use record would be
>>> -1.
>>>
>>> Then when you reserve a resource, you save the in-use record. After
>>> saving, look up the sum for the resource you reserved. If it's not equal to
>>> 0, then use a stable algorithm to determine who has to release the resource
>>> again.
>>>
>>> Would this close the race condition? Note that no documents are
>>> overwritten at reservation time, each reservation doubles as the event log.
>>> When the session clears up, the document that represents it is updated to
>>> release the resource.
>>>
>>> Does this work? Is there a better way to do it?
>>>
>>> Thanks,
>>>
>>> Wout.
>
>

Re: [user] Re: reserving resources in Couch

[Wout Mertens <wm...@cisco.com>]

Oh wow, I completely missed that functionality :-)

Actually, I think you mean that I should rewrite the resource  
documents, since they are being locked. Let's look at the sequence:

CouchDB: C
App instances: A and B
Resource: R_0 (rev 0 unused), R_1_A (rev 1, resource reserved by A),  
R_1_B (rev 1, resource reserved by B)

Time 0:
A reads R_0 from C
B reads R_0 from C

Time 1:
A writes R_1_A to C
B writes R_1_B to C

Time 2:
A gets failure from C => A knows it didn't reserve R
B gets success from C => B has the resource reserved

Is that correct? Man that's easy :)

Can I count on this always being true for a single-node CouchDB?
What about a replicating CouchDB cloud where competing instances (A  
and B) connect to the same CouchDB?

And, just out of interest, what would be a good way to do this if you  
have competing instances connecting to different CouchDBs in a  
replicating cloud? I think you'd have to make replication a part of  
the reservation process, right?

Wout.

On Feb 12, 2009, at 8:56 PM, Troy Kruthoff wrote:

> If I understand you correctly, what you need is already baked in  
> with revision #'s.
>
> 1) Get a doc that is not assigned a resource
> 2) Flag the doc as being in-use and then save it.
> 2a)  If the save fails because of conflict, you can then verify the  
> new rev is in use and forget about it
> 2b)  If save is success, you know that process has secured the "in- 
> use" lock
>
> -- troy
>
>
>
> On Feb 12, 2009, at 9:11 AM, Wout Mertens wrote:
>
>> Ok,
>>
>> (no actual code yet, I don't have time to code right now :( )
>>
>> I have a project currently using an RDBMS and I'd like to port it  
>> to CouchDB. One of the things I do is lock a table, choose a free  
>> resource from a query on a static table and the session list,  
>> assign the resource to a new session and unlock the table.
>>
>> How would I be able to do the same thing with CouchDB given that 2  
>> sessions could start at the same time? I do have the advantage that  
>> simultaneous starters would contact the same CouchDB instance.
>>
>> I was thinking of using sums: make a view that calculates the sum  
>> of resources. A resource record would count as +1 and an in-use  
>> record would be -1.
>>
>> Then when you reserve a resource, you save the in-use record. After  
>> saving, look up the sum for the resource you reserved. If it's not  
>> equal to 0, then use a stable algorithm to determine who has to  
>> release the resource again.
>>
>> Would this close the race condition? Note that no documents are  
>> overwritten at reservation time, each reservation doubles as the  
>> event log. When the session clears up, the document that represents  
>> it is updated to release the resource.
>>
>> Does this work? Is there a better way to do it?
>>
>> Thanks,
>>
>> Wout.

Re: reserving resources in Couch

[Troy Kruthoff <tk...@blit.com>]

If I understand you correctly, what you need is already baked in with  
revision #'s.

1) Get a doc that is not assigned a resource
2) Flag the doc as being in-use and then save it.
2a)  If the save fails because of conflict, you can then verify the  
new rev is in use and forget about it
2b)  If save is success, you know that process has secured the "in- 
use" lock

-- troy



On Feb 12, 2009, at 9:11 AM, Wout Mertens wrote:

> Ok,
>
> (no actual code yet, I don't have time to code right now :( )
>
> I have a project currently using an RDBMS and I'd like to port it to  
> CouchDB. One of the things I do is lock a table, choose a free  
> resource from a query on a static table and the session list, assign  
> the resource to a new session and unlock the table.
>
> How would I be able to do the same thing with CouchDB given that 2  
> sessions could start at the same time? I do have the advantage that  
> simultaneous starters would contact the same CouchDB instance.
>
> I was thinking of using sums: make a view that calculates the sum of  
> resources. A resource record would count as +1 and an in-use record  
> would be -1.
>
> Then when you reserve a resource, you save the in-use record. After  
> saving, look up the sum for the resource you reserved. If it's not  
> equal to 0, then use a stable algorithm to determine who has to  
> release the resource again.
>
> Would this close the race condition? Note that no documents are  
> overwritten at reservation time, each reservation doubles as the  
> event log. When the session clears up, the document that represents  
> it is updated to release the resource.
>
> Does this work? Is there a better way to do it?
>
> Thanks,
>
> Wout.