Re: [RFC] Conceptual locking procedure for database access [#11511]

[Keith Bostic <bo...@abyssinian.sleepycat.com>]

> I've been following the thread titled 'Subversion's use of Berkeley DB
> [#11511]', and the problem seems to be:  We have no easy way to
> determine if we need to perform recovery, and the only solution we have
> so far is to run recovery every time.

I agree, running recovery every time isn't the right solution.

Your design is pretty much where I ended up as well.  A couple
of differences: I used fcntl instead of flock, and built the
file more like a registry, with process IDs of the processes
running in the database environment.  I also used an individual
lock for each process, to detect if a process has died in the
database environment.

> I have mapped out a potential locking procedure that would make it
> possible to detect if any process exited uncleanly, and determine if
> recovery needs to be run on the database.  It should be implementable
> using POSIX flock() (meaning that it should work on Win32).  It should
> also be implementable using fcntl locking, which is NFS-friendly, but
> Win32-hostile.

POSIX 1003.1 requires fcntl, doesn't Win32 support it?

I think I see problems with your design, but I can't say for
sure.

There were places I didn't follow what you were trying to say,
so I may just be wrong -- specifically, there were two sections
labelled "startup", and they were different.  It wasn't clear
to me how they were related?

More specifically, I was concerned about the transition from the
exclusive lock to the shared lock.  It seems to me that there
was a race in there where a process could end up in the database
environment without the reference set to '1'.

I'm happy to work through this in detail with you, but we might
as well take it offline.

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic			bostic@sleepycat.com
Sleepycat Software Inc.		keithbosticim (ymsgid)
118 Tower Rd.			+1-781-259-3139
Lincoln, MA 01773		http://www.sleepycat.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: [RFC] Conceptual locking procedure for database access [#11511]

[Branko Čibej <br...@xbc.nu>]

Keith Bostic wrote:

>>I've been following the thread titled 'Subversion's use of Berkeley DB
>>[#11511]', and the problem seems to be:  We have no easy way to
>>determine if we need to perform recovery, and the only solution we have
>>so far is to run recovery every time.
>>    
>>
>
>I agree, running recovery every time isn't the right solution.
>
>Your design is pretty much where I ended up as well.
>
The proposal is a classical case of using a mutex to protect a reference 
count, so of course it will work.

>  A couple
>of differences: I used fcntl instead of flock, and built the
>file more like a registry, with process IDs of the processes
>running in the database environment.  I also used an individual
>lock for each process, to detect if a process has died in the
>database environment.
>  
>
Could you elaborate a bit? From this description, I can't see how this 
will work; I see a number of potential new races, so I'm hoping I 
misunderstood. :-)

>>I have mapped out a potential locking procedure that would make it
>>possible to detect if any process exited uncleanly, and determine if
>>recovery needs to be run on the database.  It should be implementable
>>using POSIX flock() (meaning that it should work on Win32).  It should
>>also be implementable using fcntl locking, which is NFS-friendly, but
>>Win32-hostile.
>>    
>>
>
>POSIX 1003.1 requires fcntl, doesn't Win32 support it?
>  
>
It does, but Windows also gives you a number of other useful tools. I 
don't think this locking scheme can be implemented completely portably.

>I think I see problems with your design, but I can't say for
>sure.
>
>There were places I didn't follow what you were trying to say,
>so I may just be wrong -- specifically, there were two sections
>labelled "startup", and they were different.  It wasn't clear
>to me how they were related?
>
>More specifically, I was concerned about the transition from the
>exclusive lock to the shared lock.  It seems to me that there
>was a race in there where a process could end up in the database
>environment without the reference set to '1'.
>  
>
This worries me, too. I think you'd need a separate file for the shared 
lock, and be very careful about the locking order.

>I'm happy to work through this in detail with you, but we might
>as well take it offline.
>  
>
I wish you hadn't... or at least, bring me into the loop, please?

-- Brane


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org