Re: DERBY-800, lock scheduling issue?

[Bryan Pendleton <bp...@amberpoint.com>]

> When a thread tries to create a table, it will first get a shared lock 
> on the dictionary (DataDictionaryImpl.startReading).  This is released 
> before it tries to lock the dictionary exclusively.  The way 
> DataDictionaryImpl.startwriting works is that it first checks whether 
> someone is holding a lock on the dictionary. If so, it will sleep for a 
> while a then try again.  This goes on for a while until it gets 
> impatient and actually requests an exclusive lock and enters the lock 
> queue.  In the mean time, a lot more threads have acquired a shared lock 
> and the updating thread will have to wait for all of them to release it. 

This sounds like there is a lock scheduling fairness issue.

Your description makes it sounds like this:
  - One or more transactions get shared locks on a resource
  - A transaction requests an exclusive lock on a resource, and blocks
  - Additional transactions arrive, requesting shared locks, and their
    locks are granted.

It is possible to implement a lock scheduling policy such that those
later locks, even though they are compatible with all currently granted
locks, are not immediately granted, but instead are placed onto the
resource queue behind the pending exclusive request.

Perhaps we should consider whether there is an opportunity to enhance
the locking policies to be more fair to writers?

thanks,

bryan

Re: DERBY-800, lock scheduling issue?

[Mike Matrigali <mi...@sbcglobal.net>]

The derby locking system implements livelock avoidance as you describe.
So if one or more xacts have a shared lock and then an exclusive
lock queues waiting for the resource, subsequent share locks will
wait behing the exclusive lock.  The only exception to this is the
case of compatible "release-immediate" locks, sometimes during
read committed scans we will request a shared lock and tell the lock
manager to logically grant and release it in one step.  In these
cases it will not queue behind the exclusive lock since it won't
get in its way.

Bryan Pendleton wrote:
>> When a thread tries to create a table, it will first get a shared lock 
>> on the dictionary (DataDictionaryImpl.startReading).  This is released 
>> before it tries to lock the dictionary exclusively.  The way 
>> DataDictionaryImpl.startwriting works is that it first checks whether 
>> someone is holding a lock on the dictionary. If so, it will sleep for 
>> a while a then try again.  This goes on for a while until it gets 
>> impatient and actually requests an exclusive lock and enters the lock 
>> queue.  In the mean time, a lot more threads have acquired a shared 
>> lock and the updating thread will have to wait for all of them to 
>> release it. 
> 
> 
> This sounds like there is a lock scheduling fairness issue.
> 
> Your description makes it sounds like this:
>  - One or more transactions get shared locks on a resource
>  - A transaction requests an exclusive lock on a resource, and blocks
>  - Additional transactions arrive, requesting shared locks, and their
>    locks are granted.
> 
> It is possible to implement a lock scheduling policy such that those
> later locks, even though they are compatible with all currently granted
> locks, are not immediately granted, but instead are placed onto the
> resource queue behind the pending exclusive request.
> 
> Perhaps we should consider whether there is an opportunity to enhance
> the locking policies to be more fair to writers?
> 
> thanks,
> 
> bryan
> 
> 
> 
> 

Re: DERBY-800, lock scheduling issue?

[Øystein Grøvlen <Oy...@Sun.COM>]

Bryan Pendleton wrote:
>> When a thread tries to create a table, it will first get a shared lock 
>> on the dictionary (DataDictionaryImpl.startReading).  This is released 
>> before it tries to lock the dictionary exclusively.  The way 
>> DataDictionaryImpl.startwriting works is that it first checks whether 
>> someone is holding a lock on the dictionary. If so, it will sleep for 
>> a while a then try again.  This goes on for a while until it gets 
>> impatient and actually requests an exclusive lock and enters the lock 
>> queue.  In the mean time, a lot more threads have acquired a shared 
>> lock and the updating thread will have to wait for all of them to 
>> release it. 
> 
> 
> This sounds like there is a lock scheduling fairness issue.
> 
> Your description makes it sounds like this:
>  - One or more transactions get shared locks on a resource
>  - A transaction requests an exclusive lock on a resource, and blocks
>  - Additional transactions arrive, requesting shared locks, and their
>    locks are granted.
> 
> It is possible to implement a lock scheduling policy such that those
> later locks, even though they are compatible with all currently granted
> locks, are not immediately granted, but instead are placed onto the
> resource queue behind the pending exclusive request.

The locking system behave as you describe.  However, in the particular 
case with the dictionary lock, the writer does not request the lock if 
it is already granted to some other transaction.  Instead, it sleeps, 
checks again, sleeps and so on for a while before it actually request 
the lock.  It seems to me that someone must have had a good reason in 
order to add this complexity to the code.

> 
> Perhaps we should consider whether there is an opportunity to enhance
> the locking policies to be more fair to writers?

We could look into this, but I am not sure fair scheduling of DDL 
operations is what we should spend our time on.

--
Øystein