You are viewing a plain text version of this content. The canonical link for it is here.
Posted to slide-dev@jakarta.apache.org by Andreas Probst <an...@gmx.net> on 2004/08/02 23:57:48 UTC
Re: Mea culpa / No more deadlocks
Hi Oliver,
I implemented a finer lock mechanism. For our project we need
only the plain WebDAV methods, so we do NOT use DeltaV
versioning. This means all file requests go into /files. Knowing
this I lock the URI parts either shared or exclusively depending
on the kind of request.
Example:
* GET on /files/col/file:
shared on /
shared on /files
shared on /files/col
shared on /files/col/file
* PUT on /files/col/file
shared on /
shared on /files
exclusive on /files/col (because the properties are written)
exclusive on /files/col/file
* DELETE on /files/col
shared on /
exclusive on /files (because the properties are written)
exclusive on /files/col
Database read committed. Connection pool as described in my
other email today.
Since then I did not have deadlocks any more, but several
clients can work concurrently if they don't work inside the same
collection. Without updating the lastmodified date of the
collection it would not even be necessary to exclusively lock
the collection (actually this is what we do, as lastmodified is
defined by BINDING).
This kind of lock granularity works for the plain WebDAV
methods. So far I've never thought how this can work with DeltaV
(everything in /history and /workspace and so on). I also did
not think about the security stuff (/users), as we don't need
this for our project (only one system user, no person).
Maybe something similar to this could work out inspite of all
the different operations which must be supported in general
Slide. Perhaps I could even contribute some code if this would
help, but before that I need to check this with my employer...
Regards,
Andreas
On 27 Jul 2004 at 13:39, Oliver Zeigermann wrote:
> I have just done something really bad and am pretty sure it will be hard
> to get a new job if I never need one ;)
>
> I have introduced a global read/write lock that can be configured to
> allow at most a single write request per time. It can also be configured
> to either allow read requests or disallow them while another request
> writes. However, this switch will have no effect when requests are done
> inside external transactions.
>
> No more deadlocks or failed uploads could be observed with that switch.
> Also no starvations of requests... I will make it the default in Domain.xml.
>
> While it may look like I am a complete idiot and all my work on TLocks
> and Sequences were in vain, they will become more useful in the future
> with a real concurrent Slide kernel. For now, they will at least be
> useful in externally controlled transactions.
>
> Oliver
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: Mea culpa / No more deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Andreas Probst wrote:
> On 4 Aug 2004 at 9:26, Oliver Zeigermann wrote:
>
>
>>Andreas Probst wrote:
>>
>>>On 3 Aug 2004 at 10:49, Oliver Zeigermann wrote:
>>>
>>>>What you describe is the locking the RDBMS / file store would do anyway.
>>>>However, if you acquire all those locks atomically at the very beginning
>>>>of the request deadlocks are impossible, I guess. Do you do that? May be
>>>>a good idea...
>>>
>>>
>>>Yes, I simply don't start a transaction if I can't get all the
>>>locks. So no dead lock anymore.
>>
>>What kind of lock do you use?
>
>
> I just use URI String objects. For instance if a thread has an
> exclusive lock on the String "/files", no other thread can get
> any lock on "/files". I developed some classes organizing these
> locking stuff. This way also no thread can get any lock on
> "/files/col". As long as the thread can't get the lock, it can't
> get into the store -> so no database action. It waits until the
> lock is released.
>
> The actual challenge is to know, which lock (shared/exclusive)
> to acquire for which part of the URI.
>
> You described the problem with the history folder. If a new VHR
> is created, then /history needs to be exclusively locked. If the
> lastmodified would not be set on /history - and I think it's
> meaningless there - then /history only needs a shared lock,
> which means other threads/clients can create VHR's too. Is it OK
> with Binding and DeltaV not to set the lastmodified?
As Slide explicitely represents child-parent relationship the history
folder needs to be updated as soon as it gets a new child.
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: Mea culpa / No more deadlocks
Posted by Andreas Probst <an...@gmx.net>.
On 4 Aug 2004 at 9:26, Oliver Zeigermann wrote:
> Andreas Probst wrote:
> > On 3 Aug 2004 at 10:49, Oliver Zeigermann wrote:
> >>What you describe is the locking the RDBMS / file store would do anyway.
> >>However, if you acquire all those locks atomically at the very beginning
> >>of the request deadlocks are impossible, I guess. Do you do that? May be
> >>a good idea...
> >
> >
> > Yes, I simply don't start a transaction if I can't get all the
> > locks. So no dead lock anymore.
>
> What kind of lock do you use?
I just use URI String objects. For instance if a thread has an
exclusive lock on the String "/files", no other thread can get
any lock on "/files". I developed some classes organizing these
locking stuff. This way also no thread can get any lock on
"/files/col". As long as the thread can't get the lock, it can't
get into the store -> so no database action. It waits until the
lock is released.
The actual challenge is to know, which lock (shared/exclusive)
to acquire for which part of the URI.
You described the problem with the history folder. If a new VHR
is created, then /history needs to be exclusively locked. If the
lastmodified would not be set on /history - and I think it's
meaningless there - then /history only needs a shared lock,
which means other threads/clients can create VHR's too. Is it OK
with Binding and DeltaV not to set the lastmodified?
Regards,
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: Mea culpa / No more deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Andreas Probst wrote:
> On 3 Aug 2004 at 10:49, Oliver Zeigermann wrote:
>>What you describe is the locking the RDBMS / file store would do anyway.
>>However, if you acquire all those locks atomically at the very beginning
>>of the request deadlocks are impossible, I guess. Do you do that? May be
>>a good idea...
>
>
> Yes, I simply don't start a transaction if I can't get all the
> locks. So no dead lock anymore.
What kind of lock do you use?
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: Mea culpa / No more deadlocks
Posted by Andreas Probst <an...@gmx.net>.
On 3 Aug 2004 at 10:49, Oliver Zeigermann wrote:
> Hi Andreas!
>
> The main problem is the history folder , especially when a new resource
> is uploaded and a VHR is created. If this is the case the history folder
> has to be locked which in effect results into a complicated global lock.
> This is what I simplified and made explicite with the global read/write
> lock.
Hmm, complicated.
>
> What you describe is the locking the RDBMS / file store would do anyway.
> However, if you acquire all those locks atomically at the very beginning
> of the request deadlocks are impossible, I guess. Do you do that? May be
> a good idea...
Yes, I simply don't start a transaction if I can't get all the
locks. So no dead lock anymore.
>
> Oliver
>
> Andreas Probst wrote:
> > Hi Oliver,
> >
> > I implemented a finer lock mechanism. For our project we need
> > only the plain WebDAV methods, so we do NOT use DeltaV
> > versioning. This means all file requests go into /files. Knowing
> > this I lock the URI parts either shared or exclusively depending
> > on the kind of request.
> >
> > Example:
> > * GET on /files/col/file:
> > shared on /
> > shared on /files
> > shared on /files/col
> > shared on /files/col/file
> >
> > * PUT on /files/col/file
> > shared on /
> > shared on /files
> > exclusive on /files/col (because the properties are written)
> > exclusive on /files/col/file
> >
> > * DELETE on /files/col
> > shared on /
> > exclusive on /files (because the properties are written)
> > exclusive on /files/col
> >
> > Database read committed. Connection pool as described in my
> > other email today.
> >
> > Since then I did not have deadlocks any more, but several
> > clients can work concurrently if they don't work inside the same
> > collection. Without updating the lastmodified date of the
> > collection it would not even be necessary to exclusively lock
> > the collection (actually this is what we do, as lastmodified is
> > defined by BINDING).
> >
> > This kind of lock granularity works for the plain WebDAV
> > methods. So far I've never thought how this can work with DeltaV
> > (everything in /history and /workspace and so on). I also did
> > not think about the security stuff (/users), as we don't need
> > this for our project (only one system user, no person).
> >
> > Maybe something similar to this could work out inspite of all
> > the different operations which must be supported in general
> > Slide. Perhaps I could even contribute some code if this would
> > help, but before that I need to check this with my employer...
> >
> > Regards,
> >
> > Andreas
> >
> > On 27 Jul 2004 at 13:39, Oliver Zeigermann wrote:
> >
> >
> >>I have just done something really bad and am pretty sure it will be hard
> >>to get a new job if I never need one ;)
> >>
> >>I have introduced a global read/write lock that can be configured to
> >>allow at most a single write request per time. It can also be configured
> >>to either allow read requests or disallow them while another request
> >>writes. However, this switch will have no effect when requests are done
> >>inside external transactions.
> >>
> >>No more deadlocks or failed uploads could be observed with that switch.
> >>Also no starvations of requests... I will make it the default in Domain.xml.
> >>
> >>While it may look like I am a complete idiot and all my work on TLocks
> >>and Sequences were in vain, they will become more useful in the future
> >>with a real concurrent Slide kernel. For now, they will at least be
> >>useful in externally controlled transactions.
> >>
> >>Oliver
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: Mea culpa / No more deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Hi Andreas!
The main problem is the history folder , especially when a new resource
is uploaded and a VHR is created. If this is the case the history folder
has to be locked which in effect results into a complicated global lock.
This is what I simplified and made explicite with the global read/write
lock.
What you describe is the locking the RDBMS / file store would do anyway.
However, if you acquire all those locks atomically at the very beginning
of the request deadlocks are impossible, I guess. Do you do that? May be
a good idea...
Oliver
Andreas Probst wrote:
> Hi Oliver,
>
> I implemented a finer lock mechanism. For our project we need
> only the plain WebDAV methods, so we do NOT use DeltaV
> versioning. This means all file requests go into /files. Knowing
> this I lock the URI parts either shared or exclusively depending
> on the kind of request.
>
> Example:
> * GET on /files/col/file:
> shared on /
> shared on /files
> shared on /files/col
> shared on /files/col/file
>
> * PUT on /files/col/file
> shared on /
> shared on /files
> exclusive on /files/col (because the properties are written)
> exclusive on /files/col/file
>
> * DELETE on /files/col
> shared on /
> exclusive on /files (because the properties are written)
> exclusive on /files/col
>
> Database read committed. Connection pool as described in my
> other email today.
>
> Since then I did not have deadlocks any more, but several
> clients can work concurrently if they don't work inside the same
> collection. Without updating the lastmodified date of the
> collection it would not even be necessary to exclusively lock
> the collection (actually this is what we do, as lastmodified is
> defined by BINDING).
>
> This kind of lock granularity works for the plain WebDAV
> methods. So far I've never thought how this can work with DeltaV
> (everything in /history and /workspace and so on). I also did
> not think about the security stuff (/users), as we don't need
> this for our project (only one system user, no person).
>
> Maybe something similar to this could work out inspite of all
> the different operations which must be supported in general
> Slide. Perhaps I could even contribute some code if this would
> help, but before that I need to check this with my employer...
>
> Regards,
>
> Andreas
>
> On 27 Jul 2004 at 13:39, Oliver Zeigermann wrote:
>
>
>>I have just done something really bad and am pretty sure it will be hard
>>to get a new job if I never need one ;)
>>
>>I have introduced a global read/write lock that can be configured to
>>allow at most a single write request per time. It can also be configured
>>to either allow read requests or disallow them while another request
>>writes. However, this switch will have no effect when requests are done
>>inside external transactions.
>>
>>No more deadlocks or failed uploads could be observed with that switch.
>>Also no starvations of requests... I will make it the default in Domain.xml.
>>
>>While it may look like I am a complete idiot and all my work on TLocks
>>and Sequences were in vain, they will become more useful in the future
>>with a real concurrent Slide kernel. For now, they will at least be
>>useful in externally controlled transactions.
>>
>>Oliver
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org