You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Dick Davies <ra...@hellooperator.net> on 2009/10/15 16:27:29 UTC
[users@httpd] Re: Dav Locks corrupting; need some troubleshooting tools
[sorry for the crosspost, but not sure where this should go].
To answer my own question:
got to the bottom of it; looks to me like the
lock DB is a hash of
inode <-> locktoken
Steps to reproduce:
* PUT file
* LOCK file
* PROPGET file (note down the locktoken)
use something other than DAV to delete 'file' (e.g. a cronjob)
* PUT newfile (very good chance the inode is reused)
* LOCK newfile (it will fail)
* PROPGET newfile (you'll see it has the same locktoken)
Should I report this as a bug?
On Wed, Oct 14, 2009 at 2:47 PM, Dick Davies
<ra...@hellooperator.net> wrote:
> Hi all
>
> I'm on apache 2.0.63, and seeing nonsensical Dav Locks
> (symptoms are that files show as being locked by users that couldn't
> possibly have locked them).
>
> I *think* I've tracked it down to a bug in mod_dav_fs; it appears that
> if files are deleted
> from disk (by a mechanism other than Dav), any locks that were taken
> out on that file
> 'jump' so that they apply to a different file altogether.
>
> Are there any tools that let me inspect the content of DavLockDB? And
> who's a good person
> to talk to if I need to pick over the code?
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: dropping inode keyed locks in mod_dav_fs
Posted by Greg Stein <gs...@gmail.com>.
Sorry for missing earlier messages; I don't follow httpd as closely as before.
See my replies below:
On Mon, Nov 9, 2009 at 06:28, Stefan Fritsch <sf...@sfritsch.de> wrote:
> On Friday 23 October 2009, Stefan Fritsch wrote:
>> On Thursday 22 October 2009, Joe Orton wrote:
>> > > Is the performance improvement of inode keyed locking so large
>> > > that it is worth the hassle? If mod_dav_fs used filename keyed
>> > > locking entirely, there would be an easy way to make file
>> > > replacement by PUT atomic (see PR 39815). The current behaviour
>> > > of deleting the old and the new file when the PUT fails is
>> > > really bad.
>> >
>> > I believe the intent of using inode/device-number keyed locks was
>> > to ensure that the lock database is independent of the mount
>> > point - i.e. you could move it around in the filesystem and it'd
>> > still work.
No, I don't think so. The database would be movable with or without
inodes, afaik.
That is dim history :-P, but I am pretty it is because I was looking
forward to when the BIND protocol was implemented and you could bind
multiple URLs to the same resource. Thus, if you wanted to lock the
*resource*, then the inode was your primary key. Locking one path and
allowing changes through another would be Badness.
>... [snip: bad PUT behavior] ...
>
> Since nobody disagreed, I am going ahead with this and remove inode
> keyed locks and make PUT use temp files.
>
> Using temp files and inode keyed locks would require to either copy
> (instead of move) the temp file to the target file, or to extend the
> dav_hooks_locks interface. I think both solutions are worse than just
> switching to file name keyed locks.
People aren't running around with symlinks in their repository, I
believe. mod_dav is pretty unaware of them, and Bad Things are
advertised to happen. It already takes some steps to explicitly
disallow symlinks as "files". Thus, the notion of one resource in two
locations is probably moot.
Therefore, improving the PUT situation rather than keeping inodes is a
very good tradeoff.
Thanks,
-g
Re: dropping inode keyed locks in mod_dav_fs
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Friday 23 October 2009, Stefan Fritsch wrote:
> On Thursday 22 October 2009, Joe Orton wrote:
> > > Is the performance improvement of inode keyed locking so large
> > > that it is worth the hassle? If mod_dav_fs used filename keyed
> > > locking entirely, there would be an easy way to make file
> > > replacement by PUT atomic (see PR 39815). The current behaviour
> > > of deleting the old and the new file when the PUT fails is
> > > really bad.
> >
> > I believe the intent of using inode/device-number keyed locks was
> > to ensure that the lock database is independent of the mount
> > point - i.e. you could move it around in the filesystem and it'd
> > still work.
>
> Interesting. Do you think this feature is actually used?
>
> > I certainly agree that the delete-on-PUT-failure behaviour is
> > bad; I think the correct behaviour would be to do the deletion
> > only if the resource is newly created by the PUT.
>
> That would still replace the old file with a broken new file. Even
> better would be to save the new file to a temp file and move that
> over the old file if it the transfer has completed successfully.
> But this breaks locking with inode keyed locks. Therefore I would
> like to move to filename keyed locks (which are already there for
> systems without inode numbers). Any opinions on this?
Since nobody disagreed, I am going ahead with this and remove inode
keyed locks and make PUT use temp files.
Using temp files and inode keyed locks would require to either copy
(instead of move) the temp file to the target file, or to extend the
dav_hooks_locks interface. I think both solutions are worse than just
switching to file name keyed locks.
Re: dropping inode keyed locks in mod_dav_fs (was: Dav Locks corrupting; need some troubleshooting tools)
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Thursday 22 October 2009, Joe Orton wrote:
> > Is the performance improvement of inode keyed locking so large
> > that it is worth the hassle? If mod_dav_fs used filename keyed
> > locking entirely, there would be an easy way to make file
> > replacement by PUT atomic (see PR 39815). The current behaviour
> > of deleting the old and the new file when the PUT fails is really
> > bad.
>
> I believe the intent of using inode/device-number keyed locks was
> to ensure that the lock database is independent of the mount
> point - i.e. you could move it around in the filesystem and it'd
> still work.
Interesting. Do you think this feature is actually used?
> I certainly agree that the delete-on-PUT-failure behaviour is bad;
> I think the correct behaviour would be to do the deletion only if
> the resource is newly created by the PUT.
That would still replace the old file with a broken new file. Even
better would be to save the new file to a temp file and move that over
the old file if it the transfer has completed successfully. But this
breaks locking with inode keyed locks. Therefore I would like to move
to filename keyed locks (which are already there for systems without
inode numbers). Any opinions on this?
Cheers,
Stefan
Re: dropping inode keyed locks in mod_dav_fs (was: Dav Locks
corrupting; need some troubleshooting tools)
Posted by Joe Orton <jo...@redhat.com>.
On Sun, Oct 18, 2009 at 09:50:25PM +0200, Stefan Fritsch wrote:
> On Thursday 15 October 2009, Dick Davies wrote:
> > In any event, does it made sense to use something other than the
> > inode as the key into the lockDB - the URI for example?
>
> Is the performance improvement of inode keyed locking so large that it
> is worth the hassle? If mod_dav_fs used filename keyed locking
> entirely, there would be an easy way to make file replacement by PUT
> atomic (see PR 39815). The current behaviour of deleting the old and
> the new file when the PUT fails is really bad.
I believe the intent of using inode/device-number keyed locks was to
ensure that the lock database is independent of the mount point - i.e.
you could move it around in the filesystem and it'd still work.
(It would also allow mod_dav to support BIND via hard links since locks
would be associated correctly with a resource rather than a URI path,
but that is not really a consideration here)
I certainly agree that the delete-on-PUT-failure behaviour is bad; I
think the correct behaviour would be to do the deletion only if the
resource is newly created by the PUT.
Regards, Joe
Re: dropping inode keyed locks in mod_dav_fs (was: Dav Locks
corrupting; need some troubleshooting tools)
Posted by Dick Davies <ra...@hellooperator.net>.
On Sun, Oct 18, 2009 at 8:50 PM, Stefan Fritsch <sf...@sfritsch.de> wrote:
> On Thursday 15 October 2009, Dick Davies wrote:
>> In any event, does it made sense to use something other than the
>> inode as the key into the lockDB - the URI for example?
>
> Is the performance improvement of inode keyed locking so large that it
> is worth the hassle? If mod_dav_fs used filename keyed locking
> entirely, there would be an easy way to make file replacement by PUT
> atomic (see PR 39815). The current behaviour of deleting the old and
> the new file when the PUT fails is really bad.
Agreed.
I can see how inodes are faster, but I'd like an alternative option -
maybe a similar config option to the ETags tunable ; if the DBM key was
mtime+inode number my problem would go away, for example.
[ I know it may sounds like 'Doctor it hurts when I do this'
wrt. other processes editing files underneath mod_dav, but it's a fairly
common thing to do in most installs I've seen. It's certainly the first
time I've heard that DAV requires exclusive access to a filesystem to avoid
strange behaviour. ]
dropping inode keyed locks in mod_dav_fs (was: Dav Locks corrupting; need some troubleshooting tools)
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Thursday 15 October 2009, Dick Davies wrote:
> In any event, does it made sense to use something other than the
> inode as the key into the lockDB - the URI for example?
Is the performance improvement of inode keyed locking so large that it
is worth the hassle? If mod_dav_fs used filename keyed locking
entirely, there would be an easy way to make file replacement by PUT
atomic (see PR 39815). The current behaviour of deleting the old and
the new file when the PUT fails is really bad.
[users@httpd] Re: Dav Locks corrupting; need some troubleshooting tools
Posted by Dick Davies <si...@cf.ac.uk>.
Is that documented anywhere at all?
In any event, does it made sense to use something other than the
inode as the key into the lockDB - the URI for example?
On Thu, Oct 15, 2009 at 3:33 PM, Joe Orton <jo...@redhat.com> wrote:
>> Steps to reproduce:
>>
>> * PUT file
>> * LOCK file
>> * PROPGET file (note down the locktoken)
>> use something other than DAV to delete 'file' (e.g. a cronjob)
>
> mod_dav doesn't support operations on the repository outside of its
> control - this has always been the case. You should think of the DAV
> repository as "owned" by mod_dav.
>
> Regards, Joe
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: Dav Locks corrupting; need some troubleshooting tools
Posted by Dick Davies <si...@cf.ac.uk>.
Is that documented anywhere at all?
In any event, does it made sense to use something other than the
inode as the key into the lockDB - the URI for example?
On Thu, Oct 15, 2009 at 3:33 PM, Joe Orton <jo...@redhat.com> wrote:
>> Steps to reproduce:
>>
>> * PUT file
>> * LOCK file
>> * PROPGET file (note down the locktoken)
>> use something other than DAV to delete 'file' (e.g. a cronjob)
>
> mod_dav doesn't support operations on the repository outside of its
> control - this has always been the case. You should think of the DAV
> repository as "owned" by mod_dav.
>
> Regards, Joe
>
Re: Dav Locks corrupting; need some troubleshooting tools
Posted by Joe Orton <jo...@redhat.com>.
On Thu, Oct 15, 2009 at 03:27:29PM +0100, Dick Davies wrote:
> [sorry for the crosspost, but not sure where this should go].
>
> To answer my own question:
>
> got to the bottom of it; looks to me like the
> lock DB is a hash of
>
> inode <-> locktoken
>
> Steps to reproduce:
>
> * PUT file
> * LOCK file
> * PROPGET file (note down the locktoken)
> use something other than DAV to delete 'file' (e.g. a cronjob)
mod_dav doesn't support operations on the repository outside of its
control - this has always been the case. You should think of the DAV
repository as "owned" by mod_dav.
Regards, Joe
[users@httpd] Re: Dav Locks corrupting; need some troubleshooting tools
Posted by Joe Orton <jo...@redhat.com>.
On Thu, Oct 15, 2009 at 03:27:29PM +0100, Dick Davies wrote:
> [sorry for the crosspost, but not sure where this should go].
>
> To answer my own question:
>
> got to the bottom of it; looks to me like the
> lock DB is a hash of
>
> inode <-> locktoken
>
> Steps to reproduce:
>
> * PUT file
> * LOCK file
> * PROPGET file (note down the locktoken)
> use something other than DAV to delete 'file' (e.g. a cronjob)
mod_dav doesn't support operations on the repository outside of its
control - this has always been the case. You should think of the DAV
repository as "owned" by mod_dav.
Regards, Joe
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org