You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Shahaf <d....@daniel.shahaf.name> on 2008/12/27 10:27:01 UTC
Re: Error when creating tags from working copy
[ CCing dev@ ]
Karl, is this bug still open? And how many bugs are there here (bug in
mod_dav_svn? bug in the fs layer?)?
I'd like to file an issue for it, at least, but I'm not sure I'll describe
it correctly. Could you do that?
Thanks,
Daniel
Karl Fogel wrote on Sat, 11 Oct 2008 at 18:25 -0400:
> This is a deep one. I'm posting what I've learned so far in case anyone
> has any hunches. On the client, the problem is that in
>
> libsvn_ra_neon/commit.c:commit_add_file()
>
> the following code should get a "file not found" error (i.e., an HTTP
> 404) when it probes for the existence of the file we're about to add,
> but doesn't -- instead, it's as though the file already exists:
>
> svn_ra_neon__resource_t *res;
> svn_error_t *err = svn_ra_neon__get_starting_props(&res,
> file->cc->ras,
> file->rsrc->wr_url,
> NULL, workpool);
> if (!err)
> {
> /* If the PROPFIND succeeds the file already exists */
> return svn_error_createf(SVN_ERR_RA_DAV_ALREADY_EXISTS, NULL,
> _("File '%s' already exists"),
> file->rsrc->url);
> }
>
> Here's what happens on the server side. Let's shortcut directly to
> mod_dav_svn/repos.c:get_resource(). Before the call to
> prep_resource()...
>
> /* prepare the resource for operation */
> if ((err = prep_resource(comb)) != NULL)
> return err;
>
> ...comb->exists is 0, as we would want (so is comb->res.exists, for that
> matter). But after prep_resource(), comb->exists is 1!
>
> prep_resource() is just a dispatch function; the real stuff happens in
> prep_working() (because comb->res.type == DAV_RESOURCE_TYPE_WORKING).
> In prep_working(), we get all the way to this code near the end:
>
> derr = fs_check_path(&kind, comb->priv.root.root,
> comb->priv.repos_path, pool);
>
> if (derr != NULL)
> return derr;
>
> comb->res.exists = (kind == svn_node_none) ? FALSE : TRUE;
> comb->res.collection = (kind == svn_node_dir) ? TRUE : FALSE;
>
> So that's where comb->res.exists gets set to 1. (I'm not sure where
> comb->exists gets set to 1, which might be important, but see below.)
>
> Anyway, that call to fs_check_path() sets kind to svn_node_file, even
> though the file shouldn't be present in this txn yet (that's why we're
> adding it).
>
> Tracing fs_check_path --> svn_fs_fs__check_path() --> node_kind() -->
> fs_node_id() --> get_dag() --> open_path() --> svn_fs_fs__dag_open(),
> shows that, somehow, the file is actually there, according to the FS.
> (It does not appear to be a dag node caching problem; the node isn't
> found in the cache, instead we find it the old-fashioned way, right
> there in the filesystem.)
>
> Back up in http-2.2.9/modules/dav/main/mod_dav.c:dav_method_propfind(),
> this all matters because of this code:
>
> /* Ask repository module to resolve the resource */
> err = dav_get_resource(r, 1 /* label_allowed */, 0 /* use_checked_in */,
> &resource);
> if (err != NULL)
> return dav_handle_err(r, err, NULL);
>
> if (dav_get_resource_state(r, resource) == DAV_RESOURCE_NULL) {
> /* Apache will supply a default error for this. */
> return HTTP_NOT_FOUND;
> }
>
> Not only does dav_get_resource() not get an error, but the call to
> dav_get_resource_state() will not return DAV_RESOURCE_NULL, because that
> code (in httpd-2.2.9/modules/dav/main/util_lock.c) says:
>
> if (resource->exists)
> return DAV_RESOURCE_EXISTS;
>
> Boom. If it weren't for that, the function would return
> DAV_RESOURCE_NULL and we'd be fine.
>
> Note that in the reproduction recipe, if you insert an "svn up Data/wc"
> right before the final svn cp in the reproduction recipe, then none of
> this weirdness happens, and the bug doesn't reproduce.
>
> So why does the file claim to be present in the txn already? And why
> does updating the whole working copy to r2 (instead of having a mixed
> r1/r2 working copy) cause the repository filesystem to correctly answer
> that the file is not there?
>
> I don't know. I will continue to debug, but if anyone has any inspired
> ideas, please post.
>
> -Karl
>
> Karl Fogel <kf...@red-bean.com> writes:
> > "Michael Susser" <hd...@googlemail.com> writes:
> >> i have found a bug when creating tags from working copy in svn 1.5.2
> >> (and also 1.5.3). Here is a recipe how to reproduce the error: (my
> >> apache is configured to host the repositories in E:\SVNRep)
> >>
> >> [...]
> >>
> >> svn copy -m"tagging" E:\Data\wc\trunk http://myserver/svn/repo/tags/V1.0.0
> >>>svn: Commit failed (details follow):
> >>>svn: File '/svn/repo/tags/V1.0.0/dir/file.cpp' already exists
> >
> > Thank you for the excellent recipe. I can reproduce this, and have
> > written a Unix Bourne shell reproduction script (see below). Note that
> > this only reproduces over http://. If you try it with svn:// or
> > file://, the tagging works fine.
> >
> >> This error message is absolutely wrong! I investigated what went wrong and
> >> found the following: after the commit action the working copy is not updated
> >> correctly(?). The changed file "file.cpp" gets a new revision (2), but its
> >> parent directory "dir" stays at revision 1. Is this ok?
> >
> > Well, that part is okay. When you commit, only the committed items get
> > their revisions bumped locally -- in other words, a commit does not do
> > an update on the items that are not part of the commit. However,
> > Subversion is supposed to transparently handle the resultant mixed
> > revisions. They shouldn't matter here.
> >
> > This bug seems familiar, like we've either fixed it before or filed an
> > issue for it. I'll check the logs and the issue tracker. But we have a
> > reproduction recipe, so that means we can fix it or re-fix it :-).
> >
> > ----------------------------------------------------------------------------
> > #!/bin/sh
> >
> > # The next line is the only line you should need to adjust.
> > SVNDIR=/home/kfogel/src/subversion
> >
> > SVN=${SVNDIR}/subversion/svn/svn
> > SVNSERVE=${SVNDIR}/subversion/svnserve/svnserve
> > SVNADMIN=${SVNDIR}/subversion/svnadmin/svnadmin
> >
> > # Select an access method. If svn://, the svnserve setup is
> > # handled automagically by this script; but if http://, then
> > # you'll have to configure it yourself first.
> > #
> > URL=http://localhost/ms/repos
> > # URL=svn://localhost/repos
> > # URL=file:///`pwd`/repos
> >
> > rm -rf repos wc Data import-me
> >
> > ${SVNADMIN} create repos
> >
> > # These are for svnserve only.
> > echo "[general]" > repos/conf/svnserve.conf
> > echo "anon-access = write" >> repos/conf/svnserve.conf
> > echo "auth-access = write" >> repos/conf/svnserve.conf
> >
> > # The server will only be contacted if $URL is svn://foo, of course.
> > ${SVNSERVE} --pid-file svnserve-pid -d -r `pwd`
> > # And put the kill command in a file, in case need to run it manually.
> > echo "kill -9 `cat svnserve-pid`" > k
> > chmod a+rwx k
> >
> > mkdir Data
> > svn co ${URL} Data/wc
> > mkdir Data/wc/tags
> > mkdir Data/wc/trunk
> > mkdir Data/wc/trunk/dir
> > cp /dev/null Data/wc/trunk/dir/file.cpp
> > ${SVN} add Data/wc/t*
> > ${SVN} ci -m"test" Data/wc
> >
> > echo "ABC" >> Data/wc/trunk/dir/file.cpp
> >
> > ${SVN} ci -m"test" Data/wc
> >
> > # Using http:// only, the next command fails with this error:
> > #
> > # subversion/libsvn_client/copy.c:1319: (apr_err=175005)
> > # svn: Commit failed (details follow):
> > # subversion/libsvn_ra_neon/commit.c:1036: (apr_err=175005)
> > # svn: File '/ms/repos/tags/V1.0.0/dir/file.cpp' already exists
> > #
> > ${SVN} copy -m"tagging" Data/wc/trunk ${URL}/tags/V1.0.0
> >
> > ./k
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=993658
Re: Error when creating tags from working copy
Posted by Dave Lawrence <dl...@ad-holdings.co.uk>.
I've got a few bits and bats to add to this one..
Bert Huijben wrote:
>
> I tried to reopen this issue on the developer list in the
> "Issue #3119 - File '...' already exists" (Regression since 1.5.0)" thread
> on the development list.
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=985
> 284
>
> We did some research but the thread died on the major problem that we can't
> reproduce it at will.
In contrast I find that Karl's reproduction script is a 100% reliable.
>
> To fully resolve this issue we really need a script that can reproduce the
> issue every (or at least some of the) time.
I attach my hacked version of Karl's script - this is just the current
as-is state of it the way I am using it in my set up for your ref. Also
attached is one I cooked up that *doesn't* reproduce the fault, but I
can't work out the crucial difference...
>
> I'm just guessing but maybe the following factors might help us:
> * Which server versions have the issue? (1.5.x?) (1.4.x?)
> * The issue was reported only to client versions after 1.5.0. Did somebody
> reproduce it with a trunk build?
>
Yep reproducible with trunk.
According to the previously mentioned thread:
rhuijben writes 'It looks like r31692 introduced the regression
described in issue #3119,'...
Said rev was merged to 1.5.x at 31799 and I can confirm that the bug is
there with 31799 and not with 31798
[snip]
>>
>> Karl Fogel wrote on Sat, 11 Oct 2008 at 18:25 -0400:
[snip]
>>> the following code should get a "file not found" error (i.e., an HTTP
>>> 404) when it probes for the existence of the file we're about to add,
>>> but doesn't -- instead, it's as though the file already exists:
>>>
>>> svn_ra_neon__resource_t *res;
>>> svn_error_t *err = svn_ra_neon__get_starting_props(&res,
>>> file->cc-
>>> ras,
>>> file->rsrc-
>>> wr_url,
>>> NULL,
>> workpool);
before the offending rev this used file->rsrc->url where it now uses
file->rsrc->wr_url
>>> if (!err)
>>> {
>>> /* If the PROPFIND succeeds the file already exists */
>>> return svn_error_createf(SVN_ERR_RA_DAV_ALREADY_EXISTS,
[snip]
>>>
>>> Note that in the reproduction recipe, if you insert an "svn up
>> Data/wc"
>>> right before the final svn cp in the reproduction recipe, then none
>> of
>>> this weirdness happens, and the bug doesn't reproduce.
>>>
>>> So why does the file claim to be present in the txn already? And why
>>> does updating the whole working copy to r2 (instead of having a mixed
>>> r1/r2 working copy) cause the repository filesystem to correctly
>> answer
>>> that the file is not there?
If the w/c is updated before the final copy then commit_add_file doesn't
get called. It may be that if the above mentioned code was called, it
would still malfunction.
At the risk of adding to the confusion I think I've seen this problem
from a non-mixed-rev w/c
Changing to using ra_serf is a good work around (for me and my
colleagues anyway) - is this worth noting in the issue tracker?
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=994684
RE: Error when creating tags from working copy
Posted by Bert Huijben <rh...@sharpsvn.net>.
> -----Original Message-----
> From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name]
> Sent: Saturday, December 27, 2008 11:27 AM
> To: Karl Fogel
> Cc: users@subversion.tigris.org; dev@subversion.tigris.org
> Subject: Re: Error when creating tags from working copy
>
> [ CCing dev@ ]
>
> Karl, is this bug still open? And how many bugs are there here (bug in
> mod_dav_svn? bug in the fs layer?)?
>
> I'd like to file an issue for it, at least, but I'm not sure I'll
> describe
> it correctly. Could you do that?
I tried to reopen this issue on the developer list in the
"Issue #3119 - File '...' already exists" (Regression since 1.5.0)" thread
on the development list.
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=985
284
We did some research but the thread died on the major problem that we can't
reproduce it at will.
To fully resolve this issue we really need a script that can reproduce the
issue every (or at least some of the) time.
I'm just guessing but maybe the following factors might help us:
* Which server versions have the issue? (1.5.x?) (1.4.x?)
* The issue was reported only to client versions after 1.5.0. Did somebody
reproduce it with a trunk build?
Original reporters, please fill in the missing details... we need your help
to resolve this issue.
Thanks,
Bert
>
> Thanks,
>
> Daniel
>
>
> Karl Fogel wrote on Sat, 11 Oct 2008 at 18:25 -0400:
> > This is a deep one. I'm posting what I've learned so far in case
> anyone
> > has any hunches. On the client, the problem is that in
> >
> > libsvn_ra_neon/commit.c:commit_add_file()
> >
> > the following code should get a "file not found" error (i.e., an HTTP
> > 404) when it probes for the existence of the file we're about to add,
> > but doesn't -- instead, it's as though the file already exists:
> >
> > svn_ra_neon__resource_t *res;
> > svn_error_t *err = svn_ra_neon__get_starting_props(&res,
> > file->cc-
> >ras,
> > file->rsrc-
> >wr_url,
> > NULL,
> workpool);
> > if (!err)
> > {
> > /* If the PROPFIND succeeds the file already exists */
> > return svn_error_createf(SVN_ERR_RA_DAV_ALREADY_EXISTS,
> NULL,
> > _("File '%s' already exists"),
> > file->rsrc->url);
> > }
> >
> > Here's what happens on the server side. Let's shortcut directly to
> > mod_dav_svn/repos.c:get_resource(). Before the call to
> > prep_resource()...
> >
> > /* prepare the resource for operation */
> > if ((err = prep_resource(comb)) != NULL)
> > return err;
> >
> > ...comb->exists is 0, as we would want (so is comb->res.exists, for
> that
> > matter). But after prep_resource(), comb->exists is 1!
> >
> > prep_resource() is just a dispatch function; the real stuff happens
> in
> > prep_working() (because comb->res.type == DAV_RESOURCE_TYPE_WORKING).
> > In prep_working(), we get all the way to this code near the end:
> >
> > derr = fs_check_path(&kind, comb->priv.root.root,
> > comb->priv.repos_path, pool);
> >
> > if (derr != NULL)
> > return derr;
> >
> > comb->res.exists = (kind == svn_node_none) ? FALSE : TRUE;
> > comb->res.collection = (kind == svn_node_dir) ? TRUE : FALSE;
> >
> > So that's where comb->res.exists gets set to 1. (I'm not sure where
> > comb->exists gets set to 1, which might be important, but see below.)
> >
> > Anyway, that call to fs_check_path() sets kind to svn_node_file, even
> > though the file shouldn't be present in this txn yet (that's why
> we're
> > adding it).
> >
> > Tracing fs_check_path --> svn_fs_fs__check_path() --> node_kind() -->
> > fs_node_id() --> get_dag() --> open_path() --> svn_fs_fs__dag_open(),
> > shows that, somehow, the file is actually there, according to the FS.
> > (It does not appear to be a dag node caching problem; the node isn't
> > found in the cache, instead we find it the old-fashioned way, right
> > there in the filesystem.)
> >
> > Back up in http-
> 2.2.9/modules/dav/main/mod_dav.c:dav_method_propfind(),
> > this all matters because of this code:
> >
> > /* Ask repository module to resolve the resource */
> > err = dav_get_resource(r, 1 /* label_allowed */, 0 /*
> use_checked_in */,
> > &resource);
> > if (err != NULL)
> > return dav_handle_err(r, err, NULL);
> >
> > if (dav_get_resource_state(r, resource) == DAV_RESOURCE_NULL) {
> > /* Apache will supply a default error for this. */
> > return HTTP_NOT_FOUND;
> > }
> >
> > Not only does dav_get_resource() not get an error, but the call to
> > dav_get_resource_state() will not return DAV_RESOURCE_NULL, because
> that
> > code (in httpd-2.2.9/modules/dav/main/util_lock.c) says:
> >
> > if (resource->exists)
> > return DAV_RESOURCE_EXISTS;
> >
> > Boom. If it weren't for that, the function would return
> > DAV_RESOURCE_NULL and we'd be fine.
> >
> > Note that in the reproduction recipe, if you insert an "svn up
> Data/wc"
> > right before the final svn cp in the reproduction recipe, then none
> of
> > this weirdness happens, and the bug doesn't reproduce.
> >
> > So why does the file claim to be present in the txn already? And why
> > does updating the whole working copy to r2 (instead of having a mixed
> > r1/r2 working copy) cause the repository filesystem to correctly
> answer
> > that the file is not there?
> >
> > I don't know. I will continue to debug, but if anyone has any
> inspired
> > ideas, please post.
> >
> > -Karl
> >
> > Karl Fogel <kf...@red-bean.com> writes:
> > > "Michael Susser" <hd...@googlemail.com> writes:
> > >> i have found a bug when creating tags from working copy in svn
> 1.5.2
> > >> (and also 1.5.3). Here is a recipe how to reproduce the error: (my
> > >> apache is configured to host the repositories in E:\SVNRep)
> > >>
> > >> [...]
> > >>
> > >> svn copy -m"tagging" E:\Data\wc\trunk
> http://myserver/svn/repo/tags/V1.0.0
> > >>>svn: Commit failed (details follow):
> > >>>svn: File '/svn/repo/tags/V1.0.0/dir/file.cpp' already exists
> > >
> > > Thank you for the excellent recipe. I can reproduce this, and have
> > > written a Unix Bourne shell reproduction script (see below). Note
> that
> > > this only reproduces over http://. If you try it with svn:// or
> > > file://, the tagging works fine.
> > >
> > >> This error message is absolutely wrong! I investigated what went
> wrong and
> > >> found the following: after the commit action the working copy is
> not updated
> > >> correctly(?). The changed file "file.cpp" gets a new revision (2),
> but its
> > >> parent directory "dir" stays at revision 1. Is this ok?
> > >
> > > Well, that part is okay. When you commit, only the committed items
> get
> > > their revisions bumped locally -- in other words, a commit does not
> do
> > > an update on the items that are not part of the commit. However,
> > > Subversion is supposed to transparently handle the resultant mixed
> > > revisions. They shouldn't matter here.
> > >
> > > This bug seems familiar, like we've either fixed it before or filed
> an
> > > issue for it. I'll check the logs and the issue tracker. But we
> have a
> > > reproduction recipe, so that means we can fix it or re-fix it :-).
> > >
> > > -------------------------------------------------------------------
> ---------
> > > #!/bin/sh
> > >
> > > # The next line is the only line you should need to adjust.
> > > SVNDIR=/home/kfogel/src/subversion
> > >
> > > SVN=${SVNDIR}/subversion/svn/svn
> > > SVNSERVE=${SVNDIR}/subversion/svnserve/svnserve
> > > SVNADMIN=${SVNDIR}/subversion/svnadmin/svnadmin
> > >
> > > # Select an access method. If svn://, the svnserve setup is
> > > # handled automagically by this script; but if http://, then
> > > # you'll have to configure it yourself first.
> > > #
> > > URL=http://localhost/ms/repos
> > > # URL=svn://localhost/repos
> > > # URL=file:///`pwd`/repos
> > >
> > > rm -rf repos wc Data import-me
> > >
> > > ${SVNADMIN} create repos
> > >
> > > # These are for svnserve only.
> > > echo "[general]" > repos/conf/svnserve.conf
> > > echo "anon-access = write" >> repos/conf/svnserve.conf
> > > echo "auth-access = write" >> repos/conf/svnserve.conf
> > >
> > > # The server will only be contacted if $URL is svn://foo, of
> course.
> > > ${SVNSERVE} --pid-file svnserve-pid -d -r `pwd`
> > > # And put the kill command in a file, in case need to run it
> manually.
> > > echo "kill -9 `cat svnserve-pid`" > k
> > > chmod a+rwx k
> > >
> > > mkdir Data
> > > svn co ${URL} Data/wc
> > > mkdir Data/wc/tags
> > > mkdir Data/wc/trunk
> > > mkdir Data/wc/trunk/dir
> > > cp /dev/null Data/wc/trunk/dir/file.cpp
> > > ${SVN} add Data/wc/t*
> > > ${SVN} ci -m"test" Data/wc
> > >
> > > echo "ABC" >> Data/wc/trunk/dir/file.cpp
> > >
> > > ${SVN} ci -m"test" Data/wc
> > >
> > > # Using http:// only, the next command fails with this error:
> > > #
> > > # subversion/libsvn_client/copy.c:1319: (apr_err=175005)
> > > # svn: Commit failed (details follow):
> > > # subversion/libsvn_ra_neon/commit.c:1036: (apr_err=175005)
> > > # svn: File '/ms/repos/tags/V1.0.0/dir/file.cpp' already exists
> > > #
> > > ${SVN} copy -m"tagging" Data/wc/trunk ${URL}/tags/V1.0.0
> > >
> > > ./k
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageI
> d=993658
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=993665
RE: Re: Error when creating tags from working copy
Posted by Alexandre Feblot <al...@gmail.com>.
Hoping #3119 is about that, indeed.
Here is the test script reproducing the issue I posted overthere.
The repository is hosted by collabnet, I think the server is still 1.4.
My Client is 1.5.3.
#)svn co https://a_project.company.net/svn/a_project/trunk testsvn
A testsvn/www
A testsvn/www/cn-project-pages
A testsvn/www/cn-project-pages/snippets
A testsvn/www/cn-project-pages/snippets/page.xml
A testsvn/www/index.html
Checked out revision 5.
#)cd testsvn/
#)svn cp . https://a_project.company.net/svn/a_project/branches/test1 -m 'test1: this one must work fine, since done on a fresh clean sandbox'
Committed revision 6.
#) vi www/index.html, changing the file
#)svn ci www/index.html -m "small change in index.html"
Sending www/index.html
Transmitting file data .
Committed revision 7.
#)svn cp . https://a_project.company.net/svn/a_project/branches/test2 -m 'test2: this one will fail, because of the file commit'
svn: Commit failed (details follow):
svn: File '/svn/a_project/branches/test2/www/index.html' already exists
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1561489
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Error when creating tags from working copy
Posted by Karl Fogel <kf...@red-bean.com>.
Daniel Shahaf <d....@daniel.shahaf.name> writes:
> Karl, is this bug still open? And how many bugs are there here (bug in
> mod_dav_svn? bug in the fs layer?)?
>
> I'd like to file an issue for it, at least, but I'm not sure I'll describe
> it correctly. Could you do that?
As far as I know, it's still open. A good summary would be:
"txn wrongly claims file exists (over http:// only)"
(The tag-creation error is just one side effect of that underlying bug.)
The relevant emails are:
Michael Susser's original post:
http://svn.haxx.se/users/archive-2008-10/0312.shtml
My initial response, with reproduction script:
http://svn.haxx.se/users/archive-2008-10/0352.shtml
Mail in which I noted a separate, possibly unrelated problem:
http://svn.haxx.se/users/archive-2008-10/0355.shtml
Mail in which I dig deep into mod_dav (anyone debugging the main
problem will want to read this mail carefully):
http://svn.haxx.se/users/archive-2008-10/0357.shtml
I'd like to file an issue for it, but I don't have time right now to
check whether #3119 is already the right issue or whether we need a new
one. The above list is to enable someone else to do that if they want;
I wish I could guarantee more, but can't.
-Karl
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=996060
RE: Error when creating tags from working copy
Posted by Bert Huijben <rh...@sharpsvn.net>.
> -----Original Message-----
> From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name]
> Sent: Saturday, December 27, 2008 11:27 AM
> To: Karl Fogel
> Cc: users@subversion.tigris.org; dev@subversion.tigris.org
> Subject: Re: Error when creating tags from working copy
>
> [ CCing dev@ ]
>
> Karl, is this bug still open? And how many bugs are there here (bug in
> mod_dav_svn? bug in the fs layer?)?
>
> I'd like to file an issue for it, at least, but I'm not sure I'll
> describe
> it correctly. Could you do that?
I tried to reopen this issue on the developer list in the
"Issue #3119 - File '...' already exists" (Regression since 1.5.0)" thread
on the development list.
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=985
284
We did some research but the thread died on the major problem that we can't
reproduce it at will.
To fully resolve this issue we really need a script that can reproduce the
issue every (or at least some of the) time.
I'm just guessing but maybe the following factors might help us:
* Which server versions have the issue? (1.5.x?) (1.4.x?)
* The issue was reported only to client versions after 1.5.0. Did somebody
reproduce it with a trunk build?
Original reporters, please fill in the missing details... we need your help
to resolve this issue.
Thanks,
Bert
>
> Thanks,
>
> Daniel
>
>
> Karl Fogel wrote on Sat, 11 Oct 2008 at 18:25 -0400:
> > This is a deep one. I'm posting what I've learned so far in case
> anyone
> > has any hunches. On the client, the problem is that in
> >
> > libsvn_ra_neon/commit.c:commit_add_file()
> >
> > the following code should get a "file not found" error (i.e., an HTTP
> > 404) when it probes for the existence of the file we're about to add,
> > but doesn't -- instead, it's as though the file already exists:
> >
> > svn_ra_neon__resource_t *res;
> > svn_error_t *err = svn_ra_neon__get_starting_props(&res,
> > file->cc-
> >ras,
> > file->rsrc-
> >wr_url,
> > NULL,
> workpool);
> > if (!err)
> > {
> > /* If the PROPFIND succeeds the file already exists */
> > return svn_error_createf(SVN_ERR_RA_DAV_ALREADY_EXISTS,
> NULL,
> > _("File '%s' already exists"),
> > file->rsrc->url);
> > }
> >
> > Here's what happens on the server side. Let's shortcut directly to
> > mod_dav_svn/repos.c:get_resource(). Before the call to
> > prep_resource()...
> >
> > /* prepare the resource for operation */
> > if ((err = prep_resource(comb)) != NULL)
> > return err;
> >
> > ...comb->exists is 0, as we would want (so is comb->res.exists, for
> that
> > matter). But after prep_resource(), comb->exists is 1!
> >
> > prep_resource() is just a dispatch function; the real stuff happens
> in
> > prep_working() (because comb->res.type == DAV_RESOURCE_TYPE_WORKING).
> > In prep_working(), we get all the way to this code near the end:
> >
> > derr = fs_check_path(&kind, comb->priv.root.root,
> > comb->priv.repos_path, pool);
> >
> > if (derr != NULL)
> > return derr;
> >
> > comb->res.exists = (kind == svn_node_none) ? FALSE : TRUE;
> > comb->res.collection = (kind == svn_node_dir) ? TRUE : FALSE;
> >
> > So that's where comb->res.exists gets set to 1. (I'm not sure where
> > comb->exists gets set to 1, which might be important, but see below.)
> >
> > Anyway, that call to fs_check_path() sets kind to svn_node_file, even
> > though the file shouldn't be present in this txn yet (that's why
> we're
> > adding it).
> >
> > Tracing fs_check_path --> svn_fs_fs__check_path() --> node_kind() -->
> > fs_node_id() --> get_dag() --> open_path() --> svn_fs_fs__dag_open(),
> > shows that, somehow, the file is actually there, according to the FS.
> > (It does not appear to be a dag node caching problem; the node isn't
> > found in the cache, instead we find it the old-fashioned way, right
> > there in the filesystem.)
> >
> > Back up in http-
> 2.2.9/modules/dav/main/mod_dav.c:dav_method_propfind(),
> > this all matters because of this code:
> >
> > /* Ask repository module to resolve the resource */
> > err = dav_get_resource(r, 1 /* label_allowed */, 0 /*
> use_checked_in */,
> > &resource);
> > if (err != NULL)
> > return dav_handle_err(r, err, NULL);
> >
> > if (dav_get_resource_state(r, resource) == DAV_RESOURCE_NULL) {
> > /* Apache will supply a default error for this. */
> > return HTTP_NOT_FOUND;
> > }
> >
> > Not only does dav_get_resource() not get an error, but the call to
> > dav_get_resource_state() will not return DAV_RESOURCE_NULL, because
> that
> > code (in httpd-2.2.9/modules/dav/main/util_lock.c) says:
> >
> > if (resource->exists)
> > return DAV_RESOURCE_EXISTS;
> >
> > Boom. If it weren't for that, the function would return
> > DAV_RESOURCE_NULL and we'd be fine.
> >
> > Note that in the reproduction recipe, if you insert an "svn up
> Data/wc"
> > right before the final svn cp in the reproduction recipe, then none
> of
> > this weirdness happens, and the bug doesn't reproduce.
> >
> > So why does the file claim to be present in the txn already? And why
> > does updating the whole working copy to r2 (instead of having a mixed
> > r1/r2 working copy) cause the repository filesystem to correctly
> answer
> > that the file is not there?
> >
> > I don't know. I will continue to debug, but if anyone has any
> inspired
> > ideas, please post.
> >
> > -Karl
> >
> > Karl Fogel <kf...@red-bean.com> writes:
> > > "Michael Susser" <hd...@googlemail.com> writes:
> > >> i have found a bug when creating tags from working copy in svn
> 1.5.2
> > >> (and also 1.5.3). Here is a recipe how to reproduce the error: (my
> > >> apache is configured to host the repositories in E:\SVNRep)
> > >>
> > >> [...]
> > >>
> > >> svn copy -m"tagging" E:\Data\wc\trunk
> http://myserver/svn/repo/tags/V1.0.0
> > >>>svn: Commit failed (details follow):
> > >>>svn: File '/svn/repo/tags/V1.0.0/dir/file.cpp' already exists
> > >
> > > Thank you for the excellent recipe. I can reproduce this, and have
> > > written a Unix Bourne shell reproduction script (see below). Note
> that
> > > this only reproduces over http://. If you try it with svn:// or
> > > file://, the tagging works fine.
> > >
> > >> This error message is absolutely wrong! I investigated what went
> wrong and
> > >> found the following: after the commit action the working copy is
> not updated
> > >> correctly(?). The changed file "file.cpp" gets a new revision (2),
> but its
> > >> parent directory "dir" stays at revision 1. Is this ok?
> > >
> > > Well, that part is okay. When you commit, only the committed items
> get
> > > their revisions bumped locally -- in other words, a commit does not
> do
> > > an update on the items that are not part of the commit. However,
> > > Subversion is supposed to transparently handle the resultant mixed
> > > revisions. They shouldn't matter here.
> > >
> > > This bug seems familiar, like we've either fixed it before or filed
> an
> > > issue for it. I'll check the logs and the issue tracker. But we
> have a
> > > reproduction recipe, so that means we can fix it or re-fix it :-).
> > >
> > > -------------------------------------------------------------------
> ---------
> > > #!/bin/sh
> > >
> > > # The next line is the only line you should need to adjust.
> > > SVNDIR=/home/kfogel/src/subversion
> > >
> > > SVN=${SVNDIR}/subversion/svn/svn
> > > SVNSERVE=${SVNDIR}/subversion/svnserve/svnserve
> > > SVNADMIN=${SVNDIR}/subversion/svnadmin/svnadmin
> > >
> > > # Select an access method. If svn://, the svnserve setup is
> > > # handled automagically by this script; but if http://, then
> > > # you'll have to configure it yourself first.
> > > #
> > > URL=http://localhost/ms/repos
> > > # URL=svn://localhost/repos
> > > # URL=file:///`pwd`/repos
> > >
> > > rm -rf repos wc Data import-me
> > >
> > > ${SVNADMIN} create repos
> > >
> > > # These are for svnserve only.
> > > echo "[general]" > repos/conf/svnserve.conf
> > > echo "anon-access = write" >> repos/conf/svnserve.conf
> > > echo "auth-access = write" >> repos/conf/svnserve.conf
> > >
> > > # The server will only be contacted if $URL is svn://foo, of
> course.
> > > ${SVNSERVE} --pid-file svnserve-pid -d -r `pwd`
> > > # And put the kill command in a file, in case need to run it
> manually.
> > > echo "kill -9 `cat svnserve-pid`" > k
> > > chmod a+rwx k
> > >
> > > mkdir Data
> > > svn co ${URL} Data/wc
> > > mkdir Data/wc/tags
> > > mkdir Data/wc/trunk
> > > mkdir Data/wc/trunk/dir
> > > cp /dev/null Data/wc/trunk/dir/file.cpp
> > > ${SVN} add Data/wc/t*
> > > ${SVN} ci -m"test" Data/wc
> > >
> > > echo "ABC" >> Data/wc/trunk/dir/file.cpp
> > >
> > > ${SVN} ci -m"test" Data/wc
> > >
> > > # Using http:// only, the next command fails with this error:
> > > #
> > > # subversion/libsvn_client/copy.c:1319: (apr_err=175005)
> > > # svn: Commit failed (details follow):
> > > # subversion/libsvn_ra_neon/commit.c:1036: (apr_err=175005)
> > > # svn: File '/ms/repos/tags/V1.0.0/dir/file.cpp' already exists
> > > #
> > > ${SVN} copy -m"tagging" Data/wc/trunk ${URL}/tags/V1.0.0
> > >
> > > ./k
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageI
> d=993658
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=993666
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Error when creating tags from working copy
Posted by Bert Huijben <rh...@sharpsvn.net>.
> -----Original Message-----
> From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name]
> Sent: Saturday, December 27, 2008 11:27 AM
> To: Karl Fogel
> Cc: users@subversion.tigris.org; dev@subversion.tigris.org
> Subject: Re: Error when creating tags from working copy
>
> [ CCing dev@ ]
>
> Karl, is this bug still open? And how many bugs are there here (bug in
> mod_dav_svn? bug in the fs layer?)?
>
> I'd like to file an issue for it, at least, but I'm not sure I'll
> describe
> it correctly. Could you do that?
I tried to reopen this issue on the developer list in the
"Issue #3119 - File '...' already exists" (Regression since 1.5.0)" thread
on the development list.
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=985
284
We did some research but the thread died on the major problem that we can't
reproduce it at will.
To fully resolve this issue we really need a script that can reproduce the
issue every (or at least some of the) time.
I'm just guessing but maybe the following factors might help us:
* Which server versions have the issue? (1.5.x?) (1.4.x?)
* The issue was reported only to client versions after 1.5.0. Did somebody
reproduce it with a trunk build?
Original reporters, please fill in the missing details... we need your help
to resolve this issue.
Thanks,
Bert
>
> Thanks,
>
> Daniel
>
>
> Karl Fogel wrote on Sat, 11 Oct 2008 at 18:25 -0400:
> > This is a deep one. I'm posting what I've learned so far in case
> anyone
> > has any hunches. On the client, the problem is that in
> >
> > libsvn_ra_neon/commit.c:commit_add_file()
> >
> > the following code should get a "file not found" error (i.e., an HTTP
> > 404) when it probes for the existence of the file we're about to add,
> > but doesn't -- instead, it's as though the file already exists:
> >
> > svn_ra_neon__resource_t *res;
> > svn_error_t *err = svn_ra_neon__get_starting_props(&res,
> > file->cc-
> >ras,
> > file->rsrc-
> >wr_url,
> > NULL,
> workpool);
> > if (!err)
> > {
> > /* If the PROPFIND succeeds the file already exists */
> > return svn_error_createf(SVN_ERR_RA_DAV_ALREADY_EXISTS,
> NULL,
> > _("File '%s' already exists"),
> > file->rsrc->url);
> > }
> >
> > Here's what happens on the server side. Let's shortcut directly to
> > mod_dav_svn/repos.c:get_resource(). Before the call to
> > prep_resource()...
> >
> > /* prepare the resource for operation */
> > if ((err = prep_resource(comb)) != NULL)
> > return err;
> >
> > ...comb->exists is 0, as we would want (so is comb->res.exists, for
> that
> > matter). But after prep_resource(), comb->exists is 1!
> >
> > prep_resource() is just a dispatch function; the real stuff happens
> in
> > prep_working() (because comb->res.type == DAV_RESOURCE_TYPE_WORKING).
> > In prep_working(), we get all the way to this code near the end:
> >
> > derr = fs_check_path(&kind, comb->priv.root.root,
> > comb->priv.repos_path, pool);
> >
> > if (derr != NULL)
> > return derr;
> >
> > comb->res.exists = (kind == svn_node_none) ? FALSE : TRUE;
> > comb->res.collection = (kind == svn_node_dir) ? TRUE : FALSE;
> >
> > So that's where comb->res.exists gets set to 1. (I'm not sure where
> > comb->exists gets set to 1, which might be important, but see below.)
> >
> > Anyway, that call to fs_check_path() sets kind to svn_node_file, even
> > though the file shouldn't be present in this txn yet (that's why
> we're
> > adding it).
> >
> > Tracing fs_check_path --> svn_fs_fs__check_path() --> node_kind() -->
> > fs_node_id() --> get_dag() --> open_path() --> svn_fs_fs__dag_open(),
> > shows that, somehow, the file is actually there, according to the FS.
> > (It does not appear to be a dag node caching problem; the node isn't
> > found in the cache, instead we find it the old-fashioned way, right
> > there in the filesystem.)
> >
> > Back up in http-
> 2.2.9/modules/dav/main/mod_dav.c:dav_method_propfind(),
> > this all matters because of this code:
> >
> > /* Ask repository module to resolve the resource */
> > err = dav_get_resource(r, 1 /* label_allowed */, 0 /*
> use_checked_in */,
> > &resource);
> > if (err != NULL)
> > return dav_handle_err(r, err, NULL);
> >
> > if (dav_get_resource_state(r, resource) == DAV_RESOURCE_NULL) {
> > /* Apache will supply a default error for this. */
> > return HTTP_NOT_FOUND;
> > }
> >
> > Not only does dav_get_resource() not get an error, but the call to
> > dav_get_resource_state() will not return DAV_RESOURCE_NULL, because
> that
> > code (in httpd-2.2.9/modules/dav/main/util_lock.c) says:
> >
> > if (resource->exists)
> > return DAV_RESOURCE_EXISTS;
> >
> > Boom. If it weren't for that, the function would return
> > DAV_RESOURCE_NULL and we'd be fine.
> >
> > Note that in the reproduction recipe, if you insert an "svn up
> Data/wc"
> > right before the final svn cp in the reproduction recipe, then none
> of
> > this weirdness happens, and the bug doesn't reproduce.
> >
> > So why does the file claim to be present in the txn already? And why
> > does updating the whole working copy to r2 (instead of having a mixed
> > r1/r2 working copy) cause the repository filesystem to correctly
> answer
> > that the file is not there?
> >
> > I don't know. I will continue to debug, but if anyone has any
> inspired
> > ideas, please post.
> >
> > -Karl
> >
> > Karl Fogel <kf...@red-bean.com> writes:
> > > "Michael Susser" <hd...@googlemail.com> writes:
> > >> i have found a bug when creating tags from working copy in svn
> 1.5.2
> > >> (and also 1.5.3). Here is a recipe how to reproduce the error: (my
> > >> apache is configured to host the repositories in E:\SVNRep)
> > >>
> > >> [...]
> > >>
> > >> svn copy -m"tagging" E:\Data\wc\trunk
> http://myserver/svn/repo/tags/V1.0.0
> > >>>svn: Commit failed (details follow):
> > >>>svn: File '/svn/repo/tags/V1.0.0/dir/file.cpp' already exists
> > >
> > > Thank you for the excellent recipe. I can reproduce this, and have
> > > written a Unix Bourne shell reproduction script (see below). Note
> that
> > > this only reproduces over http://. If you try it with svn:// or
> > > file://, the tagging works fine.
> > >
> > >> This error message is absolutely wrong! I investigated what went
> wrong and
> > >> found the following: after the commit action the working copy is
> not updated
> > >> correctly(?). The changed file "file.cpp" gets a new revision (2),
> but its
> > >> parent directory "dir" stays at revision 1. Is this ok?
> > >
> > > Well, that part is okay. When you commit, only the committed items
> get
> > > their revisions bumped locally -- in other words, a commit does not
> do
> > > an update on the items that are not part of the commit. However,
> > > Subversion is supposed to transparently handle the resultant mixed
> > > revisions. They shouldn't matter here.
> > >
> > > This bug seems familiar, like we've either fixed it before or filed
> an
> > > issue for it. I'll check the logs and the issue tracker. But we
> have a
> > > reproduction recipe, so that means we can fix it or re-fix it :-).
> > >
> > > -------------------------------------------------------------------
> ---------
> > > #!/bin/sh
> > >
> > > # The next line is the only line you should need to adjust.
> > > SVNDIR=/home/kfogel/src/subversion
> > >
> > > SVN=${SVNDIR}/subversion/svn/svn
> > > SVNSERVE=${SVNDIR}/subversion/svnserve/svnserve
> > > SVNADMIN=${SVNDIR}/subversion/svnadmin/svnadmin
> > >
> > > # Select an access method. If svn://, the svnserve setup is
> > > # handled automagically by this script; but if http://, then
> > > # you'll have to configure it yourself first.
> > > #
> > > URL=http://localhost/ms/repos
> > > # URL=svn://localhost/repos
> > > # URL=file:///`pwd`/repos
> > >
> > > rm -rf repos wc Data import-me
> > >
> > > ${SVNADMIN} create repos
> > >
> > > # These are for svnserve only.
> > > echo "[general]" > repos/conf/svnserve.conf
> > > echo "anon-access = write" >> repos/conf/svnserve.conf
> > > echo "auth-access = write" >> repos/conf/svnserve.conf
> > >
> > > # The server will only be contacted if $URL is svn://foo, of
> course.
> > > ${SVNSERVE} --pid-file svnserve-pid -d -r `pwd`
> > > # And put the kill command in a file, in case need to run it
> manually.
> > > echo "kill -9 `cat svnserve-pid`" > k
> > > chmod a+rwx k
> > >
> > > mkdir Data
> > > svn co ${URL} Data/wc
> > > mkdir Data/wc/tags
> > > mkdir Data/wc/trunk
> > > mkdir Data/wc/trunk/dir
> > > cp /dev/null Data/wc/trunk/dir/file.cpp
> > > ${SVN} add Data/wc/t*
> > > ${SVN} ci -m"test" Data/wc
> > >
> > > echo "ABC" >> Data/wc/trunk/dir/file.cpp
> > >
> > > ${SVN} ci -m"test" Data/wc
> > >
> > > # Using http:// only, the next command fails with this error:
> > > #
> > > # subversion/libsvn_client/copy.c:1319: (apr_err=175005)
> > > # svn: Commit failed (details follow):
> > > # subversion/libsvn_ra_neon/commit.c:1036: (apr_err=175005)
> > > # svn: File '/ms/repos/tags/V1.0.0/dir/file.cpp' already exists
> > > #
> > > ${SVN} copy -m"tagging" Data/wc/trunk ${URL}/tags/V1.0.0
> > >
> > > ./k
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageI
> d=993658
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=993666
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].