You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by William Uther <wi...@cse.unsw.edu.au> on 2002/12/13 06:10:06 UTC
ra_svn deadlock
Hi,
Having found ra_svn deadlocks on MacOS X, I just tried it on a linux
box. This machine is running RedHat 8.0. It is using the BDB 4.0 that
is part of the default install. It is using apr and apr-util from
Apache 2.0.43. I'm using subversion revision 4111.
ra_local passes make check. Then...
make install
svnserve -d
make check
BASE_URL=svn://localhost/`pwd`/subversion/tests/clients/cmdline
leads to what looks like a deadlock in the update tests. The tail of
tests.log looks as follows:
> START: copy_tests.py
> CMD: svnadmin "create" "local_tmp/repos"
> CMD: svn "import" "--username" "jrandom" "--password" "rayjandom" "-m"
> "Log message for revision 1."
> "svn://localhost//src/subversion/subversion/tests/clients/cmdline/
> local_tmp/repos" "local_tmp/greekfiles"
> CMD: svn "co" "--username" "jrandom" "--password" "rayjandom"
> "svn://localhost//src/subversion/subversion/tests/clients/cmdline/
> repositories/copy_tests-1" "working_copies/copy_tests-1"
> Interrupted
If I set it up to tunnel over ssh then I get what looks like a deadlock
in the diff tests. The tail of tests.log looks like this:
> PASS: diff_tests.py 4: replace a file with a file
> CMD: svn "co" "--username" "jrandom" "--password" "rayjandom"
> "svn://localhost//src/subversion/subversion/tests/clients/cmdline/
> repositories/diff_tests-5" "working_copies/diff_tests-5"
> CMD: svn "up" "-r" "HEAD"
> CMD: svn "add" "A/B/E/theta"
[snip]
> CMD: svn "up" "-r" "1"
> CMD: svn "diff" "-r" "4"
> CMD: svn "diff" "-r" "4"
> CMD: svn "diff" "-r" "4"
> CMD: svn "diff" "-r" "1:4"
> Interrupted
These are the same results I got on MacOS X (with an earlier revision
of svn and both BDB4 and BDB4.1). Is noone else seeing this? Is
anyone else trying 'make check' with ra_svn?
Any suggestions on where to start looking? It seems to be fairly
reproducible.
Later,
Will :-}
--
Dr William Uther National ICT Australia
Phone: +61 2 9385 6926 School of Computer Science and
Engineering
willu@cse.unsw.edu.au University of New South Wales, Australia
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Re: ra_svn deadlock
Posted by Branko Čibej <br...@xbc.nu>.
Philip Martin wrote:
>Philip Martin <ph...@codematters.co.uk> writes:
>
>
>
>>but I not sure that that is guaranteed to work either. We could do a
>>proper hot-backup, but we would either need to know the path to the
>>db4 utilities or we would have to know the names of the relevant db4
>>files, neither of which is appealing. Alternatively we could use an
>>svnadmin dump/load cycle to duplicate the repository. I haven't tried
>>that because, while I'm sure Python can handle redirecting input, I
>>don't know how to do it :)
>>
>>
>
>OK Python experts, what about this. Will it work on non-Unix
>platforms? Would it be better to dump to an intermediate file?
>
copy_repos should work on non-unix platforms, yes. We're already using
os.popen in the tests to run other binaries, no problems there.
One thing I'd like to see, though, is more serious checking of the trace
output -- at least matching against valid patterns would be nice.
Another issue is the 1k reads. Our test repositories are small enough
that it should be perfectly OK to slurp the whole dump file into memory
at once. Or maybe limit the read buffer to 100k, definitely not 1k.
>* subversion/tests/clients/cmdline/svntest/main.py (copy_repos): New
>
>* subversion/tests/clients/cmdline/externals_tests.py
> (externals_test_setup): Use copy_repos.
>
>Index: subversion/tests/clients/cmdline/svntest/main.py
>===================================================================
>--- subversion/tests/clients/cmdline/svntest/main.py (revision 4125)
>+++ subversion/tests/clients/cmdline/svntest/main.py (working copy)
>@@ -271,6 +271,32 @@
> # make the repos world-writeable, for mod_dav_svn's sake.
> chmod_tree(path, 0666, 0666)
>
>+# For copying a repository
>+def copy_repos(src_path, dst_path, head_revision):
>+ "Copy the repository SRC_PATH to DST_PATH"
>+
>+ # A BDB hot-backup procedure would be more efficient, but that would
>+ # require access to the BDB tools, and this doesn't.
>+ create_repos(dst_path)
>+ dump_in, dump_out, dump_err = os.popen3(svnadmin_binary + " dump " + src_path)
>+ load_in, load_out, load_err = os.popen3(svnadmin_binary + " load " + dst_path)
>+
>+ while 1:
>+ data = dump_out.read(1024) # Size picked at random
>+ if data == "":
>+ break
>+ load_in.write(data)
>+ load_in.close() # Tell load we are done? Seems to work.
>+
>+ # Check we have vaguely sensible looking output
>+ if len(dump_err.readlines()) != head_revision + 1: # +1 for revision 0
>+ print "Dump failed!"
>+ load_out_lines = load_out.readlines()
>+ if (load_out_lines[len(load_out_lines)-2]
>+ != "------- Committed new rev " + str(head_revision)
>+ + " (loaded from original rev " + str(head_revision) + ") >>>\n"):
>+ print "Load failed!"
>+
> def set_repos_paths(repo_dir):
> "Set current_repo_dir and current_repo_url from a relative path to the repo."
> global current_repo_dir, current_repo_url
>Index: subversion/tests/clients/cmdline/externals_tests.py
>===================================================================
>--- subversion/tests/clients/cmdline/externals_tests.py (revision 4125)
>+++ subversion/tests/clients/cmdline/externals_tests.py (working copy)
>@@ -144,7 +144,7 @@
> # the one to which the first repository's `svn:externals' properties
> # will refer. After this, both repositories have five revisions
> # of random stuff, with no svn:externals props set yet.
>- shutil.copytree(repo_dir, other_repo_dir)
>+ svntest.main.copy_repos(repo_dir, other_repo_dir, 5)
>
> # Set up the externals properties on A/B/ and A/D/.
> externals_desc = \
>
>
>
>
>
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Re: ra_svn deadlock
Posted by Philip Martin <ph...@codematters.co.uk>.
Philip Martin <ph...@codematters.co.uk> writes:
> + dump_in, dump_out, dump_err = os.popen3(svnadmin_binary + " dump " + src_path)
> + load_in, load_out, load_err = os.popen3(svnadmin_binary + " load " + dst_path)
> +
> + while 1:
> + data = dump_out.read(1024) # Size picked at random
> + if data == "":
> + break
> + load_in.write(data)
> + load_in.close() # Tell load we are done? Seems to work.
Well, I guess I should close all six files :)
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
[PATCH] Re: ra_svn deadlock
Posted by Philip Martin <ph...@codematters.co.uk>.
Philip Martin <ph...@codematters.co.uk> writes:
> but I not sure that that is guaranteed to work either. We could do a
> proper hot-backup, but we would either need to know the path to the
> db4 utilities or we would have to know the names of the relevant db4
> files, neither of which is appealing. Alternatively we could use an
> svnadmin dump/load cycle to duplicate the repository. I haven't tried
> that because, while I'm sure Python can handle redirecting input, I
> don't know how to do it :)
OK Python experts, what about this. Will it work on non-Unix
platforms? Would it be better to dump to an intermediate file?
* subversion/tests/clients/cmdline/svntest/main.py (copy_repos): New
* subversion/tests/clients/cmdline/externals_tests.py
(externals_test_setup): Use copy_repos.
Index: subversion/tests/clients/cmdline/svntest/main.py
===================================================================
--- subversion/tests/clients/cmdline/svntest/main.py (revision 4125)
+++ subversion/tests/clients/cmdline/svntest/main.py (working copy)
@@ -271,6 +271,32 @@
# make the repos world-writeable, for mod_dav_svn's sake.
chmod_tree(path, 0666, 0666)
+# For copying a repository
+def copy_repos(src_path, dst_path, head_revision):
+ "Copy the repository SRC_PATH to DST_PATH"
+
+ # A BDB hot-backup procedure would be more efficient, but that would
+ # require access to the BDB tools, and this doesn't.
+ create_repos(dst_path)
+ dump_in, dump_out, dump_err = os.popen3(svnadmin_binary + " dump " + src_path)
+ load_in, load_out, load_err = os.popen3(svnadmin_binary + " load " + dst_path)
+
+ while 1:
+ data = dump_out.read(1024) # Size picked at random
+ if data == "":
+ break
+ load_in.write(data)
+ load_in.close() # Tell load we are done? Seems to work.
+
+ # Check we have vaguely sensible looking output
+ if len(dump_err.readlines()) != head_revision + 1: # +1 for revision 0
+ print "Dump failed!"
+ load_out_lines = load_out.readlines()
+ if (load_out_lines[len(load_out_lines)-2]
+ != "------- Committed new rev " + str(head_revision)
+ + " (loaded from original rev " + str(head_revision) + ") >>>\n"):
+ print "Load failed!"
+
def set_repos_paths(repo_dir):
"Set current_repo_dir and current_repo_url from a relative path to the repo."
global current_repo_dir, current_repo_url
Index: subversion/tests/clients/cmdline/externals_tests.py
===================================================================
--- subversion/tests/clients/cmdline/externals_tests.py (revision 4125)
+++ subversion/tests/clients/cmdline/externals_tests.py (working copy)
@@ -144,7 +144,7 @@
# the one to which the first repository's `svn:externals' properties
# will refer. After this, both repositories have five revisions
# of random stuff, with no svn:externals props set yet.
- shutil.copytree(repo_dir, other_repo_dir)
+ svntest.main.copy_repos(repo_dir, other_repo_dir, 5)
# Set up the externals properties on A/B/ and A/D/.
externals_desc = \
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ra_svn deadlock
Posted by Philip Martin <ph...@codematters.co.uk>.
Philip Martin <ph...@codematters.co.uk> writes:
> > Since svnserve is holding a repository open for the lifetime of a
> > connection, then depending on how locking works, it may be impossible
> > for two client connections to access the same repository at the same
> > time. That would also affect diff-tests 5.
>
> We often have multiple "clients" holding database locks at the same
> time, I don't think that's a problem. diff-tests 5 passes :)
>
> > I don't know why I haven't run into this in my tests. Maybe a db 4.0.14
> > vs. db 4.1 issue.
>
> I'm using 4.1.
Oops, sent that before it was finished.
I'm using 4.1, and with the svnadmin recover hack all the tests pass.
In fact, they pass if I simply do a time.sleep(2) before copying the
repository, and then I don't need to run recover. So I conclude that
the cause of the problem is that the svnserve processs terminates
asychronously, and causes the repository to get modified during the
copy. That's not an svnserve bug, rather externals_tests.py is not
doing the right thing when copying a repository.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ra_svn deadlock
Posted by Philip Martin <ph...@codematters.co.uk>.
Greg Hudson <gh...@MIT.EDU> writes:
> On Fri, 2002-12-13 at 10:45, Philip Martin wrote:
> > I get a deadlock in daemon mode on Linux. I think the problem is
> > caused by externals_tests.py using a simple directory copy to
> > duplicate a repository. Is that a valid way to duplicate a
> > repository?
>
> No, but this may uncover a real bug.
>
> Since svnserve is holding a repository open for the lifetime of a
> connection, then depending on how locking works, it may be impossible
> for two client connections to access the same repository at the same
> time. That would also affect diff-tests 5.
We often have multiple "clients" holding database locks at the same
time, I don't think that's a problem. diff-tests 5 passes :)
> I don't know why I haven't run into this in my tests. Maybe a db 4.0.14
> vs. db 4.1 issue.
I'm using 4.1.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ra_svn deadlock
Posted by Brandon Ehle <az...@yahoo.com>.
Greg Hudson wrote:
>On Fri, 2002-12-13 at 10:45, Philip Martin wrote:
>
>
>>I get a deadlock in daemon mode on Linux. I think the problem is
>>caused by externals_tests.py using a simple directory copy to
>>duplicate a repository. Is that a valid way to duplicate a
>>repository?
>>
>>
>
>No, but this may uncover a real bug.
>
>Since svnserve is holding a repository open for the lifetime of a
>connection, then depending on how locking works, it may be impossible
>for two client connections to access the same repository at the same
>time. That would also affect diff-tests 5.
>
>I don't know why I haven't run into this in my tests. Maybe a db 4.0.14
>vs. db 4.1 issue.
>
>
Please note that I have found deadlocks in the db-4.0.14 code that was
fixed in db-4.1.24 with mod_dav_svn. I have not tested that the bug is
reproducible with svnserve, but its possible it exists there too.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ra_svn deadlock
Posted by Greg Hudson <gh...@MIT.EDU>.
On Fri, 2002-12-13 at 10:45, Philip Martin wrote:
> I get a deadlock in daemon mode on Linux. I think the problem is
> caused by externals_tests.py using a simple directory copy to
> duplicate a repository. Is that a valid way to duplicate a
> repository?
No, but this may uncover a real bug.
Since svnserve is holding a repository open for the lifetime of a
connection, then depending on how locking works, it may be impossible
for two client connections to access the same repository at the same
time. That would also affect diff-tests 5.
I don't know why I haven't run into this in my tests. Maybe a db 4.0.14
vs. db 4.1 issue.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ra_svn deadlock
Posted by Philip Martin <ph...@codematters.co.uk>.
Greg Hudson <gh...@MIT.EDU> writes:
> After your first post on the topic, I verified that "make check" works
> with me on Linux in daemon mode. So I'm not sure why you run into a
> deadlock in copy-tests 1.
I get a deadlock in daemon mode on Linux. I think the problem is
caused by externals_tests.py using a simple directory copy to
duplicate a repository. Is that a valid way to duplicate a
repository?
I can "fix" the problem by running 'svnadmin recover' as follows
Index: subversion/tests/clients/cmdline/externals_tests.py
===================================================================
--- subversion/tests/clients/cmdline/externals_tests.py (revision 4111)
+++ subversion/tests/clients/cmdline/externals_tests.py (working copy)
@@ -145,6 +145,8 @@
# will refer. After this, both repositories have five revisions
# of random stuff, with no svn:externals props set yet.
shutil.copytree(repo_dir, other_repo_dir)
+ out_lines, err_lines = svntest.main.run_svnadmin('recover', other_repo_dir)
+ if err_lines: return 1
# Set up the externals properties on A/B/ and A/D/.
externals_desc = \
but I not sure that that is guaranteed to work either. We could do a
proper hot-backup, but we would either need to know the path to the
db4 utilities or we would have to know the names of the relevant db4
files, neither of which is appealing. Alternatively we could use an
svnadmin dump/load cycle to duplicate the repository. I haven't tried
that because, while I'm sure Python can handle redirecting input, I
don't know how to do it :)
> It does sound like there's some kind of single-threadedness in tunnel
> mode. I'll try to find time to take a look tomorrow.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ra_svn deadlock
Posted by Greg Hudson <gh...@MIT.EDU>.
After your first post on the topic, I verified that "make check" works
with me on Linux in daemon mode. So I'm not sure why you run into a
deadlock in copy-tests 1.
It does sound like there's some kind of single-threadedness in tunnel
mode. I'll try to find time to take a look tomorrow.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org