You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Eric Kidd <er...@pobox.com> on 2002/11/16 17:05:07 UTC
Poor performance with big tree (5K files)
Hello! I decided to download and mistreat your nifty VC system. I got
Subversion to misbehave in a variety of interesting ways, but I don't
know if the problems are specific to my system.
Cheers,
Eric
What I did:
I installed subversion 0.14.5 on a Debian testing/unstable system,
using 'apt-get -t unstable install subversion-client
subversion-server'. I configured an Apache 2 server using prefork
mode (the traditional threading model), and checked in a copy of the
wxWindows 2.3.3 source distribution (which contains 5,000+ files).
I executed 'svn cp' a few times to lay out my repository, and typed
'svn co' on my newly-created trunk.
My laptop has 256MB of RAM and an 800MHz Pentium III processor. The
single '/' filesystem uses ext3 in ordered data mode.
What happened:
Subversion took approximately 25 minutes to check out the tree, and
rattled my disks like you wouldn't believe. A quick look at 'top'
'M' revealed that 'svn' was consuming 50MB of RAM, and 7 Apache
processes were consuming anywhere from 10 to 70MB of RAM:
607 www-data 9 0 76880 66M 2648 S 0.0 26.7 0:44 apache2
658 emk 17 0 56420 22M 2148 S 5.3 9.1 2:39 svn
391 www-data 9 0 20716 14M 1932 S 0.0 5.8 0:30 apache2
388 www-data 9 0 12676 11M 2032 S 0.0 4.7 0:30 apache2
661 www-data 9 0 28132 10M 6064 S 0.0 4.1 0:31 apache2
390 www-data 12 0 10756 10M 2368 R 2.3 4.0 0:23 apache2
387 www-data 9 0 10472 9980 2040 S 0.0 3.9 0:26 apache2
389 www-data 9 0 10116 9624 2032 S 0.0 3.7 0:23 apache2
This is what a fresh Apache2 process looks like on my system:
763 www-data 11 0 2372 2372 2284 S 0.0 0.9 0:00 apache2
The Apache2 processes did not shrink after the checkout was
complete. This is probably because Debian sets
'MaxRequestsPerChild' to 0, keeping alive server processes for an
indefinite amount of time.
Interestingly enough, the Apache processes don't appear to grow much
further if I do a second checkout. But I haven't investigated this
all that thoroughly.
What should have happened:
The checkout should have completed in 5 minutes or so, using a
reasonable amount of RAM for both 'svn' and 'apache2'. When 'svn'
was finished, the Apache processes should ideally have shrunk back
to a manageable size.
My theories:
Subversion's checkout speed is *much* slower than CVS's, but most of
the difference seems to be on the wall clock. Subversion checkouts
also generate a massive amount of disk I/O, even when the system has
yet to touch the swap partition. Taken together, these facts
suggest that somebody (perhaps APR) is calling sync(2) way too
often.
The massive bloat of Apache2 might be spurious--perhaps the database
is mmaping big files--but svn is definitely doing something wrong.
Unfortunately, both valgrind and mprof appear broken on my system,
so I can't check for memory leaks.
Comparison to CVS:
'svn update' and 'svn status' are faster than the corresponding CVS
operations on this tree, but they still take anywhere from 3 to 30
seconds, depending on how warm my caches are. 'svn import' and 'svn
checkout' are slower than the corresponding CVS commands by nearly
an order of magnitude.
The inability of svn to recover after an aborted update makes this
kind of testing rather awkward.
Subversion shows tremendous promise, but it's still pretty awkward
for big trees.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Poor performance with big tree (5K files)
Posted by Brandon Ehle <az...@yahoo.com>.
Eric Kidd wrote:
>Hello! I decided to download and mistreat your nifty VC system. I got
>Subversion to misbehave in a variety of interesting ways, but I don't
>know if the problems are specific to my system.
>
>
[...]
> Subversion shows tremendous promise, but it's still pretty awkward
> for big trees.
>
>
>
Checkout my Subversion Performance tracking site at
http://www.pipedreaminteractive.com/svn/
I am starting document most of what you've discovered here, but also
trying to find solutions, workarounds or create bugs in the database.
First off, apache + prefork mode is extremely slow, I've been playing
around with worker mode the last few days and having good results. I
haven't gotten around to charting worker vs prefork performance yet, but
expect to see it soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Poor performance with big tree (5K files)
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Eric Kidd <er...@pobox.com> writes:
> Hello! I decided to download and mistreat your nifty VC system. I got
> Subversion to misbehave in a variety of interesting ways, but I don't
> know if the problems are specific to my system.
You should upgrade to the latest Subversion (0.15 as of this writing)
and try again. See subversion.tigris.org for details.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org