You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by François Beausoleil <fb...@ftml.net> on 2004/11/25 15:29:53 UTC
Unbounded memory usage in large WC move operations ?
Hello everyone !
I believe there might be some problems with very large working copies.
I checked out my whole vendor-drop repository location to a WC, because
I want to do reorganizations, and I wanted them to be a single atomic
commit.
My vendor drop is organized like this:
/ <-- repos root
vendor/ <-- WC root
log4j/
1.2.8/
current/
velocity/
1.4/
current/
I want to rename all current/ to trunk/ and move all version tags to a
tags folder. Additionaly, I want to reorganize so that some vendor
drops are under a common root. For example, all Apache projects go
together, Jakarta projects go together, etc.
Server and client are on the same physical machines, and the machine is
a P4 3.0 Ghz, 1 Gb RAM and 80 Gb 7200 RPM HD. OS is Windows XP SP2, and
SVN is 1.1.1 [svn, version 1.1.0 (r11180)].
Checkout was fine. Took lots of time, but that was to be expected,
given the size of the checkout.
The problem occured when I started moving. I executed the following
from the root of my WC:
$ svn mkdir jakarta
$ svn move log4j jakarta
Believe it or not, the SVN client grew to use almost 300 Mb of RAM. I
have a picture of the Windows Task Manager with the memory and CPU graph
showing:
http://fbeausoleil.ftml.net/svn-large-move-mem-usage.gif
The whole move operation took something like 20 or 30 minutes (didn't
keep track - I expected things to go much faster). It looks like the
move operation operated in three distinct phases:
1. WC locking phase seemed to use an unbounded amount of memory - 300 MB
for my WC. This took almost half of the runtime;
2. After the peak had been reached, memory usage stayed constant, or
near enough. This part took about half of the runtime too;
3. WC unlocking began, and finished pretty quickly. Seemed like 30
seconds to a minute. Memory usage quickly dropped back to zero at that
point.
I don't have exact statistics, but my WC has 53 libraries, each with a
current/ and, on average, two version tags (so 3 * 53 = 159 vendor
drops), and each vendor drop contains hundreds of files and folders.
What I suspect is going on here is that SVN is locking the whole WC,
instead of locking the parent (vendor/) folder, and the two children it
is interested in (log4j/ and jakarta/).
Is that to be expected ? Am I missing some locking behavior here ?
Should I forget about one single atomic commit, and do everything
server-side instead ? I know it'll be much faster. Using the bindings
[a reason to learn how to do it, I guess ;)], is it possible to do
multiple server-side moves ?
Thanks !
François
Re: Unbounded memory usage in large WC move operations ?
Posted by Philip Martin <ph...@codematters.co.uk>.
François Beausoleil <fb...@ftml.net> writes:
> 1. WC locking phase seemed to use an unbounded amount of memory - 300
> MB for my WC. This took almost half of the runtime;
Yes. Most operations hold all the entries files in memory, it's a
known limitation of the current libsvn_wc implementation.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org