You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by David Young <dy...@pobox.com> on 2006/05/04 04:50:26 UTC
subversion 1.3.1 + apr 0.9.7 = memory leak?
I have installed Subversion 1.3.1 and APR 0.9.7 from pkgsrc on a NetBSD
3.99.7 box. I created a fresh fsfs database and svn import'd the
entire NetBSD source tree.
When I run commands such as 'svn co' and 'svn status -u' on the
repository, they take several minutes to run (i.e., too long), and svn's
memory usage climbs to 150MB, which seems unusually high. This is not
acceptable performance in my application, and I am wondering if there is
some easy way to speed up operations on large repositories such as mine.
I have scanned the users@subversion mailing list archives for others'
reports about high memory usage and slow commands, and I do not find
any reports concerning 1.3.x.
I am beginning to suspect that a memory leak in APR 0.9.7 is behind the
high memory usage. Can anyone confirm? Do most people use APR 1.2 with
Subversion 1.3.1?
Dave
--
David Young OJC Technologies
dyoung@ojctech.com Urbana, IL * (217) 278-3933
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 5/4/06, David Young <dy...@pobox.com> wrote:
> It seems that the cache grows without bound, without regard for the
> potential of thrashing. It ceases to improve performance after a while.
Yep.
> What is the rule for kicking entries out of the cache? Can I reduce
> the cache size?
Nope. I don't believe there is a rule for kicking entries out of the
cache, largely due to the way memory management works with memory
pools.
> I am surprised that the cost of re-parsing is so great that it pays to
> keep a parsed entries object in the cache, especially when the OS holds
> the unparsed entries file in disk cache. I am probably just missing
> something.
Well, before the cache existed performance was quite a bit worse than
it is today. It's probably a combination of the work parsing the data
and the disk access (some of which may be cached by the OS).
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion client scalability (was Re: subversion 1.3.1 + apr 0.9.7 = memory leak?)
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 6/13/06, David Young <dy...@pobox.com> wrote:
> This email, bumping issue 2167 from milestone 1.4 to 1.5, crossed my desk
> a while ago. 2167 is an aging report on facets of the client scalability
> problem discussed in this thread. I was startled by the bump. Is there
> a consensus in the Subversion leadership that the client's scalability
> should be a lower priority than before?
No, there is a consensus among the developers who took it upon
themselves to do issue triage following the branching for the 1.4
release (i.e. lundblad) that this issue was not actually solved in the
1.4 release, that it was not significant enough to hold back the
release, so it was bumped to 1.5. This is at exactly the same
priority it was at before, it's a bug, and should be fixed, but nobody
is currently working on it.
Personally, I think he was right. It would certainly be nice to solve
this problem, but it's possible to avoid it by working with smaller
trees, and nobody who needs to use larger trees has stepped up and
solved the problem.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion client scalability (was Re: subversion 1.3.1 + apr 0.9.7 = memory leak?)
Posted by David Young <dy...@pobox.com>.
On Thu, May 04, 2006 at 02:34:11PM -0500, David Young wrote:
> On Wed, May 03, 2006 at 10:05:29PM -0700, Garrett Rooney wrote:
> > On 5/3/06, David Young <dy...@pobox.com> wrote:
> > >I have installed Subversion 1.3.1 and APR 0.9.7 from pkgsrc on a NetBSD
> > >3.99.7 box. I created a fresh fsfs database and svn import'd the
> > >entire NetBSD source tree.
> > >
> > >When I run commands such as 'svn co' and 'svn status -u' on the
> > >repository, they take several minutes to run (i.e., too long), and svn's
> > >memory usage climbs to 150MB, which seems unusually high. This is not
> > >acceptable performance in my application, and I am wondering if there is
> > >some easy way to speed up operations on large repositories such as mine.
> > >
> > >I have scanned the users@subversion mailing list archives for others'
> > >reports about high memory usage and slow commands, and I do not find
> > >any reports concerning 1.3.x.
> > >
> > >I am beginning to suspect that a memory leak in APR 0.9.7 is behind the
> > >high memory usage. Can anyone confirm? Do most people use APR 1.2 with
> > >Subversion 1.3.1?
> >
> > Operations on large working copies (including checkout and status)
> > will use amounts of memory proportional to its size, due to the
> > caching of information about the entries in each directory. It's
> > either that or you get unacceptable speed hits from parsing the
> > entries file multiple times. The version of APR you use will probably
> > have no noticable effect on memory usage, and Subversion 1.3.1 is not
> > especially worse in this regard than previous versions (although it
> > might be slightly worse due to a slightly larger entries object, I
> > can't recall).
>
> It seems that the cache grows without bound, without regard for the
> potential of thrashing. It ceases to improve performance after a while.
>
> What is the rule for kicking entries out of the cache? Can I reduce
> the cache size?
>
> I am surprised that the cost of re-parsing is so great that it pays to
> keep a parsed entries object in the cache, especially when the OS holds
> the unparsed entries file in disk cache. I am probably just missing
> something.
This email, bumping issue 2167 from milestone 1.4 to 1.5, crossed my desk
a while ago. 2167 is an aging report on facets of the client scalability
problem discussed in this thread. I was startled by the bump. Is there
a consensus in the Subversion leadership that the client's scalability
should be a lower priority than before?
Dave
***
Date: 21 May 2006 18:29:48 -0000
From: lundblad@tigris.org
To: shepard@tigris.org
Subject: [UTF-8] [Issue 2167] core dump while checking o[UTF-8] ut a copy
of large repository
http://subversion.tigris.org/issues/show_bug.cgi?id=2167
User lundblad changed the following:
What |Old value |New value
================================================================================
Target milestone|1.4 |1.5
--------------------------------------------------------------------------------
------- Additional comments from lundblad@tigris.org Sun May 21 11:29:48 -0700
2006 -------
We're using memory proportional to the working copy size, that's because we
cache the entries. I'm pushing this issue to 1.5; I think our out-of-memory
handler should say at least say something before aborting().
--
David Young OJC Technologies
dyoung@ojctech.com Urbana, IL * (217) 278-3933
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by David Young <dy...@pobox.com>.
On Wed, May 03, 2006 at 10:05:29PM -0700, Garrett Rooney wrote:
> On 5/3/06, David Young <dy...@pobox.com> wrote:
> >I have installed Subversion 1.3.1 and APR 0.9.7 from pkgsrc on a NetBSD
> >3.99.7 box. I created a fresh fsfs database and svn import'd the
> >entire NetBSD source tree.
> >
> >When I run commands such as 'svn co' and 'svn status -u' on the
> >repository, they take several minutes to run (i.e., too long), and svn's
> >memory usage climbs to 150MB, which seems unusually high. This is not
> >acceptable performance in my application, and I am wondering if there is
> >some easy way to speed up operations on large repositories such as mine.
> >
> >I have scanned the users@subversion mailing list archives for others'
> >reports about high memory usage and slow commands, and I do not find
> >any reports concerning 1.3.x.
> >
> >I am beginning to suspect that a memory leak in APR 0.9.7 is behind the
> >high memory usage. Can anyone confirm? Do most people use APR 1.2 with
> >Subversion 1.3.1?
>
> Operations on large working copies (including checkout and status)
> will use amounts of memory proportional to its size, due to the
> caching of information about the entries in each directory. It's
> either that or you get unacceptable speed hits from parsing the
> entries file multiple times. The version of APR you use will probably
> have no noticable effect on memory usage, and Subversion 1.3.1 is not
> especially worse in this regard than previous versions (although it
> might be slightly worse due to a slightly larger entries object, I
> can't recall).
It seems that the cache grows without bound, without regard for the
potential of thrashing. It ceases to improve performance after a while.
What is the rule for kicking entries out of the cache? Can I reduce
the cache size?
I am surprised that the cost of re-parsing is so great that it pays to
keep a parsed entries object in the cache, especially when the OS holds
the unparsed entries file in disk cache. I am probably just missing
something.
Dave
--
David Young OJC Technologies
dyoung@ojctech.com Urbana, IL * (217) 278-3933
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by Eric Gillespie <ep...@pretzelnet.org>.
"Garrett Rooney" <ro...@electricjellyfish.net> writes:
> On 5/4/06, Eric Gillespie <ep...@pretzelnet.org> wrote:
> > Hmm. Only the entries for the directory currently being scanned
> > and all its parents need be in memory at once. There should be a
> > subpool for each directory, i would think. If it's growing
> > without bound, it sounds to me like *all* entries are staying in
> > memory. In other words, given
> >
> > wc
> > wc/bin
> > wc/bin/ls
> > wc/bin/sh
> >
> > wc/bin/ls entries should be in a pool that is freed before moving
> > on to wc/bin/sh.
> >
> > > either that or you get unacceptable speed hits from parsing the
> > > entries file multiple times.
> >
> > Once svn is finished printing status for wc/bin/ls, it will never
> > need those entries again.
>
> Sure, but doing a pool per directory means you've just multiplied your
> memory usage for small and medium sized trees dramatically, since each
> pool allocates 8k of memory.
Hrm. The status quo, with svn checkout of a large trees using
unacceptably large amounts of memory (and apparently status -u
also), is unacceptable. If the only alternative is to waste 8K
for each subdirectory, it may be best to pay that cost. I'm not
sure these are the only choices. I have not yet looked into this
in any detail in over a year, when i whined and hoped someone
else would take care of it :).
> That also assumes that we never make multiple passes, and I'm
> not sure that's the case.
I can't imagine why we'd make a second pass. Once a file has
been checked out or had its status checked, we're done with it.
Once we've checked out or checked status of everything in a
subdirectory, we're done with that set of entries.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 5/4/06, Eric Gillespie <ep...@pretzelnet.org> wrote:
> "Garrett Rooney" <ro...@electricjellyfish.net> writes:
>
> > On 5/3/06, David Young <dy...@pobox.com> wrote:
> [...]
> > > When I run commands such as 'svn co' and 'svn status -u' on the
> > > repository, they take several minutes to run (i.e., too long), and svn's
> > > memory usage climbs to 150MB, which seems unusually high. This is not
> [...]
> > Operations on large working copies (including checkout and status)
> > will use amounts of memory proportional to its size, due to the
> > caching of information about the entries in each directory.
>
> Hmm. Only the entries for the directory currently being scanned
> and all its parents need be in memory at once. There should be a
> subpool for each directory, i would think. If it's growing
> without bound, it sounds to me like *all* entries are staying in
> memory. In other words, given
>
> wc
> wc/bin
> wc/bin/ls
> wc/bin/sh
>
> wc/bin/ls entries should be in a pool that is freed before moving
> on to wc/bin/sh.
>
> > either that or you get unacceptable speed hits from parsing the
> > entries file multiple times.
>
> Once svn is finished printing status for wc/bin/ls, it will never
> need those entries again.
Sure, but doing a pool per directory means you've just multiplied your
memory usage for small and medium sized trees dramatically, since each
pool allocates 8k of memory. That also assumes that we never make
multiple passes, and I'm not sure that's the case.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by Eric Gillespie <ep...@pretzelnet.org>.
David Young <dy...@pobox.com> writes:
> I am now reading the code, and I cannot find any indication that it isn't
> (supposed) to work just as you say. Is Subversion known to leak entries
> by design? Where does this caching occur?
Take a look at svn_wc_entries_read.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by David Young <dy...@pobox.com>.
On Thu, May 04, 2006 at 03:06:19PM -0700, Eric Gillespie wrote:
> "Garrett Rooney" <ro...@electricjellyfish.net> writes:
>
> > On 5/3/06, David Young <dy...@pobox.com> wrote:
> [...]
> > > When I run commands such as 'svn co' and 'svn status -u' on the
> > > repository, they take several minutes to run (i.e., too long), and svn's
> > > memory usage climbs to 150MB, which seems unusually high. This is not
> [...]
> > Operations on large working copies (including checkout and status)
> > will use amounts of memory proportional to its size, due to the
> > caching of information about the entries in each directory.
>
> Hmm. Only the entries for the directory currently being scanned
> and all its parents need be in memory at once. There should be a
> subpool for each directory, i would think. If it's growing
> without bound, it sounds to me like *all* entries are staying in
> memory. In other words, given
>
> wc
> wc/bin
> wc/bin/ls
> wc/bin/sh
>
> wc/bin/ls entries should be in a pool that is freed before moving
> on to wc/bin/sh.
I am now reading the code, and I cannot find any indication that it isn't
(supposed) to work just as you say. Is Subversion known to leak entries
by design? Where does this caching occur?
Dave
--
David Young OJC Technologies
dyoung@ojctech.com Urbana, IL * (217) 278-3933
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by Eric Gillespie <ep...@pretzelnet.org>.
"Garrett Rooney" <ro...@electricjellyfish.net> writes:
> On 5/3/06, David Young <dy...@pobox.com> wrote:
[...]
> > When I run commands such as 'svn co' and 'svn status -u' on the
> > repository, they take several minutes to run (i.e., too long), and svn's
> > memory usage climbs to 150MB, which seems unusually high. This is not
[...]
> Operations on large working copies (including checkout and status)
> will use amounts of memory proportional to its size, due to the
> caching of information about the entries in each directory.
Hmm. Only the entries for the directory currently being scanned
and all its parents need be in memory at once. There should be a
subpool for each directory, i would think. If it's growing
without bound, it sounds to me like *all* entries are staying in
memory. In other words, given
wc
wc/bin
wc/bin/ls
wc/bin/sh
wc/bin/ls entries should be in a pool that is freed before moving
on to wc/bin/sh.
> either that or you get unacceptable speed hits from parsing the
> entries file multiple times.
Once svn is finished printing status for wc/bin/ls, it will never
need those entries again.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion 1.3.1 + apr 0.9.7 = memory leak?
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 5/3/06, David Young <dy...@pobox.com> wrote:
> I have installed Subversion 1.3.1 and APR 0.9.7 from pkgsrc on a NetBSD
> 3.99.7 box. I created a fresh fsfs database and svn import'd the
> entire NetBSD source tree.
>
> When I run commands such as 'svn co' and 'svn status -u' on the
> repository, they take several minutes to run (i.e., too long), and svn's
> memory usage climbs to 150MB, which seems unusually high. This is not
> acceptable performance in my application, and I am wondering if there is
> some easy way to speed up operations on large repositories such as mine.
>
> I have scanned the users@subversion mailing list archives for others'
> reports about high memory usage and slow commands, and I do not find
> any reports concerning 1.3.x.
>
> I am beginning to suspect that a memory leak in APR 0.9.7 is behind the
> high memory usage. Can anyone confirm? Do most people use APR 1.2 with
> Subversion 1.3.1?
Operations on large working copies (including checkout and status)
will use amounts of memory proportional to its size, due to the
caching of information about the entries in each directory. It's
either that or you get unacceptable speed hits from parsing the
entries file multiple times. The version of APR you use will probably
have no noticable effect on memory usage, and Subversion 1.3.1 is not
especially worse in this regard than previous versions (although it
might be slightly worse due to a slightly larger entries object, I
can't recall).
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org