You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by David Young <dy...@pobox.com> on 2006/06/13 22:18:01 UTC
Re: subversion client scalability (was Re: subversion 1.3.1 + apr 0.9.7 = memory leak?)
On Thu, May 04, 2006 at 02:34:11PM -0500, David Young wrote:
> On Wed, May 03, 2006 at 10:05:29PM -0700, Garrett Rooney wrote:
> > On 5/3/06, David Young <dy...@pobox.com> wrote:
> > >I have installed Subversion 1.3.1 and APR 0.9.7 from pkgsrc on a NetBSD
> > >3.99.7 box. I created a fresh fsfs database and svn import'd the
> > >entire NetBSD source tree.
> > >
> > >When I run commands such as 'svn co' and 'svn status -u' on the
> > >repository, they take several minutes to run (i.e., too long), and svn's
> > >memory usage climbs to 150MB, which seems unusually high. This is not
> > >acceptable performance in my application, and I am wondering if there is
> > >some easy way to speed up operations on large repositories such as mine.
> > >
> > >I have scanned the users@subversion mailing list archives for others'
> > >reports about high memory usage and slow commands, and I do not find
> > >any reports concerning 1.3.x.
> > >
> > >I am beginning to suspect that a memory leak in APR 0.9.7 is behind the
> > >high memory usage. Can anyone confirm? Do most people use APR 1.2 with
> > >Subversion 1.3.1?
> >
> > Operations on large working copies (including checkout and status)
> > will use amounts of memory proportional to its size, due to the
> > caching of information about the entries in each directory. It's
> > either that or you get unacceptable speed hits from parsing the
> > entries file multiple times. The version of APR you use will probably
> > have no noticable effect on memory usage, and Subversion 1.3.1 is not
> > especially worse in this regard than previous versions (although it
> > might be slightly worse due to a slightly larger entries object, I
> > can't recall).
>
> It seems that the cache grows without bound, without regard for the
> potential of thrashing. It ceases to improve performance after a while.
>
> What is the rule for kicking entries out of the cache? Can I reduce
> the cache size?
>
> I am surprised that the cost of re-parsing is so great that it pays to
> keep a parsed entries object in the cache, especially when the OS holds
> the unparsed entries file in disk cache. I am probably just missing
> something.
This email, bumping issue 2167 from milestone 1.4 to 1.5, crossed my desk
a while ago. 2167 is an aging report on facets of the client scalability
problem discussed in this thread. I was startled by the bump. Is there
a consensus in the Subversion leadership that the client's scalability
should be a lower priority than before?
Dave
***
Date: 21 May 2006 18:29:48 -0000
From: lundblad@tigris.org
To: shepard@tigris.org
Subject: [UTF-8] [Issue 2167] core dump while checking o[UTF-8] ut a copy
of large repository
http://subversion.tigris.org/issues/show_bug.cgi?id=2167
User lundblad changed the following:
What |Old value |New value
================================================================================
Target milestone|1.4 |1.5
--------------------------------------------------------------------------------
------- Additional comments from lundblad@tigris.org Sun May 21 11:29:48 -0700
2006 -------
We're using memory proportional to the working copy size, that's because we
cache the entries. I'm pushing this issue to 1.5; I think our out-of-memory
handler should say at least say something before aborting().
--
David Young OJC Technologies
dyoung@ojctech.com Urbana, IL * (217) 278-3933
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: subversion client scalability (was Re: subversion 1.3.1 + apr 0.9.7 = memory leak?)
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 6/13/06, David Young <dy...@pobox.com> wrote:
> This email, bumping issue 2167 from milestone 1.4 to 1.5, crossed my desk
> a while ago. 2167 is an aging report on facets of the client scalability
> problem discussed in this thread. I was startled by the bump. Is there
> a consensus in the Subversion leadership that the client's scalability
> should be a lower priority than before?
No, there is a consensus among the developers who took it upon
themselves to do issue triage following the branching for the 1.4
release (i.e. lundblad) that this issue was not actually solved in the
1.4 release, that it was not significant enough to hold back the
release, so it was bumped to 1.5. This is at exactly the same
priority it was at before, it's a bug, and should be fixed, but nobody
is currently working on it.
Personally, I think he was right. It would certainly be nice to solve
this problem, but it's possible to avoid it by working with smaller
trees, and nobody who needs to use larger trees has stepped up and
solved the problem.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org