svn resource requirements with large repositories?

[Chase Phillips <sh...@ameth.org>]

Recently our software development team has tried to create and use a
Subversion repository for working on NetBSD source code with Subversion
1.0.5, 1.1.0, and 1.1.1.  In each of these, basic operations fail for what
appears to be process resource requirements.  The processes grow so large
that they are killed by our server's OS.

While the initial import is of a directory that is nearly 800 MB, I'm told
that the server has more than 512 MB of RAM.  When the process is killed
it's apparently from bumping its head at having a memory image ~ 128 MB.
So far, we've experienced this type of failure in 'import', 'commit', and
'cleanup' operations (in 1.1.0, the import went well, though commit and
cleanup later failed running out of resources).  We've been in touch with
a member of the Subversion team about the problems we've experienced and
hope to get time soon to try two patches that may resolve these resource
issues in large part.

Running into this problem has made me curious who on the list is using
Subversion to manage "large" repositories and what experiences you've had
doing so.  Do you find Subversion works just as well on large repositories
as it does small repositories on your system?  Also, what OS, server
hardware, server settings, Subversion version, and back-end is recommended
for operating on large repositories.  (After researching, we've assumed
fsfs is the best back-end due to wedgeless operation, though it's new and
we have the process resource problems in the only versions where it's
supported.)

PS I reviewed the FAQ and propaganda listing prior to posting and didn't
find anything which directly addresses this issue.  Apologies if there's
already documentation on it.  If you could point me to it, I'd appreciate
it.

Thanks,
Chase Phillips
--
Developer, Champaign-Urbana Community Wireless Network

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn resource requirements with large repositories?

[Chase Phillips <sh...@ameth.org>]

On Thu, 4 Nov 2004, Garrett Rooney wrote:

> Chase Phillips wrote:
> > Recently our software development team has tried to create and use a
> > Subversion repository for working on NetBSD source code with Subversion
> > 1.0.5, 1.1.0, and 1.1.1.  In each of these, basic operations fail for what
> > appears to be process resource requirements.  The processes grow so large
> > that they are killed by our server's OS.
>
> If you could retry with a version of Subversion built from the current
> trunk of the repository that would be useful.  There have been some
> memory leaks fixed recently that might allow your import to succeed.

I guess Eric's fixes have been integrated.  Great!  We'll try that instead
of applying just his two patches to 1.1.0 or 1.1.1.

> In the 1.1.0 and 1.1.1 cases were you using Berkeley DB based
> repositories or libsvn_fs_fs based repositories?

We always used fsfs in 1.1.x.  We'd prefer fsfs if possible.

Chase

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: svn resource requirements with large repositories?

[Garrett Rooney <ro...@electricjellyfish.net>]

Chase Phillips wrote:
> Recently our software development team has tried to create and use a
> Subversion repository for working on NetBSD source code with Subversion
> 1.0.5, 1.1.0, and 1.1.1.  In each of these, basic operations fail for what
> appears to be process resource requirements.  The processes grow so large
> that they are killed by our server's OS.

If you could retry with a version of Subversion built from the current 
trunk of the repository that would be useful.  There have been some 
memory leaks fixed recently that might allow your import to succeed.

In the 1.1.0 and 1.1.1 cases were you using Berkeley DB based 
repositories or libsvn_fs_fs based repositories?

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org