svnadmin dump/load patch, please test (was: Re: How long do your svn dumps take)

[Stefan Sperling <st...@elego.de>]

On Mon, Apr 20, 2009 at 02:39:15PM -0400, Bob Archer wrote:
> > > Once other datapoint for me. Our repository has a lot of binary
> files in
> > > it (VB6 dlls). I wonder if that could have something to do with it
> also.
> > 
> > Could be. But we can only make blind guesses unless we gather some
> > profiler output. It would be greatly appreciated.
> 
> As soon as someone can give me some direction on how to do this on
> Windows... I would be happy to run it again to get a profile. 

Kevin sent me output of a profiled run of svnadmin load.

Greg Stein and I discovered that we don't do any buffering when
reading from stdin and writing to stdout.

Attached is a patch against the 1.6.x branch, and should enable
buffered reads/writes for APR versions >= 1.3.0 during svnadmin
dump and svnadmin load.

If you have APR >= 1.3.0 in your environment, please test this
patch and let us know if you see an increase in performance.

Once this has been confirmed to increase performance, we can
implement our own buffering so that people running APR < 1.3.0 
can also benefit.

Thanks,
Stefan

RE: svnadmin dump/load patch, please test (was: Re: How long do yoursvn dumps take)

[Bob Archer <bo...@amsi.com>]

If anyone is willing to send me a patched windows binary of 1.6.1 I
would love to try this patch out.

BOb


> -----Original Message-----
> From: Stefan Sperling [mailto:stsp@elego.de]
> Sent: Tuesday, April 21, 2009 1:29 PM
> To: Bob Archer
> Cc: kmradke@rockwellcollins.com; users@subversion.tigris.org
> Subject: svnadmin dump/load patch, please test (was: Re: How long do
> yoursvn dumps take)
> 
> On Mon, Apr 20, 2009 at 02:39:15PM -0400, Bob Archer wrote:
> > > > Once other datapoint for me. Our repository has a lot of binary
> > files in
> > > > it (VB6 dlls). I wonder if that could have something to do with
it
> > also.
> > >
> > > Could be. But we can only make blind guesses unless we gather some
> > > profiler output. It would be greatly appreciated.
> >
> > As soon as someone can give me some direction on how to do this on
> > Windows... I would be happy to run it again to get a profile.
> 
> Kevin sent me output of a profiled run of svnadmin load.
> 
> Greg Stein and I discovered that we don't do any buffering when
> reading from stdin and writing to stdout.
> 
> Attached is a patch against the 1.6.x branch, and should enable
> buffered reads/writes for APR versions >= 1.3.0 during svnadmin
> dump and svnadmin load.
> 
> If you have APR >= 1.3.0 in your environment, please test this
> patch and let us know if you see an increase in performance.
> 
> Once this has been confirmed to increase performance, we can
> implement our own buffering so that people running APR < 1.3.0
> can also benefit.
> 
> Thanks,
> Stefan

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=1846403

To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].

Re: svnadmin dump/load patch, please test (was: Re: How long do your svn dumps take)

[Stefan Sperling <st...@elego.de>]

On Tue, Apr 21, 2009 at 06:28:42PM +0100, Stefan Sperling wrote:
> On Mon, Apr 20, 2009 at 02:39:15PM -0400, Bob Archer wrote:
> > > > Once other datapoint for me. Our repository has a lot of binary
> > files in
> > > > it (VB6 dlls). I wonder if that could have something to do with it
> > also.
> > > 
> > > Could be. But we can only make blind guesses unless we gather some
> > > profiler output. It would be greatly appreciated.
> > 
> > As soon as someone can give me some direction on how to do this on
> > Windows... I would be happy to run it again to get a profile. 
> 
> Kevin sent me output of a profiled run of svnadmin load.
> 
> Greg Stein and I discovered that we don't do any buffering when
> reading from stdin and writing to stdout.
> 
> Attached is a patch against the 1.6.x branch, and should enable
> buffered reads/writes for APR versions >= 1.3.0 during svnadmin
> dump and svnadmin load.
> 
> If you have APR >= 1.3.0 in your environment, please test this
> patch and let us know if you see an increase in performance.

I've tested my patch now on a copy of Subversion's own repository,
up to r8385. The result is disappointing, the patch does not make
any difference at all :(
Dumping and loading the repository takes around 15 minutes on my
laptop with and without the patch.

So it looks like we're back to square one.

Stefan