You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Chris Rose <ch...@messagingdirect.com> on 2008/02/24 22:42:07 UTC
mod_dav_svn performance
We've just completed a migration from CVS to SVN, with the new
repository hosted on the same machine, same filesystem, and provided
by way of mod_dav_svn to our users. What we're seeing is some nasty
divergence of performance; SVN is taking on average twice the time to
check out source. Is there any guidance available on improving the
performance of the server? We cannot justify throwing additional
hardware at the problem at this time, so a software solution would be
preferred.
Any suggestions?
Re: mod_dav_svn performance
Posted by da...@jpmorgan.com.
Big win which I found was that our Apache server had a default of
"KeepAlive off".
Changing this to KeepAlive on improved performance considerably.
http://httpd.apache.org/docs/2.0/mod/core.html#keepalive
"In some cases this has been shown to result in an almost 50% speedup in
latency times for HTML documents with many images."
Dg.
--
David Grierson
JPMorgan - IB Architecture - Source Code Management Consultant
GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson@jpmorgan.com
Alhambra House 6th floor, 45 Waterloo Street, Glasgow G2 6HS
John Waycott <ja...@cox.net>
25/02/2008 14:45
To
cc
users@subversion.tigris.org
Subject
Re: mod_dav_svn performance
Chris Rose wrote:
> We've just completed a migration from CVS to SVN, with the new
> repository hosted on the same machine, same filesystem, and provided
> by way of mod_dav_svn to our users. What we're seeing is some nasty
> divergence of performance; SVN is taking on average twice the time to
> check out source. Is there any guidance available on improving the
> performance of the server? We cannot justify throwing additional
> hardware at the problem at this time, so a software solution would be
> preferred.
>
> Any suggestions?
>
If your clients have low-latency connections to the server, turning on
http compression (mod deflate) in the Apache server may help. We found
that it reduced our checkout times about 20%.
We also found virus scanning software can impact checkout times. We
still leave it turned on because of our IT policy.
As others have stated, SVN will always be slower for checkout than CVS,
but that is not the common operation.
-- John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Generally, this communication is for informational purposes only
and it is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation
of any transaction. In the event you are receiving the offering
materials attached below related to your interest in hedge funds or
private equity, this communication may be intended as an offer or
solicitation for the purchase or sale of such fund(s). All market
prices, data and other information are not warranted as to
completeness or accuracy and are subject to change without notice.
Any comments or statements made herein do not necessarily reflect
those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to UK legal entities.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by John Waycott <ja...@cox.net>.
Chris Rose wrote:
> We've just completed a migration from CVS to SVN, with the new
> repository hosted on the same machine, same filesystem, and provided
> by way of mod_dav_svn to our users. What we're seeing is some nasty
> divergence of performance; SVN is taking on average twice the time to
> check out source. Is there any guidance available on improving the
> performance of the server? We cannot justify throwing additional
> hardware at the problem at this time, so a software solution would be
> preferred.
>
> Any suggestions?
>
If your clients have low-latency connections to the server, turning on
http compression (mod deflate) in the Apache server may help. We found
that it reduced our checkout times about 20%.
We also found virus scanning software can impact checkout times. We
still leave it turned on because of our IT policy.
As others have stated, SVN will always be slower for checkout than CVS,
but that is not the common operation.
-- John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by Toby Thain <to...@telegraphics.com.au>.
On 25-Feb-08, at 1:02 AM, Chris Rose wrote:
> I suppose this is true, but the wide range of client environments
> in which I've seen this suggests something else. Probably
> network. And, I assume, svn only sends one copy over the wire :)
...which would again point to the client as the bottleneck?
—Toby
>
> It might be a latency issue in the network; we've got a few other
> issues like that. Given the higher count of roundtrips that would
> make sense.
>
> Thanks, guys!
>
> On 24-Feb-08, at 6:37 PM, Mark Phippard wrote:
>
>> On Sun, Feb 24, 2008 at 7:59 PM, Matt Sickler
>> <cr...@gmail.com> wrote:
>>>
>>> On Sun, Feb 24, 2008 at 6:27 PM, Mark Phippard
>>> <ma...@gmail.com> wrote:
>>>
>>>
>>>> On Sun, Feb 24, 2008 at 7:11 PM, Chris Rose
>>>>
>>>> <ch...@messagingdirect.com> wrote:
>>>>
>>>>> Waittaminute... _client_ IO is the issue? That doesn't make
>>>>> sense;
>>> we're
>>>>> using a variety of clients ranging from modern Windows and
>>>>> Linux laptop
>>> and
>>>>> Desktop machines to Solaris 8 build servers, and they're all
>>>>> slow; I'm
>>> not
>>>>> discounting it, but it seems to me that the problem in our case
>>>>> might be
>>>>> more centralized.
>>>>>
>>>>> That said, I'll look into the server-side I/O issues, too. The
>>>>> machine
>>>>> hosting the repository is using RAID and some kind of high-end
>>>>> storage
>>> that
>>>>> I don't directly deal with; I'll pester our sysadmin to look
>>>>> into it.
>>>>>
>>>>> Thanks for the input. Anyone else? :)
>>>>
>>>> Checkout is probably the worst operation to benchmark. Subversion
>>>> writes out twice as much client-side data as CVS, so unless your
>>>> network or server is a big bottleneck, it would be expected for
>>>> checkout to be much slower than CVS. Also, how often are you doing
>>>> checkout, compared to update and commit. These are the operations
>>>> where Subversion shines, not to mention the client-only features
>>>> like
>>>> diff and revert.
>>>>
>>>> There is nothing you can do to make checkout in Subversion as
>>>> fast as
>>>> CVS. As Erik pointed out, using the ra_serf library might offer a
>>>> small boost. Using svnserve instead of mod_dav_svn offers a
>>>> definite
>>>> boost. In either case, it should still be slower than CVS though.
>>>
>>> why would the network have to transfer any more data than CVS?
>>> Wasn't SVN
>>> designed to use the network most effectively?
>>
>> SVN probably sends less data over the network than CVS, although
>> perhaps not when using HTTP. What I was saying was that unless you
>> have a really slow network connection, then the client side
>> processing
>> is going to be the biggest determinant of performance. IOW, I was
>> trying to answer your question as to why it is the client that would
>> matter for performance.
>>
>> --
>> Thanks
>>
>> Mark Phippard
>> http://markphip.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: users-help@subversion.tigris.org
>>
>
> --
> Chris Rose
> Developer Planet Consulting Group
> (780) 577-8433
> crose@planetci.com
>
Re: mod_dav_svn performance
Posted by Tony Butt <tj...@cea.com.au>.
On Mon, 2008-02-25 at 11:04 +0100, Erik Huelsmann wrote:
> On 2/25/08, Chris Rose <ch...@messagingdirect.com> wrote:
> > I suppose this is true, but the wide range of client environments in which
> > I've seen this suggests something else. Probably network. And, I assume,
> > svn only sends one copy over the wire :)
>
> I'm sorry, but you have 5 people telling you it's *not* the network
> and server, it's the working copy. I'm one of the people writing the
> working copy code and I'm telling you it's the working copy. Now you
> can keep telling us its not true, but as far as I understand *you* are
> the new user here.
>
> > It might be a latency issue in the network; we've got a few other issues
> > like that. Given the higher count of roundtrips that would make sense.
>
> That would make sense, especially if you use the HTTP protocol with
> the Neon library. You could try building the soon-to-be-released
> 1.5-alpha1 version with Serf library. That could tell you the impact
> of the higher number of connections: Serf uses the HTTP/1.1 protocol
> with request-pipelining in order to reduce the number of connections
> used.
>
> But, even though it will be faster, I expect the factor 2 to stay
> roughly the same: the working copy library still has to do twice the
> I/O CVS does.
>
>
> HTH,
>
>
> Erik.
>
For us, the real bottleneck is (as always) the path based authentication
done with the apache server. The server feels the need to authenticate
each and every file access request, and if your authorisation method is
not well tuned, it is VERY painful. We started with mod_auth_krb to
authenticate against an ADS server, and found that each file fetched on
a checkout required:
1) Fetch request
2) Name resolution for ADS server
3) Name resolution for Kerberos admin ( I think)
3) Several messages for Kerberos authentication (I forget how many, but
perhaps 4)
4) Finally, the file was delivered.
To check out (for example) our local copy of the ECos source tree would
take some time.
We tweaked the kerberos config files to use IP addresses instead of
names, and things improved.
The short answer is that still find the path based authentication on the
apache server to be the limiting factor.
We run svn+ssh access on the same machine, and it is about twice as fast
to the same client.
We desperately wish that the server side credentials could be cached for
a checkout (or update, or log, or whatever), as I am pretty sure that
would provide a big speedup.
Don't get me wrong, we love subversion, even if it is slower, and think
the developers are doing a great job. If you can speed up the http +
AuthZSVN stuff, that would be a big improvement for us.
Thanks,
Tony Butt
Senior Software Engineer
CEA Technologies
Canberra, Australia
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by Erik Huelsmann <eh...@gmail.com>.
On 2/25/08, Chris Rose <ch...@messagingdirect.com> wrote:
> I suppose this is true, but the wide range of client environments in which
> I've seen this suggests something else. Probably network. And, I assume,
> svn only sends one copy over the wire :)
I'm sorry, but you have 5 people telling you it's *not* the network
and server, it's the working copy. I'm one of the people writing the
working copy code and I'm telling you it's the working copy. Now you
can keep telling us its not true, but as far as I understand *you* are
the new user here.
> It might be a latency issue in the network; we've got a few other issues
> like that. Given the higher count of roundtrips that would make sense.
That would make sense, especially if you use the HTTP protocol with
the Neon library. You could try building the soon-to-be-released
1.5-alpha1 version with Serf library. That could tell you the impact
of the higher number of connections: Serf uses the HTTP/1.1 protocol
with request-pipelining in order to reduce the number of connections
used.
But, even though it will be faster, I expect the factor 2 to stay
roughly the same: the working copy library still has to do twice the
I/O CVS does.
HTH,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by Chris Rose <ch...@messagingdirect.com>.
I suppose this is true, but the wide range of client environments in
which I've seen this suggests something else. Probably network. And,
I assume, svn only sends one copy over the wire :)
It might be a latency issue in the network; we've got a few other
issues like that. Given the higher count of roundtrips that would
make sense.
Thanks, guys!
On 24-Feb-08, at 6:37 PM, Mark Phippard wrote:
> On Sun, Feb 24, 2008 at 7:59 PM, Matt Sickler
> <cr...@gmail.com> wrote:
>>
>> On Sun, Feb 24, 2008 at 6:27 PM, Mark Phippard <ma...@gmail.com>
>> wrote:
>>
>>
>>> On Sun, Feb 24, 2008 at 7:11 PM, Chris Rose
>>>
>>> <ch...@messagingdirect.com> wrote:
>>>
>>>> Waittaminute... _client_ IO is the issue? That doesn't make
>>>> sense;
>> we're
>>>> using a variety of clients ranging from modern Windows and Linux
>>>> laptop
>> and
>>>> Desktop machines to Solaris 8 build servers, and they're all
>>>> slow; I'm
>> not
>>>> discounting it, but it seems to me that the problem in our case
>>>> might be
>>>> more centralized.
>>>>
>>>> That said, I'll look into the server-side I/O issues, too. The
>>>> machine
>>>> hosting the repository is using RAID and some kind of high-end
>>>> storage
>> that
>>>> I don't directly deal with; I'll pester our sysadmin to look into
>>>> it.
>>>>
>>>> Thanks for the input. Anyone else? :)
>>>
>>> Checkout is probably the worst operation to benchmark. Subversion
>>> writes out twice as much client-side data as CVS, so unless your
>>> network or server is a big bottleneck, it would be expected for
>>> checkout to be much slower than CVS. Also, how often are you doing
>>> checkout, compared to update and commit. These are the operations
>>> where Subversion shines, not to mention the client-only features
>>> like
>>> diff and revert.
>>>
>>> There is nothing you can do to make checkout in Subversion as fast
>>> as
>>> CVS. As Erik pointed out, using the ra_serf library might offer a
>>> small boost. Using svnserve instead of mod_dav_svn offers a
>>> definite
>>> boost. In either case, it should still be slower than CVS though.
>>
>> why would the network have to transfer any more data than CVS?
>> Wasn't SVN
>> designed to use the network most effectively?
>
> SVN probably sends less data over the network than CVS, although
> perhaps not when using HTTP. What I was saying was that unless you
> have a really slow network connection, then the client side processing
> is going to be the biggest determinant of performance. IOW, I was
> trying to answer your question as to why it is the client that would
> matter for performance.
>
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
--
Chris Rose
Developer Planet Consulting Group
(780) 577-8433
crose@planetci.com
Re: mod_dav_svn performance
Posted by Mark Phippard <ma...@gmail.com>.
On Sun, Feb 24, 2008 at 7:59 PM, Matt Sickler
<cr...@gmail.com> wrote:
>
> On Sun, Feb 24, 2008 at 6:27 PM, Mark Phippard <ma...@gmail.com> wrote:
>
>
> > On Sun, Feb 24, 2008 at 7:11 PM, Chris Rose
> >
> > <ch...@messagingdirect.com> wrote:
> >
> > > Waittaminute... _client_ IO is the issue? That doesn't make sense;
> we're
> > > using a variety of clients ranging from modern Windows and Linux laptop
> and
> > > Desktop machines to Solaris 8 build servers, and they're all slow; I'm
> not
> > > discounting it, but it seems to me that the problem in our case might be
> > > more centralized.
> > >
> > > That said, I'll look into the server-side I/O issues, too. The machine
> > > hosting the repository is using RAID and some kind of high-end storage
> that
> > > I don't directly deal with; I'll pester our sysadmin to look into it.
> > >
> > > Thanks for the input. Anyone else? :)
> >
> > Checkout is probably the worst operation to benchmark. Subversion
> > writes out twice as much client-side data as CVS, so unless your
> > network or server is a big bottleneck, it would be expected for
> > checkout to be much slower than CVS. Also, how often are you doing
> > checkout, compared to update and commit. These are the operations
> > where Subversion shines, not to mention the client-only features like
> > diff and revert.
> >
> > There is nothing you can do to make checkout in Subversion as fast as
> > CVS. As Erik pointed out, using the ra_serf library might offer a
> > small boost. Using svnserve instead of mod_dav_svn offers a definite
> > boost. In either case, it should still be slower than CVS though.
>
> why would the network have to transfer any more data than CVS? Wasn't SVN
> designed to use the network most effectively?
SVN probably sends less data over the network than CVS, although
perhaps not when using HTTP. What I was saying was that unless you
have a really slow network connection, then the client side processing
is going to be the biggest determinant of performance. IOW, I was
trying to answer your question as to why it is the client that would
matter for performance.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by Mark Phippard <ma...@gmail.com>.
On Sun, Feb 24, 2008 at 7:11 PM, Chris Rose
<ch...@messagingdirect.com> wrote:
> Waittaminute... _client_ IO is the issue? That doesn't make sense; we're
> using a variety of clients ranging from modern Windows and Linux laptop and
> Desktop machines to Solaris 8 build servers, and they're all slow; I'm not
> discounting it, but it seems to me that the problem in our case might be
> more centralized.
>
> That said, I'll look into the server-side I/O issues, too. The machine
> hosting the repository is using RAID and some kind of high-end storage that
> I don't directly deal with; I'll pester our sysadmin to look into it.
>
> Thanks for the input. Anyone else? :)
Checkout is probably the worst operation to benchmark. Subversion
writes out twice as much client-side data as CVS, so unless your
network or server is a big bottleneck, it would be expected for
checkout to be much slower than CVS. Also, how often are you doing
checkout, compared to update and commit. These are the operations
where Subversion shines, not to mention the client-only features like
diff and revert.
There is nothing you can do to make checkout in Subversion as fast as
CVS. As Erik pointed out, using the ra_serf library might offer a
small boost. Using svnserve instead of mod_dav_svn offers a definite
boost. In either case, it should still be slower than CVS though.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by Les Mikesell <le...@gmail.com>.
Chris Rose wrote:
> Waittaminute... _client_ IO is the issue? That doesn't make sense;
A checkout of svn makes 2 copies of every file. You did say it took
twice as long as cvs didn't you? That shouldn't be a surprise.
--
Les Mikesell
lesmikesell@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: mod_dav_svn performance
Posted by Chris Rose <ch...@messagingdirect.com>.
Waittaminute... _client_ IO is the issue? That doesn't make sense;
we're using a variety of clients ranging from modern Windows and Linux
laptop and Desktop machines to Solaris 8 build servers, and they're
all slow; I'm not discounting it, but it seems to me that the problem
in our case might be more centralized.
That said, I'll look into the server-side I/O issues, too. The
machine hosting the repository is using RAID and some kind of high-end
storage that I don't directly deal with; I'll pester our sysadmin to
look into it.
Thanks for the input. Anyone else? :)
On 24-Feb-08, at 4:03 PM, Erik Huelsmann wrote:
> On Sun, Feb 24, 2008 at 11:42 AM, Chris Rose
> <ch...@messagingdirect.com> wrote:
>> We've just completed a migration from CVS to SVN, with the new
>> repository hosted on the same machine, same filesystem, and provided
>> by way of mod_dav_svn to our users. What we're seeing is some nasty
>> divergence of performance; SVN is taking on average twice the time
>> to
>> check out source. Is there any guidance available on improving the
>> performance of the server? We cannot justify throwing additional
>> hardware at the problem at this time, so a software solution would be
>> preferred.
>
> A faster hard disk (on the client) is what you can throw at it:
> Subversion is extremely I/O bound.
>
> Other than that: use 'checkout' as little as possible: getting
> secondary copies works great by making local copies of your working
> copy and then 'svn switch'-ing to the branch. This retrieves deltas
> instead of full files, updating only those files which have actually
> changed.
>
> HTH,
>
> Erik.
> PS: Lots of other operations should have become faster, but checkout
> is what builds the base to be faster and that takes extra time. You
> could investigate 1.5-alpha1 (out in a few days), which supports the
> Serf HTTP/1.1 library. We have observed interesting speedups with that
> library (when compared to the pre-1.5 non-Serf behaviour).
--
Chris Rose
Developer Planet Consulting Group
(780) 577-8433
crose@planetci.com
Re: mod_dav_svn performance
Posted by Erik Huelsmann <eh...@gmail.com>.
On Sun, Feb 24, 2008 at 11:42 AM, Chris Rose
<ch...@messagingdirect.com> wrote:
> We've just completed a migration from CVS to SVN, with the new
> repository hosted on the same machine, same filesystem, and provided
> by way of mod_dav_svn to our users. What we're seeing is some nasty
> divergence of performance; SVN is taking on average twice the time to
> check out source. Is there any guidance available on improving the
> performance of the server? We cannot justify throwing additional
> hardware at the problem at this time, so a software solution would be
> preferred.
A faster hard disk (on the client) is what you can throw at it:
Subversion is extremely I/O bound.
Other than that: use 'checkout' as little as possible: getting
secondary copies works great by making local copies of your working
copy and then 'svn switch'-ing to the branch. This retrieves deltas
instead of full files, updating only those files which have actually
changed.
HTH,
Erik.
PS: Lots of other operations should have become faster, but checkout
is what builds the base to be faster and that takes extra time. You
could investigate 1.5-alpha1 (out in a few days), which supports the
Serf HTTP/1.1 library. We have observed interesting speedups with that
library (when compared to the pre-1.5 non-Serf behaviour).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org