You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Eric Lemes <er...@gmail.com> on 2008/09/30 14:12:31 UTC
Subversion Performance
Hi Guys,
I'm used to be a colaborator of this mailing lists about a year ago, when I
did a subversion implementation in Windows. I stopped because I change jobs
and the new company had a CVS implementation with some customizations that
made the migration more complex.
Now, finally we did the CVS -> SVN migration sucessfully in a repository
with about 9 years old and 29259 revisions
But, after all this work I'm very disappointed with the performance of svn.
In the same machine of the old CVS it's so much slower. The machine isn't a
very bad one: Dual Quad Core Xeon (8x2.66Ghz) CPU, 16Gb RAM, with SAS hard
drives. The OS is Debian 4 (etch) Linux with Subversion 1.4.2 over apache
2.2.x (Debian etch default).
We're using TortoiseSVN as clients.
The main problem is server CPU hitting 100%. It happens everytime a checkout
is done and in show log operations. If I go to my "trunk" and do a show log,
the server gots 100% CPU for about 5-6 minutes even stopping serving other
requests.
I really want to know if there is any workarounds or optmizations that can
be done to solve this problem.
Thank you for your attention,
Eric Lemes
Re: Subversion Performance
Posted by Troy Curtis Jr <tr...@gmail.com>.
On Tue, Sep 30, 2008 at 11:30 AM, Les Mikesell <le...@gmail.com>wrote:
> Eric Lemes wrote:
>
>> It is just a guess, but maybe using 1.5 with sharding enabled will
>> improve your
>> performance. See:
>>
>> http://subversion.tigris.org/svn_1.5_releasenotes.html#fsfs-sharding
>>
>>
>> I did some investigation about this feature, and I think it will get
>> better I/O, but I/O isn't our problem.
>>
>> The machine has lot's of RAM and after some operations almost all the db
>> is cached. We can prove it with iostat and vmstat. Our filesystem is alread
>> mounted with noatime (it's our default).
>>
>> If there's any other information that I can send to help investigate the
>> problem, please, let me know.
>>
>
> I've seen other comments about svnserve being much faster than http but it
> may really depend on subtle items like your http authentication methods and
> logging. It might be worth wading through an strace of an httpd child
> process as you do some svn operations to see if anything unexpected is
> happening.
>
> --
> Les Mikesell
> lesmikesell@gmail.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
Yeah directory level authentication can really slow things down going
through Apache. Also, for our large repository our group had to move to the
BDB backend for operations to be bearable.
Hope it helps.
Troy
--
"Beware of spyware. If you can, use the Firefox browser." - USA Today
Download now at http://getfirefox.com
Registered Linux User #354814 ( http://counter.li.org/)
Re: Subversion Performance
Posted by Les Mikesell <le...@gmail.com>.
Eric Lemes wrote:
> It is just a guess, but maybe using 1.5 with sharding enabled will
> improve your
> performance. See:
>
> http://subversion.tigris.org/svn_1.5_releasenotes.html#fsfs-sharding
>
>
> I did some investigation about this feature, and I think it will get
> better I/O, but I/O isn't our problem.
>
> The machine has lot's of RAM and after some operations almost all the db
> is cached. We can prove it with iostat and vmstat. Our filesystem is
> alread mounted with noatime (it's our default).
>
> If there's any other information that I can send to help investigate the
> problem, please, let me know.
I've seen other comments about svnserve being much faster than http but
it may really depend on subtle items like your http authentication
methods and logging. It might be worth wading through an strace of an
httpd child process as you do some svn operations to see if anything
unexpected is happening.
--
Les Mikesell
lesmikesell@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion Performance
Posted by Eric Lemes <er...@gmail.com>.
>
> It is just a guess, but maybe using 1.5 with sharding enabled will improve
> your
> performance. See:
>
> http://subversion.tigris.org/svn_1.5_releasenotes.html#fsfs-sharding
>
I did some investigation about this feature, and I think it will get better
I/O, but I/O isn't our problem.
The machine has lot's of RAM and after some operations almost all the db is
cached. We can prove it with iostat and vmstat. Our filesystem is alread
mounted with noatime (it's our default).
If there's any other information that I can send to help investigate the
problem, please, let me know.
Thank you for all your help.
Eric Lemes
Re: Subversion Performance
Posted by Eric Lemes <er...@gmail.com>.
>
>
>
> It is just a guess, but maybe using 1.5 with sharding enabled will improve
> your
> performance. See:
>
> http://subversion.tigris.org/svn_1.5_releasenotes.html#fsfs-sharding
>
Thank you for this tip. We'll analyze that and try to dosome benchmark.
Eric
Re: Subversion Performance
Posted by Konstantin Kolinko <kn...@gmail.com>.
2008/9/30 Eric Lemes <er...@gmail.com>:
> Hi Guys,
> I'm used to be a colaborator of this mailing lists about a year ago, when I
> did a subversion implementation in Windows. I stopped because I change jobs
> and the new company had a CVS implementation with some customizations that
> made the migration more complex.
> Now, finally we did the CVS -> SVN migration sucessfully in a repository
> with about 9 years old and 29259 revisions
> But, after all this work I'm very disappointed with the performance of svn.
> In the same machine of the old CVS it's so much slower. The machine isn't a
> very bad one: Dual Quad Core Xeon (8x2.66Ghz) CPU, 16Gb RAM, with SAS hard
> drives. The OS is Debian 4 (etch) Linux with Subversion 1.4.2 over apache
> 2.2.x (Debian etch default).
> We're using TortoiseSVN as clients.
> The main problem is server CPU hitting 100%. It happens everytime a checkout
> is done and in show log operations. If I go to my "trunk" and do a show log,
> the server gots 100% CPU for about 5-6 minutes even stopping serving other
> requests.
> I really want to know if there is any workarounds or optmizations that can
> be done to solve this problem.
>
> Thank you for your attention,
> Eric Lemes
>
It is just a guess, but maybe using 1.5 with sharding enabled will improve your
performance. See:
http://subversion.tigris.org/svn_1.5_releasenotes.html#fsfs-sharding
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Subversion Performance
Posted by Tennebø Frode <fr...@saabgroup.com>.
> The main problem is server CPU hitting 100%. It happens
> everytime a checkout is done and in show log operations. If I
> go to my "trunk" and do a show log, the server gots 100% CPU
> for about 5-6 minutes even stopping serving other requests.
Without having a clue about your setup, I'll venture a guess. Others will probably give more specific SVN tips, but in my experience this sounds like it's related to IO. Several factors are involved; like raid-level, filesystem, svn database type, other activities on the server, etc. Check first which process' are contributing mostly to the load and also check the output from vmstat when this happens. You'd probably also want to mount the filesystem with 'noatime'.
-Frode
--
^ Frode Tennebø | email: Frode.Tennebo@saabgroup.com ^
| SAAB Microwave Systems AS | Isebakkeveien 49 |
| N-1788 Halden | Phone: +47 45 24 99 39 |
| with Standard.Disclaimer; use Standard.Disclaimer; |
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Subversion Performance
Posted by "Gleason, Todd" <tg...@impac.com>.
Was that a typo, or did you really just now migrate to Subversion 1.4.2
(instead of 1.5.2)?
________________________________
From: Eric Lemes [mailto:ericlemes@gmail.com]
Sent: Tuesday, September 30, 2008 8:13 AM
To: users@subversion.tigris.org
Subject: Subversion Performance
Hi Guys,
I'm used to be a colaborator of this mailing lists about a year ago,
when I did a subversion implementation in Windows. I stopped because I
change jobs and the new company had a CVS implementation with some
customizations that made the migration more complex.
Now, finally we did the CVS -> SVN migration sucessfully in a repository
with about 9 years old and 29259 revisions
But, after all this work I'm very disappointed with the performance of
svn. In the same machine of the old CVS it's so much slower. The machine
isn't a very bad one: Dual Quad Core Xeon (8x2.66Ghz) CPU, 16Gb RAM,
with SAS hard drives. The OS is Debian 4 (etch) Linux with Subversion
1.4.2 over apache 2.2.x (Debian etch default).
We're using TortoiseSVN as clients.
The main problem is server CPU hitting 100%. It happens everytime a
checkout is done and in show log operations. If I go to my "trunk" and
do a show log, the server gots 100% CPU for about 5-6 minutes even
stopping serving other requests.
I really want to know if there is any workarounds or optmizations that
can be done to solve this problem.
Thank you for your attention,
Eric Lemes
Re: Fwd: Subversion Performance
Posted by km...@rockwellcollins.com.
"Eric Lemes" <er...@gmail.com> wrote on 10/01/2008 11:15:28 AM:
> Do you have some revisions with hundreds of thousands of files? This
can
> cause some issues. We have repos that are 100G+ in size. Show logs are
> typically in the few seconds range for us...
>
> Hi Kevin,
>
> No, we don't have revisions with all these files. My repo has ~75k files
and < 2Gb.
>
> Can you tell me your setup to made some comparisons?
Dual CPU/dual core 2.8GHz AMD opteron 2218 64-bit server running Solaris.
Only 4G ram currently. Data storage is on SAN connected via multiple
fiber
connections. Network is load balanced onto multiple 1Gb connections.
We are typically I/O bound, but with the upgrade to 1.5 I have seen more
CPU usage. As a snapshot, today all 4 CPUs have been basically
idle (<10%) and I'm seeing about 100Mb/sec average sustained output
network traffic. (Peak is around 350Mb/sec)
Apache 2.2.8, Subversion 1.5.1, using mod_auth_ldap for authentication
to an Active Directory cluster.
This is handling about 3000 active users on 600 repositories using
around 1Tb of storage...
I've also not seen any performance problems under Windows 2003 server...
(Albeit with only ~1000 active users and 100 repos using ~250G)
Kevin R.
Re: Fwd: Subversion Performance
Posted by Eric Lemes <er...@gmail.com>.
>
> Do you have some revisions with hundreds of thousands of files? This can
> cause some issues. We have repos that are 100G+ in size. Show logs are
> typically in the few seconds range for us...
>
Hi Kevin,
No, we don't have revisions with all these files. My repo has ~75k files and
< 2Gb.
Can you tell me your setup to made some comparisons?
Thanks,
Eric
Re: Fwd: Subversion Performance
Posted by km...@rockwellcollins.com.
"Eric Lemes" <er...@gmail.com> wrote on 10/01/2008 06:40:52 AM:
> Thanks for your help.
>
> For authentication, we're using PAM. This night we made some tests
without
> any authentication, and the result is the same. The process that gots
100%
> CPU is apache2 (apache2 on debian, httpd in other distros, I think).
Do you have some revisions with hundreds of thousands of files? This can
cause some issues. We have repos that are 100G+ in size. Show logs are
typically in the few seconds range for us...
Kevin R.
> On Tue, Sep 30, 2008 at 6:49 PM, Francesco Nesci <fr...@nesci.org>
wrote:
> Eric,
>
> What are you using for authentication? A couple of months ago we
switched
> our Apache configuration from using mod_auth_pam to mod_auth_ldap and
were
> amazed at the increase in speed. We also noticed that PAM + locking
files
> caused performance issues that were either fixed or hidden by moving to
LDAP.
> If you do a top while the CPU is pegged, which processes are using the
most
> CPU? When we had performance issues, winbind was the hog.
>
> Francesco
>
Fwd: Subversion Performance
Posted by Eric Lemes <er...@gmail.com>.
Hi Francesco,
Thanks for your help.
For authentication, we're using PAM. This night we made some tests without
any authentication, and the result is the same. The process that gots 100%
CPU is apache2 (apache2 on debian, httpd in other distros, I think).
Eric.
On Tue, Sep 30, 2008 at 6:49 PM, Francesco Nesci <fr...@nesci.org> wrote:
> Eric,
>
> What are you using for authentication? A couple of months ago we switched
> our Apache configuration from using mod_auth_pam to mod_auth_ldap and were
> amazed at the increase in speed. We also noticed that PAM + locking files
> caused performance issues that were either fixed or hidden by moving to
> LDAP. If you do a top while the CPU is pegged, which processes are using
> the most CPU? When we had performance issues, winbind was the hog.
>
> Francesco
>
>
>