You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stefan Fuhrmann <st...@wandisco.com> on 2013/02/17 02:39:23 UTC
Re: BDB vs FSFS - OMG!
On Mon, Jan 7, 2013 at 5:02 PM, C. Michael Pilato <cm...@collab.net>wrote:
>
> If we're going to force existing BDB users onto something else, they should
> expect real benefit from that change. I don't want their decision-making
> to
> boil down to choosing amongst a) enduring a dump/load process for no
> perceivable benefit, b) sticking with an old Subversion indefinitely, or c)
> just bailing to another VC system. And while the kinds of performance
> numbers folks have thrown up comparing the backends are interesting, I
> guess
> I've not yet seen a case made for FS performance being a serious bottleneck
> in real-world deployment scenarios (where networks, SSL, server load, etc.
> are involved). Is it the case that 'svn update' or 'svn merge' over HTTP
> from the moderately active Subversion server sitting in the corporate data
> center is noticeably slower with one backend versus the other?
>
Building my private "data center" is scheduled for Q2/13.
But for now, I did some testing of BDB vs. FSFS over HTTP
with real working copies, update and merge operations. The
bottom line is that we can confidently say that a fsfs setup
is "twice as fast where it hurts you" (with a greatly varying
results for individual operations).
This time, I ran tests against the repo from wesnoth.org (56112
revs, trunk is 600+MB). Here are the results:
test BDB / FSFS = rel. performance
(1) 115.8s / 45.9s = 1.00 : 2.52
(2) 1.21s / 1.64s = 1.00 : 0.74
(3) 50.3s / 28.3s = 1.00 : 1.78
(4) 22.7s / 19.5s = 1.00 : 1.16
(5) 2.38s / 0.52s = 1.00 : 4.58
(6) 10.79s / 0.81s = 1.00 :13.32
(7) 211.4s /110.0s = 1.00 : 1.92
Commands executed:
(1) $ time svn co
http://localhost/svn/wesnoth-[fsfs|bdb]/trunk~/wesnoth-[fsfs|bdb] -q
size info from null-export
866 directories
16,560 files
631,655,136 bytes in files
105,112 properties
2,856,705 bytes in properties
(2) $ time svn up . -q
empty update
(3) $ time svn up -r55000 . -q
3951 changed nodes
(4) $ time svn up . -q
reverse operation
(5) $ time svn log -v --limit 100 ./src
100 revs, 295 message lines, 1410 changes
(6) $ time svn mergeinfo ^/branches/editor --show-revs eligible
53 revisions
(7) $ time svn merge ^/branches/editor --accept p
2917 changed nodes, 1781 text conflicts, 541 tree conflicts
Apache was restarted after each operation. The machine
was rebooted after each test (pair of operations).
Tests have been conducted with maximum optimization:
./configure --enable-shared --disable-debug --enable-optimize
--with-berkeley-db=db.h:/usr/include:/usr/local/lib:db-5.1
--with-serf=/usr/local CUSERFLAGS='-march=native'
DB_CONFIG for BDB repo (~1GB cache; got same results for 10MB though)
set_cachesize 0 1048576000 1
Apache configuration (~1GB cache, mpm-worker):
SVNInMemoryCacheSize 1000000
SVNCacheFullTexts On
SVNCacheTextDeltas On
SVNCacheRevProps On
SVNCompressionLevel 0
<Location /svn/wesnoth-bdb>
DAV svn
SVNPath /home/stefan/develop/wesnoth-bdb
SVNAdvertiseV2Protocol On
</Location>
<Location /svn/wesnoth-fsfs>
DAV svn
SVNPath /home/stefan/develop/wesnoth
SVNAdvertiseV2Protocol On
</Location>
Machine:
Core2 Duo 2.4Ghz, 8GB RAM, Ubuntu 12.04, 64 bit, SMP
128GB SSD built-in, ext4
-- Stefan^2.
--
Certified & Supported Apache Subversion Downloads:
*
http://www.wandisco.com/subversion/download
*
Re: BDB vs FSFS - OMG!
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Fuhrmann <st...@wandisco.com> writes:
> On Mon, Feb 18, 2013 at 11:04 AM, Philip Martin
> <ph...@wandisco.com>wrote:
>
>> Stefan Fuhrmann <st...@wandisco.com> writes:
>>
>> > Apache configuration (~1GB cache, mpm-worker):
>> >
>> > SVNInMemoryCacheSize 1000000
>> > SVNCacheFullTexts On
>> > SVNCacheTextDeltas On
>> > SVNCacheRevProps On
>> > SVNCompressionLevel 0
>>
>> I see no mention of tuning the BDB repository.
>
>
> Just scroll up by 3 lines ;)
>
> I set the BDB cache to 1GB as well but that did not have
> a measurable effect. Also tried 10MB in case that merely
> initializing a large cache comes at an extra cost (results
> did not change, though).
oops, sorry!
>> There are several things
>> you can tweak in db/DB_CONFIG including cache size, log size, number of
>> locks, etc? Cache size in particular needs to be set:
>>
>>
>> http://docs.oracle.com/cd/E17076_02/html/programmer_reference/general_am_conf.html#am_conf_cachesize
>>
> I did not change log & lock settings because I ran r/o
> operations only.
Subversion read operations on BDB repositories use BDB transactions.
My guess would be that Subversion's default locking limits are OK
because I'm assuming your test accesses the repository with one client
at a time.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: BDB vs FSFS - OMG!
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Mon, Feb 18, 2013 at 11:04 AM, Philip Martin
<ph...@wandisco.com>wrote:
> Stefan Fuhrmann <st...@wandisco.com> writes:
>
> > Apache configuration (~1GB cache, mpm-worker):
> >
> > SVNInMemoryCacheSize 1000000
> > SVNCacheFullTexts On
> > SVNCacheTextDeltas On
> > SVNCacheRevProps On
> > SVNCompressionLevel 0
>
> I see no mention of tuning the BDB repository.
Just scroll up by 3 lines ;)
I set the BDB cache to 1GB as well but that did not have
a measurable effect. Also tried 10MB in case that merely
initializing a large cache comes at an extra cost (results
did not change, though).
> There are several things
> you can tweak in db/DB_CONFIG including cache size, log size, number of
> locks, etc? Cache size in particular needs to be set:
>
>
> http://docs.oracle.com/cd/E17076_02/html/programmer_reference/general_am_conf.html#am_conf_cachesize
>
> I did not change log & lock settings because I ran r/o
operations only.
-- Stefan^2.
--
Certified & Supported Apache Subversion Downloads:
*
http://www.wandisco.com/subversion/download
*
Re: BDB vs FSFS - OMG!
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Fuhrmann <st...@wandisco.com> writes:
> Apache configuration (~1GB cache, mpm-worker):
>
> SVNInMemoryCacheSize 1000000
> SVNCacheFullTexts On
> SVNCacheTextDeltas On
> SVNCacheRevProps On
> SVNCompressionLevel 0
I see no mention of tuning the BDB repository. There are several things
you can tweak in db/DB_CONFIG including cache size, log size, number of
locks, etc? Cache size in particular needs to be set:
http://docs.oracle.com/cd/E17076_02/html/programmer_reference/general_am_conf.html#am_conf_cachesize
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download