You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Alexis Huxley <ah...@gmx.net> on 2006/10/23 10:14:09 UTC
'svn co' stats: fsfs 50% faster than bdb?!?
I ran some loops checking out approx. 15000 files over a LAN from both
a BDB repository and an FSFS repository populated from the same dump,
using 1.4 client and server.
I also compared protocols svn://, svn+ssh://, http:// and https://.
The repository is on local mirrored SCSI disks. The working copy was
on local mirrored SCSI disks (on a different machine). The files are
C, C++, shell scripts, a few MS Word docs; part of the normal mix of
development files and support docs for a large project.
Here are the results from seven loops of checking out over four
different protocols from the two different repository formats:
ID=co-bdb-http COUNT=7 AVERAGE=1330.13
ID=co-bdb-https COUNT=7 AVERAGE=1364.61
ID=co-bdb-svn COUNT=7 AVERAGE=1153.71
ID=co-bdb-svn+ssh COUNT=7 AVERAGE=1183.68
ID=co-fsfs-http COUNT=7 AVERAGE= 741.70
ID=co-fsfs-https COUNT=7 AVERAGE= 779.40
ID=co-fsfs-svn COUNT=7 AVERAGE= 583.04
ID=co-fsfs-svn+ssh COUNT=7 AVERAGE= 644.61
That svn is fastest and https slowest was no surprise, but that FSFS
was faster than BDB was a great surprise and seems to go against
everything I've read; can anybody confirm or explain this?
For completeness, I also did some server side svnadmin operations
using a much smaller dump, this produced:
ID=create-bdb COUNT=88 AVERAGE= 1.02
ID=create-fsfs COUNT=89 AVERAGE= 0.11
ID=dump-bdb COUNT=88 AVERAGE= 66.67
ID=dump-fsfs COUNT=88 AVERAGE= 47.75
ID=hotcopy-bdb COUNT=88 AVERAGE= 2.83
ID=hotcopy-fsfs COUNT=88 AVERAGE= 2.30
ID=load-bdb COUNT=88 AVERAGE=2194.32
ID=load-fsfs COUNT=88 AVERAGE= 451.61
No surprises here: BDB (without the NOSYNC tweak) is a lot slower
loading than FSFS.
Any explanations of the co bdb/fsfs difference appreciated! Thanks!
Alexis Huxley
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: 'svn co' stats: fsfs 50% faster than bdb?!?
Posted by Alexis Huxley <ah...@gmx.net>.
On 2006-10-23, Thomas Harold <tg...@tgharold.com> wrote:
> What file system was used for the FSFS storage? I'd imagine that the #
> of folders in the db/revs folder could cause issues on some file systems?
It's ext3. There are ~6000 directories. The maximum number of entries in
any single directory is 214, with only about 25 directories containing
more than 50 files/subdirectories.
Alexis
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: 'svn co' stats: fsfs 50% faster than bdb?!?
Posted by Thomas Harold <tg...@tgharold.com>.
Alexis Huxley wrote:
> I ran some loops checking out approx. 15000 files over a LAN from both
> a BDB repository and an FSFS repository populated from the same dump,
> using 1.4 client and server.
>
> I also compared protocols svn://, svn+ssh://, http:// and https://.
>
> The repository is on local mirrored SCSI disks. The working copy was
> on local mirrored SCSI disks (on a different machine). The files are
> C, C++, shell scripts, a few MS Word docs; part of the normal mix of
> development files and support docs for a large project.
What file system was used for the FSFS storage? I'd imagine that the #
of folders in the db/revs folder could cause issues on some file systems?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: 'svn co' stats: fsfs 50% faster than bdb?!?
Posted by Alexis Huxley <ah...@gmx.net>.
>> ID=co-bdb-http COUNT=7 AVERAGE=1330.13
>> ID=co-bdb-https COUNT=7 AVERAGE=1364.61
>> ID=co-bdb-svn COUNT=7 AVERAGE=1153.71
>> ID=co-bdb-svn+ssh COUNT=7 AVERAGE=1183.68
>> ID=co-fsfs-http COUNT=7 AVERAGE= 741.70
>> ID=co-fsfs-https COUNT=7 AVERAGE= 779.40
>> ID=co-fsfs-svn COUNT=7 AVERAGE= 583.04
>> ID=co-fsfs-svn+ssh COUNT=7 AVERAGE= 644.61
>>
>> that FSFS
>> was faster than BDB was a great surprise
> You mentioned that you were using 15000 files but how many revs were
> there?
Just under 40000.
> I have found that the stats are exactly the opposite (fsfs is
> twice a long) for a large number of revs (~60000).
Indeed; this was exactly what I had expected; which is why I was
flumoxed by the above results.
Alexis
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: 'svn co' stats: fsfs 50% faster than bdb?!?
Posted by Troy Curtis Jr <tr...@gmail.com>.
On 10/23/06, Alexis Huxley <ah...@gmx.net> wrote:
> I ran some loops checking out approx. 15000 files over a LAN from both
> a BDB repository and an FSFS repository populated from the same dump,
> using 1.4 client and server.
>
> I also compared protocols svn://, svn+ssh://, http:// and https://.
>
> The repository is on local mirrored SCSI disks. The working copy was
> on local mirrored SCSI disks (on a different machine). The files are
> C, C++, shell scripts, a few MS Word docs; part of the normal mix of
> development files and support docs for a large project.
>
> Here are the results from seven loops of checking out over four
> different protocols from the two different repository formats:
>
> ID=co-bdb-http COUNT=7 AVERAGE=1330.13
> ID=co-bdb-https COUNT=7 AVERAGE=1364.61
> ID=co-bdb-svn COUNT=7 AVERAGE=1153.71
> ID=co-bdb-svn+ssh COUNT=7 AVERAGE=1183.68
> ID=co-fsfs-http COUNT=7 AVERAGE= 741.70
> ID=co-fsfs-https COUNT=7 AVERAGE= 779.40
> ID=co-fsfs-svn COUNT=7 AVERAGE= 583.04
> ID=co-fsfs-svn+ssh COUNT=7 AVERAGE= 644.61
>
> That svn is fastest and https slowest was no surprise, but that FSFS
> was faster than BDB was a great surprise and seems to go against
> everything I've read; can anybody confirm or explain this?
>
> For completeness, I also did some server side svnadmin operations
> using a much smaller dump, this produced:
>
> ID=create-bdb COUNT=88 AVERAGE= 1.02
> ID=create-fsfs COUNT=89 AVERAGE= 0.11
> ID=dump-bdb COUNT=88 AVERAGE= 66.67
> ID=dump-fsfs COUNT=88 AVERAGE= 47.75
> ID=hotcopy-bdb COUNT=88 AVERAGE= 2.83
> ID=hotcopy-fsfs COUNT=88 AVERAGE= 2.30
> ID=load-bdb COUNT=88 AVERAGE=2194.32
> ID=load-fsfs COUNT=88 AVERAGE= 451.61
>
> No surprises here: BDB (without the NOSYNC tweak) is a lot slower
> loading than FSFS.
>
> Any explanations of the co bdb/fsfs difference appreciated! Thanks!
>
> Alexis Huxley
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
You mentioned that you were using 15000 files but how many revs were
there? I have found that the stats are exactly the opposite (fsfs is
twice a long) for a large number of revs (~60000).
I can certainly confirm the quicker load time for fsfs. The large
dump that I just mentioned (the 60000 revs one) took over 10 hours to
load into a bdb repo and only 3 to load into a fsfs.
Thought you would like to have the information.
Troy
--
"Beware of spyware. If you can, use the Firefox browser." - USA Today
Download now at http://getfirefox.com
Registered Linux User #354814 ( http://counter.li.org/)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org