You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Ben Smith-Mannschott <bs...@gmail.com> on 2011/09/12 15:44:20 UTC
an observation regarding FSFS performance on BTRFS
I've made the observation that FSFS repositories perform better on
EXT4 than BTRFS. This probably isn't ground-breaking, but I thought
I'd share it.
I've got two Linux machines:
- colossus, using BTRFS spanned over two disks.
2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011
x86_64 x86_64 x86_64 GNU/Linux
- oberon, using EXT4 on a 2-disk software RAID-1 set.
Linux oberon 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011
x86_64 GNU/Linux
I've noticed that writes to FSFS repositories are 5x faster under EXT4
than BTRFS. When svnsyncing form the same svn:// source to an local
repository (file://), oberon completes about 400 revisions in the time
it takes colossus to grind through 80.
The BTRFS machine is our build server. Performance with (1.6.x)
working copies is quite acceptable, but I'm glad I'm not using it to
host svn repositories.
Looks like the BTRFS people have some work to do. Maybe current
Kernels have already improved this picture. I know there has been
recent work on reducing the cost of meta-data operations (e.g. file
creation, ...) and that work is ongoing on defragmentation
functionality because of poor performance on files that are modified
in place heavily (e.g. sqlite).
// ben
Re: an observation regarding FSFS performance on BTRFS
Posted by Ben Smith-Mannschott <bs...@gmail.com>.
Out of curiosity, I built the 1.7.0 rc3 that's currently in
preparation and tried it on both systems. The EXT4 system managed 200
revisions in the time it took the BTRFS system to complete 70, so the
EXT4 system is about 3x faster for this particular test. It appears as
if the server-side changes in 1.7.0 have reduced the performance
penalty of using BTRFS in place of EXT4 for an FSF repository.
// ben
On Mon, Sep 12, 2011 at 18:07, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> 1.7 includes performance optimizations by Stefan2, new cache modules,
> new cache users, etc.
>
> branches/performance includes, among other things, a file handles cache
> for FSFS. The plan is to merge that for 1.8.
>
> branches/revprop-packing packs revprops into flat files (not to sqlite).
> A basic form currently works and the final form will be included in 1.8.
>
> Ben Smith-Mannschott wrote on Mon, Sep 12, 2011 at 17:28:28 +0200:
>> I'm using 1.6.x. I wasn't aware that there'd been sufficient
>> server-side work in 1.7.x as to make this distinction important.
>>
>> // ben
>>
>> On Mon, Sep 12, 2011 at 16:12, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>> > You haven't mentioned what version of svn you use. As you say, there
>> > has been work recently --- some of it is in 1.7, some of it is on
>> > ^/subversion/branches/performance, some of it is on
>> > ^/subversion/branches/revprop-packing, and some additional ideas
>> > are in notes/fsfs-improvements.txt in trunk.
>> >
>> > Ben Smith-Mannschott wrote on Mon, Sep 12, 2011 at 15:44:20 +0200:
>> >> I've made the observation that FSFS repositories perform better on
>> >> EXT4 than BTRFS. This probably isn't ground-breaking, but I thought
>> >> I'd share it.
>> >>
>> >> I've got two Linux machines:
>> >>
>> >> - colossus, using BTRFS spanned over two disks.
>> >> 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011
>> >> x86_64 x86_64 x86_64 GNU/Linux
>> >>
>> >> - oberon, using EXT4 on a 2-disk software RAID-1 set.
>> >> Linux oberon 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011
>> >> x86_64 GNU/Linux
>> >>
>> >> I've noticed that writes to FSFS repositories are 5x faster under EXT4
>> >> than BTRFS. When svnsyncing form the same svn:// source to an local
>> >> repository (file://), oberon completes about 400 revisions in the time
>> >> it takes colossus to grind through 80.
>> >>
>> >> The BTRFS machine is our build server. Performance with (1.6.x)
>> >> working copies is quite acceptable, but I'm glad I'm not using it to
>> >> host svn repositories.
>> >>
>> >> Looks like the BTRFS people have some work to do. Maybe current
>> >> Kernels have already improved this picture. I know there has been
>> >> recent work on reducing the cost of meta-data operations (e.g. file
>> >> creation, ...) and that work is ongoing on defragmentation
>> >> functionality because of poor performance on files that are modified
>> >> in place heavily (e.g. sqlite).
>> >>
>> >> // ben
>> >
>
Re: an observation regarding FSFS performance on BTRFS
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
1.7 includes performance optimizations by Stefan2, new cache modules,
new cache users, etc.
branches/performance includes, among other things, a file handles cache
for FSFS. The plan is to merge that for 1.8.
branches/revprop-packing packs revprops into flat files (not to sqlite).
A basic form currently works and the final form will be included in 1.8.
Ben Smith-Mannschott wrote on Mon, Sep 12, 2011 at 17:28:28 +0200:
> I'm using 1.6.x. I wasn't aware that there'd been sufficient
> server-side work in 1.7.x as to make this distinction important.
>
> // ben
>
> On Mon, Sep 12, 2011 at 16:12, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > You haven't mentioned what version of svn you use. As you say, there
> > has been work recently --- some of it is in 1.7, some of it is on
> > ^/subversion/branches/performance, some of it is on
> > ^/subversion/branches/revprop-packing, and some additional ideas
> > are in notes/fsfs-improvements.txt in trunk.
> >
> > Ben Smith-Mannschott wrote on Mon, Sep 12, 2011 at 15:44:20 +0200:
> >> I've made the observation that FSFS repositories perform better on
> >> EXT4 than BTRFS. This probably isn't ground-breaking, but I thought
> >> I'd share it.
> >>
> >> I've got two Linux machines:
> >>
> >> - colossus, using BTRFS spanned over two disks.
> >> 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011
> >> x86_64 x86_64 x86_64 GNU/Linux
> >>
> >> - oberon, using EXT4 on a 2-disk software RAID-1 set.
> >> Linux oberon 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011
> >> x86_64 GNU/Linux
> >>
> >> I've noticed that writes to FSFS repositories are 5x faster under EXT4
> >> than BTRFS. When svnsyncing form the same svn:// source to an local
> >> repository (file://), oberon completes about 400 revisions in the time
> >> it takes colossus to grind through 80.
> >>
> >> The BTRFS machine is our build server. Performance with (1.6.x)
> >> working copies is quite acceptable, but I'm glad I'm not using it to
> >> host svn repositories.
> >>
> >> Looks like the BTRFS people have some work to do. Maybe current
> >> Kernels have already improved this picture. I know there has been
> >> recent work on reducing the cost of meta-data operations (e.g. file
> >> creation, ...) and that work is ongoing on defragmentation
> >> functionality because of poor performance on files that are modified
> >> in place heavily (e.g. sqlite).
> >>
> >> // ben
> >
Re: an observation regarding FSFS performance on BTRFS
Posted by Ben Smith-Mannschott <bs...@gmail.com>.
I'm using 1.6.x. I wasn't aware that there'd been sufficient
server-side work in 1.7.x as to make this distinction important.
// ben
On Mon, Sep 12, 2011 at 16:12, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> You haven't mentioned what version of svn you use. As you say, there
> has been work recently --- some of it is in 1.7, some of it is on
> ^/subversion/branches/performance, some of it is on
> ^/subversion/branches/revprop-packing, and some additional ideas
> are in notes/fsfs-improvements.txt in trunk.
>
> Ben Smith-Mannschott wrote on Mon, Sep 12, 2011 at 15:44:20 +0200:
>> I've made the observation that FSFS repositories perform better on
>> EXT4 than BTRFS. This probably isn't ground-breaking, but I thought
>> I'd share it.
>>
>> I've got two Linux machines:
>>
>> - colossus, using BTRFS spanned over two disks.
>> 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011
>> x86_64 x86_64 x86_64 GNU/Linux
>>
>> - oberon, using EXT4 on a 2-disk software RAID-1 set.
>> Linux oberon 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011
>> x86_64 GNU/Linux
>>
>> I've noticed that writes to FSFS repositories are 5x faster under EXT4
>> than BTRFS. When svnsyncing form the same svn:// source to an local
>> repository (file://), oberon completes about 400 revisions in the time
>> it takes colossus to grind through 80.
>>
>> The BTRFS machine is our build server. Performance with (1.6.x)
>> working copies is quite acceptable, but I'm glad I'm not using it to
>> host svn repositories.
>>
>> Looks like the BTRFS people have some work to do. Maybe current
>> Kernels have already improved this picture. I know there has been
>> recent work on reducing the cost of meta-data operations (e.g. file
>> creation, ...) and that work is ongoing on defragmentation
>> functionality because of poor performance on files that are modified
>> in place heavily (e.g. sqlite).
>>
>> // ben
>
Re: an observation regarding FSFS performance on BTRFS
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
You haven't mentioned what version of svn you use. As you say, there
has been work recently --- some of it is in 1.7, some of it is on
^/subversion/branches/performance, some of it is on
^/subversion/branches/revprop-packing, and some additional ideas
are in notes/fsfs-improvements.txt in trunk.
Ben Smith-Mannschott wrote on Mon, Sep 12, 2011 at 15:44:20 +0200:
> I've made the observation that FSFS repositories perform better on
> EXT4 than BTRFS. This probably isn't ground-breaking, but I thought
> I'd share it.
>
> I've got two Linux machines:
>
> - colossus, using BTRFS spanned over two disks.
> 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011
> x86_64 x86_64 x86_64 GNU/Linux
>
> - oberon, using EXT4 on a 2-disk software RAID-1 set.
> Linux oberon 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011
> x86_64 GNU/Linux
>
> I've noticed that writes to FSFS repositories are 5x faster under EXT4
> than BTRFS. When svnsyncing form the same svn:// source to an local
> repository (file://), oberon completes about 400 revisions in the time
> it takes colossus to grind through 80.
>
> The BTRFS machine is our build server. Performance with (1.6.x)
> working copies is quite acceptable, but I'm glad I'm not using it to
> host svn repositories.
>
> Looks like the BTRFS people have some work to do. Maybe current
> Kernels have already improved this picture. I know there has been
> recent work on reducing the cost of meta-data operations (e.g. file
> creation, ...) and that work is ongoing on defragmentation
> functionality because of poor performance on files that are modified
> in place heavily (e.g. sqlite).
>
> // ben