Subversion Performance (was Re: fs-rep-sharing branch)

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

Listman wrote:

> if rep-sharing is a performance drain then please make it optional.
> 
> Actually anything that is going to slow us should be optional (begin
> "svn is slow" rant yet again..)

So, does this mean that we shouldn't be storing deltas *at all* on the server,
but rather always storing the full text for every revision?  One of the benefits
of Subversion over CVS in the bad ol' days was that we treated every file as
just a binary blob, and did deltification on every object.  I doubt many people
want to sacrifice deltificiation for lower server CPU.

This leads me to a more general question that seems to be coming to a head: what
use cases should Subversion be optimized for?  Please pardon the jumbled
collection of thoughts that follows as I attempt to explore this question.

Subversion is used in so many more places than we ever dreamed it would, and
each has their own needs.  Some prefer economy of disk over CPU performance,
while others don't care about CPU, but require small network turnaround times.
Other users trust us to ship with reasonable defaults, and will just use those
whatever they be.  Subversion can not, and should not, be all things to all people.

Adding new features usually requires a performance hit.  Generally though, the
capability of the hardware increases to more than make up for whatever features
we add.  We've traditionally been pretty conservative, and running 1.0 on
hardware of that time is definitely slower than running 1.5 on today's hardware.
 I doubt the same can be said of many other projects.

Moore's "Law" is not a panacea for bad software, and I'm not suggesting we be
lazy in our coding practices, but I think that we can reasonably make
compromises when advancing the state of Subversion.  No one is forcing anybody
to upgrade, and users are free to build customized versions of the software
which make different compromises than the one we ship.

(The one caution I *would* make, though, is that we don't want to be too
customizable.  Doing so adds can easily lead to spaghetti-ified code, and user
confusion.  As much as users enjoy tweaking knobs, at the end of the day, most
just want something that works out-of-the-box, and our default experience should
be a positive one.)

Anyway, if people want to rip the rep-sharing stuff out, go ahead.  If
fsfs-packing should disappear, so be it.  However, there is a substantial
segment of our userbase who wants (and arguably needs) these types of features,
just as there others who care about massive concurrency and server CPU usage.
Let's not leave them hanging high-and-dry.

-Hyrum

Re: Subversion Performance (was Re: fs-rep-sharing branch)

[Erik Huelsmann <eh...@gmail.com>]

>>
>>> please do a google for "slow svn" and see what the users are saying on
>>> this issue
>>> (word of warning, its a little ugly). features are great and svn
>>> certainly has some
>>> good ones, but the main issue with svn today is performance.
>>
>> Again, patches welcome
>
>
>
> you have to face your dragons before you can defeat them - all i'm doing is
> trying to keep people aware of the real issues that svn users face today.
>
> please don't get defensive when we have this kind of discussion, this is not
> how we're going to make Subversion best in class..

The problem is "best in class for what"? Just by looking how many
people complain about svn being slow doesn't actually buy us anything:
it doesn't say why they say it, nor does it say if they're actually
exercising use-cases which we "support": doing things we think
subversion can be used for.

Just for comparison (google hits):

svn slow: 0,7 mln
p4 slow: 3,2 mln
rational slow: 6,3 mln
git slow: 5,7 mln

Just to show how useless this inquiry is: "git slow" returns as its
second link a youtube video which is hardly VC related.

What we need are hard facts preferably traceable to subversion system
components, not some sentiments to be found on google.

Bye,

Erik.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion Performance (was Re: fs-rep-sharing branch)

[Listman <li...@burble.net>]

On Oct 22, 2008, at 7:51 PM- Oct 22, 2008, Hyrum K. Wright wrote:

> Listman wrote:
>>
>>
>> On Oct 22, 2008, at 7:21 PM- Oct 22, 2008, Hyrum K. Wright wrote:
>>
>>> Listman wrote:
>>>
>>>> if rep-sharing is a performance drain then please make it optional.
>>>>
>>>> Actually anything that is going to slow us should be optional  
>>>> (begin
>>>> "svn is slow" rant yet again..)
>>>
>>> So, does this mean that we shouldn't be storing deltas *at all* on  
>>> the
>>> server,
>>> but rather always storing the full text for every revision?  One of
>>> the benefits
>>> of Subversion over CVS in the bad ol' days was that we treated every
>>> file as
>>> just a binary blob, and did deltification on every object.  I doubt
>>> many people
>>> want to sacrifice deltificiation for lower server CPU.
>>
>>
>> actually i don't care about the delta's to be honest. i'd like to see
>> that an option too.
>> that would actually be awesome..
>
> Patches welcome.
>
>>> This leads me to a more general question that seems to be coming  
>>> to a
>>> head: what
>>> use cases should Subversion be optimized for?  Please pardon the  
>>> jumbled
>>> collection of thoughts that follows as I attempt to explore this
>>> question.
>>>
>>> Subversion is used in so many more places than we ever dreamed it
>>> would, and
>>> each has their own needs.  Some prefer economy of disk over CPU
>>> performance,
>>> while others don't care about CPU, but require small network
>>> turnaround times.
>>> Other users trust us to ship with reasonable defaults, and will just
>>> use those
>>> whatever they be.  Subversion can not, and should not, be all things
>>> to all people.
>>>
>>> Adding new features usually requires a performance hit.  Generally
>>> though, the
>>> capability of the hardware increases to more than make up for  
>>> whatever
>>> features
>>> we add.  We've traditionally been pretty conservative, and running  
>>> 1.0 on
>>> hardware of that time is definitely slower than running 1.5 on  
>>> today's
>>> hardware.
>>> I doubt the same can be said of many other projects.
>>>
>>> Moore's "Law" is not a panacea for bad software, and I'm not
>>> suggesting we be
>>> lazy in our coding practices, but I think that we can reasonably  
>>> make
>>> compromises when advancing the state of Subversion.  No one is  
>>> forcing
>>> anybody
>>> to upgrade, and users are free to build customized versions of the
>>> software
>>> which make different compromises than the one we ship.
>>>
>>> (The one caution I *would* make, though, is that we don't want to  
>>> be too
>>> customizable.  Doing so adds can easily lead to spaghetti-ified  
>>> code,
>>> and user
>>> confusion.  As much as users enjoy tweaking knobs, at the end of the
>>> day, most
>>> just want something that works out-of-the-box, and our default
>>> experience should
>>> be a positive one.)
>>>
>>> Anyway, if people want to rip the rep-sharing stuff out, go  
>>> ahead.  If
>>> fsfs-packing should disappear, so be it.  However, there is a  
>>> substantial
>>> segment of our userbase who wants (and arguably needs) these types  
>>> of
>>> features,
>>> just as there others who care about massive concurrency and server  
>>> CPU
>>> usage.
>>> Let's not leave them hanging high-and-dry.
>>
>> i'm sure some folks find it useful, so why not just make it an  
>> option.
>> why force these
>> performance draining features down the throats of users that are  
>> already
>> struggling
>> with slowness issues...
>
> No one is forcing anything down anybody's throat.  Just because  
> users choose to
> upgrade to a newer version of a piece of software they receive for  
> free does
> *not* mean new features are being forced upon them.

err, ok..

>
>
>> please do a google for "slow svn" and see what the users are saying  
>> on
>> this issue
>> (word of warning, its a little ugly). features are great and svn
>> certainly has some
>> good ones, but the main issue with svn today is performance.
>
> Again, patches welcome



you have to face your dragons before you can defeat them - all i'm  
doing is
trying to keep people aware of the real issues that svn users face  
today.

please don't get defensive when we have this kind of discussion, this  
is not
how we're going to make Subversion best in class..




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion Performance (was Re: fs-rep-sharing branch)

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

Listman wrote:
> 
> 
> On Oct 22, 2008, at 7:21 PM- Oct 22, 2008, Hyrum K. Wright wrote:
> 
>> Listman wrote:
>>
>>> if rep-sharing is a performance drain then please make it optional.
>>>
>>> Actually anything that is going to slow us should be optional (begin
>>> "svn is slow" rant yet again..)
>>
>> So, does this mean that we shouldn't be storing deltas *at all* on the
>> server,
>> but rather always storing the full text for every revision?  One of
>> the benefits
>> of Subversion over CVS in the bad ol' days was that we treated every
>> file as
>> just a binary blob, and did deltification on every object.  I doubt
>> many people
>> want to sacrifice deltificiation for lower server CPU.
> 
> 
> actually i don't care about the delta's to be honest. i'd like to see
> that an option too.
> that would actually be awesome..

Patches welcome.

>> This leads me to a more general question that seems to be coming to a
>> head: what
>> use cases should Subversion be optimized for?  Please pardon the jumbled
>> collection of thoughts that follows as I attempt to explore this
>> question.
>>
>> Subversion is used in so many more places than we ever dreamed it
>> would, and
>> each has their own needs.  Some prefer economy of disk over CPU
>> performance,
>> while others don't care about CPU, but require small network
>> turnaround times.
>> Other users trust us to ship with reasonable defaults, and will just
>> use those
>> whatever they be.  Subversion can not, and should not, be all things
>> to all people.
>>
>> Adding new features usually requires a performance hit.  Generally
>> though, the
>> capability of the hardware increases to more than make up for whatever
>> features
>> we add.  We've traditionally been pretty conservative, and running 1.0 on
>> hardware of that time is definitely slower than running 1.5 on today's
>> hardware.
>> I doubt the same can be said of many other projects.
>>
>> Moore's "Law" is not a panacea for bad software, and I'm not
>> suggesting we be
>> lazy in our coding practices, but I think that we can reasonably make
>> compromises when advancing the state of Subversion.  No one is forcing
>> anybody
>> to upgrade, and users are free to build customized versions of the
>> software
>> which make different compromises than the one we ship.
>>
>> (The one caution I *would* make, though, is that we don't want to be too
>> customizable.  Doing so adds can easily lead to spaghetti-ified code,
>> and user
>> confusion.  As much as users enjoy tweaking knobs, at the end of the
>> day, most
>> just want something that works out-of-the-box, and our default
>> experience should
>> be a positive one.)
>>
>> Anyway, if people want to rip the rep-sharing stuff out, go ahead.  If
>> fsfs-packing should disappear, so be it.  However, there is a substantial
>> segment of our userbase who wants (and arguably needs) these types of
>> features,
>> just as there others who care about massive concurrency and server CPU
>> usage.
>> Let's not leave them hanging high-and-dry.
> 
> i'm sure some folks find it useful, so why not just make it an option.
> why force these
> performance draining features down the throats of users that are already
> struggling
> with slowness issues...

No one is forcing anything down anybody's throat.  Just because users choose to
upgrade to a newer version of a piece of software they receive for free does
*not* mean new features are being forced upon them.

> please do a google for "slow svn" and see what the users are saying on
> this issue
> (word of warning, its a little ugly). features are great and svn
> certainly has some
> good ones, but the main issue with svn today is performance.

Again, patches welcome.

-Hyrum

Re: Subversion Performance (was Re: fs-rep-sharing branch)

[Listman <li...@burble.net>]

On Oct 22, 2008, at 7:21 PM- Oct 22, 2008, Hyrum K. Wright wrote:

> Listman wrote:
>
>> if rep-sharing is a performance drain then please make it optional.
>>
>> Actually anything that is going to slow us should be optional (begin
>> "svn is slow" rant yet again..)
>
> So, does this mean that we shouldn't be storing deltas *at all* on  
> the server,
> but rather always storing the full text for every revision?  One of  
> the benefits
> of Subversion over CVS in the bad ol' days was that we treated every  
> file as
> just a binary blob, and did deltification on every object.  I doubt  
> many people
> want to sacrifice deltificiation for lower server CPU.


actually i don't care about the delta's to be honest. i'd like to see  
that an option too.
that would actually be awesome..

>
>
> This leads me to a more general question that seems to be coming to  
> a head: what
> use cases should Subversion be optimized for?  Please pardon the  
> jumbled
> collection of thoughts that follows as I attempt to explore this  
> question.
>
> Subversion is used in so many more places than we ever dreamed it  
> would, and
> each has their own needs.  Some prefer economy of disk over CPU  
> performance,
> while others don't care about CPU, but require small network  
> turnaround times.
> Other users trust us to ship with reasonable defaults, and will just  
> use those
> whatever they be.  Subversion can not, and should not, be all things  
> to all people.
>
> Adding new features usually requires a performance hit.  Generally  
> though, the
> capability of the hardware increases to more than make up for  
> whatever features
> we add.  We've traditionally been pretty conservative, and running  
> 1.0 on
> hardware of that time is definitely slower than running 1.5 on  
> today's hardware.
> I doubt the same can be said of many other projects.
>
> Moore's "Law" is not a panacea for bad software, and I'm not  
> suggesting we be
> lazy in our coding practices, but I think that we can reasonably make
> compromises when advancing the state of Subversion.  No one is  
> forcing anybody
> to upgrade, and users are free to build customized versions of the  
> software
> which make different compromises than the one we ship.
>
> (The one caution I *would* make, though, is that we don't want to be  
> too
> customizable.  Doing so adds can easily lead to spaghetti-ified  
> code, and user
> confusion.  As much as users enjoy tweaking knobs, at the end of the  
> day, most
> just want something that works out-of-the-box, and our default  
> experience should
> be a positive one.)
>
> Anyway, if people want to rip the rep-sharing stuff out, go ahead.  If
> fsfs-packing should disappear, so be it.  However, there is a  
> substantial
> segment of our userbase who wants (and arguably needs) these types  
> of features,
> just as there others who care about massive concurrency and server  
> CPU usage.
> Let's not leave them hanging high-and-dry.

i'm sure some folks find it useful, so why not just make it an option.  
why force these
performance draining features down the throats of users that are  
already struggling
with slowness issues...

please do a google for "slow svn" and see what the users are saying on  
this issue
(word of warning, its a little ugly). features are great and svn  
certainly has some
good ones, but the main issue with svn today is performance.



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion Performance (was Re: fs-rep-sharing branch)

[Blair Zajac <bl...@orcaware.com>]

On Oct 22, 2008, at 7:21 PM, Hyrum K. Wright wrote:

> Listman wrote:
>
>> if rep-sharing is a performance drain then please make it optional.
>>
>> Actually anything that is going to slow us should be optional (begin
>> "svn is slow" rant yet again..)
>
> Anyway, if people want to rip the rep-sharing stuff out, go ahead.  If
> fsfs-packing should disappear, so be it.  However, there is a  
> substantial
> segment of our userbase who wants (and arguably needs) these types  
> of features,
> just as there others who care about massive concurrency and server  
> CPU usage.
> Let's not leave them hanging high-and-dry.

I'd like to see fsfs-packing.  Our svn backend is going to be seeing  
10 commits per second and fsck'ing a filesystem so many inodes after  
many months of running is going to be painful.

Blair


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org