You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Hyrum K. Wright" <hy...@mail.utexas.edu> on 2008/11/06 21:48:06 UTC
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
hwright@tigris.org wrote:
> Author: hwright
> Date: Thu Nov 6 13:43:15 2008
> New Revision: 34088
>
> Log:
> Allow FSFS rep-caching to be configurable.
The question now becomes what's the default. Currently, it's set to "off"
mostly due to status quo bias. However, I think there's a strong case for "on".
The ones who want the better runtime performance are the ones who will probably
feel more comfortable configuring their FS. These are the large,
high-concurrency users, and there probably aren't too many of them.
On the other hand, the small teams are ones who will just stick with whatever
the default is, and are probably on more limited disk and hardware systems. It
makes sense to enable rep-sharing by default for these users.
Thoughts?
-Hyrum
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by Greg Stein <gs...@gmail.com>.
I concur. Default to "on".
On Thu, Nov 6, 2008 at 1:48 PM, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
> hwright@tigris.org wrote:
>> Author: hwright
>> Date: Thu Nov 6 13:43:15 2008
>> New Revision: 34088
>>
>> Log:
>> Allow FSFS rep-caching to be configurable.
>
> The question now becomes what's the default. Currently, it's set to "off"
> mostly due to status quo bias. However, I think there's a strong case for "on".
> The ones who want the better runtime performance are the ones who will probably
> feel more comfortable configuring their FS. These are the large,
> high-concurrency users, and there probably aren't too many of them.
>
> On the other hand, the small teams are ones who will just stick with whatever
> the default is, and are probably on more limited disk and hardware systems. It
> makes sense to enable rep-sharing by default for these users.
>
> Thoughts?
>
> -Hyrum
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by Bert Huijben <b....@competence.biz>.
> -----Original Message-----
> From: Hyrum K. Wright [mailto:hyrum_wright{a}mail.utexas.edu]
> Sent: donderdag 6 november 2008 22:48
> To: dev@subversion.tigris.org
> Subject: Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
>
> hwright@tigris.org wrote:
> > Author: hwright
> > Date: Thu Nov 6 13:43:15 2008
> > New Revision: 34088
> >
> > Log:
> > Allow FSFS rep-caching to be configurable.
>
> The question now becomes what's the default. Currently, it's set to
> "off"
> mostly due to status quo bias. However, I think there's a strong case
> for "on".
> The ones who want the better runtime performance are the ones who will
> probably
> feel more comfortable configuring their FS. These are the large,
> high-concurrency users, and there probably aren't too many of them.
>
> On the other hand, the small teams are ones who will just stick with
> whatever
> the default is, and are probably on more limited disk and hardware
> systems. It
> makes sense to enable rep-sharing by default for these users.
+1 to default on.
The chances you would have collision problems in normal use are virtually zero and the power users who need the additional safety for their research files can always disable it.
Defaulting to less disk space would be a quick win. (I would assume most disk space is won when you try to keep branches with only a few changes up to date with a single trunk; a quite common scenario for many users).
(And on Ivans reply)
And even if you are a hacker trying to create a collision you would have to create a collision against a /future/ version, as the old version would be the one in the repository.
You would need file level fs level write access to the repository to really make use of the collision as a hacker, and I don't understand why you would take the trouble of creating the collision if you could just change revisions.
Bert
>
> Thoughts?
>
> -Hyrum
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by David Glasser <gl...@davidglasser.net>.
On Tue, Nov 25, 2008 at 8:02 AM, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
> David Glasser wrote:
>> I'm not clear on what was resolved here (I still think this should
>> default to off, since the space savings is only noticeable on very
>> large repositories, and it is an extra unnecessary moving part if
>> you're not really pressed for space, and slows down commits), but the
>> current status of trunk is confusing. The svn_config_get_bool calls
>> give FALSE as the default value, which would imply to me that it's off
>> by default, but the generated config file seems to have an uncommented
>> (!) line setting it to true. I don't think this is how we ever
>> generate our config files: we should only be generating commented
>> lines. This means that newly-created and newly-upgraded repositories
>> have the same default values; generating lines that disagree with the
>> code's defaults doesn't really make much sense to me.
>
> Good point. The initial reason for leaving it as default was so that older
> repositories didn't start using rep sharing. Now that we're protecting the rep
> sharing functionality with a conditional on the format number, I think we can
> safely switch the default for svn_config_get_bool() to TRUE. I've done so in
> r34413.
OK. But I think the comment should be reworded and the commented-out
line made to be "= false"... I think that's our usual pattern (ie,
"uncomment this to do something", not "commented thing describes the
default").
> On the larger topic of enabling rep-sharing by default: I still think it makes
> sense to do so. We say that it "slows down commits," but I think the amount of
> slowdown is less than the potential variation in network latencies, in other
> words, negligible. The real place where it could hurt is high-concurrency
> shops, which can choose to disable it as part of their tuning process. Our repo
> isn't very high concurrency, yet the space savings are over 10%, which is about
> about the same as the ASF repo.
>
> The out-of-the-box default should work for our simplest users, and I think this
> is the best choice for them. The Berkeley DB backend doesn't even *have* the
> option to turn off rep sharing, yet I haven't heard any concerns about it.
Well, sure, but adding rep-sharing to the BDB backend didn't involve
adding an entirely new piece of database technology with its own
non-trivial concurrency and consistency concerns. (And BDB wasn't
explicitly designed as "the simple backend with minimal prereqs that
works on the largest variety of filesystems, including NFS, AFS,
etc".)
--dave
>> On Fri, Nov 7, 2008 at 7:47 AM, David Glasser <gl...@davidglasser.net> wrote:
>>> I'd like to keep it off by default until I see a solid case made with
>>> benchmarks etc that the space gain is more noticeable than the speed
>>> loss.
>>>
>>> (And I still want it reverted before 1.6 if the concurrency bugs aren't fixed.)
>>>
>>> --dave
>>>
>>> On Fri, Nov 7, 2008 at 1:16 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>>>> On Fri, Nov 7, 2008 at 12:48 AM, Hyrum K. Wright
>>>> <hy...@mail.utexas.edu> wrote:
>>>>> hwright@tigris.org wrote:
>>>>>> Author: hwright
>>>>>> Date: Thu Nov 6 13:43:15 2008
>>>>>> New Revision: 34088
>>>>>>
>>>>>> Log:
>>>>>> Allow FSFS rep-caching to be configurable.
>>>>> The question now becomes what's the default. Currently, it's set to "off"
>>>>> mostly due to status quo bias. However, I think there's a strong case for "on".
>>>>> The ones who want the better runtime performance are the ones who will probably
>>>>> feel more comfortable configuring their FS. These are the large,
>>>>> high-concurrency users, and there probably aren't too many of them.
>>>>>
>>>>> On the other hand, the small teams are ones who will just stick with whatever
>>>>> the default is, and are probably on more limited disk and hardware systems. It
>>>>> makes sense to enable rep-sharing by default for these users.
>>>>>
>>>> I'd like to keep this feature off by default. It will be regression if
>>>> we enable rep-sharing by default:
>>>> performance slowdown and possible colision and hacking. On other hand
>>>> is some kind of improvement for some groups of customers.
>>>>
>>>> So it's better prevent regression than introduce some improvement.
>
>
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
David Glasser wrote:
> I'm not clear on what was resolved here (I still think this should
> default to off, since the space savings is only noticeable on very
> large repositories, and it is an extra unnecessary moving part if
> you're not really pressed for space, and slows down commits), but the
> current status of trunk is confusing. The svn_config_get_bool calls
> give FALSE as the default value, which would imply to me that it's off
> by default, but the generated config file seems to have an uncommented
> (!) line setting it to true. I don't think this is how we ever
> generate our config files: we should only be generating commented
> lines. This means that newly-created and newly-upgraded repositories
> have the same default values; generating lines that disagree with the
> code's defaults doesn't really make much sense to me.
Good point. The initial reason for leaving it as default was so that older
repositories didn't start using rep sharing. Now that we're protecting the rep
sharing functionality with a conditional on the format number, I think we can
safely switch the default for svn_config_get_bool() to TRUE. I've done so in
r34413.
On the larger topic of enabling rep-sharing by default: I still think it makes
sense to do so. We say that it "slows down commits," but I think the amount of
slowdown is less than the potential variation in network latencies, in other
words, negligible. The real place where it could hurt is high-concurrency
shops, which can choose to disable it as part of their tuning process. Our repo
isn't very high concurrency, yet the space savings are over 10%, which is about
about the same as the ASF repo.
The out-of-the-box default should work for our simplest users, and I think this
is the best choice for them. The Berkeley DB backend doesn't even *have* the
option to turn off rep sharing, yet I haven't heard any concerns about it.
> On Fri, Nov 7, 2008 at 7:47 AM, David Glasser <gl...@davidglasser.net> wrote:
>> I'd like to keep it off by default until I see a solid case made with
>> benchmarks etc that the space gain is more noticeable than the speed
>> loss.
>>
>> (And I still want it reverted before 1.6 if the concurrency bugs aren't fixed.)
>>
>> --dave
>>
>> On Fri, Nov 7, 2008 at 1:16 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>>> On Fri, Nov 7, 2008 at 12:48 AM, Hyrum K. Wright
>>> <hy...@mail.utexas.edu> wrote:
>>>> hwright@tigris.org wrote:
>>>>> Author: hwright
>>>>> Date: Thu Nov 6 13:43:15 2008
>>>>> New Revision: 34088
>>>>>
>>>>> Log:
>>>>> Allow FSFS rep-caching to be configurable.
>>>> The question now becomes what's the default. Currently, it's set to "off"
>>>> mostly due to status quo bias. However, I think there's a strong case for "on".
>>>> The ones who want the better runtime performance are the ones who will probably
>>>> feel more comfortable configuring their FS. These are the large,
>>>> high-concurrency users, and there probably aren't too many of them.
>>>>
>>>> On the other hand, the small teams are ones who will just stick with whatever
>>>> the default is, and are probably on more limited disk and hardware systems. It
>>>> makes sense to enable rep-sharing by default for these users.
>>>>
>>> I'd like to keep this feature off by default. It will be regression if
>>> we enable rep-sharing by default:
>>> performance slowdown and possible colision and hacking. On other hand
>>> is some kind of improvement for some groups of customers.
>>>
>>> So it's better prevent regression than introduce some improvement.
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by David Glasser <gl...@davidglasser.net>.
I'm not clear on what was resolved here (I still think this should
default to off, since the space savings is only noticeable on very
large repositories, and it is an extra unnecessary moving part if
you're not really pressed for space, and slows down commits), but the
current status of trunk is confusing. The svn_config_get_bool calls
give FALSE as the default value, which would imply to me that it's off
by default, but the generated config file seems to have an uncommented
(!) line setting it to true. I don't think this is how we ever
generate our config files: we should only be generating commented
lines. This means that newly-created and newly-upgraded repositories
have the same default values; generating lines that disagree with the
code's defaults doesn't really make much sense to me.
--dave
On Fri, Nov 7, 2008 at 7:47 AM, David Glasser <gl...@davidglasser.net> wrote:
> I'd like to keep it off by default until I see a solid case made with
> benchmarks etc that the space gain is more noticeable than the speed
> loss.
>
> (And I still want it reverted before 1.6 if the concurrency bugs aren't fixed.)
>
> --dave
>
> On Fri, Nov 7, 2008 at 1:16 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>> On Fri, Nov 7, 2008 at 12:48 AM, Hyrum K. Wright
>> <hy...@mail.utexas.edu> wrote:
>>> hwright@tigris.org wrote:
>>>> Author: hwright
>>>> Date: Thu Nov 6 13:43:15 2008
>>>> New Revision: 34088
>>>>
>>>> Log:
>>>> Allow FSFS rep-caching to be configurable.
>>>
>>> The question now becomes what's the default. Currently, it's set to "off"
>>> mostly due to status quo bias. However, I think there's a strong case for "on".
>>> The ones who want the better runtime performance are the ones who will probably
>>> feel more comfortable configuring their FS. These are the large,
>>> high-concurrency users, and there probably aren't too many of them.
>>>
>>> On the other hand, the small teams are ones who will just stick with whatever
>>> the default is, and are probably on more limited disk and hardware systems. It
>>> makes sense to enable rep-sharing by default for these users.
>>>
>> I'd like to keep this feature off by default. It will be regression if
>> we enable rep-sharing by default:
>> performance slowdown and possible colision and hacking. On other hand
>> is some kind of improvement for some groups of customers.
>>
>> So it's better prevent regression than introduce some improvement.
>>
>> --
>> Ivan Zhakov
>> VisualSVN Team
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: dev-help@subversion.tigris.org
>>
>>
>
>
>
> --
> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by Peter Samuelson <pe...@p12n.org>.
[Peter Wemm]
> FWIW, Size is a huge factor for us at freebsd.org. Right now the
> repo fits 100% in ram on our servers.
Right ... but you're exactly the sort of administrator for whom the
default does not matter, because you are willing and able to tune your
install. The question is more whether the size / speed tradeoff still
makes sense for typical deployments whose administrators are not.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by Peter Wemm <pe...@wemm.org>.
On Fri, Nov 7, 2008 at 11:42 AM, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
> David Glasser wrote:
>> I'd like to keep it off by default until I see a solid case made with
>> benchmarks etc that the space gain is more noticeable than the speed
>> loss.
>
> I'm curious as to how we can quantitatively compare space gain vs. speed loss.
> It's a tradeoff to be sure, but doing a direct comparison seems like
> apples-to-oranges. Have you run Mike's script on the new, fsfs-disabled trunk
> repo? (See http://svn.haxx.se/dev/archive-2008-10/0985.shtml)
FWIW, Size is a huge factor for us at freebsd.org. Right now the repo
fits 100% in ram on our servers. We've seen as much as an order of
magnitude (or more) slowdown as a consequence of hitting disk while
bringing things up. A 3.5GB working set is fairly easy to manage on
current servers, but I'd hate to see a massive scaling up of size.
--
Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV
"All of this is for nothing if we don't go to the stars" - JMS/B5
"If Java had true garbage collection, most programs would delete
themselves upon execution." -- Robert Sewell
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
David Glasser wrote:
> I'd like to keep it off by default until I see a solid case made with
> benchmarks etc that the space gain is more noticeable than the speed
> loss.
I'm curious as to how we can quantitatively compare space gain vs. speed loss.
It's a tradeoff to be sure, but doing a direct comparison seems like
apples-to-oranges. Have you run Mike's script on the new, fsfs-disabled trunk
repo? (See http://svn.haxx.se/dev/archive-2008-10/0985.shtml)
> (And I still want it reverted before 1.6 if the concurrency bugs aren't fixed.)
I've been buried underneath stuff this past week, but haven't forgotten. I'm
planning on looking at these issues next week (prior to branching).
-Hyrum
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by David Glasser <gl...@davidglasser.net>.
I'd like to keep it off by default until I see a solid case made with
benchmarks etc that the space gain is more noticeable than the speed
loss.
(And I still want it reverted before 1.6 if the concurrency bugs aren't fixed.)
--dave
On Fri, Nov 7, 2008 at 1:16 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
> On Fri, Nov 7, 2008 at 12:48 AM, Hyrum K. Wright
> <hy...@mail.utexas.edu> wrote:
>> hwright@tigris.org wrote:
>>> Author: hwright
>>> Date: Thu Nov 6 13:43:15 2008
>>> New Revision: 34088
>>>
>>> Log:
>>> Allow FSFS rep-caching to be configurable.
>>
>> The question now becomes what's the default. Currently, it's set to "off"
>> mostly due to status quo bias. However, I think there's a strong case for "on".
>> The ones who want the better runtime performance are the ones who will probably
>> feel more comfortable configuring their FS. These are the large,
>> high-concurrency users, and there probably aren't too many of them.
>>
>> On the other hand, the small teams are ones who will just stick with whatever
>> the default is, and are probably on more limited disk and hardware systems. It
>> makes sense to enable rep-sharing by default for these users.
>>
> I'd like to keep this feature off by default. It will be regression if
> we enable rep-sharing by default:
> performance slowdown and possible colision and hacking. On other hand
> is some kind of improvement for some groups of customers.
>
> So it's better prevent regression than introduce some improvement.
>
> --
> Ivan Zhakov
> VisualSVN Team
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r34088 - trunk/subversion/libsvn_fs_fs
Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Fri, Nov 7, 2008 at 12:48 AM, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
> hwright@tigris.org wrote:
>> Author: hwright
>> Date: Thu Nov 6 13:43:15 2008
>> New Revision: 34088
>>
>> Log:
>> Allow FSFS rep-caching to be configurable.
>
> The question now becomes what's the default. Currently, it's set to "off"
> mostly due to status quo bias. However, I think there's a strong case for "on".
> The ones who want the better runtime performance are the ones who will probably
> feel more comfortable configuring their FS. These are the large,
> high-concurrency users, and there probably aren't too many of them.
>
> On the other hand, the small teams are ones who will just stick with whatever
> the default is, and are probably on more limited disk and hardware systems. It
> makes sense to enable rep-sharing by default for these users.
>
I'd like to keep this feature off by default. It will be regression if
we enable rep-sharing by default:
performance slowdown and possible colision and hacking. On other hand
is some kind of improvement for some groups of customers.
So it's better prevent regression than introduce some improvement.
--
Ivan Zhakov
VisualSVN Team
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org