non-skelta update editor mode in ra_serf (was Re: svn commit: r1415864 - /subversion/trunk/subversion/libsvn_ra_serf/update.c)

[Ivan Zhakov <iv...@visualsvn.com>]

[ changing subject to make topic more visible]

On Sat, Dec 1, 2012 at 9:00 PM, Mark Phippard <ma...@gmail.com> wrote:
> On Sat, Dec 1, 2012 at 12:36 AM, Justin Erenkrantz
> <ju...@erenkrantz.com> wrote:
>> On Fri, Nov 30, 2012 at 4:54 PM, <cm...@apache.org> wrote:
>>>
>>> Author: cmpilato
>>> Date: Fri Nov 30 21:54:35 2012
>>> New Revision: 1415864
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1415864&view=rev
>>> Log:
>>> Implement in ra_serf "send-all" mode support for update-style REPORTs
>>> and their responses.  (Currently disabled by compile-time conditionals.)
>>>
>>> (This one goes out to Ivan Zhakov.)
>>
>>
>> I've stated for a long time that I think the send-all mode is a huge mistake
>> architecturally because it is too prone to double-compression and TCP
>> pipeline stalls and is a tremendous burden on a properly-configured httpd
>> (by not taking advantage of server-side parallelism), it's nice to see it's
>> not *too* hard to shoehorn this bad idea back into ra_serf.  We'd never be
>> able to shove the non-send-all approach into ra_neon.  =)
>
> Just to be clear, I do not believe anyone is suggesting we completely
> abandon the non-send-all approach.  I like that this approach can
> offer good performance on a well-configured server as well as enable
> new features/ideas such as not even fetching the full-texts that we
> already have locally.  I think the question is simply what is the best
> way to deliver this.
Completely agree.

My point was that in theory skelta-mode is cool, but it still needs a
lot of work to get it really done. So let's release ra_serf by
piecemeal, because we also have significant amount of ra_serf issues
unrelated to update editor.

>
>> Here's my suggestion for consideration - let's experiment with this setting
>> in the beta release process with the setting as-is - that is we always do
>> the parallel updates unconditionally (except perhaps when svnrdump is being
>> stupid).  If we get real users complaining about the update during that
>> cycle, we can then figure out either switching the default and/or adding a
>> config-option or even allowing some control via capabilities exchange.
>
> I feel pretty strongly that we should at minimum use the send-all
> approach when talking to pre-1.8 servers.  Even though in some
> situations it could still offer good performance.  I just think it
> would be more respectful to our users (server admins in this case) to
> not change this behavior in a way that could surprise them.  Maybe we
> could come up with exceptions, such as older servers that are using
> the SVNAllowBulkUpdates off directive.  In that situation we should
> use the new behavior since that is basically what that directive is
> asking for.
>
> As I said in another thread, I think we should treat a 1.8 server the
> same way and require someone that was upgrading to add some new
> directive to enable the new feature.  This would allow a server admin
> to setup his server correctly, including using things like
> mod_deflate, and turn on the new behavior rather than get it
> automatically simply because they upgraded their binaries.
>
> This seems like it satisfies everyone.  Existing users, especially
> those running older server versions, would not be surprised by new and
> unwanted client behavior, and it would still be easy to configure a
> new server properly to support the non-send-all mode when it was
> desired.  I just do not see what the downside would be to approaching
> it this way.
>
+1.

-- 
Ivan Zhakov

Re: non-skelta update editor mode in ra_serf (was Re: svn commit: r1415864 - /subversion/trunk/subversion/libsvn_ra_serf/update.c)

[Lieven Govaerts <sv...@mobsol.be>]

Hi,

On Sat, Dec 1, 2012 at 10:18 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
> [ changing subject to make topic more visible]
>
> On Sat, Dec 1, 2012 at 9:00 PM, Mark Phippard <ma...@gmail.com> wrote:
>> On Sat, Dec 1, 2012 at 12:36 AM, Justin Erenkrantz
>> <ju...@erenkrantz.com> wrote:
>>> On Fri, Nov 30, 2012 at 4:54 PM, <cm...@apache.org> wrote:
>>>>
>>>> Author: cmpilato
>>>> Date: Fri Nov 30 21:54:35 2012
>>>> New Revision: 1415864
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1415864&view=rev
>>>> Log:
>>>> Implement in ra_serf "send-all" mode support for update-style REPORTs
>>>> and their responses.  (Currently disabled by compile-time conditionals.)
>>>>
>>>> (This one goes out to Ivan Zhakov.)
>>>
>>>
>>> I've stated for a long time that I think the send-all mode is a huge mistake
>>> architecturally because it is too prone to double-compression and TCP
>>> pipeline stalls and is a tremendous burden on a properly-configured httpd
>>> (by not taking advantage of server-side parallelism), it's nice to see it's
>>> not *too* hard to shoehorn this bad idea back into ra_serf.  We'd never be
>>> able to shove the non-send-all approach into ra_neon.  =)
>>
>> Just to be clear, I do not believe anyone is suggesting we completely
>> abandon the non-send-all approach.  I like that this approach can
>> offer good performance on a well-configured server as well as enable
>> new features/ideas such as not even fetching the full-texts that we
>> already have locally.  I think the question is simply what is the best
>> way to deliver this.
> Completely agree.
>
> My point was that in theory skelta-mode is cool, but it still needs a
> lot of work to get it really done. So let's release ra_serf by
> piecemeal, because we also have significant amount of ra_serf issues
> unrelated to update editor.
>
>>
>>> Here's my suggestion for consideration - let's experiment with this setting
>>> in the beta release process with the setting as-is - that is we always do
>>> the parallel updates unconditionally (except perhaps when svnrdump is being
>>> stupid).  If we get real users complaining about the update during that
>>> cycle, we can then figure out either switching the default and/or adding a
>>> config-option or even allowing some control via capabilities exchange.
>>
>> I feel pretty strongly that we should at minimum use the send-all
>> approach when talking to pre-1.8 servers.  Even though in some
>> situations it could still offer good performance.  I just think it
>> would be more respectful to our users (server admins in this case) to
>> not change this behavior in a way that could surprise them.  Maybe we
>> could come up with exceptions, such as older servers that are using
>> the SVNAllowBulkUpdates off directive.  In that situation we should
>> use the new behavior since that is basically what that directive is
>> asking for.
>>
>> As I said in another thread, I think we should treat a 1.8 server the
>> same way and require someone that was upgrading to add some new
>> directive to enable the new feature.  This would allow a server admin
>> to setup his server correctly, including using things like
>> mod_deflate, and turn on the new behavior rather than get it
>> automatically simply because they upgraded their binaries.
>>
>> This seems like it satisfies everyone.  Existing users, especially
>> those running older server versions, would not be surprised by new and
>> unwanted client behavior, and it would still be easy to configure a
>> new server properly to support the non-send-all mode when it was
>> desired.  I just do not see what the downside would be to approaching
>> it this way.
>>
> +1.
>

r1417639 and r1417642 respectively give the server admin and the user
the option to force the use of bulk updates.
Changes are ready for review and use.

I haven't changed any of the defaults in ra_serf and mod_dav_svn.

Lieven

Re: non-skelta update editor mode in ra_serf (was Re: svn commit: r1415864 - /subversion/trunk/subversion/libsvn_ra_serf/update.c)

[Johan Corveleyn <jc...@gmail.com>]

On Sat, Dec 1, 2012 at 10:40 PM, Branko Čibej <br...@wandisco.com> wrote:
> On 01.12.2012 22:18, Ivan Zhakov wrote:
>> Completely agree.
>>
>> My point was that in theory skelta-mode is cool, but it still needs a
>> lot of work to get it really done.
>
> Sorry? I've had "http-library = serf" in ~/.subversion/config for years,
> and have seen no problems. What does it take, in your opinion, to get it
> "really done"?

It has become abundantly clear during the last year that, just because
some people have been using it for years, doesn't mean it's stable /
done / ready for prime time. What do you think I spent my entire
Berlin hackathon week on last June, together with Justin and others?
And weeks before that and after ... and the time spent by others
during the last months fixing difficult to reproduce, yet important
bugs.

Those issues were not due to wrong configurations, or total edge
cases, but things that people with big installations would have run
into the first day they would have started using it.

I don't want to rehash all the discussions, I think there's mostly
consensus on the general direction we can / should go (which is
forward, not backward). But I find "I have been using it for years
without problems" not a very strong argument in this case.

Getting back to the actual topic ... as an svn admin, I'd definitely
like to have the choice between send-all and skelta mode, if that's
possible. Which one should be the default ... I don't know. But having
the choice is the most important thing. I'd sleep better at night,
knowing that I can flip a switch to reduce the number of requests
etc., in the unlikely case that would turn out to be a problem :-).

-- 
Johan

Re: non-skelta update editor mode in ra_serf (was Re: svn commit: r1415864 - /subversion/trunk/subversion/libsvn_ra_serf/update.c)

[Branko Čibej <br...@wandisco.com>]

On 01.12.2012 22:18, Ivan Zhakov wrote:
> Completely agree.
>
> My point was that in theory skelta-mode is cool, but it still needs a
> lot of work to get it really done.

Sorry? I've had "http-library = serf" in ~/.subversion/config for years,
and have seen no problems. What does it take, in your opinion, to get it
"really done"?

>  So let's release ra_serf by
> piecemeal, because we also have significant amount of ra_serf issues
> unrelated to update editor.

We've been through that. There's one or maybe two bugs that are actual
bugs that pop up in edge cases, everything else is related to server
configuration etc.

Is your goal shipping without bugs?

-- Brane

-- 
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com