You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Malcolm Rowe <ma...@farside.org.uk> on 2006/02/10 11:52:38 UTC
Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should 'svnadmin dump --deltas' write svndiff1 diffs?)
On Wed, Feb 08, 2006 at 02:19:48PM -0500, C. Michael Pilato wrote:
> Daniel Berlin wrote:
> >>So while the --deltas option saves quite a bit of space,
> >>'--deltas' + gzip is actually better than '--deltas --svndiff1', or even
> >>'--deltas --svndiff1' + gzip. In this case anyway.
> >
> > This will always be the case.
Of course it will. Silly me.
Though that does make me wonder: did we gain any benefit from making
ra_dav support svndiff1 diffs in the transport, when the whole stream
is almost certainly going to be compressed? Or did we actually make
things worse?
I know that the output of 'svnadmin dump --deltas' isn't completely
comparable to that of an ra_dav session, but presumably it's not too
different.
I can't easily set up a mod_dav_svn test locally, so if anyone could put
up a (semi-)public mod_dav_svn/svndiff1 server, I'll try to profile the
network traffic with and without svndiff1 enabled, and see whether it
makes a positive or negative impact on the results.
I feel like I might be missing something here; feel free to slap me down
if I've made some mindblowingly stupid assumption (e.g. "Not all httpd
servers run with compression enabled, you insensitive clod!" :-)).
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should 'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Fri, Feb 10, 2006 at 09:14:33AM -0500, C. Michael Pilato wrote:
> Malcolm Rowe wrote:
> > I feel like I might be missing something here; feel free to slap me down
> > if I've made some mindblowingly stupid assumption (e.g. "Not all httpd
> > servers run with compression enabled, you insensitive clod!" :-)).
>
> Well, couldn't the choice of svndiff0 or svndiff1 take into account
> whether or not wire compression is available? Can mod_dav_svn know that
> at the time the deltas are generated?
>
Maybe, though it might be easier for the client to condition the offer
of an svdiff1 capability based upon the 'http-compression' setting in
the config file.
I'd still like to test whether it makes a difference though.
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should
'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by "C. Michael Pilato" <cm...@collab.net>.
Malcolm Rowe wrote:
> Though that does make me wonder: did we gain any benefit from making
> ra_dav support svndiff1 diffs in the transport, when the whole stream
> is almost certainly going to be compressed? Or did we actually make
> things worse?
[...]
> I feel like I might be missing something here; feel free to slap me down
> if I've made some mindblowingly stupid assumption (e.g. "Not all httpd
> servers run with compression enabled, you insensitive clod!" :-)).
Well, couldn't the choice of svndiff0 or svndiff1 take into account
whether or not wire compression is available? Can mod_dav_svn know that
at the time the deltas are generated?
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should
'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by Daniel Berlin <db...@dberlin.org>.
On Fri, 2006-02-10 at 15:31 +0000, Malcolm Rowe wrote:
> On Fri, Feb 10, 2006 at 09:47:21AM -0500, Daniel Berlin wrote:
> > > Though that does make me wonder: did we gain any benefit from making
> > > ra_dav support svndiff1 diffs in the transport, when the whole stream
> > > is almost certainly going to be compressed? Or did we actually make
> > > things worse?
> >
> > It depends on a lot of factors :).
> >
>
> Sure. We really need to measure it :)
>
> > IIRC, isn't the data sent base64 encoded by us beforehand anyway?
> >
>
> Er, not entirely sure. I think that either we or Neon base64 the
> binary blob before transmission, yes. So that would mean that we need
> to determine whether
>
> (((svndiff ins. || svndiff data) => base64) || plaintext) => zlib
>
> was significantly worse or better than
>
> (((svndiff ins. || svndiff data => zlib) => base64) || plaintext) => zlib
>
> ... (or something like that, anyway)
>
> _If_ (big if) the tests I performed with 'dump --deltas' are in any
> way representative, we'll actually send more data in the second case,
> and burn more CPU time, too.
Possibly, and if so, we should definitely turn it off.
It was originally intended as an on-disk format anyway, the fact that we
transmit it over the wire is just kinda gravy.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should
'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by Daniel Berlin <db...@dberlin.org>.
On Fri, 2006-02-10 at 08:08 -0800, Justin Erenkrantz wrote:
> On 2/10/06, Malcolm Rowe <ma...@farside.org.uk> wrote:
> > Er, not entirely sure. I think that either we or Neon base64 the
> > binary blob before transmission, yes. So that would mean that we need
> > to determine whether
>
> FWIW, ra_serf doesn't use any base64 trickery. My plan is to have
> ra_serf request gzip encoding for plain-text files (i.e. no base) and
> svndiff (no gzip) for when we have a delta base.
It'd be really nice to request it for things like "log" as well, if
possible.
One of my few http users's complaints was that svn log information was
being transferred in plaintext, and there was megabytes of it in some
cases.
(I believe this is now fixed with gzip compression support in the
client. This was quite a while ago. However, in the interest of full
disclosure, i haven't checked :P)
> The knowledge of the
> delta base is way way better than requesting the entire plain text and
> zlibing it back in most cases.
Yes. In a lot of cases, we should also be able to transfer the deltas
directly from the repository (combining them if necessary), instead of
continually reencoding like we do now :(
>
> Greg and I have talked about mod_dav_svn not svndiff'ing a response
> when the file is under some X bytes because it's just better to give
> it back in plain text and avoid any diff overhead... -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should 'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On 2/10/06, Malcolm Rowe <ma...@farside.org.uk> wrote:
> Er, not entirely sure. I think that either we or Neon base64 the
> binary blob before transmission, yes. So that would mean that we need
> to determine whether
FWIW, ra_serf doesn't use any base64 trickery. My plan is to have
ra_serf request gzip encoding for plain-text files (i.e. no base) and
svndiff (no gzip) for when we have a delta base. The knowledge of the
delta base is way way better than requesting the entire plain text and
zlibing it back in most cases.
Greg and I have talked about mod_dav_svn not svndiff'ing a response
when the file is under some X bytes because it's just better to give
it back in plain text and avoid any diff overhead... -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should 'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Fri, Feb 10, 2006 at 09:47:21AM -0500, Daniel Berlin wrote:
> > Though that does make me wonder: did we gain any benefit from making
> > ra_dav support svndiff1 diffs in the transport, when the whole stream
> > is almost certainly going to be compressed? Or did we actually make
> > things worse?
>
> It depends on a lot of factors :).
>
Sure. We really need to measure it :)
> IIRC, isn't the data sent base64 encoded by us beforehand anyway?
>
Er, not entirely sure. I think that either we or Neon base64 the
binary blob before transmission, yes. So that would mean that we need
to determine whether
(((svndiff ins. || svndiff data) => base64) || plaintext) => zlib
was significantly worse or better than
(((svndiff ins. || svndiff data => zlib) => base64) || plaintext) => zlib
... (or something like that, anyway)
_If_ (big if) the tests I performed with 'dump --deltas' are in any
way representative, we'll actually send more data in the second case,
and burn more CPU time, too.
(I think that we can agree that it's always better to use svndiff1 if
the HTTP stream itself isn't compressed).
> Or am i just off my rocker?
>
Isn't that orthogonal to the above question? :)
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Should mod_dav_svn/ra_dav support svndiff1? (was: Re: Should
'svnadmin dump --deltas' write svndiff1 diffs?)
Posted by Daniel Berlin <db...@dberlin.org>.
> Of course it will. Silly me.
>
> Though that does make me wonder: did we gain any benefit from making
> ra_dav support svndiff1 diffs in the transport, when the whole stream
> is almost certainly going to be compressed? Or did we actually make
> things worse?
It depends on a lot of factors :).
IIRC, isn't the data sent base64 encoded by us beforehand anyway?
Or am i just off my rocker?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org