You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Ben Collins-Sussman <su...@red-bean.com> on 2008/09/30 22:30:58 UTC
Re: svn commit: r33366 - trunk/notes
On Tue, Sep 30, 2008 at 3:24 PM, <gs...@tigris.org> wrote:
> Unfortunately, DeltaV is an insanely complex and inefficient protocol,
> and doesn't fit Subversion's model well at all. The result is that
> Subversion speaks a "limited portion" of DeltaV, and pays a huge price
> for this complexity: speed.
>
> +### GJS: doesn't fit? heh. IMO, it does... you should have seen it
> + *before* I grok'd the design and started providing feedback. it
> + just doesn't match it *precisely*
Sure, I remember. But still, we only implement "just enough" DeltaV
to look like a DAV server to a dumb DAV client. But there are no
3rd-party DeltaV clients that run against mod_dav_svn, and there are
no 3rd-party DeltaV servers that can talk to an svn client. Nearly
every single "interesting" svn_ra.h interface is done through a custom
REPORT -- checkout, update, log, etc. We've surrounded ourselves with
DeltaV formalities that provide a lot of complexity and zero value.
> +
> A typical network trace involves dozens of unnecessary turnarounds
> where the client keeps asking for the same information over and over
> again, all for the sake of following DeltaV. And then once the client
> @@ -33,6 +38,19 @@ has "discovered" the information it need
> custom REPORT request anyway. Most svn operations are at least twice
> as slow over HTTP than over the custom svnserve protocol.
>
> +### GJS: that is the fault of the client, not the protocol. somewhere
> + around the time that I stopped working on SVN for a while, I
> + identified several things that an RA layer could cache in order to
> + avoid looking them up again. nobody ever implemented that. thus,
> + it is hard to claim the fault lies in the protocol when we KNOW we
> + don't have to re-fetch certain items.
But this just makes our existing clients more and more complex. Why
are we doing PROPFINDs at all, ever? Unless we're actually trying to
implement svn_ra_get_prop(), every single PROPFIND we do before our
custom REPORTs are just weird formalities related to DeltaV discovery.
Even if we completely remove all the redundant PROPFIND requests, we
still get absolutely nothing out of following the DeltaV rules in
these cases.
> +
> +### GJS: turnarounds don't have to be expensive if you can pipeline
> + them. that is one of the (design) reasons for Serf. Again,
> + unrelated to the protocol, but simply the implementation. And
> + proposing a *new* implementation rather than fixing an existing
> + one seems to be a much more monumental task.
I have to do a reality check here: the promise of serf is that
checkouts/updates would be faster than neon, because we could pipeline
a bunch of GET requests rather than suck down the tree in one
response. In practice, serf has proven to be no faster than neon in
this regard. (Or if faster, only by a tiny percent.)
I understand that the "real" promise here is that of caching proxy
servers, which will supposedly deliver serf's original speed promises
to us... but I've lost my faith in this idea:
* nobody's actually done it yet and demonstrated it
* given that BDB and/or FSFS is (typically) already serving popular
fs-nodes out of RAM (due to OS caching), I don't think a proxy-server
serving nodes out of RAM will be any faster.
> +
>
> PROPOSAL
> ========
> @@ -41,15 +59,22 @@ Write a new HTTP protocol for svn; map
> requests.
>
> * svn over HTTP would be much faster (eliminate turnarounds)
> +
> +### GJS: again: pipelining.
Are we going to pipeline every single request in svn_ra.h? Does it
even make sense to do so? I don't view it as a magic bullet.
>
> * svn over HTTP would be almost as easy to extend as svnserve.
>
> +### GJS: have we had an issue with extending our current HTTP use?
Let's ask folks who have implement the mod_dav_svn get-lock-report,
get-locations report, mergeinfo-report, etc.
But meanwhile, I maintain it's not so much about extending as it is
about maintaining. Anyone should be able to understand what's
happening in the HTTP layer, debug it, rev an interface by adding a
new parameter. Nobody is scared of doing this in svnserve. To do
this in mod_dav_svn, people need a 3 hour lecture in the architecture
of the system, DeltaV terminology, etc.
> +
> * svn over HTTP would be comprehensible to devs and users both
> (require no knowledge of DeltaV concepts).
>
> * We'd still maintain almost all of Apache's advantages
> (propositions A through E above).
>
> +### GJS: and with the right design, should be able to get the G that I
> + added above.
> +
> * We'd give up proposition F, which hasn't given us anything but the
> ability to mount svn repositories as network drives. (For those
> who *really* need this rarely-used feature, mod_dav_svn would
> @@ -58,6 +83,11 @@ requests.
> * We could freely advertise a fixed syntax for browsing older
> revisions, without worrying about violating DeltaV.
>
> +### GJS: we can do this today. DeltaV certainly doesn't proscribe what
> + we can do in our URL namespace. my reluctance to advertise one was
> + to give us maximum flexibility on the server, and to defer browing
> + to external tools like ViewVC or Trac.
Agreed, this feature isn't tied to a protocol rewrite at all. I'll
remove it from the doc.
> +
>
> MILE-HIGH DESIGN
> ================
> @@ -124,6 +154,9 @@ DESIGN
>
> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>
> +### GJS: in order to allow for intermediate caches to work, the URLs
> + need to be well-designed up-front. I can help with that.
Is caching really a goal? Why is it so important on your list?
> +
> For requests which require large data exchanges, the big payloads go
> into request and/or response bodies. For example:
>
> @@ -131,6 +164,11 @@ DESIGN
>
> [body contains complete 'update report' describing working copy's
> revisions; response is a complete editor-drive.]
> +
> +### GJS: no. I've never seen a GET request with a body. You want to
> + avoid that, or you could end up being incompatible with many
> + proxies. If you need input data, then make it a POST, or use one
> + of the other HTTP methods defined by HTTP, WebDAV, or DeltaV.
Ah, ok, so any request bodies will probably need to be POSTs or somesuch.
>
> POST /repos?cmd=commit&keeplocks=true
>
> @@ -170,6 +208,12 @@ DESIGN
> embedded s-expressions, exactly like svnserve does. Heck, maybe we
> could even use svnserve's parsing/unparsing code!)
>
> +### GJS: personally, I'd recommend protocol buffers, or Apache Thrift
> + (a third-party reimplementation of PBs done at facebook). just
> + toss out the formats used in FS and the svn protocol, and let a
> + new/external library do the marshalling for us.
> + http://stuartsierra.com/2008/07/10/thrift-vs-protocol-buffers
Hmmm, interesting.
Honestly, it sounds to me like you might be willing to toss away all
the DeltaV junk, just as long as we still make individual GETs of
(rev, path) objects (1) possible, (2) cacheable, (3) pipelineable.
Those seem to be the things that are important to you.
If that's the case, I *still* think we need a new protocol, and a new
server and client to implement it. The idea is to start clean, build
the smallest possible protocol with the smallest amount of code,
directly corresponding with svn_ra.h as much as possible... and heck,
if we can get pipelining and cacheability in there too, great. But I
cannot comprehend achieving this goal by modifying the existing
codebase.... it would just be adding even more complexity to something
already incomprehensible to most of the svn developers.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
On Tue, Sep 30, 2008 at 4:50 PM, Mark Phippard <ma...@gmail.com> wrote:
>...
> That said, correct me if I am wrong, but if your server is using SSL
> isn't it impossible for a proxy to ever cache anything? I was looking
> into this recently for a corporate customer and this seemed like an
> absolute dead-end because of SSL.
It is possible to have the proxy perform the SSL connection for you,
and you'd do plain HTTP to that (local) proxy. In this situation, the
proxy will see/cache content. If you're worried about keeping the
local connection private, that can be done under a separate SSL
connection.
(it's cool shit like this as one of my reasons for suggesting HTTP all
those years ago...)
Cheers,
-g
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Mark Phippard <ma...@gmail.com>.
On Tue, Sep 30, 2008 at 7:00 PM, Greg Stein <gs...@gmail.com> wrote:
> And yes, I want cachability of (rev, path) nodes. If a request can be
> satisfied locally, then that is a huge win. You say "serve out of
> RAM", and both will do that. But a proxy can do it on your LAN rather
> than the server via the WAN. You say "nobody has done it", but that is
> an incorrect argument. Nobody has been ABLE to do it because we
> glommed shit into a mother REPORT. And serf needs some more work
> before people can TRY it. So I respectfully disagree with any argument
> about cachability being a non-requirement.
It does not make sense to come up with a new design and not try to
make it work well (ideally very well) with caching proxy servers.
That said, correct me if I am wrong, but if your server is using SSL
isn't it impossible for a proxy to ever cache anything? I was looking
into this recently for a corporate customer and this seemed like an
absolute dead-end because of SSL.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Eric Gillespie <ep...@pretzelnet.org>.
I debated joining this thread, since I don't really work here
anymore ;->. But what the hey.
Greg Stein <gs...@gmail.com> writes:
> Recall that svnserve was started as a simplification over Apache. But
> since then, it has added multiple auth modes, logging, threading, and
> all the other goodies that Apache was providing us that svnserve was
> "going to stay away from in staying simple". I believe we may want to
Last time the snickering about svnserve catching up with apache
came up (it seems to come up a lot), I figured it was about time
someone posted some numbers, but I dropped it.
I don't think anyone ever promised svnserve would not gain such
features. It can and has grown them while remaining simpler,
easier to understand, and easier to manage. I'm well aware of
the pitfalls of such a comparison, but comparing lines of code
may be illuminating:
mod_dav_svn 14559
+ httpd/modules/dav/main 11612
+ pick your poison:
libsvn_ra_neon 13731 => 39902
libsvn_ra_serf 16285 => 42456
svnserve 5141
+ libsvn_ra_svn 5996 => 11137
Beyond that, it's pretty fuzzy: do you count all Apache against
svndav? What about OpenSSL? If you do either of those, it's
only fair to count cyrus sasl against svnserve, but it's a much
smaller dependency. And openssl and cyrus sasl are already
required for even the most minimal server installation of every
modern Linux distribution. So, too fuzzy.
Now you guys are talking about adding a second HTTP layer.
Unless we dump svndav and declare 2.0, how on earth is this
reducing the burden? If you have to hack 5 ra clients (2 of
which are difficult to understand, and I'm doubtful the new one
can be much better) and 3 servers (1 of which is impossible to
understand, and I'm doubtful the new one can be much better) to
add a new ra call, how many will bother? How much will it slow
down those who do?
I saw at least one reference to having ra_serf speak both svndav
*and* the new protocol. I don't know how serious that suggestion
was, but it sure scared me. Maybe I misunderstood, though.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Branko Čibej <br...@xbc.nu>.
Greg Stein wrote:
> On Tue, Sep 30, 2008 at 5:46 PM, Branko Čibej <br...@xbc.nu> wrote:
>
>> Greg Stein wrote:
>>
>>> In general, I'm not crazy-opposed. You're entirely right: the vision
>>> of WebDAV (or "WebDA") came to fruition. DeltaV did not, so attempting
>>> to adhere strongly to DeltaV really makes little pragmatic sense.
>>>
>>> Within the scope of the (new) design, I *do* think it would be
>>> interesting to make it DAV-capable. i.e. is the URL namespace both
>>> DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
>>> maintain you could probably mesh the two pretty easily. The "new"
>>> client would do some interesting GETs and POSTs, and a DAV client (not
>>> svn! a downlevel client) would throw in a couple PROPFINDs, and if we
>>> reach a bit, then some autoversioning around PUT and DELETE.
>>>
>>> IOW, what I might suggest is a mesh of your simplified protocol, with
>>> the related DAV support for Windows, Mac, Linux, and other software
>>> DAV-users. An admin could install mod_svn and get speed *and* DAV
>>> capability.
>>>
>>>
>> I agree that a SVN DAV plugin for dumb clients is a good thing ... well,
>> a major selling point in the non-techie parts of the corporate
>> environment, heh. But cramming both into a single httpd module seems
>> like serious overkill and suspiciously close to what we have now.
>> Unmaintainable nightmare, don't y'know. Keep 'em separate and simple.
>>
>
> Possibly. Downlevel support is not that hard if it is read-only, so
> the answer might be to scrap auto-versioning from the "DAV add-on"
> portion. I'd still be curious about auto-versioning, and what the
> additional work *really* means.
>
I should've said "DAV autoversioning" rather than "DAV plugin for dumb
clients". Frankly the only real benefit of DAV over a custom protocol
that I've ever seen is the autoversioning bit.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
On Tue, Sep 30, 2008 at 5:46 PM, Branko Čibej <br...@xbc.nu> wrote:
> Greg Stein wrote:
>> In general, I'm not crazy-opposed. You're entirely right: the vision
>> of WebDAV (or "WebDA") came to fruition. DeltaV did not, so attempting
>> to adhere strongly to DeltaV really makes little pragmatic sense.
>>
>> Within the scope of the (new) design, I *do* think it would be
>> interesting to make it DAV-capable. i.e. is the URL namespace both
>> DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
>> maintain you could probably mesh the two pretty easily. The "new"
>> client would do some interesting GETs and POSTs, and a DAV client (not
>> svn! a downlevel client) would throw in a couple PROPFINDs, and if we
>> reach a bit, then some autoversioning around PUT and DELETE.
>>
>> IOW, what I might suggest is a mesh of your simplified protocol, with
>> the related DAV support for Windows, Mac, Linux, and other software
>> DAV-users. An admin could install mod_svn and get speed *and* DAV
>> capability.
>>
>
> I agree that a SVN DAV plugin for dumb clients is a good thing ... well,
> a major selling point in the non-techie parts of the corporate
> environment, heh. But cramming both into a single httpd module seems
> like serious overkill and suspiciously close to what we have now.
> Unmaintainable nightmare, don't y'know. Keep 'em separate and simple.
Possibly. Downlevel support is not that hard if it is read-only, so
the answer might be to scrap auto-versioning from the "DAV add-on"
portion. I'd still be curious about auto-versioning, and what the
additional work *really* means.
Recall that svnserve was started as a simplification over Apache. But
since then, it has added multiple auth modes, logging, threading, and
all the other goodies that Apache was providing us that svnserve was
"going to stay away from in staying simple". I believe we may want to
consider auto-versioning from the start rather than bolting in on,
with the idea that mod_dav_svn gets deprecated, and mod_svn becomes
the new HTTP/DAV recommendation. And in that regard, the biz users
will *really* want the auto-versioning.
>> And yes, I want cachability of (rev, path) nodes. If a request can be
>> satisfied locally, then that is a huge win. You say "serve out of
>> RAM", and both will do that. But a proxy can do it on your LAN rather
>> than the server via the WAN.
>
> The problem is of course that most of the traffic from an SVN repository
> server, at least as far as I've seen, is *not* (rev, path) requests but
> (rev, rev, path) that return a delta. Since no two working copies are
Actually, the second rev *is* there. X-SVN-Delta-Base or some header
like that. And the response has a Vary: header with that in there. A
proxy can use that to cache the (rev, rev, path) response. And the
good ones *do* look at that Vary header. They basically *have* to in
order to function properly.
> likely to be in the same state at any one time, that implies an order of
> magnitude more caching required -- and the better solution then is a
> local slave repository that serves the read-only requests.
I'd actually disagree somewhat here. How often does a *given* file
change? Sure, from update to update, there may be 20 revisions. But
file X might not get changed for a while, so working copies A, B, and
C all have that same, old version, and the 3-tuple will be the same
for all of them during an update.
This is precisely why the "delta base" URL is manufactured from the
*changed-rev* rather than the base-rev.
Cheers,
-g
RE: svn commit: r33366 - trunk/notes
Posted by Bert Huijben <b....@competence.biz>.
> -----Original Message-----
> From: Branko Čibej <br...@xbc.nu> [mailto:=?ISO-8859-
> 2?Q?Branko_=C8ibej_<br...@xbc.nu>?=]
> Sent: woensdag 1 oktober 2008 2:46
> To: Greg Stein
> Cc: Ben Collins-Sussman; dev@subversion.tigris.org
> Subject: Re: svn commit: r33366 - trunk/notes
>
> Greg Stein wrote:
> > In general, I'm not crazy-opposed. You're entirely right: the vision
> > of WebDAV (or "WebDA") came to fruition. DeltaV did not, so
> attempting
> > to adhere strongly to DeltaV really makes little pragmatic sense.
> >
> > Within the scope of the (new) design, I *do* think it would be
> > interesting to make it DAV-capable. i.e. is the URL namespace both
> > DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
> > maintain you could probably mesh the two pretty easily. The "new"
> > client would do some interesting GETs and POSTs, and a DAV client
> (not
> > svn! a downlevel client) would throw in a couple PROPFINDs, and if we
> > reach a bit, then some autoversioning around PUT and DELETE.
> >
> > IOW, what I might suggest is a mesh of your simplified protocol, with
> > the related DAV support for Windows, Mac, Linux, and other software
> > DAV-users. An admin could install mod_svn and get speed *and* DAV
> > capability.
> >
>
> I agree that a SVN DAV plugin for dumb clients is a good thing ...
> well,
> a major selling point in the non-techie parts of the corporate
> environment, heh. But cramming both into a single httpd module seems
> like serious overkill and suspiciously close to what we have now.
> Unmaintainable nightmare, don't y'know. Keep 'em separate and simple.
Keeping the implementations apart would be a big plus, but it would be a real nice to have if we could keep existing repository Urls compatible with both new and old clients; automatically upgrading to the new protocol if possible.
I would prefer not to create two repository URLs for the same project. One for pre 1.X clients+slow access and one for 1.X+ clients for faster access.
I see using a single Url to refer to a repository and a file (and with a peg revision as an immutable reference to a file-version) as one of the major advantages of using Subversion.
Moving to a new url scheme breaks this use case, as I would have to support both urls indefinitely:
Here at TCG we annotate all debugging symbols (.pdb files) with this information on all source files to be able to retrieve the exact source files on debugging (this could be years after building the binary). My Visual Studio debugger automatically downloads the source files from Subversion if it can't find the exact file version locally..
I can't switch the old location to the new protocol in a big bang to support older Subversion releases. And if I introduce a new url for the repository I have to maintain this url forever.
If all requests are routed over the repository root Url it shouldn't be too hard to handle the new style requests on the same public reposity Url as the old style requests.
Just allowing an apache rewrite rule to handle the conversion might be an option, but wht can't we just use a part of the Url namespace that isn't used by mod_dav_svn, for the new mod_svn?
Bert
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On Wed, Oct 1, 2008 at 10:54 AM, Mark Phippard <ma...@gmail.com> wrote:
> 1) It is likely we will get to a point that we do not add new features
> to mod_dav_svn since the new client will be speaking the new protocol.
> Therefore, we will have a somewhat confusion situation where users
> will "upgrade" their server and see it not supporting new features.
> Basically, because they missed the part where they have to change
> their configuration. Updating the existing module makes this much
> simpler for users.
Man, you are one step ahead of me, I was *just* about to write about this. :-)
One of the condundrums of implementing a separate mod_svn module is
deciding how we market it to users. Would mod_svn be the new
'officially supported' module where all new server-side features
(going forward) get implemented? If so, then admin would be
*required* to install mod_svn to get any new features, and mod_dav_svn
would just become an unchanging thing that can be (optionally)
installed to provide backward compatibility with older clients. On
the other hand, we could instead advertise mod_svn as the "optional"
thing, existing only to provide a speed boost when installed on top of
mod_dav_svn. Then we'd have to make new server-side features
available in both modules.
Personally, I prefer the former over the latter. But, as you say,
there may be a bit of confusion when admins upgrade. I don't think it
would be *big* amount of confusion, though. Asking admins to "load an
extra module" doesn't seem any more disruptive then asking them to run
'svnadmin upgrade' on their repositories.
>
> 2) It seems to me that using the existing module gives us complete
> freedom to add this feature incrementally.
I think the freedom is basically the same either way -- mod_svn can
grow incrementally whether it's an independent module (using DECLINE
to pass requests to mod_dav_svn), or whether it's embedded inside
mod_dav_svn (and we pass requests around internally).
> Wouldn't that give us just as much separation of code complexity? Or
> as I said before, does the mere fact that we are a DAV module
> interject complexity into the new code?
Yes, having mod_dav act as an uber-gateway does have some weird
effects, and may affect the implementation of the new protocol. It's
a monkey I'd like off our back, if possible. The other advantage to
having mod_svn as an indpendent module is that it allows for better
testing. In theory, our whole client-server should be able to pass
when it's the only module present; and when tests fail, it's *much*
easier to debug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Oct 1, 2008 at 11:44 AM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> On Wed, Oct 1, 2008 at 10:08 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>
>> Two apache modules could easy handle the same public url. You just to
>> register handler hook before mod_dav_svn and then return DECLINED if
>> url is not accepted. For example if it doesn't have cmd=something in
>> args. And return OK in other cases. mod_svn_view module uses this
>> approach for implementing Subversion web interface on the same URLs as
>> Subversion.
>
> Aha! I completely forgot about this option. Thanks Ivan!
>
> This strategy gives us the best of both worlds. It allows us to write
> a 'clean' mod_svn module with no baggage, minimal code, totally
> streamlined and aligned with svn_ra.h. But it also allows mod_dav_svn
> to coexist and speak the older protocol at the same public URL.
>
> If an admin wants to upgrade to the faster protocol, he just installs
> mod_svn next to mod_dav_svn, then asks his users to upgrade their
> clients when they're ready.
>
> Am I missing anything here?
I am fine with this. Having the single URL space is most important.
That said, let me play devil's advocate:
1) It is likely we will get to a point that we do not add new features
to mod_dav_svn since the new client will be speaking the new protocol.
Therefore, we will have a somewhat confusion situation where users
will "upgrade" their server and see it not supporting new features.
Basically, because they missed the part where they have to change
their configuration. Updating the existing module makes this much
simpler for users.
2) It seems to me that using the existing module gives us complete
freedom to add this feature incrementally. Tackle the places where we
get the biggest gains first and add more over time. Do we really need
to write new code to handle lock/unlock and other less used commands
right away? Can't the code be partitioned in such a way that when a
request comes into our module the URL is analyzed and routed to either
the old or new code based on whether the new code can handle it?
Wouldn't that give us just as much separation of code complexity? Or
as I said before, does the mere fact that we are a DAV module
interject complexity into the new code?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
Sorry, I should have been clearer. I wasn't objecting to Ivan's
suggestion, just confused about why he was thinking about building
URIs manually. :-)
On Wed, Oct 1, 2008 at 2:49 PM, Greg Stein <gs...@gmail.com> wrote:
> I'm with Ivan: put any path into the URL path, not as a parameter.
>
> On Wed, Oct 1, 2008 at 12:48 PM, Ben Collins-Sussman
> <su...@red-bean.com> wrote:
>> On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>>
>>> This is great. Only one thing I'm missing what is the reason to use
>>> repository root and encode path parameter in query args? From my point
>>> of it's much better to work on target URL, without knowledge of
>>> repository root. I mean:
>>> GET /repos/trunk/foo.c?cmd=get-file&r=23
>>> instead of
>>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>>
>>> Because resolving repository root takes additional workaround unless
>>> we have working copy.
>>
>> ? I don't understand. Who has to 'resolve' a repository root?
>>
>> I'm focused on aligning our HTTP requests exactly with RA apis.
>> Every RA api takes an ra_session object which represents the 'root' of
>> a repository operation. Granted, it's not necessarily always the root
>> of the repository, but I think we should preserve the same concepts.
>> The HTTP request should be against a URI that matches whatever
>> ra_session is anchored on; if a &path parameter is present, then it
>> should considered relative to the anchor.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: dev-help@subversion.tigris.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
I'm with Ivan: put any path into the URL path, not as a parameter.
On Wed, Oct 1, 2008 at 12:48 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>
>> This is great. Only one thing I'm missing what is the reason to use
>> repository root and encode path parameter in query args? From my point
>> of it's much better to work on target URL, without knowledge of
>> repository root. I mean:
>> GET /repos/trunk/foo.c?cmd=get-file&r=23
>> instead of
>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>
>> Because resolving repository root takes additional workaround unless
>> we have working copy.
>
> ? I don't understand. Who has to 'resolve' a repository root?
>
> I'm focused on aligning our HTTP requests exactly with RA apis.
> Every RA api takes an ra_session object which represents the 'root' of
> a repository operation. Granted, it's not necessarily always the root
> of the repository, but I think we should preserve the same concepts.
> The HTTP request should be against a URI that matches whatever
> ra_session is anchored on; if a &path parameter is present, then it
> should considered relative to the anchor.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
Well, you *should* be able to return the UUID of the repository for
non-existent paths under a repos root. The sticky point is simply that
the "proper" response to the PROPFIND in this case is a 404, rather
than a 207 with the UUID. But from an Apache standpoint, the
repos-root is there and easily seen.
Cheers,
-g
On Thu, Oct 2, 2008 at 3:12 AM, Lieven Govaerts <sv...@mobsol.be> wrote:
> Ivan Zhakov wrote:
>> On Wed, Oct 1, 2008 at 11:48 PM, Ben Collins-Sussman
>> <su...@red-bean.com> wrote:
>>> On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>>>
>>>> This is great. Only one thing I'm missing what is the reason to use
>>>> repository root and encode path parameter in query args? From my point
>>>> of it's much better to work on target URL, without knowledge of
>>>> repository root. I mean:
>>>> GET /repos/trunk/foo.c?cmd=get-file&r=23
>>>> instead of
>>>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>>>
>>>> Because resolving repository root takes additional workaround unless
>>>> we have working copy.
>>> ? I don't understand. Who has to 'resolve' a repository root?
>> (sorry for my english)
>>
>> I meant that serf often resolves real repository root using
>> svn_ra_serf__discover_root() and then requests some properties like
>> repository UUID only on root of repository.
>
> You don't need the repo-root to get the UUID of a repo, you need to know
> a path in rHEAD in that repo. Problem is, if you're working on
> path@PEGREV where path does not exist in HEAD you don't know such a
> path. So, as workaround, ra_serf first finds out the repository root
> because that's the only path that's sure to exist in HEAD.
>
> Lieven
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Lieven Govaerts <sv...@mobsol.be>.
Ivan Zhakov wrote:
> On Wed, Oct 1, 2008 at 11:48 PM, Ben Collins-Sussman
> <su...@red-bean.com> wrote:
>> On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>>
>>> This is great. Only one thing I'm missing what is the reason to use
>>> repository root and encode path parameter in query args? From my point
>>> of it's much better to work on target URL, without knowledge of
>>> repository root. I mean:
>>> GET /repos/trunk/foo.c?cmd=get-file&r=23
>>> instead of
>>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>>
>>> Because resolving repository root takes additional workaround unless
>>> we have working copy.
>> ? I don't understand. Who has to 'resolve' a repository root?
> (sorry for my english)
>
> I meant that serf often resolves real repository root using
> svn_ra_serf__discover_root() and then requests some properties like
> repository UUID only on root of repository.
You don't need the repo-root to get the UUID of a repo, you need to know
a path in rHEAD in that repo. Problem is, if you're working on
path@PEGREV where path does not exist in HEAD you don't know such a
path. So, as workaround, ra_serf first finds out the repository root
because that's the only path that's sure to exist in HEAD.
Lieven
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
I think that most cases would simply use 2 repository paths, rather
than 2 URLs. You could have the target of the operation as the main
URL path, and the supporting URL converted to a repository-relative
path.
One day, if we ever do cross-repository work from the server side,
then yes... you'd bury the two URLs into the body of a POST.
Cheers,
-g
On Wed, Oct 1, 2008 at 3:40 PM, Mark Phippard <ma...@gmail.com> wrote:
> On Wed, Oct 1, 2008 at 5:18 PM, Ben Collins-Sussman
> <su...@red-bean.com> wrote:
>
>> Actually, gstein just persuaded me (in IRC) to never have a &path
>> query parameter -- it's too weird and "un-URL-ish".
>
> Would requests that need 2 URL's use POST?
>
> --
> Thanks
>
> Mark Phippard
> http://markphip.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Oct 1, 2008 at 5:18 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> Actually, gstein just persuaded me (in IRC) to never have a &path
> query parameter -- it's too weird and "un-URL-ish".
Would requests that need 2 URL's use POST?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On Wed, Oct 1, 2008 at 3:08 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
> On Wed, Oct 1, 2008 at 11:48 PM, Ben Collins-Sussman
> <su...@red-bean.com> wrote:
>> On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>>
>>> This is great. Only one thing I'm missing what is the reason to use
>>> repository root and encode path parameter in query args? From my point
>>> of it's much better to work on target URL, without knowledge of
>>> repository root. I mean:
>>> GET /repos/trunk/foo.c?cmd=get-file&r=23
>>> instead of
>>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>>
>>> Because resolving repository root takes additional workaround unless
>>> we have working copy.
>>
>> ? I don't understand. Who has to 'resolve' a repository root?
> (sorry for my english)
>
> I meant that serf often resolves real repository root using
> svn_ra_serf__discover_root() and then requests some properties like
> repository UUID only on root of repository.
In the new system, svn_ra_serf__discover_root() wouldn't need to do a
bunch of PROPFINDs, it could just do a single GET
/repos/path?cmd=get_repos_root
>
>>
>> I'm focused on aligning our HTTP requests exactly with RA apis.
>> Every RA api takes an ra_session object which represents the 'root' of
>> a repository operation. Granted, it's not necessarily always the root
>> of the repository,
> "The administrator makes an svn repository available at a specific
> URI, which we'll refer to as the "repository root URI"."
>
> Sorry I was confused with term "repository root URI" in design notes
> which is actually "session root (anchor) URI". Maybe you'd like to
> change term in design notes to prevent such confusion.
Will do. I'm busy refining the doc right now.
>
>> but I think we should preserve the same concepts.
>> The HTTP request should be against a URI that matches whatever
>> ra_session is anchored on; if a &path parameter is present, then it
>> should considered relative to the anchor.
>>
> I'm OK with this approach if root URI don't have to be root of repository.
>
> I see only one minor problem: the same data can be accessed from different URIs:
> GET /repos/trunk?cmd=get-file&r=23&path=foo.c
> or
> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>
> It could make testing/debugging of server-side harder, because will
> have to care about all cases every time.
Actually, gstein just persuaded me (in IRC) to never have a &path
query parameter -- it's too weird and "un-URL-ish".
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Wed, Oct 1, 2008 at 11:48 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>
>> This is great. Only one thing I'm missing what is the reason to use
>> repository root and encode path parameter in query args? From my point
>> of it's much better to work on target URL, without knowledge of
>> repository root. I mean:
>> GET /repos/trunk/foo.c?cmd=get-file&r=23
>> instead of
>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>
>> Because resolving repository root takes additional workaround unless
>> we have working copy.
>
> ? I don't understand. Who has to 'resolve' a repository root?
(sorry for my english)
I meant that serf often resolves real repository root using
svn_ra_serf__discover_root() and then requests some properties like
repository UUID only on root of repository.
>
> I'm focused on aligning our HTTP requests exactly with RA apis.
> Every RA api takes an ra_session object which represents the 'root' of
> a repository operation. Granted, it's not necessarily always the root
> of the repository,
"The administrator makes an svn repository available at a specific
URI, which we'll refer to as the "repository root URI"."
Sorry I was confused with term "repository root URI" in design notes
which is actually "session root (anchor) URI". Maybe you'd like to
change term in design notes to prevent such confusion.
> but I think we should preserve the same concepts.
> The HTTP request should be against a URI that matches whatever
> ra_session is anchored on; if a &path parameter is present, then it
> should considered relative to the anchor.
>
I'm OK with this approach if root URI don't have to be root of repository.
I see only one minor problem: the same data can be accessed from different URIs:
GET /repos/trunk?cmd=get-file&r=23&path=foo.c
or
GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
It could make testing/debugging of server-side harder, because will
have to care about all cases every time.
--
Ivan Zhakov
VisualSVN Team
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On Wed, Oct 1, 2008 at 2:38 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
> This is great. Only one thing I'm missing what is the reason to use
> repository root and encode path parameter in query args? From my point
> of it's much better to work on target URL, without knowledge of
> repository root. I mean:
> GET /repos/trunk/foo.c?cmd=get-file&r=23
> instead of
> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>
> Because resolving repository root takes additional workaround unless
> we have working copy.
? I don't understand. Who has to 'resolve' a repository root?
I'm focused on aligning our HTTP requests exactly with RA apis.
Every RA api takes an ra_session object which represents the 'root' of
a repository operation. Granted, it's not necessarily always the root
of the repository, but I think we should preserve the same concepts.
The HTTP request should be against a URI that matches whatever
ra_session is anchored on; if a &path parameter is present, then it
should considered relative to the anchor.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Wed, Oct 1, 2008 at 7:44 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> On Wed, Oct 1, 2008 at 10:08 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>
>> Two apache modules could easy handle the same public url. You just to
>> register handler hook before mod_dav_svn and then return DECLINED if
>> url is not accepted. For example if it doesn't have cmd=something in
>> args. And return OK in other cases. mod_svn_view module uses this
>> approach for implementing Subversion web interface on the same URLs as
>> Subversion.
>
> Aha! I completely forgot about this option. Thanks Ivan!
>
> This strategy gives us the best of both worlds. It allows us to write
> a 'clean' mod_svn module with no baggage, minimal code, totally
> streamlined and aligned with svn_ra.h. But it also allows mod_dav_svn
> to coexist and speak the older protocol at the same public URL.
>
> If an admin wants to upgrade to the faster protocol, he just installs
> mod_svn next to mod_dav_svn, then asks his users to upgrade their
> clients when they're ready.
>
> Am I missing anything here?
>
This is great. Only one thing I'm missing what is the reason to use
repository root and encode path parameter in query args? From my point
of it's much better to work on target URL, without knowledge of
repository root. I mean:
GET /repos/trunk/foo.c?cmd=get-file&r=23
instead of
GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
Because resolving repository root takes additional workaround unless
we have working copy.
Another bikeshed comment: I propose use 'svn-cmd' instead of 'cmd' to
be able serve several services (Subversion web access for example) on
the same URL.
--
Ivan Zhakov
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On Wed, Oct 1, 2008 at 10:08 AM, Ivan Zhakov <iv...@visualsvn.com> wrote:
> Two apache modules could easy handle the same public url. You just to
> register handler hook before mod_dav_svn and then return DECLINED if
> url is not accepted. For example if it doesn't have cmd=something in
> args. And return OK in other cases. mod_svn_view module uses this
> approach for implementing Subversion web interface on the same URLs as
> Subversion.
Aha! I completely forgot about this option. Thanks Ivan!
This strategy gives us the best of both worlds. It allows us to write
a 'clean' mod_svn module with no baggage, minimal code, totally
streamlined and aligned with svn_ra.h. But it also allows mod_dav_svn
to coexist and speak the older protocol at the same public URL.
If an admin wants to upgrade to the faster protocol, he just installs
mod_svn next to mod_dav_svn, then asks his users to upgrade their
clients when they're ready.
Am I missing anything here?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Wed, Oct 1, 2008 at 6:13 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> 2008/10/1 Bert Huijben <B....@competence.biz>:
>
>
>> I think providing both mod_dav_svn and mod_svn over the same public url encourages users to upgrade their servers to support mod_svn, while requiring all users to upgrade their clients before the migration will hold the server upgrade until the adminstrators are sure all users have updated.
>> Which might take a couple of years for big hosters like sourceforge.
>
> I agree this would be ideal, but it's not technically possible. As I
> said before, two modules cannot both "own" the same public url. If
> you have a brilliant idea on how to implement this, I'm all ears.
Two apache modules could easy handle the same public url. You just to
register handler hook before mod_dav_svn and then return DECLINED if
url is not accepted. For example if it doesn't have cmd=something in
args. And return OK in other cases. mod_svn_view module uses this
approach for implementing Subversion web interface on the same URLs as
Subversion.
--
Ivan Zhakov
VisualSVN Team
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
2008/10/1 Bert Huijben <B....@competence.biz>:
> I think providing both mod_dav_svn and mod_svn over the same public url encourages users to upgrade their servers to support mod_svn, while requiring all users to upgrade their clients before the migration will hold the server upgrade until the adminstrators are sure all users have updated.
> Which might take a couple of years for big hosters like sourceforge.
I agree this would be ideal, but it's not technically possible. As I
said before, two modules cannot both "own" the same public url. If
you have a brilliant idea on how to implement this, I'm all ears.
> Disabling old clients should be a user decision, not ours.
You're preaching to the choir... no need to write many paragraphs
explaining why this is a desirable goal. :-) I just can't see how to
make it happen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: svn commit: r33366 - trunk/notes
Posted by Bert Huijben <B....@competence.biz>.
> -----Original Message-----
> From: sussman@gmail.com [mailto:sussman@gmail.com] On Behalf Of Ben
> Collins-Sussman
> Sent: woensdag 1 oktober 2008 14:09
> To: Bert Huijben
> Cc: dev@subversion.tigris.org
> Subject: Re: svn commit: r33366 - trunk/notes
>
> On Wed, Oct 1, 2008 at 2:26 AM, Bert Huijben <b....@competence.biz>
> wrote:
> >> -----Original Message-----
> >> From: Branko Čibej <br...@xbc.nu> [mailto:=?ISO-8859-
> >> 2?Q?Branko_=C8ibej_<br...@xbc.nu>?=]
> >> Sent: woensdag 1 oktober 2008 2:46
> >> To: Greg Stein
> >> Cc: Ben Collins-Sussman; dev@subversion.tigris.org
> >> Subject: Re: svn commit: r33366 - trunk/notes
> >>
> >> Greg Stein wrote:
> >> > In general, I'm not crazy-opposed. You're entirely right: the
> vision
> >> > of WebDAV (or "WebDA") came to fruition. DeltaV did not, so
> >> attempting
> >> > to adhere strongly to DeltaV really makes little pragmatic sense.
> >> >
> >> > Within the scope of the (new) design, I *do* think it would be
> >> > interesting to make it DAV-capable. i.e. is the URL namespace both
> >> > DAV-aware *and* svn-aware? Given that DAV does not use POST, then
> I
> >> > maintain you could probably mesh the two pretty easily. The "new"
> >> > client would do some interesting GETs and POSTs, and a DAV client
> >> (not
> >> > svn! a downlevel client) would throw in a couple PROPFINDs, and if
> we
> >> > reach a bit, then some autoversioning around PUT and DELETE.
> >> >
> >> > IOW, what I might suggest is a mesh of your simplified protocol,
> with
> >> > the related DAV support for Windows, Mac, Linux, and other
> software
> >> > DAV-users. An admin could install mod_svn and get speed *and* DAV
> >> > capability.
> >> >
> >>
> >> I agree that a SVN DAV plugin for dumb clients is a good thing ...
> >> well,
> >> a major selling point in the non-techie parts of the corporate
> >> environment, heh. But cramming both into a single httpd module seems
> >> like serious overkill and suspiciously close to what we have now.
> >> Unmaintainable nightmare, don't y'know. Keep 'em separate and
> simple.
> >
> > Keeping the implementations apart would be a big plus, but it would
> be a real nice to have if we could keep existing repository Urls
> compatible with both new and old clients; automatically upgrading to
> the new protocol if possible.
>
> I don't see how this is technically possible. There's no way to have
> two different apache modules "own" a single URI at the same time. So
> the only way that a URI could respond to both protocols would be to
> (somehow) hack the entire new protocol into the existing mod_dav_svn
> module.... and that would be a nightmare.
>
> > I see using a single Url to refer to a repository and a file (and
> with a peg revision as an immutable reference to a file-version) as one
> of the major advantages of using Subversion.
>
> The *public* syntax used by an svn client isn't going to change. In
> other words, "svn list http://host/repos/path@23" will still do the
> same thing, regardless of whether we're using old or new protocols.
> What's invisible to the user (or to Visual Studio) is how the svn
> client library converts "http://host/repos/path@23" into an *internal*
> HTTP request. In the old protocol, the public URL is converted into a
> whole bunch of PROPFIND requests, followed by a 'GET
> /repos/!svn/bc/23/path'. In the new protocol, the public URL would be
> converted into a single 'GET /repos?cmd=get-file&path=/path&rev=23'
> (or something similar.)
>
> The point is, you could just switch your apache server from
> mod_dav_svn to mod_svn, and as long as you upgrade your svn clients,
> the same old working-copy URLs should keep working. We're only
> changing the internal protocol, not the URLs that users use
> day-to-day.
I'm not talking about the internal urls. My users never see the !svn or ?cmd urls, unless there is a network problem and neon shows the raw error.
I'm just talking about the public Urls.
My repositories are currently on https://subversion.<company>.com/svn/netdev/<projectgroup>/<project>/trunk/
And I would like to keep using that exact public Url with both /new/ and /old/ subversion clients; upgrading to the enhanced protocol when possible.
(The requests libsvn* sends and how they are formatted don't matter to our users, but the public repository Urls shown in UIs do)
If I would manage a commercial subversion hosting company I would prefer a single Url that 'just works'. Instead of:
* http://.../svn/<repos> for pre 1.X
* http://.../fastsvn/<repos> for 1.X+
Or if I would take googlecode as an example also:
* https://.../svn/<repos> for read+write pre 1.X
and
* https://.../fastsvn/<repos> for read+write 1.X+
I think providing both mod_dav_svn and mod_svn over the same public url encourages users to upgrade their servers to support mod_svn, while requiring all users to upgrade their clients before the migration will hold the server upgrade until the adminstrators are sure all users have updated.
Which might take a couple of years for big hosters like sourceforge.
I don't think returning the new fastsvn service connection point Url from the initial DAV request like we do with the '!svn' now, will cause real problems. And it would allow us to keep the existing public urls working for both implementations.
Another example:
When would we switch http://svn.collab.net/repos/svn/ over to mod_svn?
I personally would like that url to keep working with all 1.X subversion clients, old and new. Providing the best experience for everybody.
Formally we wouldn't break our API if we make existing urls inaccessible to old clients, but I think many users will see this in a different way.
And I don't think it should be really hard to keep things working in this scenario.
Disabling old clients should be a user decision, not ours.
Bert
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On Wed, Oct 1, 2008 at 2:26 AM, Bert Huijben <b....@competence.biz> wrote:
>> -----Original Message-----
>> From: Branko Čibej <br...@xbc.nu> [mailto:=?ISO-8859-
>> 2?Q?Branko_=C8ibej_<br...@xbc.nu>?=]
>> Sent: woensdag 1 oktober 2008 2:46
>> To: Greg Stein
>> Cc: Ben Collins-Sussman; dev@subversion.tigris.org
>> Subject: Re: svn commit: r33366 - trunk/notes
>>
>> Greg Stein wrote:
>> > In general, I'm not crazy-opposed. You're entirely right: the vision
>> > of WebDAV (or "WebDA") came to fruition. DeltaV did not, so
>> attempting
>> > to adhere strongly to DeltaV really makes little pragmatic sense.
>> >
>> > Within the scope of the (new) design, I *do* think it would be
>> > interesting to make it DAV-capable. i.e. is the URL namespace both
>> > DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
>> > maintain you could probably mesh the two pretty easily. The "new"
>> > client would do some interesting GETs and POSTs, and a DAV client
>> (not
>> > svn! a downlevel client) would throw in a couple PROPFINDs, and if we
>> > reach a bit, then some autoversioning around PUT and DELETE.
>> >
>> > IOW, what I might suggest is a mesh of your simplified protocol, with
>> > the related DAV support for Windows, Mac, Linux, and other software
>> > DAV-users. An admin could install mod_svn and get speed *and* DAV
>> > capability.
>> >
>>
>> I agree that a SVN DAV plugin for dumb clients is a good thing ...
>> well,
>> a major selling point in the non-techie parts of the corporate
>> environment, heh. But cramming both into a single httpd module seems
>> like serious overkill and suspiciously close to what we have now.
>> Unmaintainable nightmare, don't y'know. Keep 'em separate and simple.
>
> Keeping the implementations apart would be a big plus, but it would be a real nice to have if we could keep existing repository Urls compatible with both new and old clients; automatically upgrading to the new protocol if possible.
I don't see how this is technically possible. There's no way to have
two different apache modules "own" a single URI at the same time. So
the only way that a URI could respond to both protocols would be to
(somehow) hack the entire new protocol into the existing mod_dav_svn
module.... and that would be a nightmare.
> I see using a single Url to refer to a repository and a file (and with a peg revision as an immutable reference to a file-version) as one of the major advantages of using Subversion.
The *public* syntax used by an svn client isn't going to change. In
other words, "svn list http://host/repos/path@23" will still do the
same thing, regardless of whether we're using old or new protocols.
What's invisible to the user (or to Visual Studio) is how the svn
client library converts "http://host/repos/path@23" into an *internal*
HTTP request. In the old protocol, the public URL is converted into a
whole bunch of PROPFIND requests, followed by a 'GET
/repos/!svn/bc/23/path'. In the new protocol, the public URL would be
converted into a single 'GET /repos?cmd=get-file&path=/path&rev=23'
(or something similar.)
The point is, you could just switch your apache server from
mod_dav_svn to mod_svn, and as long as you upgrade your svn clients,
the same old working-copy URLs should keep working. We're only
changing the internal protocol, not the URLs that users use
day-to-day.
Re: svn commit: r33366 - trunk/notes
Posted by Branko Čibej <br...@xbc.nu>.
Greg Stein wrote:
> In general, I'm not crazy-opposed. You're entirely right: the vision
> of WebDAV (or "WebDA") came to fruition. DeltaV did not, so attempting
> to adhere strongly to DeltaV really makes little pragmatic sense.
>
> Within the scope of the (new) design, I *do* think it would be
> interesting to make it DAV-capable. i.e. is the URL namespace both
> DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
> maintain you could probably mesh the two pretty easily. The "new"
> client would do some interesting GETs and POSTs, and a DAV client (not
> svn! a downlevel client) would throw in a couple PROPFINDs, and if we
> reach a bit, then some autoversioning around PUT and DELETE.
>
> IOW, what I might suggest is a mesh of your simplified protocol, with
> the related DAV support for Windows, Mac, Linux, and other software
> DAV-users. An admin could install mod_svn and get speed *and* DAV
> capability.
>
I agree that a SVN DAV plugin for dumb clients is a good thing ... well,
a major selling point in the non-techie parts of the corporate
environment, heh. But cramming both into a single httpd module seems
like serious overkill and suspiciously close to what we have now.
Unmaintainable nightmare, don't y'know. Keep 'em separate and simple.
> And yes, I want cachability of (rev, path) nodes. If a request can be
> satisfied locally, then that is a huge win. You say "serve out of
> RAM", and both will do that. But a proxy can do it on your LAN rather
> than the server via the WAN.
The problem is of course that most of the traffic from an SVN repository
server, at least as far as I've seen, is *not* (rev, path) requests but
(rev, rev, path) that return a delta. Since no two working copies are
likely to be in the same state at any one time, that implies an order of
magnitude more caching required -- and the better solution then is a
local slave repository that serves the read-only requests.
Just sayin'. There's nothing wrong in principle with making the new
protocol proxy-friendly.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Branko Čibej <br...@xbc.nu>.
Ben Collins-Sussman wrote:
> There's still an open question in my mind as to whether an 'svn
> checkout' would be single proprietary GET request and massive response
> (as RA models it), or whether we could use a hybrid of the core & dav
> protocols (GET a skelta, then GET each individual file via
> pipelining.) We can work that out.
>
Anything that amortizes connection and request latency is good. Also
keep in mind that, for the SSL/TLS case, you want too keep the number of
handshajes to a minimum.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
On Tue, Sep 30, 2008 at 5:20 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> OK, I think I'm game for this plan. I can start revising the
> designdoc in this direction:
>
> 1. We can design a 'core' protocol which is basically a direct mapping
> of RA calls to (proprietary) single-turnaround GET and POST requests.
> No mess, no fuss. Lean and mean.
Sounds good.
> 2. We can also throw in a secondary protocol which is pure WebDAV (no
> DeltaV), so that drives are still mountable via {GET, PUT, PROPFIND,
> PROPPATCH, COPY, MOVE, MKCOL, LOCK, UNLOCK}; we'd design the URI
> space to map *sanely* to svn objects, and make sure that immutable
> objects are explicitly flagged as cacheable. The URI spaces will be
> publically advertised as well. This would preserve autoversioning, as
> well as the built-in web-browsing feature.
Yup. Basically, just concentrating on what an arbitrary DAV client
will use... *actual* Subversion operations use the gunk from (1).
> There's still an open question in my mind as to whether an 'svn
> checkout' would be single proprietary GET request and massive response
> (as RA models it), or whether we could use a hybrid of the core & dav
> protocols (GET a skelta, then GET each individual file via
> pipelining.) We can work that out.
The latter. Reason: the client might already have all of the files
stashed away in its new checksum-keyed datastore of pristine files!
:-D (aka wc-ng)
Ideally, the server would respond with: "fetch <these> contents, and
here's their checksum values". The client can fetch or not, based on
whether it already has the pristine file. The individual fetches can
be pipelined (we *have* identified that as similar-perf to
mother-REPORT, *and* they're cacheable this way).
Cheers,
-g
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Ben Collins-Sussman <su...@red-bean.com>.
OK, I think I'm game for this plan. I can start revising the
designdoc in this direction:
1. We can design a 'core' protocol which is basically a direct mapping
of RA calls to (proprietary) single-turnaround GET and POST requests.
No mess, no fuss. Lean and mean.
2. We can also throw in a secondary protocol which is pure WebDAV (no
DeltaV), so that drives are still mountable via {GET, PUT, PROPFIND,
PROPPATCH, COPY, MOVE, MKCOL, LOCK, UNLOCK}; we'd design the URI
space to map *sanely* to svn objects, and make sure that immutable
objects are explicitly flagged as cacheable. The URI spaces will be
publically advertised as well. This would preserve autoversioning, as
well as the built-in web-browsing feature.
There's still an open question in my mind as to whether an 'svn
checkout' would be single proprietary GET request and massive response
(as RA models it), or whether we could use a hybrid of the core & dav
protocols (GET a skelta, then GET each individual file via
pipelining.) We can work that out.
Keep in mind that the reason I'm volunteering to work on this is that
it makes for a good 20% project -- and also a really good "I'm about
to go on 4+ weeks of paternity leave" project. :-)
On Tue, Sep 30, 2008 at 6:00 PM, Greg Stein <gs...@gmail.com> wrote:
> In general, I'm not crazy-opposed. You're entirely right: the vision
> of WebDAV (or "WebDA") came to fruition. DeltaV did not, so attempting
> to adhere strongly to DeltaV really makes little pragmatic sense.
>
> Within the scope of the (new) design, I *do* think it would be
> interesting to make it DAV-capable. i.e. is the URL namespace both
> DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
> maintain you could probably mesh the two pretty easily. The "new"
> client would do some interesting GETs and POSTs, and a DAV client (not
> svn! a downlevel client) would throw in a couple PROPFINDs, and if we
> reach a bit, then some autoversioning around PUT and DELETE.
>
> IOW, what I might suggest is a mesh of your simplified protocol, with
> the related DAV support for Windows, Mac, Linux, and other software
> DAV-users. An admin could install mod_svn and get speed *and* DAV
> capability.
>
> And yes, I want cachability of (rev, path) nodes. If a request can be
> satisfied locally, then that is a huge win. You say "serve out of
> RAM", and both will do that. But a proxy can do it on your LAN rather
> than the server via the WAN. You say "nobody has done it", but that is
> an incorrect argument. Nobody has been ABLE to do it because we
> glommed shit into a mother REPORT. And serf needs some more work
> before people can TRY it. So I respectfully disagree with any argument
> about cachability being a non-requirement.
>
> Cheers,
> -g
>
> On Tue, Sep 30, 2008 at 3:30 PM, Ben Collins-Sussman
> <su...@red-bean.com> wrote:
>> On Tue, Sep 30, 2008 at 3:24 PM, <gs...@tigris.org> wrote:
>>
>>> Unfortunately, DeltaV is an insanely complex and inefficient protocol,
>>> and doesn't fit Subversion's model well at all. The result is that
>>> Subversion speaks a "limited portion" of DeltaV, and pays a huge price
>>> for this complexity: speed.
>>>
>>> +### GJS: doesn't fit? heh. IMO, it does... you should have seen it
>>> + *before* I grok'd the design and started providing feedback. it
>>> + just doesn't match it *precisely*
>>
>> Sure, I remember. But still, we only implement "just enough" DeltaV
>> to look like a DAV server to a dumb DAV client. But there are no
>> 3rd-party DeltaV clients that run against mod_dav_svn, and there are
>> no 3rd-party DeltaV servers that can talk to an svn client. Nearly
>> every single "interesting" svn_ra.h interface is done through a custom
>> REPORT -- checkout, update, log, etc. We've surrounded ourselves with
>> DeltaV formalities that provide a lot of complexity and zero value.
>>
>>
>>> +
>>> A typical network trace involves dozens of unnecessary turnarounds
>>> where the client keeps asking for the same information over and over
>>> again, all for the sake of following DeltaV. And then once the client
>>> @@ -33,6 +38,19 @@ has "discovered" the information it need
>>> custom REPORT request anyway. Most svn operations are at least twice
>>> as slow over HTTP than over the custom svnserve protocol.
>>>
>>> +### GJS: that is the fault of the client, not the protocol. somewhere
>>> + around the time that I stopped working on SVN for a while, I
>>> + identified several things that an RA layer could cache in order to
>>> + avoid looking them up again. nobody ever implemented that. thus,
>>> + it is hard to claim the fault lies in the protocol when we KNOW we
>>> + don't have to re-fetch certain items.
>>
>> But this just makes our existing clients more and more complex. Why
>> are we doing PROPFINDs at all, ever? Unless we're actually trying to
>> implement svn_ra_get_prop(), every single PROPFIND we do before our
>> custom REPORTs are just weird formalities related to DeltaV discovery.
>> Even if we completely remove all the redundant PROPFIND requests, we
>> still get absolutely nothing out of following the DeltaV rules in
>> these cases.
>>
>>> +
>>> +### GJS: turnarounds don't have to be expensive if you can pipeline
>>> + them. that is one of the (design) reasons for Serf. Again,
>>> + unrelated to the protocol, but simply the implementation. And
>>> + proposing a *new* implementation rather than fixing an existing
>>> + one seems to be a much more monumental task.
>>
>> I have to do a reality check here: the promise of serf is that
>> checkouts/updates would be faster than neon, because we could pipeline
>> a bunch of GET requests rather than suck down the tree in one
>> response. In practice, serf has proven to be no faster than neon in
>> this regard. (Or if faster, only by a tiny percent.)
>>
>> I understand that the "real" promise here is that of caching proxy
>> servers, which will supposedly deliver serf's original speed promises
>> to us... but I've lost my faith in this idea:
>>
>> * nobody's actually done it yet and demonstrated it
>>
>> * given that BDB and/or FSFS is (typically) already serving popular
>> fs-nodes out of RAM (due to OS caching), I don't think a proxy-server
>> serving nodes out of RAM will be any faster.
>>
>>
>>> +
>>>
>>> PROPOSAL
>>> ========
>>> @@ -41,15 +59,22 @@ Write a new HTTP protocol for svn; map
>>> requests.
>>>
>>> * svn over HTTP would be much faster (eliminate turnarounds)
>>> +
>>> +### GJS: again: pipelining.
>>
>> Are we going to pipeline every single request in svn_ra.h? Does it
>> even make sense to do so? I don't view it as a magic bullet.
>>
>>
>>>
>>> * svn over HTTP would be almost as easy to extend as svnserve.
>>>
>>> +### GJS: have we had an issue with extending our current HTTP use?
>>
>> Let's ask folks who have implement the mod_dav_svn get-lock-report,
>> get-locations report, mergeinfo-report, etc.
>>
>> But meanwhile, I maintain it's not so much about extending as it is
>> about maintaining. Anyone should be able to understand what's
>> happening in the HTTP layer, debug it, rev an interface by adding a
>> new parameter. Nobody is scared of doing this in svnserve. To do
>> this in mod_dav_svn, people need a 3 hour lecture in the architecture
>> of the system, DeltaV terminology, etc.
>>
>>
>>> +
>>> * svn over HTTP would be comprehensible to devs and users both
>>> (require no knowledge of DeltaV concepts).
>>>
>>> * We'd still maintain almost all of Apache's advantages
>>> (propositions A through E above).
>>>
>>> +### GJS: and with the right design, should be able to get the G that I
>>> + added above.
>>> +
>>> * We'd give up proposition F, which hasn't given us anything but the
>>> ability to mount svn repositories as network drives. (For those
>>> who *really* need this rarely-used feature, mod_dav_svn would
>>> @@ -58,6 +83,11 @@ requests.
>>> * We could freely advertise a fixed syntax for browsing older
>>> revisions, without worrying about violating DeltaV.
>>>
>>> +### GJS: we can do this today. DeltaV certainly doesn't proscribe what
>>> + we can do in our URL namespace. my reluctance to advertise one was
>>> + to give us maximum flexibility on the server, and to defer browing
>>> + to external tools like ViewVC or Trac.
>>
>> Agreed, this feature isn't tied to a protocol rewrite at all. I'll
>> remove it from the doc.
>>
>>
>>
>>> +
>>>
>>> MILE-HIGH DESIGN
>>> ================
>>> @@ -124,6 +154,9 @@ DESIGN
>>>
>>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>>
>>> +### GJS: in order to allow for intermediate caches to work, the URLs
>>> + need to be well-designed up-front. I can help with that.
>>
>> Is caching really a goal? Why is it so important on your list?
>>
>>
>>> +
>>> For requests which require large data exchanges, the big payloads go
>>> into request and/or response bodies. For example:
>>>
>>> @@ -131,6 +164,11 @@ DESIGN
>>>
>>> [body contains complete 'update report' describing working copy's
>>> revisions; response is a complete editor-drive.]
>>> +
>>> +### GJS: no. I've never seen a GET request with a body. You want to
>>> + avoid that, or you could end up being incompatible with many
>>> + proxies. If you need input data, then make it a POST, or use one
>>> + of the other HTTP methods defined by HTTP, WebDAV, or DeltaV.
>>
>> Ah, ok, so any request bodies will probably need to be POSTs or somesuch.
>>
>>
>>
>>>
>>> POST /repos?cmd=commit&keeplocks=true
>>>
>>> @@ -170,6 +208,12 @@ DESIGN
>>> embedded s-expressions, exactly like svnserve does. Heck, maybe we
>>> could even use svnserve's parsing/unparsing code!)
>>>
>>> +### GJS: personally, I'd recommend protocol buffers, or Apache Thrift
>>> + (a third-party reimplementation of PBs done at facebook). just
>>> + toss out the formats used in FS and the svn protocol, and let a
>>> + new/external library do the marshalling for us.
>>> + http://stuartsierra.com/2008/07/10/thrift-vs-protocol-buffers
>>
>> Hmmm, interesting.
>>
>> Honestly, it sounds to me like you might be willing to toss away all
>> the DeltaV junk, just as long as we still make individual GETs of
>> (rev, path) objects (1) possible, (2) cacheable, (3) pipelineable.
>> Those seem to be the things that are important to you.
>>
>> If that's the case, I *still* think we need a new protocol, and a new
>> server and client to implement it. The idea is to start clean, build
>> the smallest possible protocol with the smallest amount of code,
>> directly corresponding with svn_ra.h as much as possible... and heck,
>> if we can get pipelining and cacheability in there too, great. But I
>> cannot comprehend achieving this goal by modifying the existing
>> codebase.... it would just be adding even more complexity to something
>> already incomprehensible to most of the svn developers.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: dev-help@subversion.tigris.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r33366 - trunk/notes
Posted by Greg Stein <gs...@gmail.com>.
In general, I'm not crazy-opposed. You're entirely right: the vision
of WebDAV (or "WebDA") came to fruition. DeltaV did not, so attempting
to adhere strongly to DeltaV really makes little pragmatic sense.
Within the scope of the (new) design, I *do* think it would be
interesting to make it DAV-capable. i.e. is the URL namespace both
DAV-aware *and* svn-aware? Given that DAV does not use POST, then I
maintain you could probably mesh the two pretty easily. The "new"
client would do some interesting GETs and POSTs, and a DAV client (not
svn! a downlevel client) would throw in a couple PROPFINDs, and if we
reach a bit, then some autoversioning around PUT and DELETE.
IOW, what I might suggest is a mesh of your simplified protocol, with
the related DAV support for Windows, Mac, Linux, and other software
DAV-users. An admin could install mod_svn and get speed *and* DAV
capability.
And yes, I want cachability of (rev, path) nodes. If a request can be
satisfied locally, then that is a huge win. You say "serve out of
RAM", and both will do that. But a proxy can do it on your LAN rather
than the server via the WAN. You say "nobody has done it", but that is
an incorrect argument. Nobody has been ABLE to do it because we
glommed shit into a mother REPORT. And serf needs some more work
before people can TRY it. So I respectfully disagree with any argument
about cachability being a non-requirement.
Cheers,
-g
On Tue, Sep 30, 2008 at 3:30 PM, Ben Collins-Sussman
<su...@red-bean.com> wrote:
> On Tue, Sep 30, 2008 at 3:24 PM, <gs...@tigris.org> wrote:
>
>> Unfortunately, DeltaV is an insanely complex and inefficient protocol,
>> and doesn't fit Subversion's model well at all. The result is that
>> Subversion speaks a "limited portion" of DeltaV, and pays a huge price
>> for this complexity: speed.
>>
>> +### GJS: doesn't fit? heh. IMO, it does... you should have seen it
>> + *before* I grok'd the design and started providing feedback. it
>> + just doesn't match it *precisely*
>
> Sure, I remember. But still, we only implement "just enough" DeltaV
> to look like a DAV server to a dumb DAV client. But there are no
> 3rd-party DeltaV clients that run against mod_dav_svn, and there are
> no 3rd-party DeltaV servers that can talk to an svn client. Nearly
> every single "interesting" svn_ra.h interface is done through a custom
> REPORT -- checkout, update, log, etc. We've surrounded ourselves with
> DeltaV formalities that provide a lot of complexity and zero value.
>
>
>> +
>> A typical network trace involves dozens of unnecessary turnarounds
>> where the client keeps asking for the same information over and over
>> again, all for the sake of following DeltaV. And then once the client
>> @@ -33,6 +38,19 @@ has "discovered" the information it need
>> custom REPORT request anyway. Most svn operations are at least twice
>> as slow over HTTP than over the custom svnserve protocol.
>>
>> +### GJS: that is the fault of the client, not the protocol. somewhere
>> + around the time that I stopped working on SVN for a while, I
>> + identified several things that an RA layer could cache in order to
>> + avoid looking them up again. nobody ever implemented that. thus,
>> + it is hard to claim the fault lies in the protocol when we KNOW we
>> + don't have to re-fetch certain items.
>
> But this just makes our existing clients more and more complex. Why
> are we doing PROPFINDs at all, ever? Unless we're actually trying to
> implement svn_ra_get_prop(), every single PROPFIND we do before our
> custom REPORTs are just weird formalities related to DeltaV discovery.
> Even if we completely remove all the redundant PROPFIND requests, we
> still get absolutely nothing out of following the DeltaV rules in
> these cases.
>
>> +
>> +### GJS: turnarounds don't have to be expensive if you can pipeline
>> + them. that is one of the (design) reasons for Serf. Again,
>> + unrelated to the protocol, but simply the implementation. And
>> + proposing a *new* implementation rather than fixing an existing
>> + one seems to be a much more monumental task.
>
> I have to do a reality check here: the promise of serf is that
> checkouts/updates would be faster than neon, because we could pipeline
> a bunch of GET requests rather than suck down the tree in one
> response. In practice, serf has proven to be no faster than neon in
> this regard. (Or if faster, only by a tiny percent.)
>
> I understand that the "real" promise here is that of caching proxy
> servers, which will supposedly deliver serf's original speed promises
> to us... but I've lost my faith in this idea:
>
> * nobody's actually done it yet and demonstrated it
>
> * given that BDB and/or FSFS is (typically) already serving popular
> fs-nodes out of RAM (due to OS caching), I don't think a proxy-server
> serving nodes out of RAM will be any faster.
>
>
>> +
>>
>> PROPOSAL
>> ========
>> @@ -41,15 +59,22 @@ Write a new HTTP protocol for svn; map
>> requests.
>>
>> * svn over HTTP would be much faster (eliminate turnarounds)
>> +
>> +### GJS: again: pipelining.
>
> Are we going to pipeline every single request in svn_ra.h? Does it
> even make sense to do so? I don't view it as a magic bullet.
>
>
>>
>> * svn over HTTP would be almost as easy to extend as svnserve.
>>
>> +### GJS: have we had an issue with extending our current HTTP use?
>
> Let's ask folks who have implement the mod_dav_svn get-lock-report,
> get-locations report, mergeinfo-report, etc.
>
> But meanwhile, I maintain it's not so much about extending as it is
> about maintaining. Anyone should be able to understand what's
> happening in the HTTP layer, debug it, rev an interface by adding a
> new parameter. Nobody is scared of doing this in svnserve. To do
> this in mod_dav_svn, people need a 3 hour lecture in the architecture
> of the system, DeltaV terminology, etc.
>
>
>> +
>> * svn over HTTP would be comprehensible to devs and users both
>> (require no knowledge of DeltaV concepts).
>>
>> * We'd still maintain almost all of Apache's advantages
>> (propositions A through E above).
>>
>> +### GJS: and with the right design, should be able to get the G that I
>> + added above.
>> +
>> * We'd give up proposition F, which hasn't given us anything but the
>> ability to mount svn repositories as network drives. (For those
>> who *really* need this rarely-used feature, mod_dav_svn would
>> @@ -58,6 +83,11 @@ requests.
>> * We could freely advertise a fixed syntax for browsing older
>> revisions, without worrying about violating DeltaV.
>>
>> +### GJS: we can do this today. DeltaV certainly doesn't proscribe what
>> + we can do in our URL namespace. my reluctance to advertise one was
>> + to give us maximum flexibility on the server, and to defer browing
>> + to external tools like ViewVC or Trac.
>
> Agreed, this feature isn't tied to a protocol rewrite at all. I'll
> remove it from the doc.
>
>
>
>> +
>>
>> MILE-HIGH DESIGN
>> ================
>> @@ -124,6 +154,9 @@ DESIGN
>>
>> GET /repos?cmd=get-file&r=23&path=/trunk/foo.c
>>
>> +### GJS: in order to allow for intermediate caches to work, the URLs
>> + need to be well-designed up-front. I can help with that.
>
> Is caching really a goal? Why is it so important on your list?
>
>
>> +
>> For requests which require large data exchanges, the big payloads go
>> into request and/or response bodies. For example:
>>
>> @@ -131,6 +164,11 @@ DESIGN
>>
>> [body contains complete 'update report' describing working copy's
>> revisions; response is a complete editor-drive.]
>> +
>> +### GJS: no. I've never seen a GET request with a body. You want to
>> + avoid that, or you could end up being incompatible with many
>> + proxies. If you need input data, then make it a POST, or use one
>> + of the other HTTP methods defined by HTTP, WebDAV, or DeltaV.
>
> Ah, ok, so any request bodies will probably need to be POSTs or somesuch.
>
>
>
>>
>> POST /repos?cmd=commit&keeplocks=true
>>
>> @@ -170,6 +208,12 @@ DESIGN
>> embedded s-expressions, exactly like svnserve does. Heck, maybe we
>> could even use svnserve's parsing/unparsing code!)
>>
>> +### GJS: personally, I'd recommend protocol buffers, or Apache Thrift
>> + (a third-party reimplementation of PBs done at facebook). just
>> + toss out the formats used in FS and the svn protocol, and let a
>> + new/external library do the marshalling for us.
>> + http://stuartsierra.com/2008/07/10/thrift-vs-protocol-buffers
>
> Hmmm, interesting.
>
> Honestly, it sounds to me like you might be willing to toss away all
> the DeltaV junk, just as long as we still make individual GETs of
> (rev, path) objects (1) possible, (2) cacheable, (3) pipelineable.
> Those seem to be the things that are important to you.
>
> If that's the case, I *still* think we need a new protocol, and a new
> server and client to implement it. The idea is to start clean, build
> the smallest possible protocol with the smallest amount of code,
> directly corresponding with svn_ra.h as much as possible... and heck,
> if we can get pipelining and cacheability in there too, great. But I
> cannot comprehend achieving this goal by modifying the existing
> codebase.... it would just be adding even more complexity to something
> already incomprehensible to most of the svn developers.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org