You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Michael Sinz <Mi...@sinz.org> on 2005/05/08 02:49:30 UTC
mod_dav_svn enhancement idea
In my working on the Insurrection project, I happened to run into a few
items that may be reasonable to update/enhance/fix in the mod_dav_svn.
Note, I have not gone and "just done it" as (1) I have a family and,
more importantly, (2) I wanted to make sure that this is not unreasonable.
In the normal HTTP GET operations for the directories, the current mod_dav_svn
produces XML that looks very much like this:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="/insurrection.xsl"?>
<!DOCTYPE svn [
<!ELEMENT svn (index)>
<!ATTLIST svn version CDATA #REQUIRED
href CDATA #REQUIRED>
<!ELEMENT index (updir?, (file | dir)*)>
<!ATTLIST index name CDATA #IMPLIED
path CDATA #IMPLIED
rev CDATA #IMPLIED>
<!ELEMENT updir EMPTY>
<!ELEMENT file (prop)*>
<!ATTLIST file name CDATA #REQUIRED
href CDATA #REQUIRED>
<!ELEMENT dir (prop)*>
<!ATTLIST dir name CDATA #REQUIRED
href CDATA #REQUIRED>
<!ELEMENT prop (#PCDATA)>
<!ATTLIST prop name CDATA #REQUIRED>
]>
<svn version="1.1.4 (r13838)"
href="http://subversion.tigris.org/">
<index rev="258" path="/">
<file name=".svn_index" href=".svn_index"></file>
<dir name="branches" href="branches/"></dir>
<dir name="releases" href="releases/"></dir>
<dir name="tags" href="tags/"></dir>
<dir name="trunk" href="trunk/"></dir>
</index>
</svn>
The first thing that caused some consternation is the fact that the repository
name is not included. The XML DTD shows that it is optional but it would
be handy. There is even code in mod_dav_svn/repos.c to check for and include
the name from the dav_resource * that is passed into dav_svn_deliver() but
there never seems to be a name. Now, I have worked around this with some
mod_rewrite tricks in Apache, so it is not an issue, but it would be nice to have.
The second question is why not include the same information that "svn ls"
produces. Something on the order of:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="/insurrection.xsl"?>
<!DOCTYPE svn [
<!ELEMENT svn (index)>
<!ATTLIST svn version CDATA #REQUIRED
href CDATA #REQUIRED>
<!ELEMENT index (updir?, (file | dir)*)>
<!ATTLIST index name CDATA #IMPLIED
path CDATA #IMPLIED
rev CDATA #IMPLIED>
<!ELEMENT updir EMPTY>
<!ELEMENT file EMPTY>
<!ATTLIST file name CDATA #REQUIRED
author CDATA #REQUIRED
revision CDATA #REQUIRED
size CDATA #REQUIRED
href CDATA #REQUIRED>
<!ELEMENT dir EMPTY>
<!ATTLIST dir name CDATA #REQUIRED
author CDATA #REQUIRED
revision CDATA #REQUIRED
href CDATA #REQUIRED>
]>
<svn version="$Id$" href="http://www.sinz.org/Michael.Sinz/Insurrection/">
<index rev="258" path="/" name="Web">
<file name=".svn_index" href=".svn_index"
author="mks" revision="93" size="3646"/>
<dir name="branches" href="branches/" author="mks" revision="1"/>
<dir name="releases" href="releases/" author="mks" revision="1"/>
<dir name="tags" href="tags/" author="mks" revision="233"/>
<dir name="trunk" href="trunk/" author="mks" revision="258"/>
</index>
</svn>
(The above is from some test code that I wrote as a wrapper to the svn ls and
not from the mod_dav_svn - yet - it has also been re-formatted to look reasonable
in this EMail.)
Note that I have removed the prop element as it has never actually happened.
(I see the comment about where it maybe would go but...)
The point being here, is to include data that is already easily available
and without much risk of dramatic bandwidth increase. (In fact, if that was a
worry, extra white space could be removed - after all, this is XML and most any
XML view will automatically produce the XML tree)
Would something like this be acceptable? Should I make some of the fields
optional? (Why or why not...)
Note that I have a perl hack that does just this after parsing the svn ls
output. This is not, by any means, the best choice of implementation but
it was mainly for proof of concept.
The advantages are that the data now contains more information that would
be useful for the display of the "directory" listing. (And not just in the
XML, the HTML could also be addressed.)
What do those of you who are deeper in the code think of the impact of
doing the extra svn_repos_get_committed_info() operations to the overall
system? My take on it is that it only impacts those who use the mod_dav_svn
to list/enumerate the repository and this is generally over the network.
The advantage is that there is no need to do multiple client-server interactions.
(Maybe the svn ls code could just grow XML output instead ;-)?
I need to do a lot more thinking about this, but before I charge down a path,
is this even a reasonable suggestion/idea? What are the plans for the prop
element and should it be removed? (It has not been generated, from what I can
see and has been optional.)
BTW - I really wish svn log --limit existed in the 1.1.x - it would really
be a life-saver for those larger repositories.
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:michael.sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Michael Sinz <Mi...@sinz.org>.
Ben Collins-Sussman wrote:
>
> On May 7, 2005, at 9:49 PM, Michael Sinz wrote:
>
>>
>> Would something like this be acceptable? Should I make some of the
>> fields optional? (Why or why not...)
>
> People have asked for improvements like this many times in the past,
> and our standard answer has been "no, please, don't go there."
I understand and that is why I asked. I have no real problem doing whst
I have been doing so far. I just thought it could reduce the server load
a bit by having some of the simple data directly available.
I do understand your point of view and have seen it on the list before.
No problem, and sorry if I bothered the list with the request again. It
just "looked so easy" when I looked at the code.
> WebDAV says that a GET request on a directory is "undefined", meaning
> we can do whatever we want. Instead of returning an error or a blank
> page, we decided to be a little bit nice and show a directory listing.
I may just use mod_rewrite to have the get on a directory go elsewhere...
It just makes the mod_authz situation a bit harder to deal with.
[...]
> Our standard reply is:
>
> * please, no more code bloat in mod_dav_svn.
Fully understand - I am a major "keep it simple, stupid" type of guy myself.
> * go install one of the many repository viewers instead.
;-) The reason I asked was to try to improve the Insurrection system a bit.
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:michael.sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Ben Collins-Sussman <su...@collab.net>.
On May 7, 2005, at 9:49 PM, Michael Sinz wrote:
>
> Would something like this be acceptable? Should I make some of the
> fields
> optional? (Why or why not...)
>
People have asked for improvements like this many times in the past,
and our standard answer has been "no, please, don't go there."
WebDAV says that a GET request on a directory is "undefined", meaning
we can do whatever we want. Instead of returning an error or a blank
page, we decided to be a little bit nice and show a directory
listing. Unfortunately, it's a bit of a tease... people constantly
want to add more and more information to the list, more features,
until we ultimately end up duplicating a full-blown repository viewer.
It's a slippery slope that we don't want to go down: there are
already what -- 7 or 8 repository viewers now (including
Insurrection!) -- and there's just no need to build one directly into
mod_dav_svn.
Our standard reply is:
* please, no more code bloat in mod_dav_svn.
* go install one of the many repository viewers instead.
It's nothing personal, of course. :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
John Belmonte wrote:
> Ben Collins-Sussman wrote:
>
>>On May 7, 2005, at 9:49 PM, Michael Sinz wrote:
>>
>>
>>>BTW - I really wish svn log --limit existed in the 1.1.x - it would
>>>really
>>>be a life-saver for those larger repositories.
>>>
>>
>>Not possible to backport to 1.1.x, because
>>
>> * patch releases are bugfixes only, not for adding new features, and
>>
>> * it involves an API change, which guarantees that it must be in a
>>minor release.
>>
>>Those 'larger repositories' will just have to upgrade to svn 1.2 next
>>week. :-)
>
>
> Except that the current implementation of --limit in 1.2 doesn't seem to
> provide any better performance than doing a full log and filtering the
> result (see http://subversion.tigris.org/issues/show_bug.cgi?id=2290).
> This is a shame-- I would really like to use this for interfaces to log
> that are simple and efficient (e.g. dynamically generate the RSS feed of
> changes on a path by sending "svn log --limit 10 --xml TARGET" through
> an appropriate XSLT).
Well, it's not sending all the data over the wire (if you're talking to
a 1.2.x server anyway), but yes, it is doing more server-side work than
is truly needed in that it's still calling svn_repos_history2 with the
original revision range (i.e. 0:HEAD if no range is specified by the
user). This means it has to go through all the revisions to see if the
path in question was modified, which is kind of slow. This could be
changed to make multiple calls to svn_repos_history2, starting with more
recent ranges and moving backwards in time until you have enough to
satisfy the limit argument, but nobody has done that yet. It would be a
server-side change only, clients would still work fine, and it wouldn't
require any changes in APIs, so there's no reason this couldn't be in a
forthcoming 1.2.x release if someone got motivated enough to fix it.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Michael Sinz <Mi...@sinz.org>.
John Belmonte wrote:
> Ben Collins-Sussman wrote:
>
>>On May 7, 2005, at 9:49 PM, Michael Sinz wrote:
>>
>>
>>>BTW - I really wish svn log --limit existed in the 1.1.x - it would
>>>really
>>>be a life-saver for those larger repositories.
>>>
>>
>>Not possible to backport to 1.1.x, because
>>
>> * patch releases are bugfixes only, not for adding new features, and
>>
>> * it involves an API change, which guarantees that it must be in a
>>minor release.
>>
>>Those 'larger repositories' will just have to upgrade to svn 1.2 next
>>week. :-)
>
>
> Except that the current implementation of --limit in 1.2 doesn't seem to
> provide any better performance than doing a full log and filtering the
> result (see http://subversion.tigris.org/issues/show_bug.cgi?id=2290).
> This is a shame-- I would really like to use this for interfaces to log
> that are simple and efficient (e.g. dynamically generate the RSS feed of
> changes on a path by sending "svn log --limit 10 --xml TARGET" through
> an appropriate XSLT).
You can look at what Insurrection does for the RSS feeds. It is in the
http://svn.sinz.com:8000/svn/Web/trunk/rss.cgi
It is "stupid" as some of the comments state, but it handles getting only
a subset of the log for RSS using a date range. Now, I don't use XSLT
in this case since most RSS readers do not support that, but that is a minor
nit. (I really should use XSLT to be completely correct, but RSS is so
simple that the perl regexp trick worked just fine.)
Yes the limit would be nice in case there are more than 20 (or whatever limit
is needed) entries within that date range.
I also have the date range start at the newest entry in that part of the
repository. This way if no changes over the weekend, you still get the
last n-days of changes rather than missing those 2 days.
You can try the RSS feed here: http://svn.sinz.com:8000/svn/Web/trunk/?Insurrection=rss
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:michael.sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by John Belmonte <jo...@neggie.net>.
Ben Collins-Sussman wrote:
>
> On May 7, 2005, at 9:49 PM, Michael Sinz wrote:
>
>>
>> BTW - I really wish svn log --limit existed in the 1.1.x - it would
>> really
>> be a life-saver for those larger repositories.
>>
>
> Not possible to backport to 1.1.x, because
>
> * patch releases are bugfixes only, not for adding new features, and
>
> * it involves an API change, which guarantees that it must be in a
> minor release.
>
> Those 'larger repositories' will just have to upgrade to svn 1.2 next
> week. :-)
Except that the current implementation of --limit in 1.2 doesn't seem to
provide any better performance than doing a full log and filtering the
result (see http://subversion.tigris.org/issues/show_bug.cgi?id=2290).
This is a shame-- I would really like to use this for interfaces to log
that are simple and efficient (e.g. dynamically generate the RSS feed of
changes on a path by sending "svn log --limit 10 --xml TARGET" through
an appropriate XSLT).
--John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Ben Collins-Sussman <su...@collab.net>.
On May 7, 2005, at 9:49 PM, Michael Sinz wrote:
>
> BTW - I really wish svn log --limit existed in the 1.1.x - it would
> really
> be a life-saver for those larger repositories.
>
Not possible to backport to 1.1.x, because
* patch releases are bugfixes only, not for adding new features, and
* it involves an API change, which guarantees that it must be in a
minor release.
Those 'larger repositories' will just have to upgrade to svn 1.2 next
week. :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Ben Collins-Sussman <su...@collab.net>.
On May 8, 2005, at 11:53 AM, Michael Sinz wrote:
>
> The other problem I have is when working on the server side but
> that is just
> my fault for not building this as a direct SVN module but as a
> layer on top of
> the SVN commands. I am wondering if that was a mistake. I like
> the fact that
> doing it this way does not depend on library linkages (old-school
> unix pipe/etc
> connections) but it does seem to have a number of limitations. I
> was just
> trying to find out which way the Subversion project would best like
> to move.
>
If you're asking the question, "is parsing the output from existing
binaries more limiting than using the library APIs?", then the answer
is "absolutely". :-)
But you're the only one who can judge whether the limitations are
worth the benefit of easier end-user installation.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Michael Sinz <Mi...@sinz.org>.
Ben Collins-Sussman wrote:
> On May 8, 2005, at 7:28 AM, Michael Sinz wrote:
>>> Try setting the SVNReposName parameter in the <Location> in httpd.conf.
>>> How else do you expect mod_dav_svn could know what the repository
>>> name is?
>> That only works when you have only one repository. When you use SVNParentPath
>> the repository name is "implied" by the next element in the URL which is not exposed
>> in the XML or the HTML returned by mod_dav_svn.
> Michael -- I'm not sure I understand the problem you're trying to solve.
The situation is that I have a number of clients that have more than one
repository on their servers. When browsing a repository using the features
of the mod_dav_svn HTTP GET, there is no visual indication of which repository
you are in other than by the user looking at the URL in the URL bar (if visible)
The XML/XSLT or the HTML shows the path within the repository but not which one.
This means that printing a directory listing or otherwise just looking at the
page does not give that information.
I have a nasty workaround where some JavaScript in the page generated by the XSLT
parses the document.location for the repository name. It works but is a kludge
and requires some trickery to get the repository name correct depending on where
the location is set for the SVNParentPath. (Is it /svn/* or /repos/* or /work/cms/* ?)
> You know that mod_dav_svn provides a public API which examines any
> incoming URL and gives you back (1) the repository's name, (2) the path
> within the repository that the URL refers to... right?
Yes, that is understood for custom clients. I was building this with web browsers
and XMLHttp and JavaScript. No software to install for read-only access to the
repository.
> Modules like mod_authz_svn (and marcus's new mod_authn_svn module)
> depend on this API, since only mod_dav_svn knows how to interpret the
> funky-looking opaque urls it generates and forces the svn client to
> use. It alsos work on normal public urls as well. Maybe Insurrection
> could use it too? Look at dav_svn_split_uri() in mod_dav_svn.h.
The other problem I have is when working on the server side but that is just
my fault for not building this as a direct SVN module but as a layer on top of
the SVN commands. I am wondering if that was a mistake. I like the fact that
doing it this way does not depend on library linkages (old-school unix pipe/etc
connections) but it does seem to have a number of limitations. I was just
trying to find out which way the Subversion project would best like to move.
BTW - I really do like Subversion a lot and mainly use the command line tools
for all of my work. I have a number of clients that are now switched over to
or are switching to Subversion from other solutions. (Most of them are moving
from SourceSafe - what a horrible tool that is) And TortoiseSVN is a great
way to sell the non-engineers (and some engineers) on using revision control.
(Those requirements documents and project plans should be revision tracked too!)
--
Michael Sinz Technology and Engineering Director/Consultant
"Starting Startups" mailto:michael.sinz@sinz.org
My place on the web http://www.sinz.org/Michael.Sinz
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Ben Collins-Sussman <su...@collab.net>.
On May 8, 2005, at 7:28 AM, Michael Sinz wrote:
>>
>> Try setting the SVNReposName parameter in the <Location> in
>> httpd.conf.
>> How else do you expect mod_dav_svn could know what the repository
>> name is?
>>
>
> That only works when you have only one repository. When you use
> SVNParentPath
> the repository name is "implied" by the next element in the URL which
> is not exposed
> in the XML or the HTML returned by mod_dav_svn.
>
Michael -- I'm not sure I understand the problem you're trying to solve.
You know that mod_dav_svn provides a public API which examines any
incoming URL and gives you back (1) the repository's name, (2) the
path within the repository that the URL refers to... right?
Modules like mod_authz_svn (and marcus's new mod_authn_svn module)
depend on this API, since only mod_dav_svn knows how to interpret the
funky-looking opaque urls it generates and forces the svn client to
use. It alsos work on normal public urls as well. Maybe
Insurrection could use it too? Look at dav_svn_split_uri() in
mod_dav_svn.h.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: mod_dav_svn enhancement idea
Posted by Michael Sinz <mi...@gmail.com>.
On 5/8/05, Branko Čibej <br...@xbc.nu> wrote:> Michael Sinz wrote:> > > The first thing that caused some consternation is the fact that the> > repository> > name is not included.> > Try setting the SVNReposName parameter in the <Location> in httpd.conf.> How else do you expect mod_dav_svn could know what the repository name is?
That only works when you have only one repository. When you use SVNParentPaththe repository name is "implied" by the next element in the URL whichis not exposedin the XML or the HTML returned by mod_dav_svn.
> [snip]> > > Note that I have removed the prop element as it has never actually> > happened.> > (I see the comment about where it maybe would go but...)> > What you did was replace a generic property transport mechanism with one> that only supports some props.
Yes, I know. It was to try to reduce the complexity (as everyone wants to keepmod_dav_svn small/light/simple.
> That's quite apart from the fact that we decided not to add more> browsing capabilities into mod_dav_svn. In fact, I think I'll remove the> prop element from that DTD anyway, since we won't ever be using it.
Ok... And I will not bother "adding bloat" to mod_dav_svn.-- Michael Sinz Technology and Engineering Director/Consultant"Starting Startups" mailto:Michael.Sinz@sinz.orgMy place on the web http://www.sinz.org/Michael.Sinz
Re: mod_dav_svn enhancement idea
Posted by Branko Čibej <br...@xbc.nu>.
Michael Sinz wrote:
> The first thing that caused some consternation is the fact that the
> repository
> name is not included.
Try setting the SVNReposName parameter in the <Location> in httpd.conf.
How else do you expect mod_dav_svn could know what the repository name is?
[snip]
> Note that I have removed the prop element as it has never actually
> happened.
> (I see the comment about where it maybe would go but...)
What you did was replace a generic property transport mechanism with one
that only supports some props.
That's quite apart from the fact that we decided not to add more
browsing capabilities into mod_dav_svn. In fact, I think I'll remove the
prop element from that DTD anyway, since we won't ever be using it.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org