You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Simon Odou <si...@lrde.epita.fr> on 2004/07/19 17:41:41 UTC
svn log causes XML parse error (internal error)
For the same project, the svn behavior is different with "file://" and
"http://". Commits with particular log message can be rejected with "http" but
accepted with "file". Then the command "svn log" might fail.
$ svn co http://localhost/test && cd test
Checked out revision 0.
$ touch a && svn add a
A a
$ echo "a" >> a && svn ci -m `echo -ne "\0x1"`
svn: Commit failed (details follow):
svn: applying log message
to /test/!svn/wbl/0f562004-95df-0310-837a-e85ba3689386/0: 400 Bad Request
(http://localhost)
If you checkout your project using file system, then your commit is accepted:
$ svn co file:///svn/test && cd test
Checked out revision 0.
$ touch a && svn add a
A a
$ echo "a" >> a && svn ci -m `echo -ne "\0x1"`
Adding a
Transmitting file data .
Committed revision 1.
Using the file system, the svn log command succeed:
$ svn log | cat -e
No commit for revision 0.$
------------------------------------------------------------------------$
r1 | simon | 2004-07-19 19:01:53 +0200 (Mon, 19 Jul 2004) | 1 line$
$
^A$
------------------------------------------------------------------------$
But using http, it fails:
$ svn log
svn: REPORT request failed on '/test/!svn/bc/2'
svn: The REPORT request returned invalid XML in the response: XML parse error
at line 7: internal error. (/test/!svn/bc/2)
NB: it seems to be the same problem for most of special characters between 0x1
and 0x1f.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Julian Reschke <ju...@gmx.de>.
Vincent Lefevre wrote:
> Are generic WebDAV clients useful for Subversion? I thought the only
> point of using DAV here was with a Subversion client.
Of course they are. Why else would all the work have been done for
auto-versioning (that will enable Delta-V unaware clients to edit files
in a Subversion store)?
Best regards, Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Vincent Lefevre <vi...@vinc17.org>.
On 2004-07-22 08:32:20 +0200, Julian Reschke wrote:
> Vincent Lefevre wrote:
> >Otherwise, couldn't Subversion properties be transported with DAV in
> >an encoded form, encapsulated in DAV properties?
>
> Yes, but then they wouldn't work as expected in generic WebDAV clients,
> right?
Are generic WebDAV clients useful for Subversion? I thought the only
point of using DAV here was with a Subversion client.
--
Vincent Lefèvre <vi...@vinc17.org> - Web: <http://www.vinc17.org/>
100% validated (X)HTML - Acorn / RISC OS / ARM, free software, YP17,
Championnat International des Jeux Mathématiques et Logiques, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Julian Reschke <ju...@gmx.de>.
Vincent Lefevre wrote:
> Are DAV properties exactly the same notion as Subversion properties?
Protocol-wise, I think so.
> Otherwise, couldn't Subversion properties be transported with DAV in
> an encoded form, encapsulated in DAV properties?
Yes, but then they wouldn't work as expected in generic WebDAV clients,
right?
Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Vincent Lefevre <vi...@vinc17.org>.
On 2004-07-21 10:52:37 -0700, Ben Reser wrote:
> On Wed, Jul 21, 2004 at 09:45:48AM -0500, kfogel@collab.net wrote:
> > Now we're talking about "/" being disallowed in property names. Why
> > is it disallowed? I'm not saying there should be no limitations on
> > property names -- newlines and other control chars seem like a bad
> > idea, for example. But "/"? What's wrong with "/"?
>
> Remember that a / is a reserved character in XML element names and that
> properties over DAV are named in XML elements. This is a horrible
> failing of the DAV spec, but it's what it is.
Are DAV properties exactly the same notion as Subversion properties?
Otherwise, couldn't Subversion properties be transported with DAV in
an encoded form, encapsulated in DAV properties?
--
Vincent Lefèvre <vi...@vinc17.org> - Web: <http://www.vinc17.org/>
100% validated (X)HTML - Acorn / RISC OS / ARM, free software, YP17,
Championnat International des Jeux Mathématiques et Logiques, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by kf...@collab.net.
Julian Reschke <ju...@gmx.de> writes:
> Subversion may own the "svn:" namespace in the context of Subversion,
> but it certainly doesn't in the context of XML namespaces (and thus
> WebDAV). The simple reason being that it's not a registered URI scheme
> and thus is owned by nobody (as far as the IETF is concerned).
What I'm trying to get us away from is the idea that the Subversion
property namespace is an XML namespace. The "context of XML
namespaces" does not matter, if we're talking about ownership.
It happens that one of our transport layers uses XML to send
Subversion property names, and does so in such a way that some of the
limitations of XML end up being imposed on the names. This is
unfortunate. The best solution seems to be to have Subversion impose
the limitation on users up front, so the "dangerous" data doesn't
enter the system. Okay, that's fine.
However, this solution to an implementation bug does *not* mean that
Subversion properties are anything other than just that: Subversion
properties. For over 4 years, since even before there were any
Subversion repositories in the world, Subversion has clearly
documented that property names beginning with "svn:" are under
Subversion's control.
Subversion is perfectly free to declare that. It invented Subversion
properties. They do not exist apart from Subversion. It's too bad
about the implementation bug, and we're having to take some practical
steps to deal with that. But that's no reason to suddenly declare
that Subversion properties are officially defined by the
specifications of some other piece of software (or protocol, or
format, or whatever).
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Julian Reschke <ju...@gmx.de>.
kfogel@collab.net wrote:
> Since Subversion owns the "svn:" namespace, we don't have to worry
> about compatibility with older clients *or* servers. No one else is
> creating "svn:" properties, and if they are, it's in violation of a
> long-published standard.
Subversion may own the "svn:" namespace in the context of Subversion,
but it certainly doesn't in the context of XML namespaces (and thus
WebDAV). The simple reason being that it's not a registered URI scheme
and thus is owned by nobody (as far as the IETF is concerned).
I always thought that one of the longer term goals of Subversion was to
be a RFC3253-compliant server (plus extensions). This also implies that
other CMS systems would be able to implement RFC3253 (+ the extensions)
and then would be able to interact with Subversion clients. It would be
a pity if that goal is gone. If it isn't, there should be a one-to-one
mapping of Subversion properties to WebDAV properties; and the simplest
way to achieve this is to simply use the same name format.
> ...
> It seems quite reasonable to me that people want "/" in property
> names, just as they want ":". We've already had real-world examples
> of both.
> ...
It doesn't seem reasonable to me at all; and I've spent the last 3 years
working on a CMS (incl versioning and resource properties), and nobody
ever has asked for that.
Asking again: what's the use case? Is it possible that people use the
*name* for display purposes (in the absence of something else such as a
property *display* name that would be I18N'd)?
> ...
> Good point.
>
> I certainly agree that we should decide on a valid syntax for property
> names (and values, for that matter), among other things. But the
> limitations encouraged by DAV are far too narrow, and we shouldn't
> settle for them unless we have absolutely no choice.
DAV simply relies on XML marshalling, just like *many* other protocols
as well (RDF in XML, Atom, RSS...). Using XML name syntax at least
guarantees a well-known and well-defined set of valid names that will be
compatible across many protocols/formats using XML as well. This by
itself is a big advantage because it allows sharing of property names
between these formats (for instance, I can make any feed-level Atom
extension element a WebDAV property, and the other way around).
I think giving that up would be a mistake, in particular when it's
absolutely unclear why one would want these characters inside an
*identifier*.
Best regards, Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Ben Reser <be...@reser.org>.
On Fri, Jul 23, 2004 at 10:14:08AM -0500, kfogel@collab.net wrote:
> Hey, now :-). Neglecting to specify the encoding scheme doesn't make
> it a drive-by suggestion. Obviously we *can* encode data safely,
> using whatever minimal set of characters we're allowed as long as
> there are at least two distinct characters available.
>
> I'm not denying the ugliness of the solution, any more than of the
> problem that motivates it. But there's a lot of room between
> "ready-to-publish RFC spec" and "drive-by suggestion". If we insisted
> that every proposal made on this list be a complete and
> ready-to-implement specification, we'd never have gotten a lot of the
> features and bugfixes we have today. Leave some room for discussion,
> please.
>
> You can disagree with a proposal on technical or aesthetic grounds,
> without libeling it as "drive-by".
Well all of this has somewhat come up before and encoding it was
suggested then. Everyone just assumed that it could be done with URI
encoding. Leaving out how you plan on encoding it was a pretty
important thing to leave out becuase the method we'd probably most
likely want to use won't work.
Considering that this was all discussed on the colon issue, it seemed
rather driveby to come along and say "Let's just encode it." When
past discussions have shown it's really not that easy.
> Yeah, but wait a sec.
>
> You just proposed, and tossed, three solutions based on namespaces.
> None of them were the solution I proposed, which is also based on a
> namespace, but as it's *Subversion*'s namespace, it maintains
> compatibility (as far as we know) -- which is exactly why I used it.
>
> There are an infinite number of non-working solutions for any problem.
> Proposing three of them here, only to shoot them down, doesn't add
> anything to the discussion. There's surely some Latin name for this
> rhetorical technique :-), but I don't know it. All I know is, it
> doesn't mean anything. The number of non-working solutions thought of
> has *no relevance* to the solubility of a problem. Once you know they
> don't work, there's just no reason to bring them up again.
Sure I had plenty of reason to bring them out again. Because by leaving
off the encoding scheme in your suggestion it seemed as though you
weren't aware of them. Bringing them out again saved everyone the time
of thinking them up again and then trying to implement them, because we
found your solution inelegent.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"A nation that forgets its past is doomed to repeat it."
- Sir Winston Churchill
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by kf...@collab.net.
Ben Reser <be...@reser.org> writes:
> On Wed, Jul 21, 2004 at 04:06:36PM -0500, kfogel@collab.net wrote:
> > No, we didn't get the idea of properties from DAV. We knew we wanted
> > key/value pairs even before we'd heard of DAV. Now we are imposing
> > limitations on the keys, because of one (out of three) RA protocols.
>
> Well I didn't set the limitations. juliafoad added it in r7528.
Oh, I'm not blaming you at all. Just correcting the misimpression
that our properties were meant to be DAV-restricted from the beginning
(and adding in some unnecessary griping, I'll admit, but that's not
directed at you).
> Do we really want to go down paths like this? It sounds really really
> ugly. And for what? How many people have asked for property names to
> be able to support slashes? So far we've had one complaint on IRC
> because he was trying to change a property that SVN::Mirror set. As
> ghudson is fond of saying I don't think it's worth the complexity to
> solve until it proves to be a problem.
>
> And this isn't a problem. The restriction on the names has been in
> there for nearly a year and hardly anyone has noticed or complained.
> How difficult is it to pick another name in the valid namespace? Use -
> or . or _ instead of /.
>
> Further, your suggested solution is a totally drive by suggestion. How
> exactly are you going to quote it? You could use base64, but jeez how
> ugly. Can't use URI encoding because % isn't a valid character in XML
> element names. You could make up your own encoding scheme, but again
> how ugly.
Hey, now :-). Neglecting to specify the encoding scheme doesn't make
it a drive-by suggestion. Obviously we *can* encode data safely,
using whatever minimal set of characters we're allowed as long as
there are at least two distinct characters available.
I'm not denying the ugliness of the solution, any more than of the
problem that motivates it. But there's a lot of room between
"ready-to-publish RFC spec" and "drive-by suggestion". If we insisted
that every proposal made on this list be a complete and
ready-to-implement specification, we'd never have gotten a lot of the
features and bugfixes we have today. Leave some room for discussion,
please.
You can disagree with a proposal on technical or aesthetic grounds,
without libeling it as "drive-by".
> The only other option is to use another special namespace where you
> place all the of the name in the xmlns URI (which of course is URI
> encoded) and then decode that and throw away the element name, e.g.:
>
> <P:customprop
> xmlns="http://subversion.tigris.org/xmlns/encoded/foo/bar%20whatever">
>
> But ohh we did something really silly and we barf in the old clients if
> they see a namespace they don't know about. Whoops can't do that.
>
> What about putting them in the existing namespace won't that work?
>
> <P:customprop
> xmlns="http://subversion.tigris.org/xmlns/svn/quoted-propname/foo/bar%20whatever">
>
> Oops that won't work either. Old clients will once again think it's a
> different namespace. And even if it did work they'd think there were
> gobs of properties named "customprop".
>
> So why don't we do the xmlns trick I showed above and make servers and
> clients negotiate if they know about it. What a great idea. Except DAV
> is stateless. We have the OPTIONS method. But we don't always call it
> and the only way we have of putting flags in it is also really ugly.
>
> We could try the new way and if it fails fallback on the old way. But
> we do dozens of PROPFINDs in some operations and that could
> significantly increase our round trips against old servers.
>
> There is no way to do this that is reasonably backwards compatable and
> elegent. Believe me I tried. I finally gave up and worked around the
> colon issue.
Yeah, but wait a sec.
You just proposed, and tossed, three solutions based on namespaces.
None of them were the solution I proposed, which is also based on a
namespace, but as it's *Subversion*'s namespace, it maintains
compatibility (as far as we know) -- which is exactly why I used it.
There are an infinite number of non-working solutions for any problem.
Proposing three of them here, only to shoot them down, doesn't add
anything to the discussion. There's surely some Latin name for this
rhetorical technique :-), but I don't know it. All I know is, it
doesn't mean anything. The number of non-working solutions thought of
has *no relevance* to the solubility of a problem. Once you know they
don't work, there's just no reason to bring them up again.
Regarding my proposal: you didn't like it because of inelegantness,
not incorrectness. I pretty much agree, and if the only practical
problem we have is that once a year someone posts saying they can't
use "/" in a property name, then I can see this not being worth the
trouble to fix. You're right that I don't like the creeping way these
restrictions came upon us. I wish I'd been paying more attention
before :-(.
> > I'm not saying this enhancement should delay 1.1 or even 1.2. I am
> > merely asking if anyone disagrees that the situation is a bug, and
> > secondarily asking if the above method seems like it would fix it.
>
> No I don't agree it is a bug. We've explicitly disallowed these names.
> Right or wrong that's the way it is until 2.0.
Okay.
> > It seems quite reasonable to me that people want "/" in property
> > names, just as they want ":". We've already had real-world examples
> > of both.
>
> Sure it seems reasonable enough. But here's the deal. The colon
> properties were already being allowed by the client software. They only
> failed to work in one specific case and then only as a revprop. So many
> people were probably using them completely unaware of the problems with
> them. I suspect very few people set revprops other than the ones we use
> internally (svn:log, svn:author, etc...).
>
> But the / is an entirely different situation. It's disallowed by the
> client library already. It has much more serious consequences if it
> gets into a repo. It doesn't just fail in some obscure case. It will
> cause operations involving it to have completely invalid XML.
>
> The bug is that we're allowing people driving the repos/fs layers
> directly to set properties that are invalid as far as our client libs
> are concerned. I suppose someone else might want to use these libs for
> something completely different that might not care about our RA issues.
> But I don't see what choice we have.
These three paragraphs, and the one preceding them, were all the
response I was actually asking for.
> > I certainly agree that we should decide on a valid syntax for property
> > names (and values, for that matter), among other things. But the
> > limitations encouraged by DAV are far too narrow, and we shouldn't
> > settle for them unless we have absolutely no choice.
>
> Well I think we have. You just don't like it. :)
No, I don't, you're right :-). But you've persuaded me to live with
it, though I'm embarrassed that I didn't notice it was happening until
it was too late. Sigh.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Ben Reser <be...@reser.org>.
On Wed, Jul 21, 2004 at 04:06:36PM -0500, kfogel@collab.net wrote:
> No, we didn't get the idea of properties from DAV. We knew we wanted
> key/value pairs even before we'd heard of DAV. Now we are imposing
> limitations on the keys, because of one (out of three) RA protocols.
Well I didn't set the limitations. juliafoad added it in r7528.
> I'm not sure it has to wait until 2.0. We could add the feature
> compatibly with a new "svn:quoted-propname:" prefix. (I'm being
> verbose; obviously it doesn't have to be that long.)
>
> Since Subversion owns the "svn:" namespace, we don't have to worry
> about compatibility with older clients *or* servers. No one else is
> creating "svn:" properties, and if they are, it's in violation of a
> long-published standard.
>
> With this enhancement, anytime someone sets a property whose name
> contains special characters, we transform it into
> "svn:quoted-propname:<quoted-str>" for transport. In the actual
> working copy, and in the repository, it does not need the quoting,
> because our storage formats there can handle the real names. However,
> if an older repository happens to keep the quoting, that's okay! Old
> clients will simply fail to unquote it; new clients will correctly
> unquote it. No one will crash, and all information will be preserved
> in one of two possible forms, until eventually everyone is upgraded.
Do we really want to go down paths like this? It sounds really really
ugly. And for what? How many people have asked for property names to
be able to support slashes? So far we've had one complaint on IRC
because he was trying to change a property that SVN::Mirror set. As
ghudson is fond of saying I don't think it's worth the complexity to
solve until it proves to be a problem.
And this isn't a problem. The restriction on the names has been in
there for nearly a year and hardly anyone has noticed or complained.
How difficult is it to pick another name in the valid namespace? Use -
or . or _ instead of /.
Further, your suggested solution is a totally drive by suggestion. How
exactly are you going to quote it? You could use base64, but jeez how
ugly. Can't use URI encoding because % isn't a valid character in XML
element names. You could make up your own encoding scheme, but again
how ugly.
The only other option is to use another special namespace where you
place all the of the name in the xmlns URI (which of course is URI
encoded) and then decode that and throw away the element name, e.g.:
<P:customprop
xmlns="http://subversion.tigris.org/xmlns/encoded/foo/bar%20whatever">
But ohh we did something really silly and we barf in the old clients if
they see a namespace they don't know about. Whoops can't do that.
What about putting them in the existing namespace won't that work?
<P:customprop
xmlns="http://subversion.tigris.org/xmlns/svn/quoted-propname/foo/bar%20whatever">
Oops that won't work either. Old clients will once again think it's a
different namespace. And even if it did work they'd think there were
gobs of properties named "customprop".
So why don't we do the xmlns trick I showed above and make servers and
clients negotiate if they know about it. What a great idea. Except DAV
is stateless. We have the OPTIONS method. But we don't always call it
and the only way we have of putting flags in it is also really ugly.
We could try the new way and if it fails fallback on the old way. But
we do dozens of PROPFINDs in some operations and that could
significantly increase our round trips against old servers.
There is no way to do this that is reasonably backwards compatable and
elegent. Believe me I tried. I finally gave up and worked around the
colon issue.
> I'm not saying this enhancement should delay 1.1 or even 1.2. I am
> merely asking if anyone disagrees that the situation is a bug, and
> secondarily asking if the above method seems like it would fix it.
No I don't agree it is a bug. We've explicitly disallowed these names.
Right or wrong that's the way it is until 2.0.
> It seems quite reasonable to me that people want "/" in property
> names, just as they want ":". We've already had real-world examples
> of both.
Sure it seems reasonable enough. But here's the deal. The colon
properties were already being allowed by the client software. They only
failed to work in one specific case and then only as a revprop. So many
people were probably using them completely unaware of the problems with
them. I suspect very few people set revprops other than the ones we use
internally (svn:log, svn:author, etc...).
But the / is an entirely different situation. It's disallowed by the
client library already. It has much more serious consequences if it
gets into a repo. It doesn't just fail in some obscure case. It will
cause operations involving it to have completely invalid XML.
The bug is that we're allowing people driving the repos/fs layers
directly to set properties that are invalid as far as our client libs
are concerned. I suppose someone else might want to use these libs for
something completely different that might not care about our RA issues.
But I don't see what choice we have.
> Good point.
>
> I certainly agree that we should decide on a valid syntax for property
> names (and values, for that matter), among other things. But the
> limitations encouraged by DAV are far too narrow, and we shouldn't
> settle for them unless we have absolutely no choice.
Well I think we have. You just don't like it. :)
Of course Julian's log for the addition said it was easier to make a
restrictive set and expand it later. That sounds really true. But in
this case it just isn't the case.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Klaus Rennecke <kr...@tigris.org>.
kfogel@collab.net wrote:
> Ben Reser <be...@reser.org> writes:
>
>>As far as this being DAV specific. I presume that DAV is where we got
>>the whole concept of properties and our properties are designed around
>>that specification. I don't see what's so bad about having some
>>limitations on property names. So long as those limitations are spelled
>>out. Which I think we've done a poor job of doing.
>
>
> No, we didn't get the idea of properties from DAV. We knew we wanted
> key/value pairs even before we'd heard of DAV. Now we are imposing
> limitations on the keys, because of one (out of three) RA protocols.
> [...]
Sorry if this is a stupid idea; I haven't had the time yet to read all
the specs.
Could the pf:property-displayname be bent to express the real property
name? I am aware that it is supposed to be language specific, but I
don't see anything wrong with fixing it at xml:lang="en" for that purpose.
http://www.ietf.org/internet-drafts/draft-reschke-webdav-property-datatypes-06.txt
/Klaus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Amelia A Lewis <al...@tibco.com>.
On Wed, 21 Jul 2004 16:06:36 -0500
kfogel@collab.net wrote:
> I'm not sure it has to wait until 2.0. We could add the feature
> compatibly with a new "svn:quoted-propname:" prefix. (I'm being
> verbose; obviously it doesn't have to be that long.)
Just a quick note. Once "svn:quoted-propname:xyzzy" is used as the name
of an element or attribute in XML, the XML is ill-formed (fatal error,
stop processing).
Except for older processors that permit colon in a name, but do not
interpret the colon. For XML+Namespaces, more than one colon causes
breakage.
In other words, as a fix for the DAV restriction to names valid in XML,
this one doesn't work (although the basic concept would work, mostly,
apart from the fact that whatever you use as a separator character isn't
reserved and other people may be using it for something else).
> With this enhancement, anytime someone sets a property whose name
> contains special characters, we transform it into
> "svn:quoted-propname:<quoted-str>" for transport. In the actual
Illegal as a name in XML+Namespaces.
Amy!
--
Amelia A. Lewis
Senior Architect
TIBCO/Extensibility, Inc.
alewis@tibco.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by kf...@collab.net.
Ben Reser <be...@reser.org> writes:
> As far as this being DAV specific. I presume that DAV is where we got
> the whole concept of properties and our properties are designed around
> that specification. I don't see what's so bad about having some
> limitations on property names. So long as those limitations are spelled
> out. Which I think we've done a poor job of doing.
No, we didn't get the idea of properties from DAV. We knew we wanted
key/value pairs even before we'd heard of DAV. Now we are imposing
limitations on the keys, because of one (out of three) RA protocols.
> Well this was already done a long time ago. I could easily open up the
> available number of characters in property names by using a namespace
> hack. However, this is competely incompatable with our existing
> clients. Plus it's a total pain in the butt to try and negotiate this
> with clients. So if we're going to do this it's a 2.0 thing in my mind.
I'm not sure it has to wait until 2.0. We could add the feature
compatibly with a new "svn:quoted-propname:" prefix. (I'm being
verbose; obviously it doesn't have to be that long.)
Since Subversion owns the "svn:" namespace, we don't have to worry
about compatibility with older clients *or* servers. No one else is
creating "svn:" properties, and if they are, it's in violation of a
long-published standard.
With this enhancement, anytime someone sets a property whose name
contains special characters, we transform it into
"svn:quoted-propname:<quoted-str>" for transport. In the actual
working copy, and in the repository, it does not need the quoting,
because our storage formats there can handle the real names. However,
if an older repository happens to keep the quoting, that's okay! Old
clients will simply fail to unquote it; new clients will correctly
unquote it. No one will crash, and all information will be preserved
in one of two possible forms, until eventually everyone is upgraded.
I'm not saying this enhancement should delay 1.1 or even 1.2. I am
merely asking if anyone disagrees that the situation is a bug, and
secondarily asking if the above method seems like it would fix it.
It seems quite reasonable to me that people want "/" in property
names, just as they want ":". We've already had real-world examples
of both.
> But for now, we need to follow what we've got. The acceptance by the
> lower level layers of property names not accepted by the client layers
> is a bug IMHO.
Sure. While we have the bug, we have the bug, and we should deal
gracefully with it.
> But if you really want to talk about the more subtle issue... The more
> subtle issue is a lack of thought into what constitutes valid syntaxes
> for a variety of things. As a result we're not enforcing these syntaxes
> and as a result people get really strange behavior when they get outside
> of what people normally use.
>
> Pathnames and property names are good examples that we've already run
> into. There are a number of reasons why we should define these things
> clearly:
>
> * Helps people adopting the software know our limitations.
> * Lets anyone planning on implementing another RA layer or FS layer or
> anything else know what they have to support and don't have to support.
> * Helps us write better tests to look for bugs.
> * Allows people writing code to better take into account the
> consequences of their code. Some implementations of network and fs
> layers will by necessity apply special meaning to characters. They
> better make sure that they escape or handle such characters that are
> allowed to avoid potential security issues for their special meanings.
>
> We've also got issues like this in the svn protocol. Just how long is a
> valid string? There's absolutely no definition. So theoretically we
> have to allow any amount. Which ended up causing security implicatons
> for us.
>
> The length of what a property value is allowed to be played into this.
> It's not specified either. So once again we had to just stick our heads
> in the sand and support an unlimited amount.
>
> I don't think you can ever support every imaginable naming or length
> that a user might want. But we have to pick something and then follow
> it. If we want to be fairly unrestrictive then we need to say that so
> implementors know to expect unusual characters.
Good point.
I certainly agree that we should decide on a valid syntax for property
names (and values, for that matter), among other things. But the
limitations encouraged by DAV are far too narrow, and we shouldn't
settle for them unless we have absolutely no choice.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Ben Reser <be...@reser.org>.
On Wed, Jul 21, 2004 at 09:45:48AM -0500, kfogel@collab.net wrote:
> Now we're talking about "/" being disallowed in property names. Why
> is it disallowed? I'm not saying there should be no limitations on
> property names -- newlines and other control chars seem like a bad
> idea, for example. But "/"? What's wrong with "/"?
Remember that a / is a reserved character in XML element names and that
properties over DAV are named in XML elements. This is a horrible
failing of the DAV spec, but it's what it is.
At any rate here's where we stand with property names:
Right now libsvn_client restricts the valid property names in the
is_valid_prop_name() static function. I think it's clear that this
function is implementing the Name syntax as specified in the XML
specification:
http://www.w3.org/TR/2004/REC-xml-20040204/#sec-common-syn
Issue #1807 exists because : while not technically reserved is somewhat
reserved for use by XML namespaces. Unfortunately the parser on the
Apache end was being too strict and rejecting XML that contained the
character outside the context of the use of a namespace. It's arguable
in my reading of the specification if this is legal or not. While, the
specification says that the parser must allow colons as name characters
it doesn't really say that to do with documents using namespaces and
that have element names with colons in them that aren't part of the
namespace use.
Of course the XML Namespace Recommendation redefines the space of the
valid element names to disallow the colon, so the Apache XML parsers
error is probably correct:
http://www.w3.org/TR/REC-xml-names/#ns-decl
Turning more specifically to DAV. RFC 2518 defines property names based
on XML namespace valid names. Which means if we're following the DAV
specification we're technically in violation becuase we're permitting
property names with colons.
http://www.webdav.org/specs/rfc2518.htm#rfc.section.4.5
Unfortunately, we probably encouraged the use of colon named properties
with our own svn: prefixed properties. It probably would have been much
better to use svn.log instead. But that's water under the bridge.
> If the issue is that a particular transport layer (DAV) makes it
> difficult to allow certain otherwise-desirable chars, then we have a
> bug in our transport layer, and should treat it as such. Now, maybe
> we need to disallow those chars in propset for a while, until we fix
> the bug. But we shouldn't be treating this as a permanent and
> acceptable state of affairs. The property namespace is Subversion's;
> the fact that we use XML or XML-based protocols is an implementation
> detail that shouldn't be leaking out into userland.
These characters were already disallowed by propset. The only reason
this issue came up is because SVN::Mirror doesn't use the client libs or
the ra layer libs. It uses the repos and fs libraries directly. As a
result it could place property names that the client libs would
disallow.
If we're going to disallow clients from setting property names then we
should disallow it all the way through the API. My intention is to do
that and to make is_valid_prop_name() a public function (though renamed)
so that anyone can tell if they have a valid property name.
As far as this being DAV specific. I presume that DAV is where we got
the whole concept of properties and our properties are designed around
that specification. I don't see what's so bad about having some
limitations on property names. So long as those limitations are spelled
out. Which I think we've done a poor job of doing.
> Is there some more subtle issue here? Is it that we need to limit
> these properties because DAV has its own idea of properties and we
> want to remain compatible with DAV clients? If so, perhaps we should
> be thinking of a compatibility strategy other than "Limit everyone to
> whatever DAV can handle."
Well this was already done a long time ago. I could easily open up the
available number of characters in property names by using a namespace
hack. However, this is competely incompatable with our existing
clients. Plus it's a total pain in the butt to try and negotiate this
with clients. So if we're going to do this it's a 2.0 thing in my mind.
But for now, we need to follow what we've got. The acceptance by the
lower level layers of property names not accepted by the client layers
is a bug IMHO.
But if you really want to talk about the more subtle issue... The more
subtle issue is a lack of thought into what constitutes valid syntaxes
for a variety of things. As a result we're not enforcing these syntaxes
and as a result people get really strange behavior when they get outside
of what people normally use.
Pathnames and property names are good examples that we've already run
into. There are a number of reasons why we should define these things
clearly:
* Helps people adopting the software know our limitations.
* Lets anyone planning on implementing another RA layer or FS layer or
anything else know what they have to support and don't have to support.
* Helps us write better tests to look for bugs.
* Allows people writing code to better take into account the
consequences of their code. Some implementations of network and fs
layers will by necessity apply special meaning to characters. They
better make sure that they escape or handle such characters that are
allowed to avoid potential security issues for their special meanings.
We've also got issues like this in the svn protocol. Just how long is a
valid string? There's absolutely no definition. So theoretically we
have to allow any amount. Which ended up causing security implicatons
for us.
The length of what a property value is allowed to be played into this.
It's not specified either. So once again we had to just stick our heads
in the sand and support an unlimited amount.
I don't think you can ever support every imaginable naming or length
that a user might want. But we have to pick something and then follow
it. If we want to be fairly unrestrictive then we need to say that so
implementors know to expect unusual characters.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Julian Reschke <ju...@gmx.de>.
kfogel@collab.net wrote:
> I'm a bit confused about what characters we're disallowing in property
> names, and about the reasons we're disallowing them.
>
> My understanding was that we had a problem with ":" in a property
> name, due to a WebDAV protocol problem. We all agreed this was a bug.
> Ben Reser went to fix it one way, but ended up having to fix it
> another way, see issue #1807 for details.
>
> Now we're talking about "/" being disallowed in property names. Why
> is it disallowed? I'm not saying there should be no limitations on
> property names -- newlines and other control chars seem like a bad
> idea, for example. But "/"? What's wrong with "/"?
In WebDAV properties are marshalled as XML elements, that is, a property
name must be a valid XML name. XML defines the set of chracters allowed
in names in
<http://www.w3.org/TR/2004/REC-xml-20040204/#NT-Name>
Note that this is similar to allowed characters in variable names in
common programming languages; you can't have whitespace or a "/" in them.
> If the issue is that a particular transport layer (DAV) makes it
> difficult to allow certain otherwise-desirable chars, then we have a
> bug in our transport layer, and should treat it as such. Now, maybe
> we need to disallow those chars in propset for a while, until we fix
> the bug. But we shouldn't be treating this as a permanent and
There is no simple way to fix that bug, except for adding a new layer of
character escaping. I don't think this is a good idea.
> acceptable state of affairs. The property namespace is Subversion's;
> the fact that we use XML or XML-based protocols is an implementation
> detail that shouldn't be leaking out into userland.
>
> Is there some more subtle issue here? Is it that we need to limit
> these properties because DAV has its own idea of properties and we
> want to remain compatible with DAV clients? If so, perhaps we should
> be thinking of a compatibility strategy other than "Limit everyone to
> whatever DAV can handle."
>
> (Are there other chars we don't allow, and if so, why?)
I'd like to understand why someone would *want* to use these
non-XML-name characters inside *identifiers* (and this is what property
names are). Keep in mind that identifiers are *not* meant to be used in
a UI; you need a separate layer of I18N on top of that anyway.
Best regards, Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by kf...@collab.net.
I'm a bit confused about what characters we're disallowing in property
names, and about the reasons we're disallowing them.
My understanding was that we had a problem with ":" in a property
name, due to a WebDAV protocol problem. We all agreed this was a bug.
Ben Reser went to fix it one way, but ended up having to fix it
another way, see issue #1807 for details.
Now we're talking about "/" being disallowed in property names. Why
is it disallowed? I'm not saying there should be no limitations on
property names -- newlines and other control chars seem like a bad
idea, for example. But "/"? What's wrong with "/"?
If the issue is that a particular transport layer (DAV) makes it
difficult to allow certain otherwise-desirable chars, then we have a
bug in our transport layer, and should treat it as such. Now, maybe
we need to disallow those chars in propset for a while, until we fix
the bug. But we shouldn't be treating this as a permanent and
acceptable state of affairs. The property namespace is Subversion's;
the fact that we use XML or XML-based protocols is an implementation
detail that shouldn't be leaking out into userland.
Is there some more subtle issue here? Is it that we need to limit
these properties because DAV has its own idea of properties and we
want to remain compatible with DAV clients? If so, perhaps we should
be thinking of a compatibility strategy other than "Limit everyone to
whatever DAV can handle."
(Are there other chars we don't allow, and if so, why?)
-Karl
Ben Reser <be...@reser.org> writes:
> On Mon, Jul 19, 2004 at 05:00:33PM -0700, Michael Brouwer wrote:
> > Of course svk (or more precisely SVN:Mirror) uses / in property names
> > which isn't even allowed anymore in 1.1 (not sure if it worked prior to
> > 1.1 either). You can't use svn ps to modify or set a property with a /
> > in it, though setting it though the perl-swig bindings does work since
> > that's what SVN:Mirror does.
>
> Never were allowed. clkao is changing that hopefully in some future
> version of SVN::Mirror and then in 1.2 I intend to fix our API to
> totally disallow this "bug".
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Ben Reser <be...@reser.org>.
On Mon, Jul 19, 2004 at 05:00:33PM -0700, Michael Brouwer wrote:
> Of course svk (or more precisely SVN:Mirror) uses / in property names
> which isn't even allowed anymore in 1.1 (not sure if it worked prior to
> 1.1 either). You can't use svn ps to modify or set a property with a /
> in it, though setting it though the perl-swig bindings does work since
> that's what SVN:Mirror does.
Never were allowed. clkao is changing that hopefully in some future
version of SVN::Mirror and then in 1.2 I intend to fix our API to
totally disallow this "bug".
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Michael Brouwer <mi...@tlaloc.net>.
Of course svk (or more precisely SVN:Mirror) uses / in property names
which isn't even allowed anymore in 1.1 (not sure if it worked prior to
1.1 either). You can't use svn ps to modify or set a property with a /
in it, though setting it though the perl-swig bindings does work since
that's what SVN:Mirror does.
Michael
On Jul 19, 2004, at 3:23 PM, Ben Reser wrote:
> On Tue, Jul 20, 2004 at 12:16:56AM +0200, Julian Reschke wrote:
>> What's the use case for having control characters in properties?
>
> Someone was on IRC trying to put ANSI color codes in their response
> from
> their hooks. Guess they had a syntax checker and wanted to show green
> for files that passed and red for files that failed. I can imagine the
> same request is there for log messages.
>
> Given the dangers of control characters, I don't think we should bother
> to try and support them. Otherwise we end up trying to decide when to
> filter them from the terminal. And open ourself up as an attack vector
> against some buggy terminal.
>
> --
> Ben Reser <be...@reser.org>
> http://ben.reser.org
>
> "Conscience is the inner voice which warns us somebody may be looking."
> - H.L. Mencken
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Ben Reser <be...@reser.org>.
On Tue, Jul 20, 2004 at 12:16:56AM +0200, Julian Reschke wrote:
> What's the use case for having control characters in properties?
Someone was on IRC trying to put ANSI color codes in their response from
their hooks. Guess they had a syntax checker and wanted to show green
for files that passed and red for files that failed. I can imagine the
same request is there for log messages.
Given the dangers of control characters, I don't think we should bother
to try and support them. Otherwise we end up trying to decide when to
filter them from the terminal. And open ourself up as an attack vector
against some buggy terminal.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Julian Reschke <ju...@gmx.de>.
Peter N. Lundblad wrote:
> On Mon, 19 Jul 2004, Simon Odou wrote:
>
>
>>For the same project, the svn behavior is different with "file://" and
>>"http://". Commits with particular log message can be rejected with "http" but
>>accepted with "file". Then the command "svn log" might fail.
>>
>>$ svn co http://localhost/test && cd test
>>Checked out revision 0.
>>$ touch a && svn add a
>>A a
>>$ echo "a" >> a && svn ci -m `echo -ne "\0x1"`
>>svn: Commit failed (details follow):
>>svn: applying log message
>>to /test/!svn/wbl/0f562004-95df-0310-837a-e85ba3689386/0: 400 Bad Request
>>(http://localhost)
>>
>
> [...]
>
> This is because there is no way to put control characters in XML (well,
> one could introduce ones own escaping mechanism, see last weeks discussion
> about control chars in file names). I think we should filter out
> characters that are not valid XML characters. OK, we could base64-encode
Control characters in *filenames* aren't problem for WebDAV, because
WebDAV is talking about URIs (which never ever contain control
characters, using %hh escaping).
If control characters appear anywhere else in WebDAV XML
request/response bodies (properties?), that simply should be fixed.
> them, but does the DAV elements allow that? Also, are those potential
It allows anything you want that is compatible to what the spec says,
but you can't expect interoperability with other clients/servers then.
> compability problems worht it just for the fun of being able to screw up
> someone else's terminal? :-) Couldn't we just check all svn: properties
> and be done with it?
>
> (Probably not for some reason I don't think of...)
What's the use case for having control characters in properties?
Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Klaus Rennecke <kr...@tigris.org>.
Julian Reschke wrote:
> [...] In WebDAV's XML
> request/response bodies, filenames never occur. WebDAV uses URIs, and
> URIs are plain ASCII.
Yes, that's because they are encoded.
> [...] The wish to introduce spaces in names often comes from people using a
> property name for *display* purposes. That's the wrong approach. If you
> need user-friendly display, you'll need to think about I18N as well;
> just adding whitespace doesn't solve that (think about that: do variable
> names in programming languages allow whitespace?).
Good point, I think you are right. I retract that idea about encoding
property names :-)
/Klaus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Julian Reschke <ju...@gmx.de>.
Klaus Rennecke wrote:
> If encoding names, I think the best approach would be to URL-encode
> them. Base64 bloats the text even if it's totally fine, but URL-encoding
> would leave the polite names readable.
I think it would be really great if we'd have a clear understanding
about where the problem lies (I don't yet). In WebDAV's XML
request/response bodies, filenames never occur. WebDAV uses URIs, and
URIs are plain ASCII.
If there *are* other parts of message bodies (properties?) that
transport filenames, it's absolutely a good idea to use URI-escaping for
those.
> That goes for file and property names alike.
>
> As for the dangers of control characters: File names have been an attack
> vector on unix for a loong time. Root guys should be aware of that, with
> or without subversion.
>
> The standard approach is to display a ? in place of control characters
> for user output. In this case however, I think it would be easier (like,
> no-op) to just leave the URL-encoded string as-is except for actually
> creating the files. And this would have the added bonus that wrapping
> tools had a real chance at safely parsing it.
>
> This would allow spaces in the property names as well. I think there was
> at least one request on users@ about this. It will not break the skel
> syntax if you just leave it URL-encoded in the database. All existing
> properties except the ones with enclosed % characters will remain
> unaffected.
Control characters (and other non-name characters) in property names are
deeply incompatible with WebDAV and also other metadata formats (such as
RDF or DC). I think it's a very bad idea to introduce them.
The wish to introduce spaces in names often comes from people using a
property name for *display* purposes. That's the wrong approach. If you
need user-friendly display, you'll need to think about I18N as well;
just adding whitespace doesn't solve that (think about that: do variable
names in programming languages allow whitespace?).
Best regards, Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by Klaus Rennecke <kr...@tigris.org>.
Peter N. Lundblad wrote:
> [...] I think we should filter out
> characters that are not valid XML characters. OK, we could base64-encode
> them, but does the DAV elements allow that? Also, are those potential
> compability problems worht it just for the fun of being able to screw up
> someone else's terminal? [...]
If encoding names, I think the best approach would be to URL-encode
them. Base64 bloats the text even if it's totally fine, but URL-encoding
would leave the polite names readable.
That goes for file and property names alike.
As for the dangers of control characters: File names have been an attack
vector on unix for a loong time. Root guys should be aware of that, with
or without subversion.
The standard approach is to display a ? in place of control characters
for user output. In this case however, I think it would be easier (like,
no-op) to just leave the URL-encoded string as-is except for actually
creating the files. And this would have the added bonus that wrapping
tools had a real chance at safely parsing it.
This would allow spaces in the property names as well. I think there was
at least one request on users@ about this. It will not break the skel
syntax if you just leave it URL-encoded in the database. All existing
properties except the ones with enclosed % characters will remain
unaffected.
/Klaus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn log causes XML parse error (internal error)
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Mon, 19 Jul 2004, Simon Odou wrote:
> For the same project, the svn behavior is different with "file://" and
> "http://". Commits with particular log message can be rejected with "http" but
> accepted with "file". Then the command "svn log" might fail.
>
> $ svn co http://localhost/test && cd test
> Checked out revision 0.
> $ touch a && svn add a
> A a
> $ echo "a" >> a && svn ci -m `echo -ne "\0x1"`
> svn: Commit failed (details follow):
> svn: applying log message
> to /test/!svn/wbl/0f562004-95df-0310-837a-e85ba3689386/0: 400 Bad Request
> (http://localhost)
>
[...]
This is because there is no way to put control characters in XML (well,
one could introduce ones own escaping mechanism, see last weeks discussion
about control chars in file names). I think we should filter out
characters that are not valid XML characters. OK, we could base64-encode
them, but does the DAV elements allow that? Also, are those potential
compability problems worht it just for the fun of being able to screw up
someone else's terminal? :-) Couldn't we just check all svn: properties
and be done with it?
(Probably not for some reason I don't think of...)
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org