You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Ben Reser <be...@reser.org> on 2004/07/12 21:59:51 UTC
Re: some valid Linux filenames break subversion
On Mon, Jul 12, 2004 at 02:48:53PM -0000, kfogel@tigris.org wrote:
> More specifically, we put filenames in XML attributes, and there's no
> way (?) to encode these characters for use in XML attribute names or
> values. If we did the filenames as CDATA inside <name>...</name>
> element or something, then we could encode.
Uhh yeah what he said. :)
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Amelia A Lewis <al...@tibco.com>.
On Wed, 14 Jul 2004 13:54:51 -0400
Greg Hudson <gh...@MIT.EDU> wrote:
> On Wed, 2004-07-14 at 12:03, kfogel@collab.net wrote:
> > But if we *did* encode the attribute values somehow, we could support
> > all possible filenames.
>
> No, because we also pass filenames over HTTP, using standard DAV and
> DeltaV methods, and we can't just decide to base64-encode those
> pathnames.
At a guess, your best bet would be to URL-encode pathnames in the XML.
You then have to know to URL-decode them, and there is a potential for
problems if you're using URL encoding for other purposes, but it should
answer. This would allow roughly the same thing to be passed over
HTTP/DAV (although it may URL-encode more than is strictly necessary; I
don't recall offhand what overlap there is between the set of characters
illegal in an HTTP URL path and the set of characters illegal in XML).
> (If I understand correctly.)
What he said.
BTW, URL encoding might be an interesting solution, around 2.0, for
client-side handling of characters unsupported by the file system. For
instance, HFS permits slashes in names, but does not permit colon. Unix
filesystems are typically the reverse. In both cases, the reserved
character is the path separator. Unix would permit the filename
"Proposal%201%2f2" and HFS would permit "Proposal%3a%20Part%201" (or
"Proposal 1%2f2" and "Proposal%3a Part 1") as valid (if icky) filenames.
Might be problems with a filesystem that uses "%" as a path separator, I
suppose. The difficulty is in communicating with the repo: how can *it*
tell the difference between the file Proposal%201%2f2 and the path
Proposal%201/2 ? (where the filename is "2"; the directory is "Proposal 1"
and there are presumably several parts, of which this is the second).
Amy!
--
Amelia A. Lewis
Senior Architect
TIBCO/Extensibility, Inc.
alewis@tibco.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Julian Reschke <ju...@gmx.de>.
Greg Hudson wrote:
>>But if we *did* encode the attribute values somehow, we could support
>>all possible filenames.
>
>
> No, because we also pass filenames over HTTP, using standard DAV and
> DeltaV methods, and we can't just decide to base64-encode those
> pathnames.
>
> (If I understand correctly.)
That's no problem. In WebDAV you never pass *filenames*. You always pass
URIs, and in URIs control characters are escaped using the %hh notation
anyway.
Best regards, Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Greg Hudson <gh...@MIT.EDU>.
On Wed, 2004-07-14 at 12:03, kfogel@collab.net wrote:
> But if we *did* encode the attribute values somehow, we could support
> all possible filenames.
No, because we also pass filenames over HTTP, using standard DAV and
DeltaV methods, and we can't just decide to base64-encode those
pathnames.
(If I understand correctly.)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by kf...@collab.net.
Amelia A Lewis <al...@tibco.com> writes:
> If you want any characters in the range 0-31 or 128-159 (C0 and C1), apart
> from tab, cr, and lf, then you *must* encode them in some fashion, such as
> Base64. Their appearance, as characters, including such appearance in a
> CDATA attribute or a CDATA section (<![CDATA[ ]]>) is a well-formedness
> error, and it is illegal for the parser to attempt to continue parsing.
>
> These characters may *not* be represented by numeric character entities
> ( is as illegal as the appearance of a bare BEL character). I don't
> recall, off the top of my head, whether the appearance of such a character
> entity is treated as a well-formedness error (I think that it is), or is
> treated as an unrecognized entity (which, in fact, may be a
> well-formedness error as well; the rules on entities are fairly
> unpleasant, all things considered).
Ah.
Okay, so the real issue here is not
"There's no way to represent these funny characters in attribute
values"
but rather
"We (the Subversion project) are unwilling to Base64 or otherwise
encode the attribute values in our .svn/entries files, probably
because that would make them so much harder to debug."
But if we *did* encode the attribute values somehow, we could support
all possible filenames.
> Umm. No. If you are placing information that contains characters that
> fall into the Unicode C0 and C1 (control) sets, then you *must* encode
> that information, because those characters are *illegal*.
>
> Note that if you are using a character set encoding of some sort that
> happens to re-use the bit patterns of C0 for some printable unicode
> characters, that's legal (I can't think of any character set that does
> this, though; perhaps EBCDIC? Almost every other character set uses the
> ASCII bit patterns for the bottom seven bits, and Unicode uses the same
> mapping (ASCII is a proper subset of Unicode (at least of UTF-8; things
> are more complex with UTF-16, UCS-2, and the like))).
Thanks for the education! I'll be saving this mail.
> Did I close all my parens? My mail editor doesn't do paren matching ....
Yup, my editor thinks you did anyway :-).
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Amelia A Lewis <al...@tibco.com>.
On Wed, 14 Jul 2004 10:24:39 -0500
kfogel@collab.net wrote:
> I'm operating under the assumption that XML has no standard way, in
> attribute values, to represent every possible character. Whereas in
> CDATA, there *is* a way to represent every possible character.
Umm, no, there isn't.
CDATA has precisely the same set of restrictions as PCDATA (and attributes
default to CDATA, using DTD), except that CDATA doesn't interpret pointy
brackets and ampersands, so it isn't necessary to escape them.
If you want any characters in the range 0-31 or 128-159 (C0 and C1), apart
from tab, cr, and lf, then you *must* encode them in some fashion, such as
Base64. Their appearance, as characters, including such appearance in a
CDATA attribute or a CDATA section (<![CDATA[ ]]>) is a well-formedness
error, and it is illegal for the parser to attempt to continue parsing.
These characters may *not* be represented by numeric character entities
( is as illegal as the appearance of a bare BEL character). I don't
recall, off the top of my head, whether the appearance of such a character
entity is treated as a well-formedness error (I think that it is), or is
treated as an unrecognized entity (which, in fact, may be a
well-formedness error as well; the rules on entities are fairly
unpleasant, all things considered).
> If there were a way to do it in attributes as well, then we'd be fine
> (even if that way were different from the CDATA way).
Umm. No. If you are placing information that contains characters that
fall into the Unicode C0 and C1 (control) sets, then you *must* encode
that information, because those characters are *illegal*.
Note that if you are using a character set encoding of some sort that
happens to re-use the bit patterns of C0 for some printable unicode
characters, that's legal (I can't think of any character set that does
this, though; perhaps EBCDIC? Almost every other character set uses the
ASCII bit patterns for the bottom seven bits, and Unicode uses the same
mapping (ASCII is a proper subset of Unicode (at least of UTF-8; things
are more complex with UTF-16, UCS-2, and the like))).
Did I close all my parens? My mail editor doesn't do paren matching ....
Amy!
(provoked into being an XML geek)
--
Amelia A. Lewis
Senior Architect
TIBCO/Extensibility, Inc.
alewis@tibco.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Julian Reschke <ju...@gmx.de>.
kfogel@collab.net wrote:
> Ben Reser <br...@siaer.com> writes:
>
>>On Tue, Jul 13, 2004 at 01:08:01AM +0200, Julian Reschke wrote:
>>
>>>>Uhh yeah what he said. :)
>>>
>>>Sorry? The set of valid characters inside a CDATA section is exactly the
>>>same as outside; just the escaping rules differ. If you want to
>>>represent control characters (other than TAB, CR, LF) inside XML, you
>>>need to use a custom escape format (such as BASE64, quoted printable or
>>>URI-percent encoding).
>>
>>That's what I thought he meant...
>
> Right, that's what I meant.
>
> I'm operating under the assumption that XML has no standard way, in
> attribute values, to represent every possible character. Whereas in
> CDATA, there *is* a way to represent every possible character.
No. Again: the set of legal characters in XML content is the same for
attributes, plain text or CDATA sections. Only the escaping rules
differ. You can't have any control character except CR, LF and TAB
inside XML content.
> If there were a way to do it in attributes as well, then we'd be fine
> (even if that way were different from the CDATA way).
Best regards,
Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by kf...@collab.net.
Ben Reser <br...@siaer.com> writes:
> On Tue, Jul 13, 2004 at 01:08:01AM +0200, Julian Reschke wrote:
> > >Uhh yeah what he said. :)
> >
> > Sorry? The set of valid characters inside a CDATA section is exactly the
> > same as outside; just the escaping rules differ. If you want to
> > represent control characters (other than TAB, CR, LF) inside XML, you
> > need to use a custom escape format (such as BASE64, quoted printable or
> > URI-percent encoding).
>
> That's what I thought he meant...
Right, that's what I meant.
I'm operating under the assumption that XML has no standard way, in
attribute values, to represent every possible character. Whereas in
CDATA, there *is* a way to represent every possible character.
If there were a way to do it in attributes as well, then we'd be fine
(even if that way were different from the CDATA way).
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Ben Reser <br...@siaer.com>.
On Tue, Jul 13, 2004 at 01:08:01AM +0200, Julian Reschke wrote:
> >Uhh yeah what he said. :)
>
> Sorry? The set of valid characters inside a CDATA section is exactly the
> same as outside; just the escaping rules differ. If you want to
> represent control characters (other than TAB, CR, LF) inside XML, you
> need to use a custom escape format (such as BASE64, quoted printable or
> URI-percent encoding).
That's what I thought he meant...
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Ben Reser <br...@siaer.com>.
On Tue, Jul 13, 2004 at 12:37:44AM +0100, Barry Scott wrote:
> Doesn't the &#nnn; syntax work in attribute values?
>
> <example ctrl_c=""/>
Yes, but control characters are explicitly disallowed in XML:
http://www.w3.org/TR/2004/REC-xml-20040204/#charsets
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Julian Reschke <ju...@gmx.de>.
Barry Scott wrote:
> Doesn't the &#nnn; syntax work in attribute values?
>
> <example ctrl_c=""/>
> ...
No. The set of valid *characters* is the same everywhere in text
content; whether it's an attribute or in a CDATA section just changes
the escaping rules. And in XML 1.0, any control character except TAB, LF
and CR is forbidden.
Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Barry Scott <ba...@barrys-emacs.org>.
Doesn't the &#nnn; syntax work in attribute values?
<example ctrl_c=""/>
Barry
On Jul 13, 2004, at 00:08, Julian Reschke wrote:
> Ben Reser wrote:
>> On Mon, Jul 12, 2004 at 02:48:53PM -0000, kfogel@tigris.org wrote:
>>> More specifically, we put filenames in XML attributes, and there's no
>>> way (?) to encode these characters for use in XML attribute names or
>>> values. If we did the filenames as CDATA inside <name>...</name>
>>> element or something, then we could encode.
>> Uhh yeah what he said. :)
>
> Sorry? The set of valid characters inside a CDATA section is exactly
> the same as outside; just the escaping rules differ. If you want to
> represent control characters (other than TAB, CR, LF) inside XML, you
> need to use a custom escape format (such as BASE64, quoted printable
> or URI-percent encoding).
>
> Best regards, Julian
>
> --
> <green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: some valid Linux filenames break subversion
Posted by Julian Reschke <ju...@gmx.de>.
Ben Reser wrote:
> On Mon, Jul 12, 2004 at 02:48:53PM -0000, kfogel@tigris.org wrote:
>
>>More specifically, we put filenames in XML attributes, and there's no
>>way (?) to encode these characters for use in XML attribute names or
>>values. If we did the filenames as CDATA inside <name>...</name>
>>element or something, then we could encode.
>
>
> Uhh yeah what he said. :)
Sorry? The set of valid characters inside a CDATA section is exactly the
same as outside; just the escaping rules differ. If you want to
represent control characters (other than TAB, CR, LF) inside XML, you
need to use a custom escape format (such as BASE64, quoted printable or
URI-percent encoding).
Best regards, Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org