You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Greg Hudson <gh...@MIT.EDU> on 2004/06/05 16:38:57 UTC
Re: URI encoding URLs on the cmdline?
On Sat, 2004-06-05 at 12:46, Peter N. Lundblad wrote:
> Hi,
>
> I'm still on issue 1770. There is a comment that really doesn't belong to
> the issue, but that's still interesting. Why don't we URI-encode the URLs
> given on the command line for the user?
I assume we just never implemented it. I don't see a problem with doing
that.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: RFC: URL encoding switch and option (was Re: URI encoding URLs on the cmdline?)
Posted by Pierre THIERRY <pi...@moine-fou.org>.
> Maybe this clears things out
Sure. I thought the IRI thing was a separate problem, so I didn't read
carefully the thread on it...
Lightly,
le Moine Fou
--
pierre.thierry@moine-fou.org
OpenPGP 0xD9D50D8A
Re: RFC: URL encoding switch and option (was Re: URI encoding URLs
on the cmdline?)
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Fri, 11 Jun 2004, Pierre THIERRY wrote:
> No need of my 'knob' if we have a magical way of knowing when and what
> to encode without any hint from the user. I'm pretty impatient of
> discovering the guessing algorithm...
>
I don't know what guessing algorithm you are referring to. I'm not talking
about guessing anything. You should read up on IRIs. There is a draft for
an RFC by Martin Dúrst. I don't have the URL handy, but you should be able
to find it via W3C or IETF.
As I explained earlier, to support IRIs, we just need to %-escape bytes
between 0x80 and 0xff. We already have the input in UTF-8. ONe concern is
that the IRI draft specifies that when converting from other encodings to
UTF-8, Unicode Normalization form C. I don't know if apr-iconv can
guarantee this. In any case, we can't do this normalization in the IRI ->
URI function, since the spec also says that if the input already is in an
Unicode encoding (which it is if the user uses an UTF-8 locale, for
example), we shall not do any normalization. But this isn't about
guessing, so let's leave it for now. So, IRI support is straight-forward.
Then, I also proposed to automatically escape some characters that are
always illegal in an URI. For example, space is illegal, so if we
encounter a space, we can just encode it as %20. Then, I suggested that we
can be more smart than that. For example, when we are in the path
component, [] are *not* reserved (they are in the eariler parts, for IPv6
addresses). This is no guessing, since we can always parse the URI. But it
is more complex, and may seem less predictable to the user, so whether
this is desirable can be debated further.
Maybe this clears things out, or I (and Greg) have missed something.
Regards,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: RFC: URL encoding switch and option (was Re: URI encoding URLs
on the cmdline?)
Posted by Greg Hudson <gh...@MIT.EDU>.
On Thu, 2004-06-10 at 19:19, Pierre THIERRY wrote:
> No need of my 'knob' if we have a magical way of knowing when and what
> to encode without any hint from the user. I'm pretty impatient of
> discovering the guessing algorithm...
Web browsers do this without a knob; we can do so as well. Impatience
is not a reason to complicate the UI.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: RFC: URL encoding switch and option (was Re: URI encoding URLs on the cmdline?)
Posted by Pierre THIERRY <pi...@moine-fou.org>.
> -1. We don't need a knob for this.
No need of my 'knob' if we have a magical way of knowing when and what
to encode without any hint from the user. I'm pretty impatient of
discovering the guessing algorithm...
Curiously,
le Moine Fou
--
pierre.thierry@moine-fou.org
OpenPGP 0xD9D50D8A
Re: RFC: URL encoding switch and option (was Re: URI encoding URLs
on the cmdline?)
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Thu, 10 Jun 2004, Greg Hudson wrote:
> On Thu, 2004-06-10 at 18:45, Pierre THIERRY wrote:
> > The easiest way to deal with URL encoding seems to me to add an option
> > for it:
>
> -1. We don't need a knob for this.
>
>
Exactly. A URI is an IRI, so there is no backwards compatibility problem.
I hadn't read up on IRIs when I wrote the thing about compatibility.
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: RFC: URL encoding switch and option (was Re: URI encoding URLs
on the cmdline?)
Posted by Greg Hudson <gh...@MIT.EDU>.
On Thu, 2004-06-10 at 18:45, Pierre THIERRY wrote:
> The easiest way to deal with URL encoding seems to me to add an option
> for it:
-1. We don't need a knob for this.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RFC: URL encoding switch and option (was Re: URI encoding URLs on the cmdline?)
Posted by Pierre THIERRY <pi...@moine-fou.org>.
> We could have compatibility problems if scripts assume they have to
> encode the URLs themselves.
I don't think the default behaviour should be changed. SVN should
continue expecting URIs.
The easiest way to deal with URL encoding seems to me to add an option
for it:
--url-encode (-e) LOCATION
Apply the RFC 2396 encoding to non-ASCII charecters in LOCATION and
use the result as the repository URL.
--url-encode-all (-E) LOCATION
Apply the RFC 2396 encoding to non-ASCII and reserved characters
(except '/') to LOCATION and use the result as the repository URL.
And there should also be a configuration option, maybe in the
miscellany section:
url-encode
This tells Subversion when to automatically encode the repository
location. The flag has three possible values:
- none. This is the default value. If the location is not a valid
URL, the user will get an error.
- non-ascii. It's equivalent to using the command-line -e switch.
- all. It's equivalent to using the command-line -E switch.
Optionally,
le Moine Fou
--
pierre.thierry@moine-fou.org
OpenPGP 0xD9D50D8A
Re: URI encoding URLs on the cmdline?
Posted by Ben Collins-Sussman <su...@collab.net>.
On Sat, 2004-06-05 at 18:52, C. Michael Pilato wrote:
> "error: You might need to URI-encode some of your URLs, namely the
> ones that have a percent sign (%) in the decoded path,
> because rather than encourage proper user behavior, we
> want to cater to mindless masses."
This is a really difficult UI question. I totally sympathize with
cmpilato's desire to stick to KISS designs:
I mean, on the one hand, we seem to
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Julian Reschke <ju...@gmx.de>.
Jim Correia wrote:
> ...
> The rule would have to be more strict than "you can use any character
> you want in a URI
> on the command line except for '%'". The rule would have to apply to all
> URI reserved characters, not just % since there is no way to know
> whether the author of the URI intended for the character to be escaped
> or not. In fact RFC 2396 says that one needs to consider URIs always
> escaped (2.4.2) for this very reason.
Correct. There's no such thing as an "unescaped" URI. If it needs
escaping, it's not yet a URI.
That being said, a mechanism to use *IRIs* probably would be useful
(even iff they'd need to be explicitly markes as IRI).
Julian
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Sun, 6 Jun 2004, Greg Hudson wrote:
> On Sun, 2004-06-06 at 08:56, Jim Correia wrote:
> > The rule would have to be more strict than "you can use any character
> > you want in a URI
> > on the command line except for '%'". The rule would have to apply to
> > all URI reserved characters, not just % since there is no way to know
> > whether the author of the URI intended for the character to be escaped
> > or not. In fact RFC 2396 says that one needs to consider URIs always
> > escaped (2.4.2) for this very reason.
>
> We're mostly talking about the path part here, where only a few reserved
> characters are disallowed. But you're right, characters like '?' and
> '#' are also at issue, not just '%'.
>
> I remain confident that we would be doing our users a service by
> auto-escaping international characters in URLs, even if we remain picky
I fully aggree. Try using a non-ASCII charset for a while. Even if I know
how to URI-encode my räksmörgås in principal, I don't have an
UTF-8-encoder built in to my head.
To make things simple, we could say:
Use every character you want, but if a pathname component contains one of
;?:@&=+$%, (/ excluded for obvious reasons)
you have to escape it using %hh. If you don't trust us in escaping your
characters, you are free to do it yourself.
This is backwards compatible and more user friendly. The above nine
characters should be quite uncommon in our URLs, but they can still be
used if necessary.
As Greg points out, for the path component, we could remove :@&+$, leaving
just ; (parameter
info), ? (delimits query string, which we don't use, but anyway) and =
(which is also reserved). But that's not necessary and would be
backwrads-compatible if we want to do it in the future. This is according
to 3.3. in RFC2396.
We could be more smart, like escaping an percent sing not followed by two
hex digits, but *this* is where the rules start to get confusing.
I *really* can't see the problem with this at all. If we take I18N
seriously, we need to do this IMO.
Regards,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sun, 2004-06-06 at 08:56, Jim Correia wrote:
> The rule would have to be more strict than "you can use any character
> you want in a URI
> on the command line except for '%'". The rule would have to apply to
> all URI reserved characters, not just % since there is no way to know
> whether the author of the URI intended for the character to be escaped
> or not. In fact RFC 2396 says that one needs to consider URIs always
> escaped (2.4.2) for this very reason.
We're mostly talking about the path part here, where only a few reserved
characters are disallowed. But you're right, characters like '?' and
'#' are also at issue, not just '%'.
I remain confident that we would be doing our users a service by
auto-escaping international characters in URLs, even if we remain picky
about reserved punctuation. There's no need to introduce CMike's
hypothetical verbose error message; if we see confusing puncutation, we
can bounce out the URI with exactly the same error message as we do
now. Since the URI encoding rules are already complex, the naive user
already has no hope of understanding what to do; we wouldn't be making
that any different, just less frequent.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Jim Correia <ji...@pobox.com>.
On Jun 6, 2004, at 1:40 AM, Greg Hudson wrote:
> On Sat, 2004-06-05 at 23:51, Jim Correia wrote:
>>> How hard is it to explain "you can use any character you want in a
>>> URI
>>> on the command line except for '%'"?
>
>> The rule would have to be more strict than that - it would be an URI
>> reserved character. Unless you are the author of the URL you can't say
>> for certain whether the URI reserved char is being used for its
>> reserved purpose, or needs to be escaped.
>
> Please elaborate. I can't understand this argument, partially because
> of an unclear antecedent in the first sentence ("it" in "it would be an
> URI reserved character").
The rule would have to be more strict than "you can use any character
you want in a URI
on the command line except for '%'". The rule would have to apply to
all URI reserved characters, not just % since there is no way to know
whether the author of the URI intended for the character to be escaped
or not. In fact RFC 2396 says that one needs to consider URIs always
escaped (2.4.2) for this very reason.
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sat, 2004-06-05 at 23:51, Jim Correia wrote:
> > How hard is it to explain "you can use any character you want in a URI
> > on the command line except for '%'"?
> The rule would have to be more strict than that - it would be an URI
> reserved character. Unless you are the author of the URL you can't say
> for certain whether the URI reserved char is being used for its
> reserved purpose, or needs to be escaped.
Please elaborate. I can't understand this argument, partially because
of an unclear antecedent in the first sentence ("it" in "it would be an
URI reserved character").
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Jim Correia <ji...@pobox.com>.
On Jun 5, 2004, at 10:50 PM, Greg Hudson wrote:
> On Sat, 2004-06-05 at 21:32, Ben Collins-Sussman wrote:
>> If we do "auto-escaping", then we've got users typing technically
>> invalid psuedo-urls, with fuzzy rules that convert things most of the
>> time. And it's hard to explain those rules.
>
> How hard is it to explain "you can use any character you want in a URI
> on the command line except for '%'"?
The rule would have to be more strict than that - it would be an URI
reserved character. Unless you are the author of the URL you can't say
for certain whether the URI reserved char is being used for its
reserved purpose, or needs to be escaped.
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Branko Čibej <br...@xbc.nu>.
Ben Collins-Sussman wrote:
>On Sun, 2004-06-06 at 12:51, Brian Denny wrote:
>
>
>>On Sat, Jun 05, 2004 at 10:50:54PM -0400, Greg Hudson wrote:
>>
>>
>>>>And then users start pasting those fuzzy urls into web browsers and
>>>>wondering why they don't work.
>>>>
>>>>
>>>What makes you think they don't work in web browsers? Web browsers are
>>>more than happy to auto-escape URIs. (I just checked; spaces work fine,
>>>and so do UTF-8 characters. % characters, not so much, for obvious
>>>reasons.)
>>>
>>>
>>To me, this is key. To the extent that the most widely used web browsers
>>accept unescaped URLs, svn should too.
>>
>>
>
>I have to agree. To me, it proves that (1) a "reasonable" amount of
>auto-escaping can be done without creating widespread confusion, and (2)
>users already expect this behavior.
>
>(The fact that users might be referring to "http://host/snögubbe" as a
>URL, when in fact it isn't, is something that only bothers programmers.)
>
>
+1 FWIW. I'd also lean strongly towards fixing this in 1.1 if at all
possible.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Ben Collins-Sussman <su...@collab.net>.
On Sun, 2004-06-06 at 12:51, Brian Denny wrote:
> On Sat, Jun 05, 2004 at 10:50:54PM -0400, Greg Hudson wrote:
> >
> > > And then users start pasting those fuzzy urls into web browsers and
> > > wondering why they don't work.
> >
> > What makes you think they don't work in web browsers? Web browsers are
> > more than happy to auto-escape URIs. (I just checked; spaces work fine,
> > and so do UTF-8 characters. % characters, not so much, for obvious
> > reasons.)
>
> To me, this is key. To the extent that the most widely used web browsers
> accept unescaped URLs, svn should too.
I have to agree. To me, it proves that (1) a "reasonable" amount of
auto-escaping can be done without creating widespread confusion, and (2)
users already expect this behavior.
(The fact that users might be referring to "http://host/snögubbe" as a
URL, when in fact it isn't, is something that only bothers programmers.)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Brian Denny <br...@briandenny.net>.
On Sat, Jun 05, 2004 at 10:50:54PM -0400, Greg Hudson wrote:
>
> > And then users start pasting those fuzzy urls into web browsers and
> > wondering why they don't work.
>
> What makes you think they don't work in web browsers? Web browsers are
> more than happy to auto-escape URIs. (I just checked; spaces work fine,
> and so do UTF-8 characters. % characters, not so much, for obvious
> reasons.)
To me, this is key. To the extent that the most widely used web browsers
accept unescaped URLs, svn should too.
-brian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Greg Hudson <gh...@MIT.EDU>.
On Sat, 2004-06-05 at 21:32, Ben Collins-Sussman wrote:
> If we do "auto-escaping", then we've got users typing technically
> invalid psuedo-urls, with fuzzy rules that convert things most of the
> time. And it's hard to explain those rules.
How hard is it to explain "you can use any character you want in a URI
on the command line except for '%'"?
> And then users start pasting those fuzzy urls into web browsers and
> wondering why they don't work.
What makes you think they don't work in web browsers? Web browsers are
more than happy to auto-escape URIs. (I just checked; spaces work fine,
and so do UTF-8 characters. % characters, not so much, for obvious
reasons.)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Ben Collins-Sussman <su...@collab.net>.
On Sat, 2004-06-05 at 18:52, C. Michael Pilato wrote:
> "error: You might need to URI-encode some of your URLs, namely the
> ones that have a percent sign (%) in the decoded path,
> because rather than encourage proper user behavior, we
> want to cater to mindless masses."
Let's try this again, grr.
On the one hand, we have no qualms about users encoding filenames using
their native locale. Non-ascii filenames "just work", and by virtue of
our general silence on the topic, we're implicitly encouraging users to
use them.
But on the other hand, there's a set of strings attached to this feature
which we don't advertise. Sure, you can version control non-ascii
paths, _as long as you understand how to escape them in URIs_. And any
user who deals with branches and tags is going to be using URIs. I
think it's a pretty ridiculous barrier to entry. People who don't
understand rules of URI escaping aren't "mindless masses", it's the
other way around. People who *do* understand URI escaping are elite web
haX0rs.
So there's no nice way out of this problem.
If we do "auto-escaping", then we've got users typing technically
invalid psuedo-urls, with fuzzy rules that convert things most of the
time. And it's hard to explain those rules. And then users start
pasting those fuzzy urls into web browsers and wondering why they don't
work.
But if we keep the status quo, then users are shocked into realization
that they suddenly have to learn how to do URI-escaping. svn comes
across as extremely unfriendly software. . The only fix for this is
education (i.e. explaining in the book, I guess.)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by "C. Michael Pilato" <cm...@collab.net>.
"Peter N. Lundblad" <pe...@famlundblad.se> writes:
> One could choose not to encode the % under the assumption that percent
> characters aren't very common in repository URLs.
<rant mood="snotty">
...and turn a simple error message of:
"error: You need to URI-encode your URLs"
into a confusing one:
"error: You might need to URI-encode some of your URLs, namely the
ones that have a percent sign (%) in the decoded path,
because rather than encourage proper user behavior, we
want to cater to mindless masses."
Subversion uses URLs, folks, not filesystem paths with URL-looking
prefixes on them.
</rant>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Sat, 5 Jun 2004, Jim Correia wrote:
> It is probably safe to URI encode non-ascii values based on the current
> locale. However, if you hit a URI reserved character, like %, it isn't
> safe to assume it is needs encoding. (If what follows are not two hex
> digits, the % needed encoded, but if what follows *is* two hex digits,
> it may still need encoding. Only the person who wrote the URI knows for
> sure.)
>
One could choose not to encode the % under the assumption that percent
characters aren't very common in repository URLs. So the user has to
encode %, but not "räksmörgås". I think that would be a better solution
than the current situation. As it stands now, using non-ASCII in
subversion URLs is just not very fun at all...
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by Jim Correia <ji...@pobox.com>.
On Jun 5, 2004, at 1:21 PM, Peter N. Lundblad wrote:
> I thik I'll try it out then sometimes.
>
> We could have compatibility problems if scripts assume they have to
> encode
> the URLs themselves. I don't know if this is a real concern.
Check the archives - this was discussed before.
It is probably safe to URI encode non-ascii values based on the current
locale. However, if you hit a URI reserved character, like %, it isn't
safe to assume it is needs encoding. (If what follows are not two hex
digits, the % needed encoded, but if what follows *is* two hex digits,
it may still need encoding. Only the person who wrote the URI knows for
sure.)
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: URI encoding URLs on the cmdline?
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Sat, 5 Jun 2004, Greg Hudson wrote:
> On Sat, 2004-06-05 at 12:46, Peter N. Lundblad wrote:
> > Hi,
> >
> > I'm still on issue 1770. There is a comment that really doesn't belong to
> > the issue, but that's still interesting. Why don't we URI-encode the URLs
> > given on the command line for the user?
>
> I assume we just never implemented it. I don't see a problem with doing
> that.
I thik I'll try it out then sometimes.
We could have compatibility problems if scripts assume they have to encode
the URLs themselves. I don't know if this is a real concern.
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org