issue report - problem with command line argument parsing using tr_TR locale

[Ozgur Murat Homurlu <oz...@gmx.net>]

Hello,

Trying a to use "COMMITTED" revision using tr_TR locale (normally
LC_CTYPE is an exported environment variable, it's explicitly set here
to better demonstrate the issue):

$ LC_CTYPE=tr_TR svn log -r COMMITTED
svn: Syntax error in revision argument 'COMMITTED'
$


Workarounds:
-------------------
$ LC_CTYPE=C svn log -r COMMITTED
(works normally)

$ LC_CTYPE=tr_TR svn log -r COMMiTTED
(works normally)

$ LC_CTYPE=tr_TR svn log -r committed
(works normally)


Some insight (possibly related with the issue):
----------------------------------------------------------------
There's a well known cause of problems related to tr_TR locale: lower
case of the character 'I' is 'ı' with tr_TR locale (not the
English-usual 'i'). So if you use C library's "tolower" type function
with tr_TR locale you shouldn't expect i.e. 'COMMITTED' converted to
'committed'. Correct programming style is setting the correct locale (
for the word ) before using case conversions.

Please CC me for the answers and questions, I'm not subscribed to the
list.
Thank you,

Regards...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: issue report - problem with command line argument parsing using tr_TR locale

[Philip Martin <ph...@codematters.co.uk>]

Julian Foad <ju...@btopenworld.com> writes:

> * subversion/libsvn_subr/opt.c (svn_opt_subcommand_help):
>   Allow revision keywords to be upper case or lower case but not mixed
>   case.

As far as it goes it looks reasonable to me, however strcasecmp is
also used by the config stuff and the keyword stuff, so I guess these
have the same problem.  Perhaps we need a more general solution?  An
svn_ascii_casecmp() that doesn't use the locale perhaps?

-- 
Philip Martin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: issue report - problem with command line argument parsing using tr_TR locale

[Julian Foad <ju...@btopenworld.com>]

Ozgur Murat Homurlu wrote:
> 
> $ LC_CTYPE=tr_TR svn log -r COMMITTED
> svn: Syntax error in revision argument 'COMMITTED'
[...]
> $ LC_CTYPE=tr_TR svn log -r COMMiTTED
> (works normally)
[...]
> There's a well known cause of problems related to tr_TR locale: lower
> case of the character 'I' is 'ı' with tr_TR locale (not the
> English-usual 'i'). So if you use C library's "tolower" type function
> with tr_TR locale you shouldn't expect i.e. 'COMMITTED' converted to
> 'committed'. Correct programming style is setting the correct locale (
> for the word ) before using case conversions.

We had better fix this as best we can, perhaps by doing the case-insensitive comparison correctly, but...

I would use this as an example of how case-insensitivity leads to problems, and recommend again that, when possible, we allow only one version (e.g. the all-capitals version) of such keywords.  The writers of the XML standard decided that it must be case-sensitive, unlike HTML, for this reason that non-English languages do not always have simple one-to-one mappings between upper and lower case.

It's probably too late now to disallow lower case for these keywords, so close to the first release, but perhaps we could get away with explicitly allowing the all-lower and all-upper-case variants, but not arbitrary mixed-case variants.  The fewer variations the better.

Patch attached for your consideration.

I understand that this is probably not urgent enough to fix in 1.0.0 and that if we release 1.0.0 allowing mixed case then we shouldn't really retract that afterwards.

- Julian

Re: issue report - problem with command line argument parsing using tr_TR locale

[Tobias Ringström <to...@ringstrom.mine.nu>]

Ozgur Murat Homurlu wrote:

>Some insight (possibly related with the issue):
>----------------------------------------------------------------
>There's a well known cause of problems related to tr_TR locale: lower
>case of the character 'I' is 'ı' with tr_TR locale (not the
>English-usual 'i'). So if you use C library's "tolower" type function
>with tr_TR locale you shouldn't expect i.e. 'COMMITTED' converted to
>'committed'. Correct programming style is setting the correct locale (
>for the word ) before using case conversions.
>  
>
I agree that strcmp of tolower:ed strings would be a problem, but 
Subversion uses libc's strcasecmp function for these comparisons, e.g:

    strcasecmp (word, "committed")

It sounds like a libc bug to me. I wonder if it would cause a similar 
problem for some other locale if that was changed to

    strcasecmp (word, "COMMITTED")

Anybody?

/Tobias


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org