You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Hojin Choi <py...@users.kldp.net> on 2004/04/28 05:27:05 UTC

Checking Invalidity of UTF8 filename, is that work properly?

I have a freebsd machine. and I use svn client ver. 1.0.1. from freebsd
ports.
And I am in migrating my cvs repository to that of subversion.

I was met this message when I was treating my hangul(Korean character) file
name.

$ svn add HHHH.txt
svn: Can't recode string

(above HHHH is really a korean character file name)

And I checked the source code where the message is created. and it is found
from
UTF8 handling file. Maybe it is not UTF8 conforming filename.
But that's not a utf8. It's euc-kr..

What if? Can subversion accept only UTF8?
Because of that reason, character encoding conversion must be occurred
automatically.
But the conversion is not smart yet, or subversion treat all non-ascii file
name as a utf8 one.

I found this string from version 1.0.1 release note.
"detect invalid UTF8 filenames when native locale is UTF8."

Is that related what I concerned?



Regards.

Hojin Choi.




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Checking Invalidity of UTF8 filename, is that work properly?

Posted by Hojin Choi <py...@users.kldp.net>.
Arrrrghs!!

$ export LC_CTYPE
$ LC_CTYPE=ko_KR.eucKR

Works well..

Thank you.

----- Original Message ----- 
From: Ulrich Eckhardt
To: users@subversion.tigris.org
Sent: Wednesday, April 28, 2004 4:25 PM
Subject: Re: Checking Invalidity of UTF8 filename, is that work properly?


Hojin Choi wrote:
> $ svn add HHHH.txt
> svn: Can't recode string
> (above HHHH is really a korean character file name)
>
> And I checked the source code where the message is created. and it is
found
> from UTF8 handling file. Maybe it is not UTF8 conforming filename.
> But that's not a utf8. It's euc-kr..
>
> What if? Can subversion accept only UTF8?
> Because of that reason, character encoding conversion must be occurred
> automatically.
> But the conversion is not smart yet, or subversion treat all non-ascii
file
> name as a utf8 one.

You're really close to the solution. ;)
The point is that Subversion uses UTF-8 internally for all paths. Anything
you
give it is converted to UTF-8 according to the current locale(!). Now, if
your locale says it is UTF-8 but your encoding is really EUC, this
conversion
fails. Luckily, as it might as well produce garbage.

So, the only thing to do is to set the locale correctly. LC_CTYPE is the
env.
variable IIRC, but check the manpages.

Uli


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Checking Invalidity of UTF8 filename, is that work properly?

Posted by Ulrich Eckhardt <ec...@satorlaser.com>.
Hojin Choi wrote:
> $ svn add HHHH.txt
> svn: Can't recode string
> (above HHHH is really a korean character file name)
>
> And I checked the source code where the message is created. and it is found
> from UTF8 handling file. Maybe it is not UTF8 conforming filename.
> But that's not a utf8. It's euc-kr..
>
> What if? Can subversion accept only UTF8?
> Because of that reason, character encoding conversion must be occurred
> automatically.
> But the conversion is not smart yet, or subversion treat all non-ascii file
> name as a utf8 one.

You're really close to the solution. ;)
The point is that Subversion uses UTF-8 internally for all paths. Anything you 
give it is converted to UTF-8 according to the current locale(!). Now, if 
your locale says it is UTF-8 but your encoding is really EUC, this conversion 
fails. Luckily, as it might as well produce garbage.

So, the only thing to do is to set the locale correctly. LC_CTYPE is the env. 
variable IIRC, but check the manpages.

Uli


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org