You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Заболотный Андрей <za...@yandex.ru> on 2012/11/15 11:50:20 UTC
Renaming UTF-8 file names is broken
Hello!
I've got an issue here. We have a lot of file names in repository using UTF-8 characters. All worked well till I decided to do move one of these files to another directory.
The 'svn move' command worked fine. But when I tried to commit the transaction, that's what I've got:
Adding doc/forth-asm/Карта памяти.txt
svn: E155011: Commit failed (details follow):
svn: E155011: File '/home/zap/projects/СУ АКБ/doc/forth-asm/Карта памяти.txt' is out of date
svn: E160013: File not found: transaction '12-m', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
svn: E155011: Your commit message was left in a temporary file:
svn: E155011: '/home/zap/projects/svn-commit.tmp'
It looks like it tries to access the file with the wrong filename containing all those %D0 etc, and naturally fails.
Can this be worked around somehow, or it's a dead end?
--
Andrew
Re: Renaming UTF-8 file names is broken
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Nov 15, 2012 at 02:50:20PM +0400, Заболотный Андрей wrote:
> Hello!
>
> I've got an issue here. We have a lot of file names in repository using UTF-8 characters. All worked well till I decided to do move one of these files to another directory.
Hmm, that looks bad and unexpected.
Which version of Subversion (server and client) are you running?
Is a write-through proxy involved?
> The 'svn move' command worked fine. But when I tried to commit the transaction, that's what I've got:
>
> Adding doc/forth-asm/Карта памяти.txt
> svn: E155011: Commit failed (details follow):
> svn: E155011: File '/home/zap/projects/СУ АКБ/doc/forth-asm/Карта памяти.txt' is out of date
> svn: E160013: File not found: transaction '12-m', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
> svn: E155011: Your commit message was left in a temporary file:
> svn: E155011: '/home/zap/projects/svn-commit.tmp'
>
> It looks like it tries to access the file with the wrong filename containing all those %D0 etc, and naturally fails.
>
> Can this be worked around somehow, or it's a dead end?
You could try a URL->URL move, i.e. without using a working copy.
Maybe that will work around the issue.
Re: Renaming UTF-8 file names is broken
Posted by Andreas Mohr <an...@lisas.de>.
Hi,
On Sat, Nov 17, 2012 at 12:59:15AM +0400, Заболотный Андрей wrote:
> Okay, so thanks to Nico Kadel-Garcia I've upgraded subversion to version 1.7.7 on my Centos server.
>
> Unfortunately, this did not help. The error is still the same:
>
> svn: E160013: File not found: transaction '18-16', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
> svn: E160013: Your commit message was left in a temporary file:
> svn: E160013: 'svn-commit.2.tmp'
>
> I'm out of ideas.
Since that discussion has been a bit longer now and you're at a bit of a
loss, I'll try to explain things a bit.
I've recently started having to fight with an overly nonreliable
and non-supported (non-)interop(er)ability product "of a large so-called PC software company".
That product did not support international (read: non-ASCII range)
handling yet, thus I had to add support for that (I could send you the patch
to have a look at what I had to do to fix the problems in my case).
Encoding of URIs is usually governed by RFC 2396 originally.
This RFC strictly covered ASCII range content only.
It indicates which set of within-payload-content (i.e., between-delimiters!)
characters to encode (in order to avoid having a payload character end up
misinterpreted as a delimiter).
In your case, we *are* talking about international characters.
As far as RFC 2396 is concerned, it does not say much about non-ASCII
range, thus AFAICS it usually is chosen to simply hex-encode all non-ASCII chars
as well.
Possibly for RFC2396 purposes internationalization is simply not allowed
in its scope at all. RFC3986 (http://www.ietf.org/rfc/rfc3986.txt)
says the following:
"
Percent-encoded
octets (Section 2.1) may be used within a URI to represent characters
outside the range of the US-ASCII coded character set if this
Berners-Lee, et al. Standards Track [Page 8]
RFC 3986 URI Generic Syntax January 2005
representation is allowed by the scheme or by the protocol element in
which the URI is referenced. Such a definition should specify the
character encoding used to map those characters to octets prior to
being percent-encoded for the URI.
"
In your case the problem could be that SVN knows to correctly apply hex encoding
of non-ASCII range chars in user-defined content
(i.e. specific directory names, file names, ...).
What then might be what happens is that the *other* party sends
back a request (a possibly rather unrelated one some time later!!)
with exactly this hex-encoded string data content
(the actual meaning of that content may be unknown to the sender,
i.e. it's just an opaque "token"/"handle"),
which SVN then fails to *decode* symmetrically prior to doing
actual filesystem item lookup.
General note: encoding handling should always be perfectly symmetric
(and thus reversible without any information loss!!),
and *layered* (i.e. URI en/decoding requirements should be handled by a
different transcoding layer than e.g. predefined entity escape
requirements for transport in XML protocol,
and further transport paths might need to add another transcoding layer!).
It's just a "do the required and correct transcoding per each transport mechanism" thing.
There might also be issues such as *duplicate* encoding involved
(in which case hex-encoding would be (in most cases improperly)
changed into hex-encoded hex-encoding,
and the receiving party would then of course only decode *once*
and thus end up with one hex-encoding remaining rather than
original data).
Or it might be server vs. client mismatching in their implementation
conformance (e.g. one conforms to RFC2396 vs. the other RFC3986,
as one example).
Please take my statements with a grain of salt since it was a very Q&D
report without much research. Anyway, these pointers should be quite
useful and to the point, especially since I had to go through pretty
much the same thing recently.
HTH,
Andreas Mohr
--
GNU/Linux. It's not the software that's free, it's you.
Re: Renaming UTF-8 file names is broken
Posted by Stefan Sperling <st...@elego.de>.
On Sat, Nov 17, 2012 at 12:59:15AM +0400, Заболотный Андрей wrote:
> Okay, so thanks to Nico Kadel-Garcia I've upgraded subversion to version 1.7.7 on my Centos server.
>
> Unfortunately, this did not help. The error is still the same:
>
> svn: E160013: File not found: transaction '18-16', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
> svn: E160013: Your commit message was left in a temporary file:
> svn: E160013: 'svn-commit.2.tmp'
>
> I'm out of ideas.
Can you try to commit the rename over the svn:// or file:// repository
access method to see if that works?
Does anything in your httpd configuration munge URLs?
Are you using a write-through proxy setup?
Re: Renaming UTF-8 file names is broken
Posted by Заболотный Андрей <za...@yandex.ru>.
Okay, so thanks to Nico Kadel-Garcia I've upgraded subversion to version 1.7.7 on my Centos server.
Unfortunately, this did not help. The error is still the same:
svn: E160013: File not found: transaction '18-16', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
svn: E160013: Your commit message was left in a temporary file:
svn: E160013: 'svn-commit.2.tmp'
I'm out of ideas.
--
Andrew
Re: Renaming UTF-8 file names is broken
Posted by Nico Kadel-Garcia <nk...@gmail.com>.
On Fri, Nov 16, 2012 at 3:06 AM, Заболотный Андрей <za...@yandex.ru> wrote:
> Also, since the problem now seems server-related, here's the version information about the server:
>
> OS: CentOS 6.2
> Subversion: subversion-1.6.11-7.el6.x86_64, mod_dav_svn-1.6.11-7.el6.x86_64
>
> These are the latest versions from the repositories of CentOS. Is there a chance that if I install the latest SVN from sources the problem will be fixed?
Hop on over to https://github.com/nkadel/subversion-1.6.18-srpm and
grab my build tools. I've been busy at a new job and haven't had a
chance to push the updates to 1.6.19.
Re: Renaming UTF-8 file names is broken
Posted by Заболотный Андрей <za...@yandex.ru>.
Also, since the problem now seems server-related, here's the version information about the server:
OS: CentOS 6.2
Subversion: subversion-1.6.11-7.el6.x86_64, mod_dav_svn-1.6.11-7.el6.x86_64
These are the latest versions from the repositories of CentOS. Is there a chance that if I install the latest SVN from sources the problem will be fixed?
--
Andrew
Re: Renaming UTF-8 file names is broken
Posted by Заболотный Андрей <za...@yandex.ru>.
Found some interesting information in Apache log file.
Attached to this message are the related portions of the main and error log files.
Maybe they can shed some light on what happens.
--
Andrew
Re: Renaming UTF-8 file names is broken
Posted by Заболотный Андрей <za...@yandex.ru>.
15.11.2012, 17:14, "Daniel Shahaf" <d....@daniel.shahaf.name>:
> You could try 'svn rename URL URL'.
>
> What filesystem is your working copy on?
>
> Заболотный Андрей wrote on Thu, Nov 15, 2012 at 14:50:20 +0400:
>
>> Hello!
>>
>> I've got an issue here. We have a lot of file names in repository using UTF-8 characters. All worked well till I decided to do move one of these files to another directory.
>>
>> The 'svn move' command worked fine. But when I tried to commit the transaction, that's what I've got:
>>
>> Adding doc/forth-asm/Карта памяти.txt
>> svn: E155011: Commit failed (details follow):
>> svn: E155011: File '/home/zap/projects/СУ АКБ/doc/forth-asm/Карта памяти.txt' is out of date
>> svn: E160013: File not found: transaction '12-m', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
>> svn: E155011: Your commit message was left in a temporary file:
>> svn: E155011: '/home/zap/projects/svn-commit.tmp'
>>
>> It looks like it tries to access the file with the wrong filename containing all those %D0 etc, and naturally fails.
>>
>> Can this be worked around somehow, or it's a dead end?
svn rename URL1 URL2 results in a similar error:
# svn rename "https://xxx.yyy.ru/svn/СУ АКБ/tools/AVR/ПамятьAVR" "https://xxx.yyy.ru/svn/СУ АКБ/doc/forth-asm/Карта памяти.txt"
svn: E160013: File not found: transaction '12-n', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
svn: E160013: Your commit message was left in a temporary file:
svn: E160013: 'svn-commit.tmp'
The file system is ext4.
--
С уважением,
Андрей
Re: Renaming UTF-8 file names is broken
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
You could try 'svn rename URL URL'.
What filesystem is your working copy on?
Заболотный Андрей wrote on Thu, Nov 15, 2012 at 14:50:20 +0400:
> Hello!
>
> I've got an issue here. We have a lot of file names in repository using UTF-8 characters. All worked well till I decided to do move one of these files to another directory.
>
> The 'svn move' command worked fine. But when I tried to commit the transaction, that's what I've got:
>
> Adding doc/forth-asm/Карта памяти.txt
> svn: E155011: Commit failed (details follow):
> svn: E155011: File '/home/zap/projects/СУ АКБ/doc/forth-asm/Карта памяти.txt' is out of date
> svn: E160013: File not found: transaction '12-m', path '/%D0%A1%D0%A3%20%D0%90%D0%9A%D0%91/doc/forth-asm/%D0%9A%D0%B0%D1%80%D1%82%D0%B0%20%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D0%B8.txt'
> svn: E155011: Your commit message was left in a temporary file:
> svn: E155011: '/home/zap/projects/svn-commit.tmp'
>
> It looks like it tries to access the file with the wrong filename containing all those %D0 etc, and naturally fails.
>
> Can this be worked around somehow, or it's a dead end?
>
> --
> Andrew