You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by QXO <qx...@gmail.com> on 2013/05/23 04:13:34 UTC
svnadmin upgrade output message i18n issue
os: windows
encoding:GBK ( chcp 936 )
The svnadmin upgrade command output message first line encoding
issue(UTF-8 show in GBK),But the second line is right encoding!
已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
���������
if change console encoding to UTF-8 (chcp 65001),output message is :
Repository lock acquired.
Please wait; upgrading the repository may take some time...
Upgrade completed.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
I have download a binary package from win32svn[1], and confirmed your issue.
I check the subversion.mo file:
msgunfmt.exe subversion.mo -o subversion.po
It looks OK. Then I replaced the intl3_svn.dll file with gettext
0.18.2, it give me another output:
C:\var\tmp>svnadmin upgrade test
已取得版本库锁定。
请稍候；升级版本库可能需要一段时间...
?\205?\234?\179?\201?\201?\253?\188?\182?\161?\163
C:\var\tmp>
The I write a simple test program:
/*
* cl /MD /I. t-intl.c libintl-8.lib
*/
#include <stdio.h>
#include <locale.h>
#include <libintl.h>
#include <iconv.h>
#define _(S) gettext(S)
#define PACKAGE_NAME "subversion"
int main(int argc, char **argv)
{
iconv_t cd;
size_t nc, inbytesleft, outbytesleft;
char *msg, msg2[256];
setlocale(LC_ALL,"");
setlocale(LC_CTYPE, "");
bindtextdomain(PACKAGE_NAME, "../share/locale");
/* bind_textdomain_codeset(PACKAGE_NAME, "UTF-8"); */
textdomain(PACKAGE_NAME);
#undef printf
printf(_("Repository lock acquired.\n"
"Please wait; recovering the repository may take some time...\n"));
printf(_("\n"
"Upgrade completed.\n"));
return 0;
}
C:\var\tmp\svn-win32-1.7.9\bin>cl /nologo /MD /I. t-intl.c libintl-8.lib
C:\var\tmp\svn-win32-1.7.9\bin>t-intl.exe
已取得版本库锁定。
请稍候；修复版本库可能需要一段时间...
完成升级。
So this is a binary package build issue, not subversion issue.
[1] http://sourceforge.net/projects/win32svn/files/1.7.9/apache24/svn-win32-1.7.9-ap24.zip/download
On Thu, May 23, 2013 at 10:13 AM, QXO <qx...@gmail.com> wrote:
> os: windows
> encoding:GBK ( chcp 936 )
>
> The svnadmin upgrade command output message first line encoding
> issue(UTF-8 show in GBK),But the second line is right encoding!
>
> 宸插彇寰楃増鏈簱閿佸畾銆?璇风◢鍊欙紱鍗囩骇鐗堟湰搴撳彲鑳介渶瑕佷竴娈垫椂闂?..
>
> 完成升级。
>
> if change console encoding to UTF-8 (chcp 65001),output message is :
>
> Repository lock acquired.
> Please wait; upgrading the repository may take some time...
>
> Upgrade completed.
>
>
Re: svnadmin upgrade output message i18n issue
Posted by Branko Čibej <br...@wandisco.com>.
On 23.05.2013 16:06, Philip Martin wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>>> Dongsheng Song <do...@gmail.com> writes:
>>>
>>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>>>> <ph...@wandisco.com> wrote:
>>>>> Philip Martin <ph...@wandisco.com> writes:
>>>>>
>>>>>> So it appears the UTF8 to native conversion is missing from
>>>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>>>> for 1.8.
>>>>>
>>>> As GETTEXT(3) man pages said, If and only if
>>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>>>> your commit is OK.
>>>>
>>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>>>> svn_cmdline_cstring_from_utf8.
>>> Are you saying there is a problem with my change? If there is a problem
>>> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>>>
>> I thinks so. In the subversion/libsvn_subr/nls.c file:
>>
>> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>>
>> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
>> defined. In this case, you can assume GETTEXT(3) returned string is
>> UTF-8 encoded.
> I still don't understand if you are claiming my change has a problem or
> if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
>
> I recall a related thread from last year:
>
> http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
> http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
>
> I think we assume that the translations are UTF-8.
We do not "assume" the translations are UTF-8, we require them to be.
http://subversion.apache.org/docs/community-guide/l10n.html#po-mo-requirements
-- Brane
--
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com
Re: svnadmin upgrade output message i18n issue
Posted by QXO <qx...@gmail.com>.
http://sourceforge.net/projects/win32svn/
2013/9/6 Bert Huijben <be...@qqmail.nl>
> Which build of the Windows binaries do you use?****
>
> ** **
>
> The Subversion project doesn��t produce binaries, so can��t really support
> build issues.****
>
> ** **
>
> Personally I do build the SlikSvn binaries (http://sliksvn.com/en/download),
> so if you find issues in that I might be able to help you.****
>
> ** **
>
> Bert****
>
> ** **
>
> *From:* QXO [mailto:qxodream@gmail.com]
> *Sent:* vrijdag 6 september 2013 08:49
> *To:* Philip Martin
> *Cc:* Dongsheng Song; users@subversion.apache.org;
> dev@subversion.apache.org
> *Subject:* Re: svnadmin upgrade output message i18n issue****
>
> ** **
>
> The bug found in svn 1.8.3(r1516576) again :(****
>
> ** **
>
> ** **
>
> E:\svnmirror>D:\tools\svn-win32-1.8.3\bin\svnadmin.exe upgrade ZmccPrj****
>
> 已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..****
>
> ** **
>
> ���������****
>
> ** **
>
> ** **
>
> E:\svnmirror>D:\tools\svn-win32-1.8.0\bin\svnadmin.exe upgrade ZmccPrj****
>
> ��ȡ�ð汾��������****
>
> ���Ժ������汾�������Ҫһ��ʱ��...****
>
> ** **
>
> ���������****
>
> ** **
>
> ** **
>
> ** **
>
> 2013/6/19 QXO <qx...@gmail.com>****
>
> The bug fixed in svn 1.8.0,Thanks:)****
>
> ** **
>
> 2013/5/24 Philip Martin <ph...@wandisco.com>****
>
> Dongsheng Song <do...@gmail.com> writes:
>
> >> We do call bind_textdomain_codeset if it is available so we should be
> >> getting UTF8 translations.
> >>
> >
> > For non-autotools system, e.g. Windows, user may not define
> > HAVE_BIND_TEXTDOMAIN_CODESET.****
>
> If you build the software with the wrong settings it may not work
> properly. The solution is to build it with the correct settings.
> Perhaps you can improve the Windows build.****
>
>
> > Or we should call bind_textdomain_codeset as possible, and warn the
> > user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
> >
> > #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> > bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> > #else
> > fprintf(sdterr, "bind_textdomain_codeset not available, or not
> > configured. Non-UTF8 locales maybe see garbled output.\n");
> > #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */****
>
> That error would be annoying if it was a false alarm. Perhaps we could
> verify the correct behaviour: call gettext and check whether the
> returned string is valid UTF8? However, we are not getting reports of
> problems so it's probably not worth the effort. The bug that started
> this thread is about the exact opposite: gettext was returning UTF8 and
> the output code was failing to convert to locale encoding.****
>
>
> --
> Certified & Supported Apache Subversion Downloads:
> http://www.wandisco.com/subversion/download****
>
> ** **
>
> ** **
>
RE: svnadmin upgrade output message i18n issue
Posted by Bert Huijben <be...@qqmail.nl>.
Which build of the Windows binaries do you use?
The Subversion project doesn’t produce binaries, so can’t really support build issues.
Personally I do build the SlikSvn binaries (http://sliksvn.com/en/download), so if you find issues in that I might be able to help you.
Bert
From: QXO [mailto:qxodream@gmail.com]
Sent: vrijdag 6 september 2013 08:49
To: Philip Martin
Cc: Dongsheng Song; users@subversion.apache.org; dev@subversion.apache.org
Subject: Re: svnadmin upgrade output message i18n issue
The bug found in svn 1.8.3(r1516576) again :(
E:\svnmirror>D:\tools\svn-win32-1.8.3\bin\svnadmin.exe upgrade ZmccPrj
宸插彇寰楃増鏈簱閿佸畾銆?璇风◢鍊欙紱鍗囩骇鐗堟湰搴撳彲鑳介渶瑕佷竴娈垫椂闂?..
完成升级。
E:\svnmirror>D:\tools\svn-win32-1.8.0\bin\svnadmin.exe upgrade ZmccPrj
已取得版本库锁定。
请稍候；升级版本库可能需要一段时间...
完成升级。
2013/6/19 QXO < <ma...@gmail.com> qxodream@gmail.com>
The bug fixed in svn 1.8.0,Thanks:)
2013/5/24 Philip Martin < <ma...@wandisco.com> philip.martin@wandisco.com>
Dongsheng Song < <ma...@gmail.com> dongsheng.song@gmail.com> writes:
>> We do call bind_textdomain_codeset if it is available so we should be
>> getting UTF8 translations.
>>
>
> For non-autotools system, e.g. Windows, user may not define
> HAVE_BIND_TEXTDOMAIN_CODESET.
If you build the software with the wrong settings it may not work
properly. The solution is to build it with the correct settings.
Perhaps you can improve the Windows build.
> Or we should call bind_textdomain_codeset as possible, and warn the
> user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
>
> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> #else
> fprintf(sdterr, "bind_textdomain_codeset not available, or not
> configured. Non-UTF8 locales maybe see garbled output.\n");
> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
That error would be annoying if it was a false alarm. Perhaps we could
verify the correct behaviour: call gettext and check whether the
returned string is valid UTF8? However, we are not getting reports of
problems so it's probably not worth the effort. The bug that started
this thread is about the exact opposite: gettext was returning UTF8 and
the output code was failing to convert to locale encoding.
--
Certified & Supported Apache Subversion Downloads:
<http://www.wandisco.com/subversion/download> http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
QXO <qx...@gmail.com> writes:
> The bug found in svn 1.8.3(r1516576) again :(
>
> E:\svnmirror>D:\tools\svn-win32-1.8.3\bin\svnadmin.exe upgrade ZmccPrj
> 已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
> ���������
>
> E:\svnmirror>D:\tools\svn-win32-1.8.0\bin\svnadmin.exe upgrade ZmccPrj
> ��ȡ�ð汾��������
> ���Ժ������汾�������Ҫһ��ʱ��...
> ���������
>
> 2013/6/19 QXO <qx...@gmail.com>
>
>> The bug fixed in svn 1.8.0,Thanks:)
As I recall you are using a non-UTF8 encoding. There was a bug in 1.8.0
that caused double UTF8-to-native conversion for svnadmin, that was
fixed in 1.8.3. Since you say 1.8.0 worked your build must be missing
one of the conversion. How did you build Subversion? Was it built with
ENABLE_NLS and HAVE_BIND_TEXTDOMAIN_CODESET?
Is anyone else using Windows in a non-UTF8 setup?
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
QXO <qx...@gmail.com> writes:
> The bug found in svn 1.8.3(r1516576) again :(
>
> E:\svnmirror>D:\tools\svn-win32-1.8.3\bin\svnadmin.exe upgrade ZmccPrj
> 已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
> ���������
>
> E:\svnmirror>D:\tools\svn-win32-1.8.0\bin\svnadmin.exe upgrade ZmccPrj
> ��ȡ�ð汾��������
> ���Ժ������汾�������Ҫһ��ʱ��...
> ���������
>
> 2013/6/19 QXO <qx...@gmail.com>
>
>> The bug fixed in svn 1.8.0,Thanks:)
As I recall you are using a non-UTF8 encoding. There was a bug in 1.8.0
that caused double UTF8-to-native conversion for svnadmin, that was
fixed in 1.8.3. Since you say 1.8.0 worked your build must be missing
one of the conversion. How did you build Subversion? Was it built with
ENABLE_NLS and HAVE_BIND_TEXTDOMAIN_CODESET?
Is anyone else using Windows in a non-UTF8 setup?
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: svnadmin upgrade output message i18n issue
Posted by QXO <qx...@gmail.com>.
The bug found in svn 1.8.3(r1516576) again :(
E:\svnmirror>D:\tools\svn-win32-1.8.3\bin\svnadmin.exe upgrade ZmccPrj
已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
���������
E:\svnmirror>D:\tools\svn-win32-1.8.0\bin\svnadmin.exe upgrade ZmccPrj
��ȡ�ð汾��������
���Ժ������汾�������Ҫһ��ʱ��...
���������
2013/6/19 QXO <qx...@gmail.com>
> The bug fixed in svn 1.8.0,Thanks:)
>
>
> 2013/5/24 Philip Martin <ph...@wandisco.com>
>
>> Dongsheng Song <do...@gmail.com> writes:
>>
>> >> We do call bind_textdomain_codeset if it is available so we should be
>> >> getting UTF8 translations.
>> >>
>> >
>> > For non-autotools system, e.g. Windows, user may not define
>> > HAVE_BIND_TEXTDOMAIN_CODESET.
>>
>> If you build the software with the wrong settings it may not work
>> properly. The solution is to build it with the correct settings.
>> Perhaps you can improve the Windows build.
>>
>> > Or we should call bind_textdomain_codeset as possible, and warn the
>> > user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
>> >
>> > #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> > bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> > #else
>> > fprintf(sdterr, "bind_textdomain_codeset not available, or not
>> > configured. Non-UTF8 locales maybe see garbled output.\n");
>> > #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>>
>> That error would be annoying if it was a false alarm. Perhaps we could
>> verify the correct behaviour: call gettext and check whether the
>> returned string is valid UTF8? However, we are not getting reports of
>> problems so it's probably not worth the effort. The bug that started
>> this thread is about the exact opposite: gettext was returning UTF8 and
>> the output code was failing to convert to locale encoding.
>>
>> --
>> Certified & Supported Apache Subversion Downloads:
>> http://www.wandisco.com/subversion/download
>>
>
>
Re: svnadmin upgrade output message i18n issue
Posted by QXO <qx...@gmail.com>.
The bug found in svn 1.8.3(r1516576) again :(
E:\svnmirror>D:\tools\svn-win32-1.8.3\bin\svnadmin.exe upgrade ZmccPrj
已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
���������
E:\svnmirror>D:\tools\svn-win32-1.8.0\bin\svnadmin.exe upgrade ZmccPrj
��ȡ�ð汾��������
���Ժ������汾�������Ҫһ��ʱ��...
���������
2013/6/19 QXO <qx...@gmail.com>
> The bug fixed in svn 1.8.0,Thanks:)
>
>
> 2013/5/24 Philip Martin <ph...@wandisco.com>
>
>> Dongsheng Song <do...@gmail.com> writes:
>>
>> >> We do call bind_textdomain_codeset if it is available so we should be
>> >> getting UTF8 translations.
>> >>
>> >
>> > For non-autotools system, e.g. Windows, user may not define
>> > HAVE_BIND_TEXTDOMAIN_CODESET.
>>
>> If you build the software with the wrong settings it may not work
>> properly. The solution is to build it with the correct settings.
>> Perhaps you can improve the Windows build.
>>
>> > Or we should call bind_textdomain_codeset as possible, and warn the
>> > user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
>> >
>> > #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> > bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> > #else
>> > fprintf(sdterr, "bind_textdomain_codeset not available, or not
>> > configured. Non-UTF8 locales maybe see garbled output.\n");
>> > #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>>
>> That error would be annoying if it was a false alarm. Perhaps we could
>> verify the correct behaviour: call gettext and check whether the
>> returned string is valid UTF8? However, we are not getting reports of
>> problems so it's probably not worth the effort. The bug that started
>> this thread is about the exact opposite: gettext was returning UTF8 and
>> the output code was failing to convert to locale encoding.
>>
>> --
>> Certified & Supported Apache Subversion Downloads:
>> http://www.wandisco.com/subversion/download
>>
>
>
Re: svnadmin upgrade output message i18n issue
Posted by QXO <qx...@gmail.com>.
The bug fixed in svn 1.8.0,Thanks:)
2013/5/24 Philip Martin <ph...@wandisco.com>
> Dongsheng Song <do...@gmail.com> writes:
>
> >> We do call bind_textdomain_codeset if it is available so we should be
> >> getting UTF8 translations.
> >>
> >
> > For non-autotools system, e.g. Windows, user may not define
> > HAVE_BIND_TEXTDOMAIN_CODESET.
>
> If you build the software with the wrong settings it may not work
> properly. The solution is to build it with the correct settings.
> Perhaps you can improve the Windows build.
>
> > Or we should call bind_textdomain_codeset as possible, and warn the
> > user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
> >
> > #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> > bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> > #else
> > fprintf(sdterr, "bind_textdomain_codeset not available, or not
> > configured. Non-UTF8 locales maybe see garbled output.\n");
> > #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>
> That error would be annoying if it was a false alarm. Perhaps we could
> verify the correct behaviour: call gettext and check whether the
> returned string is valid UTF8? However, we are not getting reports of
> problems so it's probably not worth the effort. The bug that started
> this thread is about the exact opposite: gettext was returning UTF8 and
> the output code was failing to convert to locale encoding.
>
> --
> Certified & Supported Apache Subversion Downloads:
> http://www.wandisco.com/subversion/download
>
Re: svnadmin upgrade output message i18n issue
Posted by QXO <qx...@gmail.com>.
The bug fixed in svn 1.8.0,Thanks:)
2013/5/24 Philip Martin <ph...@wandisco.com>
> Dongsheng Song <do...@gmail.com> writes:
>
> >> We do call bind_textdomain_codeset if it is available so we should be
> >> getting UTF8 translations.
> >>
> >
> > For non-autotools system, e.g. Windows, user may not define
> > HAVE_BIND_TEXTDOMAIN_CODESET.
>
> If you build the software with the wrong settings it may not work
> properly. The solution is to build it with the correct settings.
> Perhaps you can improve the Windows build.
>
> > Or we should call bind_textdomain_codeset as possible, and warn the
> > user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
> >
> > #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> > bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> > #else
> > fprintf(sdterr, "bind_textdomain_codeset not available, or not
> > configured. Non-UTF8 locales maybe see garbled output.\n");
> > #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>
> That error would be annoying if it was a false alarm. Perhaps we could
> verify the correct behaviour: call gettext and check whether the
> returned string is valid UTF8? However, we are not getting reports of
> problems so it's probably not worth the effort. The bug that started
> this thread is about the exact opposite: gettext was returning UTF8 and
> the output code was failing to convert to locale encoding.
>
> --
> Certified & Supported Apache Subversion Downloads:
> http://www.wandisco.com/subversion/download
>
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
>> We do call bind_textdomain_codeset if it is available so we should be
>> getting UTF8 translations.
>>
>
> For non-autotools system, e.g. Windows, user may not define
> HAVE_BIND_TEXTDOMAIN_CODESET.
If you build the software with the wrong settings it may not work
properly. The solution is to build it with the correct settings.
Perhaps you can improve the Windows build.
> Or we should call bind_textdomain_codeset as possible, and warn the
> user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
>
> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> #else
> fprintf(sdterr, "bind_textdomain_codeset not available, or not
> configured. Non-UTF8 locales maybe see garbled output.\n");
> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
That error would be annoying if it was a false alarm. Perhaps we could
verify the correct behaviour: call gettext and check whether the
returned string is valid UTF8? However, we are not getting reports of
problems so it's probably not worth the effort. The bug that started
this thread is about the exact opposite: gettext was returning UTF8 and
the output code was failing to convert to locale encoding.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
>> We do call bind_textdomain_codeset if it is available so we should be
>> getting UTF8 translations.
>>
>
> For non-autotools system, e.g. Windows, user may not define
> HAVE_BIND_TEXTDOMAIN_CODESET.
If you build the software with the wrong settings it may not work
properly. The solution is to build it with the correct settings.
Perhaps you can improve the Windows build.
> Or we should call bind_textdomain_codeset as possible, and warn the
> user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
>
> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> #else
> fprintf(sdterr, "bind_textdomain_codeset not available, or not
> configured. Non-UTF8 locales maybe see garbled output.\n");
> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
That error would be annoying if it was a false alarm. Perhaps we could
verify the correct behaviour: call gettext and check whether the
returned string is valid UTF8? However, we are not getting reports of
problems so it's probably not worth the effort. The bug that started
this thread is about the exact opposite: gettext was returning UTF8 and
the output code was failing to convert to locale encoding.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 11:29 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> Even ALL the translations are UTF-8, GETTEXT(3) still return the
>> string encoded by the ***current locale's codeset***.
>>
>> Here is sniped from the GETTEXT(3) man pages:
>>
>> In both cases, the functions also use the LC_CTYPE locale facet in
>> order to convert the translated message from the translator's
>> codeset to the ***current locale's codeset***, unless overridden by a
>> prior call to the bind_textdomain_codeset function.
>
> We do call bind_textdomain_codeset if it is available so we should be
> getting UTF8 translations.
>
For non-autotools system, e.g. Windows, user may not define
HAVE_BIND_TEXTDOMAIN_CODESET.
>> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
>> coded, it it encoded to the ***current locale's codeset***.
>>
>> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
>> messages, write it ***AS IS***, since GETTEXT(3) already do the
>> correct conversion for us.
>
> It's not that simple. We would have to change almost every error:
>
> svn_error_createf(SVN_ERR_BAD_RELATIVE_PATH, NULL, \
> _("Path '%s' must be an immediate child of " \
> "the directory '%s'"), path, relative_to_dir)
>
> and convert variable like 'path' and 'relative_to_dir' from UTF8 to
> native before combining with the native translation.
>
> What would be the gain for all that work? The only problem at present
> is a system that doesn't have bind_textdomain_codeset but where gettext
> returns the current locale encoding having converted it from the UTF8 in
> the file. Are there any such systems? What about the opposite problem:
> systems that don't have bind_textdomain_codeset and where gettext
> returns UTF8 because that is the encoding in the file. Are there any
> systems like that?
>
Or we should call bind_textdomain_codeset as possible, and warn the
user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
#ifdef HAVE_BIND_TEXTDOMAIN_CODESET
bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
#else
fprintf(sdterr, "bind_textdomain_codeset not available, or not
configured. Non-UTF8 locales maybe see garbled output.\n");
#endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 11:29 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> Even ALL the translations are UTF-8, GETTEXT(3) still return the
>> string encoded by the ***current locale's codeset***.
>>
>> Here is sniped from the GETTEXT(3) man pages:
>>
>> In both cases, the functions also use the LC_CTYPE locale facet in
>> order to convert the translated message from the translator's
>> codeset to the ***current locale's codeset***, unless overridden by a
>> prior call to the bind_textdomain_codeset function.
>
> We do call bind_textdomain_codeset if it is available so we should be
> getting UTF8 translations.
>
For non-autotools system, e.g. Windows, user may not define
HAVE_BIND_TEXTDOMAIN_CODESET.
>> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
>> coded, it it encoded to the ***current locale's codeset***.
>>
>> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
>> messages, write it ***AS IS***, since GETTEXT(3) already do the
>> correct conversion for us.
>
> It's not that simple. We would have to change almost every error:
>
> svn_error_createf(SVN_ERR_BAD_RELATIVE_PATH, NULL, \
> _("Path '%s' must be an immediate child of " \
> "the directory '%s'"), path, relative_to_dir)
>
> and convert variable like 'path' and 'relative_to_dir' from UTF8 to
> native before combining with the native translation.
>
> What would be the gain for all that work? The only problem at present
> is a system that doesn't have bind_textdomain_codeset but where gettext
> returns the current locale encoding having converted it from the UTF8 in
> the file. Are there any such systems? What about the opposite problem:
> systems that don't have bind_textdomain_codeset and where gettext
> returns UTF8 because that is the encoding in the file. Are there any
> systems like that?
>
Or we should call bind_textdomain_codeset as possible, and warn the
user if HAVE_BIND_TEXTDOMAIN_CODESET not defined:
#ifdef HAVE_BIND_TEXTDOMAIN_CODESET
bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
#else
fprintf(sdterr, "bind_textdomain_codeset not available, or not
configured. Non-UTF8 locales maybe see garbled output.\n");
#endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
> Even ALL the translations are UTF-8, GETTEXT(3) still return the
> string encoded by the ***current locale's codeset***.
>
> Here is sniped from the GETTEXT(3) man pages:
>
> In both cases, the functions also use the LC_CTYPE locale facet in
> order to convert the translated message from the translator's
> codeset to the ***current locale's codeset***, unless overridden by a
> prior call to the bind_textdomain_codeset function.
We do call bind_textdomain_codeset if it is available so we should be
getting UTF8 translations.
> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
> coded, it it encoded to the ***current locale's codeset***.
>
> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> messages, write it ***AS IS***, since GETTEXT(3) already do the
> correct conversion for us.
It's not that simple. We would have to change almost every error:
svn_error_createf(SVN_ERR_BAD_RELATIVE_PATH, NULL, \
_("Path '%s' must be an immediate child of " \
"the directory '%s'"), path, relative_to_dir)
and convert variable like 'path' and 'relative_to_dir' from UTF8 to
native before combining with the native translation.
What would be the gain for all that work? The only problem at present
is a system that doesn't have bind_textdomain_codeset but where gettext
returns the current locale encoding having converted it from the UTF8 in
the file. Are there any such systems? What about the opposite problem:
systems that don't have bind_textdomain_codeset and where gettext
returns UTF8 because that is the encoding in the file. Are there any
systems like that?
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Fri, May 24, 2013 at 12:52 AM, Erik Huelsmann <eh...@gmail.com> wrote:
> One application has multiple active code page settings on Windows. Or course
> if your example was the only option, we would not be having this discussion.
>
Very interesting. In my mind, application only can have 1 active
locale in 1 thread. When gettext() got called, the current locale is
uniquely. Could you give me a sample ?
Regards,
Dongsheng
> Bye,
>
> Erik.
>
> sent from my phone
>
> On May 23, 2013 6:44 PM, "Dongsheng Song" <do...@gmail.com> wrote:
>>
>> On Thu, May 23, 2013 at 11:38 PM, Erik Huelsmann <eh...@gmail.com> wrote:
>> > That was not my point nor the point we discussed back then. As long as
>> > gettext tries to convert its translations to *any* encoding, it's flawed
>> > by
>> > design, because some systems have multiple active output encodings (e.g.
>> > Windows).
>> >
>>
>> This does not matter. If I open 2 console window, one is CP437, the
>> other is CP936. Then svn in CP437 windows generate English (ASCII)
>> output, CP936 windows generate Chinese (GBK/GB18030) output.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Fri, May 24, 2013 at 12:52 AM, Erik Huelsmann <eh...@gmail.com> wrote:
> One application has multiple active code page settings on Windows. Or course
> if your example was the only option, we would not be having this discussion.
>
Very interesting. In my mind, application only can have 1 active
locale in 1 thread. When gettext() got called, the current locale is
uniquely. Could you give me a sample ?
Regards,
Dongsheng
> Bye,
>
> Erik.
>
> sent from my phone
>
> On May 23, 2013 6:44 PM, "Dongsheng Song" <do...@gmail.com> wrote:
>>
>> On Thu, May 23, 2013 at 11:38 PM, Erik Huelsmann <eh...@gmail.com> wrote:
>> > That was not my point nor the point we discussed back then. As long as
>> > gettext tries to convert its translations to *any* encoding, it's flawed
>> > by
>> > design, because some systems have multiple active output encodings (e.g.
>> > Windows).
>> >
>>
>> This does not matter. If I open 2 console window, one is CP437, the
>> other is CP936. Then svn in CP437 windows generate English (ASCII)
>> output, CP936 windows generate Chinese (GBK/GB18030) output.
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
One application has multiple active code page settings on Windows. Or
course if your example was the only option, we would not be having this
discussion.
Bye,
Erik.
sent from my phone
On May 23, 2013 6:44 PM, "Dongsheng Song" <do...@gmail.com> wrote:
> On Thu, May 23, 2013 at 11:38 PM, Erik Huelsmann <eh...@gmail.com> wrote:
> > That was not my point nor the point we discussed back then. As long as
> > gettext tries to convert its translations to *any* encoding, it's flawed
> by
> > design, because some systems have multiple active output encodings (e.g.
> > Windows).
> >
>
> This does not matter. If I open 2 console window, one is CP437, the
> other is CP936. Then svn in CP437 windows generate English (ASCII)
> output, CP936 windows generate Chinese (GBK/GB18030) output.
>
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
One application has multiple active code page settings on Windows. Or
course if your example was the only option, we would not be having this
discussion.
Bye,
Erik.
sent from my phone
On May 23, 2013 6:44 PM, "Dongsheng Song" <do...@gmail.com> wrote:
> On Thu, May 23, 2013 at 11:38 PM, Erik Huelsmann <eh...@gmail.com> wrote:
> > That was not my point nor the point we discussed back then. As long as
> > gettext tries to convert its translations to *any* encoding, it's flawed
> by
> > design, because some systems have multiple active output encodings (e.g.
> > Windows).
> >
>
> This does not matter. If I open 2 console window, one is CP437, the
> other is CP936. Then svn in CP437 windows generate English (ASCII)
> output, CP936 windows generate Chinese (GBK/GB18030) output.
>
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 11:38 PM, Erik Huelsmann <eh...@gmail.com> wrote:
> That was not my point nor the point we discussed back then. As long as
> gettext tries to convert its translations to *any* encoding, it's flawed by
> design, because some systems have multiple active output encodings (e.g.
> Windows).
>
This does not matter. If I open 2 console window, one is CP437, the
other is CP936. Then svn in CP437 windows generate English (ASCII)
output, CP936 windows generate Chinese (GBK/GB18030) output.
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
Found at least one of the related discussions:
http://svn.haxx.se/dev/archive-2004-05/0078.shtml
bye,
Erik.
On May 23, 2013 5:38 PM, "Erik Huelsmann" <eh...@gmail.com> wrote:
>
> > >
> > >> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> > >> messages, write it ***AS IS***, since GETTEXT(3) already do the
> > >> correct conversion for us.
> > >
> > > Well, even though gettext may want us to believe otherwise, this
> doesn't
> > > work for cross platform applications: e.g. in windows the locale for
> output
> > > on the console may be different from the locale for other uses. Back
> when we
> > > went with gettext (2004?), we've hashed this through pretty
> thoroughly. I
> > > hope that discussion is still available in the archives.
> > >
> >
> > As I said in the first email of this thread, gettext 0.18.2 and 0.14.1
> > give me the different behavior, it seems that gettext 0.14.1 do not do
> > the correct thing. But do we still need support this OLD and BUGGY
> > version ?
>
> That was not my point nor the point we discussed back then. As long as
> gettext tries to convert its translations to *any* encoding, it's flawed by
> design, because some systems have multiple active output encodings (e.g.
> Windows).
>
> Unless this design has changed between 0.14 and 0.18, gettext() is still
> as broken as it was. Translating or not translating doesn't matter: it'll
> just be broken on other systems. Too bad the rest of it is actually pretty
> good.
>
> Bye,
>
> Erik.
>
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
Found at least one of the related discussions:
http://svn.haxx.se/dev/archive-2004-05/0078.shtml
bye,
Erik.
On May 23, 2013 5:38 PM, "Erik Huelsmann" <eh...@gmail.com> wrote:
>
> > >
> > >> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> > >> messages, write it ***AS IS***, since GETTEXT(3) already do the
> > >> correct conversion for us.
> > >
> > > Well, even though gettext may want us to believe otherwise, this
> doesn't
> > > work for cross platform applications: e.g. in windows the locale for
> output
> > > on the console may be different from the locale for other uses. Back
> when we
> > > went with gettext (2004?), we've hashed this through pretty
> thoroughly. I
> > > hope that discussion is still available in the archives.
> > >
> >
> > As I said in the first email of this thread, gettext 0.18.2 and 0.14.1
> > give me the different behavior, it seems that gettext 0.14.1 do not do
> > the correct thing. But do we still need support this OLD and BUGGY
> > version ?
>
> That was not my point nor the point we discussed back then. As long as
> gettext tries to convert its translations to *any* encoding, it's flawed by
> design, because some systems have multiple active output encodings (e.g.
> Windows).
>
> Unless this design has changed between 0.14 and 0.18, gettext() is still
> as broken as it was. Translating or not translating doesn't matter: it'll
> just be broken on other systems. Too bad the rest of it is actually pretty
> good.
>
> Bye,
>
> Erik.
>
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 11:38 PM, Erik Huelsmann <eh...@gmail.com> wrote:
> That was not my point nor the point we discussed back then. As long as
> gettext tries to convert its translations to *any* encoding, it's flawed by
> design, because some systems have multiple active output encodings (e.g.
> Windows).
>
This does not matter. If I open 2 console window, one is CP437, the
other is CP936. Then svn in CP437 windows generate English (ASCII)
output, CP936 windows generate Chinese (GBK/GB18030) output.
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
> >
> >> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> >> messages, write it ***AS IS***, since GETTEXT(3) already do the
> >> correct conversion for us.
> >
> > Well, even though gettext may want us to believe otherwise, this doesn't
> > work for cross platform applications: e.g. in windows the locale for
output
> > on the console may be different from the locale for other uses. Back
when we
> > went with gettext (2004?), we've hashed this through pretty thoroughly.
I
> > hope that discussion is still available in the archives.
> >
>
> As I said in the first email of this thread, gettext 0.18.2 and 0.14.1
> give me the different behavior, it seems that gettext 0.14.1 do not do
> the correct thing. But do we still need support this OLD and BUGGY
> version ?
That was not my point nor the point we discussed back then. As long as
gettext tries to convert its translations to *any* encoding, it's flawed by
design, because some systems have multiple active output encodings (e.g.
Windows).
Unless this design has changed between 0.14 and 0.18, gettext() is still as
broken as it was. Translating or not translating doesn't matter: it'll just
be broken on other systems. Too bad the rest of it is actually pretty good.
Bye,
Erik.
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
> >
> >> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> >> messages, write it ***AS IS***, since GETTEXT(3) already do the
> >> correct conversion for us.
> >
> > Well, even though gettext may want us to believe otherwise, this doesn't
> > work for cross platform applications: e.g. in windows the locale for
output
> > on the console may be different from the locale for other uses. Back
when we
> > went with gettext (2004?), we've hashed this through pretty thoroughly.
I
> > hope that discussion is still available in the archives.
> >
>
> As I said in the first email of this thread, gettext 0.18.2 and 0.14.1
> give me the different behavior, it seems that gettext 0.14.1 do not do
> the correct thing. But do we still need support this OLD and BUGGY
> version ?
That was not my point nor the point we discussed back then. As long as
gettext tries to convert its translations to *any* encoding, it's flawed by
design, because some systems have multiple active output encodings (e.g.
Windows).
Unless this design has changed between 0.14 and 0.18, gettext() is still as
broken as it was. Translating or not translating doesn't matter: it'll just
be broken on other systems. Too bad the rest of it is actually pretty good.
Bye,
Erik.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 11:02 PM, Erik Huelsmann <eh...@gmail.com> wrote:
> sent from my phone
>
>
> On May 23, 2013 4:43 PM, "Dongsheng Song" <do...@gmail.com> wrote:
>>
>> On Thu, May 23, 2013 at 10:06 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>> > Dongsheng Song <do...@gmail.com> writes:
>> >
>> >> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
>> >> <ph...@wandisco.com> wrote:
>> >>> Dongsheng Song <do...@gmail.com> writes:
>> >>>
>> >>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>> >>>> <ph...@wandisco.com> wrote:
>> >>>>> Philip Martin <ph...@wandisco.com> writes:
>> >>>>>
>> >>>>>> So it appears the UTF8 to native conversion is missing from
>> >>>>>> repos_notify_handler. I think repos_notify_handler should be using
>> >>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>> >>>>>
>> >>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed
>> >>>>> it
>> >>>>> for 1.8.
>> >>>>>
>> >>>>
>> >>>> As GETTEXT(3) man pages said, If and only if
>> >>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>> >>>> your commit is OK.
>> >>>>
>> >>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>> >>>> svn_cmdline_cstring_from_utf8.
>> >>>
>> >>> Are you saying there is a problem with my change? If there is a
>> >>> problem
>> >>> doesn't already apply to all other uses of
>> >>> svn_cmdline_cstring_from_utf8?
>> >>>
>> >>
>> >> I thinks so. In the subversion/libsvn_subr/nls.c file:
>> >>
>> >> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> >> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> >> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>> >>
>> >> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
>> >> defined. In this case, you can assume GETTEXT(3) returned string is
>> >> UTF-8 encoded.
>> >
>> > I still don't understand if you are claiming my change has a problem or
>> > if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
>> >
>> > I recall a related thread from last year:
>> >
>> > http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
>> >
>> > http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
>> >
>> > I think we assume that the translations are UTF-8.
>> >
>> > Is there some code change you think we should make?
>> >
>>
>> Even ALL the translations are UTF-8, GETTEXT(3) still return the
>> string encoded by the ***current locale's codeset***.
>>
>> Here is sniped from the GETTEXT(3) man pages:
>>
>> In both cases, the functions also use the LC_CTYPE locale facet in
>> order to convert the translated message from the translator's
>> codeset to the ***current locale's codeset***, unless overridden by a
>> prior call to the bind_textdomain_codeset function.
>>
>> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
>> coded, it it encoded to the ***current locale's codeset***.
>
> But we call the codeset function to make sure we do not generate output in
> the current locale encoding.
>
>> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
>> messages, write it ***AS IS***, since GETTEXT(3) already do the
>> correct conversion for us.
>
> Well, even though gettext may want us to believe otherwise, this doesn't
> work for cross platform applications: e.g. in windows the locale for output
> on the console may be different from the locale for other uses. Back when we
> went with gettext (2004?), we've hashed this through pretty thoroughly. I
> hope that discussion is still available in the archives.
>
As I said in the first email of this thread, gettext 0.18.2 and 0.14.1
give me the different behavior, it seems that gettext 0.14.1 do not do
the correct thing. But do we still need support this OLD and BUGGY
version ?
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 11:02 PM, Erik Huelsmann <eh...@gmail.com> wrote:
> sent from my phone
>
>
> On May 23, 2013 4:43 PM, "Dongsheng Song" <do...@gmail.com> wrote:
>>
>> On Thu, May 23, 2013 at 10:06 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>> > Dongsheng Song <do...@gmail.com> writes:
>> >
>> >> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
>> >> <ph...@wandisco.com> wrote:
>> >>> Dongsheng Song <do...@gmail.com> writes:
>> >>>
>> >>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>> >>>> <ph...@wandisco.com> wrote:
>> >>>>> Philip Martin <ph...@wandisco.com> writes:
>> >>>>>
>> >>>>>> So it appears the UTF8 to native conversion is missing from
>> >>>>>> repos_notify_handler. I think repos_notify_handler should be using
>> >>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>> >>>>>
>> >>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed
>> >>>>> it
>> >>>>> for 1.8.
>> >>>>>
>> >>>>
>> >>>> As GETTEXT(3) man pages said, If and only if
>> >>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>> >>>> your commit is OK.
>> >>>>
>> >>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>> >>>> svn_cmdline_cstring_from_utf8.
>> >>>
>> >>> Are you saying there is a problem with my change? If there is a
>> >>> problem
>> >>> doesn't already apply to all other uses of
>> >>> svn_cmdline_cstring_from_utf8?
>> >>>
>> >>
>> >> I thinks so. In the subversion/libsvn_subr/nls.c file:
>> >>
>> >> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> >> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> >> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>> >>
>> >> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
>> >> defined. In this case, you can assume GETTEXT(3) returned string is
>> >> UTF-8 encoded.
>> >
>> > I still don't understand if you are claiming my change has a problem or
>> > if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
>> >
>> > I recall a related thread from last year:
>> >
>> > http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
>> >
>> > http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
>> >
>> > I think we assume that the translations are UTF-8.
>> >
>> > Is there some code change you think we should make?
>> >
>>
>> Even ALL the translations are UTF-8, GETTEXT(3) still return the
>> string encoded by the ***current locale's codeset***.
>>
>> Here is sniped from the GETTEXT(3) man pages:
>>
>> In both cases, the functions also use the LC_CTYPE locale facet in
>> order to convert the translated message from the translator's
>> codeset to the ***current locale's codeset***, unless overridden by a
>> prior call to the bind_textdomain_codeset function.
>>
>> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
>> coded, it it encoded to the ***current locale's codeset***.
>
> But we call the codeset function to make sure we do not generate output in
> the current locale encoding.
>
>> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
>> messages, write it ***AS IS***, since GETTEXT(3) already do the
>> correct conversion for us.
>
> Well, even though gettext may want us to believe otherwise, this doesn't
> work for cross platform applications: e.g. in windows the locale for output
> on the console may be different from the locale for other uses. Back when we
> went with gettext (2004?), we've hashed this through pretty thoroughly. I
> hope that discussion is still available in the archives.
>
As I said in the first email of this thread, gettext 0.18.2 and 0.14.1
give me the different behavior, it seems that gettext 0.14.1 do not do
the correct thing. But do we still need support this OLD and BUGGY
version ?
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
sent from my phone
On May 23, 2013 4:43 PM, "Dongsheng Song" <do...@gmail.com> wrote:
>
> On Thu, May 23, 2013 at 10:06 PM, Philip Martin
> <ph...@wandisco.com> wrote:
> > Dongsheng Song <do...@gmail.com> writes:
> >
> >> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
> >> <ph...@wandisco.com> wrote:
> >>> Dongsheng Song <do...@gmail.com> writes:
> >>>
> >>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
> >>>> <ph...@wandisco.com> wrote:
> >>>>> Philip Martin <ph...@wandisco.com> writes:
> >>>>>
> >>>>>> So it appears the UTF8 to native conversion is missing from
> >>>>>> repos_notify_handler. I think repos_notify_handler should be using
> >>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
> >>>>>
> >>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed
it
> >>>>> for 1.8.
> >>>>>
> >>>>
> >>>> As GETTEXT(3) man pages said, If and only if
> >>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
> >>>> your commit is OK.
> >>>>
> >>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
> >>>> svn_cmdline_cstring_from_utf8.
> >>>
> >>> Are you saying there is a problem with my change? If there is a
problem
> >>> doesn't already apply to all other uses of
svn_cmdline_cstring_from_utf8?
> >>>
> >>
> >> I thinks so. In the subversion/libsvn_subr/nls.c file:
> >>
> >> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> >> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> >> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
> >>
> >> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
> >> defined. In this case, you can assume GETTEXT(3) returned string is
> >> UTF-8 encoded.
> >
> > I still don't understand if you are claiming my change has a problem or
> > if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
> >
> > I recall a related thread from last year:
> >
> > http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
> >
http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
> >
> > I think we assume that the translations are UTF-8.
> >
> > Is there some code change you think we should make?
> >
>
> Even ALL the translations are UTF-8, GETTEXT(3) still return the
> string encoded by the ***current locale's codeset***.
>
> Here is sniped from the GETTEXT(3) man pages:
>
> In both cases, the functions also use the LC_CTYPE locale facet in
> order to convert the translated message from the translator's
> codeset to the ***current locale's codeset***, unless overridden by a
> prior call to the bind_textdomain_codeset function.
>
> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
> coded, it it encoded to the ***current locale's codeset***.
But we call the codeset function to make sure we do not generate output in
the current locale encoding.
> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> messages, write it ***AS IS***, since GETTEXT(3) already do the
> correct conversion for us.
Well, even though gettext may want us to believe otherwise, this doesn't
work for cross platform applications: e.g. in windows the locale for output
on the console may be different from the locale for other uses. Back when
we went with gettext (2004?), we've hashed this through pretty thoroughly.
I hope that discussion is still available in the archives.
Bye,
Erik.
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
> Even ALL the translations are UTF-8, GETTEXT(3) still return the
> string encoded by the ***current locale's codeset***.
>
> Here is sniped from the GETTEXT(3) man pages:
>
> In both cases, the functions also use the LC_CTYPE locale facet in
> order to convert the translated message from the translator's
> codeset to the ***current locale's codeset***, unless overridden by a
> prior call to the bind_textdomain_codeset function.
We do call bind_textdomain_codeset if it is available so we should be
getting UTF8 translations.
> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
> coded, it it encoded to the ***current locale's codeset***.
>
> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> messages, write it ***AS IS***, since GETTEXT(3) already do the
> correct conversion for us.
It's not that simple. We would have to change almost every error:
svn_error_createf(SVN_ERR_BAD_RELATIVE_PATH, NULL, \
_("Path '%s' must be an immediate child of " \
"the directory '%s'"), path, relative_to_dir)
and convert variable like 'path' and 'relative_to_dir' from UTF8 to
native before combining with the native translation.
What would be the gain for all that work? The only problem at present
is a system that doesn't have bind_textdomain_codeset but where gettext
returns the current locale encoding having converted it from the UTF8 in
the file. Are there any such systems? What about the opposite problem:
systems that don't have bind_textdomain_codeset and where gettext
returns UTF8 because that is the encoding in the file. Are there any
systems like that?
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Erik Huelsmann <eh...@gmail.com>.
sent from my phone
On May 23, 2013 4:43 PM, "Dongsheng Song" <do...@gmail.com> wrote:
>
> On Thu, May 23, 2013 at 10:06 PM, Philip Martin
> <ph...@wandisco.com> wrote:
> > Dongsheng Song <do...@gmail.com> writes:
> >
> >> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
> >> <ph...@wandisco.com> wrote:
> >>> Dongsheng Song <do...@gmail.com> writes:
> >>>
> >>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
> >>>> <ph...@wandisco.com> wrote:
> >>>>> Philip Martin <ph...@wandisco.com> writes:
> >>>>>
> >>>>>> So it appears the UTF8 to native conversion is missing from
> >>>>>> repos_notify_handler. I think repos_notify_handler should be using
> >>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
> >>>>>
> >>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed
it
> >>>>> for 1.8.
> >>>>>
> >>>>
> >>>> As GETTEXT(3) man pages said, If and only if
> >>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
> >>>> your commit is OK.
> >>>>
> >>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
> >>>> svn_cmdline_cstring_from_utf8.
> >>>
> >>> Are you saying there is a problem with my change? If there is a
problem
> >>> doesn't already apply to all other uses of
svn_cmdline_cstring_from_utf8?
> >>>
> >>
> >> I thinks so. In the subversion/libsvn_subr/nls.c file:
> >>
> >> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> >> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> >> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
> >>
> >> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
> >> defined. In this case, you can assume GETTEXT(3) returned string is
> >> UTF-8 encoded.
> >
> > I still don't understand if you are claiming my change has a problem or
> > if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
> >
> > I recall a related thread from last year:
> >
> > http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
> >
http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
> >
> > I think we assume that the translations are UTF-8.
> >
> > Is there some code change you think we should make?
> >
>
> Even ALL the translations are UTF-8, GETTEXT(3) still return the
> string encoded by the ***current locale's codeset***.
>
> Here is sniped from the GETTEXT(3) man pages:
>
> In both cases, the functions also use the LC_CTYPE locale facet in
> order to convert the translated message from the translator's
> codeset to the ***current locale's codeset***, unless overridden by a
> prior call to the bind_textdomain_codeset function.
>
> So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
> coded, it it encoded to the ***current locale's codeset***.
But we call the codeset function to make sure we do not generate output in
the current locale encoding.
> I think the best solution is: DO NOTconvert the GETTEXT(3) returned
> messages, write it ***AS IS***, since GETTEXT(3) already do the
> correct conversion for us.
Well, even though gettext may want us to believe otherwise, this doesn't
work for cross platform applications: e.g. in windows the locale for output
on the console may be different from the locale for other uses. Back when
we went with gettext (2004?), we've hashed this through pretty thoroughly.
I hope that discussion is still available in the archives.
Bye,
Erik.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 10:06 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>>> Dongsheng Song <do...@gmail.com> writes:
>>>
>>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>>>> <ph...@wandisco.com> wrote:
>>>>> Philip Martin <ph...@wandisco.com> writes:
>>>>>
>>>>>> So it appears the UTF8 to native conversion is missing from
>>>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>>>
>>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>>>> for 1.8.
>>>>>
>>>>
>>>> As GETTEXT(3) man pages said, If and only if
>>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>>>> your commit is OK.
>>>>
>>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>>>> svn_cmdline_cstring_from_utf8.
>>>
>>> Are you saying there is a problem with my change? If there is a problem
>>> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>>>
>>
>> I thinks so. In the subversion/libsvn_subr/nls.c file:
>>
>> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>>
>> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
>> defined. In this case, you can assume GETTEXT(3) returned string is
>> UTF-8 encoded.
>
> I still don't understand if you are claiming my change has a problem or
> if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
>
> I recall a related thread from last year:
>
> http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
> http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
>
> I think we assume that the translations are UTF-8.
>
> Is there some code change you think we should make?
>
Even ALL the translations are UTF-8, GETTEXT(3) still return the
string encoded by the ***current locale's codeset***.
Here is sniped from the GETTEXT(3) man pages:
In both cases, the functions also use the LC_CTYPE locale facet in
order to convert the translated message from the translator's
codeset to the ***current locale's codeset***, unless overridden by a
prior call to the bind_textdomain_codeset function.
So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
coded, it it encoded to the ***current locale's codeset***.
I think the best solution is: DO NOTconvert the GETTEXT(3) returned
messages, write it ***AS IS***, since GETTEXT(3) already do the
correct conversion for us.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 10:06 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>>> Dongsheng Song <do...@gmail.com> writes:
>>>
>>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>>>> <ph...@wandisco.com> wrote:
>>>>> Philip Martin <ph...@wandisco.com> writes:
>>>>>
>>>>>> So it appears the UTF8 to native conversion is missing from
>>>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>>>
>>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>>>> for 1.8.
>>>>>
>>>>
>>>> As GETTEXT(3) man pages said, If and only if
>>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>>>> your commit is OK.
>>>>
>>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>>>> svn_cmdline_cstring_from_utf8.
>>>
>>> Are you saying there is a problem with my change? If there is a problem
>>> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>>>
>>
>> I thinks so. In the subversion/libsvn_subr/nls.c file:
>>
>> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
>> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
>> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>>
>> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
>> defined. In this case, you can assume GETTEXT(3) returned string is
>> UTF-8 encoded.
>
> I still don't understand if you are claiming my change has a problem or
> if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
>
> I recall a related thread from last year:
>
> http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
> http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
>
> I think we assume that the translations are UTF-8.
>
> Is there some code change you think we should make?
>
Even ALL the translations are UTF-8, GETTEXT(3) still return the
string encoded by the ***current locale's codeset***.
Here is sniped from the GETTEXT(3) man pages:
In both cases, the functions also use the LC_CTYPE locale facet in
order to convert the translated message from the translator's
codeset to the ***current locale's codeset***, unless overridden by a
prior call to the bind_textdomain_codeset function.
So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
coded, it it encoded to the ***current locale's codeset***.
I think the best solution is: DO NOTconvert the GETTEXT(3) returned
messages, write it ***AS IS***, since GETTEXT(3) already do the
correct conversion for us.
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
> <ph...@wandisco.com> wrote:
>> Dongsheng Song <do...@gmail.com> writes:
>>
>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>>> <ph...@wandisco.com> wrote:
>>>> Philip Martin <ph...@wandisco.com> writes:
>>>>
>>>>> So it appears the UTF8 to native conversion is missing from
>>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>>
>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>>> for 1.8.
>>>>
>>>
>>> As GETTEXT(3) man pages said, If and only if
>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>>> your commit is OK.
>>>
>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>>> svn_cmdline_cstring_from_utf8.
>>
>> Are you saying there is a problem with my change? If there is a problem
>> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>>
>
> I thinks so. In the subversion/libsvn_subr/nls.c file:
>
> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>
> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
> defined. In this case, you can assume GETTEXT(3) returned string is
> UTF-8 encoded.
I still don't understand if you are claiming my change has a problem or
if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
I recall a related thread from last year:
http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
I think we assume that the translations are UTF-8.
Is there some code change you think we should make?
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
> On Thu, May 23, 2013 at 9:28 PM, Philip Martin
> <ph...@wandisco.com> wrote:
>> Dongsheng Song <do...@gmail.com> writes:
>>
>>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>>> <ph...@wandisco.com> wrote:
>>>> Philip Martin <ph...@wandisco.com> writes:
>>>>
>>>>> So it appears the UTF8 to native conversion is missing from
>>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>>
>>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>>> for 1.8.
>>>>
>>>
>>> As GETTEXT(3) man pages said, If and only if
>>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>>> your commit is OK.
>>>
>>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>>> svn_cmdline_cstring_from_utf8.
>>
>> Are you saying there is a problem with my change? If there is a problem
>> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>>
>
> I thinks so. In the subversion/libsvn_subr/nls.c file:
>
> #ifdef HAVE_BIND_TEXTDOMAIN_CODESET
> bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
> #endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
>
> bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
> defined. In this case, you can assume GETTEXT(3) returned string is
> UTF-8 encoded.
I still don't understand if you are claiming my change has a problem or
if there is a problem in all uses of svn_cmdline_cstring_from_utf8.
I recall a related thread from last year:
http://svn.haxx.se/dev/archive-2012-08/index.shtml#34
http://mail-archives.apache.org/mod_mbox/subversion-dev/201208.mbox/%3Cop.wilcelggnngjn5@tortoise%3E
I think we assume that the translations are UTF-8.
Is there some code change you think we should make?
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 9:28 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>>> Philip Martin <ph...@wandisco.com> writes:
>>>
>>>> So it appears the UTF8 to native conversion is missing from
>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>
>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>> for 1.8.
>>>
>>
>> As GETTEXT(3) man pages said, If and only if
>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>> your commit is OK.
>>
>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>> svn_cmdline_cstring_from_utf8.
>
> Are you saying there is a problem with my change? If there is a problem
> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>
I thinks so. In the subversion/libsvn_subr/nls.c file:
#ifdef HAVE_BIND_TEXTDOMAIN_CODESET
bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
#endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
defined. In this case, you can assume GETTEXT(3) returned string is
UTF-8 encoded.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 9:28 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Dongsheng Song <do...@gmail.com> writes:
>
>> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
>> <ph...@wandisco.com> wrote:
>>> Philip Martin <ph...@wandisco.com> writes:
>>>
>>>> So it appears the UTF8 to native conversion is missing from
>>>> repos_notify_handler. I think repos_notify_handler should be using
>>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>>
>>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>>> for 1.8.
>>>
>>
>> As GETTEXT(3) man pages said, If and only if
>> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
>> your commit is OK.
>>
>> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
>> svn_cmdline_cstring_from_utf8.
>
> Are you saying there is a problem with my change? If there is a problem
> doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
>
I thinks so. In the subversion/libsvn_subr/nls.c file:
#ifdef HAVE_BIND_TEXTDOMAIN_CODESET
bind_textdomain_codeset(PACKAGE_NAME, "UTF-8");
#endif /* HAVE_BIND_TEXTDOMAIN_CODESET */
bind_textdomain_codeset only called when HAVE_BIND_TEXTDOMAIN_CODESET
defined. In this case, you can assume GETTEXT(3) returned string is
UTF-8 encoded.
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
> <ph...@wandisco.com> wrote:
>> Philip Martin <ph...@wandisco.com> writes:
>>
>>> So it appears the UTF8 to native conversion is missing from
>>> repos_notify_handler. I think repos_notify_handler should be using
>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>
>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>> for 1.8.
>>
>
> As GETTEXT(3) man pages said, If and only if
> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
> your commit is OK.
>
> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
> svn_cmdline_cstring_from_utf8.
Are you saying there is a problem with my change? If there is a problem
doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Dongsheng Song <do...@gmail.com> writes:
> On Thu, May 23, 2013 at 9:11 PM, Philip Martin
> <ph...@wandisco.com> wrote:
>> Philip Martin <ph...@wandisco.com> writes:
>>
>>> So it appears the UTF8 to native conversion is missing from
>>> repos_notify_handler. I think repos_notify_handler should be using
>>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>>
>> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
>> for 1.8.
>>
>
> As GETTEXT(3) man pages said, If and only if
> defined(HAVE_BIND_TEXTDOMAIN_CODESET),
> your commit is OK.
>
> So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
> svn_cmdline_cstring_from_utf8.
Are you saying there is a problem with my change? If there is a problem
doesn't already apply to all other uses of svn_cmdline_cstring_from_utf8?
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 9:11 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Philip Martin <ph...@wandisco.com> writes:
>
>> So it appears the UTF8 to native conversion is missing from
>> repos_notify_handler. I think repos_notify_handler should be using
>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>
> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
> for 1.8.
>
As GETTEXT(3) man pages said, If and only if
defined(HAVE_BIND_TEXTDOMAIN_CODESET),
your commit is OK.
So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
svn_cmdline_cstring_from_utf8.
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 9:11 PM, Philip Martin
<ph...@wandisco.com> wrote:
> Philip Martin <ph...@wandisco.com> writes:
>
>> So it appears the UTF8 to native conversion is missing from
>> repos_notify_handler. I think repos_notify_handler should be using
>> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>
> I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
> for 1.8.
>
As GETTEXT(3) man pages said, If and only if
defined(HAVE_BIND_TEXTDOMAIN_CODESET),
your commit is OK.
So you should check HAVE_BIND_TEXTDOMAIN_CODESET when you use
svn_cmdline_cstring_from_utf8.
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> So it appears the UTF8 to native conversion is missing from
> repos_notify_handler. I think repos_notify_handler should be using
> svn_stream_printf_from_utf8 rather than svn_stream_printf.
I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
for 1.8.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> So it appears the UTF8 to native conversion is missing from
> repos_notify_handler. I think repos_notify_handler should be using
> svn_stream_printf_from_utf8 rather than svn_stream_printf.
I've fixed trunk to use svn_cmdline_cstring_from_utf8 and proposed it
for 1.8.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 4:17 PM, Philip Martin
<ph...@wandisco.com> wrote:
> [bringing in dev@s.a.o]
>
> QXO <qx...@gmail.com> writes:
>
>> os: windows
>> encoding:GBK ( chcp 936 )
>>
>> The svnadmin upgrade command output message first line encoding
>> issue(UTF-8 show in GBK),But the second line is right encoding!
>>
>> 宸插彇寰楃増鏈簱閿佸畾銆?璇风◢鍊欙紱鍗囩骇鐗堟湰搴撳彲鑳介渶瑕佷竴娈垫椂闂?..
>>
>> 完成升级。
>>
>> if change console encoding to UTF-8 (chcp 65001),output message is :
>>
>> Repository lock acquired.
>> Please wait; upgrading the repository may take some time...
>>
>> Upgrade completed.
>
> Those two lines are produced by different code paths. The first line
> is produced by repos_notify_handler:
>
> svn_error_clear(svn_stream_printf(feedback_stream, scratch_pool,
> _("Repository lock acquired.\n"
> "Please wait; upgrading the"
> " repository may take some time...\n")));
>
> The second line is produced by:
>
> SVN_ERR(svn_cmdline_printf(pool, _("\nUpgrade completed.\n")));
>
> and svn_cmdline_printf uses svn_cmdline_cstring_from_utf8 to do a UTF8
> to native conversion.
>
> So it appears the UTF8 to native conversion is missing from
> repos_notify_handler. I think repos_notify_handler should be using
> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>
NO. From GETTEXT(3) man pages:
In both cases, the functions also use the LC_CTYPE locale facet in
order to convert the translated message from the translator's
codeset to the ***current locale's codeset***, unless overridden by a
prior call to the bind_textdomain_codeset function.
So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
coded, it it encoded to the ***current locale's codeset***.
--
Regards,
Dongsheng
Re: svnadmin upgrade output message i18n issue
Posted by Dongsheng Song <do...@gmail.com>.
On Thu, May 23, 2013 at 4:17 PM, Philip Martin
<ph...@wandisco.com> wrote:
> [bringing in dev@s.a.o]
>
> QXO <qx...@gmail.com> writes:
>
>> os: windows
>> encoding:GBK ( chcp 936 )
>>
>> The svnadmin upgrade command output message first line encoding
>> issue(UTF-8 show in GBK),But the second line is right encoding!
>>
>> 宸插彇寰楃増鏈簱閿佸畾銆?璇风◢鍊欙紱鍗囩骇鐗堟湰搴撳彲鑳介渶瑕佷竴娈垫椂闂?..
>>
>> 完成升级。
>>
>> if change console encoding to UTF-8 (chcp 65001),output message is :
>>
>> Repository lock acquired.
>> Please wait; upgrading the repository may take some time...
>>
>> Upgrade completed.
>
> Those two lines are produced by different code paths. The first line
> is produced by repos_notify_handler:
>
> svn_error_clear(svn_stream_printf(feedback_stream, scratch_pool,
> _("Repository lock acquired.\n"
> "Please wait; upgrading the"
> " repository may take some time...\n")));
>
> The second line is produced by:
>
> SVN_ERR(svn_cmdline_printf(pool, _("\nUpgrade completed.\n")));
>
> and svn_cmdline_printf uses svn_cmdline_cstring_from_utf8 to do a UTF8
> to native conversion.
>
> So it appears the UTF8 to native conversion is missing from
> repos_notify_handler. I think repos_notify_handler should be using
> svn_stream_printf_from_utf8 rather than svn_stream_printf.
>
NO. From GETTEXT(3) man pages:
In both cases, the functions also use the LC_CTYPE locale facet in
order to convert the translated message from the translator's
codeset to the ***current locale's codeset***, unless overridden by a
prior call to the bind_textdomain_codeset function.
So svn_cmdline_printf SHOULD NOT assume the input string is UTF-8
coded, it it encoded to the ***current locale's codeset***.
--
Regards,
Dongsheng
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
[bringing in dev@s.a.o]
QXO <qx...@gmail.com> writes:
> os: windows
> encoding:GBK ( chcp 936 )
>
> The svnadmin upgrade command output message first line encoding
> issue(UTF-8 show in GBK),But the second line is right encoding!
>
> 已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
>
> ���������
>
> if change console encoding to UTF-8 (chcp 65001),output message is :
>
> Repository lock acquired.
> Please wait; upgrading the repository may take some time...
>
> Upgrade completed.
Those two lines are produced by different code paths. The first line
is produced by repos_notify_handler:
svn_error_clear(svn_stream_printf(feedback_stream, scratch_pool,
_("Repository lock acquired.\n"
"Please wait; upgrading the"
" repository may take some time...\n")));
The second line is produced by:
SVN_ERR(svn_cmdline_printf(pool, _("\nUpgrade completed.\n")));
and svn_cmdline_printf uses svn_cmdline_cstring_from_utf8 to do a UTF8
to native conversion.
So it appears the UTF8 to native conversion is missing from
repos_notify_handler. I think repos_notify_handler should be using
svn_stream_printf_from_utf8 rather than svn_stream_printf.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: svnadmin upgrade output message i18n issue
Posted by Philip Martin <ph...@wandisco.com>.
[bringing in dev@s.a.o]
QXO <qx...@gmail.com> writes:
> os: windows
> encoding:GBK ( chcp 936 )
>
> The svnadmin upgrade command output message first line encoding
> issue(UTF-8 show in GBK),But the second line is right encoding!
>
> 已取得版本库锁定�?请稍候；升级版本库可能需要一段时�?..
>
> ���������
>
> if change console encoding to UTF-8 (chcp 65001),output message is :
>
> Repository lock acquired.
> Please wait; upgrading the repository may take some time...
>
> Upgrade completed.
Those two lines are produced by different code paths. The first line
is produced by repos_notify_handler:
svn_error_clear(svn_stream_printf(feedback_stream, scratch_pool,
_("Repository lock acquired.\n"
"Please wait; upgrading the"
" repository may take some time...\n")));
The second line is produced by:
SVN_ERR(svn_cmdline_printf(pool, _("\nUpgrade completed.\n")));
and svn_cmdline_printf uses svn_cmdline_cstring_from_utf8 to do a UTF8
to native conversion.
So it appears the UTF8 to native conversion is missing from
repos_notify_handler. I think repos_notify_handler should be using
svn_stream_printf_from_utf8 rather than svn_stream_printf.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download