You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Danny Trebbien <dt...@gmail.com> on 2011/02/12 23:16:25 UTC
Best way to issue a warning in a C test
I want to add a test to
`subversion/tests/libsvn_subr/subst_translate-test.c` that will set
the locale with setlocale(LC_ALL, "eo.ISO-8859-3"), issuing a warning
if it is unable to because the eo.ISO-8859-3 locale is not installed.
What is a good way to do this? I don't think that the test should fail
simply because the system does not have the eo.ISO-8859-3 locale, but
I would like a warning to be conspicuously printed in the test
results.
Re: Best way to issue a warning in a C test
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Feb 13, 2011 at 11:15:13AM -0800, Danny Trebbien wrote:
> Per Philip's suggestion, I have switched the test to trying a list of
> locales. While I have not tested it on Windows, I am using examples
> from MSDN verbatim. It should work.
>
> I also try the Windows-specific locale strings first because I know
> that a Linux system successfully ignores them. Using the following,
> for example, the locale ends up being en_US.ISO-8859-1 on my Debian
> Linux "Squeeze" system supporting locales C, en_US, en_US.iso88591,
> en_US.utf8, eo, eo.iso88593, eo.utf8, and POSIX:
>
> if ((! setlocale(LC_ALL, "English.1252")) &&
> (! setlocale(LC_ALL, "German.1252")) &&
> (! setlocale(LC_ALL, "French.1252")) &&
> (! setlocale(LC_ALL, "en_US.ISO-8859-1")) &&
> (! setlocale(LC_ALL, "en_GB.ISO-8859-1")) &&
> (! setlocale(LC_ALL, "de_DE.ISO-8859-1")))
> return svn_error_createf(SVN_ERR_TEST_SKIPPED, NULL, "None of the
> locales English.1252, German.1252, French.1252, en_US.ISO-8859-1,
> en_GB.ISO-8859-1, and de_DE.ISO-8859-1 are installed.");
Fair enough.
But you will also need to try "en_US.ISO8859-1" (only one dash!)
for this work on OpenBSD, and possibly other BSDs.
Re: Best way to issue a warning in a C test
Posted by Danny Trebbien <dt...@gmail.com>.
>> I also try the Windows-specific locale strings first because I know
>> that a Linux system successfully ignores them.
>
> Does setlocale even have a real effect on Windows? Last time I looked,
> you couldn't change the per-process locale.
Yes. I have developed some websites in PHP on Windows where I used
PHP's `setlocale` function which basically just calls the standard C
function of the same name. There are differences in how the locales
are specified (e.g. Windows uses "English_USA" instead of Linux'
"en_US"), but it does work.
Re: Best way to issue a warning in a C test
Posted by Branko Čibej <br...@e-reka.si>.
On 13.02.2011 20:15, Danny Trebbien wrote:
> I also try the Windows-specific locale strings first because I know
> that a Linux system successfully ignores them.
Does setlocale even have a real effect on Windows? Last time I looked,
you couldn't change the per-process locale.
-- Brane
Re: Best way to issue a warning in a C test
Posted by Danny Trebbien <dt...@gmail.com>.
>> >> Yes, I basically just want a non-UTF-8 locale. Which one do you suggest?
>
> Danny, what is this new test for (sorry if this has been explained in
> some other thread -- I briefly checked and couldn't find any).
Hi Stefan,
This is for a new test. See:
http://thread.gmane.org/gmane.comp.version-control.subversion.devel/125782
> And are you aware of utf8_tests.py and the related issue?
> http://subversion.tigris.org/issues/show_bug.cgi?id=2079
> Will your new test have similar problems?
I was not aware of this issue, but I don't think that my new test has
the same problem. For one thing, the definition of the variable
`i18n_filename` in `subversion/tests/cmdline/utf8_tests.py` and the
way that it is used assumes that the system-default character encoding
is ISO-8859-1. If it isn't, then the `i18n_filename` either is
invalid in the current encoding or represents a different sequence of
characters entirely.
In my test, I was careful to pick a byte sequence (0xc6) that
represents the same character (Æ) when viewed as a CP-1252 or
ISO-8859-1 encoded string. Also, the test sets the locale to use
either of those two encodings, skipping the test if it is unable to do
so.
> Instead of printing a warning, I would suggest to skip the test if
> the locale cannot be configured.
That's what it does.
>> > How about trying a list? en_US.8859-1 is widely available even if not
>> > actually installed. I don't know which non-utf8 locales are widely
>> > installed, personally I have en_GB.8859-1. We have several German devs
>> > so maybe de_DE.8859-1? Perhaps jp_JP.EUC-JP?
>>
>> I see utf8_tests.py is using ".1251" and "en_US.ISO8859-1".
>
> It isn't easy to find a locale that works on all platforms.
>
> Locales are poorly standardized -- a lot things, including names of locales,
> are left to the implementation. AFAIK the only thing that is standardized is
> the C99 API for using locales (setlocale() etc.). There are various
> implementations and they differ greatly in the amount of character
> encodings and languages provided.
>
> You'll mostly have to consider all operating systems that svn developers
> use, which is Windows (not sure if that even has a locale concept),
> various Linux distributions (some of which allow users to configure the
> set of available locales, so that they might only have e.g. en_US.UTF-8),
> Mac OS X, and all of the BSD systems.
>
> I don't think any of us is using Solaris or other proprietary UNIX systems.
> Though you should strive to allow your test to run there, too.
> Some users run the tests to verify their own binary builds, and those
> people might well be using proprietary UNIX.
>
> So if you must do this, please use some latin-1 locale.
> That's likely a good common denominator.
> If the language doesn't matter (it probably doesn't if all you want to
> test is encoding conversion), I'd recommend using en_US.ISO88591.
> And if that isn't there, just skip the test.
> Note that this may be called en_US.ISO8859-1 on some systems (e.g. BSD).
> Maybe there are other naming variations, not sure.
>
> The esperanto locale is definitely a bad idea. I would expect that
> none of the BSD systems have it, for instance.
Per Philip's suggestion, I have switched the test to trying a list of
locales. While I have not tested it on Windows, I am using examples
from MSDN verbatim. It should work.
I also try the Windows-specific locale strings first because I know
that a Linux system successfully ignores them. Using the following,
for example, the locale ends up being en_US.ISO-8859-1 on my Debian
Linux "Squeeze" system supporting locales C, en_US, en_US.iso88591,
en_US.utf8, eo, eo.iso88593, eo.utf8, and POSIX:
if ((! setlocale(LC_ALL, "English.1252")) &&
(! setlocale(LC_ALL, "German.1252")) &&
(! setlocale(LC_ALL, "French.1252")) &&
(! setlocale(LC_ALL, "en_US.ISO-8859-1")) &&
(! setlocale(LC_ALL, "en_GB.ISO-8859-1")) &&
(! setlocale(LC_ALL, "de_DE.ISO-8859-1")))
return svn_error_createf(SVN_ERR_TEST_SKIPPED, NULL, "None of the
locales English.1252, German.1252, French.1252, en_US.ISO-8859-1,
en_GB.ISO-8859-1, and de_DE.ISO-8859-1 are installed.");
Re: Best way to issue a warning in a C test
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Feb 13, 2011 at 12:09:32AM +0000, Philip Martin wrote:
> Philip Martin <ph...@wandisco.com> writes:
>
> > Danny Trebbien <dt...@gmail.com> writes:
> >
> >>> I had to look that up, it's Esperanto. Why do you want that one? What
> >>> are you testing? It doesn't appear to be available on my machine. Do
> >>> you just want a non-utf8 locale?
> >>
> >> Yes, I basically just want a non-UTF-8 locale. Which one do you suggest?
Danny, what is this new test for (sorry if this has been explained in
some other thread -- I briefly checked and couldn't find any).
And are you aware of utf8_tests.py and the related issue?
http://subversion.tigris.org/issues/show_bug.cgi?id=2079
Will your new test have similar problems?
Instead of printing a warning, I would suggest to skip the test if
the locale cannot be configured.
> > How about trying a list? en_US.8859-1 is widely available even if not
> > actually installed. I don't know which non-utf8 locales are widely
> > installed, personally I have en_GB.8859-1. We have several German devs
> > so maybe de_DE.8859-1? Perhaps jp_JP.EUC-JP?
>
> I see utf8_tests.py is using ".1251" and "en_US.ISO8859-1".
It isn't easy to find a locale that works on all platforms.
Locales are poorly standardized -- a lot things, including names of locales,
are left to the implementation. AFAIK the only thing that is standardized is
the C99 API for using locales (setlocale() etc.). There are various
implementations and they differ greatly in the amount of character
encodings and languages provided.
You'll mostly have to consider all operating systems that svn developers
use, which is Windows (not sure if that even has a locale concept),
various Linux distributions (some of which allow users to configure the
set of available locales, so that they might only have e.g. en_US.UTF-8),
Mac OS X, and all of the BSD systems.
I don't think any of us is using Solaris or other proprietary UNIX systems.
Though you should strive to allow your test to run there, too.
Some users run the tests to verify their own binary builds, and those
people might well be using proprietary UNIX.
So if you must do this, please use some latin-1 locale.
That's likely a good common denominator.
If the language doesn't matter (it probably doesn't if all you want to
test is encoding conversion), I'd recommend using en_US.ISO88591.
And if that isn't there, just skip the test.
Note that this may be called en_US.ISO8859-1 on some systems (e.g. BSD).
Maybe there are other naming variations, not sure.
The esperanto locale is definitely a bad idea. I would expect that
none of the BSD systems have it, for instance.
Re: Best way to issue a warning in a C test
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> Danny Trebbien <dt...@gmail.com> writes:
>
>>> I had to look that up, it's Esperanto. Why do you want that one? What
>>> are you testing? It doesn't appear to be available on my machine. Do
>>> you just want a non-utf8 locale?
>>
>> Yes, I basically just want a non-UTF-8 locale. Which one do you suggest?
>
> How about trying a list? en_US.8859-1 is widely available even if not
> actually installed. I don't know which non-utf8 locales are widely
> installed, personally I have en_GB.8859-1. We have several German devs
> so maybe de_DE.8859-1? Perhaps jp_JP.EUC-JP?
I see utf8_tests.py is using ".1251" and "en_US.ISO8859-1".
--
Philip
Re: Best way to issue a warning in a C test
Posted by Philip Martin <ph...@wandisco.com>.
Danny Trebbien <dt...@gmail.com> writes:
>> While I couldn't get SVN_TEST_SKIP2 to work because the availability
>> of a locale cannot be tested at preprocessing time
>
> Actually, at compile time.
Ah, yes. Perhaps you could extend svn_test_mode_t and add a new
svn_test_skip_runtime mode: add support in svn_test_main.c for calling
the predicate function, and a new SVN_TEST_ macro.
--
Philip
Re: Best way to issue a warning in a C test
Posted by Danny Trebbien <dt...@gmail.com>.
> While I couldn't get SVN_TEST_SKIP2 to work because the availability
> of a locale cannot be tested at preprocessing time
Actually, at compile time.
Re: Best way to issue a warning in a C test
Posted by Danny Trebbien <dt...@gmail.com>.
> How about trying a list? en_US.8859-1 is widely available even if not
> actually installed. I don't know which non-utf8 locales are widely
> installed, personally I have en_GB.8859-1. We have several German devs
> so maybe de_DE.8859-1? Perhaps jp_JP.EUC-JP?
While I couldn't get SVN_TEST_SKIP2 to work because the availability
of a locale cannot be tested at preprocessing time, I found out that
returning an SVN_ERR_TEST_SKIPPED error has the same effect.
I like your idea of trying a list. Here is what I ended up with:
if ((! setlocale(LC_ALL, "English.1252")) &&
(! setlocale(LC_ALL, "German.1252")) &&
(! setlocale(LC_ALL, "French.1252")) &&
(! setlocale(LC_ALL, "en_US.ISO-8859-1")) &&
(! setlocale(LC_ALL, "en_GB.ISO-8859-1")) &&
(! setlocale(LC_ALL, "de_DE.ISO-8859-1")))
return svn_error_createf(SVN_ERR_TEST_SKIPPED, NULL, "None of
the locales English.1252, German.1252, French.1252, en_US.ISO-8859-1,
en_GB.ISO-8859-1, and de_DE.ISO-8859-1 are installed.");
Re: Best way to issue a warning in a C test
Posted by Philip Martin <ph...@wandisco.com>.
Danny Trebbien <dt...@gmail.com> writes:
>> I had to look that up, it's Esperanto. Why do you want that one? What
>> are you testing? It doesn't appear to be available on my machine. Do
>> you just want a non-utf8 locale?
>
> Yes, I basically just want a non-UTF-8 locale. Which one do you suggest?
How about trying a list? en_US.8859-1 is widely available even if not
actually installed. I don't know which non-utf8 locales are widely
installed, personally I have en_GB.8859-1. We have several German devs
so maybe de_DE.8859-1? Perhaps jp_JP.EUC-JP?
--
Philip
Re: Best way to issue a warning in a C test
Posted by Danny Trebbien <dt...@gmail.com>.
> I had to look that up, it's Esperanto. Why do you want that one? What
> are you testing? It doesn't appear to be available on my machine. Do
> you just want a non-utf8 locale?
Yes, I basically just want a non-UTF-8 locale. Which one do you suggest?
> You could write a function to determine if the locale is available and
> then use that in SVN_TEST_SKIP2 to skip the test if the locale is not
> available. The the test should be appear in the summary as SKIP.
Okay.
Re: Best way to issue a warning in a C test
Posted by Philip Martin <ph...@wandisco.com>.
Danny Trebbien <dt...@gmail.com> writes:
> I want to add a test to
> `subversion/tests/libsvn_subr/subst_translate-test.c` that will set
> the locale with setlocale(LC_ALL, "eo.ISO-8859-3"), issuing a warning
> if it is unable to because the eo.ISO-8859-3 locale is not installed.
>
> What is a good way to do this? I don't think that the test should fail
> simply because the system does not have the eo.ISO-8859-3 locale, but
> I would like a warning to be conspicuously printed in the test
> results.
I had to look that up, it's Esperanto. Why do you want that one? What
are you testing? It doesn't appear to be available on my machine. Do
you just want a non-utf8 locale?
You could write a function to determine if the locale is available and
then use that in SVN_TEST_SKIP2 to skip the test if the locale is not
available. The the test should be appear in the summary as SKIP.
--
Philip