You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Richard Liang <ri...@gmail.com> on 2006/07/19 12:12:19 UTC
Re: [testing] locale dependent tests
Tim Ellison wrote:
> Richard Liang wrote:
>
>> Although the spec does not require the round-trip of applyPattern and
>> toPattern, we still want to get one *certain* pattern through toPattern.
>> Now the problem is the returned pattern is locale-dependent. I'm
>> uncertain about the reason to remove the assertion:
>> 1) Because the behavior is not required by spec, or
>> 2) Because the behavior is locale-dependent
>>
>
> It would seem that rather than forcing the locale to be en_US to make
> the test assertions valid, you could work with the default locale and
> guard any assertions that are locale-specific. It would seem a shame to
> loose the diversity of testing on multiple locale machines if the tests
> always force everyone to look like an American (horror! ;-) )
>
> I would expect the fix therefore to be something like, if locale is
> en_US go ahead and assert more round-tripping code, otherwise can't test
> it as the spec allows variances.
>
>
If a test case passes in all locales, could we think that the behavior
tested by the test case is locale-independent? We still have to think
about how to test locale-dependent behavior. For example,
java.util.Locale.getDisplayName() and java.lang.String.toUpperCase(). To
verify both the code logic and locale data, shall we have some tests
like ABC_en_US_Test, ABC_en_UK_Test, and ABC_ru_RU_Test?
Best regards,
Richard
> Regards,
> Tim
>
>
--
Richard Liang
China Software Development Lab, IBM
Re: [testing] locale dependent tests
Posted by Alexey Petrenko <al...@gmail.com>.
Sounds reasonable.
SY, Alexey
2006/7/20, Paulex Yang <pa...@gmail.com>:
> Andrew Zhang wrote:
> > On 7/19/06, Richard Liang <ri...@gmail.com> wrote:
> >>
> >>
> >>
> >> Tim Ellison wrote:
> >> > Richard Liang wrote:
> >> >
> >> >> Although the spec does not require the round-trip of applyPattern and
> >> >> toPattern, we still want to get one *certain* pattern through
> >> toPattern.
> >> >> Now the problem is the returned pattern is locale-dependent. I'm
> >> >> uncertain about the reason to remove the assertion:
> >> >> 1) Because the behavior is not required by spec, or
> >> >> 2) Because the behavior is locale-dependent
> >> >>
> >> >
> >> > It would seem that rather than forcing the locale to be en_US to make
> >> > the test assertions valid, you could work with the default locale and
> >> > guard any assertions that are locale-specific. It would seem a
> >> shame to
> >> > loose the diversity of testing on multiple locale machines if the
> >> tests
> >> > always force everyone to look like an American (horror! ;-) )
> >> >
> >> > I would expect the fix therefore to be something like, if locale is
> >> > en_US go ahead and assert more round-tripping code, otherwise can't
> >> test
> >> > it as the spec allows variances.
> >> >
> >> >
> >> If a test case passes in all locales, could we think that the behavior
> >> tested by the test case is locale-independent? We still have to think
> >> about how to test locale-dependent behavior. For example,
> >> java.util.Locale.getDisplayName() and java.lang.String.toUpperCase(). To
> >> verify both the code logic and locale data, shall we have some tests
> >> like ABC_en_US_Test, ABC_en_UK_Test, and ABC_ru_RU_Test?
>
>
> I still confuse what we want to test, the logic or the data? I think
> most (if not all) i18n related methods actually have same single
> executable with multiple resource bundles, i.e., the single executable
> should be locale-independent, the different return value is due to the
> resource data difference. I think at least for now, we should pay our
> attention to logic of single executable, and leave the data verification
> to the i18n libraries' author, say, ICU, they have much more knowledge
> and authority (at least than me) on this area.
>
> If we can get agree on the above, so the i18n related test cases
> organization are easier to judge: the logic is locale-independent, so
> ideally the tests should be locale-independent, but we have some
> exceptional cases, say, the en_UK in MessageFormat case, so we cannot
> make our tests rely on the default locale, then we just specify one
> locale(en_US) to the tests, and supplement some exceptional case when we
> find some. i.e., I don't think we need ABC_en_US_Test, or so.
>
> Comments?
>
>
> >
> >
> > Hi Richard,
> > For getDisplayName, getDisplayLanguage() and methods like so, which are
> > locale-dependent, I suggest we write implementation tests for them.
> > The test
> > may look like:
> > 1. get default locale
> > 2. get i18n string from ResourceBundle directly
> > 3. get i18n string by locale-dependent method
> > 4. assertEquals
> If we write test cases like this, these tests are probably
> locale-independent, because: the executable is probably single. I don't
> think we should have many "locale-dependent" methods, we just have many
> methods with "locale-dependent" data.
> >
> > Sounds reasonable?
> >
> > Any comments? Thank you!
> >
> > Best regards,
> >> Richard
> >>
> >> > Regards,
> >> > Tim
> >> >
> >> >
> >>
> >> --
> >> Richard Liang
> >> China Software Development Lab, IBM
> >>
> >>
> >>
> >
> >
>
>
> --
> Paulex Yang
> China Software Development Lab
> IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Alexey A. Petrenko
Intel Middleware Products Division
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [testing] locale dependent tests
Posted by Paulex Yang <pa...@gmail.com>.
Tim Ellison wrote:
> Paulex Yang wrote:
>
>> I still confuse what we want to test, the logic or the data? I think
>> most (if not all) i18n related methods actually have same single
>> executable with multiple resource bundles, i.e., the single executable
>> should be locale-independent, the different return value is due to the
>> resource data difference. I think at least for now, we should pay our
>> attention to logic of single executable, and leave the data verification
>> to the i18n libraries' author, say, ICU, they have much more knowledge
>> and authority (at least than me) on this area.
>>
>
> Agreed
>
>
>> If we can get agree on the above, so the i18n related test cases
>> organization are easier to judge: the logic is locale-independent,
>>
>
> Ah, that is why I was trying to determine. If the logic is
> locale-independent then picking a locale to test with is ok; but it was
> unclear that was the case when changing locale caused assertion failures.
>
I think the reason why most these kind of tests fail on different
locales is the test cases leverage some locale-dependent data, say, use
',' as separator of numbers or so, so it is the test's bug, to fix them,
we only need to specify a locale to the test.
>
>> so ideally the tests should be locale-independent, but we have some
>> exceptional cases, say, the en_UK in MessageFormat case,
>>
>
> Do you mean that in this case the logic /is/ locale dependent? I'm
> confused again <g>.
>
And about the "exceptional case", say, the MessageFormat, the code is
still locale independent, but accidentally some different MessageFormat
in en_UK can have same behavior, it's still a data issue. But this data
issue is valuable here, please see the [1] for the MessageFormat's code
fragment used by toPattern(), in en_UK, the MessageFormat with "time,
long" pattern is always equals to the one with "time, full", so it never
returns ",time,full", the check order of RI only can be found in
en_UK(so far). So I called it "exceptional case".
[1]MessageFormat fragment
if (format.equals(DateFormat
.getTimeInstance(DateFormat.DEFAULT, locale))) {
buffer.append(",time");
} else if
(format.equals(DateFormat.getDateInstance(DateFormat.DEFAULT,
locale))) {
buffer.append(",date");
} else if
(format.equals(DateFormat.getTimeInstance(DateFormat.SHORT,
locale))) {
buffer.append(",time,short");
} else if
(format.equals(DateFormat.getDateInstance(DateFormat.SHORT,
locale))) {
buffer.append(",date,short");
} else if (format.equals(DateFormat.getTimeInstance(DateFormat.LONG,
locale))) {
buffer.append(",time,long");
} else if (format.equals(DateFormat.getDateInstance(DateFormat.LONG,
locale))) {
buffer.append(",date,long");
} else if (format.equals(DateFormat.getTimeInstance(DateFormat.FULL,
locale))) {
buffer.append(",time,full");
} else if (format.equals(DateFormat.getDateInstance(DateFormat.FULL,
locale))) {
buffer.append(",date,full");
} else {
buffer.append(",date,");
return ((SimpleDateFormat) format).toPattern();
}
>
>> so we cannot make our tests rely on the default locale, then we just
>> specify one locale(en_US) to the tests, and supplement some
>> exceptional case when we find some. i.e., I don't think we need
>> ABC_en_US_Test, or so.
>>
>
> Right, I don't think we need to have test_en_US blah, unless perhaps we
> pick one as the base for locale-dependent tests, otherwise we just run
> all tests in the machine default locale.
>
> Regards,
> Tim
>
>
--
Paulex Yang
China Software Development Lab
IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [testing] locale dependent tests
Posted by Tim Ellison <t....@gmail.com>.
Paulex Yang wrote:
> I still confuse what we want to test, the logic or the data? I think
> most (if not all) i18n related methods actually have same single
> executable with multiple resource bundles, i.e., the single executable
> should be locale-independent, the different return value is due to the
> resource data difference. I think at least for now, we should pay our
> attention to logic of single executable, and leave the data verification
> to the i18n libraries' author, say, ICU, they have much more knowledge
> and authority (at least than me) on this area.
Agreed
> If we can get agree on the above, so the i18n related test cases
> organization are easier to judge: the logic is locale-independent,
Ah, that is why I was trying to determine. If the logic is
locale-independent then picking a locale to test with is ok; but it was
unclear that was the case when changing locale caused assertion failures.
> so ideally the tests should be locale-independent, but we have some
> exceptional cases, say, the en_UK in MessageFormat case,
Do you mean that in this case the logic /is/ locale dependent? I'm
confused again <g>.
> so we cannot make our tests rely on the default locale, then we just
> specify one locale(en_US) to the tests, and supplement some
> exceptional case when we find some. i.e., I don't think we need
> ABC_en_US_Test, or so.
Right, I don't think we need to have test_en_US blah, unless perhaps we
pick one as the base for locale-dependent tests, otherwise we just run
all tests in the machine default locale.
Regards,
Tim
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [testing] locale dependent tests
Posted by Paulex Yang <pa...@gmail.com>.
Andrew Zhang wrote:
> On 7/19/06, Richard Liang <ri...@gmail.com> wrote:
>>
>>
>>
>> Tim Ellison wrote:
>> > Richard Liang wrote:
>> >
>> >> Although the spec does not require the round-trip of applyPattern and
>> >> toPattern, we still want to get one *certain* pattern through
>> toPattern.
>> >> Now the problem is the returned pattern is locale-dependent. I'm
>> >> uncertain about the reason to remove the assertion:
>> >> 1) Because the behavior is not required by spec, or
>> >> 2) Because the behavior is locale-dependent
>> >>
>> >
>> > It would seem that rather than forcing the locale to be en_US to make
>> > the test assertions valid, you could work with the default locale and
>> > guard any assertions that are locale-specific. It would seem a
>> shame to
>> > loose the diversity of testing on multiple locale machines if the
>> tests
>> > always force everyone to look like an American (horror! ;-) )
>> >
>> > I would expect the fix therefore to be something like, if locale is
>> > en_US go ahead and assert more round-tripping code, otherwise can't
>> test
>> > it as the spec allows variances.
>> >
>> >
>> If a test case passes in all locales, could we think that the behavior
>> tested by the test case is locale-independent? We still have to think
>> about how to test locale-dependent behavior. For example,
>> java.util.Locale.getDisplayName() and java.lang.String.toUpperCase(). To
>> verify both the code logic and locale data, shall we have some tests
>> like ABC_en_US_Test, ABC_en_UK_Test, and ABC_ru_RU_Test?
I still confuse what we want to test, the logic or the data? I think
most (if not all) i18n related methods actually have same single
executable with multiple resource bundles, i.e., the single executable
should be locale-independent, the different return value is due to the
resource data difference. I think at least for now, we should pay our
attention to logic of single executable, and leave the data verification
to the i18n libraries' author, say, ICU, they have much more knowledge
and authority (at least than me) on this area.
If we can get agree on the above, so the i18n related test cases
organization are easier to judge: the logic is locale-independent, so
ideally the tests should be locale-independent, but we have some
exceptional cases, say, the en_UK in MessageFormat case, so we cannot
make our tests rely on the default locale, then we just specify one
locale(en_US) to the tests, and supplement some exceptional case when we
find some. i.e., I don't think we need ABC_en_US_Test, or so.
Comments?
>
>
> Hi Richard,
> For getDisplayName, getDisplayLanguage() and methods like so, which are
> locale-dependent, I suggest we write implementation tests for them.
> The test
> may look like:
> 1. get default locale
> 2. get i18n string from ResourceBundle directly
> 3. get i18n string by locale-dependent method
> 4. assertEquals
If we write test cases like this, these tests are probably
locale-independent, because: the executable is probably single. I don't
think we should have many "locale-dependent" methods, we just have many
methods with "locale-dependent" data.
>
> Sounds reasonable?
>
> Any comments? Thank you!
>
> Best regards,
>> Richard
>>
>> > Regards,
>> > Tim
>> >
>> >
>>
>> --
>> Richard Liang
>> China Software Development Lab, IBM
>>
>>
>>
>
>
--
Paulex Yang
China Software Development Lab
IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: [testing] locale dependent tests
Posted by Andrew Zhang <zh...@gmail.com>.
On 7/19/06, Richard Liang <ri...@gmail.com> wrote:
>
>
>
> Tim Ellison wrote:
> > Richard Liang wrote:
> >
> >> Although the spec does not require the round-trip of applyPattern and
> >> toPattern, we still want to get one *certain* pattern through
> toPattern.
> >> Now the problem is the returned pattern is locale-dependent. I'm
> >> uncertain about the reason to remove the assertion:
> >> 1) Because the behavior is not required by spec, or
> >> 2) Because the behavior is locale-dependent
> >>
> >
> > It would seem that rather than forcing the locale to be en_US to make
> > the test assertions valid, you could work with the default locale and
> > guard any assertions that are locale-specific. It would seem a shame to
> > loose the diversity of testing on multiple locale machines if the tests
> > always force everyone to look like an American (horror! ;-) )
> >
> > I would expect the fix therefore to be something like, if locale is
> > en_US go ahead and assert more round-tripping code, otherwise can't test
> > it as the spec allows variances.
> >
> >
> If a test case passes in all locales, could we think that the behavior
> tested by the test case is locale-independent? We still have to think
> about how to test locale-dependent behavior. For example,
> java.util.Locale.getDisplayName() and java.lang.String.toUpperCase(). To
> verify both the code logic and locale data, shall we have some tests
> like ABC_en_US_Test, ABC_en_UK_Test, and ABC_ru_RU_Test?
Hi Richard,
For getDisplayName, getDisplayLanguage() and methods like so, which are
locale-dependent, I suggest we write implementation tests for them. The test
may look like:
1. get default locale
2. get i18n string from ResourceBundle directly
3. get i18n string by locale-dependent method
4. assertEquals
Sounds reasonable?
Any comments? Thank you!
Best regards,
> Richard
>
> > Regards,
> > Tim
> >
> >
>
> --
> Richard Liang
> China Software Development Lab, IBM
>
>
>
--
Andrew Zhang
China Software Development Lab, IBM