You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Dawid Weiss (JIRA)" <ji...@apache.org> on 2012/04/25 23:48:20 UTC
[jira] [Created] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Dawid Weiss created LUCENE-4021:
-----------------------------------
Summary: Locale representations don't round trip from toString->new Locale.
Key: LUCENE-4021
URL: https://issues.apache.org/jira/browse/LUCENE-4021
Project: Lucene - Java
Issue Type: Bug
Reporter: Dawid Weiss
Priority: Minor
The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
{noformat}
public void testLocaleRoundTrip() {
for (Locale locale : Locale.getAvailableLocales()) {
String s = locale.toString();
try {
Locale back = LuceneTestCase.localeForName(s);
assertEquals(locale, back);
} catch (Exception e) {
System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
} catch (AssertionError e) {
System.err.println("Not equals: " + s + " (" + e.toString() + ")");
}
}
}
{noformat}
Example locales that don't pass.
{noformat}
th_TH_TH_#u-nu-thai
ja_JP_JP_#u-ca-japanese
{noformat}
Interestingly, some that do pass the reverse trip are not .equal to their originals:
{noformat}
Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Dawid Weiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262201#comment-13262201 ]
Dawid Weiss commented on LUCENE-4021:
-------------------------------------
Heavy reflection or test ivy dependency on beanshell or some other simple scripting language... then you'd just write different snippets of code for different JVMs and execute them as scripts. This is a heavy beast for this but...
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Dawid Weiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262464#comment-13262464 ]
Dawid Weiss commented on LUCENE-4021:
-------------------------------------
Thanks Robert. Funny, I looked at that Javadoc but somehow skipped that special cases paragraph!
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
> Attachments: LUCENE-4021.patch
>
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-4021:
--------------------------------
Attachment: LUCENE-4021.patch
Here's the patch: now they round-trip.
Problem 1 was our switch statement on the number of elements:
"For compatibility reasons, two non-conforming locales are treated as special cases. These are ja_JP_JP and th_TH_TH. These are ill-formed in BCP 47 since the variants are too short. To ease migration to BCP 47, these are treated specially during construction. These two cases (and only these) cause a constructor to generate an extension, all other values behave exactly as they did prior to Java 7.
Java has used ja_JP_JP to represent Japanese as used in Japan together with the Japanese Imperial calendar. This is now representable using a Unicode locale extension, by specifying the Unicode locale key ca (for "calendar") and type japanese. When the Locale constructor is called with the arguments "ja", "JP", "JP", the extension "u-ca-japanese" is automatically added.
Java has used th_TH_TH to represent Thai as used in Thailand together with Thai digits. This is also now representable using a Unicode locale extension, by specifying the Unicode locale key nu (for "number") and value thai. When the Locale constructor is called with the arguments "th", "TH", "TH", the extension "u-nu-thai" is automatically added."
http://docs.oracle.com/javase/7/docs/api/java/util/Locale.html
Problem 2 was the assertEquals, I think like TimeZone we cannot guarantee this. So simply change the test to use toString(), which is well-defined too: http://docs.oracle.com/javase/7/docs/api/java/util/Locale.html#toString%28%29
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
> Attachments: LUCENE-4021.patch
>
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262175#comment-13262175 ]
Robert Muir commented on LUCENE-4021:
-------------------------------------
yeah we have to fix that Locale code i added.
The general problem is that java7 locales really shouldnt be parsed this way (imo) but
with forLanguageTag.
Any way i can guarantee you this is gonna be messy even if you know what you are doing
(e.g. http://bugs.icu-project.org/trac/ticket/8734).
And reproducibility between java6 and 7 is just out of the question.
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262197#comment-13262197 ]
Uwe Schindler commented on LUCENE-4021:
---------------------------------------
Heavy reflection only :-(
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-4021.
---------------------------------
Resolution: Fixed
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
> Attachments: LUCENE-4021.patch
>
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-4021) Locale representations don't round
trip from toString->new Locale.
Posted by "Dawid Weiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262194#comment-13262194 ]
Dawid Weiss commented on LUCENE-4021:
-------------------------------------
I'm just pointing out the fact, I don't plan to fix it. I realize it's going to be messy (there is not even a way to create a locale with script with regular constructors and Locale.Builder is only 1.7+). I'm not going to touch it.
> Locale representations don't round trip from toString->new Locale.
> ------------------------------------------------------------------
>
> Key: LUCENE-4021
> URL: https://issues.apache.org/jira/browse/LUCENE-4021
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Dawid Weiss
> Priority: Minor
>
> The script and extensions are not currently parsed. Seems like they were added in 1.7 only too so it'll probably require Java-version-specific workarounds.
> {noformat}
> public void testLocaleRoundTrip() {
> for (Locale locale : Locale.getAvailableLocales()) {
> String s = locale.toString();
> try {
> Locale back = LuceneTestCase.localeForName(s);
> assertEquals(locale, back);
> } catch (Exception e) {
> System.err.println("Failed roundtrip: " + s + " (" + e.toString() + ")");
> } catch (AssertionError e) {
> System.err.println("Not equals: " + s + " (" + e.toString() + ")");
> }
> }
> }
> {noformat}
> Example locales that don't pass.
> {noformat}
> th_TH_TH_#u-nu-thai
> ja_JP_JP_#u-ca-japanese
> {noformat}
> Interestingly, some that do pass the reverse trip are not .equal to their originals:
> {noformat}
> Not equals: sr_ME_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_ME_#Latn> but was: java.util.Locale<sr_ME_#Latn>)
> Not equals: sr_BA_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_BA_#Latn> but was: java.util.Locale<sr_BA_#Latn>)
> Not equals: sr__#Latn (java.lang.AssertionError: expected: java.util.Locale<sr__#Latn> but was: java.util.Locale<sr__#Latn>)
> Not equals: sr_RS_#Latn (java.lang.AssertionError: expected: java.util.Locale<sr_RS_#Latn> but was: java.util.Locale<sr_RS_#Latn>)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org