You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Sachin (JIRA)" <ji...@apache.org> on 2009/03/17 10:02:50 UTC
[jira] Created: (SOLR-1073) StrField should allow locale sensitive
sorting
StrField should allow locale sensitive sorting
----------------------------------------------
Key: SOLR-1073
URL: https://issues.apache.org/jira/browse/SOLR-1073
Project: Solr
Issue Type: Improvement
Environment: All
Reporter: Sachin
Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
snip:
public SortField getSortField(SchemaField field,boolean reverse)
{
...
Locale locale = new Locale(lang,country);
return new SortField(field.getName(), locale, reverse);
}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1073) StrField should allow locale
sensitive sorting
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703884#action_12703884 ]
Hoss Man commented on SOLR-1073:
--------------------------------
The patch seems ok to me, ideally it should validate the Locale during init and fail if it doesn't exist, but that doesn't seem like a show stopper. I'm personally not comfortable committing it without a basic test (Locale stuff is tricky, it would be good to know we aren't tickling some weird Lucene-Java bug in an odd way).
The only other big catch in my mind is that this new class doesn't respect the sortMissingLast & sortMissingTrue options that StrField does. We should at least document this in the javadocs, but ideally we should make them work as well. I *think* it would be possible if we changed MissingStringLastComparatorSource so that it could wrap another SortComparatorSource (do the null checks first, then delegate). if we managed to get that working, then there wouldn't be any reason for a new LocaleStrField, it could just be a new option on StrField.
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
> More details about this issue here:
> http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1073) StrField should allow locale
sensitive sorting
Posted by "Walter Underwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703893#action_12703893 ]
Walter Underwood commented on SOLR-1073:
----------------------------------------
Using the locale of the JVM is very, very bad for a multilingual server. Solr should always use the same, simple locale. It is OK to set a Locale in configuration for single-language installations, but using the JVM locale is a recipe for disaster. You move Solr to a different server and everything breaks. Very, very bad.
In a multi-lingual config, locales must be set per-request.
Ideally, requests should send an ISO language code as context for the query.
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
> More details about this issue here:
> http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1073) StrField should allow locale sensitive
sorting
Posted by "Sachin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sachin updated SOLR-1073:
-------------------------
Description:
Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
snip:
public SortField getSortField(SchemaField field,boolean reverse)
{
...
Locale locale = new Locale(lang,country);
return new SortField(field.getName(), locale, reverse);
}
More details about this issue here:
http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
was:
Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
snip:
public SortField getSortField(SchemaField field,boolean reverse)
{
...
Locale locale = new Locale(lang,country);
return new SortField(field.getName(), locale, reverse);
}
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
> More details about this issue here:
> http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1073) StrField should allow locale
sensitive sorting
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785185#action_12785185 ]
Robert Muir commented on SOLR-1073:
-----------------------------------
fyi, SOLR-1571 is an alternative implementation for this feature, but instead works as a TokenFilter, indexing collation keys. (You then just sort on that field).
It doesn't have the performance problems of Lucene's Locale-based sort, but on the other hand it requires some additional space in the index (the work is done at index time).
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Fix For: 1.5
>
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
> More details about this issue here:
> http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1073) StrField should allow locale sensitive
sorting
Posted by "Sachin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sachin updated SOLR-1073:
-------------------------
Attachment: LocaleStrField.java
A new data type which extends StrField and provides locale specific sorting.
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1073) StrField should allow locale sensitive
sorting
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-1073:
----------------------------------------
Fix Version/s: 1.5
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Fix For: 1.5
>
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
> More details about this issue here:
> http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1073) StrField should allow locale
sensitive sorting
Posted by "Sachin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706949#action_12706949 ]
Sachin commented on SOLR-1073:
------------------------------
Hoss:
1) I didn't add the validation as currently it falls back on the default implementation of String sort in case the locale is not specified.
2) About the basic test, we've using this on http://cn.allrecipes.com for last month or so and haven't had users complaining about it (yet).
3) sortMissingLast & sortMissingTrue wasn't implemented as I only need sorting for a field which doesn't allow null :) but I'll be quite happy to get that in.
Walter:
Sorry for being a little slow but I didn't understand much (my lack of understanding of java), can you please elaborate a little?
Thanks!
> StrField should allow locale sensitive sorting
> ----------------------------------------------
>
> Key: SOLR-1073
> URL: https://issues.apache.org/jira/browse/SOLR-1073
> Project: Solr
> Issue Type: Improvement
> Environment: All
> Reporter: Sachin
> Attachments: LocaleStrField.java
>
>
> Currently, StrField does not take a parameter which it can pass to ctor of SortField making the StrField's sorting rely on the locale of the JVM. Ideally, StrField should allow setting the locale in the schema.xml and use it to create a new instance of the SortField in getSortField() method, something like:
> snip:
> public SortField getSortField(SchemaField field,boolean reverse)
> {
> ...
> Locale locale = new Locale(lang,country);
> return new SortField(field.getName(), locale, reverse);
> }
> More details about this issue here:
> http://www.nabble.com/CJKAnalyzer-and-Chinese-Text-sort-td22374195.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.