You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2011/01/28 03:04:43 UTC
[jira] Created: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
No error reported when sorting on a field Solr knows you shouldn't sort on.
---------------------------------------------------------------------------
Key: SOLR-2339
URL: https://issues.apache.org/jira/browse/SOLR-2339
Project: Solr
Issue Type: Bug
Components: search
Reporter: Hoss Man
Assignee: Hoss Man
Fix For: 3.1, 4.0
In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989499#comment-12989499 ]
Hoss Man commented on SOLR-2339:
--------------------------------
I should have mentioned when i attached the patch: the approach i took here was to add a "void checkSortability()" method to SchemaField that tests isMultivalued -- i also moved the "is this field indexed?" check here (it use to be part of QueryParsing.parseSortSpec) and changed all the out of hte box FieldTypes to call this method in their getSortField implementation -- this way we have acommon (documented) hook point where we can add future helper tests for things that don't make sense of in sorting, but it's still up to the FieldType to decide what to do if someone wants to make a really exotic one (RandomSortField for example doesn't even care if it's indexed).
If anyone has any better suggestions please chime in.
> No error reported when sorting on a field Solr knows you shouldn't sort on.
> ---------------------------------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989016#comment-12989016 ]
Hoss Man commented on SOLR-2339:
--------------------------------
yonik pointed out on IRC that the main motivation behind LUCENE-2142 could easily apply to solr users as well...
bq. So this means... you could happily think all is good, until some day, well into production, once you've updated enough docs, suddenly the check will catch you and throw an unhandled exception, stopping all searches [that need to sort by this string field] in their tracks. It's not gracefully degrading.
...arguing that for solr users as well: the loss of this particular check is a feature and not a bug.
i feel like the user bases and use cases are different enough that the "default" behavior should be different -- but not enough to fight strongly for it.
if anyone else has an opinion, please comment -- for now i'm going to just tackle the case where there is not doubt you have a problem: sorting on a multiValued field
> No error reported when sorting on a field Solr knows you shouldn't sort on.
> ---------------------------------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-2339:
---------------------------
Attachment: SOLR-2339.patch
Updated patch that fixes the remaining sort related test failures.
TestDistributedSearch and BasicDistributedZkTest were fundementally flawed in that they expected to get deterministic sort orderings on fields that they were deliberately putting multiple values into (ie: not just sorting on a multivalued field; sorting on a multivalued field that was actually used to store multiple values for each doc)
I'm amazed they ever passed.
> No error reported when sorting on a field Solr knows you shouldn't sort on.
> ---------------------------------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch, SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-2339:
---------------------------
Attachment: SOLR-2339.patch
patch that forces failure for multivalued fields, along with an example of how i think we should call this behavior change out in CHANGES.txt
this uncovered a lot of tests that were missusing fields declared as multivalued -- BasicDistributedZkTest and TestDistributedSearch are still failing in this patch, largely because i'm so confused about how they work it's not even clear to me what schema.xml file they are using to try and fix it.
> No error reported when sorting on a field Solr knows you shouldn't sort on.
> ---------------------------------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2339) No error reported when sorting on a
multiValued field
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-2339:
---------------------------
Summary: No error reported when sorting on a multiValued field (was: No error reported when sorting on a field Solr knows you shouldn't sort on.)
narrowing issue summary to be specific about multiValued fields since that's all we've decided to tackle for now.
> No error reported when sorting on a multiValued field
> -----------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch, SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989500#comment-12989500 ]
Hoss Man commented on SOLR-2339:
--------------------------------
It also occurs to me that we should probably put this same check in for dealing with ValueSources that are FieldCache based ... not in ValueSource.getSortField, but in FieldType.getValueSource for the affected FieldTypes.
> No error reported when sorting on a field Solr knows you shouldn't sort on.
> ---------------------------------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2339) No error reported when sorting on a
multiValued field
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990378#comment-12990378 ]
Hoss Man commented on SOLR-2339:
--------------------------------
Committed revision 1067030. - trunk
Still working on the backport to 3x (this is hte first time svn has ever given me a "Tree conflict" for a file)
> No error reported when sorting on a multiValued field
> -----------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch, SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2339) No error reported when sorting on a
field Solr knows you shouldn't sort on.
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987888#action_12987888 ]
Hoss Man commented on SOLR-2339:
--------------------------------
I'm not sure the best way to go about this ... we could reimplement the basic termcount vs doccount check that use to be done at a higher level before using the FieldCache -- but that may be overkill and i'm not convinced we need to behave in exactly the same way as before.
At a minimum we should probably start logging a warning if someone does a sort on a field that is configured to be multiValued=true.
Alternately: we could consider adding a property to the StringIndex and DocTermsIndex classes (on their respective branches) so that FieldCache could indicate when it suspects there may be a problem (using simple doccount < termcount, or something more complex if it's easy), and Solr could error/log if that property is set.
> No error reported when sorting on a field Solr knows you shouldn't sort on.
> ---------------------------------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2339) No error reported when sorting on a
multiValued field
Posted by "Bill Bell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034492#comment-13034492 ]
Bill Bell commented on SOLR-2339:
---------------------------------
Guys,
How are we going to support sorting on multiValued fields?
Would a function work for this?
> No error reported when sorting on a multiValued field
> -----------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch, SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (SOLR-2339) No error reported when sorting on a
multiValued field
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man resolved SOLR-2339.
----------------------------
Resolution: Fixed
Committed revision 1067042.
backported to 3x
> No error reported when sorting on a multiValued field
> -----------------------------------------------------
>
> Key: SOLR-2339
> URL: https://issues.apache.org/jira/browse/SOLR-2339
> Project: Solr
> Issue Type: Bug
> Components: search
> Reporter: Hoss Man
> Assignee: Hoss Man
> Fix For: 3.1, 4.0
>
> Attachments: SOLR-2339.patch, SOLR-2339.patch
>
>
> In the past, Solr has relied on the underlying FieldCache to throw an error in situations where sorting on a field was not possible. however LUCENE-2142 has changed this, so that FieldCache never throws an error.
> In order to maintain the functionality of past Solr releases (ie: error when users attempt to sort on a field that we known will produce meaningless results) we should add some sort of check at the Solr level.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org