You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Michael Gibney (Jira)" <ji...@apache.org> on 2019/10/31 18:04:00 UTC
[jira] [Commented] (SOLR-12457) field(x,min|max) sorting doesn't
work on trie or str fields in multi-shard collections
[ https://issues.apache.org/jira/browse/SOLR-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964263#comment-16964263 ]
Michael Gibney commented on SOLR-12457:
---------------------------------------
Thanks [~hossman] for the tests and thorough exploration of the problem! After looking into this, I feel there are three issues (the first of which is largely already covered by comments on this issue); all should be addressed (I think) with [PR 989|https://github.com/apache/lucene-solr/pull/989]:
# Field function values bypass field type sort value marshal/unmarshal, despite the fact that field-function-derived values (e.g., indexed) are exactly the same as those generated from a simple field sort (e.g., "sort=my_field asc"). Presumably, any FieldType-specific logic that would leverage marshal/unmarshal for simple field sort would _also_ call for similar handling of values from the field _function_. Trie fields are perhaps an imperfect example here, but I think it's probably worth invoking marshal/unmarshal on all field-function-generated values, for consistency with "simple field sort", as a matter of general principle.
# Trie fields traffic in BytesRef values because they implement multivalue sort via SortedSetSortField. Marshal/unmarshal would be [one way to handle this situation|https://github.com/apache/lucene-solr/compare/0af7b62...91bd715], but potentially a [cleaner way|https://github.com/apache/lucene-solr/commit/830f44b] would be to subclass SortedSetFieldSource to produce numeric values, with a corresponding SortField capable of consuming such a FieldSource.
# Another, more general problem is that field function SortFields are not capable of accommodating missingValue, which makes them brittle when used with sortMissingFirst or sortMissingLast. This problem affects all field types, not just Trie, etc. [PR 989|https://github.com/apache/lucene-solr/pull/989] adds tests and a fix for this case.
> field(x,min|max) sorting doesn't work on trie or str fields in multi-shard collections
> --------------------------------------------------------------------------------------
>
> Key: SOLR-12457
> URL: https://issues.apache.org/jira/browse/SOLR-12457
> Project: Solr
> Issue Type: Bug
> Affects Versions: 7.1
> Reporter: Varun Thacker
> Priority: Major
> Labels: numeric-tries-to-points
> Attachments: SOLR-12457.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When I go to sort on a multi-valued field in a 2 shard collection, which has trie fields the query fails.
> To reproduce we need 2+ shards, a multi-valued trie field and "desc" sort criteria.
> Here's my schema
> {code:java}
> <dynamicField name="*_is" type="int" indexed="true" stored="true" multiValued="true" docValues="true"/>
> <fieldType class="solr.TrieIntField" docValues="true" name="int" positionIncrementGap="0" precisionStep="0"/>
> <dynamicField name="*_i" type="pint" indexed="true" stored="true" multiValued="true"/>
> <fieldType name="pint" class="solr.IntPointField" docValues="true"/>
> {code}
> Now If I add a few docs
> {code:java}
> [
> {"id" : "1", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2", "3", "4", "5"]},
> {"id" : "2", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2", "3", "4", "5"]},
> {"id" : "3", "test_is" : ["1", "2", "3", "4", "5"], "test_i" : ["1", "2", "3", "4", "5"]}
> ]{code}
> Works:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*&sort=field(test_i,max)%20desc]
>
> Doesn't Work:
> [http://localhost:8983/solr/gettingstarted/select?q=*:*&sort=field(test_is,max)%20desc]
>
> To be more clear when I say it doesn't work , the query throws and error and here's the stack trace for it:
> {code:java}
> ERROR - 2018-06-06 22:55:06.599; [c:gettingstarted s:shard2 r:core_node8 x:gettingstarted_shard2_replica_n5] org.apache.solr.common.SolrException; null:java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.lucene.util.BytesRef
> at org.apache.lucene.search.FieldComparator$TermOrdValComparator.compareValues(FieldComparator.java:561)
> at org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:161)
> at org.apache.solr.handler.component.ShardFieldSortedHitQueue$1.compare(ShardFieldSortedHitQueue.java:153)
> at org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:91)
> at org.apache.solr.handler.component.ShardFieldSortedHitQueue.lessThan(ShardFieldSortedHitQueue.java:33)
> at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:263)
> at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
> at org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:156)
> at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:924)
> at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
> at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2484)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org