You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pablo Recio <pr...@yaco.es> on 2010/10/28 12:59:25 UTC

Possible bug in query sorting

Hi all. I'm having a problem with solr sorting search results.

When I try to make a query and sort it by title:

http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&sort=title%20desc

I get that error [1]. If I try to sort by other indexed field it works, indeed
if I change in solr schema title name to titlx, for example, it works.

It's a bug? Anyone has the same problem?

[1] HTTP ERROR: 500

501

java.lang.ArrayIndexOutOfBoundsException: 501
    at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:721)
    at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:224)
    at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:692)
    at org.apache.lucene.search.FieldComparator$StringOrdValComparator.setNextReader(FieldComparator.java:667)
    at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:249)
    at org.apache.lucene.search.Searcher.search(Searcher.java:171)
    at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
    at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
    at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
    at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
    at org.mortbay.jetty.Server.handle(Server.java:285)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
    at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)

RequestURI=*/solr/select/*

Powered by Jetty://

Re: Possible bug in query sorting

Posted by Gora Mohanty <go...@mimirtech.com>.
On Fri, Oct 29, 2010 at 1:47 PM, Pablo Recio <pr...@yaco.es> wrote:
> That's my schema XML:

>   <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
>     <analyzer type="index">
>       <tokenizer class="solr.
> WhitespaceTokenizerFactory"/>
>       <filter class="solr.LowerCaseFilterFactory" />
>       <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
>       <filter class="solr.ISOLatin1AccentFilterFactory" />
>     </analyzer>
>     <analyzer type="query">
>       <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>       <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/>
>       <filter class="solr.LowerCaseFilterFactory" />
>       <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
>       <filter class="solr.ISOLatin1AccentFilterFactory" />
>     </analyzer>
>   </fieldType>
>  </types>
>
>  <fields>
[...]
>  <field name="title" type="text" indexed="true" stored="true"
> required="true" multiValued="false" omitNorms="false" />
>  <field name="contributor" type="text" indexed="false" stored="false"
> required="false" multiValued="false" omitNorms="false" />
>  ....
>  </fields>
[...]

The issue is that you are using the WhitespaceTokenizerFactory
as an analyzer for the field. This is resulting in a different number
of tokens in different documents, which is causing the error.

Use a field that is non-tokenized, e.g., change the type of the
"title" field to "string". If you need a tokenized "title" field, copy
the field to another of type "string", and sort on that field instead.
Please see http://wiki.apache.org/solr/CommonQueryParameters#sort

Regards,
Gora

Re: Possible bug in query sorting

Posted by Pablo Recio <pr...@yaco.es>.
That's my schema XML:

<?xml version="1.0" encoding="UTF-8" ?>
<schema name="example" version="1.2">
 <types>
   <fieldType name="string" class="solr.StrField" sortMissingLast="true"
omitNorms="true"/>
   <fieldType name="uuid" class="solr.UUIDField" indexed="true"
required="true" omitNorms="true"/>
   <fieldType name="date" class="solr.TrieDateField" omitNorms="true"
precisionStep="0" positionIncrementGap="0"/>
   <fieldType name="integer" class="solr.IntField" omitNorms="true"/>
   <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
     <analyzer type="index">
       <tokenizer class="solr.
WhitespaceTokenizerFactory"/>
       <filter class="solr.LowerCaseFilterFactory" />
       <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
       <filter class="solr.ISOLatin1AccentFilterFactory" />
     </analyzer>
     <analyzer type="query">
       <tokenizer class="solr.WhitespaceTokenizerFactory"/>
       <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/>
       <filter class="solr.LowerCaseFilterFactory" />
       <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
       <filter class="solr.ISOLatin1AccentFilterFactory" />
     </analyzer>
   </fieldType>
 </types>

 <fields>
  <field name="text" type="text" indexed="true" stored="false"
required="false" multiValued="false" omitNorms="false" />
  <field name="icms_collection" type="text" indexed="true" stored="true"
required="true" multiValued="false" omitNorms="false" />
  <field name="link" type="text" indexed="true" stored="true"
required="true" multiValued="false" omitNorms="false" />
  <field name="title" type="text" indexed="true" stored="true"
required="true" multiValued="false" omitNorms="false" />
  <field name="contributor" type="text" indexed="false" stored="false"
required="false" multiValued="false" omitNorms="false" />
  ....
 </fields>

 <uniqueKey>link</uniqueKey>
 <defaultSearchField>text</defaultSearchField>

 <solrQueryParser defaultOperator="AND"/>

 <copyField source="title" dest="text"/>
 <copyField source="contributor" dest="text"/>
 ...

</schema>


2010/10/28 Gora Mohanty <go...@mimirtech.com>

> On Thu, Oct 28, 2010 at 5:18 PM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
> > Is it somehow possible that you are trying to sort by a multi-valued
> field?
> [...]
>
> Either that, or or your field gets processed into multiple tokens via the
> analyzer/tokenizer path in your schema. The reported error is a
> consequence of the fact that different documents might result in a
> different number of tokens.
>
> Please show us the part of schema.xml that defines the field type for
> the field "title".
>
> Regards,
> Gora
>

Re: Possible bug in query sorting

Posted by Gora Mohanty <go...@mimirtech.com>.
On Thu, Oct 28, 2010 at 5:18 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Is it somehow possible that you are trying to sort by a multi-valued field?
[...]

Either that, or or your field gets processed into multiple tokens via the
analyzer/tokenizer path in your schema. The reported error is a
consequence of the fact that different documents might result in a
different number of tokens.

Please show us the part of schema.xml that defines the field type for
the field "title".

Regards,
Gora

Re: Possible bug in query sorting

Posted by Michael McCandless <lu...@mikemccandless.com>.
Is it somehow possible that you are trying to sort by a multi-valued field?

Mike

On Thu, Oct 28, 2010 at 6:59 AM, Pablo Recio <pr...@yaco.es> wrote:
> Hi all. I'm having a problem with solr sorting search results.
>
> When I try to make a query and sort it by title:
>
> http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&sort=title%20desc
>
> I get that error [1]. If I try to sort by other indexed field it works, indeed
> if I change in solr schema title name to titlx, for example, it works.
>
> It's a bug? Anyone has the same problem?
>
> [1] HTTP ERROR: 500
>
> 501
>
> java.lang.ArrayIndexOutOfBoundsException: 501
>    at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:721)
>    at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:224)
>    at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:692)
>    at org.apache.lucene.search.FieldComparator$StringOrdValComparator.setNextReader(FieldComparator.java:667)
>    at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
>    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:249)
>    at org.apache.lucene.search.Searcher.search(Searcher.java:171)
>    at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
>    at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
>    at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
>    at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
>    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
>    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
>    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
>    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>    at org.mortbay.jetty.Server.handle(Server.java:285)
>    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
>    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
>    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
>    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
>    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
>    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
>    at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>
> RequestURI=*/solr/select/*
>
> Powered by Jetty://
>