You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pablo Recio <pr...@yaco.es> on 2010/10/28 12:59:25 UTC
Possible bug in query sorting
Hi all. I'm having a problem with solr sorting search results.
When I try to make a query and sort it by title:
http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&sort=title%20desc
I get that error [1]. If I try to sort by other indexed field it works, indeed
if I change in solr schema title name to titlx, for example, it works.
It's a bug? Anyone has the same problem?
[1] HTTP ERROR: 500
501
java.lang.ArrayIndexOutOfBoundsException: 501
at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:721)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:224)
at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:692)
at org.apache.lucene.search.FieldComparator$StringOrdValComparator.setNextReader(FieldComparator.java:667)
at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:249)
at org.apache.lucene.search.Searcher.search(Searcher.java:171)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
RequestURI=*/solr/select/*
Powered by Jetty://
Re: Possible bug in query sorting
Posted by Gora Mohanty <go...@mimirtech.com>.
On Fri, Oct 29, 2010 at 1:47 PM, Pablo Recio <pr...@yaco.es> wrote:
> That's my schema XML:
> <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.
> WhitespaceTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
> <filter class="solr.ISOLatin1AccentFilterFactory" />
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/>
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
> <filter class="solr.ISOLatin1AccentFilterFactory" />
> </analyzer>
> </fieldType>
> </types>
>
> <fields>
[...]
> <field name="title" type="text" indexed="true" stored="true"
> required="true" multiValued="false" omitNorms="false" />
> <field name="contributor" type="text" indexed="false" stored="false"
> required="false" multiValued="false" omitNorms="false" />
> ....
> </fields>
[...]
The issue is that you are using the WhitespaceTokenizerFactory
as an analyzer for the field. This is resulting in a different number
of tokens in different documents, which is causing the error.
Use a field that is non-tokenized, e.g., change the type of the
"title" field to "string". If you need a tokenized "title" field, copy
the field to another of type "string", and sort on that field instead.
Please see http://wiki.apache.org/solr/CommonQueryParameters#sort
Regards,
Gora
Re: Possible bug in query sorting
Posted by Pablo Recio <pr...@yaco.es>.
That's my schema XML:
<?xml version="1.0" encoding="UTF-8" ?>
<schema name="example" version="1.2">
<types>
<fieldType name="string" class="solr.StrField" sortMissingLast="true"
omitNorms="true"/>
<fieldType name="uuid" class="solr.UUIDField" indexed="true"
required="true" omitNorms="true"/>
<fieldType name="date" class="solr.TrieDateField" omitNorms="true"
precisionStep="0" positionIncrementGap="0"/>
<fieldType name="integer" class="solr.IntField" omitNorms="true"/>
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.
WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory" />
<filter class="solr.ISOLatin1AccentFilterFactory" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.RemoveDuplicatesTokenFilterFactory" />
<filter class="solr.ISOLatin1AccentFilterFactory" />
</analyzer>
</fieldType>
</types>
<fields>
<field name="text" type="text" indexed="true" stored="false"
required="false" multiValued="false" omitNorms="false" />
<field name="icms_collection" type="text" indexed="true" stored="true"
required="true" multiValued="false" omitNorms="false" />
<field name="link" type="text" indexed="true" stored="true"
required="true" multiValued="false" omitNorms="false" />
<field name="title" type="text" indexed="true" stored="true"
required="true" multiValued="false" omitNorms="false" />
<field name="contributor" type="text" indexed="false" stored="false"
required="false" multiValued="false" omitNorms="false" />
....
</fields>
<uniqueKey>link</uniqueKey>
<defaultSearchField>text</defaultSearchField>
<solrQueryParser defaultOperator="AND"/>
<copyField source="title" dest="text"/>
<copyField source="contributor" dest="text"/>
...
</schema>
2010/10/28 Gora Mohanty <go...@mimirtech.com>
> On Thu, Oct 28, 2010 at 5:18 PM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
> > Is it somehow possible that you are trying to sort by a multi-valued
> field?
> [...]
>
> Either that, or or your field gets processed into multiple tokens via the
> analyzer/tokenizer path in your schema. The reported error is a
> consequence of the fact that different documents might result in a
> different number of tokens.
>
> Please show us the part of schema.xml that defines the field type for
> the field "title".
>
> Regards,
> Gora
>
Re: Possible bug in query sorting
Posted by Gora Mohanty <go...@mimirtech.com>.
On Thu, Oct 28, 2010 at 5:18 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Is it somehow possible that you are trying to sort by a multi-valued field?
[...]
Either that, or or your field gets processed into multiple tokens via the
analyzer/tokenizer path in your schema. The reported error is a
consequence of the fact that different documents might result in a
different number of tokens.
Please show us the part of schema.xml that defines the field type for
the field "title".
Regards,
Gora
Re: Possible bug in query sorting
Posted by Michael McCandless <lu...@mikemccandless.com>.
Is it somehow possible that you are trying to sort by a multi-valued field?
Mike
On Thu, Oct 28, 2010 at 6:59 AM, Pablo Recio <pr...@yaco.es> wrote:
> Hi all. I'm having a problem with solr sorting search results.
>
> When I try to make a query and sort it by title:
>
> http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&sort=title%20desc
>
> I get that error [1]. If I try to sort by other indexed field it works, indeed
> if I change in solr schema title name to titlx, for example, it works.
>
> It's a bug? Anyone has the same problem?
>
> [1] HTTP ERROR: 500
>
> 501
>
> java.lang.ArrayIndexOutOfBoundsException: 501
> at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:721)
> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:224)
> at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:692)
> at org.apache.lucene.search.FieldComparator$StringOrdValComparator.setNextReader(FieldComparator.java:667)
> at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:249)
> at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>
> RequestURI=*/solr/select/*
>
> Powered by Jetty://
>