You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Jon Poulton <Jo...@vyre.com> on 2010/10/08 13:14:13 UTC

NumberFormatException upon reading a Trie field during search

Hi there,
I have recently upgraded our Solr instance and have reindexed all of the items in our store, and for at least one search I am getting some unusual error messages back for a search that previously worked. It reads as follows:

HTTP Status 500 - Invalid shift value in prefixCoded string (is encoded value really an INT?)
java.lang.NumberFormatException: Invalid shift value in prefixCoded string (is encoded value really an INT?)
	at org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
	at org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
	at org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
	at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
	at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
	at org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
	at org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
	at org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
	at org.apache.lucene.search.Searcher.search(Searcher.java:171)
	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:619)
java.lang.NumberFormatException: Invalid shift value in prefixCoded string (is encoded value really an INT?)
	at org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
	at org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
	at org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
	at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
	at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
	at org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
	at org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
	at org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
	at org.apache.lucene.search.Searcher.search(Searcher.java:171)
	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:619)
java.lang.NumberFormatException: Invalid shift value in prefixCoded string (is encoded value really an INT?)
	at org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
	at org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
	at org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
	at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
	at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
	at org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
	at org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
	at org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
	at org.apache.lucene.search.Searcher.search(Searcher.java:171)
	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:619)

A quick Google search revealed that this may have something to do with the use of Trie fields (which we have just started using) as the Exception is being thrown initially from a class called NumericUtils:

http://www.jarvana.com/jarvana/view/org/apache/lucene/lucene-core/3.0.2/lucene-core-3.0.2-sources.jar!/org/apache/lucene/util/NumericUtils.java?format=ok

I have a copy of the index open in Luke and I'm trying to find the source of the problem, but the stack trace doesn't give the name of the field, or document, and the only integer Trie field we use only has a few discrete values (1,2,3,4,5 in a field called root_priority), so I'm at a loss at where to look next. When I run the original query and the filter queries through Luke they seem to work just fine and bring back a set of documents. 

The Solr config for the trie fields looks fine to me:

        <fieldtype name="integer" class="solr.TrieField" type="integer" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
        <fieldtype name="long" class="solr.TrieField" type="long" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
        <fieldtype name="float" class="solr.TrieField" type="float" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
        <fieldtype name="double" class="solr.TrieField" type="double" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
        <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>

Any ideas on where I can go from here?

Thanks

Jon

Re: NumberFormatException upon reading a Trie field during search

Posted by Erick Erickson <er...@gmail.com>.

Thanks for letting us know...

Erick

On Fri, Oct 8, 2010 at 11:42 AM, Jon Poulton <Jo...@vyre.com> wrote:

> Hi all,
>
> Just to let you know, deleting the index and reindexing our data appears to
> have fixed this problem, at least for the moment. My guess is that the old
> index wasn't deleted cleanly, as I assumed it had been.
>
> Thanks
>
> Jon
>
>
> On 8 Oct 2010, at 12:14, Jon Poulton wrote:
>
> > Hi there,
> > I have recently upgraded our Solr instance and have reindexed all of the
> items in our store, and for at least one search I am getting some unusual
> error messages back for a search that previously worked. It reads as
> follows:
> >
> > HTTP Status 500 - Invalid shift value in prefixCoded string (is encoded
> value really an INT?)
> > java.lang.NumberFormatException: Invalid shift value in prefixCoded
> string (is encoded value really an INT?)
> >       at
> org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
> >       at
> org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
> >       at
> org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
> >       at
> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> >       at
> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
> >       at
> org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
> >       at
> org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
> >       at
> org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
> >       at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
> >       at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> >       at
> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> >       at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> >       at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> >       at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> >       at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> >       at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> >       at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> >       at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> >       at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> >       at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> >       at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >       at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> >       at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> >       at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> >       at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >       at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >       at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> >       at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> >       at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> >       at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> >       at java.lang.Thread.run(Thread.java:619)
> > java.lang.NumberFormatException: Invalid shift value in prefixCoded
> string (is encoded value really an INT?)
> >       at
> org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
> >       at
> org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
> >       at
> org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
> >       at
> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> >       at
> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
> >       at
> org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
> >       at
> org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
> >       at
> org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
> >       at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
> >       at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> >       at
> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> >       at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> >       at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> >       at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> >       at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> >       at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> >       at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> >       at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> >       at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> >       at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> >       at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >       at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> >       at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> >       at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> >       at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >       at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >       at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> >       at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> >       at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> >       at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> >       at java.lang.Thread.run(Thread.java:619)
> > java.lang.NumberFormatException: Invalid shift value in prefixCoded
> string (is encoded value really an INT?)
> >       at
> org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
> >       at
> org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
> >       at
> org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
> >       at
> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> >       at
> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
> >       at
> org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
> >       at
> org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
> >       at
> org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
> >       at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
> >       at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> >       at
> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> >       at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> >       at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> >       at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> >       at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> >       at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> >       at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> >       at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> >       at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> >       at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> >       at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> >       at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> >       at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> >       at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> >       at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >       at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >       at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> >       at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> >       at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> >       at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> >       at java.lang.Thread.run(Thread.java:619)
> >
> > A quick Google search revealed that this may have something to do with
> the use of Trie fields (which we have just started using) as the Exception
> is being thrown initially from a class called NumericUtils:
> >
> >
> http://www.jarvana.com/jarvana/view/org/apache/lucene/lucene-core/3.0.2/lucene-core-3.0.2-sources.jar!/org/apache/lucene/util/NumericUtils.java?format=ok
> >
> > I have a copy of the index open in Luke and I'm trying to find the source
> of the problem, but the stack trace doesn't give the name of the field, or
> document, and the only integer Trie field we use only has a few discrete
> values (1,2,3,4,5 in a field called root_priority), so I'm at a loss at
> where to look next. When I run the original query and the filter queries
> through Luke they seem to work just fine and bring back a set of documents.
> >
> > The Solr config for the trie fields looks fine to me:
> >
> >        <fieldtype name="integer" class="solr.TrieField" type="integer"
> precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
> >        <fieldtype name="long" class="solr.TrieField" type="long"
> precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
> >        <fieldtype name="float" class="solr.TrieField" type="float"
> precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
> >        <fieldtype name="double" class="solr.TrieField" type="double"
> precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
> >        <fieldType name="date" class="solr.TrieDateField" omitNorms="true"
> precisionStep="6" positionIncrementGap="0"/>
> >
> > Any ideas on where I can go from here?
> >
> > Thanks
> >
> > Jon
>
>

Re: NumberFormatException upon reading a Trie field during search

Posted by Jon Poulton <Jo...@vyre.com>.

Hi all,

Just to let you know, deleting the index and reindexing our data appears to have fixed this problem, at least for the moment. My guess is that the old index wasn't deleted cleanly, as I assumed it had been. 

Thanks

Jon


On 8 Oct 2010, at 12:14, Jon Poulton wrote:

> Hi there,
> I have recently upgraded our Solr instance and have reindexed all of the items in our store, and for at least one search I am getting some unusual error messages back for a search that previously worked. It reads as follows:
> 
> HTTP Status 500 - Invalid shift value in prefixCoded string (is encoded value really an INT?)
> java.lang.NumberFormatException: Invalid shift value in prefixCoded string (is encoded value really an INT?)
> 	at org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
> 	at org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
> 	at org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
> 	at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> 	at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
> 	at org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
> 	at org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
> 	at org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> 	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> 	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> 	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> 	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> 	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> 	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> 	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> 	at java.lang.Thread.run(Thread.java:619)
> java.lang.NumberFormatException: Invalid shift value in prefixCoded string (is encoded value really an INT?)
> 	at org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
> 	at org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
> 	at org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
> 	at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> 	at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
> 	at org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
> 	at org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
> 	at org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> 	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> 	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> 	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> 	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> 	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> 	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> 	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> 	at java.lang.Thread.run(Thread.java:619)
> java.lang.NumberFormatException: Invalid shift value in prefixCoded string (is encoded value really an INT?)
> 	at org.apache.lucene.util.NumericUtils.prefixCodedToInt(NumericUtils.java:233)
> 	at org.apache.lucene.search.FieldCache$7.parseInt(FieldCache.java:237)
> 	at org.apache.lucene.search.FieldCacheImpl$IntCache.createValue(FieldCacheImpl.java:441)
> 	at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> 	at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:414)
> 	at org.apache.lucene.search.FieldComparator$IntComparator.setNextReader(FieldComparator.java:332)
> 	at org.apache.lucene.search.TopFieldCollector$MultiComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:435)
> 	at org.apache.solr.search.DocSetDelegateCollector.setNextReader(DocSetHitCollector.java:140)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:253)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:171)
> 	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> 	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> 	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> 	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
> 	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
> 	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
> 	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> 	at java.lang.Thread.run(Thread.java:619)
> 
> A quick Google search revealed that this may have something to do with the use of Trie fields (which we have just started using) as the Exception is being thrown initially from a class called NumericUtils:
> 
> http://www.jarvana.com/jarvana/view/org/apache/lucene/lucene-core/3.0.2/lucene-core-3.0.2-sources.jar!/org/apache/lucene/util/NumericUtils.java?format=ok
> 
> I have a copy of the index open in Luke and I'm trying to find the source of the problem, but the stack trace doesn't give the name of the field, or document, and the only integer Trie field we use only has a few discrete values (1,2,3,4,5 in a field called root_priority), so I'm at a loss at where to look next. When I run the original query and the filter queries through Luke they seem to work just fine and bring back a set of documents. 
> 
> The Solr config for the trie fields looks fine to me:
> 
>        <fieldtype name="integer" class="solr.TrieField" type="integer" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
>        <fieldtype name="long" class="solr.TrieField" type="long" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
>        <fieldtype name="float" class="solr.TrieField" type="float" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
>        <fieldtype name="double" class="solr.TrieField" type="double" precisionStep="4" omitNorms="true" indexed="true" stored="true"/>
>        <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
> 
> Any ideas on where I can go from here?
> 
> Thanks
> 
> Jon