You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pablo Mercado <pa...@sbnation.com> on 2010/02/26 15:59:50 UTC
SEVERE: java.lang.NumberFormatException: For input string: "104708<"
Hello,
Solr is raising the following exception when processing queries that
sort on integer attribute. The same queries and sorts have been
running fine in production for almost a year now. If I run the query
without the sort on the integer attribute, the query runs fine. If I
run a query that would return 0 results, but still has a sort
parameter the exception is raised. The stack trace is the same no
matter what the query.
I need help troubleshooting this issue. Any clues, or suggested
approaches would be helpful. Thank you in advance!.
The stack trace is as follows:
SEVERE: java.lang.NumberFormatException: For input string: "104708<"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:456)
at java.lang.Integer.parseInt(Integer.java:497)
at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
at org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:291)
at org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:188)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)
at org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
Our solr info is:
Solr Specification Version: 1.3.0
Solr Implementation Version: 1.3.0 694707 - grantingersoll - 2008-09-12 11:06:47
Lucene Specification Version: 2.4-dev
Lucene Implementation Version: 2.4-dev 691741 - 2008-09-03 15:25:16
Re: SEVERE: java.lang.NumberFormatException: For input string:
"104708<"
Posted by Pablo Mercado <pa...@sbnation.com>.
A big thanks to Yonik and Mark. Using the raw term query I was able
to find the range(!) of documents that had bad integer field values.
Deleting those documents, committing and optimizing cleared up the
issue.
Still not sure how the bad values were inserted in the first place,
but that is another task. Thanks again for being so helpful.
On Fri, Feb 26, 2010 at 11:29, Yonik Seeley <yo...@lucidimagination.com> wrote:
> On Fri, Feb 26, 2010 at 10:59 AM, Mark Miller <ma...@gmail.com> wrote:
>> You have to find the document with the bad value somehow.
>>
>> In the past I have used Luke to help with this.
>>
>> Then you need to delete the document.
>
> You can also find the document with a raw term query.
>
> q={!raw f=myfield}104708<
>
> -Yonik
> http://www.lucidimagination.com
>
Re: SEVERE: java.lang.NumberFormatException: For input string:
"104708<"
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Feb 26, 2010 at 10:59 AM, Mark Miller <ma...@gmail.com> wrote:
> You have to find the document with the bad value somehow.
>
> In the past I have used Luke to help with this.
>
> Then you need to delete the document.
You can also find the document with a raw term query.
q={!raw f=myfield}104708<
-Yonik
http://www.lucidimagination.com
Re: SEVERE: java.lang.NumberFormatException: For input string: "104708<"
Posted by Mark Miller <ma...@gmail.com>.
You have to find the document with the bad value somehow.
In the past I have used Luke to help with this.
Then you need to delete the document.
Finally, you have to get the deleted document out of the index through a
merge (else the bad term will still be loaded by the FieldCache) -
easiest way is to do this is an optimize.
--
- Mark
http://www.lucidimagination.com
On 02/26/2010 10:49 AM, Pablo Mercado wrote:
> Thank you for taking the time to look at my issue and respond.
>
> Do you have any suggestions for purging the document with this field
> from the index? Would that even help?
>
> I do not know which document has the corrupt value, and searching for
> the document with something like
>
> pk_i:104708<
>
> does return a document with that value.
>
> (pk_i is the integer field that we try to sort on and that,
> presumably, has a non-integer value stored for some document)
>
>
>
>
> On Fri, Feb 26, 2010 at 10:26, Yonik Seeley<yo...@lucidimagination.com> wrote:
>
>> One of your field values isn't a valid integer, it's "104708<"
>> You're probably using the straight integer type in 1.3, which is meant
>> for back compat with existing lucene indexes and currently doesn't do
>> validation on it's input.
>>
>> For Solr 1.4, "int" is a new field type (example schema maps it to
>> TrieIntField) that does do validation at index time, and is just as
>> efficient for sorting.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>>
>> On Fri, Feb 26, 2010 at 9:59 AM, Pablo Mercado<pa...@sbnation.com> wrote:
>>
>>> Hello,
>>>
>>> Solr is raising the following exception when processing queries that
>>> sort on integer attribute. The same queries and sorts have been
>>> running fine in production for almost a year now. If I run the query
>>> without the sort on the integer attribute, the query runs fine. If I
>>> run a query that would return 0 results, but still has a sort
>>> parameter the exception is raised. The stack trace is the same no
>>> matter what the query.
>>>
>>> I need help troubleshooting this issue. Any clues, or suggested
>>> approaches would be helpful. Thank you in advance!.
>>>
>>> The stack trace is as follows:
>>>
>>> SEVERE: java.lang.NumberFormatException: For input string: "104708<"
>>> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>>> at java.lang.Integer.parseInt(Integer.java:456)
>>> at java.lang.Integer.parseInt(Integer.java:497)
>>> at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
>>> at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
>>> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>>> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
>>> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
>>> at org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:291)
>>> at org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:188)
>>> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>>> at org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)
>>> at org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
>>> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)
>>> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
>>> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
>>> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
>>> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169)
>>> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
>>> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
>>> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
>>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
>>> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>>> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
>>> at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>>> at org.mortbay.jetty.Server.handle(Server.java:285)
>>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
>>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
>>> at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
>>> at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>>>
>>>
>>> Our solr info is:
>>> Solr Specification Version: 1.3.0
>>> Solr Implementation Version: 1.3.0 694707 - grantingersoll - 2008-09-12 11:06:47
>>> Lucene Specification Version: 2.4-dev
>>> Lucene Implementation Version: 2.4-dev 691741 - 2008-09-03 15:25:16
>>>
>>>
>>
Re: SEVERE: java.lang.NumberFormatException: For input string:
"104708<"
Posted by Pablo Mercado <pa...@sbnation.com>.
Thank you for taking the time to look at my issue and respond.
Do you have any suggestions for purging the document with this field
from the index? Would that even help?
I do not know which document has the corrupt value, and searching for
the document with something like
pk_i:104708<
does return a document with that value.
(pk_i is the integer field that we try to sort on and that,
presumably, has a non-integer value stored for some document)
On Fri, Feb 26, 2010 at 10:26, Yonik Seeley <yo...@lucidimagination.com> wrote:
> One of your field values isn't a valid integer, it's "104708<"
> You're probably using the straight integer type in 1.3, which is meant
> for back compat with existing lucene indexes and currently doesn't do
> validation on it's input.
>
> For Solr 1.4, "int" is a new field type (example schema maps it to
> TrieIntField) that does do validation at index time, and is just as
> efficient for sorting.
>
> -Yonik
> http://www.lucidimagination.com
>
>
> On Fri, Feb 26, 2010 at 9:59 AM, Pablo Mercado <pa...@sbnation.com> wrote:
>> Hello,
>>
>> Solr is raising the following exception when processing queries that
>> sort on integer attribute. The same queries and sorts have been
>> running fine in production for almost a year now. If I run the query
>> without the sort on the integer attribute, the query runs fine. If I
>> run a query that would return 0 results, but still has a sort
>> parameter the exception is raised. The stack trace is the same no
>> matter what the query.
>>
>> I need help troubleshooting this issue. Any clues, or suggested
>> approaches would be helpful. Thank you in advance!.
>>
>> The stack trace is as follows:
>>
>> SEVERE: java.lang.NumberFormatException: For input string: "104708<"
>> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>> at java.lang.Integer.parseInt(Integer.java:456)
>> at java.lang.Integer.parseInt(Integer.java:497)
>> at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
>> at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
>> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
>> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
>> at org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:291)
>> at org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:188)
>> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>> at org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)
>> at org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
>> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)
>> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
>> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
>> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
>> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169)
>> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
>> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
>> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
>> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
>> at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>> at org.mortbay.jetty.Server.handle(Server.java:285)
>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
>> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
>> at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
>> at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>>
>>
>> Our solr info is:
>> Solr Specification Version: 1.3.0
>> Solr Implementation Version: 1.3.0 694707 - grantingersoll - 2008-09-12 11:06:47
>> Lucene Specification Version: 2.4-dev
>> Lucene Implementation Version: 2.4-dev 691741 - 2008-09-03 15:25:16
>>
>
Re: SEVERE: java.lang.NumberFormatException: For input string:
"104708<"
Posted by Yonik Seeley <yo...@lucidimagination.com>.
One of your field values isn't a valid integer, it's "104708<"
You're probably using the straight integer type in 1.3, which is meant
for back compat with existing lucene indexes and currently doesn't do
validation on it's input.
For Solr 1.4, "int" is a new field type (example schema maps it to
TrieIntField) that does do validation at index time, and is just as
efficient for sorting.
-Yonik
http://www.lucidimagination.com
On Fri, Feb 26, 2010 at 9:59 AM, Pablo Mercado <pa...@sbnation.com> wrote:
> Hello,
>
> Solr is raising the following exception when processing queries that
> sort on integer attribute. The same queries and sorts have been
> running fine in production for almost a year now. If I run the query
> without the sort on the integer attribute, the query runs fine. If I
> run a query that would return 0 results, but still has a sort
> parameter the exception is raised. The stack trace is the same no
> matter what the query.
>
> I need help troubleshooting this issue. Any clues, or suggested
> approaches would be helpful. Thank you in advance!.
>
> The stack trace is as follows:
>
> SEVERE: java.lang.NumberFormatException: For input string: "104708<"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:456)
> at java.lang.Integer.parseInt(Integer.java:497)
> at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
> at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
> at org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:291)
> at org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:188)
> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
> at org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:168)
> at org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:907)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:169)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204)
> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
> at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
> at org.mortbay.jetty.Server.handle(Server.java:285)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
> at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
> at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
>
>
> Our solr info is:
> Solr Specification Version: 1.3.0
> Solr Implementation Version: 1.3.0 694707 - grantingersoll - 2008-09-12 11:06:47
> Lucene Specification Version: 2.4-dev
> Lucene Implementation Version: 2.4-dev 691741 - 2008-09-03 15:25:16
>