You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Stephen Duncan Jr <st...@gmail.com> on 2009/08/11 20:09:22 UTC
ArrayIndexOutOfBounds on Some Searches
This is with trunk for Solr 1.4. It happened both with a build from 1 week
ago as well as with a build from today, so I'm not sure if it's something
recent, or even if it would happen on Solr 1.3 or not. Here's the stack
trace indicating that a value looping around from Integer.MAX_VALUE to
Integer.MIN_VALUE causing the exception:
-2147483648 java.lang.ArrayIndexOutOfBoundsException: -2147483648
at
org.apache.solr.search.DocSetDelegateCollector.collect(DocSetHitCollector.java:113)
at
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:301)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:245)
at org.apache.lucene.search.Searcher.search(Searcher.java:173)
at
org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:174)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at ...
Any ideas? The requests causing this are often never returning, or taking a
very long time to return, and it's causing serious problems for us in our
testing phase (including one occassion where the problem was triggered while
warming a searcher during commit, and the commit never completed).
--
Stephen Duncan Jr
www.stephenduncanjr.com
Re: ArrayIndexOutOfBounds on Some Searches
Posted by Mark Miller <ma...@gmail.com>.
Ah - right - sorry, getting lost looking at the diff I guess.
It looks to me like you are never actually returning NO_MORE_DOCS in
nextDoc based on the exception.
I can't see how that is at the moment though.
- Mark
Stephen Duncan Jr wrote:
> Doesn't that happen with the call to
>
> this.termDocs.next()
>
> ? Since I'm essentially delegating the tracking to the TermDocs object?
>
>
> On Wed, Aug 12, 2009 at 10:08 AM, Mark Miller <ma...@gmail.com> wrote:
>
>
>> First thing I see (and it may be it) is that nextDoc must also set the doc
>> - not just return it.
>>
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> Stephen Duncan Jr wrote:
>>
>>
>>> On Tue, Aug 11, 2009 at 2:09 PM, Stephen Duncan Jr <
>>> stephen.duncan@gmail.com
>>>
>>>
>>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>
>>>
>>>> This is with trunk for Solr 1.4. It happened both with a build from 1
>>>> week
>>>> ago as well as with a build from today, so I'm not sure if it's something
>>>> recent, or even if it would happen on Solr 1.3 or not. Here's the stack
>>>> trace indicating that a value looping around from Integer.MAX_VALUE to
>>>> Integer.MIN_VALUE causing the exception:
>>>>
>>>> -2147483648 java.lang.ArrayIndexOutOfBoundsException: -2147483648
>>>> at
>>>>
>>>> org.apache.solr.search.DocSetDelegateCollector.collect(DocSetHitCollector.java:113)
>>>> at
>>>> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:301)
>>>> at
>>>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:245)
>>>> at org.apache.lucene.search.Searcher.search(Searcher.java:173)
>>>> at
>>>>
>>>> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
>>>> at
>>>>
>>>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
>>>> at
>>>>
>>>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
>>>> at
>>>>
>>>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:174)
>>>> at
>>>>
>>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>>>> at
>>>>
>>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299)
>>>> at
>>>>
>>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>>>> at
>>>>
>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>>>> at ...
>>>>
>>>> Any ideas? The requests causing this are often never returning, or
>>>> taking
>>>> a very long time to return, and it's causing serious problems for us in
>>>> our
>>>> testing phase (including one occassion where the problem was triggered
>>>> while
>>>> warming a searcher during commit, and the commit never completed).
>>>>
>>>> --
>>>> Stephen Duncan Jr
>>>> www.stephenduncanjr.com
>>>>
>>>>
>>>>
>>>>
>>> I think I narrowed the problem down to being caused by a custom Lucence
>>> Query we were using, specifically my attempt to upgrade it to avoid
>>> deprecation warnings with the latest Lucene coming in from the Solr trunk
>>> build.
>>>
>>> Here's the a diff of our attempted change that seems to have lead to this
>>> issue: http://gist.github.com/166075
>>>
>>> We attempted to follow the deprecation information from Lucene. Not sure
>>> where we went wrong. If anyone has an idea, I'd love to hear it, though
>>> I'm
>>> guessing this part of the question would better belong on the Lucene
>>> mailing
>>> list (mostly wanted to make it clear that I'd figured out the source of
>>> the
>>> problem).
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
--
- Mark
http://www.lucidimagination.com
Re: ArrayIndexOutOfBounds on Some Searches
Posted by Stephen Duncan Jr <st...@gmail.com>.
Doesn't that happen with the call to
this.termDocs.next()
? Since I'm essentially delegating the tracking to the TermDocs object?
On Wed, Aug 12, 2009 at 10:08 AM, Mark Miller <ma...@gmail.com> wrote:
> First thing I see (and it may be it) is that nextDoc must also set the doc
> - not just return it.
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> Stephen Duncan Jr wrote:
>
>> On Tue, Aug 11, 2009 at 2:09 PM, Stephen Duncan Jr <
>> stephen.duncan@gmail.com
>>
>>
>>> wrote:
>>>
>>>
>>
>>
>>
>>> This is with trunk for Solr 1.4. It happened both with a build from 1
>>> week
>>> ago as well as with a build from today, so I'm not sure if it's something
>>> recent, or even if it would happen on Solr 1.3 or not. Here's the stack
>>> trace indicating that a value looping around from Integer.MAX_VALUE to
>>> Integer.MIN_VALUE causing the exception:
>>>
>>> -2147483648 java.lang.ArrayIndexOutOfBoundsException: -2147483648
>>> at
>>>
>>> org.apache.solr.search.DocSetDelegateCollector.collect(DocSetHitCollector.java:113)
>>> at
>>> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:301)
>>> at
>>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:245)
>>> at org.apache.lucene.search.Searcher.search(Searcher.java:173)
>>> at
>>>
>>> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
>>> at
>>>
>>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
>>> at
>>>
>>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
>>> at
>>>
>>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:174)
>>> at
>>>
>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>>> at
>>>
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299)
>>> at
>>>
>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>>> at
>>>
>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>>> at ...
>>>
>>> Any ideas? The requests causing this are often never returning, or
>>> taking
>>> a very long time to return, and it's causing serious problems for us in
>>> our
>>> testing phase (including one occassion where the problem was triggered
>>> while
>>> warming a searcher during commit, and the commit never completed).
>>>
>>> --
>>> Stephen Duncan Jr
>>> www.stephenduncanjr.com
>>>
>>>
>>>
>>
>>
>> I think I narrowed the problem down to being caused by a custom Lucence
>> Query we were using, specifically my attempt to upgrade it to avoid
>> deprecation warnings with the latest Lucene coming in from the Solr trunk
>> build.
>>
>> Here's the a diff of our attempted change that seems to have lead to this
>> issue: http://gist.github.com/166075
>>
>> We attempted to follow the deprecation information from Lucene. Not sure
>> where we went wrong. If anyone has an idea, I'd love to hear it, though
>> I'm
>> guessing this part of the question would better belong on the Lucene
>> mailing
>> list (mostly wanted to make it clear that I'd figured out the source of
>> the
>> problem).
>>
>>
>>
>
>
>
>
--
Stephen Duncan Jr
www.stephenduncanjr.com
Re: ArrayIndexOutOfBounds on Some Searches
Posted by Mark Miller <ma...@gmail.com>.
First thing I see (and it may be it) is that nextDoc must also set the
doc - not just return it.
--
- Mark
http://www.lucidimagination.com
Stephen Duncan Jr wrote:
> On Tue, Aug 11, 2009 at 2:09 PM, Stephen Duncan Jr <stephen.duncan@gmail.com
>
>> wrote:
>>
>
>
>> This is with trunk for Solr 1.4. It happened both with a build from 1 week
>> ago as well as with a build from today, so I'm not sure if it's something
>> recent, or even if it would happen on Solr 1.3 or not. Here's the stack
>> trace indicating that a value looping around from Integer.MAX_VALUE to
>> Integer.MIN_VALUE causing the exception:
>>
>> -2147483648 java.lang.ArrayIndexOutOfBoundsException: -2147483648
>> at
>> org.apache.solr.search.DocSetDelegateCollector.collect(DocSetHitCollector.java:113)
>> at
>> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:301)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:245)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:173)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
>> at
>> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
>> at
>> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
>> at
>> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:174)
>> at
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
>> at ...
>>
>> Any ideas? The requests causing this are often never returning, or taking
>> a very long time to return, and it's causing serious problems for us in our
>> testing phase (including one occassion where the problem was triggered while
>> warming a searcher during commit, and the commit never completed).
>>
>> --
>> Stephen Duncan Jr
>> www.stephenduncanjr.com
>>
>>
>
>
> I think I narrowed the problem down to being caused by a custom Lucence
> Query we were using, specifically my attempt to upgrade it to avoid
> deprecation warnings with the latest Lucene coming in from the Solr trunk
> build.
>
> Here's the a diff of our attempted change that seems to have lead to this
> issue: http://gist.github.com/166075
>
> We attempted to follow the deprecation information from Lucene. Not sure
> where we went wrong. If anyone has an idea, I'd love to hear it, though I'm
> guessing this part of the question would better belong on the Lucene mailing
> list (mostly wanted to make it clear that I'd figured out the source of the
> problem).
>
>
Re: ArrayIndexOutOfBounds on Some Searches
Posted by Stephen Duncan Jr <st...@gmail.com>.
On Tue, Aug 11, 2009 at 2:09 PM, Stephen Duncan Jr <stephen.duncan@gmail.com
> wrote:
> This is with trunk for Solr 1.4. It happened both with a build from 1 week
> ago as well as with a build from today, so I'm not sure if it's something
> recent, or even if it would happen on Solr 1.3 or not. Here's the stack
> trace indicating that a value looping around from Integer.MAX_VALUE to
> Integer.MIN_VALUE causing the exception:
>
> -2147483648 java.lang.ArrayIndexOutOfBoundsException: -2147483648
> at
> org.apache.solr.search.DocSetDelegateCollector.collect(DocSetHitCollector.java:113)
> at
> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:301)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:245)
> at org.apache.lucene.search.Searcher.search(Searcher.java:173)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListAndSetNC(SolrIndexSearcher.java:1101)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:880)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:174)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1299)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
> at ...
>
> Any ideas? The requests causing this are often never returning, or taking
> a very long time to return, and it's causing serious problems for us in our
> testing phase (including one occassion where the problem was triggered while
> warming a searcher during commit, and the commit never completed).
>
> --
> Stephen Duncan Jr
> www.stephenduncanjr.com
>
I think I narrowed the problem down to being caused by a custom Lucence
Query we were using, specifically my attempt to upgrade it to avoid
deprecation warnings with the latest Lucene coming in from the Solr trunk
build.
Here's the a diff of our attempted change that seems to have lead to this
issue: http://gist.github.com/166075
We attempted to follow the deprecation information from Lucene. Not sure
where we went wrong. If anyone has an idea, I'd love to hear it, though I'm
guessing this part of the question would better belong on the Lucene mailing
list (mostly wanted to make it clear that I'd figured out the source of the
problem).
--
Stephen Duncan Jr
www.stephenduncanjr.com