You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Soujanya (JIRA)" <ji...@apache.org> on 2010/12/07 10:20:09 UTC
[jira] Created: (LUCENE-2806) ArrayIndexOutOfBoundsException thrown
from TermScorer.score while doing a regular index search.
ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
-----------------------------------------------------------------------------------------------
Key: LUCENE-2806
URL: https://issues.apache.org/jira/browse/LUCENE-2806
Project: Lucene - Java
Issue Type: Bug
Components: Search
Affects Versions: 3.0.3
Environment: Ubuntu linux
Reporter: Soujanya
java.lang.ArrayIndexOutOfBoundsException: 51064
org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
org.apache.lucene.search.Searcher.search(Searcher.java:67)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2806) ArrayIndexOutOfBoundsException
thrown from TermScorer.score while doing a regular index search.
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968643#action_12968643 ]
Shai Erera commented on LUCENE-2806:
------------------------------------
Can you please post a patch with a short test case that reproduces the bug? If not, can you please describe the environment: index size, num docs, query used etc.
>From TermScorer's code, the AIOOBE can be thrown if the 'freq' is larger than the scoreCache array size, however there is an if validating that freq is smaller than the array size, so I don't get it ...
> ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
> -----------------------------------------------------------------------------------------------
>
> Key: LUCENE-2806
> URL: https://issues.apache.org/jira/browse/LUCENE-2806
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Affects Versions: 3.0.3
> Environment: Ubuntu linux
> Reporter: Soujanya
>
> java.lang.ArrayIndexOutOfBoundsException: 51064
> org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> org.apache.lucene.search.Searcher.search(Searcher.java:67)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2806) ArrayIndexOutOfBoundsException
thrown from TermScorer.score while doing a regular index search.
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968654#action_12968654 ]
Shai Erera commented on LUCENE-2806:
------------------------------------
Ok -- I looked at 3x source code, but I see you marked the issue as 3.0.3 and that TermScorer's lines are different between 3x and 3.0.3. The offending line looks like:
{code}
return norms == null ? raw : raw * SIM_NORM_DECODER[norms[doc] & 0xFF]; // normalize for field
{code}
And it can be thrown by either invalid access to the 'norms' array, or the SIM_NORM_DECODER array. Are you using a custom Similarity maybe? I think it's less likely that the invalid access is caused from the 'norms' array, as it is allocated in the size of the index (one norm per document).
Are you able to rebuild the index? If so, I'd suggest to rebuild it and add all content as ANALYZED_NO_NORMS to disable norms. If the AIOOBE still happens, then we know it's from the SIM_NORM_DECODER, otherwise it's from the norms[] array.
Also, does it happen in other queries too? Maybe to remove some noise, remove the *field:word** part and only keep the type:document part? Trying to minimize the query down to few select terms to ease on the debugging.
Can you perhaps post the index here?
> ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
> -----------------------------------------------------------------------------------------------
>
> Key: LUCENE-2806
> URL: https://issues.apache.org/jira/browse/LUCENE-2806
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Affects Versions: 3.0.3
> Environment: Ubuntu linux
> Reporter: Soujanya
>
> java.lang.ArrayIndexOutOfBoundsException: 51064
> org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> org.apache.lucene.search.Searcher.search(Searcher.java:67)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2806) ArrayIndexOutOfBoundsException
thrown from TermScorer.score while doing a regular index search.
Posted by "Soujanya (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968647#action_12968647 ]
Soujanya commented on LUCENE-2806:
----------------------------------
I used SimpleAnalyzer to build the index. There are totally 240,000+
documents (multiple fielded documents).
I use SimpleAnalyzer to generate the query. The query has two fields to
search like this: field1:searchstring* AND field2:searchstring2. The first
one has a regular expression '*' at the end of the query string. To match
obtain matches when the query string is not entered completely. The query
string is a single word appended with a '*', say like 'field1:glamour* AND
type:document'.
I run this query on the index and it throws this error.
Could you please tell me what 'freq' is? Is the frequency of a word or is it
an array?
Thanks,
Soujanya
> ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
> -----------------------------------------------------------------------------------------------
>
> Key: LUCENE-2806
> URL: https://issues.apache.org/jira/browse/LUCENE-2806
> Project: Lucene - Java
> Issue Type: Bug
> Components: Search
> Affects Versions: 3.0.3
> Environment: Ubuntu linux
> Reporter: Soujanya
>
> java.lang.ArrayIndexOutOfBoundsException: 51064
> org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> org.apache.lucene.search.Searcher.search(Searcher.java:67)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org