You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Albert MacSweeny <al...@profium.com> on 2020/09/23 10:31:23 UTC

ArrayIndexOutOfBoundsException errors querying MemoryIndex

Hi all,

After upgrading projects from Lucene 3.0.0 to 8.5.2, I'm getting an error appearing sometimes during a load test.

Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 128
	at org.apache.lucene.util.BytesRefHash$1.get(BytesRefHash.java:181) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.StringMSBRadixSorter.byteAt(StringMSBRadixSorter.java:30) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.MSBRadixSorter.getBucket(MSBRadixSorter.java:199) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.MSBRadixSorter.buildHistogram(MSBRadixSorter.java:249) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.MSBRadixSorter.computeCommonPrefixLengthAndBuildHistogram(MSBRadixSorter.java:236) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.MSBRadixSorter.radixSort(MSBRadixSorter.java:148) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:128) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.MSBRadixSorter.sort(MSBRadixSorter.java:121) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.util.BytesRefHash.sort(BytesRefHash.java:185) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.index.memory.MemoryIndex$Info.sortTerms(MemoryIndex.java:858) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
	at org.apache.lucene.index.memory.MemoryIndex$MemoryIndexReader$MemoryTermsEnum.<init>(MemoryIndex.java:1342) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
	at org.apache.lucene.index.memory.MemoryIndex$MemoryIndexReader$MemoryFields$1.iterator(MemoryIndex.java:1279) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]
	at org.apache.lucene.index.TermStates.loadTermsEnum(TermStates.java:123) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.index.TermStates.build(TermStates.java:109) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:199) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.search.BooleanWeight.<init>(BooleanWeight.java:63) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:231) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445) ~[lucene-core-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:05]
	at org.apache.lucene.index.memory.MemoryIndex.search(MemoryIndex.java:690) ~[lucene-memory-8.5.2.jar:8.5.2 384dadd9141cec3f848d8c416315dc2384749814 - mdrob - 2020-05-19 13:13:43]

It's not easily reproducible as it doesn't happen for the same queries or documents each time the load test is repeated.

The queries are generally quite simple field:value ones e.g. "doc_lead:mayenne"

The queries are evaluated in concurrent threads and I noticed the freeze() method had been added since 3.0.0 but even calling this between populating the index and evaluating the queries did not help.

Thanks in advance for any help,
Albert

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org