You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Trejkaz <tr...@trypticon.org> on 2017/01/16 22:27:27 UTC
Weird corruption symptom, not making sense
I have this thing where our UninvertingReader is getting an
ArrayIndexOutOfBoundsException in production. I'm sure the index is
corrupt, but I tried investigating the code and it still seems a bit
odd.
Caused by: java.lang.ArrayIndexOutOfBoundsException: -48116
at org.apache.lucene.util.PagedBytes$Reader.fill(PagedBytes.java:118)
at OurFieldCacheImpl.BinaryDocValuesImpl.get(SourceFile:844)
In BinaryDocValuesImpl :
return new BinaryDocValues()
{
@Override
public BytesRef get(int docID)
{
int pointer = (int) docToOffset.get(docID);
if (pointer == 0) {
term.length = 0;
} else {
bytes.fill(term, pointer);
}
return term;
}
};
So "pointer" is the negative value presumably? Implying that somehow a
negative value got into the docToOffset mappings.
The value it's putting in comes from:
long pointer = bytes.copyUsingLengthPrefix(term);
postingsEnum = termsEnum.postings(postingsEnum, PostingsEnum.NONE);
while (true) {
int docID = postingsEnum.nextDoc();
if (docID == DocIdSetIterator.NO_MORE_DOCS) {
break;
}
docToOffset.set(docID, pointer);
}
So it seems like bytes.copyUsingLengthPrefix can return a negative
value? But I looked in PagedBytes and couldn't see an obvious way to
get a negative value.
Is it possible some kind of overflow is happening here?
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Weird corruption symptom, not making sense
Posted by Trejkaz <tr...@trypticon.org>.
On Tue, Jan 17, 2017 at 9:31 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> ...or a JVM bug. We have seen those around PagedBytes in the past. What Java version?
Actually I just did a bit more digging and found this:
https://issues.apache.org/jira/browse/LUCENE-6948
And of course, even though we're on a later version, our copy of
UninvertingReader was cloned from before that fix. *facepalm*
TX
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Weird corruption symptom, not making sense
Posted by Uwe Schindler <uw...@thetaphi.de>.
...or a JVM bug. We have seen those around PagedBytes in the past. What Java version?
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Trejkaz [mailto:trejkaz@trypticon.org]
> Sent: Monday, January 16, 2017 11:27 PM
> To: Lucene Users Mailing List <ja...@lucene.apache.org>
> Subject: Weird corruption symptom, not making sense
>
> I have this thing where our UninvertingReader is getting an
> ArrayIndexOutOfBoundsException in production. I'm sure the index is
> corrupt, but I tried investigating the code and it still seems a bit
> odd.
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -48116
> at org.apache.lucene.util.PagedBytes$Reader.fill(PagedBytes.java:118)
> at OurFieldCacheImpl.BinaryDocValuesImpl.get(SourceFile:844)
>
> In BinaryDocValuesImpl :
>
> return new BinaryDocValues()
> {
> @Override
> public BytesRef get(int docID)
> {
> int pointer = (int) docToOffset.get(docID);
> if (pointer == 0) {
> term.length = 0;
> } else {
> bytes.fill(term, pointer);
> }
> return term;
> }
> };
>
> So "pointer" is the negative value presumably? Implying that somehow a
> negative value got into the docToOffset mappings.
> The value it's putting in comes from:
>
> long pointer = bytes.copyUsingLengthPrefix(term);
> postingsEnum = termsEnum.postings(postingsEnum,
> PostingsEnum.NONE);
> while (true) {
> int docID = postingsEnum.nextDoc();
> if (docID == DocIdSetIterator.NO_MORE_DOCS) {
> break;
> }
> docToOffset.set(docID, pointer);
> }
>
> So it seems like bytes.copyUsingLengthPrefix can return a negative
> value? But I looked in PagedBytes and couldn't see an obvious way to
> get a negative value.
>
> Is it possible some kind of overflow is happening here?
>
> TX
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org