exception while writing to index

[nischal reddy <ni...@gmail.com>]

Hi,

I am getting an exception while indexing files, i tried debugging but
couldnt figure out the problem.

I have a custom analyzer which creates the token stream , i am indexing
around 15k files, when i start the indexing after some time i get this
exception:


java.lang.IllegalArgumentException: maxValue must be non-negative (got: -1)
    at
org.apache.lucene.util.packed.PackedInts.bitsRequired(PackedInts.java:1184)
    at
org.apache.lucene.codecs.lucene41.ForUtil.bitsRequired(ForUtil.java:243)
    at
org.apache.lucene.codecs.lucene41.ForUtil.writeBlock(ForUtil.java:164)
    at
org.apache.lucene.codecs.lucene41.Lucene41PostingsWriter.addPosition(Lucene41PostingsWriter.java:322)
    at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:534)
    at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
    at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
    at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
    at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
    at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
    at
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:478)
    at
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:615)
    at
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2748)
    at
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2897)
    at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2872)
    at
com.progress.openedge.pdt.search.index.OEIndexer.flushWriter(OEIndexer.java:597)
    at
com.progress.openedge.pdt.search.index.OEIndexer.access$8(OEIndexer.java:594)
    at
com.progress.openedge.pdt.search.index.OEIndexer$3.run(OEIndexer.java:282)
    at
com.progress.openedge.pdt.search.index.OEIndexer$IndexJob.run(OEIndexer.java:620)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)


I seem to be getting this error when i hit the max rambuffer size and
lucene is trying to flush my ram.

When i debugged i found out that there were couple of negative values in
offsetStartDeltaBuffer field of "Lucene41PostingsWriter".

am i screwing up somewhere with the offsets, i am not sending any negative
offset values.

What would i possibly be doing wrong? need your help.

Thanks in advance,
Nischal Y

Re: exception while writing to index

[Michael McCandless <lu...@mikemccandless.com>]

On Mon, Sep 16, 2013 at 3:12 PM, nischal reddy
<ni...@gmail.com> wrote:

> I seem to be getting this error when i hit the max rambuffer size and
> lucene is trying to flush my ram.
>
> When i debugged i found out that there were couple of negative values in
> offsetStartDeltaBuffer field of "Lucene41PostingsWriter".
>
> am i screwing up somewhere with the offsets, i am not sending any negative
> offset values.
>
> What would i possibly be doing wrong? need your help.

This is spooky: if you are indeed messing up offsets in some way,
IndexWriter should have detected this and thrown an exception on that
doc that did it.

Yet, you are not seeing an exception until much later, which is wrong.
 So there seems to be a bug somewhere...

Which Lucene version are you using?

Can you run with assertions enabled?  Can you whittle this down to a
small test case?

Which JVM are you using?

Mike McCandless

http://blog.mikemccandless.com

> Thanks in advance,
> Nischal Y

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org