[jira] [Updated] (LUCENE-2645) False assertion of >0 position delta in StandardPostingsWriterImpl

["KuroSaka TeruHiko (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

KuroSaka TeruHiko updated LUCENE-2645:
--------------------------------------

    Attachment: LuceneTrunkAssertErrorReproducer.java

This is preventing one of unit tests for our Tokenizer/Filter product to fail.  Although it looks as though the unit test is failing, actually it is an assertion error from this assertion.  
We have analyzed the Token sequence that causes this assertion to fail, but they look valid.
The attached program can reproduce this false assertion failure. Please give it higher priority and remove this assertion.
{noformat}
$ javac -cp lucene-core-4.0-SNAPSHOT.jar LuceneTrunkAssertErrorReproducer.java
$ java -ea -cp .:lucene-core-4.0-SNAPSHOT.jar LuceneTrunkAssertErrorReproducer
Exception in thread "main" java.lang.AssertionError: position=1 lastPosition=1
	at org.apache.lucene.index.codecs.standard.StandardPostingsWriter.addPosition(StandardPostingsWriter.java:197)
	at org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:385)
	at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:88)
	at org.apache.lucene.index.TermsHash.flush(TermsHash.java:117)
	at org.apache.lucene.index.DocInverter.flush(DocInverter.java:80)
	at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:75)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:457)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:421)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:548)
	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:2776)
	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2753)
	at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2619)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2692)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2674)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2658)
	at LuceneTrunkAssertErrorReproducer.main(LuceneTrunkAssertErrorReproducer.java:39)
{noformat}

> False assertion of >0 position delta in StandardPostingsWriterImpl
> ------------------------------------------------------------------
>
>                 Key: LUCENE-2645
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2645
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: David Smiley
>            Priority: Minor
>         Attachments: LuceneTrunkAssertErrorReproducer.java
>
>
> StandardPostingsWriterImpl line 159 is:
> {code:java}
>     assert delta > 0 || position == 0 || position == -1: "position=" + position + " lastPosition=" + lastPosition;            // not quite right (if pos=0 is repeated twice we don't catch it)
> {code}
> I enable assertions when I run my unit tests and I've found this assertion to fail when delta is 0 which occurs when the same position value is sent in twice in arrow.  Once I added RemoveDuplicatesTokenFilter, this problem went away.  Should I really be forced to add this filter?  I think delta >= 0 would be a better assertion.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org