[jira] [Created] (LUCENE-5401) Field.StringTokenStream#end() does not call super.end()

["Michael Busch (JIRA)" <ji...@apache.org>]

Michael Busch created LUCENE-5401:
-------------------------------------

             Summary: Field.StringTokenStream#end() does not call super.end()
                 Key: LUCENE-5401
                 URL: https://issues.apache.org/jira/browse/LUCENE-5401
             Project: Lucene - Core
          Issue Type: Bug
          Components: core/other
    Affects Versions: 4.6
            Reporter: Michael Busch
            Assignee: Michael Busch
            Priority: Minor
             Fix For: 4.6.1


Field.StringTokenStream#end() currently does not call super.end(). This prevents resetting the PositionIncrementAttribute to 0 in end(), which can lead to wrong positions in the index under certain conditions.

I added a test to TestDocument which indexes two Fields with the same name, String values, indexed=true, tokenized=false and IndexOptions.DOCS_AND_FREQS_AND_POSITIONS. Without the fix the test fails. The first token gets the correct position 0, but the second token gets position 2 instead of 1. The reason is that in DocInverterPerField line 176 (which is just after the call to end()) we increment the position a second time, because end() didn't reset the increment to 0.

All tests pass with the fix.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org