[jira] [Commented] (LUCENE-8250) Should FilteringTokenFilter handle positionLength

["Robert Muir (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436277#comment-16436277 ] 

Robert Muir commented on LUCENE-8250:
-------------------------------------

can we make a simple test case for the issue? It would help me understand it better. The last example we looked at, we tentatively decided that stopfilter was doing the right thing, so maybe we need another one.

> Should FilteringTokenFilter handle positionLength
> -------------------------------------------------
>
>                 Key: LUCENE-8250
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8250
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Jim Ferenczi
>            Priority: Major
>
> FilteringTokenFilter does not handle the position length graph attribute when removing a token from the stream. This doesn't work well with graph token stream that sets position length since removing a token from the stream can invalidate the position length set on the previous tokens. 
> This issue was first discussed in https://issues.apache.org/jira/browse/LUCENE-4065 but it has a different purpose which is why I am opening a new issue here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org