You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Steve Rowe (JIRA)" <ji...@apache.org> on 2013/04/22 07:57:15 UTC
[jira] [Reopened] (LUCENE-4810) Positions are incremented for each
ngram in EdgeNGramTokenFilter
[ https://issues.apache.org/jira/browse/LUCENE-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Rowe reopened LUCENE-4810:
--------------------------------
On the 4.3 vote thread, Karol Sikora reported seeing an exception when EdgeNGramTokenFilter followed the Morfologik lemmatizing filter. My summary of the new EdgeNGramTokenFilter bug is on the thread here: [http://markmail.org/message/7dkd6edz2p7fat2h].
As of the changes committed on this issue, EdgeNGramTokenFilter passes through as-is the position increment on its first emitted token for a given input token. When the input token has a position increment of 0, e.g. for synonyms, and this is the first output token, EdgeNGramTokenFilter is then guilty of producing a stream whose first token has a position increment of 0.
> Positions are incremented for each ngram in EdgeNGramTokenFilter
> ----------------------------------------------------------------
>
> Key: LUCENE-4810
> URL: https://issues.apache.org/jira/browse/LUCENE-4810
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Walter Underwood
> Assignee: Michael McCandless
> Fix For: 5.0, 4.3
>
> Attachments: LUCENE-4810.diff, LUCENE-4810.patch, LUCENE-4810.patch
>
>
> Edge ngrams should be like synonyms, with all the ngrams generated from a token having the same position as that original token. The current code increments position.
> For the text "molecular biology", the query "mol bio" should match as a phrase in neighboring positions. It does not.
> You can see this in the Analysis page in the admin UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org