You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Grant Ingersoll (Assigned) (JIRA)" <ji...@apache.org> on 2011/12/20 03:31:30 UTC
[jira] [Assigned] (LUCENE-1224) NGramTokenFilter creates bad
TokenStream
[ https://issues.apache.org/jira/browse/LUCENE-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll reassigned LUCENE-1224:
---------------------------------------
Assignee: (was: Grant Ingersoll)
> NGramTokenFilter creates bad TokenStream
> ----------------------------------------
>
> Key: LUCENE-1224
> URL: https://issues.apache.org/jira/browse/LUCENE-1224
> Project: Lucene - Java
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Hiroaki Kawai
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-1224.patch, NGramTokenFilter.patch, NGramTokenFilter.patch
>
>
> With current trunk NGramTokenFilter(min=2,max=4) , I index "abcdef" string into an index, but I can't query it with "abc". If I query with "ab", I can get a hit result.
> The reason is that the NGramTokenFilter generates badly ordered TokenStream. Query is based on the Token order in the TokenStream, that how stemming or phrase should be anlayzed is based on the order (Token.positionIncrement).
> With current filter, query string "abc" is tokenized to : ab bc abc
> meaning "query a string that has ab bc abc in this order".
> Expected filter will generate : ab abc(positionIncrement=0) bc
> meaning "query a string that has (ab|abc) bc in this order"
> I'd like to submit a patch for this issue. :-)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org