You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2018/04/25 15:08:00 UTC
[jira] [Commented] (LUCENE-8273) Add a ConditionalTokenFilter
[ https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452428#comment-16452428 ]
Robert Muir commented on LUCENE-8273:
-------------------------------------
{quote}
This latter one is a bit clunky, as this TokenFilter won't work with filters that consume more than one token at a time - eg ShingleFilter or SynonymGraphFilter. At the moment I have a blacklist, but there may be a better way of isolating that - preferably one that throws errors when you build the TokenStream. Speak up if you have any suggestions.
{quote}
I don't understand the problem yet, but I will try to fight with it. You mean any filter that uses captureState?
> Add a ConditionalTokenFilter
> ----------------------------
>
> Key: LUCENE-8273
> URL: https://issues.apache.org/jira/browse/LUCENE-8273
> Project: Lucene - Core
> Issue Type: New Feature
> Reporter: Alan Woodward
> Priority: Major
> Attachments: LUCENE-8273.patch, LUCENE-8273.patch
>
>
> Spinoff of LUCENE-8265. It would be useful to be able to wrap a TokenFilter in such a way that it could optionally be bypassed based on the current state of the TokenStream. This could be used to, for example, only apply WordDelimiterFilter to terms that contain hyphens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org