You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by Arne Claassen <ar...@mindtouch.com> on 2010/06/14 23:11:59 UTC
Are custom tokenizers not invoked when a wildcard is detected
I created a tokenizer that replaces '-', whitespace and "_" all with
'_'.
Now if i query on that field with "foo-", the parser turns it into
"foo_", but if i parse "foo-*" it stays "foo-*" and my tokenizer is
never invoked. Am I not implementing something in my tokenizer, is
this intentionial or is it a bug?
thanks,
Arne Claassen
MindTouch
San Diego, CA
http://twitter.com/sdether
Re: Are custom tokenizers not invoked when a wildcard is detected
Posted by digy digy <di...@gmail.com>.
Please see the issue https://issues.apache.org/jira/browse/LUCENENET-51
> Hoss Man commented on LUCENENET-51:
> this is known behavior in the Java library due to the fact that there is no
> "clean" way to let an analyzer process a prefix (or fuzzy or wildcard) query
> ... it is covered in the FAQ...
> http://wiki.apache.org/lucene-java/LuceneFAQ#head-133cf44dd3dff3680c96c1316a663e881eeac35a
> "Are Wildcard, Prefix, and Fuzzy queries case sensitive?"
DIGY
On Tue, Jun 15, 2010 at 12:11 AM, Arne Claassen <ar...@mindtouch.com> wrote:
> I created a tokenizer that replaces '-', whitespace and "_" all with '_'.
>
> Now if i query on that field with "foo-", the parser turns it into "foo_",
> but if i parse "foo-*" it stays "foo-*" and my tokenizer is never invoked.
> Am I not implementing something in my tokenizer, is this intentionial or is
> it a bug?
>
> thanks,
> Arne Claassen
>
> MindTouch
> San Diego, CA
> http://twitter.com/sdether
>
>