You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by John Wang <jo...@gmail.com> on 2005/08/23 19:15:17 UTC
i18n query normalization
Hi:
We have a multi-languaged index and we need to match accented
characters with non accented characters. For example, if a document
contains: mângão, the query: mangao should match it.
I guess I would have to build some sort of analyzer/tokenizer for this.
I was wondering if there are tokenizers already built for lucene.
Thanks
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: i18n query normalization
Posted by Ken Krugler <kk...@transpac.com>.
> We have a multi-languaged index and we need to match accented
>characters with non accented characters. For example, if a document
>contains: mângão, the query: mangao should match it.
>
> I guess I would have to build some sort of analyzer/tokenizer for this.
>
> I was wondering if there are tokenizers already built for lucene.
Search the archives for a discussion about this,
back in June I believe. I'd suggested using ICU
to generate sort keys, and indexing those.
-- Ken
--
Ken Krugler
TransPac Software, Inc.
<http://www.transpac.com>
+1 530-470-9200
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: i18n query normalization
Posted by Daniel Naber <lu...@danielnaber.de>.
On Tuesday 23 August 2005 19:15, John Wang wrote:
> We have a multi-languaged index and we need to match accented
> characters with non accented characters. For example, if a document
> contains: mângão, the query: mangao should match it.
See ISOLatin1AccentFilter in contrib/analyzers in SVN.
regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org