You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by John Wang <jo...@gmail.com> on 2005/08/23 19:15:17 UTC

i18n query normalization

Hi:

   We have a multi-languaged index and we need to match accented
characters with non accented characters. For example, if a document
contains: mângão, the query: mangao should match it.

    I guess I would have to build some sort of analyzer/tokenizer for this.

    I was wondering if there are tokenizers already built for lucene.


Thanks

-John

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: i18n query normalization

Posted by Ken Krugler <kk...@transpac.com>.
>    We have a multi-languaged index and we need to match accented
>characters with non accented characters. For example, if a document
>contains: mângão, the query: mangao should match it.
>
>     I guess I would have to build some sort of analyzer/tokenizer for this.
>
>     I was wondering if there are tokenizers already built for lucene.

Search the archives for a discussion about this, 
back in June I believe. I'd suggested using ICU 
to generate sort keys, and indexing those.

-- Ken
-- 
Ken Krugler
TransPac Software, Inc.
<http://www.transpac.com>
+1 530-470-9200

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: i18n query normalization

Posted by Daniel Naber <lu...@danielnaber.de>.
On Tuesday 23 August 2005 19:15, John Wang wrote:

>  We have a multi-languaged index and we need to match accented
> characters with non accented characters. For example, if a document
> contains: mângão, the query: mangao should match it.

See ISOLatin1AccentFilter in contrib/analyzers in SVN.

regards
 Daniel

-- 
http://www.danielnaber.de

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org