You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Andreas Hauser (JIRA)" <ji...@apache.org> on 2009/02/20 16:26:01 UTC
[jira] Updated: (LUCENE-1545) Standard analyzer does not correctly
tokenize combining character U+0364 COMBINING LATIN SMALL LETTRE E
[ https://issues.apache.org/jira/browse/LUCENE-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andreas Hauser updated LUCENE-1545:
-----------------------------------
Attachment: AnalyzerTest.java
$ java -Dfile.encoding=UTF-8 -cp lib/lucene-core-2.4-20090219.021329-1.jar:. AnalyzerTest
(mo,0,2,type=<ALPHANUM>)
(chte,3,7,type=<ALPHANUM>)
(m,8,9,type=<ALPHANUM>)
(mo,10,12,type=<ALPHANUM>)
(chte,13,17,type=<ALPHANUM>)
$locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE=de_DE.UTF-8
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES=de_DE.UTF-8
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=
> Standard analyzer does not correctly tokenize combining character U+0364 COMBINING LATIN SMALL LETTRE E
> -------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1545
> URL: https://issues.apache.org/jira/browse/LUCENE-1545
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 2.4
> Environment: Linux x86_64, Sun Java 1.6
> Reporter: Andreas Hauser
> Fix For: 2.9
>
> Attachments: AnalyzerTest.java
>
>
> Standard analyzer does not correctly tokenize combining character U+0364 COMBINING LATIN SMALL LETTRE E.
> The word "moͤchte" is incorrectly tokenized into "mo" "chte", the combining character is lost.
> Expected result is only on token "moͤchte".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org