[jira] [Resolved] (LUCENE-9853) Use CJKWidthCharFilter as the default character normalizer for JapaneseAnalyzer instead of CJKWidthFilter

["Tomoko Uchida (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-9853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tomoko Uchida resolved LUCENE-9853.
-----------------------------------
    Fix Version/s: main (9.0)
       Resolution: Fixed

> Use CJKWidthCharFilter as the default character normalizer for JapaneseAnalyzer instead of CJKWidthFilter
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-9853
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9853
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: modules/analysis
>    Affects Versions: main (9.0)
>            Reporter: Tomoko Uchida
>            Assignee: Tomoko Uchida
>            Priority: Minor
>             Fix For: main (9.0)
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> Follow-up issue of LUCENE-9413.
> We now have CJKWidthCharFilter in analyzers-common. I believe in many situations it is recommended applying half-width/full-width character normalization before tokenization for consistency in analysis.
> The change slightly affects on the analyzer's outputs. We can provide a parameter to switch back to CJKWidthFilter for backward compatibility.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org