You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "Richard Zowalla (Jira)" <ji...@apache.org> on 2026/09/17 17:06:00 UTC

[jira] [Created] (OPENNLP-1964) Remove the Japanese add-on

Richard Zowalla created OPENNLP-1964:
----------------------------------------

             Summary: Remove the Japanese add-on
                 Key: OPENNLP-1964
                 URL: https://issues.apache.org/jira/browse/OPENNLP-1964
             Project: OpenNLP
          Issue Type: Task
          Components: Name Finder
            Reporter: Richard Zowalla
            Assignee: Richard Zowalla


Remove the {{japanese}} module and its leftover Ant {{build.xml}} from opennlp-addons. Most of its feature generators are copies of the ones in {{opennlp.tools.util.featuregen}}.

*Replacement:* core now handles Japanese itself:
* {{StringPattern}} detects Han, Hiragana and Katakana, and {{FeatureGeneratorUtil}} emits the {{jah}}/{{jak}} token classes.
* {{LatticeTokenizer}} with a MeCab dictionary (OPENNLP-1894) segments the text and returns each morpheme with its features, which covers the {{word/POS}} tokens {{AuxiliaryInfo*}} expected.

The module also put classes in {{opennlp.tools.namefind}} and {{opennlp.tools.util.featuregen}}, packages that {{opennlp-runtime}} already owns.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)