You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@opennlp.apache.org by kr...@apache.org on 2026/07/24 18:53:21 UTC

(opennlp) branch OPENNLP-1894-lattice-cjk updated (b5723b45d -> 65579e9cd)

This is an automated email from the ASF dual-hosted git repository.

krickert pushed a change to branch OPENNLP-1894-lattice-cjk
in repository https://gitbox.apache.org/repos/asf/opennlp.git


    from b5723b45d OPENNLP-1894: Apply the review-convention pass and drop unreferenced lexicon accessors
     add 65579e9cd OPENNLP-1894: Trim parsed lines as Unicode whitespace and document the tokenizer overrides

No new revisions were added by this update.

Summary of changes:
 .../opennlp/tools/tokenize/lattice/LatticeTokenizer.java | 16 +++++++++++++++-
 .../opennlp/tools/tokenize/lattice/MecabDictionary.java  |  8 ++++----
 .../opennlp/tools/tokenize/lattice/UnigramSegmenter.java | 16 +++++++++++++++-
 .../tools/tokenize/lattice/UnigramSegmenterTest.java     | 15 +++++++++++++++
 4 files changed, 49 insertions(+), 6 deletions(-)