You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opennlp.apache.org by kr...@apache.org on 2026/08/06 02:11:36 UTC
(opennlp) branch OPENNLP-1894-lattice-cjk updated (6716542d3 -> 7a98076c8)
This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1894-lattice-cjk
in repository https://gitbox.apache.org/repos/asf/opennlp.git
discard 6716542d3 OPENNLP-1894: Address review: complete javadoc, hoist constants, and fold fixture duplication
discard ceca48832 OPENNLP-1894: Trim parsed lines as Unicode whitespace and document the tokenizer overrides
discard e689f3f8d OPENNLP-1894: Apply the review-convention pass and drop unreferenced lexicon accessors
discard 7536cf13b OPENNLP-1894: Document lattice CJK tokenization with a mirror-tested example
discard 19c58d265 OPENNLP-1894: Chain lattice nodes intrusively instead of allocating per-position lists
discard 075b343c8 OPENNLP-1894: Hold the lexicon in a double-array trie with frequency-recoded labels
discard d8870784b OPENNLP-1894: Precompute category runs, unbox the trie, and reject inexpressible dictionary values
discard 7a4ee6e70 OPENNLP-1894: Categorize by code point, keep unknown candidates inside their category run, and validate context ids at load
discard 41cb53573 OPENNLP-1894: Document dictionary acquisition with a checksum-verifying download helper
discard 264c5feab OPENNLP-1894: Keep lattice test sources ASCII-only via Unicode escapes, add an EUC-JP loading example
discard b96496662 OPENNLP-1894: Usage example and edge-case tests for the lattice and unigram segmenters, corrected javadoc
discard 376e419b6 OPENNLP-1894: Frequency-driven segmentation over user-supplied lexicons
discard 67d7643c6 OPENNLP-1894: Character trie for lattice prefix search
discard b892c9d48 OPENNLP-1894: Lattice segmentation over user-supplied mecab-format dictionaries
add 014c0182a OPENNLP-1899: Harden SymSpell model deserialization against oversized count fields (#1197)
add 757716610 OPENNLP-1907 - Bump io.github.classgraph:classgraph from 4.8.184 to 4.8.186 (#1201)
add b4226529a Bump actions/setup-java from 5.6.0 to 5.7.0 (#1199)
add ed51a6a6f OPENNLP-1908 - Bump onnxruntime.version from 1.27.0 to 1.28.0 (#1203)
add 4e77bb67b Minor: Regenerated NOTICE File for ed51a6a6f974fd0bf3be900458125fb57b011c04 (#1209)
add fc9824a97 OPENNLP-1900: Document and publish the OpenNLP security model (#1198)
add f733fcffc OPENNLP-1894: Lattice segmentation over user-supplied mecab-format dictionaries
add 781c22214 OPENNLP-1894: Character trie for lattice prefix search
add f1fada5c7 OPENNLP-1894: Frequency-driven segmentation over user-supplied lexicons
add 05737234a OPENNLP-1894: Usage example and edge-case tests for the lattice and unigram segmenters, corrected javadoc
add c7fadc913 OPENNLP-1894: Keep lattice test sources ASCII-only via Unicode escapes, add an EUC-JP loading example
add a256b90a6 OPENNLP-1894: Document dictionary acquisition with a checksum-verifying download helper
add 559fdd072 OPENNLP-1894: Categorize by code point, keep unknown candidates inside their category run, and validate context ids at load
add 04d380ce7 OPENNLP-1894: Precompute category runs, unbox the trie, and reject inexpressible dictionary values
add 625452d92 OPENNLP-1894: Hold the lexicon in a double-array trie with frequency-recoded labels
add cac8a31ea OPENNLP-1894: Chain lattice nodes intrusively instead of allocating per-position lists
add f28c02fc6 OPENNLP-1894: Document lattice CJK tokenization with a mirror-tested example
add 701f1db57 OPENNLP-1894: Apply the review-convention pass and drop unreferenced lexicon accessors
add fdc9f93a4 OPENNLP-1894: Trim parsed lines as Unicode whitespace and document the tokenizer overrides
add 7a98076c8 OPENNLP-1894: Address review: complete javadoc, hoist constants, and fold fixture duplication
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6716542d3)
\
N -- N -- N refs/heads/OPENNLP-1894-lattice-cjk (7a98076c8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/license.yml | 2 +-
.github/workflows/maven.yml | 8 +-
.github/workflows/publish-snapshots.yml | 2 +-
NOTICE | 6 +-
README.md | 9 +
SECURITY.md | 244 +++++++++++++++++++++
.../tools/ml/model/AbstractModelReader.java | 5 +-
opennlp-distr/src/main/readme/NOTICE | 6 +-
.../dictionary/SymSpellModelSerializer.java | 11 +-
.../SymSpellModelSerializerLimitsTest.java | 109 +++++++++
pom.xml | 4 +-
11 files changed, 387 insertions(+), 19 deletions(-)
create mode 100644 SECURITY.md
create mode 100644 opennlp-extensions/opennlp-spellcheck/src/test/java/opennlp/spellcheck/dictionary/SymSpellModelSerializerLimitsTest.java