You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@opennlp.apache.org by kr...@apache.org on 2026/07/17 12:15:20 UTC

(opennlp) branch sentencepiece updated (7f20e7663 -> 6c19162e8)

This is an automated email from the ASF dual-hosted git repository.

krickert pushed a change to branch sentencepiece
in repository https://gitbox.apache.org/repos/asf/opennlp.git


    from 7f20e7663 OPENNLP-1885: Make the tokenizer graph serializable with computed UIDs, name the format constants, document every helper
     add 6c19162e8 OPENNLP-1885: Guard tokenizer deserialization with an allow-listing ObjectInputFilter

No new revisions were added by this update.

Summary of changes:
 .../sentencepiece/SentencePieceTokenizer.java      | 201 ++++++++++++++++++++-
 .../SentencePieceTokenizerSerializationTest.java   |  86 +++++++++
 2 files changed, 286 insertions(+), 1 deletion(-)