You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/09/24 11:11:00 UTC
[jira] [Commented] (OPENNLP-1931) RegEx removal part 3: JSON vocabulary and id2label scrape in opennlp-dl
[ https://issues.apache.org/jira/browse/OPENNLP-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118807#comment-18118807 ]
ASF GitHub Bot commented on OPENNLP-1931:
-----------------------------------------
krickert commented on PR #1277:
URL: https://github.com/apache/opennlp/pull/1277#issuecomment-5812923554
Once green, going to run the eval build
> RegEx removal part 3: JSON vocabulary and id2label scrape in opennlp-dl
> -----------------------------------------------------------------------
>
> Key: OPENNLP-1931
> URL: https://issues.apache.org/jira/browse/OPENNLP-1931
> Project: OpenNLP
> Issue Type: Improvement
> Components: Deep Learning
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 2.5.13, 3.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{AbstractDL.loadJsonVocab}} and {{DocumentCategorizerConfig.fromJson}} located string keys and values with regular expressions. Both now use a small cursor helper, {{opennlp.dl.JsonScan}} (public, marked internal), that finds string literals honoring backslash escapes, skips ASCII whitespace, and reads digit runs.
> Behavior is unchanged, including the quirks of the old patterns, which the tests pin: a value with a decimal fraction yields its integer prefix, a negative id is skipped, the id2label object is cut at the first closing brace even inside a nested value, and a key spanning a newline is accepted while a value with a newline is skipped.
> Verification: the old patterns and the new scan were compared over the real vocabulary and configuration resources, 57 hand-written edge cases, and 400,000 generated inputs with zero differences. {{JsonScanTest}} adds 71 cases, {{LoadVocabTest}} 27, {{DocumentCategorizerConfigTest}} 24.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)