You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/09/29 02:18:00 UTC
[jira] [Commented] (OPENNLP-1933) RegEx removal part 5a: Remove regex from Leipzig filenames and dictionary loading
[ https://issues.apache.org/jira/browse/OPENNLP-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18120444#comment-18120444 ]
ASF GitHub Bot commented on OPENNLP-1933:
-----------------------------------------
krickert commented on PR #1320:
URL: https://github.com/apache/opennlp/pull/1320#issuecomment-5882375303
I replaced regex in Leipzig file name checks, Morfologik suffix handling and frequency dictionary parsing with direct character checks.
It matches only a trailing .info suffix. Split frequency columns on tabs or spaces, preserve Unicode terms and digits, and report line numbers for invalid counts.
I added tests for file names, separators, Unicode input and invalid counts.
Documented the dictionary formats in the manual.
LMK if I need to do anything else.
> RegEx removal part 5a: Remove regex from Leipzig filenames and dictionary loading
> ----------------------------------------------------------------------------------
>
> Key: OPENNLP-1933
> URL: https://issues.apache.org/jira/browse/OPENNLP-1933
> Project: OpenNLP
> Issue Type: Improvement
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 2.5.13, 3.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Replace regex operations in Leipzig filename checks, Morfologik suffix handling, and frequency-dictionary column parsing with character scans. Preserve Unicode dictionary terms and supported integer-count representations, removing the proposed ASCII-only count restriction. Keep space/tab column separators and existing negative-count and overflow checks. Add focused regression tests and document the supported format, referencing SymSpell's dictionary documentation: https://github.com/wolfgarbe/SymSpell#dictionary-file-format
--
This message was sent by Atlassian Jira
(v8.20.10#820010)