You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/09/29 18:24:00 UTC
[jira] [Commented] (OPENNLP-1990) Regex removal (5c/10): Validate MASC identifiers and anchors consistently
[ https://issues.apache.org/jira/browse/OPENNLP-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18120857#comment-18120857 ]
ASF GitHub Bot commented on OPENNLP-1990:
-----------------------------------------
rzo1 commented on code in PR #1322:
URL: https://github.com/apache/opennlp/pull/1322#discussion_r4136879559
##########
opennlp-core/opennlp-formats/src/test/java/opennlp/tools/formats/masc/MascIdentifiersTest.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package opennlp.tools.formats.masc;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import opennlp.tools.util.Span;
+
+public class MascIdentifiersTest {
+
+ @ParameterizedTest
+ @CsvSource({"ne-n7, ne-n, 7", "penn-n12, penn-n, 12", "seg-r0, seg-r, 0", "seg-r00, seg-r, 0",
+ "penn-n007, penn-n, 7", "ne-n2147483647, ne-n, 2147483647"})
+ void testParseIdReadsTheNumberAfterThePrefix(String id, String prefix, int expected) {
+ Assertions.assertEquals(expected, MascIdentifiers.parseId(id, prefix));
+ }
+
+ @ParameterizedTest
+ // other or missing prefix, prefix later in the text, doubled prefix, no digits, sign,
+ // digits of another script, trailing text, whitespace, and empty
+ @ValueSource(strings = {"7", "xne-n7", "NE-N7", "ne\u2011n7", "ne-nne-n7", "ne-n", "ne-n-7",
+ "ne-n+7", "ne-n\u0661", "ne-n\uFF17", "ne-n7\u0661", "ne-n\u06F7", "ne-n\u00B2", "ne-n\u2167",
+ "ne-n\uD835\uDFCE", "ne-n7x", "ne-n7 ", " ne-n7", "ne-n7\n", ""})
+ void testParseIdRejectsAnythingElse(String id) {
+ Assertions.assertThrows(IllegalArgumentException.class,
+ () -> MascIdentifiers.parseId(id, MascIdentifiers.NAMED_ENTITY_ID_PREFIX));
+ }
+
+ @Test
+ void testParseIdRejectsNull() {
+ Assertions.assertThrows(IllegalArgumentException.class,
+ () -> MascIdentifiers.parseId(null, MascIdentifiers.NAMED_ENTITY_ID_PREFIX));
+ }
Review Comment:
`""` is in the `@ValueSource` and null has its own test. Use `@NullAndEmptySource`.
##########
opennlp-core/opennlp-formats/src/main/java/opennlp/tools/formats/masc/MascIdentifiers.java:
##########
@@ -0,0 +1,182 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package opennlp.tools.formats.masc;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import opennlp.tools.util.Span;
+import opennlp.tools.util.StringUtil;
+
+/**
+ * Parses MASC identifier and anchor attributes; lists are separated by
+ * <a href="https://www.w3.org/TR/xml/#NT-S">XML whitespace</a>.
+ */
+final class MascIdentifiers {
+
+ /** The prefix of a named entity node identifier, as in {@code ne-n7}. */
+ static final String NAMED_ENTITY_ID_PREFIX = "ne-n";
+
+ /** The prefix of a Penn token node identifier, as in {@code penn-n7}. */
+ static final String PENN_TOKEN_ID_PREFIX = "penn-n";
+
+ /** The prefix of a segmentation region identifier, as in {@code seg-r7}. */
+ static final String REGION_ID_PREFIX = "seg-r";
+
+ private MascIdentifiers() {
+ }
+
+ /**
+ * Parses the number of an identifier that starts with {@code prefix} and continues with
+ * one or more ASCII digits only.
+ *
+ * @param id The identifier, such as {@code penn-n7}.
+ * @param prefix The expected prefix, such as {@link #PENN_TOKEN_ID_PREFIX}.
+ * @return The number after the prefix.
+ * @throws IllegalArgumentException Thrown if {@code id} is {@code null}, does not start with
+ * {@code prefix}, is not followed by digits only, or the number does not fit
+ * an {@code int}.
+ */
+ static int parseId(String id, String prefix) {
+ if (id == null || !id.startsWith(prefix)) {
+ throw invalidId(id, prefix, null);
+ }
+ try {
+ return parseAsciiInt(id, prefix.length());
+ } catch (NumberFormatException e) {
+ // reached only when the digits overflow an int
+ throw new IllegalArgumentException("MASC identifier number does not fit an int: " + id, e);
+ } catch (IllegalArgumentException e) {
+ throw invalidId(id, prefix, e);
+ }
+ }
+
+ /**
+ * Describes an identifier that is not {@code prefix} followed by ASCII digits.
+ *
+ * @param id The rejected identifier.
+ * @param prefix The expected prefix.
+ * @param cause The underlying error, or {@code null}.
+ * @return The exception to throw.
+ */
+ private static IllegalArgumentException invalidId(String id, String prefix, Throwable cause) {
+ return new IllegalArgumentException(
+ "MASC identifier must be " + prefix + " followed by digits: " + id, cause);
+ }
+
+ /**
+ * Parses an XML whitespace separated list of identifiers as {@link #parseId(String, String)}
+ * does.
+ *
+ * @param ids The identifiers, such as {@code seg-r1 seg-r2}.
+ * @param prefix The expected prefix of each identifier.
+ * @return The numbers in order.
+ * @throws IllegalArgumentException Thrown if {@code ids} is {@code null}, names no identifier, or
+ * contains one that {@link #parseId(String, String)} rejects.
+ */
+ static int[] parseIds(String ids, String prefix) {
+ if (ids == null) {
+ throw new IllegalArgumentException("MASC identifier list must not be null");
+ }
+ String[] items = splitOnXmlWhitespace(ids);
+ if (items.length == 0) {
+ throw new IllegalArgumentException("MASC identifier list must name at least one identifier");
+ }
+ int[] numbers = new int[items.length];
+ for (int i = 0; i < items.length; i++) {
+ numbers[i] = parseId(items[i], prefix);
+ }
+ return numbers;
+ }
+
+ /**
+ * Splits an attribute value on runs of XML whitespace. Leading, trailing, and repeated separators
+ * produce no empty item.
+ *
+ * @param value The attribute value. Must not be {@code null}.
+ * @return The non-empty items in order; empty for a value without one.
+ */
+ private static String[] splitOnXmlWhitespace(String value) {
+ List<String> items = new ArrayList<>();
+ int start = -1;
+ for (int i = 0; i <= value.length(); i++) {
+ if (i == value.length() || isXmlWhitespace(value.charAt(i))) {
+ if (start >= 0) {
+ items.add(value.substring(start, i));
+ start = -1;
+ }
+ } else if (start < 0) {
+ start = i;
+ }
+ }
+ return items.toArray(new String[0]);
+ }
+
+ /**
+ * Parses the two ordered, nonnegative offsets of a contiguous text region.
+ *
+ * @param anchors The XML whitespace separated offsets.
+ * @return The region span.
+ * @throws IllegalArgumentException Thrown if the anchors are missing, malformed, out of range,
+ * negative, or reversed.
+ */
+ static Span parseAnchors(String anchors) {
+ if (anchors == null) {
+ throw new IllegalArgumentException("MASC region anchors must not be null");
+ }
+ String[] items = splitOnXmlWhitespace(anchors);
+ if (items.length != 2) {
+ throw new IllegalArgumentException("MASC region anchors must contain exactly two offsets: " + anchors);
+ }
+ try {
+ return new Span(Integer.parseInt(items[0]), Integer.parseInt(items[1]));
+ } catch (IllegalArgumentException e) {
+ throw new IllegalArgumentException("Invalid MASC region anchors: " + anchors, e);
+ }
+ }
+
+ /**
+ * Parses the ASCII digits from {@code from} to the end of {@code text} as a nonnegative
+ * {@code int}. A sign and digits of other scripts are not accepted.
+ *
+ * @param text The text holding the digits.
+ * @param from The index of the first digit.
+ * @return The number.
+ * @throws IllegalArgumentException Thrown if there is no digit at {@code from} or the digits
+ * do not reach the end of {@code text}.
+ * @throws NumberFormatException Thrown if the number does not fit an {@code int}.
+ */
+ private static int parseAsciiInt(String text, int from) {
+ int end = StringUtil.endOfAsciiDigits(text, from);
+ if (end == from || end != text.length()) {
+ throw new IllegalArgumentException("Expected ASCII digits only: " + text.substring(from));
+ }
+ return Integer.parseInt(text, from, end, 10);
+ }
+
+ /**
+ * Tests for <a href="https://www.w3.org/TR/xml/#NT-S">XML whitespace</a>: space, tab,
+ * carriage return or line feed.
+ *
+ * @param c The character to test.
+ * @return {@code true} if {@code c} is XML whitespace, {@code false} otherwise.
+ */
+ private static boolean isXmlWhitespace(char c) {
+ return c == ' ' || c == '\t' || c == '\r' || c == '\n';
Review Comment:
Please declare `' '`, `'\t'`, `'\r'` and `'\n'` as constants, like in `EventFields`.
##########
opennlp-core/opennlp-formats/src/test/java/opennlp/tools/formats/masc/MascNamedEntityParserTest.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package opennlp.tools.formats.masc;
+
+import java.util.List;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import org.xml.sax.SAXException;
+
+public class MascNamedEntityParserTest {
+
+ /**
+ * Parses an annotation fixture.
+ *
+ * @param xml The annotation XML.
+ * @return The parser containing the annotations.
+ * @throws Exception Thrown if parsing fails.
+ */
+ private static MascNamedEntityParser parse(String xml) throws Exception {
+ return MascParserTestUtil.parse(xml, new MascNamedEntityParser());
+ }
+
+ /**
+ * Builds a person annotation for a node reference.
+ *
+ * @param ref The node reference.
+ * @return The annotation XML.
+ */
+ private static String entity(String ref) {
+ return "<a ref=\"" + ref + "\" label=\"person\"/>";
+ }
+
+ /**
+ * Checks that malformed annotations retain their validation cause.
+ *
+ * @param xml The malformed annotation XML.
+ */
+ private static void assertRejected(String xml) {
Review Comment:
`assertRejected` and `parse` are copied into four test classes (here, `MascPennTagParserTest:57`, `MascSentenceParserTest:60`, `MascWordParserTest:59`). Please move them to `MascParserTestUtil`.
##########
opennlp-docs/src/docbkx/corpora.xml:
##########
@@ -844,4 +844,15 @@ Organization: precision: 85.11%; recall: 79.38%; F1: 82.14%. [target: 130
</para>
</section>
</section>
+ <section xml:id="tools.corpora.masc">
+ <title>MASC</title>
+ <para>
+ The <link xlink:href="https://anc.org/data/masc/masc-structure">MASC</link> reader requires
+ the identifiers <code>seg-r</code>, <code>penn-n</code> and <code>ne-n</code> followed by
+ ASCII digits, at most 2147483647, and exactly two nonnegative, ordered offsets per region.
+ Lists of identifiers and offsets are separated by space, tab, carriage return or line feed.
+ Since 3.0.0, a malformed annotation stops reading with an <code>IOException</code> whose
+ cause names the value.
+ </para>
+ </section>
Review Comment:
The MASC section only lists validation rules. It doesn't say what MASC is or how to use the reader. "Since 3.0.0, a malformed annotation stops reading with an IOException" is also not right, it threw an `IOException` before, just without the cause. Either write a real section (what it is, which formats, one example) or drop it.
##########
opennlp-core/opennlp-formats/src/test/java/opennlp/tools/formats/masc/MascPennTagParserTest.java:
##########
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package opennlp.tools.formats.masc;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+import org.xml.sax.SAXException;
+
+public class MascPennTagParserTest {
+
+ /**
+ * Parses an annotation fixture.
+ *
+ * @param xml The annotation XML.
+ * @return The parser containing the annotations.
+ * @throws Exception Thrown if parsing fails.
+ */
+ private static MascPennTagParser parse(String xml) throws Exception {
+ return MascParserTestUtil.parse(xml, new MascPennTagParser());
+ }
+
+ /**
+ * Builds a token node linked to segmentation regions.
+ *
+ * @param id The token identifier.
+ * @param targets The region identifiers.
+ * @return The annotation XML.
+ */
+ private static String tokenWithTargets(String id, String targets) {
+ return "<graph><node xml:id=\"" + id + "\"><link targets=\"" + targets + "\"/></node></graph>";
+ }
+
+ /**
+ * Checks that malformed annotations retain their validation cause.
+ *
+ * @param xml The malformed annotation XML.
+ * @return The parsing exception.
+ */
+ private static SAXException assertRejected(String xml) {
+ SAXException e = Assertions.assertThrows(SAXException.class, () -> parse(xml));
+ Assertions.assertInstanceOf(IllegalArgumentException.class, e.getCause());
+ return e;
+ }
+
+ @Test
+ void testTokenIdsLoseTheirPrefix() throws Exception {
+ MascPennTagParser parser = parse("<graph>"
+ + "<node xml:id=\"penn-n10\"><link targets=\"seg-r0 seg-r1\"/></node>"
+ + "<a ref=\"penn-n10\"><fs>"
+ + "<f name=\"msd\" value=\"NN\"/><f name=\"base\" value=\"test\"/>"
+ + "</fs></a>"
+ + "</graph>");
+ Assertions.assertArrayEquals(new int[] {0, 1}, parser.getTokenToQuarks().get(10));
+ Assertions.assertEquals("NN", parser.getTags().get(10));
+ Assertions.assertEquals("test", parser.getBases().get(10));
+ }
+
+ @ParameterizedTest
+ // a tab or a line break written directly into the attribute is a space after XML
+ // attribute-value normalization, a character reference keeps the character
+ @MethodSource("opennlp.tools.formats.masc.MascParserTestUtil#xmlWhitespaceSeparators")
+ void testLinkTargetsUseXmlWhitespace(String separator) throws Exception {
+ MascPennTagParser parser = parse(tokenWithTargets("penn-n10", " seg-r0" + separator + "seg-r1 "));
+ Assertions.assertArrayEquals(new int[] {0, 1}, parser.getTokenToQuarks().get(10));
+ }
+
+ @ParameterizedTest
+ // these characters are not XML whitespace, even when introduced through a reference
+ @ValueSource(strings = {"seg-r0 seg-r1", "seg-r0　seg-r1", "seg-r0…seg-r1"})
+ void testLinkTargetsSeparatedByOtherWhitespaceAreRejected(String targets) {
+ assertRejected(tokenWithTargets("penn-n10", targets));
+ }
+
+ @ParameterizedTest
+ // doubled prefix, missing prefix, other prefix, no digits, trailing text
+ @ValueSource(strings = {"penn-npenn-n2", "2", "ne-n2", "penn-n", "penn-n2x"})
+ void testMalformedTokenIdsAreRejected(String id) {
Review Comment:
The test checks the node id and the `ref`. Please split it.
##########
opennlp-core/opennlp-formats/src/main/java/opennlp/tools/formats/masc/MascIdentifiers.java:
##########
@@ -0,0 +1,182 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package opennlp.tools.formats.masc;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import opennlp.tools.util.Span;
+import opennlp.tools.util.StringUtil;
+
+/**
+ * Parses MASC identifier and anchor attributes; lists are separated by
+ * <a href="https://www.w3.org/TR/xml/#NT-S">XML whitespace</a>.
+ */
+final class MascIdentifiers {
+
+ /** The prefix of a named entity node identifier, as in {@code ne-n7}. */
+ static final String NAMED_ENTITY_ID_PREFIX = "ne-n";
+
+ /** The prefix of a Penn token node identifier, as in {@code penn-n7}. */
+ static final String PENN_TOKEN_ID_PREFIX = "penn-n";
+
+ /** The prefix of a segmentation region identifier, as in {@code seg-r7}. */
+ static final String REGION_ID_PREFIX = "seg-r";
+
+ private MascIdentifiers() {
+ }
+
+ /**
+ * Parses the number of an identifier that starts with {@code prefix} and continues with
+ * one or more ASCII digits only.
+ *
+ * @param id The identifier, such as {@code penn-n7}.
+ * @param prefix The expected prefix, such as {@link #PENN_TOKEN_ID_PREFIX}.
+ * @return The number after the prefix.
+ * @throws IllegalArgumentException Thrown if {@code id} is {@code null}, does not start with
+ * {@code prefix}, is not followed by digits only, or the number does not fit
+ * an {@code int}.
+ */
+ static int parseId(String id, String prefix) {
+ if (id == null || !id.startsWith(prefix)) {
+ throw invalidId(id, prefix, null);
+ }
+ try {
+ return parseAsciiInt(id, prefix.length());
+ } catch (NumberFormatException e) {
+ // reached only when the digits overflow an int
+ throw new IllegalArgumentException("MASC identifier number does not fit an int: " + id, e);
+ } catch (IllegalArgumentException e) {
+ throw invalidId(id, prefix, e);
+ }
+ }
+
+ /**
+ * Describes an identifier that is not {@code prefix} followed by ASCII digits.
+ *
+ * @param id The rejected identifier.
+ * @param prefix The expected prefix.
+ * @param cause The underlying error, or {@code null}.
+ * @return The exception to throw.
+ */
+ private static IllegalArgumentException invalidId(String id, String prefix, Throwable cause) {
+ return new IllegalArgumentException(
+ "MASC identifier must be " + prefix + " followed by digits: " + id, cause);
+ }
+
+ /**
+ * Parses an XML whitespace separated list of identifiers as {@link #parseId(String, String)}
+ * does.
+ *
+ * @param ids The identifiers, such as {@code seg-r1 seg-r2}.
+ * @param prefix The expected prefix of each identifier.
+ * @return The numbers in order.
+ * @throws IllegalArgumentException Thrown if {@code ids} is {@code null}, names no identifier, or
+ * contains one that {@link #parseId(String, String)} rejects.
+ */
+ static int[] parseIds(String ids, String prefix) {
+ if (ids == null) {
+ throw new IllegalArgumentException("MASC identifier list must not be null");
+ }
+ String[] items = splitOnXmlWhitespace(ids);
+ if (items.length == 0) {
+ throw new IllegalArgumentException("MASC identifier list must name at least one identifier");
+ }
+ int[] numbers = new int[items.length];
+ for (int i = 0; i < items.length; i++) {
+ numbers[i] = parseId(items[i], prefix);
+ }
+ return numbers;
+ }
+
+ /**
+ * Splits an attribute value on runs of XML whitespace. Leading, trailing, and repeated separators
+ * produce no empty item.
+ *
+ * @param value The attribute value. Must not be {@code null}.
+ * @return The non-empty items in order; empty for a value without one.
+ */
+ private static String[] splitOnXmlWhitespace(String value) {
+ List<String> items = new ArrayList<>();
+ int start = -1;
+ for (int i = 0; i <= value.length(); i++) {
+ if (i == value.length() || isXmlWhitespace(value.charAt(i))) {
+ if (start >= 0) {
+ items.add(value.substring(start, i));
+ start = -1;
+ }
+ } else if (start < 0) {
+ start = i;
+ }
+ }
+ return items.toArray(new String[0]);
+ }
+
+ /**
+ * Parses the two ordered, nonnegative offsets of a contiguous text region.
+ *
+ * @param anchors The XML whitespace separated offsets.
+ * @return The region span.
+ * @throws IllegalArgumentException Thrown if the anchors are missing, malformed, out of range,
+ * negative, or reversed.
+ */
+ static Span parseAnchors(String anchors) {
+ if (anchors == null) {
+ throw new IllegalArgumentException("MASC region anchors must not be null");
+ }
+ String[] items = splitOnXmlWhitespace(anchors);
+ if (items.length != 2) {
+ throw new IllegalArgumentException("MASC region anchors must contain exactly two offsets: " + anchors);
+ }
+ try {
+ return new Span(Integer.parseInt(items[0]), Integer.parseInt(items[1]));
Review Comment:
ids only take ASCII digits, but the anchors go through `Integer.parseInt`. So `"+0 4"`, `"-0 +4"` and `"0 ٤"` are accepted, and `MascParserTestUtil:66-68` even tests for that. Please use `parseAsciiInt` for the anchors too and move the `equivalentAnchors` cases to the rejected ones.
> Regex removal (5c/10): Validate MASC identifiers and anchors consistently
> -------------------------------------------------------------------------
>
> Key: OPENNLP-1990
> URL: https://issues.apache.org/jira/browse/OPENNLP-1990
> Project: OpenNLP
> Issue Type: Improvement
> Reporter: Kristian Rickert
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Unify MASC identifier, target-list, and anchor parsing across the relevant readers. Validate expected prefixes, numeric ranges, and exactly two ordered, nonnegative anchors for supported text spans, preserving useful SAX error causes. Apply whitespace rules supported by the MASC/GrAF format and document their relationship to XML attribute normalization: https://www.w3.org/TR/xml/#AVNormalize. Preserve source text and offsets, add malformed-input and Unicode regressions, and validate the changes against a real MASC corpus.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)