You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opennlp.apache.org by rz...@apache.org on 2026/08/30 17:46:16 UTC
(opennlp) branch 2.x-OPENNLP-1922 created (now 0fe04b17d)
This is an automated email from the ASF dual-hosted git repository.
rzo1 pushed a change to branch 2.x-OPENNLP-1922
in repository https://gitbox.apache.org/repos/asf/opennlp.git
at 0fe04b17d OPENNLP-1922: Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns (#1250)
This branch includes the following new commits:
new 0fe04b17d OPENNLP-1922: Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns (#1250)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(opennlp) 01/01: OPENNLP-1922: Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns (#1250)
Posted by rz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rzo1 pushed a commit to branch 2.x-OPENNLP-1922
in repository https://gitbox.apache.org/repos/asf/opennlp.git
commit 0fe04b17d8ff99f422f38fe65b6a64d0680deecd
Author: Suneel Marthi <sm...@apache.org>
AuthorDate: Sun Aug 30 09:11:08 2026 -0400
OPENNLP-1922: Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns (#1250)
* Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns
The built-in EMAIL and URL regexes in DEFAULT_REGEX_NAME_FINDER were
vulnerable to catastrophic backtracking and deep recursion
(CWE-1333 / CWE-400, and CWE-674 for URL). RegexNameFinder feeds the
concatenated token string straight into Matcher.find(), so a single
crafted long token drives the matcher into quadratic backtracking
(EMAIL) or a StackOverflowError (URL).
- EMAIL: bounded, possessive quantifiers with RFC 5321 length limits;
drop the malformed host:garbage sub-branch of the IP-literal
alternative. Quoted-string local part is retained.
- URL: flatten the nested (((/(...)+)+|/)+ ...) and (...(&...)*)*
groups to single-level possessive repetition with segment caps.
Also fixes two latent bugs: unescaped dot in "www." and the
"%[a-f\d{2}]" character class that never matched a percent octet.
- Capturing groups made non-capturing (unused by RegexNameFinder).
Adds a regression test asserting the built-in finders complete
promptly on 100k-char attack inputs.
* Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns
The built-in EMAIL and URL regexes in DEFAULT_REGEX_NAME_FINDER were
vulnerable to catastrophic backtracking and deep recursion
(CWE-1333 / CWE-400, and CWE-674 for URL). RegexNameFinder feeds the
concatenated token string straight into Matcher.find(), so a single
crafted long token drives the matcher into quadratic backtracking
(EMAIL) or a StackOverflowError (URL).
- EMAIL: bounded, possessive quantifiers with RFC 5321 length limits;
drop the malformed host:garbage sub-branch of the IP-literal
alternative. Quoted-string local part is retained.
- URL: flatten the nested (((/(...)+)+|/)+ ...) and (...(&...)*)*
groups to single-level possessive repetition with segment caps.
Also fixes two latent bugs: unescaped dot in "www." and the
"%[a-f\d{2}]" character class that never matched a percent octet.
- Capturing groups made non-capturing (unused by RegexNameFinder).
Adds a regression test asserting the built-in finders complete
promptly on 100k-char attack inputs.
# Conflicts:
# opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/namefind/RegexNameFinderFactoryTest.java
* Fix ReDoS / stack exhaustion in RegexNameFinderFactory EMAIL and URL patterns
The built-in EMAIL and URL regexes in DEFAULT_REGEX_NAME_FINDER were
vulnerable to catastrophic backtracking and deep recursion
(CWE-1333 / CWE-400, and CWE-674 for URL). RegexNameFinder feeds the
concatenated token string straight into Matcher.find(), so a single
crafted long token drives the matcher into quadratic backtracking
(EMAIL) or a StackOverflowError (URL).
Both patterns are rewritten so that every quantifier is bounded by a
constant. Bounds alone remove the exponential backtracking and cap the
recursion depth; the repetitions stay bounded ({0,255} / {1,63}) rather
than becoming * / + so that the trailing \b can still backtrack into a
matched run (a plain * reintroduces the overflow on a long path segment).
- EMAIL: bounded quantifiers with RFC 5321 length limits; drop the
malformed host:garbage sub-branch of the IP-literal alternative.
Quoted-string local part is retained.
- URL: flatten the nested (((/(...)+)+|/)+ ...) and (...(&...)*)* groups
to single-level bounded repetition with segment caps. Also fixes two
latent bugs: unescaped dot in "www." and the "%[a-f\d{2}]" character
class that never matched a percent octet. Capturing groups made
non-capturing (unused by RegexNameFinder).
Adds regression tests: the built-in finders complete promptly on
100k-char attack inputs (incl. the reported ?a&a&a&... query string),
and the URL finder still matches URLs with a trailing slash, sentence
period, '#' or '&'.
* Apply suggestion from @rzo1
Co-authored-by: Richard Zowalla <13...@users.noreply.github.com>
* Apply suggestion from @rzo1
Co-authored-by: Richard Zowalla <13...@users.noreply.github.com>
---------
Co-authored-by: Richard Zowalla <13...@users.noreply.github.com>
(cherry picked from commit 8f7befe4439530eda66221b1f4a927a00fd91a20)
---
.../tools/namefind/RegexNameFinderFactory.java | 39 +++++----
.../tools/namefind/RegexNameFinderFactoryTest.java | 94 ++++++++++++++++++++++
2 files changed, 118 insertions(+), 15 deletions(-)
diff --git a/opennlp-tools/src/main/java/opennlp/tools/namefind/RegexNameFinderFactory.java b/opennlp-tools/src/main/java/opennlp/tools/namefind/RegexNameFinderFactory.java
index 7c2bc79ec..0890ae051 100644
--- a/opennlp-tools/src/main/java/opennlp/tools/namefind/RegexNameFinderFactory.java
+++ b/opennlp-tools/src/main/java/opennlp/tools/namefind/RegexNameFinderFactory.java
@@ -107,12 +107,18 @@ public class RegexNameFinderFactory {
@Override
public Map<String, Pattern[]> getRegexMap() {
Pattern[] p = new Pattern[1];
- p[0] = Pattern.compile("([a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*" +
- "|\"([\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\\[\\x01-\\x09" +
- "\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9]([a-z0-9-]*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]" +
- "*[a-z0-9])?|\\[((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]" +
- "?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]" +
- "|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\\])", Pattern.CASE_INSENSITIVE);
+ // Every quantifier is bounded by a constant, which removes both the exponential
+ // backtracking and the recursion depth that made the old pattern a ReDoS vector.
+ // Limits follow RFC 5321: local part <= 64 chars, each domain label <= 63 chars.
+ p[0] = Pattern.compile(
+ "(?<![a-z0-9!#$%&'*+/=?^_`{|}~.-])" +
+ "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]{1,64}(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]{1,64}){0,10}" +
+ "|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]" +
+ "|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f]){0,255}\")" +
+ "@(?:(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\\.){1,20}" +
+ "[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?" +
+ "|\\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}" +
+ "(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\])", Pattern.CASE_INSENSITIVE);
Map<String, Pattern[]> regexMap = new HashMap<>();
regexMap.put(getType(), p);
return regexMap;
@@ -127,16 +133,19 @@ public class RegexNameFinderFactory {
@Override
public Map<String, Pattern[]> getRegexMap() {
Pattern[] p = new Pattern[1];
- p[0] = Pattern.compile("\\b(((ht|f)tp(s?)\\:\\/\\/|~\\/|\\/)|www.)"
- + "(\\w+:\\w+@)?(([-\\w]+\\.)+(com|org|net|gov"
+ // Flattened to single-level groups and every quantifier bounded by a constant.
+ // The bounds alone remove the nested-quantifier backtracking and the recursion
+ // depth that caused the StackOverflowError; they must stay bounded ({0,255} /
+ // {1,63} rather than * / +) or a long path segment reintroduces the overflow.
+ p[0] = Pattern.compile("\\b(?:(?:ht|f)tps?://|~/|/|www\\.)"
+ + "(?:\\w{1,63}:\\w{1,63}@)?"
+ + "(?:[-\\w]{1,63}\\.){1,20}(?:com|org|net|gov"
+ "|mil|biz|info|mobi|name|aero|jobs|museum"
- + "|travel|[a-z]{2}))(:[\\d]{1,5})?"
- + "(((\\/([-\\w~!$+|.,=]|%[a-f\\d]{2})+)+|\\/)+|\\?|#)?"
- + "((\\?([-\\w~!$+|.,*:]|%[a-f\\d{2}])+=?"
- + "([-\\w~!$+|.,*:=]|%[a-f\\d]{2})*)"
- + "(&(?:[-\\w~!$+|.,*:]|%[a-f\\d{2}])+=?"
- + "([-\\w~!$+|.,*:=]|%[a-f\\d]{2})*)*)*"
- + "(#([-\\w~!$+|.,*:=]|%[a-f\\d]{2})*)?\\b", Pattern.CASE_INSENSITIVE);
+ + "|travel|[a-z]{2})(?::\\d{1,5})?"
+ + "(?:/(?:[-\\w~!$+|.,=]|%[a-f\\d]{2}){0,255}){0,50}"
+ + "(?:\\?(?:[-\\w~!$+|.,*:=]|%[a-f\\d]{2}){0,255}"
+ + "(?:&(?:[-\\w~!$+|.,*:=]|%[a-f\\d]{2}){0,255}){0,50})?"
+ + "(?:#(?:[-\\w~!$+|.,*:=]|%[a-f\\d]{2}){0,255})?\\b", Pattern.CASE_INSENSITIVE);
Map<String, Pattern[]> regexMap = new HashMap<>();
regexMap.put(getType(), p);
return regexMap;
diff --git a/opennlp-tools/src/test/java/opennlp/tools/namefind/RegexNameFinderFactoryTest.java b/opennlp-tools/src/test/java/opennlp/tools/namefind/RegexNameFinderFactoryTest.java
index 809429925..6fdea6414 100644
--- a/opennlp-tools/src/test/java/opennlp/tools/namefind/RegexNameFinderFactoryTest.java
+++ b/opennlp-tools/src/test/java/opennlp/tools/namefind/RegexNameFinderFactoryTest.java
@@ -17,6 +17,7 @@
package opennlp.tools.namefind;
+import java.time.Duration;
import java.util.Arrays;
import java.util.List;
@@ -90,6 +91,99 @@ public class RegexNameFinderFactoryTest {
Assertions.assertEquals("45", tokens[latLongSpan2.getStart()]);
}
+ /**
+ * Crafted inputs that used to drive the built-in EMAIL and URL patterns into
+ * catastrophic backtracking / deep recursion (ReDoS, CWE-1333 / CWE-400 / CWE-674).
+ * The hardened patterns must finish quickly regardless of input length.
+ */
+ @Test
+ void testBuiltinPatternsAreNotVulnerableToReDoS() {
+ final String emailLocalBlowup = "a".repeat(100_000) + "@ ";
+ final String emailDomainBlowup = "x@a" + "-a".repeat(60_000) + " ";
+ final String urlPathRecursion = "http://a.com/" + "a".repeat(100_000) + " ";
+ final String urlNestedBlowup = "http://a.com" + "/a".repeat(50_000) + "%z";
+ // The reported StackOverflowError repro: a long ?a&a&a&... query string, which
+ // drove the nested (&(...)+ ... )* group in the old URL pattern into deep recursion.
+ final String urlQueryRecursion = "http://a.com/p?a" + "&a".repeat(50_000) + "= ";
+
+ Assertions.assertTimeoutPreemptively(Duration.ofSeconds(2), () -> {
+ for (String attack : new String[] {
+ emailLocalBlowup, emailDomainBlowup, urlPathRecursion, urlNestedBlowup, urlQueryRecursion}) {
+ String[] tokens = WhitespaceTokenizer.INSTANCE.tokenize(attack);
+ regexNameFinder.find(tokens);
+ }
+ });
+ }
+
+ /**
+ * Regression tests for the hardened built-in patterns: a trailing slash or a
+ * sentence-final period must not cause the match to be abandoned entirely.
+ */
+ @Test
+ void testUrlWithTrailingSlashOrSentencePunctuation() {
+ RegexNameFinder urlFinder = RegexNameFinderFactory.getDefaultRegexNameFinders(
+ RegexNameFinderFactory.DEFAULT_REGEX_NAME_FINDER.URL);
+ final String input = "see http://a.com/ or http://example.com/path/ or www.google.com.";
+ Span[] spans = urlFinder.find(input);
+ Assertions.assertEquals(3, spans.length);
+ Assertions.assertEquals("http://a.com",
+ input.substring(spans[0].getStart(), spans[0].getEnd()));
+ Assertions.assertEquals("http://example.com/path",
+ input.substring(spans[1].getStart(), spans[1].getEnd()));
+ Assertions.assertEquals("www.google.com",
+ input.substring(spans[2].getStart(), spans[2].getEnd()));
+ }
+
+ @Test
+ void testUrlKeepsPortWhenFollowedBySlash() {
+ RegexNameFinder urlFinder = RegexNameFinderFactory.getDefaultRegexNameFinders(
+ RegexNameFinderFactory.DEFAULT_REGEX_NAME_FINDER.URL);
+ final String input = "http://example.com:8080/";
+ Span[] spans = urlFinder.find(input);
+ Assertions.assertEquals(1, spans.length);
+ Assertions.assertEquals("http://example.com:8080",
+ input.substring(spans[0].getStart(), spans[0].getEnd()));
+ }
+
+ @Test
+ void testEmailAtEndOfSentence() {
+ RegexNameFinder emailFinder = RegexNameFinderFactory.getDefaultRegexNameFinders(
+ RegexNameFinderFactory.DEFAULT_REGEX_NAME_FINDER.EMAIL);
+ final String input = "mail me at a@b.com. and a@sub.b.co.uk.";
+ Span[] spans = emailFinder.find(input);
+ Assertions.assertEquals(2, spans.length);
+ Assertions.assertEquals("a@b.com", input.substring(spans[0].getStart(), spans[0].getEnd()));
+ Assertions.assertEquals("a@sub.b.co.uk",
+ input.substring(spans[1].getStart(), spans[1].getEnd()));
+ }
+
+ /**
+ * The hardened URL pattern must still match a URL that is followed by a trailing
+ * delimiter (a path slash, a sentence-final period, {@code #} or {@code &}), stopping
+ * the span before the delimiter rather than abandoning the match.
+ */
+ @Test
+ void testUrlTrailingDelimiters() {
+ assertFirstUrl("http://a.com", "http://a.com/");
+ assertFirstUrl("http://example.com/path", "http://example.com/path/");
+ assertFirstUrl("ftp://files.example.com/pub", "ftp://files.example.com/pub/");
+ assertFirstUrl("www.google.com", "check www.google.com/ now");
+ assertFirstUrl("http://example.com", "I saw http://example.com. Then");
+ assertFirstUrl("www.google.com", "www.google.com.");
+ assertFirstUrl("http://a.com/path", "http://a.com/path.");
+ assertFirstUrl("http://a.com", "http://a.com/#");
+ assertFirstUrl("http://example.com:8080", "http://example.com:8080/");
+ assertFirstUrl("http://a.com/p?q=1", "http://a.com/p?q=1&");
+ }
+
+ private static void assertFirstUrl(String expected, String input) {
+ RegexNameFinder urlFinder = RegexNameFinderFactory.getDefaultRegexNameFinders(
+ RegexNameFinderFactory.DEFAULT_REGEX_NAME_FINDER.URL);
+ Span[] spans = urlFinder.find(input);
+ Assertions.assertTrue(spans.length > 0, "no URL match for: " + input);
+ Assertions.assertEquals(expected, input.substring(spans[0].getStart(), spans[0].getEnd()));
+ }
+
@Test
void testMgrs() {
String[] tokens = WhitespaceTokenizer.INSTANCE.tokenize(text);