You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@allura.apache.org by br...@apache.org on 2025/11/21 19:04:31 UTC
(allura) branch db/8588 updated (ea9d48c8c -> 596f00968)
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a change to branch db/8588
in repository https://gitbox.apache.org/repos/asf/allura.git
discard ea9d48c8c [#8588] convert all markdown.inlinepatterns.Pattern to markdown.inlinepatterns.InlineProcessor based on https://github.com/Python-Markdown/markdown/commit/d18c3d0acab0e7469c3284c897afcb61f9dd1fea
discard 755171026 [#8588] use *+ instead of * (possessive quantifiers) and regular re library. Seems like both *+ are not needed, but doing both for good measure
discard a28da9def [#8588] no longer need regex-as-re-globally hacky package
discard 87fb1a290 [#8588] recompile this bad regexp with the "regex" library which has better worst-case performance
add aa817a88f restrict optional bcrypt dep since passlib isn't updated for it
new 62065c38b [#8588] recompile this bad regexp with the "regex" library which has better worst-case performance
new a8ad3b7ae [#8588] no longer need regex-as-re-globally hacky package
new f689e79b6 [#8588] use *+ instead of * (possessive quantifiers) and regular re library. Seems like both *+ are not needed, but doing both for good measure
new 596f00968 [#8588] convert all markdown.inlinepatterns.Pattern to markdown.inlinepatterns.InlineProcessor based on https://github.com/Python-Markdown/markdown/commit/d18c3d0acab0e7469c3284c897afcb61f9dd1fea
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ea9d48c8c)
\
N -- N -- N refs/heads/db/8588 (596f00968)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
requirements-optional.txt | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
(allura) 01/04: [#8588] recompile this bad regexp with the "regex" library which has better worst-case performance
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8588
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 62065c38bc3b9a2409594c04db18e7243dc42a15
Author: Dave Brondsema <db...@slashdotmedia.com>
AuthorDate: Thu Nov 20 17:21:42 2025 -0500
[#8588] recompile this bad regexp with the "regex" library which has better worst-case performance
---
Allura/allura/lib/markdown_extensions.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Allura/allura/lib/markdown_extensions.py b/Allura/allura/lib/markdown_extensions.py
index b576ab1ff..93a2904af 100644
--- a/Allura/allura/lib/markdown_extensions.py
+++ b/Allura/allura/lib/markdown_extensions.py
@@ -24,6 +24,7 @@
from urllib.parse import urljoin
+import regex
from tg import config
from bs4 import BeautifulSoup, MarkupResemblesLocatorWarning
import html5lib
@@ -361,6 +362,7 @@ class ForgeLinkPattern(markdown.inlinepatterns.Pattern):
def __init__(self, *args, **kwargs):
self.ext = kwargs.pop('ext')
super().__init__(*args, **kwargs)
+ self.compiled_re = regex.compile(self.compiled_re.pattern, flags=self.compiled_re.flags)
def handleMatch(self, m):
el = etree.Element('a')
(allura) 04/04: [#8588] convert all markdown.inlinepatterns.Pattern to markdown.inlinepatterns.InlineProcessor based on https://github.com/Python-Markdown/markdown/commit/d18c3d0acab0e7469c3284c897afcb61f9dd1fea
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8588
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 596f00968fca6a05d3ce873a5596378544449ed9
Author: Dave Brondsema <db...@slashdotmedia.com>
AuthorDate: Thu Nov 20 18:24:21 2025 -0500
[#8588] convert all markdown.inlinepatterns.Pattern to markdown.inlinepatterns.InlineProcessor based on https://github.com/Python-Markdown/markdown/commit/d18c3d0acab0e7469c3284c897afcb61f9dd1fea
---
Allura/allura/lib/markdown_extensions.py | 68 +++++++++++++++-----------------
1 file changed, 31 insertions(+), 37 deletions(-)
diff --git a/Allura/allura/lib/markdown_extensions.py b/Allura/allura/lib/markdown_extensions.py
index 4579e1970..52eb10b10 100644
--- a/Allura/allura/lib/markdown_extensions.py
+++ b/Allura/allura/lib/markdown_extensions.py
@@ -47,7 +47,7 @@
SHORT_REF_RE = markdown.inlinepatterns.NOIMG + r'\[([^\]]+)\]'
# FORGE_LINK_RE copied from markdown pre 3.0's LINK_RE
-# TODO: replace these with newer approach, see ForgeLinkPattern
+# TODO: further simplify now that InlineProcessor is used, see ForgeLinkPattern? or not needed since possessive quantifiers are used
NOBRACKET = r'[^\]\[]*+'
BRK = (
r'\[(' +
@@ -56,7 +56,7 @@
NOBRACKET + r')\]'
)
FORGE_LINK_RE = markdown.inlinepatterns.NOIMG + BRK + \
- r'''\(\s*(<.*?>|((?:(?:\(.*?\))|[^\(\)]))*?)\s*((['"])(.*?)\12\s*)?\)'''
+ r'''\(\s*(<.*?>|((?:(?:\(.*?\))|[^\(\)]))*?)\s*((['"])(.*?)\11\s*)?\)'''
def clear_markdown_registry(reg: markdown.util.Registry, keep: list[str] = []):
@@ -286,7 +286,7 @@ def extendMarkdown(self, md):
# this has to be before the 'escape' processor, otherwise weird
# placeholders are inserted for escaped chars within urls, and then the
# autolink can't match the whole url
- md.inlinePatterns.register(AutolinkPattern(r'(http(?:s?)://[a-zA-Z0-9./\-\\_%?&=+#;~:!@]+)', md),
+ md.inlinePatterns.register(AutolinkPattern(r'(?:(?<=\s)|^)(http(?:s?)://[a-zA-Z0-9./\-\\_%?&=+#;~:!@]+)', md),
'autolink_without_brackets',
185) # was '<escape' and 'escape' is priority 180; great num runs first, so: 185
# replace the link pattern with our extended version
@@ -318,11 +318,11 @@ def extendMarkdown(self, md):
md.inlinePatterns.register(EmojiInlinePattern(self.EMOJI_RE), 'emoji', 0)
-class EmojiInlinePattern(markdown.inlinepatterns.Pattern):
+class EmojiInlinePattern(markdown.inlinepatterns.InlineProcessor):
- def handleMatch(self, m):
- emoji_code = m.group(2)
- return emoji.emojize(emoji_code, language="alias")
+ def handleMatch(self, m: re.Match[str], data: str) -> tuple[etree.Element | None, int | None, int | None]:
+ emoji_code = m.group(1)
+ return emoji.emojize(emoji_code, language="alias"), m.start(0), m.end(0)
class UserMentionExtension(markdown.Extension):
@@ -334,10 +334,10 @@ def extendMarkdown(self, md):
md.inlinePatterns.register(UserMentionInlinePattern(self.UM_RE), 'user_mentions', 0)
-class UserMentionInlinePattern(markdown.inlinepatterns.Pattern):
+class UserMentionInlinePattern(markdown.inlinepatterns.InlineProcessor):
- def handleMatch(self, m):
- user_name = m.group(2).replace("@", "")
+ def handleMatch(self, m: re.Match[str], data: str) -> tuple[etree.Element | None, int | None, int | None]:
+ user_name = m.group(1).replace("@", "")
user = M.User.by_username(user_name)
result = None
@@ -348,13 +348,14 @@ def handleMatch(self, m):
result.set('class', 'user-mention')
else:
result = "@%s" % user_name
- return result
+ return result, m.start(0), m.end(0)
-class ForgeLinkPattern(markdown.inlinepatterns.Pattern):
- # TODO: convert from extending Pattern to extending InlineProcessor
- # which is how core Markdown library in 3.0 made its base link parsing much faster.
+class ForgeLinkPattern(markdown.inlinepatterns.InlineProcessor):
+ # TODO: consider further changes to align with InlineProcessor patterns
+ # but maybe isn't needed since NOBRACKET and BRK are fast now (see their comments)
# https://github.com/Python-Markdown/markdown/commit/d18c3d0acab0e7469c3284c897afcb61f9dd1fea
+ # https://github.com/brondsem/allura/pull/1 maybe a starting point
artifact_re = re.compile(r'((.*?):)?((.*?):)?(.+)')
@@ -362,26 +363,26 @@ def __init__(self, *args, **kwargs):
self.ext = kwargs.pop('ext')
super().__init__(*args, **kwargs)
- def handleMatch(self, m):
+ def handleMatch(self, m: re.Match[str], data: str) -> tuple[etree.Element | None, int | None, int | None]:
el = etree.Element('a')
- el.text = m.group(2)
+ el.text = m.group(1)
is_link_with_brackets = False
try:
- href = m.group(9)
+ href = m.group(8)
except IndexError:
- href = m.group(2)
+ href = m.group(1)
is_link_with_brackets = True
if el.text == 'x' or el.text == ' ': # skip [ ] and [x] for markdown checklist
- return '[' + el.text + ']'
+ return None, None, None
try:
- title = m.group(13)
+ title = m.group(12)
except IndexError:
title = None
classes = ''
if href:
if href == 'TOC':
- return '[TOC]' # skip TOC
+ return None, None, None
if self.artifact_re.match(href):
href, classes = self._expand_alink(href, is_link_with_brackets)
el.set('href', self.unescape(href.strip()))
@@ -397,7 +398,7 @@ def handleMatch(self, m):
text = el.text
el = etree.Element('span')
el.text = '[%s]' % text
- return el
+ return el, m.start(0), m.end(0)
def _expand_alink(self, link, is_link_with_brackets):
'''Return (href, classes) for an artifact link'''
@@ -426,17 +427,17 @@ def _expand_alink(self, link, is_link_with_brackets):
return href, classes
-class ForgeMacroPattern(markdown.inlinepatterns.Pattern):
+class ForgeMacroPattern(markdown.inlinepatterns.InlineProcessor):
def __init__(self, *args, **kwargs):
self.ext = kwargs.pop('ext')
self.macro = macro.parse(self.ext._macro_context)
super().__init__(*args, **kwargs)
- def handleMatch(self, m):
- html = self.macro(m.group(2))
+ def handleMatch(self, m: re.Match[str], data: str) -> tuple[etree.Element | None, int | None, int | None]:
+ html = self.macro(m.group(1))
placeholder = self.md.htmlStash.store(html)
- return placeholder
+ return placeholder, m.start(0), m.end(0)
class ForgeLinkTreeProcessor(markdown.treeprocessors.Treeprocessor):
@@ -546,17 +547,10 @@ def run(self, text):
return s.render(walker)
-class AutolinkPattern(markdown.inlinepatterns.Pattern):
-
- def __init__(self, pattern, markdown_instance=None):
- super().__init__(pattern, markdown_instance)
- # override the complete regex, requiring the preceding text (.*?) to end
- # with whitespace or beginning of line "\s|^"
- self.compiled_re = re.compile(r"^(.*?\s|^)%s(.*?)$" % pattern,
- re.DOTALL | re.UNICODE)
+class AutolinkPattern(markdown.inlinepatterns.InlineProcessor):
- def handleMatch(self, mo):
- old_link = mo.group(2)
+ def handleMatch(self, m: re.Match[str], data: str) -> tuple[etree.Element | None, int | None, int | None]:
+ old_link = m.group(1)
result = etree.Element('a')
result.text = old_link
# since this is run before the builtin 'escape' processor, we have to
@@ -564,7 +558,7 @@ def handleMatch(self, mo):
for char in self.md.ESCAPED_CHARS:
old_link = old_link.replace('\\' + char, char)
result.set('href', old_link)
- return result
+ return result, m.start(0), m.end(0)
class ForgeMacroIncludePreprocessor(markdown.preprocessors.Preprocessor):
(allura) 03/04: [#8588] use *+ instead of * (possessive quantifiers) and regular re library. Seems like both *+ are not needed, but doing both for good measure
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8588
in repository https://gitbox.apache.org/repos/asf/allura.git
commit f689e79b641470bf7ffa694004424b0d3ff39b02
Author: Dave Brondsema <db...@slashdotmedia.com>
AuthorDate: Thu Nov 20 17:38:44 2025 -0500
[#8588] use *+ instead of * (possessive quantifiers) and regular re library. Seems like both *+ are not needed, but doing both for good measure
---
Allura/allura/lib/markdown_extensions.py | 6 ++----
Allura/allura/tests/test_globals.py | 3 ++-
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/Allura/allura/lib/markdown_extensions.py b/Allura/allura/lib/markdown_extensions.py
index 93a2904af..4579e1970 100644
--- a/Allura/allura/lib/markdown_extensions.py
+++ b/Allura/allura/lib/markdown_extensions.py
@@ -24,7 +24,6 @@
from urllib.parse import urljoin
-import regex
from tg import config
from bs4 import BeautifulSoup, MarkupResemblesLocatorWarning
import html5lib
@@ -49,11 +48,11 @@
# FORGE_LINK_RE copied from markdown pre 3.0's LINK_RE
# TODO: replace these with newer approach, see ForgeLinkPattern
-NOBRACKET = r'[^\]\[]*' # if not using regex-as-re-globally, must change "*" to {0,50} for performance mitigation
+NOBRACKET = r'[^\]\[]*+'
BRK = (
r'\[(' +
(NOBRACKET + r'(\[')*6 +
- (NOBRACKET + r'\])*')*6 +
+ (NOBRACKET + r'\])*+')*6 +
NOBRACKET + r')\]'
)
FORGE_LINK_RE = markdown.inlinepatterns.NOIMG + BRK + \
@@ -362,7 +361,6 @@ class ForgeLinkPattern(markdown.inlinepatterns.Pattern):
def __init__(self, *args, **kwargs):
self.ext = kwargs.pop('ext')
super().__init__(*args, **kwargs)
- self.compiled_re = regex.compile(self.compiled_re.pattern, flags=self.compiled_re.flags)
def handleMatch(self, m):
el = etree.Element('a')
diff --git a/Allura/allura/tests/test_globals.py b/Allura/allura/tests/test_globals.py
index c93abed71..04f788ec6 100644
--- a/Allura/allura/tests/test_globals.py
+++ b/Allura/allura/tests/test_globals.py
@@ -601,7 +601,8 @@ def test_markdown_invalid_classes_ids(self):
assert r == '<div class="markdown_content"><p><i class="fa fa-cog">gear icon</i></p></div>'
def test_markdown_extremely_slow(self):
- # regex-as-re-globally package mitigates this being slow; also see `NOBRACKET` comments
+ # this used to be extremely slow due to catastrophic backtracking in FORGE_LINK_RE / BRK regex composition
+ # initially mitigated by using the `regex` lib instead of `re` but now is mitigated by possessive quantifiers
g.markdown.convert(inspect.cleandoc('''bonjour, voila ce que j'obtient en voulant ajouter un utilisateur a un groupe de sécurite, que ce soit sur un groupe pre-existant, ou sur un groupe crée.
message d'erreur:
(allura) 02/04: [#8588] no longer need regex-as-re-globally hacky package
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8588
in repository https://gitbox.apache.org/repos/asf/allura.git
commit a8ad3b7ae44198a4d9a1cf43d60470bb3b77900f
Author: Dave Brondsema <db...@slashdotmedia.com>
AuthorDate: Thu Nov 20 17:22:01 2025 -0500
[#8588] no longer need regex-as-re-globally hacky package
---
requirements.in | 1 -
requirements.txt | 8 ++------
2 files changed, 2 insertions(+), 7 deletions(-)
diff --git a/requirements.in b/requirements.in
index 44b5786c0..31e2955c9 100644
--- a/requirements.in
+++ b/requirements.in
@@ -43,7 +43,6 @@ python-magic
python-oembed
pytz
qrcode
-regex-as-re-globally
requests
requests-oauthlib
# for taskd proc name switching
diff --git a/requirements.txt b/requirements.txt
index 572660703..6bef39099 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -250,12 +250,8 @@ pyyaml==6.0.3
# via pre-commit
qrcode==8.2
# via -r requirements.in
-regex==2025.9.18
- # via
- # regex-as-re-globally
- # textile
-regex-as-re-globally==0.0.6
- # via -r requirements.in
+regex==2025.9.18 ; implementation_name != 'pypy'
+ # via textile
repoze-lru==0.7
# via turbogears2
requests==2.32.5