You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "dan-robertson (via GitHub)" <gi...@apache.org> on 2025/09/12 15:07:46 UTC
[I] Parsing a regex with groups that may match empty strings seems to be incorrect sometimes [lucene]
dan-robertson opened a new issue, #15182:
URL: https://github.com/apache/lucene/issues/15182
### Description
Consider a regex of the form `a(b|)c`. This should match `ac` and `abc`. Some regexes of this form seem to behave incorrectly. To try to make a slightly more concrete or realistic example, one might want to search some logs for something like `[a-z-]*(|-prod|-main) `.
I cobbled together some tests against a recent version of the lucene repo. I have three docs which respectively contain:
- `foo-bar-baz`
- `foo--baz`
- `foo-test-baz`
And then I spell a regex to match only the first two docs a bunch of different ways:
1. `.*foo-(bar|)-baz.*` - IllegalArgumentException: expected ')' at position 18
2. `.*foo-(|bar)-baz.*` - 0 matches but 2 expected
3. `.*(foo-(bar|)-baz).*` - IllegalArgumentException: expected ')' at position 20
4. `.*(foo-(|bar)-baz).*` - 0 matches but 2 expected
5. `.*foo-(bar|())-baz.*` - 2 matches
6. `.*foo-(bar|()?)-baz.*` - 2 matches
7. `.*foo-(bar|#?)-baz.*` - 2 matches
8. `.*(foo-(bar|())-baz).*` - 2 matches
9. `.*(foo-(bar|()?)-baz).*` - 2 matches
10. `.*(foo-(bar|#?)-baz).*` - 2 matches
11. `.*foo-(bar)?-baz.*` - 2 matches
The first four cases seem incorrect to me.
I came to this investigation after some problems with elasticsearch (v8.12.2, using lucene 9.9.2) where regexes following pattern number 5 also failed. Maybe that is some useful context.
### Version and environment details
I added tests by modifying `lucene/core/src/test/org/apache/lucene/search/TestRegexpQuery.java` with a base revision of `cd1a4ecc9ead8e06b08f3bc2016297525f65b37c`.
Here are other details in case they are relevant:
OS: linux (el8 with a 6.12.39 kernel)
Java: openjdk version "24.0.1" 2025-04-15
This is on an x86_64 box.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Parsing a regex with groups that may match empty strings seems to be incorrect sometimes [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on issue #15182:
URL: https://github.com/apache/lucene/issues/15182#issuecomment-3286675220
Other regex engines do a lot of other things that this one does not. I think the syntax should continue to match https://www.brics.dk/automaton/ for our own sanity, and ability to maintain it.
We've added custom FLAGS only to provide behavior such as caseless matching.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Parsing a regex with groups that may match empty strings seems to be incorrect sometimes [lucene]
Posted by "dan-robertson (via GitHub)" <gi...@apache.org>.
dan-robertson commented on issue #15182:
URL: https://github.com/apache/lucene/issues/15182#issuecomment-3286165141
Hmm, reading the docs of RegExp, I think you’re right, but those docs also imply that `(|b)` and `(b|)` should be treated similarly (either both allowed or both disallowed) which is not what we observe above.
Nevertheless, it seems like not accepting such expressions is undesirable considering that most otherwise similar regexp syntaxes do accept such expressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Parsing a regex with groups that may match empty strings seems to be incorrect sometimes [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on issue #15182:
URL: https://github.com/apache/lucene/issues/15182#issuecomment-3286058098
> Consider a regex of the form `a(b|)c`. This should match `ac` and `abc`.
This isn't true, this isn't the syntax supported by the regex engine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org