You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "neilconway (via GitHub)" <gi...@apache.org> on 2026/09/29 01:14:26 UTC
[PR] perf: Reuse regexp capture locations [arrow-rs]
neilconway opened a new pull request, #11274:
URL: https://github.com/apache/arrow-rs/pull/11274
# Which issue does this PR close?
- N/A
# Rationale for this change
When `regexp_match` is used with a scalar pattern, we previously allocated a new `CaptureLocations` value for every row that was matched against. We can do better by allocating the storage once and then reusing it for every row via the `Regex::captures_read()` API.
Benchmarks: (M4 Max)
- Utf8, matching: 1,463.97 → 1,070.79 µs — 26.9% less time.
- Utf8, mixed: 829.28 → 607.46 µs — 26.7% less time.
- Utf8, no matches: 299.01 → 148.47 µs — 50.3% less time.
- Utf8View, matching: 1,496.74 → 1,159.14 µs — 22.6% less time.
- Utf8View, mixed: 849.78 → 649.64 µs — 23.6% less time.
- Utf8View, no matches: 253.79 → 151.30 µs — 40.4% less time.
# What changes are included in this PR?
* Switch to `Regex::captures_read` in `regexp_scalar_match` and `regexp_scalar_match_utf8view`
* Add benchmark
* Improve test coverage of scalar regexp matching
# Are these changes tested?
Yes. New tests added to increase test coverage of this function overall.
# Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: Reuse capture locations in `regexp_match` [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #11274:
URL: https://github.com/apache/arrow-rs/pull/11274#issuecomment-5915401095
Thanks again @neilconway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: Reuse capture locations in `regexp_match` [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #11274:
URL: https://github.com/apache/arrow-rs/pull/11274#discussion_r4144910389
##########
arrow-string/src/regexp.rs:
##########
@@ -332,6 +332,7 @@ fn get_scalar_pattern_flag_utf8view<'a>(
macro_rules! process_regexp_match {
($array:expr, $regex:expr, $list_builder:expr) => {
+ let mut capture_locations = None;
Review Comment:
Can we add a small comment about the use of this (that it is a cache to reuse the locations)?
##########
arrow-string/src/regexp.rs:
##########
@@ -342,19 +343,25 @@ macro_rules! process_regexp_match {
$list_builder.values().append_value("");
$list_builder.append(true);
}
- Some(value) => match $regex.captures(value) {
- Some(caps) => {
- let mut iter = caps.iter();
- if caps.len() > 1 {
- iter.next();
- }
- for m in iter.flatten() {
- $list_builder.values().append_value(m.as_str());
+ Some(value) => {
+ let locations =
+ capture_locations.get_or_insert_with(|| $regex.capture_locations());
+ match $regex.captures_read(locations, value) {
Review Comment:
I think this is the key change -- to use https://docs.rs/regex/latest/regex/struct.Regex.html#method.captures_read instead of `captures`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: Reuse capture locations in `regexp_match` [arrow-rs]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on code in PR #11274:
URL: https://github.com/apache/arrow-rs/pull/11274#discussion_r4145948993
##########
arrow-string/src/regexp.rs:
##########
@@ -342,19 +343,25 @@ macro_rules! process_regexp_match {
$list_builder.values().append_value("");
$list_builder.append(true);
}
- Some(value) => match $regex.captures(value) {
- Some(caps) => {
- let mut iter = caps.iter();
- if caps.len() > 1 {
- iter.next();
- }
- for m in iter.flatten() {
- $list_builder.values().append_value(m.as_str());
+ Some(value) => {
+ let locations =
+ capture_locations.get_or_insert_with(|| $regex.capture_locations());
+ match $regex.captures_read(locations, value) {
Review Comment:
Yep!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: Reuse capture locations in `regexp_match` [arrow-rs]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on code in PR #11274:
URL: https://github.com/apache/arrow-rs/pull/11274#discussion_r4145966293
##########
arrow-string/src/regexp.rs:
##########
@@ -332,6 +332,7 @@ fn get_scalar_pattern_flag_utf8view<'a>(
macro_rules! process_regexp_match {
($array:expr, $regex:expr, $list_builder:expr) => {
+ let mut capture_locations = None;
Review Comment:
Thanks, done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: Reuse capture locations in `regexp_match` [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #11274:
URL: https://github.com/apache/arrow-rs/pull/11274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org