You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/07/14 17:44:32 UTC
[PR] perf: optimize spark_regexp_extract_all in datafusion-comet-spark-expr [datafusion-comet]
andygrove opened a new pull request, #4923:
URL: https://github.com/apache/datafusion-comet/pull/4923
> This PR was created by an LLM as a draft PR. I will mark it as ready for review after human review.
## Which issue does this PR close?
N/A — autonomous exploratory PR.
## Rationale for this change
Reuse a single CaptureLocations across all matches and rows instead of letting captures_iter heap-allocate capture slots per match, and use find_iter when only the whole match (idx=0) is needed.
## What changes are included in this PR?
Reuse a single CaptureLocations across all matches and rows instead of letting captures_iter heap-allocate capture slots per match, and use find_iter when only the whole match (idx=0) is needed.
## How are these changes tested?
Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs `main`).
Benchmark (criterion):
- capture_group: 24.244% faster (base 1052449ns -> cand 797294ns)
- whole_match: 61.571% faster (base 1046798ns -> cand 402278ns)
Full criterion output:
```text
regexp_extract_all/capture_group
time: [797.09 µs 797.74 µs 798.40 µs]
change: [−24.419% −24.244% −24.094%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
regexp_extract_all/whole_match
time: [402.08 µs 402.21 µs 402.42 µs]
change: [−61.621% −61.571% −61.519%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
3 (3.00%) high mild
4 (4.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: optimize spark_regexp_extract_all in datafusion-comet-spark-expr [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed pull request #4923: perf: optimize spark_regexp_extract_all in datafusion-comet-spark-expr
URL: https://github.com/apache/datafusion-comet/pull/4923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org