You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "sam-1112 (via GitHub)" <gi...@apache.org> on 2026/08/04 08:54:28 UTC
[I] Add Criterion bench for native RLike (is_match loop) [datafusion-comet]
sam-1112 opened a new issue, #5246:
URL: https://github.com/apache/datafusion-comet/issues/5246
### What is the problem the feature request solves?
Native RLike documents that `is_match` hand-rolls a loop over the input string array instead of calling Arrow's `regexp_is_match(_scalar)`, which recompiles the pattern on every batch (see arrow-string `regexp.rs`). #5102 (and follow-up PRs) rewrote that loop (e.g. `iter`/`map`/`collect`). The rationale lives in docs, but nothing in `native/spark-expr/benches/` measures it, so future changes to this hot path can evaporate without a baseline.
### Describe the potential solution
Add `native/spark-expr/benches/rlike.rs` alongside the existing `regexp_extract.rs` Criterion bench, and register it in `native/spark-expr/Cargo.toml` (`[[bench]]`, `harness = false`).
Suggested coverage (mirroring review discussion):
- Scalar pattern, Utf8 subject array (common path through `is_match`)
- Mix of matches / non-matches / nulls
- Optionally Dictionary(Int32, Utf8), since that is the reachable
dictionary shape in Comet today
This does not need to land in the string-layout / dictionary fix PR;
it is follow-up so the rewritten loop has a measurable baseline.
### Additional context
Requested in review on the RLike string-layout PR: the null-check removal is likely lost in noise next to `Regex::is_match`, so this is not a merge blocker, but worth tracking rather than dropping.
Reference: `native/spark-expr/benches/regexp_extract.rs`
Related PR: https://github.com/apache/datafusion-comet/pull/5215#pullrequestreview-4849576694
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Criterion bench for native RLike (is_match loop) [datafusion-comet]
Posted by "sam-1112 (via GitHub)" <gi...@apache.org>.
sam-1112 closed issue #5246: Add Criterion bench for native RLike (is_match loop)
URL: https://github.com/apache/datafusion-comet/issues/5246
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Criterion bench for native RLike (is_match loop) [datafusion-comet]
Posted by "dwsmith1983 (via GitHub)" <gi...@apache.org>.
dwsmith1983 commented on issue #5246:
URL: https://github.com/apache/datafusion-comet/issues/5246#issuecomment-5637280454
#5598 (2949fd0d2) added native/spark-expr/benches/rlike.rs and registered it under `[[bench]]` in native/spark-expr/Cargo.toml. It benchmarks the scalar-pattern `is_match` path over Utf8 input at several null ratios, which is the baseline this issue asked for; the optional dictionary-input case is not included but was listed as optional. Suggest closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Criterion bench for native RLike (is_match loop) [datafusion-comet]
Posted by "sam-1112 (via GitHub)" <gi...@apache.org>.
sam-1112 commented on issue #5246:
URL: https://github.com/apache/datafusion-comet/issues/5246#issuecomment-5637752529
Thanks for checking. I agree that #5598 covers the baseline this issue was intended to add. The dictionary case was optional, so I'll close this as completed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org