You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/07/28 18:04:45 UTC

[I] rlike: consider arrow regexp_is_match kernel and fix non-StringArray panic [datafusion-comet]

andygrove opened a new issue, #5102:
URL: https://github.com/apache/datafusion-comet/issues/5102

   ### What is the problem the feature request solves?
   
   `RLike::is_match` in `native/spark-expr/src/predicate_funcs/rlike.rs` loops over a `StringArray` calling `regex::Regex::is_match` per row into a `BooleanBuilder`. Arrow's `regexp_is_match` kernel does the same thing: it is built on the same Rust `regex` crate, so the documented divergence from Java regex semantics is unchanged, and null propagation matches.
   
   The kernel would also make the array path more robust: it supports Utf8/LargeUtf8/Utf8View, while the current code hard-`expect`s a `StringArray` and would panic on other string layouts.
   
   ### Describe the potential solution
   
   Replace the row loop with `regexp_is_match(array, &StringArray::new_scalar(pattern), None)`.
   
   Trade-offs to resolve:
   - The struct currently pre-compiles the `Regex` once at plan build; the kernel takes a pattern string and compiles per batch. For complex patterns this is a measurable regression; there is no arrow API accepting a pre-built `Regex`. Either accept per-batch compilation or keep the loop and only fix the downcast robustness.
   - The dictionary fast path (run on dictionary values then `take`) and the scalar-input branch stay as wrappers.
   
   ### Additional context
   
   Found during an audit of native code that replicates existing arrow-rs kernels.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] rlike: consider arrow regexp_is_match kernel and fix non-StringArray panic [datafusion-comet]

Posted by "sunchao (via GitHub)" <gi...@apache.org>.
sunchao closed issue #5102: rlike: consider arrow regexp_is_match kernel and fix non-StringArray panic
URL: https://github.com/apache/datafusion-comet/issues/5102


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] rlike: consider arrow regexp_is_match kernel and fix non-StringArray panic [datafusion-comet]

Posted by "sam-1112 (via GitHub)" <gi...@apache.org>.
sam-1112 commented on issue #5102:
URL: https://github.com/apache/datafusion-comet/issues/5102#issuecomment-5150827046

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org