[GitHub] [arrow-datafusion] isidentical opened a new issue, #3762: Allow specialized physical functions to provide hints for the array adapter

[GitBox <gi...@apache.org>]

isidentical opened a new issue, #3762:
URL: https://github.com/apache/arrow-datafusion/issues/3762

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   We currently have a specialization system (used by the `regex_replace` at the moment, but can be extended to similar functions) that always produces functions with adapters, but sometimes the newly created functions might not need padded arrays for scalars (`_regexp_replace_static_pattern_replace` only needs an array of length-1 for the last three arguments). Instead of requiring the specialized functions to implement APIs that process raw `ColumnarValue`'s and provides multiple implementations (because not all arguments can be hinted/known), we can extend the array->scalar adapter (`make_scalar_function`) to also take a list of hints and produce more optimized inputs.
   
   This is something that was noticed during the [`regex_replace` speed-up](https://github.com/apache/arrow-datafusion/issues/3518#issuecomment-1272095620) discussion, where on some inputs the time spent on `into_array()` can be as high as %25 of the whole execution.
   
   **Describe the solution you'd like**
   A new API that can take some hints about what the decorated function expects in terms.
   
   **Describe alternatives you've considered**
   Not doing this and writing multiple implementations for specialized functions that directly consumes `ColumnarValue`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb closed issue #3762: Allow specialized physical functions to provide hints for the array adapter

[GitBox <gi...@apache.org>]

alamb closed issue #3762: Allow specialized physical functions to provide hints for the array adapter
URL: https://github.com/apache/arrow-datafusion/issues/3762


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org