[GitHub] [arrow-rs] jhorstmann opened a new issue #997: lexicographical_partition_ranges does not need to materialize indices

[GitBox <gi...@apache.org>]

jhorstmann opened a new issue #997:
URL: https://github.com/apache/arrow-rs/issues/997


   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   the [`lexicographical_partition_ranges` function](https://github.com/apache/arrow-rs/blob/9fb2a5fab32d50feacc1f600b45606ba402017df/arrow/src/compute/kernels/partition.rs#L65) currently materializes a vector of incrementing indices, which it then uses to call `slice::partition_point`. The benefit is that we can directly reuse a standard rust function, the downside is that the vector of indices can be quite large and that it adds another indirection. I think by duplicating the functionality of `partition_points` to work with a range we could avoid this allocation and indirection.
   
   **Describe the solution you'd like**
   Reimplement `partition_points` to work with a range. The implementation in the standard library calls a relatively simple binary search which could be simplified for this usecase. The predicate could be used directly so there is no need for using the `Ordering` enum while searching.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb closed issue #997: lexicographical_partition_ranges does not need to materialize indices

[GitBox <gi...@apache.org>]

alamb closed issue #997:
URL: https://github.com/apache/arrow-rs/issues/997


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org