[GitHub] [arrow-rs] alamb opened a new issue #1371: Improve performance of `compare_dict_op`

[GitBox <gi...@apache.org>]

alamb opened a new issue #1371:
URL: https://github.com/apache/arrow-rs/issues/1371


   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   https://github.com/apache/arrow-rs/pull/1330 introduced a significant cleanup for comparing dictionary arrays 🎉 
   
   However, it also may reduce performance as it indirectly calls `value()` repeatedly in an array which checks the bounds. This bounds check is necessary for known good dictionary arrays (where all the value entries are known to be valid indexes into the values array)
   
   **Describe the solution you'd like**
   
   As suggested by @viirya in https://github.com/apache/arrow-rs/pull/1330#issuecomment-1043252680:
   1. A benchmark showing the speed of comparing dictionary arrays
   2. Implement something like `unsafe take_iter_unchecked()` that is used in `compare_dict_op`
   3. Demonstrate that the benchmark is faster with the specialized approach
   
   **Additional context**
   All the context is on #1330 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb closed issue #1371: Improve performance of `compare_dict_op`

[GitBox <gi...@apache.org>]

alamb closed issue #1371:
URL: https://github.com/apache/arrow-rs/issues/1371


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org