[GitHub] [arrow-rs] tustvold opened a new issue, #2186: Use BitChunks in equal_bits

[GitBox <gi...@apache.org>]

tustvold opened a new issue, #2186:
URL: https://github.com/apache/arrow-rs/issues/2186

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   `arrow::array::equal::utils::equal_bits` currently is implemented as
   
   ```
   (0..len).all(|i| {
           bit_util::get_bit(lhs_values, lhs_start + i)
               == bit_util::get_bit(rhs_values, rhs_start + i)
       })
   ```
   
   This is likely extremely sub-optimal, performing a large number of operations for every bit
   
   **Describe the solution you'd like**
   
   Something like the below, not tested, should be significantly faster
   ```
   let lhs = BitChunks::new(lhs_values, lhs_start, len);
   let rhs = BitChunks::new(rhs_values, rhs_start, len);
   
   for (a, b) in lhs.iter().zip(rhs.iter()) {
       if a != b {
           return false
       }
   }
   
   lhs.remainder_bits() == rhs.remainder_bits()
   ```
   
   **Describe alternatives you've considered**
   
   We could not do this
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold closed issue #2186: Use BitChunks in equal_bits

[GitBox <gi...@apache.org>]

tustvold closed issue #2186: Use BitChunks in equal_bits
URL: https://github.com/apache/arrow-rs/issues/2186


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org