[GitHub] [arrow-rs] YjyJeff opened a new issue, #3031: Nullif with incorrect valid_count

[GitBox <gi...@apache.org>]

YjyJeff opened a new issue, #3031:
URL: https://github.com/apache/arrow-rs/issues/3031

   **Describe the bug**
   The [`nullif`] function in `compute/kernels/boolean.rs` has incorrect `valid_count` when the left's null_buffer is None. 
   
   [`nullif`]: https://github.com/apache/arrow-rs/blob/master/arrow/src/compute/kernels/boolean.rs#L525
   
   **To Reproduce**
   ```rust 
   let a = Int32Array::from(vec![Some(15), Some(7), Some(8), Some(1), Some(9)]);
   let comp = BooleanArray::from(vec![Some(false), None, Some(true), Some(false), None]);
   let res = nullif(&a, &comp).unwrap(); // panic here
   ```
   **Why it happens**
   `bitwise_unary_op_helper` will apply the `op` to the remainder bytes, the problem happens here. `valid_count` will take the ones that should be ignored into consideration.  
   
   **Solution**
   ```rust
   let left = Bitmap::new(len);
   bitwise_bin_op_helper(left.buffer(), 0, &right, right_offset, len, |l, r| {
       let t = l & !r;
       valid_count += t.count_ones() as usize;
       t
   })
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb commented on issue #3031: Nullif with incorrect valid_count

[GitBox <gi...@apache.org>]

alamb commented on issue #3031:
URL: https://github.com/apache/arrow-rs/issues/3031#issuecomment-1312141444

   `label_issue.py` automatically added labels {'arrow'} from #3034


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold closed issue #3031: Nullif with incorrect valid_count

[GitBox <gi...@apache.org>]

tustvold closed issue #3031: Nullif with incorrect valid_count
URL: https://github.com/apache/arrow-rs/issues/3031


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org