[GitHub] [arrow-datafusion] alamb commented on a change in pull request #2118: Fix case evaluation with nulls

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #2118:
URL: https://github.com/apache/arrow-datafusion/pull/2118#discussion_r837648566



##########
File path: datafusion/physical-expr/src/physical_expr.rs
##########
@@ -87,6 +87,10 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug {
 fn scatter(mask: &BooleanArray, truthy: &dyn Array) -> Result<ArrayRef> {
     let truthy = truthy.data();
 
+    // update the mask so that any null null values become false
+    // (SlicesIterator doesn't respect nulls)
+    let mask = and_kleene(mask, &is_not_null(mask)?)?;

Review comment:
       Here is the fix for the bug  -- convert NULLs --> false

##########
File path: datafusion/physical-expr/src/physical_expr.rs
##########
@@ -109,8 +113,8 @@ fn scatter(mask: &BooleanArray, truthy: &dyn Array) -> Result<ArrayRef> {
         filled = end;
     });
     // the remaining part is falsy
-    if filled < truthy.len() {
-        mutable.extend_nulls(truthy.len() - filled);
+    if filled < mask.len() {

Review comment:
       this is a bug I found while writing some more tests (namely that the output is supposed to be the size of `mask` not the size of `truthy` I think




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org