You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "sdf-jkl (via GitHub)" <gi...@apache.org> on 2025/10/31 20:42:19 UTC

[I] Flatten using the wrong validity buffer for LargeList(LargeList) [datafusion]

sdf-jkl opened a new issue, #18419:
URL: https://github.com/apache/datafusion/issues/18419

   ### Describe the bug
   
   While working on #18363 I saw an inconsistency within match arm logic for different inner List types for `LargeList`.
   ```diff
   -let (inner_field, inner_offsets, inner_values, nulls) =  // in LargeList(LargeList)
   +let (inner_field, inner_offsets, inner_values, _) = // in LargeList(List)
                           as_list_array(&values)?.clone().into_parts();
   
   let flattened_array = GenericListArray::<i64>::new(
                           inner_field,
                           offsets,
                           inner_values,
                           nulls, <------- the validity buffer is used here
                       );
   ```
   
   ### To Reproduce
   
   To reproduce the error, I fixed an existing `sqllogictest` to have `LargeList(nullable LargeList(nullable Int64))`
   This way when we generate a `flattened_array` the validity buffer's len and values difference will cause an error.
   ```
   External error: task 26 panicked with message "called `Result::unwrap()` on an `Err` value: InvalidArgumentError(\"Incorrect length of null buffer for LargeListArray, expected 1 got 5\")"
   ```
   
   ### Expected behavior
   
   The expected behavior is to pass the new `sqllogictest`
   ```
   flatten(arrow_cast(make_array([1], null, [2, 3], [null], make_array(4, null, 5)), 'LargeList(LargeList(Int64))')),
   ----
   [1, NULL, 2, 3, NULL, 4, NULL, 5]
   ```
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Flatten using the wrong validity buffer for LargeList(LargeList) [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #18419: Flatten using the wrong validity buffer for LargeList(LargeList)
URL: https://github.com/apache/datafusion/issues/18419


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org