You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "sdf-jkl (via GitHub)" <gi...@apache.org> on 2025/10/31 20:42:19 UTC
[I] Flatten using the wrong validity buffer for LargeList(LargeList) [datafusion]
sdf-jkl opened a new issue, #18419:
URL: https://github.com/apache/datafusion/issues/18419
### Describe the bug
While working on #18363 I saw an inconsistency within match arm logic for different inner List types for `LargeList`.
```diff
-let (inner_field, inner_offsets, inner_values, nulls) = // in LargeList(LargeList)
+let (inner_field, inner_offsets, inner_values, _) = // in LargeList(List)
as_list_array(&values)?.clone().into_parts();
let flattened_array = GenericListArray::<i64>::new(
inner_field,
offsets,
inner_values,
nulls, <------- the validity buffer is used here
);
```
### To Reproduce
To reproduce the error, I fixed an existing `sqllogictest` to have `LargeList(nullable LargeList(nullable Int64))`
This way when we generate a `flattened_array` the validity buffer's len and values difference will cause an error.
```
External error: task 26 panicked with message "called `Result::unwrap()` on an `Err` value: InvalidArgumentError(\"Incorrect length of null buffer for LargeListArray, expected 1 got 5\")"
```
### Expected behavior
The expected behavior is to pass the new `sqllogictest`
```
flatten(arrow_cast(make_array([1], null, [2, 3], [null], make_array(4, null, 5)), 'LargeList(LargeList(Int64))')),
----
[1, NULL, 2, 3, NULL, 4, NULL, 5]
```
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Flatten using the wrong validity buffer for LargeList(LargeList) [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #18419: Flatten using the wrong validity buffer for LargeList(LargeList)
URL: https://github.com/apache/datafusion/issues/18419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org