You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Neville Dipale (Jira)" <ji...@apache.org> on 2020/12/29 09:08:00 UTC
[jira] [Created] (ARROW-11061) [Rust] Validate array properties
against schema
Neville Dipale created ARROW-11061:
--------------------------------------
Summary: [Rust] Validate array properties against schema
Key: ARROW-11061
URL: https://issues.apache.org/jira/browse/ARROW-11061
Project: Apache Arrow
Issue Type: Improvement
Reporter: Neville Dipale
We have a problem when it comes to nested arrays, where one could create a <list[null]<primitive[non-null]>> where the array fields can't be null, but the list can have null slots.
This creates a lot of work when working with such nested arrays, because we have to create work-arounds to account for this, and take unnecessarily slower paths.
I propose that we prevent this problem at the source, by:
* checking that a batch can't be created with arrays that have incompatible null contracts
* preventing list and struct children from being non-null if any descendant of such children are null (might be less of an issue for structs)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)