You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Neville Dipale (Jira)" <ji...@apache.org> on 2020/11/22 10:16:00 UTC

[jira] [Created] (ARROW-10684) [Rust] Logical equality should consider parent array nullability

Neville Dipale created ARROW-10684:
--------------------------------------

             Summary: [Rust] Logical equality should consider parent array nullability
                 Key: ARROW-10684
                 URL: https://issues.apache.org/jira/browse/ARROW-10684
             Project: Apache Arrow
          Issue Type: Bug
          Components: Rust
    Affects Versions: 2.0.0
            Reporter: Neville Dipale


When creating a struct array with a primitive child array, it is possible for the child to be non-nullable, while its parent struct array is nullable.

In this scenario, the child array's slots where the parent is null, become invalidated, such that an array with [1, 2, 3] having slot 2 being null, should be interpreted as [1, 0, 3].

This issue becomes evident in Parquet roundtrip tests, as we end up not able to correctly compare nested structures that have non-null children.

The specification caters for the above behaviour, see [http://arrow.apache.org/docs/format/Columnar.html#struct-layout] .

When a struct has nulls, its child array(s) nullability is subject to the parent struct.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)