You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/23 13:44:01 UTC
[GitHub] [arrow-rs] jhorstmann opened a new issue, #1611: Validate nullable flag on StructArray fields
jhorstmann opened a new issue, #1611:
URL: https://github.com/apache/arrow-rs/issues/1611
**Describe the bug**
Extracted from [a discussion on the equals kernels](https://github.com/apache/arrow-rs/pull/1589#issuecomment-1107444994)
It is currently very easy to get inconsistant state between the `nullable` flag on `StructArray` fields and the presence of their validity bitmap. This can lead to very difficult to debug issues, as for example the equality comparison of `StructArray` includes this flag, but it is not printed by the `Debug` impl, for example in assertions.
**To Reproduce**
Example of a confusing test:
```rust
#[test]
fn compare_nullable_struct_array_fields() {
let array1 = StructArray::try_from(vec![(
"a",
Arc::new(Int32Array::from(vec![1, 2, 3])) as ArrayRef,
)])
.unwrap();
let array2 = StructArray::try_from(vec![(
"a",
Arc::new(Int32Array::from(vec![Some(1), Some(2), Some(3)])) as ArrayRef,
)])
.unwrap();
// assert_eq!(array1.data_type(), array2.data_type());
assert_eq!(&array1, &array2);
}
```
```
panicked at 'assertion failed: `(left == right)`
left: `StructArray
[
-- child 0: "a" (Int32)
PrimitiveArray<Int32>
[
1,
2,
3,
]
]`,
right: `StructArray
[
-- child 0: "a" (Int32)
PrimitiveArray<Int32>
[
1,
2,
3,
]
]`'
```
**Expected behavior**
- if a field is not nullable it should not have a validity bitmap
- a `StructArray` itself should only have a validity bitmap if all its fields are nullable
- maybe `StructArray::from(Vec<(&str, ArrayRef>)` should always set the `nullable` flag?
A clear and concise description of what you expected to happen.
**Additional context**
Alternatives could be to ignore the `nullable` flag when comparing arrays
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on issue #1611: Validate nullable flag on StructArray fields
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #1611:
URL: https://github.com/apache/arrow-rs/issues/1611#issuecomment-1471878188
Closed by https://github.com/apache/arrow-rs/pull/3244
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold closed issue #1611: Validate nullable flag on StructArray fields
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #1611: Validate nullable flag on StructArray fields
URL: https://github.com/apache/arrow-rs/issues/1611
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org