You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "kawadakk (via GitHub)" <gi...@apache.org> on 2024/10/10 07:32:05 UTC
[I] `ListArray::try_new` rejects `{Union,Dictionary}Array` incorrectly if `field` is not nullable [arrow-rs]
kawadakk opened a new issue, #6538:
URL: https://github.com/apache/arrow-rs/issues/6538
**Describe the bug**
`ListArray::try_new()` uses `Array::is_nullable()` to check the presence of null values if `field` is not marked as nullable.
<https://github.com/apache/arrow-rs/blob/5508978a3c5c4eb65ef6410e097887a8adaba38a/arrow-array/src/array/list_array.rs#L191-L197>
While this works for most types, this can cause a false positive for `DictionaryArray` and `UnionArray` because `Array::is_nullable()` is allowed to return `true` if it's expensive to prove the absence of logical nulls.
<https://github.com/apache/arrow-rs/blob/5508978a3c5c4eb65ef6410e097887a8adaba38a/arrow-array/src/array/mod.rs#L287-L291>
**To Reproduce**
```rust
let offsets = OffsetBuffer::new(vec![0, 1, 4, 5].into());
let mut builder = UnionBuilder::new_dense();
builder.append::<Int32Type>("a", 1).unwrap();
builder.append::<Int32Type>("b", 2).unwrap();
builder.append::<Int32Type>("b", 3).unwrap();
builder.append::<Int32Type>("a", 4).unwrap();
builder.append::<Int32Type>("a", 5).unwrap();
let values = builder.build().unwrap();
let field = Arc::new(Field::new("element", values.data_type().clone(), false));
ListArray::new(field.clone(), offsets, Arc::new(values), None);
```
This produces:
```
thread 'foo' panicked at foo.rs:10:46:
called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Non-nullable field of ListArray \"element\" cannot contain nulls")
```
**Expected behavior**
Successful execution
**Additional context**
<!--
Add any other context about the problem here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `ListArray::try_new` rejects `{Union,Dictionary}Array` incorrectly if `field` is not nullable [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #6538:
URL: https://github.com/apache/arrow-rs/issues/6538#issuecomment-2404663685
This looks to have been introduced for UnionArray in https://github.com/apache/arrow-rs/pull/6303. This isn't so much of a problem for DictionaryArray given how rare nulls in dictionary values are
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `ListArray::try_new` rejects `{Union,Dictionary}Array` incorrectly if `field` is not nullable [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey closed issue #6538: `ListArray::try_new` rejects `{Union,Dictionary}Array` incorrectly if `field` is not nullable
URL: https://github.com/apache/arrow-rs/issues/6538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org