You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2023/10/29 08:08:37 UTC
[I] `try_from_array` returns incorrect value for NULL value in ListArray [arrow-datafusion]
viirya opened a new issue, #7968:
URL: https://github.com/apache/arrow-datafusion/issues/7968
### Describe the bug
For NULL value in ListArray, `try_from_array` produces `ScalarValue::List(new_null_array(&DataType::Null, 0))` which isn't consistent with the case when invoking `try_from_array` on valid value of `ListArray`.
Fro valid value, `try_from_array` produces a `ListArray` containing single element of the specified value in provided `ListArray`. For example, if input array is `ListArray[[1, 2, 3], [4, 5, 6]]]`, `try_from_array(arr, 0)` will produce `ListArray[[1, 2, 3]]`.
For NULL value, `try_from_array` currently produces a null array of`DataType::Null` type. Not only it is not a `ListArray`, the data type is not the element type of provided `ListArray` too.
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `try_from_array` returns incorrect value for NULL value in ListArray [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #7968:
URL: https://github.com/apache/arrow-datafusion/issues/7968#issuecomment-1784083797
Thank you @viirya -- the way `ARRAY` datatypes handle null now is non ideal -- they use `DataType::Null` inconsistenty rather than `DataType::List` with an appropriate Null literal -- perhaps this is related to https://github.com/apache/arrow-datafusion/issues/7142
cc @jayzhan211 -- maybe https://github.com/apache/arrow-datafusion/issues/7142 would be a good simplification to try and work on in the context of moving DataFusion array support forward
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `try_from_array` returns incorrect value for NULL value in ListArray [arrow-datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on issue #7968:
URL: https://github.com/apache/arrow-datafusion/issues/7968#issuecomment-1784093704
> Thank you @viirya -- the way `ARRAY` datatypes handle null now is non ideal -- they use `DataType::Null` inconsistenty rather than `DataType::List` with an appropriate Null literal -- perhaps this is related to #7142
>
> cc @jayzhan211 -- maybe #7142 would be a good simplification to try and work on in the context of moving DataFusion array support forward
Great, I was thinking of it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Incorrect and inconsistency handling of NULL values in `ScalarValue` for ListArray [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #7968: Incorrect and inconsistency handling of NULL values in `ScalarValue` for ListArray
URL: https://github.com/apache/arrow-datafusion/issues/7968
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `try_from_array` returns incorrect value for NULL value in ListArray [arrow-datafusion]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #7968:
URL: https://github.com/apache/arrow-datafusion/issues/7968#issuecomment-1784245104
No, this issue is due to incorrect and inconsistency handling of NULL value on `Scalar::List`. It doesn't involve array functions. Please see the proposed PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Incorrect and inconsistency handling of NULL values in `ScalarValue` for ListArray [arrow-datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on issue #7968:
URL: https://github.com/apache/arrow-datafusion/issues/7968#issuecomment-1784286323
> No, this is another issue, I think.
>
> This issue is due to incorrect and inconsistency handling of NULL value on `ScalarValue::List`. It doesn't involve array functions. I added some description to this issue. Please also see the proposed PR.
We won't have `Null` but something like `Int32(None)` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org