You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "codephage2020 (via GitHub)" <gi...@apache.org> on 2025/07/03 05:55:30 UTC
[PR] Fix union slice logical_nulls length [arrow-rs]
codephage2020 opened a new pull request, #7855:
URL: https://github.com/apache/arrow-rs/pull/7855
# Which issue does this PR close?
- Closes #7647.
# Rationale for this change
Fixes the incorrect length of logical_nulls for sliced single-field dense union arrays.
# What changes are included in this PR?
N/A
# Are these changes tested?
add test slice_union_array_single_field();
# Are there any user-facing changes?
N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#issuecomment-3048853812
Thanks again @codephage2020
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#discussion_r2187136506
##########
arrow-array/src/array/union_array.rs:
##########
@@ -781,13 +781,23 @@ impl Array for UnionArray {
};
if fields.len() <= 1 {
- return self
+ if let Some(logical_nulls) = self
.fields
.iter()
.flatten()
.map(Array::logical_nulls)
.next()
- .flatten();
+ .flatten()
+ {
+ if logical_nulls.len() != self.len() {
Review Comment:
I think you can make this significiantly simpler and just add a single check for dense nulls. This works for me locally:
```rust
return self
.fields
.iter()
.flatten()
.map(Array::logical_nulls)
.next()
.flatten()
.map(|logical_nulls| {
if self.is_dense() {
// Dense union: gather nulls based on offsets
self.gather_nulls(vec![(0, logical_nulls)]).into()
} else {
// Sparse union: use the logical nulls directly
logical_nulls
}
})
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#discussion_r2183235866
##########
arrow-array/src/array/union_array.rs:
##########
@@ -781,13 +781,23 @@ impl Array for UnionArray {
};
if fields.len() <= 1 {
- return self
+ if let Some(logical_nulls) = self
.fields
.iter()
.flatten()
.map(Array::logical_nulls)
.next()
- .flatten();
+ .flatten()
+ {
+ if logical_nulls.len() != self.len() {
Review Comment:
it seems like this can only happen for Dense unions -- I think the code would be clearer if it it explicitly had two paths, one (the existing code) for dense unions and one for sparse unions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "codephage2020 (via GitHub)" <gi...@apache.org>.
codephage2020 commented on code in PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#discussion_r2184186509
##########
arrow-array/src/array/union_array.rs:
##########
@@ -781,13 +781,23 @@ impl Array for UnionArray {
};
if fields.len() <= 1 {
- return self
+ if let Some(logical_nulls) = self
.fields
.iter()
.flatten()
.map(Array::logical_nulls)
.next()
- .flatten();
+ .flatten()
+ {
+ if logical_nulls.len() != self.len() {
Review Comment:
Yes, I have made some improvements for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "codephage2020 (via GitHub)" <gi...@apache.org>.
codephage2020 commented on code in PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#discussion_r2184186509
##########
arrow-array/src/array/union_array.rs:
##########
@@ -781,13 +781,23 @@ impl Array for UnionArray {
};
if fields.len() <= 1 {
- return self
+ if let Some(logical_nulls) = self
.fields
.iter()
.flatten()
.map(Array::logical_nulls)
.next()
- .flatten();
+ .flatten()
+ {
+ if logical_nulls.len() != self.len() {
Review Comment:
Yes, I have made some improvements to this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "codephage2020 (via GitHub)" <gi...@apache.org>.
codephage2020 commented on PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#issuecomment-3038956627
> I left a suggestion -- let me know what you think
I saw your suggestion. This is indeed a more concise and elegant implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "codephage2020 (via GitHub)" <gi...@apache.org>.
codephage2020 commented on PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#issuecomment-3049270946
> Thanks again
@alamb 🫡 Happy to contribute! I'd love to help more with this project. If there are any other good first issues or medium-level tasks available, please feel free to assign them to me . I'll be glad to work on them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#issuecomment-3049284279
> > Thanks again
>
> @alamb 🫡 Happy to contribute! I'd love to help more with this project. If there are any other good first issues or medium-level tasks available, please feel free to assign them to me . I'll be glad to work on them.
We have a log of activity recently on issues related to variant which might be interesting
- #6736
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix union slice logical_nulls length [arrow-rs]
Posted by "codephage2020 (via GitHub)" <gi...@apache.org>.
codephage2020 commented on PR #7855:
URL: https://github.com/apache/arrow-rs/pull/7855#issuecomment-3049308998
> We have a log of activity recently on issues related to variant which might be interesting
Thanks, I'll take a look at these issues. If I have any questions during the process, I'll reach out for guidance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org