You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "scovich (via GitHub)" <gi...@apache.org> on 2025/09/20 00:10:28 UTC
[PR] [Variant] Fix NULL handling for shredded object fields [arrow-rs]
scovich opened a new pull request, #8395:
URL: https://github.com/apache/arrow-rs/pull/8395
# Which issue does this PR close?
- Fast-follow for https://github.com/apache/arrow-rs/pull/8366
- Related to https://github.com/apache/arrow-rs/pull/8392
# Rationale for this change
Somehow, https://github.com/apache/arrow-rs/pull/8392 exposes a latent bug in https://github.com/apache/arrow-rs/pull/8366, which has improper NULL handling for shredded object fields. The shredding PR originally attempted to handle this case, but somehow the test did not trigger the bug and so the (admittedly incomplete) code was removed. See https://github.com/apache/arrow-rs/pull/8366#discussion_r2357552451. To be honest, I have no idea how the original ever worked correctly, nor why the new PR is able to expose the problem.
# What changes are included in this PR?
When used as a top-level builder, `VariantToShreddedVariantRowBuilder::append_null` must append NULL to its own `NullBufferBuilder`; but when used as a shredded object field builder, it must append non-NULL. Plumb a new `top_level` parameter through the various functions and into the two sub-builders it relies on, so they can correctly implement the new semantics.
# Are these changes tested?
In theory, yes (I don't know how the object shredding test ever passed). And it fixes the breakage in https://github.com/apache/arrow-rs/pull/8392.
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Variant] Fix NULL handling for shredded object fields [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8395:
URL: https://github.com/apache/arrow-rs/pull/8395#issuecomment-3315973069
> @alamb -- we should track a follow-on item somewhere: Once we support array element shredding, we will have _three_ different semantics to deal with, and this fix will need to expand from a bool to an enum:
Filed
- https://github.com/apache/arrow-rs/issues/8400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Variant] Fix NULL handling for shredded object fields [arrow-rs]
Posted by "scovich (via GitHub)" <gi...@apache.org>.
scovich commented on PR #8395:
URL: https://github.com/apache/arrow-rs/pull/8395#issuecomment-3314277966
@alamb -- we should track a follow-on item somewhere: Once we support array element shredding, we will have _three_ different semantics to deal with, and this fix will need to expand from a bool to an enum:
When appending NULL:
||struct|value|typed_value|
|-|-|-|-|
|Variant|NULL|NULL|NULL
|Object field|valid|NULL|NULL
|Array element|valid|Variant::Null|NULL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Variant] Fix NULL handling for shredded object fields [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #8395:
URL: https://github.com/apache/arrow-rs/pull/8395
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org