You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "scovich (via GitHub)" <gi...@apache.org> on 2025/09/06 03:49:07 UTC
[I] [Variant] cast_to_variant conflates empty map with NULL [arrow-rs]
scovich opened a new issue, #8289:
URL: https://github.com/apache/arrow-rs/issues/8289
**Describe the bug**
While trying to refactor `cast_to_variant` to be more efficient, I noticed that the logic for maps does not check for NULL entries. It only looks at the offsets, which technically only say how many entries the map has.
**To Reproduce**
`test_cast_map_to_variant_object_with_nulls` produces a `MapArray` with no NULL rows, and the test expects a NULL output for the row that had an empty map.
**Expected behavior**
The test should produce a `MapArray` where one row is an empty map and another row is NULL; the former should become a variant object with no fields and the other should become NULL or Variant::Null (depending on whether the map is top-level or nested inside another variant value).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] cast_to_variant conflates empty map with NULL [arrow-rs]
Posted by "scovich (via GitHub)" <gi...@apache.org>.
scovich closed issue #8289: [Variant] cast_to_variant conflates empty map with NULL
URL: https://github.com/apache/arrow-rs/issues/8289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] cast_to_variant conflates empty map with NULL [arrow-rs]
Posted by "scovich (via GitHub)" <gi...@apache.org>.
scovich commented on issue #8289:
URL: https://github.com/apache/arrow-rs/issues/8289#issuecomment-3281188200
* Fixed by https://github.com/apache/arrow-rs/pull/8299
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] cast_to_variant conflates empty map with NULL [arrow-rs]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 commented on issue #8289:
URL: https://github.com/apache/arrow-rs/issues/8289#issuecomment-3264427700
@scovich I can help to fix this, it's straightforward to fix the empty map case, for the `MapArray` with a `NULL` row, we need to return `NULL` for top-level and `Variant::NULL` for nested-level. Can we assume that the first call to the function `parquet-variant-compute/src/cast_to_variant.rs#cast_to_variant` is the top level, and the recursive calll from `cast_to_variant` to the function as the nested level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] cast_to_variant conflates empty map with NULL [arrow-rs]
Posted by "klion26 (via GitHub)" <gi...@apache.org>.
klion26 commented on issue #8289:
URL: https://github.com/apache/arrow-rs/issues/8289#issuecomment-3268721379
Thanks for the info, it seems this will be fixed in #8299 , we don't need a separate pr for this now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] cast_to_variant conflates empty map with NULL [arrow-rs]
Posted by "scovich (via GitHub)" <gi...@apache.org>.
scovich commented on issue #8289:
URL: https://github.com/apache/arrow-rs/issues/8289#issuecomment-3265986677
The current cast design treats every `convert_map` call as "top level" because it always converts every column to variant independently, and complex types like struct combine child variants into parent variants. So I think it suffices to make `convert_map` check for a null input and append_null to its builder? And then a bunch of weird offset calculations can go away.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org