You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "sdf-jkl (via GitHub)" <gi...@apache.org> on 2026/05/04 15:41:55 UTC
Re: [PR] [Variant] remove `BorrowedShreddingState` [arrow-rs]
sdf-jkl commented on code in PR #9791:
URL: https://github.com/apache/arrow-rs/pull/9791#discussion_r3182714893
##########
parquet-variant-compute/src/unshred_variant.rs:
##########
@@ -175,12 +174,17 @@ impl<'a> UnshredVariantRowBuilder<'a> {
}
}
- /// Creates a new UnshredVariantRowBuilder from shredding state
- /// Returns None for None/None case - caller decides how to handle based on context
- fn try_new_opt(shredding_state: BorrowedShreddingState<'a>) -> Result<Option<Self>> {
- let value = shredding_state.value_field();
- let typed_value = shredding_state.typed_value_field();
- let Some(typed_value) = typed_value else {
+ /// Creates a new UnshredVariantRowBuilder from the `(value, typed_value)` pair of a shredded
+ /// variant struct. Returns None for the None/None case - caller decides how to handle based on
+ /// context.
+ fn try_new_opt(inner_struct: &'a StructArray) -> Result<Option<Self>> {
Review Comment:
1. Nothing serious. I thought the `BorrowedShreddingState` redundant and wanted to remove it as dead code.
2. We tried this in #8354. There we follow the `variant_get` path and extract the shredding state. For Struct arrays we can use `BorrowedShreddingState`, but for List arrays, we write a new array using `take` kernel. We can't use `BorrowedShreddingState` here as reference outlives the value.
3. `PrimitiveArray` is only cloned in `ShreddingState` and `unshred_variant` builder. `StructArray` is cloned in: `VariantArray::try_new`, `ShreddedVariantFieldArray::try_new` and `canonicalize_shredded_types`. We make the situation a little worse for `ShreddingState` by introducing one extra `ArrayRef` Arc bump per present field. Same for `PrimitiveArray`s for unshred builders.
4. I think a good alternative is Arc'ing the inner `StructArray`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org