You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "friendlymatthew (via GitHub)" <gi...@apache.org> on 2025/10/20 17:12:24 UTC
[I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
friendlymatthew opened a new issue, #8665:
URL: https://github.com/apache/arrow-rs/issues/8665
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
`shred_variant` currently panics when attempting to shred `Variant`s containing values of the `FixedSizeBinary(16)` data type
```rs
let input = VariantArray::from_iter([ Some(Variant::from(Uuid::new_v4())), None]);
let out = shred_variant(&input, &DataType::FixedSizeBinary(16)).unwrap(); // panics!
```
**Describe the solution you'd like**
`shred_variant` should handle this case gracefully instead of panicking, and correctly shred variants of `FixedSizeBinary(16)`.
Note, since `FixedSizeBinary(16)` represents a UUID, it's super important that the implementation respects the canonical UUID extension type
Related https://github.com/apache/arrow-rs/issues/8420
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
Posted by "friendlymatthew (via GitHub)" <gi...@apache.org>.
friendlymatthew commented on issue #8665:
URL: https://github.com/apache/arrow-rs/issues/8665#issuecomment-3423018870
cc @scovich @alamb
Unless I'm missing something, this repro seems sensible to me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
Posted by "scovich (via GitHub)" <gi...@apache.org>.
scovich commented on issue #8665:
URL: https://github.com/apache/arrow-rs/issues/8665#issuecomment-3423325006
Yeah, as long as `shred_variant` only takes a `DataType` instead of a `Field`, we are forced to assume 16-byte fixed binary is UUID. If it accepted a `Field`, we should instead require the UUID extension type. Otherwise, we run into problems because Decimal128 can _also_ use 16-byte fixed binary!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #8665:
URL: https://github.com/apache/arrow-rs/issues/8665#issuecomment-3553616470
`label_issue.py` automatically added labels {'parquet-variant'} from #8673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #8665:
URL: https://github.com/apache/arrow-rs/issues/8665#issuecomment-3423159512
makes sense to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #8665: [Variant] Support Uuid/`FixedSizeBinary(16)` shredding
URL: https://github.com/apache/arrow-rs/issues/8665
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Variant] Support Uuid/`FixedSizeBinary(16)` shredding [arrow-rs]
Posted by "friendlymatthew (via GitHub)" <gi...@apache.org>.
friendlymatthew commented on issue #8665:
URL: https://github.com/apache/arrow-rs/issues/8665#issuecomment-3423329112
> Yeah, as long as `shred_variant` only takes a `DataType` instead of a `Field`, we are forced to assume 16-byte fixed binary is UUID. If it accepted a `Field`, we should additionally require the UUID extension type. Otherwise, we potentially run into problems because Decimal128 can _also_ use 16-byte fixed binary!
I plan on pushing up another PR for this exact change!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org