You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "CurtHagenlocher (via GitHub)" <gi...@apache.org> on 2026/09/27 04:23:37 UTC
[PR] fix: Read large_binary and binary_view storage in shredded variant readers [arrow-dotnet]
CurtHagenlocher opened a new pull request, #449:
URL: https://github.com/apache/arrow-dotnet/pull/449
## What's Changed
`VariantArray` accepts `binary`, `large_binary` or `binary_view` for `metadata` and `value` at every nesting level, and `ShredSchema.FromArrowType` maps `large_utf8` / `large_binary` `typed_value` columns. The shredded readers (`ShreddedVariant`, `ShreddedObject`, `ShreddedArray`) cast those columns to `BinaryArray` / `StringArray`, so any other representation threw `InvalidCastException`. This even affected `GetLogicalVariantValue` on unshredded `large_binary` columns.
This replaces the casts with two small helpers in `ShreddingHelpers`, `GetBytes` and `GetString`, that read from any binary or string representation. (`IBinaryArray` would have been the natural abstraction, but it's internal to `Apache.Arrow`.)
The new tests shred a column that has residuals at the top level, in a partially shredded object, in an object field, and in a list element. They then convert every binary field, recursively, to `large_binary` or `binary_view` (and string/binary `typed_value` columns to `large_utf8` / `large_binary`), and read everything back. All of these tests fail without the fix.
Closes #448.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Read large_binary and binary_view storage in shredded variant readers [arrow-dotnet]
Posted by "CurtHagenlocher (via GitHub)" <gi...@apache.org>.
CurtHagenlocher merged PR #449:
URL: https://github.com/apache/arrow-dotnet/pull/449
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org