You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "veeceey (via GitHub)" <gi...@apache.org> on 2026/02/23 03:58:25 UTC
[PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
veeceey opened a new pull request, #49373:
URL: https://github.com/apache/arrow/pull/49373
`ReferencedBufferSize` (used by pyarrow's `Table.nbytes` / `Array.nbytes`) was missing a visitor for `BinaryViewType`, which caused:
```
ArrowTypeError: Extracting byte ranges not supported for type string_view
```
when calling `.nbytes` on any table/array containing `string_view` or `binary_view` columns.
The fix adds a `Visit(const BinaryViewType&)` handler to `GetByteRangesArray` that accounts for:
1. The validity bitmap (buffer 0)
2. The views buffer (buffer 1) - fixed-width, 16 bytes per element
3. Out-of-line data buffers (buffers 2+) - only the ranges actually referenced by non-inline views
Since `StringViewType` inherits from `BinaryViewType`, both types are handled.
Also added tests for inline-only, mixed, and out-of-line binary_view/string_view arrays.
Fixes #47279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49373:
URL: https://github.com/apache/arrow/pull/49373#issuecomment-3942479725
:warning: GitHub issue #47279 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "veeceey (via GitHub)" <gi...@apache.org>.
veeceey commented on PR #49373:
URL: https://github.com/apache/arrow/pull/49373#issuecomment-3976481932
Thanks @rok — makes sense to consolidate. Happy to review #47418 and follow up with any missing pieces once it merges. Will keep this open for now in case there are gaps to address afterward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "veeceey (via GitHub)" <gi...@apache.org>.
veeceey closed pull request #49373: GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize
URL: https://github.com/apache/arrow/pull/49373
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on PR #49373:
URL: https://github.com/apache/arrow/pull/49373#issuecomment-3960065312
Thanks for working on this @veeceey ! It seems we already have a PR (https://github.com/apache/arrow/pull/47418) for this. Would you be so kind and give https://github.com/apache/arrow/pull/47418 a review and perhaps open a follow up PR to it once it merges if you're missing features from it? I'll take a look here once we merge https://github.com/apache/arrow/pull/47418 too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "veeceey (via GitHub)" <gi...@apache.org>.
veeceey commented on PR #49373:
URL: https://github.com/apache/arrow/pull/49373#issuecomment-4043890075
closing this - #47418 already covers the BinaryView/StringView support and has been merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on PR #49373:
URL: https://github.com/apache/arrow/pull/49373#issuecomment-4046052455
Thanks @veeceey and feel free to open a PR if something is still missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47279: [C++] Support BinaryView/StringView in ReferencedBufferSize [arrow]
Posted by "veeceey (via GitHub)" <gi...@apache.org>.
veeceey commented on PR #49373:
URL: https://github.com/apache/arrow/pull/49373#issuecomment-3942479910
### Test Results
Code structure verified - braces balanced, visitor correctly added. The C++ build requires cmake setup which I don't have locally, but the implementation follows the exact same patterns as the existing `VisitBaseBinary` handler.
The visitor handles:
- Validity bitmap via `VisitBitmap()`
- Views buffer as fixed-width (16 bytes per view)
- Out-of-line data buffers with min/max offset tracking per buffer index
Added test cases:
- `BinaryViewInline`: all-inline strings (no data buffers)
- `StringViewInline`: string_view (inherits BinaryViewType)
- `BinaryViewOutOfLine`: strings >12 bytes stored out-of-line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org