You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "tustvold (via GitHub)" <gi...@apache.org> on 2024/05/08 13:37:36 UTC
[I] Structured ByteView Access [arrow-rs]
tustvold opened a new issue, #5736:
URL: https://github.com/apache/arrow-rs/issues/5736
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
<!--
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
-->
There have been a couple of proposals to improve the ergonomics of interacting with byte views
* https://github.com/apache/arrow-rs/pull/5735
* https://github.com/apache/arrow-rs/pull/5619
The current design of the byte views assumes calling sites know the structure of the `u128`, #5735 tweaks this a little bit but the onus is still on the user to understand the layout of the views.
**Describe the solution you'd like**
<!--
A clear and concise description of what you want to happen.
-->
TBD
**Describe alternatives you've considered**
<!--
A clear and concise description of any alternative solutions or features you've considered.
-->
**Additional context**
<!--
Add any other context or screenshots about the feature request here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2108627323
Related discussion https://github.com/apache/arrow-rs/issues/5654#issuecomment-2108376421
I think this basically means the intention is that we (well @tustvold ) plans to replace `u128` in `ByteViewArray` with an actual structured type of some sort
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2160454051
@tustvold do you have any more plans for this issue in the next week? I think @XiangpengHao is going to be helping work on ByteView features over the next few days but also that that shouldn't really affect this ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2140171921
I like the plan of going with what we have (which is now a bit nicer) and waiting another three months for any breaking change
We can always add things on top of the `u128`s (e.g. similar to https://github.com/apache/arrow-rs/pull/5619) if we need something more ergonomic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2160620628
I do not intend to do anything further with this ticket at this time, no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2139604972
#5786 provides a high-level interface for constructing views, which mirrors the high-level interface exposed by ByteViewArray for accessing them.
This means users shouldn't ever need to deal with the underlying representation. I think we therefore could just close this ticket.
For completeness I have spent some time trying to come up with a better abstraction than u128 and have struggled.
There are a couple of major challenges:
* String prefixes are not necessarily valid unicode
* Certain operations exploit the layout in ways LLVM seems unable to come up with, e.g.
* Compare the length and prefix in a single u64 comparison
* Elide length checking once checked length and prefix together
That isn't to say a better abstraction isn't possible, but it is fiddly and the major forcing function of construction is resolved by #5786. I think a smaller incremental PR like #5735 may therefore be all we need for the time being
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2135598822
https://github.com/apache/arrow-rs/pull/5796 is up for feedback
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5736:
URL: https://github.com/apache/arrow-rs/issues/5736#issuecomment-2204013716
`label_issue.py` automatically added labels {'arrow'} from #5619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Structured ByteView Access (underlying StringView/BinaryView representation) [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #5736: Structured ByteView Access (underlying StringView/BinaryView representation)
URL: https://github.com/apache/arrow-rs/issues/5736
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org