You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/12/22 16:29:27 UTC
[GitHub] [arrow-rs] tustvold commented on pull request #1082: Optimized ByteArrayReader (#1040)
tustvold commented on pull request #1082:
URL: https://github.com/apache/arrow-rs/pull/1082#issuecomment-999707518
> would you be able to share some performance benchmark results?
They're very preliminary at this stage, I'm not totally confident this code is correct nor have I spent any time trying to optimise it, but here you go. My primary focus has been proving out the interface from #1041, not polishing up the specific optimisations yet.
"Old" is the new ByteArrayReader, "new" is the StringArrayReader
```
arrow_array_reader/read StringArray, plain encoded, mandatory, no NULLs - old
time: [110.98 us 111.00 us 111.03 us]
arrow_array_reader/read StringArray, plain encoded, mandatory, no NULLs - new
time: [124.77 us 124.99 us 125.32 us]
arrow_array_reader/read StringArray, plain encoded, optional, no NULLs - old
time: [122.15 us 122.17 us 122.20 us]
arrow_array_reader/read StringArray, plain encoded, optional, no NULLs - new
time: [136.72 us 136.76 us 136.81 us]
arrow_array_reader/read StringArray, plain encoded, optional, half NULLs - old
time: [117.26 us 117.35 us 117.43 us]
arrow_array_reader/read StringArray, plain encoded, optional, half NULLs - new
time: [258.05 us 258.17 us 258.30 us]
arrow_array_reader/read StringArray, dictionary encoded, mandatory, no NULLs - old
time: [145.30 us 145.35 us 145.41 us]
arrow_array_reader/read StringArray, dictionary encoded, mandatory, no NULLs - new
time: [117.14 us 117.18 us 117.22 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, no NULLs - old
time: [159.07 us 159.11 us 159.15 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, no NULLs - new
time: [134.39 us 134.41 us 134.43 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, half NULLs - old
time: [108.28 us 108.30 us 108.33 us]
arrow_array_reader/read StringArray, dictionary encoded, optional, half NULLs - new
time: [230.15 us 230.23 us 230.32 us]
```
Aside from dictionary encoded columns with no nulls, it performs better. This is probably just something suboptimal in the way I'm decoding RLE data, and should be rectifiable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org