You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/06/07 11:01:00 UTC
[GitHub] [arrow-rs] tustvold opened a new pull request, #1810: Separate Page IO from Page Decode
tustvold opened a new pull request, #1810:
URL: https://github.com/apache/arrow-rs/pull/1810
_Draft as builds on #1807_
# Which issue does this PR close?
Part of #1163
# Rationale for this change
This is the first part of reworking the read path to use less custom abstractions, in particular it splits reading the page bytes from interpreting those bytes.
# What changes are included in this PR?
Splits the decode logic out of SerializedPageReader into free functions, and then uses this within ParquetRecordBatchStream to elide a copy.
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter commented on pull request #1810: Separate Page IO from Page Decode
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #1810:
URL: https://github.com/apache/arrow-rs/pull/1810#issuecomment-1148540881
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1810?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1810](https://codecov.io/gh/apache/arrow-rs/pull/1810?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a70efd6) into [master](https://codecov.io/gh/apache/arrow-rs/commit/96a4d6c75cc6700c74b7d0d553732458691ed089?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (96a4d6c) will **decrease** coverage by `0.03%`.
> The diff coverage is `66.94%`.
> :exclamation: Current head a70efd6 differs from pull request most recent head fd6dfa4. Consider uploading reports for the commit fd6dfa4 to get more accurate results
```diff
@@ Coverage Diff @@
## master #1810 +/- ##
==========================================
- Coverage 83.42% 83.38% -0.04%
==========================================
Files 199 199
Lines 56632 56683 +51
==========================================
+ Hits 47244 47265 +21
- Misses 9388 9418 +30
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1810?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [parquet/src/arrow/async\_reader.rs](https://codecov.io/gh/apache/arrow-rs/pull/1810/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvYXJyb3cvYXN5bmNfcmVhZGVyLnJz) | `0.00% <0.00%> (ø)` | |
| [parquet/src/file/serialized\_reader.rs](https://codecov.io/gh/apache/arrow-rs/pull/1810/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZmlsZS9zZXJpYWxpemVkX3JlYWRlci5ycw==) | `94.95% <88.40%> (+0.49%)` | :arrow_up: |
| [parquet/src/compression.rs](https://codecov.io/gh/apache/arrow-rs/pull/1810/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvY29tcHJlc3Npb24ucnM=) | `89.68% <100.00%> (+1.08%)` | :arrow_up: |
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1810/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `65.75% <0.00%> (-0.23%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1810?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1810?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [96a4d6c...fd6dfa4](https://codecov.io/gh/apache/arrow-rs/pull/1810?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold merged pull request #1810: Separate Page IO from Page Decode
Posted by GitBox <gi...@apache.org>.
tustvold merged PR #1810:
URL: https://github.com/apache/arrow-rs/pull/1810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org