You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/06/28 09:04:11 UTC
[GitHub] [arrow-rs] tustvold opened a new pull request, #1956: Use InMemoryColumnChunkReader (~20% faster)
tustvold opened a new pull request, #1956:
URL: https://github.com/apache/arrow-rs/pull/1956
# Which issue does this PR close?
Part of #1163
# Rationale for this change
https://github.com/apache/arrow-rs/pull/1810 added the InMemoryColumnChunkReader, but I stupidly neglected to actually hook it up :facepalm:
# What changes are included in this PR?
Alters ParquetRecordBatchStream to use the InMemoryColumnChunkReader to avoid an unnecessary copy. For large column chunks, this represents a significant saving.
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter commented on pull request #1956: Use InMemoryColumnChunkReader (~20% faster)
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #1956:
URL: https://github.com/apache/arrow-rs/pull/1956#issuecomment-1168463040
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1956?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1956](https://codecov.io/gh/apache/arrow-rs/pull/1956?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9edb297) into [master](https://codecov.io/gh/apache/arrow-rs/commit/464e8d199d0dd875b09241cc0608da73d6771e29?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (464e8d1) will **increase** coverage by `0.00%`.
> The diff coverage is `0.00%`.
> :exclamation: Current head 9edb297 differs from pull request most recent head 7e308a4. Consider uploading reports for the commit 7e308a4 to get more accurate results
```diff
@@ Coverage Diff @@
## master #1956 +/- ##
=======================================
Coverage 83.47% 83.47%
=======================================
Files 221 221
Lines 57049 57046 -3
=======================================
Hits 47621 47621
+ Misses 9428 9425 -3
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1956?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [parquet/src/arrow/async\_reader.rs](https://codecov.io/gh/apache/arrow-rs/pull/1956/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvYXJyb3cvYXN5bmNfcmVhZGVyLnJz) | `0.00% <0.00%> (ø)` | |
| [arrow/src/datatypes/datatype.rs](https://codecov.io/gh/apache/arrow-rs/pull/1956/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2RhdGF0eXBlcy9kYXRhdHlwZS5ycw==) | `65.42% <0.00%> (-0.38%)` | :arrow_down: |
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1956/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `65.98% <0.00%> (+0.22%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1956?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1956?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [464e8d1...7e308a4](https://codecov.io/gh/apache/arrow-rs/pull/1956?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] Ted-Jiang commented on pull request #1956: Use InMemoryColumnChunkReader (~20% faster)
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on PR #1956:
URL: https://github.com/apache/arrow-rs/pull/1956#issuecomment-1170708214
@tustvold does this means we not support `SerializedPageReader` any more?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold merged pull request #1956: Use InMemoryColumnChunkReader (~20% faster)
Posted by GitBox <gi...@apache.org>.
tustvold merged PR #1956:
URL: https://github.com/apache/arrow-rs/pull/1956
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #1956: Use InMemoryColumnChunkReader (~20% faster)
Posted by GitBox <gi...@apache.org>.
tustvold commented on PR #1956:
URL: https://github.com/apache/arrow-rs/pull/1956#issuecomment-1170891109
SerializedPageReader is still used by the sync APIs, i.e. SerializedFileReader, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org