You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/30 11:21:17 UTC
[GitHub] [arrow-datafusion] alamb opened a new pull request, #2988: remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing
alamb opened a new pull request, #2988:
URL: https://github.com/apache/arrow-datafusion/pull/2988
Draft as it builds on https://github.com/apache/arrow-datafusion/pull/2985
# Which issue does this PR close?
Closes https://github.com/apache/arrow-datafusion/issues/2987
# Rationale for this change
The duplication was confusing.
However, I am not sure about this change as now the different formats are treated differently, whereas before the formats all had a uniform implementation
# What changes are included in this PR?
Move shared options into a `ParquetFormatOptions` shared structure
# Are there any user-facing changes?
This is backards compatibel for anyone who maniuplated the fields directly. If they used the builder API it will be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #2988: remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2988:
URL: https://github.com/apache/arrow-datafusion/pull/2988#issuecomment-1214138111
I don't think this is worth pursuing at the current time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed pull request #2988: remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing
Posted by GitBox <gi...@apache.org>.
alamb closed pull request #2988: remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing
URL: https://github.com/apache/arrow-datafusion/pull/2988
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #2988: remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2988:
URL: https://github.com/apache/arrow-datafusion/pull/2988#issuecomment-1200143611
# [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/2988?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2988](https://codecov.io/gh/apache/arrow-datafusion/pull/2988?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a0274a1) into [master](https://codecov.io/gh/apache/arrow-datafusion/commit/3d1de1557143efa7514e73839f5c54a6808d388c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3d1de15) will **increase** coverage by `0.00%`.
> The diff coverage is `91.66%`.
```diff
@@ Coverage Diff @@
## master #2988 +/- ##
=======================================
Coverage 85.78% 85.78%
=======================================
Files 281 282 +1
Lines 51580 51656 +76
=======================================
+ Hits 44246 44313 +67
- Misses 7334 7343 +9
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-datafusion/pull/2988?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [benchmarks/src/bin/tpch.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVuY2htYXJrcy9zcmMvYmluL3RwY2gucnM=) | `39.52% <0.00%> (ø)` | |
| [datafusion/core/tests/sql/mod.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3Rlc3RzL3NxbC9tb2QucnM=) | `98.26% <ø> (ø)` | |
| [datafusion/core/tests/sql/parquet.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3Rlc3RzL3NxbC9wYXJxdWV0LnJz) | `100.00% <ø> (ø)` | |
| [datafusion/proto/src/logical\_plan.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9wcm90by9zcmMvbG9naWNhbF9wbGFuLnJz) | `17.42% <0.00%> (ø)` | |
| [datafusion/core/src/execution/options.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9leGVjdXRpb24vb3B0aW9ucy5ycw==) | `59.45% <77.77%> (+0.03%)` | :arrow_up: |
| [...afusion/core/src/datasource/file\_format/parquet.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9kYXRhc291cmNlL2ZpbGVfZm9ybWF0L3BhcnF1ZXQucnM=) | `85.20% <83.33%> (-0.70%)` | :arrow_down: |
| [datafusion/core/tests/sql/parquet\_schema.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3Rlc3RzL3NxbC9wYXJxdWV0X3NjaGVtYS5ycw==) | `98.79% <98.79%> (ø)` | |
| [...tafusion/core/src/physical\_plan/file\_format/mod.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFuL2ZpbGVfZm9ybWF0L21vZC5ycw==) | `97.36% <100.00%> (ø)` | |
| [datafusion/expr/src/window\_frame.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy93aW5kb3dfZnJhbWUucnM=) | `92.43% <0.00%> (-0.85%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/apache/arrow-datafusion/pull/2988/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #2988: remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2988:
URL: https://github.com/apache/arrow-datafusion/pull/2988#issuecomment-1200140308
🤔 now that I see how this looks I am not convinced it saves much duplication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org