You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/15 12:50:24 UTC
[GitHub] [arrow] nevi-me opened a new pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
nevi-me opened a new pull request #8926:
URL: https://github.com/apache/arrow/pull/8926
Contains the following:
- Fixes the date64 writer by writing the array as a timestamp milli
- Adds support for reading and writing interval types
> Apache Spark doesn't read intervals (as they're fixed len binary)
> It looks like the CPP impl at v2.0.0 just reads back the binary data.
The binary data read is however correct.
The result is that we can now correctly read and write the temporal types.
The Duration type remains unsupported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nevi-me commented on pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
Posted by GitBox <gi...@apache.org>.
nevi-me commented on pull request #8926:
URL: https://github.com/apache/arrow/pull/8926#issuecomment-745272405
CC @carols10cents @houqp
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8926:
URL: https://github.com/apache/arrow/pull/8926#issuecomment-745277001
https://issues.apache.org/jira/browse/ARROW-8425
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao closed pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
Posted by GitBox <gi...@apache.org>.
jorgecarleitao closed pull request #8926:
URL: https://github.com/apache/arrow/pull/8926
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8926:
URL: https://github.com/apache/arrow/pull/8926#issuecomment-745273384
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=h1) Report
> Merging [#8926](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=desc) (c760e88) into [master](https://codecov.io/gh/apache/arrow/commit/091df202ceb586b92882f67577ff720664e63eff?el=desc) (091df20) will **decrease** coverage by `0.03%`.
> The diff coverage is `64.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8926 +/- ##
==========================================
- Coverage 83.22% 83.18% -0.04%
==========================================
Files 196 196
Lines 48232 48341 +109
==========================================
+ Hits 40142 40214 +72
- Misses 8090 8127 +37
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/parquet/src/arrow/array\_reader.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJheV9yZWFkZXIucnM=) | `75.49% <48.43%> (-1.51%)` | :arrow_down: |
| [rust/parquet/src/arrow/schema.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9zY2hlbWEucnM=) | `91.05% <56.52%> (-0.27%)` | :arrow_down: |
| [rust/parquet/src/arrow/arrow\_writer.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJvd193cml0ZXIucnM=) | `95.58% <78.57%> (-1.76%)` | :arrow_down: |
| [rust/parquet/src/arrow/arrow\_reader.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJvd19yZWFkZXIucnM=) | `91.35% <100.00%> (+0.09%)` | :arrow_up: |
| [rust/parquet/src/arrow/converter.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9jb252ZXJ0ZXIucnM=) | `73.04% <100.00%> (+4.35%)` | :arrow_up: |
| [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.24% <0.00%> (-0.20%)` | :arrow_down: |
| [rust/arrow/src/compute/kernels/cast.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL2Nhc3QucnM=) | `96.84% <0.00%> (+0.06%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=footer). Last update [091df20...c760e88](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nevi-me commented on pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
Posted by GitBox <gi...@apache.org>.
nevi-me commented on pull request #8926:
URL: https://github.com/apache/arrow/pull/8926#issuecomment-748497278
> I did not see any changes in the tests, which I would expect for a change in semantics. Aren't we not testing this yet, or how can I verify that it is now correct?
There were existing tests, but they were ignored. So I removed the `#[ignore]` so they run.
I was writing date64 as timestampmilli, but this wasn't per the spec, so I fixed that now. I also added an interval test to increase test coverage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io commented on pull request #8926: ARROW-8425: [Rust] [Parquet] Correct temporal IO
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8926:
URL: https://github.com/apache/arrow/pull/8926#issuecomment-745273384
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=h1) Report
> Merging [#8926](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=desc) (3a5d822) into [master](https://codecov.io/gh/apache/arrow/commit/8df91c93676f8ac2b4d072e8ef0df62d3430cd57?el=desc) (8df91c9) will **increase** coverage by `23.76%`.
> The diff coverage is `62.50%`.
[](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8926 +/- ##
===========================================
+ Coverage 51.89% 75.66% +23.76%
===========================================
Files 173 181 +8
Lines 31081 41107 +10026
===========================================
+ Hits 16131 31103 +14972
+ Misses 14950 10004 -4946
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/parquet/src/arrow/array\_reader.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJheV9yZWFkZXIucnM=) | `71.18% <43.10%> (+71.18%)` | :arrow_up: |
| [rust/parquet/src/arrow/arrow\_writer.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9hcnJvd193cml0ZXIucnM=) | `95.56% <75.00%> (+95.56%)` | :arrow_up: |
| [rust/parquet/src/arrow/converter.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9jb252ZXJ0ZXIucnM=) | `62.88% <100.00%> (+62.88%)` | :arrow_up: |
| [rust/parquet/src/arrow/schema.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9hcnJvdy9zY2hlbWEucnM=) | `91.63% <100.00%> (+91.63%)` | :arrow_up: |
| [rust/parquet/src/util/hash\_util.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy91dGlsL2hhc2hfdXRpbC5ycw==) | `95.77% <0.00%> (ø)` | |
| [rust/parquet/src/bin/parquet-rowcount.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9iaW4vcGFycXVldC1yb3djb3VudC5ycw==) | `0.00% <0.00%> (ø)` | |
| [rust/parquet/src/util/test\_common/file\_util.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy91dGlsL3Rlc3RfY29tbW9uL2ZpbGVfdXRpbC5ycw==) | `77.77% <0.00%> (ø)` | |
| [rust/parquet/src/bin/parquet-schema.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9iaW4vcGFycXVldC1zY2hlbWEucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/parquet\_derive/src/lib.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0X2Rlcml2ZS9zcmMvbGliLnJz) | `0.00% <0.00%> (ø)` | |
| [rust/parquet/src/util/bit\_packing.rs](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy91dGlsL2JpdF9wYWNraW5nLnJz) | `99.96% <0.00%> (ø)` | |
| ... and [52 more](https://codecov.io/gh/apache/arrow/pull/8926/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=footer). Last update [0e8de08...3a5d822](https://codecov.io/gh/apache/arrow/pull/8926?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org