You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/16 20:12:47 UTC
[GitHub] [arrow] seddonm1 opened a new pull request #8943: ARROW-10947: [Rust][DataFusion] Optimize UTF8 to Date32 Conversion
seddonm1 opened a new pull request #8943:
URL: https://github.com/apache/arrow/pull/8943
After adding benchmarking capability to the UTF8 to Date32/Date64 CAST functions there was opportunity to improve the performance.
This PR uses inbuilt `chrono` functionality to calculate the number of days since CE then uses a constant to calculate the offset days relative to 1970-01-01. This improves performance around 10% for this operation relative to the `since` function presumably as `chrono` does not have to ensure the `from_ymd` is a valid date.
Before:
```
cast utf8 to date32 512 time: [41.966 us 42.508 us 43.087 us]
cast utf8 to date32 512 time: [40.591 us 40.661 us 40.740 us]
cast utf8 to date32 512 time: [40.825 us 40.878 us 40.916 us]
```
After:
```
cast utf8 to date32 512 time: [36.557 us 36.839 us 37.200 us]
cast utf8 to date32 512 time: [35.997 us 36.442 us 36.919 us]
cast utf8 to date32 512 time: [35.750 us 35.969 us 36.160 us]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao closed pull request #8943: ARROW-10947: [Rust][DataFusion] Optimize UTF8 to Date32 Conversion
Posted by GitBox <gi...@apache.org>.
jorgecarleitao closed pull request #8943:
URL: https://github.com/apache/arrow/pull/8943
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8943: ARROW-10947: [Rust][DataFusion] Optimize UTF8 to Date32 Conversion
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8943:
URL: https://github.com/apache/arrow/pull/8943#issuecomment-746971433
https://issues.apache.org/jira/browse/ARROW-10947
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io commented on pull request #8943: ARROW-10947: [Rust][DataFusion] Optimize UTF8 to Date32 Conversion
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8943:
URL: https://github.com/apache/arrow/pull/8943#issuecomment-747019019
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=h1) Report
> Merging [#8943](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=desc) (639fb71) into [master](https://codecov.io/gh/apache/arrow/commit/71e37e23e5a3aad396585df484625c2d0641840d?el=desc) (71e37e2) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8943 +/- ##
=======================================
Coverage 83.26% 83.27%
=======================================
Files 195 195
Lines 48066 48062 -4
=======================================
- Hits 40024 40022 -2
+ Misses 8042 8040 -2
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/cast.rs](https://codecov.io/gh/apache/arrow/pull/8943/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL2Nhc3QucnM=) | `96.77% <100.00%> (-0.01%)` | :arrow_down: |
| [rust/arrow/src/csv/reader.rs](https://codecov.io/gh/apache/arrow/pull/8943/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY3N2L3JlYWRlci5ycw==) | `94.33% <100.00%> (+0.36%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=footer). Last update [6cedab0...639fb71](https://codecov.io/gh/apache/arrow/pull/8943?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org