You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/21 23:10:47 UTC
[GitHub] [arrow] Dandandan opened a new pull request #8986: ARROW-5950: [Rust][DataFusion] Add logger
Dandandan opened a new pull request #8986:
URL: https://github.com/apache/arrow/pull/8986
It seems useful to me to do some extra (debug, info, etc) logging.
I added to the benchmark.
Executing with `RUST_LOG="debug"` gives (next to sqlparser logs):
```
[2020-12-21T23:04:59Z DEBUG datafusion::execution::context] Logical plan:
Sort: #l_returnflag ASC NULLS FIRST, #l_linestatus ASC NULLS FIRST
Projection: #l_returnflag, #l_linestatus, #SUM(l_quantity) AS sum_qty, #SUM(l_extendedprice) AS sum_base_price, #SUM(l_extendedprice Multiply Int64(1) Minus l_discount) AS sum_disc_price, #SUM(l_extendedprice Multiply Int64(1) Minus l_discount Multiply Int64(1) Plus l_tax) AS sum_charge, #AVG(l_quantity) AS avg_qty, #AVG(l_extendedprice) AS avg_price, #AVG(l_discount) AS avg_disc, #COUNT(UInt8(1)) AS count_order
Aggregate: groupBy=[[#l_returnflag, #l_linestatus]], aggr=[[SUM(#l_quantity), SUM(#l_extendedprice), SUM(#l_extendedprice Multiply Int64(1) Minus #l_discount), SUM(#l_extendedprice Multiply Int64(1) Minus #l_discount Multiply Int64(1) Plus #l_tax), AVG(#l_quantity), AVG(#l_extendedprice), AVG(#l_discount), COUNT(UInt8(1))]]
Filter: #l_shipdate LtEq CAST(Utf8("1998-09-02") AS Date32(Day))
TableScan: lineitem projection=None
[2020-12-21T23:04:59Z DEBUG datafusion::execution::context] Optimized logical plan:
Sort: #l_returnflag ASC NULLS FIRST, #l_linestatus ASC NULLS FIRST
Projection: #l_returnflag, #l_linestatus, #SUM(l_quantity) AS sum_qty, #SUM(l_extendedprice) AS sum_base_price, #SUM(l_extendedprice Multiply Int64(1) Minus l_discount) AS sum_disc_price, #SUM(l_extendedprice Multiply Int64(1) Minus l_discount Multiply Int64(1) Plus l_tax) AS sum_charge, #AVG(l_quantity) AS avg_qty, #AVG(l_extendedprice) AS avg_price, #AVG(l_discount) AS avg_disc, #COUNT(UInt8(1)) AS count_order
Aggregate: groupBy=[[#l_returnflag, #l_linestatus]], aggr=[[SUM(#l_quantity), SUM(#l_extendedprice), SUM(#l_extendedprice Multiply Int64(1) Minus #l_discount), SUM(#l_extendedprice Multiply Int64(1) Minus #l_discount Multiply Int64(1) Plus #l_tax), AVG(#l_quantity), AVG(#l_extendedprice), AVG(#l_discount), COUNT(UInt8(1))]]
Filter: #l_shipdate LtEq CAST(Utf8("1998-09-02") AS Date32(Day))
TableScan: lineitem projection=Some([4, 5, 6, 7, 8, 9, 10])
Query 1 iteration 9 took 802.7 ms
Query 1 avg time: 915.30 ms
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io commented on pull request #8986: ARROW-5950: [Rust][DataFusion] Add logger
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8986:
URL: https://github.com/apache/arrow/pull/8986#issuecomment-749251591
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=h1) Report
> Merging [#8986](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=desc) (aacc33d) into [master](https://codecov.io/gh/apache/arrow/commit/38ba81b02186c039e804d079195012da915311eb?el=desc) (38ba81b) will **decrease** coverage by `0.00%`.
> The diff coverage is `66.66%`.
[](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8986 +/- ##
==========================================
- Coverage 82.65% 82.65% -0.01%
==========================================
Files 200 200
Lines 49675 49678 +3
==========================================
+ Hits 41058 41060 +2
- Misses 8617 8618 +1
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow-flight/src/arrow.flight.protocol.rs](https://codecov.io/gh/apache/arrow/pull/8986/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy1mbGlnaHQvc3JjL2Fycm93LmZsaWdodC5wcm90b2NvbC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/benchmarks/src/bin/tpch.rs](https://codecov.io/gh/apache/arrow/pull/8986/diff?src=pr&el=tree#diff-cnVzdC9iZW5jaG1hcmtzL3NyYy9iaW4vdHBjaC5ycw==) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/execution/context.rs](https://codecov.io/gh/apache/arrow/pull/8986/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `89.45% <100.00%> (+0.02%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=footer). Last update [38ba81b...aacc33d](https://codecov.io/gh/apache/arrow/pull/8986?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb closed pull request #8986: ARROW-5950: [Rust][DataFusion] Add logger
Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8986:
URL: https://github.com/apache/arrow/pull/8986
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8986: ARROW-5950: [Rust][DataFusion] Add logger
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8986:
URL: https://github.com/apache/arrow/pull/8986#issuecomment-749255612
https://issues.apache.org/jira/browse/ARROW-5950
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Dandandan commented on pull request #8986: ARROW-5950: [Rust][DataFusion] Add logger
Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8986:
URL: https://github.com/apache/arrow/pull/8986#issuecomment-749395880
> FWIW for me we can add it to the bin, also.
I think that's a good idea, but don't want to add a logger as dependency directly to datafusion. Doing so at the bin level is not yet supported https://github.com/rust-lang/cargo/issues/1982 It will probably change though https://github.com/rust-lang/rfcs/pull/3020
Maybe at some point they can move to their own project.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Dandandan commented on pull request #8986: ARROW-5950: [Rust][DataFusion] Add logger
Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8986:
URL: https://github.com/apache/arrow/pull/8986#issuecomment-749519484
Thanks @alamb updated the PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org