You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/09 04:57:47 UTC
[GitHub] [arrow] jorgecarleitao opened a new pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
jorgecarleitao opened a new pull request #8874:
URL: https://github.com/apache/arrow/pull/8874
This PR observes that `ExecutionContext::collect(&self, plan: Arc<dyn ExecutionPlan>)` does not use `self` on its implementation.
Using this observation, it refactors out `collect` out of `ExecutionContext` (into `physical_plan::collect`), thereby simplifying the execution of the plans, by not requiring creating an `ExecutionContext` just to execute a physical plan.
From a design's perspective, this makes it obvious that the execution of a physical plan is entirely independent of a context and its state.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8874:
URL: https://github.com/apache/arrow/pull/8874#issuecomment-741530586
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=h1) Report
> Merging [#8874](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=desc) (976878f) into [master](https://codecov.io/gh/apache/arrow/commit/7d509dce3cc684eb74b56a17048f2a2ed3f43dcd?el=desc) (7d509dc) will **increase** coverage by `23.49%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8874 +/- ##
===========================================
+ Coverage 52.90% 76.39% +23.49%
===========================================
Files 171 176 +5
Lines 30655 40428 +9773
===========================================
+ Hits 16218 30885 +14667
+ Misses 14437 9543 -4894
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/datafusion/examples/flight\_server.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL2V4YW1wbGVzL2ZsaWdodF9zZXJ2ZXIucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/execution/context.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/execution/dataframe\_impl.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vZGF0YWZyYW1lX2ltcGwucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/physical\_plan/filter.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL2ZpbHRlci5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/mod.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL21vZC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/sort.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL3NvcnQucnM=) | `0.00% <ø> (ø)` | |
| [rust/parquet/src/util/bit\_packing.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy91dGlsL2JpdF9wYWNraW5nLnJz) | `99.96% <0.00%> (ø)` | |
| [rust/parquet/src/util/test\_common/file\_util.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy91dGlsL3Rlc3RfY29tbW9uL2ZpbGVfdXRpbC5ycw==) | `77.77% <0.00%> (ø)` | |
| [rust/arrow/src/array/transform/fixed\_binary.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL2ZpeGVkX2JpbmFyeS5ycw==) | `42.10% <0.00%> (ø)` | |
| [rust/parquet/src/file/footer.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9maWxlL2Zvb3Rlci5ycw==) | `96.22% <0.00%> (ø)` | |
| ... and [51 more](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=footer). Last update [6ba022f...976878f](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb closed pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8874:
URL: https://github.com/apache/arrow/pull/8874
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8874:
URL: https://github.com/apache/arrow/pull/8874#issuecomment-741530586
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=h1) Report
> Merging [#8874](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=desc) (af52402) into [master](https://codecov.io/gh/apache/arrow/commit/7d509dce3cc684eb74b56a17048f2a2ed3f43dcd?el=desc) (7d509dc) will **increase** coverage by `24.15%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8874 +/- ##
===========================================
+ Coverage 52.90% 77.05% +24.15%
===========================================
Files 171 173 +2
Lines 30655 40078 +9423
===========================================
+ Hits 16218 30884 +14666
+ Misses 14437 9194 -5243
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/datafusion/examples/flight\_server.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL2V4YW1wbGVzL2ZsaWdodF9zZXJ2ZXIucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/execution/context.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/execution/dataframe\_impl.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vZGF0YWZyYW1lX2ltcGwucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/physical\_plan/filter.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL2ZpbHRlci5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/mod.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL21vZC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/sort.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL3NvcnQucnM=) | `0.00% <ø> (ø)` | |
| [rust/datafusion/tests/dataframe.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3Rlc3RzL2RhdGFmcmFtZS5ycw==) | `0.00% <ø> (ø)` | |
| [...ion-testing/src/bin/arrow-json-integration-test.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9pbnRlZ3JhdGlvbi10ZXN0aW5nL3NyYy9iaW4vYXJyb3ctanNvbi1pbnRlZ3JhdGlvbi10ZXN0LnJz) | | |
| [...ntegration-testing/src/bin/arrow-file-to-stream.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9pbnRlZ3JhdGlvbi10ZXN0aW5nL3NyYy9iaW4vYXJyb3ctZmlsZS10by1zdHJlYW0ucnM=) | | |
| [...ntegration-testing/src/bin/arrow-stream-to-file.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9pbnRlZ3JhdGlvbi10ZXN0aW5nL3NyYy9iaW4vYXJyb3ctc3RyZWFtLXRvLWZpbGUucnM=) | | |
| ... and [55 more](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=footer). Last update [6ba022f...af52402](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io commented on pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8874:
URL: https://github.com/apache/arrow/pull/8874#issuecomment-741530586
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=h1) Report
> Merging [#8874](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=desc) (562bc1f) into [master](https://codecov.io/gh/apache/arrow/commit/7d509dce3cc684eb74b56a17048f2a2ed3f43dcd?el=desc) (7d509dc) will **increase** coverage by `0.05%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8874 +/- ##
==========================================
+ Coverage 52.90% 52.96% +0.05%
==========================================
Files 171 172 +1
Lines 30655 30687 +32
==========================================
+ Hits 16218 16253 +35
+ Misses 14437 14434 -3
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/datafusion/examples/flight\_server.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL2V4YW1wbGVzL2ZsaWdodF9zZXJ2ZXIucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/execution/context.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/execution/dataframe\_impl.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vZGF0YWZyYW1lX2ltcGwucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/physical\_plan/filter.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL2ZpbHRlci5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/mod.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL21vZC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/sort.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL3NvcnQucnM=) | `0.00% <ø> (ø)` | |
| [rust/arrow/src/array/transform/fixed\_binary.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL2ZpeGVkX2JpbmFyeS5ycw==) | `42.10% <0.00%> (ø)` | |
| [rust/arrow/src/array/array\_binary.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvYXJyYXlfYmluYXJ5LnJz) | `90.82% <0.00%> (+0.20%)` | :arrow_up: |
| [rust/arrow/src/array/equal/mod.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvZXF1YWwvbW9kLnJz) | `91.50% <0.00%> (+0.37%)` | :arrow_up: |
| [rust/arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL21vZC5ycw==) | `76.85% <0.00%> (+0.80%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=footer). Last update [6ba022f...562bc1f](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8874:
URL: https://github.com/apache/arrow/pull/8874#issuecomment-741529902
https://issues.apache.org/jira/browse/ARROW-10859
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8874: ARROW-10859: [Rust] [DataFusion] Made collect not require ExecutionContext
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8874:
URL: https://github.com/apache/arrow/pull/8874#issuecomment-741530586
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=h1) Report
> Merging [#8874](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=desc) (8d15deb) into [master](https://codecov.io/gh/apache/arrow/commit/38cfdc3c95c0066b7b6a9b7fb5a47494decf7805?el=desc) (38cfdc3) will **increase** coverage by `1.18%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8874 +/- ##
==========================================
+ Coverage 52.92% 54.10% +1.18%
==========================================
Files 172 173 +1
Lines 30710 31062 +352
==========================================
+ Hits 16253 16806 +553
+ Misses 14457 14256 -201
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/datafusion/examples/flight\_server.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL2V4YW1wbGVzL2ZsaWdodF9zZXJ2ZXIucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/execution/context.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/execution/dataframe\_impl.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9leGVjdXRpb24vZGF0YWZyYW1lX2ltcGwucnM=) | `0.00% <0.00%> (ø)` | |
| [rust/datafusion/src/physical\_plan/filter.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL2ZpbHRlci5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/mod.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL21vZC5ycw==) | `0.00% <ø> (ø)` | |
| [rust/datafusion/src/physical\_plan/sort.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3NyYy9waHlzaWNhbF9wbGFuL3NvcnQucnM=) | `0.00% <ø> (ø)` | |
| [rust/datafusion/tests/dataframe.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9kYXRhZnVzaW9uL3Rlc3RzL2RhdGFmcmFtZS5ycw==) | `0.00% <ø> (ø)` | |
| [rust/arrow/src/array/cast.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvY2FzdC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/equal/decimal.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvZXF1YWwvZGVjaW1hbC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/transform/utils.rs](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3V0aWxzLnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [107 more](https://codecov.io/gh/apache/arrow/pull/8874/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=footer). Last update [38cfdc3...8d15deb](https://codecov.io/gh/apache/arrow/pull/8874?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org