You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/02/09 02:03:15 UTC
[GitHub] [arrow-rs] dianaclarke opened a new pull request #1289: Arrow Rust + Conbench Integration
dianaclarke opened a new pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289
Here's a minimal Arrow Rust + Conbench proof of concept.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033279753
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1289](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4d7e6bb) into [master](https://codecov.io/gh/apache/arrow-rs/commit/35e16be01e680e9381b2d1393c2e3f8e7acb7b13?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (35e16be) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1289 +/- ##
==========================================
- Coverage 83.04% 83.04% -0.01%
==========================================
Files 180 180
Lines 52424 52424
==========================================
- Hits 43537 43536 -1
- Misses 8887 8888 +1
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [arrow/src/datatypes/datatype.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2RhdGF0eXBlcy9kYXRhdHlwZS5ycw==) | `66.40% <0.00%> (-0.40%)` | :arrow_down: |
| [parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2VuY29kaW5nLnJz) | `93.52% <0.00%> (-0.20%)` | :arrow_down: |
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `66.21% <0.00%> (ø)` | |
| [arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L3RyYW5zZm9ybS9tb2QucnM=) | `84.64% <0.00%> (+0.12%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [35e16be...4d7e6bb](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033279753
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1289](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (dc59492) into [master](https://codecov.io/gh/apache/arrow-rs/commit/35e16be01e680e9381b2d1393c2e3f8e7acb7b13?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (35e16be) will **not change** coverage.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1289 +/- ##
=======================================
Coverage 83.04% 83.04%
=======================================
Files 180 180
Lines 52424 52424
=======================================
Hits 43537 43537
Misses 8887 8887
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2VuY29kaW5nLnJz) | `93.52% <0.00%> (-0.20%)` | :arrow_down: |
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `66.21% <0.00%> (ø)` | |
| [arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L3RyYW5zZm9ybS9tb2QucnM=) | `84.64% <0.00%> (+0.12%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [35e16be...dc59492](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dianaclarke commented on a change in pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
dianaclarke commented on a change in pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#discussion_r802240975
##########
File path: conbench/.flake8
##########
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
From a Python perspective, it's probably a mistake to name this directory `conbench`. Thoughts on what you would like `arrow-rs/conbench/` to be named?
- `arrow-rs/_conbench/`?
- `arrow-rs/conbench-benchmarks/`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] andygrove commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
andygrove commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1046077340
If there are no objections, I plan to merge this PR next week so that I can proceed to the next step of integrating this with CI. The changes in this PR are self-contained so I think it is low risk to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dianaclarke commented on a change in pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
dianaclarke commented on a change in pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#discussion_r802294900
##########
File path: conbench/.gitignore
##########
@@ -0,0 +1,130 @@
+# Byte-compiled / optimized / DLL files
Review comment:
This is just the GitHub generated `.gitignore` for Python packages. I can trim in down, but I usually don't bother.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dianaclarke commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
dianaclarke commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033936044
> Thanks @dianaclarke I can take it from here for sure .. and it turns out this benchmark was failing even before this PR. I should have checked that first.
Ah, I was getting an error with `cargo bench` in `arrow-rs`, but I assumed I just didn't have my env set up correctly (I don't know Rust aside from what I learned doing this).
But I can `cargo bench` in `arrow-datafusion`, so perhaps it's easier to start with this pull request:
- https://github.com/apache/arrow-datafusion/pull/1791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033279753
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1289](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ca33a0a) into [master](https://codecov.io/gh/apache/arrow-rs/commit/35e16be01e680e9381b2d1393c2e3f8e7acb7b13?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (35e16be) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1289 +/- ##
==========================================
- Coverage 83.04% 83.04% -0.01%
==========================================
Files 180 180
Lines 52424 52424
==========================================
- Hits 43537 43536 -1
- Misses 8887 8888 +1
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `65.98% <0.00%> (-0.23%)` | :arrow_down: |
| [parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2VuY29kaW5nLnJz) | `93.52% <0.00%> (-0.20%)` | :arrow_down: |
| [arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L3RyYW5zZm9ybS9tb2QucnM=) | `84.64% <0.00%> (+0.12%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [35e16be...ca33a0a](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] andygrove commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
andygrove commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033887256
One of the tests failed because it makes an assumption that a `target` directory exists and it did not. My assumption is that somehow the current working directory was not correct. This is how I initiated the benchmark run.
```bash
~/git/arrow-rs/conbench$ conbench arrow-rs --src-dir=`pwd`/..
```
I see that the Python code should have changed the directory so I am not sure what went wrong. I will start debugging this tonight. We may also want to have our tests write output to a temporary directory rather than making any assumptions about current working directory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dianaclarke commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
dianaclarke commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033366659
> I pulled the branch locally and followed the instructions in the README and benchmarks are currently running. It looks like it will take a while so I will check in on results in the morning and continue reviewing the PR.
Nice!
99% of the time is spent executing `cargo bench`.
Note that this is only the first step – you should see json formatted benchmark results spew to your terminal, but they won't yet publish to the Arrow Conbench server[1].
You'll want to work with @ElenaHenderson on getting the results to publish to the Arrow Conbench server via buildkite etc.
[1] https://conbench.ursa.dev/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dianaclarke commented on a change in pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
dianaclarke commented on a change in pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#discussion_r802240975
##########
File path: conbench/.flake8
##########
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
From a Python perspective, it's probably a mistake to name this directory `conbench`. Thoughts on what you would like `arrow-rs/conbench/` to be named?
- `arrow-rs/_conbench/`?
- `arrow-rs/conbench-benchmarks/`?
- `arrow-rs/conbench-integration/`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] andygrove commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
andygrove commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033358632
I pulled the branch locally and followed the instructions in the README and benchmarks are currently running. It looks like it will take a while so I will check in on results in the morning and continue reviewing the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] andygrove commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
andygrove commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033924207
Thanks @dianaclarke I can take it from here for sure .. and it turns out this benchmark was failing even before this PR. I should have checked that first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] nevi-me merged pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
nevi-me merged pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] andygrove commented on a change in pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
andygrove commented on a change in pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#discussion_r802261507
##########
File path: conbench/README.md
##########
@@ -0,0 +1,251 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Arrow Rust + Conbench Integration
+
+
+## Quick start
+
+```
+$ cd ~/arrow-rs/conbench/
+$ conda create -y -n conbench python=3.9
+$ conda activate conbench
+(conbench) $ pip install -r requirements.txt
+(conbench) $ conbench arrow-rs --src-dir=/Users/diana/workspace/arrow-rs
+```
+
+## Example output
+
+```
+{
+ "batch_id": "b68c559358cc43a3aab02d893d2693f4",
+ "context": {
+ "benchmark_language": "Rust"
+ },
+ "github": {
+ "commit": "ca33a0a50494f95840ade2e9509c3c3d4df35249",
+ "repository": "https://github.com/dianaclarke/arrow-rs"
+ },
+ "info": {},
+ "machine_info": {
+ "architecture_name": "x86_64",
+ "cpu_core_count": "8",
+ "cpu_frequency_max_hz": "2400000000",
+ "cpu_l1d_cache_bytes": "65536",
+ "cpu_l1i_cache_bytes": "131072",
+ "cpu_l2_cache_bytes": "4194304",
+ "cpu_l3_cache_bytes": "0",
+ "cpu_model_name": "Apple M1",
+ "cpu_thread_count": "8",
+ "gpu_count": "0",
+ "gpu_product_names": [],
+ "kernel_name": "20.6.0",
+ "memory_bytes": "17179869184",
+ "name": "diana",
+ "os_name": "macOS",
+ "os_version": "10.16"
+ },
+ "run_id": "08353595bde147fb9deebdb4facd019a",
+ "stats": {
+ "data": [
+ "0.000287",
+ "0.000286",
+ "0.000285",
+ "0.000281",
+ "0.000282",
+ "0.000277",
+ "0.000277",
+ "0.000286",
+ "0.000279",
+ "0.000282",
+ "0.000277",
+ "0.000277",
+ "0.000282",
+ "0.000276",
+ "0.000281",
+ "0.000281",
+ "0.000281",
+ "0.000281",
+ "0.000281",
+ "0.000284",
+ "0.000288",
+ "0.000278",
+ "0.000276",
+ "0.000278",
+ "0.000275",
+ "0.000275",
+ "0.000275",
+ "0.000275",
+ "0.000281",
+ "0.000284",
+ "0.000277",
+ "0.000277",
+ "0.000278",
+ "0.000282",
+ "0.000281",
+ "0.000284",
+ "0.000282",
+ "0.000279",
+ "0.000280",
+ "0.000281",
+ "0.000281",
+ "0.000286",
+ "0.000278",
+ "0.000278",
+ "0.000281",
+ "0.000276",
+ "0.000284",
+ "0.000281",
+ "0.000276",
+ "0.000276",
+ "0.000279",
+ "0.000283",
+ "0.000282",
+ "0.000278",
+ "0.000281",
+ "0.000284",
+ "0.000279",
+ "0.000276",
+ "0.000278",
+ "0.000283",
+ "0.000282",
+ "0.000276",
+ "0.000281",
+ "0.000279",
+ "0.000276",
+ "0.000277",
+ "0.000283",
+ "0.000279",
+ "0.000281",
+ "0.000283",
+ "0.000279",
+ "0.000282",
+ "0.000283",
+ "0.000278",
+ "0.000281",
+ "0.000282",
+ "0.000278",
+ "0.000276",
+ "0.000281",
+ "0.000278",
+ "0.000276",
+ "0.000282",
+ "0.000281",
+ "0.000282",
+ "0.000280",
+ "0.000281",
+ "0.000282",
+ "0.000280",
+ "0.000282",
+ "0.000280",
+ "0.000280",
+ "0.000282",
+ "0.000278",
+ "0.000284",
+ "0.000290",
+ "0.000282",
+ "0.000281",
+ "0.000281",
+ "0.000281",
+ "0.000278"
+ ],
+ "iqr": "0.000004",
+ "iterations": 100,
+ "max": "0.000290",
+ "mean": "0.000280",
+ "median": "0.000281",
+ "min": "0.000275",
+ "q1": "0.000278",
+ "q3": "0.000282",
+ "stdev": "0.000003",
+ "time_unit": "s",
+ "times": [],
+ "unit": "s"
+ },
+ "tags": {
+ "name": "nlike_utf8 scalar starts with",
+ "suite": "nlike_utf8 scalar starts with"
+ },
+ "timestamp": "2022-02-09T02:33:26.792404+00:00"
+}
+```
+
+## Dubug with test benchmark
Review comment:
```suggestion
## Debug with test benchmark
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033279753
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1289](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (359e459) into [master](https://codecov.io/gh/apache/arrow-rs/commit/35e16be01e680e9381b2d1393c2e3f8e7acb7b13?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (35e16be) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1289 +/- ##
==========================================
- Coverage 83.04% 83.04% -0.01%
==========================================
Files 180 180
Lines 52424 52424
==========================================
- Hits 43537 43536 -1
- Misses 8887 8888 +1
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [arrow/src/datatypes/datatype.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2RhdGF0eXBlcy9kYXRhdHlwZS5ycw==) | `66.40% <0.00%> (-0.40%)` | :arrow_down: |
| [parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2VuY29kaW5nLnJz) | `93.52% <0.00%> (-0.20%)` | :arrow_down: |
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `66.21% <0.00%> (ø)` | |
| [arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L3RyYW5zZm9ybS9tb2QucnM=) | `84.64% <0.00%> (+0.12%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [35e16be...359e459](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033279753
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1289](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6c3e7b8) into [master](https://codecov.io/gh/apache/arrow-rs/commit/35e16be01e680e9381b2d1393c2e3f8e7acb7b13?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (35e16be) will **not change** coverage.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1289 +/- ##
=======================================
Coverage 83.04% 83.04%
=======================================
Files 180 180
Lines 52424 52424
=======================================
Hits 43537 43537
Misses 8887 8887
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [arrow/src/datatypes/datatype.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2RhdGF0eXBlcy9kYXRhdHlwZS5ycw==) | `66.40% <0.00%> (-0.40%)` | :arrow_down: |
| [arrow/src/datatypes/field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2RhdGF0eXBlcy9maWVsZC5ycw==) | `53.79% <0.00%> (-0.31%)` | :arrow_down: |
| [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `66.21% <0.00%> (ø)` | |
| [arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/1289/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L3RyYW5zZm9ybS9tb2QucnM=) | `84.77% <0.00%> (+0.25%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [35e16be...6c3e7b8](https://codecov.io/gh/apache/arrow-rs/pull/1289?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dianaclarke commented on pull request #1289: Arrow Rust + Conbench Integration
Posted by GitBox <gi...@apache.org>.
dianaclarke commented on pull request #1289:
URL: https://github.com/apache/arrow-rs/pull/1289#issuecomment-1033912687
> One of the tests failed because it makes an assumption that a `target` directory exists and it did not. My assumption is that somehow the current working directory was not correct. This is how I initiated the benchmark run.
>
> ```shell
> ~/git/arrow-rs/conbench$ conbench arrow-rs --src-dir=`pwd`/..
> ```
>
> I see that the Python code should have changed the directory so I am not sure what went wrong. I will start debugging this tonight. We may also want to have our tests write output to a temporary directory rather than making any assumptions about current working directory.
If you pull the latest, I just removed the need for `--src-dir`. Hopefully that works for you.
```
$ cd ~/arrow-rs/conbench/
$ conda create -y -n conbench python=3.9
$ conda activate conbench
(conbench) $ pip install -r requirements.txt
(conbench) $ conbench arrow-rs
```
PS. I don't work on arrow at all anymore, so I better go back to my day job now – just didn't want to leave you hanging. Cheers!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org