You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/12/06 09:38:22 UTC
[GitHub] [arrow] cyb70289 opened a new pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
cyb70289 opened a new pull request #11865:
URL: https://github.com/apache/arrow/pull/11865
If no benchmark matches the suite/benchmark filter, current benchmark
tool will build Arrow twice, run no benchmark, and fail with confusing
error message.
This patch is to fail after the first build when the benchmark tool
finds no benchmark to run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
kszucs commented on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-986840190
@cyb70289 can we cover this change with a unittest in `test_benchmarks.py`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 commented on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
cyb70289 commented on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-987513236
> @cyb70289 can we cover this change with a unittest in `test_benchmarks.py`?
@kszucs, not sure if there's simple way to test it.
Current workflow is to first build all the benchmarks, then scan `*-benchmark` files and select wanted binaries per **suite_filter**, then list benchmarks by running the binary `xxx-benchmark --benchmark_list_tests` and apply **benchmark_filter** to select wanted benchmarks. Only after that we can judge if the suite/benchmark filter is valid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#discussion_r763174999
##########
File path: dev/archery/archery/benchmark/runner.py
##########
@@ -180,8 +181,12 @@ def suites(self):
.format(suite_name))
continue
+ suite_found = True
yield suite
+ if not suite_found:
+ raise ValueError("No benchmark matches the filter")
Review comment:
Nit: "the suite filter" perhaps?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 closed pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
cyb70289 closed pull request #11865:
URL: https://github.com/apache/arrow/pull/11865
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-989433563
Benchmark runs are scheduled for baseline = 9d200f5a4466e0ae2731abbf2949fa5e118bb056 and contender = f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2. f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/a0215c0cd36a4a349f488f99f7ba97d4...976eff87fba24c4789a59764071a42e3/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a07bdb7a1a6a4af5b8c4667adbd765f4...37be7efbc9a648298a9639efad9fcfcb/)
[Finished :arrow_down:0.49% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/30066b2999c24ebf8e48d45eaa5c7cc9...c72d614a1fa14aff83e626705bfecafd/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-989433563
Benchmark runs are scheduled for baseline = 9d200f5a4466e0ae2731abbf2949fa5e118bb056 and contender = f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2. f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/a0215c0cd36a4a349f488f99f7ba97d4...976eff87fba24c4789a59764071a42e3/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a07bdb7a1a6a4af5b8c4667adbd765f4...37be7efbc9a648298a9639efad9fcfcb/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/30066b2999c24ebf8e48d45eaa5c7cc9...c72d614a1fa14aff83e626705bfecafd/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-989433563
Benchmark runs are scheduled for baseline = 9d200f5a4466e0ae2731abbf2949fa5e118bb056 and contender = f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2. f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/a0215c0cd36a4a349f488f99f7ba97d4...976eff87fba24c4789a59764071a42e3/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a07bdb7a1a6a4af5b8c4667adbd765f4...37be7efbc9a648298a9639efad9fcfcb/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/30066b2999c24ebf8e48d45eaa5c7cc9...c72d614a1fa14aff83e626705bfecafd/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-989433563
Benchmark runs are scheduled for baseline = 9d200f5a4466e0ae2731abbf2949fa5e118bb056 and contender = f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2. f0110cf26af3cc8f4bcb94da7fafd01974bbbfd2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/a0215c0cd36a4a349f488f99f7ba97d4...976eff87fba24c4789a59764071a42e3/)
[Failed :arrow_down:0.45% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a07bdb7a1a6a4af5b8c4667adbd765f4...37be7efbc9a648298a9639efad9fcfcb/)
[Finished :arrow_down:0.49% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/30066b2999c24ebf8e48d45eaa5c7cc9...c72d614a1fa14aff83e626705bfecafd/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #11865: ARROW-14976: [Dev][Archery] Fail early if no benchmark found
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11865:
URL: https://github.com/apache/arrow/pull/11865#issuecomment-986610773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org