You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "austin3dickey (via GitHub)" <gi...@apache.org> on 2024/04/05 14:07:36 UTC
[PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
austin3dickey opened a new pull request, #41031:
URL: https://github.com/apache/arrow/pull/41031
Fixes #41015. This PR allows users to run `yarn perf` to run the JS benchmarks on more machines; in particular, machines that do not support `env -S` like Amazon Linux.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "austin3dickey (via GitHub)" <gi...@apache.org>.
austin3dickey commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2039904570
@ursabot please benchmark lang=JavaScript
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040445105
Thanks for your patience. Conbench analyzed the 1 benchmarking run that has been run so far on PR commit e5f6c9382f748f9549d16394eaa7bdeab765068e.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/23501196099) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "austin3dickey (via GitHub)" <gi...@apache.org>.
austin3dickey commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040622605
I think that might be slightly difficult on the orchestration side. Cool, thanks for the review. 💜 would you mind merging? I don't have write access.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2039902806
:warning: GitHub issue #41015 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2039904754
Benchmark runs are scheduled for commit e5f6c9382f748f9549d16394eaa7bdeab765068e. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "domoritz (via GitHub)" <gi...@apache.org>.
domoritz commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040037666
Sounds good to me. We have a few other places that use the same shebang and I think we should update them as well.
Rather than yarn, maybe we should use npm run since it's more portable. But that can be a separate change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "domoritz (via GitHub)" <gi...@apache.org>.
domoritz merged PR #41031:
URL: https://github.com/apache/arrow/pull/41031
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040963098
After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 02bc6537f5989bc37896429394e8c6b0cb843889.
There was 1 benchmark result indicating a performance regression:
- Commit Run on `ursa-i9-9960x` at [2024-04-05 21:57:32Z](https://conbench.ursa.dev/compare/runs/716aee73291f4094a60e8d1c6521d48f...2ceec05a028840f7a2639a2e4578f903/)
- [`tpch` (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1](https://conbench.ursa.dev/compare/benchmarks/0661065fc57a732d80001b774432f048...0661086e13bf78dc8000a6f34eeab333)
The [full Conbench report](https://github.com/apache/arrow/runs/23512990185) has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "austin3dickey (via GitHub)" <gi...@apache.org>.
austin3dickey commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040022164
>Where else do we call perf from? Could we just change all the calls to run yarn perf / npm run perf?
As far as I know, everything that runs benchmarks just uses `yarn perf`.
>Now we have the same command in two places
If this is a problem, perhaps we should remove the shebang then? The trouble with the shebang as written is that using `/usr/bin/env -S` is non-portable. So if we're using `yarn perf` everywhere, it should be fine to remove the shebang.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "austin3dickey (via GitHub)" <gi...@apache.org>.
austin3dickey commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040071984
>We have a few other places that use the same shebang and I think we should update them as well.
Just looked into this briefly and found places in archery that might fail if we remove the shebangs:
https://github.com/apache/arrow/blob/83359d6958273be534f376f40976196c14675c8b/dev/archery/archery/integration/tester_js.py#L27-L30
So we could go through and fix all these, but it might be more trouble than it's worth. Would you mind if I keep this PR scoped to just fixing the benchmarks' portability?
(For context, I'm trying to move the orchestration of the Arrow merge-commit JS benchmarks off a machine in someone's basement to an EC2 instance. This PR unblocks that work.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably [arrow]
Posted by "domoritz (via GitHub)" <gi...@apache.org>.
domoritz commented on PR #41031:
URL: https://github.com/apache/arrow/pull/41031#issuecomment-2040371792
> So we could go through and fix all these, but it might be more trouble than it's worth. Would you mind if I keep this PR scoped to just fixing the benchmarks' portability?
Agreed. So to unblock you, can you run `node --no-warnings --loader ts-node/esm/transpile-only perf/index.ts` from the perf script? Otherwise, sure, let's merge this as is. I approve either way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org