You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/09/23 06:51:49 UTC
[GitHub] [arrow] liyafan82 opened a new pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
liyafan82 opened a new pull request #8245:
URL: https://github.com/apache/arrow/pull/8245
See https://issues.apache.org/jira/browse/ARROW-10069
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kiszk edited a comment on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
kiszk edited a comment on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-698541508
Thank you. Looks good.
In my environment, while `benchmark.forks` works, `benchmark.warmups` and `benchmark.runs` do not work. In other words, the following command executes the benchmark twice with 5 warmups and 5 runs.
```
mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly -Dbenchmark.forks=2 -Dbenchmark.warmups=6 -Dbenchmark.runs=7
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-697177304
https://issues.apache.org/jira/browse/ARROW-10069
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] liyafan82 commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
liyafan82 commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-699584585
@lidavidm Thanks for your feedback. Please see my reply in line.
> LGTM. Two questions:
>
> * Should we include this in the developer docs in Sphinx, or in the README in the Java subproject?
Thanks for the good suggestion. It seems we do not have a development guide for Java. Maybe we should add one in a separate issue.
> * How do I run the benchmarks without cleaning and re-building the project? (This seems to do it: `mvn install --also-make -Dskip.perf.benchmarks=false -Dbenchmark.filter=.\*IntBenchmarks.setIntDirectly -pl :arrow-performance`)
Good question. We can run
```
mvn exec:exec -Dskip.perf.benchmarks=false ...
```
However, it is fast to rebuild the code, as there are only a few classes in the performance module. Rebuilding is helpful for making the generated benchmark code up to date.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm closed pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
lidavidm closed pull request #8245:
URL: https://github.com/apache/arrow/pull/8245
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kiszk edited a comment on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
kiszk edited a comment on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-698541508
Thank you. Looks good.
In my environment, while `benchmark.forks` works, `benchmark.warmups` and `benchmark.runs` do not work. In other words, the following command executes the benchmark twice with 5 warmups and 5 runs.
```
mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly -Dbenchmark.forks=2 -Dbenchmark.warmups=6 -Dbenchmark.runs=7
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] liyafan82 commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
liyafan82 commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-697173842
The straightforward way of running a Java benchmark is through the following command:
```
java -jar ./performance/target/benchmarks.jar
```
However, it is like to fail with the following message:
```
ERROR: Unable to find the resource: /META-INF/BenchmarkList
```
The fundamental reason for this error is missing some jars in the class path, and it is considered a bug of maven compilation plugin (https://issues.apache.org/jira/browse/MCOMPILER-369)
Since there are many jars required (directly or indirected referenced), we recommend running the benchmarks through maven.
To run all benchmarks, please go to the arrow-performance directory, and invoke the command:
```
mvn clean install -Dskip.perf.benchmarks=false
```
We also support filtering specific benchmarks, for example:
```
mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly
```
In addtion, we also support configuring other benchmark parameters, for example:
```
mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly -Dbenchmark.forks=1 -Dbenchmark.warmups=5 -Dbenchmark.runs=5
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kiszk commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
kiszk commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-699658104
Thank you. Now, these three parameters work correctly as we expect.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kiszk commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
kiszk commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-698541508
Looks good.
In my environment, while `benchmark.forks` works, `benchmark.warmups` and `benchmark.runs` do not work. In other words, the following command executes the benchmark twice with 5 warmups and 5 runs.
```
mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly -Dbenchmark.forks=2 -Dbenchmark.warmups=6 -Dbenchmark.runs=7
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kiszk commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
kiszk commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-698541508
Looks good.
In my environment, while `benchmark.forks` works, `benchmark.warmups` and `benchmark.runs` do not work. In other words, the following command executes the benchmark twice with 5 warmups and 5 runs.
```
mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly -Dbenchmark.forks=2 -Dbenchmark.warmups=6 -Dbenchmark.runs=7
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] liyafan82 commented on pull request #8245: ARROW-10069: [Java] Support running Java benchmarks from command line
Posted by GitBox <gi...@apache.org>.
liyafan82 commented on pull request #8245:
URL: https://github.com/apache/arrow/pull/8245#issuecomment-699584686
> Thank you. Looks good.
>
> In my environment, while `benchmark.forks` works, `benchmark.warmups` and `benchmark.runs` do not work. In other words, the following command executes the benchmark twice with 5 warmups and 5 runs.
>
> ```
> mvn clean install -Dskip.perf.benchmarks=false -Dbenchmark.filter=.*IntBenchmarks.setIntDirectly -Dbenchmark.forks=2 -Dbenchmark.warmups=6 -Dbenchmark.runs=7
> ```
@kiszk Thanks for your feedback.
The problem was caused by incorrect parameter names. I have fixed it now, please check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org