You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2014/12/04 20:36:11 UTC
Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/
-----------------------------------------------------------
Review request for Aurora, Kevin Sweeney and Bill Farner.
Repository: aurora
Description
-------
This RB is superseding the https://reviews.apache.org/r/28474/.
I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
- Active development trail [2]
- Advanced featureset and built-in optimizations improving accuracy and consistency [3]
- Well documented set of examples [4]
- Large community experience and collective wisdom.
This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
[1] - http://openjdk.java.net/projects/code-tools/jmh/
[2] - http://hg.openjdk.java.net/code-tools/jmh/
[3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
[4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
[5] - https://github.com/melix/jmh-gradle-plugin
Diffs
-----
build.gradle fb729c5096108c535229e266fa9649f997e6da37
config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
Diff: https://reviews.apache.org/r/28710/diff/
Testing
-------
$ ./gradlew jmh
Sample results generated:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 1 s each
# Measurement: 3 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration 1: 3156839103.911 ops/s
Iteration 1: 3333544897.411 ops/s
Iteration 2: 3357230627.218 ops/s
Iteration 3: 3461073727.560 ops/s
Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
Confidence interval (99.9%): [2146420835.212, 4621478666.247]
# Run complete. Total time: 00:00:05
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review63889
-----------------------------------------------------------
Ship it!
Master (11da34c) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 4, 2014, 7:36 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2014, 7:36 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle fb729c5096108c535229e266fa9649f997e6da37
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review63900
-----------------------------------------------------------
Ship it!
Master (11da34c) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 4, 2014, 8:20 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2014, 8:20 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle fb729c5096108c535229e266fa9649f997e6da37
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64056
-----------------------------------------------------------
Ship it!
Master (40d2718) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 5, 2014, 6:32 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 6:32 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Kevin Sweeney <ke...@apache.org>.
> On Dec. 5, 2014, 12:06 p.m., Bill Farner wrote:
> > Not to bikeshed, but do either of you have a sense for what it would take to put the benchmarking code into a gradle subproject? My sense is that would avoid further complicating the root build.gradle, but i'm not familiar enough with gradle to know what, if any, hurdles that would present.
>
> Maxim Khutornenko wrote:
> There is not much to hide in a subproject as all you see here is configuring the plugin to run with our gradle. I don't see any sane way of hiding it in a subproject short of reimplementing the plugin ourselves.
>
> Bill Farner wrote:
> Kevin - do you agree with this? You're more versed in subprojects than i am, so i'll defer to your judgement.
>
> Bill Farner wrote:
> Ping - Kevin?
This plugin doesn't seem geared toward use in a subproject so I think leaving src/jmh in place is (if not ideal) acceptable here.
- Kevin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64066
-----------------------------------------------------------
On Dec. 9, 2014, 5:22 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 5:22 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 5, 2014, 8:06 p.m., Bill Farner wrote:
> > Not to bikeshed, but do either of you have a sense for what it would take to put the benchmarking code into a gradle subproject? My sense is that would avoid further complicating the root build.gradle, but i'm not familiar enough with gradle to know what, if any, hurdles that would present.
There is not much to hide in a subproject as all you see here is configuring the plugin to run with our gradle. I don't see any sane way of hiding it in a subproject short of reimplementing the plugin ourselves.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64066
-----------------------------------------------------------
On Dec. 5, 2014, 6:32 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 6:32 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Bill Farner <wf...@apache.org>.
> On Dec. 5, 2014, 8:06 p.m., Bill Farner wrote:
> > Not to bikeshed, but do either of you have a sense for what it would take to put the benchmarking code into a gradle subproject? My sense is that would avoid further complicating the root build.gradle, but i'm not familiar enough with gradle to know what, if any, hurdles that would present.
>
> Maxim Khutornenko wrote:
> There is not much to hide in a subproject as all you see here is configuring the plugin to run with our gradle. I don't see any sane way of hiding it in a subproject short of reimplementing the plugin ourselves.
Kevin - do you agree with this? You're more versed in subprojects than i am, so i'll defer to your judgement.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64066
-----------------------------------------------------------
On Dec. 5, 2014, 6:32 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 6:32 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Bill Farner <wf...@apache.org>.
> On Dec. 5, 2014, 8:06 p.m., Bill Farner wrote:
> > Not to bikeshed, but do either of you have a sense for what it would take to put the benchmarking code into a gradle subproject? My sense is that would avoid further complicating the root build.gradle, but i'm not familiar enough with gradle to know what, if any, hurdles that would present.
>
> Maxim Khutornenko wrote:
> There is not much to hide in a subproject as all you see here is configuring the plugin to run with our gradle. I don't see any sane way of hiding it in a subproject short of reimplementing the plugin ourselves.
>
> Bill Farner wrote:
> Kevin - do you agree with this? You're more versed in subprojects than i am, so i'll defer to your judgement.
Ping - Kevin?
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64066
-----------------------------------------------------------
On Dec. 6, 2014, 12:33 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 6, 2014, 12:33 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64066
-----------------------------------------------------------
Not to bikeshed, but do either of you have a sense for what it would take to put the benchmarking code into a gradle subproject? My sense is that would avoid further complicating the root build.gradle, but i'm not familiar enough with gradle to know what, if any, hurdles that would present.
- Bill Farner
On Dec. 5, 2014, 6:32 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 6:32 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64134
-----------------------------------------------------------
Ship it!
Master (40d2718) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 6, 2014, 12:33 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 6, 2014, 12:33 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64491
-----------------------------------------------------------
Ship it!
Ship It!
- Kevin Sweeney
On Dec. 9, 2014, 5:22 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 5:22 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64492
-----------------------------------------------------------
Ship it!
Master (2aac148) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 10, 2014, 1:22 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 1:22 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64578
-----------------------------------------------------------
Ship it!
Ship It!
- Bill Farner
On Dec. 10, 2014, 1:22 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2014, 1:22 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/
-----------------------------------------------------------
(Updated Dec. 10, 2014, 1:22 a.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
CR comments.
Repository: aurora
Description
-------
This RB is superseding the https://reviews.apache.org/r/28474/.
I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
- Active development trail [2]
- Advanced featureset and built-in optimizations improving accuracy and consistency [3]
- Well documented set of examples [4]
- Large community experience and collective wisdom.
This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
[1] - http://openjdk.java.net/projects/code-tools/jmh/
[2] - http://hg.openjdk.java.net/code-tools/jmh/
[3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
[4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
[5] - https://github.com/melix/jmh-gradle-plugin
Diffs (updated)
-----
build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
Diff: https://reviews.apache.org/r/28710/diff/
Testing
-------
$ ./gradlew jmh
Sample results generated:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 1 s each
# Measurement: 3 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration 1: 3156839103.911 ops/s
Iteration 1: 3333544897.411 ops/s
Iteration 2: 3357230627.218 ops/s
Iteration 3: 3461073727.560 ops/s
Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
Confidence interval (99.9%): [2146420835.212, 4621478666.247]
# Run complete. Total time: 00:00:05
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 10, 2014, 1:09 a.m., Kevin Sweeney wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java, lines 29-32
> > <https://reviews.apache.org/r/28710/diff/4/?file=784251#file784251line29>
> >
> > What does the main get us here? Isn't the gradle plugin wiring this up?
Good point. I used it to run benchmarks from command line but given the command line execution is rather tedious with all jmh flags passed into it, it's cleaner to support only one way via gradle. Dropped.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64487
-----------------------------------------------------------
On Dec. 6, 2014, 12:33 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 6, 2014, 12:33 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review64487
-----------------------------------------------------------
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java
<https://reviews.apache.org/r/28710/#comment107200>
What does the main get us here? Isn't the gradle plugin wiring this up?
- Kevin Sweeney
On Dec. 5, 2014, 4:33 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 4:33 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/
-----------------------------------------------------------
(Updated Dec. 6, 2014, 12:33 a.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
CR comments.
Repository: aurora
Description
-------
This RB is superseding the https://reviews.apache.org/r/28474/.
I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
- Active development trail [2]
- Advanced featureset and built-in optimizations improving accuracy and consistency [3]
- Well documented set of examples [4]
- Large community experience and collective wisdom.
This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
[1] - http://openjdk.java.net/projects/code-tools/jmh/
[2] - http://hg.openjdk.java.net/code-tools/jmh/
[3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
[4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
[5] - https://github.com/melix/jmh-gradle-plugin
Diffs (updated)
-----
build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
Diff: https://reviews.apache.org/r/28710/diff/
Testing
-------
$ ./gradlew jmh
Sample results generated:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 1 s each
# Measurement: 3 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration 1: 3156839103.911 ops/s
Iteration 1: 3333544897.411 ops/s
Iteration 2: 3357230627.218 ops/s
Iteration 3: 3461073727.560 ops/s
Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
Confidence interval (99.9%): [2146420835.212, 4621478666.247]
# Run complete. Total time: 00:00:05
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 5, 2014, 7:42 p.m., Kevin Sweeney wrote:
> > build.gradle, line 53
> > <https://reviews.apache.org/r/28710/diff/1/?file=782698#file782698line53>
> >
> > This should be unneeded.
Dropped.
> On Dec. 5, 2014, 7:42 p.m., Kevin Sweeney wrote:
> > build.gradle, line 103
> > <https://reviews.apache.org/r/28710/diff/3/?file=783833#file783833line103>
> >
> > im not convinced you want to apply this to the api subproject.
This was only needed to make :api recognize .jmh configuration. However, played a bit more and found a way to reference parent configuration. Dropped.
> On Dec. 5, 2014, 7:42 p.m., Kevin Sweeney wrote:
> > build.gradle, line 156
> > <https://reviews.apache.org/r/28710/diff/3/?file=783833#file783833line156>
> >
> > Do we want to apply jmh to the 'api' project? I'd think we'd want to add the root. That will give you the benefit of not having to disable this task on the api project at all.
Gone.
> On Dec. 5, 2014, 7:42 p.m., Kevin Sweeney wrote:
> > build.gradle, line 165
> > <https://reviews.apache.org/r/28710/diff/3/?file=783833#file783833line165>
> >
> > should this be a test scope? also should this be in the root project instead of the api project?
It's only needed to resolve /jmh deps. Test scope is irrelevant here.
> On Dec. 5, 2014, 7:42 p.m., Kevin Sweeney wrote:
> > build.gradle, line 458
> > <https://reviews.apache.org/r/28710/diff/3/?file=783833#file783833line458>
> >
> > Does
> >
> > ```groovy
> > tasks('jmh') << {
> > //...
> > }
> > ```
> >
> > work?
It does not: `> Could not find method tasks() for arguments [jmh] on root project 'aurora'.`
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review63897
-----------------------------------------------------------
On Dec. 5, 2014, 6:32 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 6:32 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review63897
-----------------------------------------------------------
build.gradle
<https://reviews.apache.org/r/28710/#comment106210>
This should be unneeded.
build.gradle
<https://reviews.apache.org/r/28710/#comment106452>
im not convinced you want to apply this to the api subproject.
build.gradle
<https://reviews.apache.org/r/28710/#comment106449>
Do we want to apply jmh to the 'api' project? I'd think we'd want to add the root. That will give you the benefit of not having to disable this task on the api project at all.
build.gradle
<https://reviews.apache.org/r/28710/#comment106451>
should this be a test scope? also should this be in the root project instead of the api project?
build.gradle
<https://reviews.apache.org/r/28710/#comment106454>
Does
```groovy
tasks('jmh') << {
//...
}
```
work?
- Kevin Sweeney
On Dec. 5, 2014, 10:32 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 10:32 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/
-----------------------------------------------------------
(Updated Dec. 5, 2014, 6:32 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Rebased.
Repository: aurora
Description
-------
This RB is superseding the https://reviews.apache.org/r/28474/.
I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
- Active development trail [2]
- Advanced featureset and built-in optimizations improving accuracy and consistency [3]
- Well documented set of examples [4]
- Large community experience and collective wisdom.
This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
[1] - http://openjdk.java.net/projects/code-tools/jmh/
[2] - http://hg.openjdk.java.net/code-tools/jmh/
[3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
[4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
[5] - https://github.com/melix/jmh-gradle-plugin
Diffs (updated)
-----
build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
Diff: https://reviews.apache.org/r/28710/diff/
Testing
-------
$ ./gradlew jmh
Sample results generated:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 1 s each
# Measurement: 3 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration 1: 3156839103.911 ops/s
Iteration 1: 3333544897.411 ops/s
Iteration 2: 3357230627.218 ops/s
Iteration 3: 3461073727.560 ops/s
Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
Confidence interval (99.9%): [2146420835.212, 4621478666.247]
# Run complete. Total time: 00:00:05
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/
-----------------------------------------------------------
(Updated Dec. 4, 2014, 8:20 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
CR comments.
Repository: aurora
Description
-------
This RB is superseding the https://reviews.apache.org/r/28474/.
I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
- Active development trail [2]
- Advanced featureset and built-in optimizations improving accuracy and consistency [3]
- Well documented set of examples [4]
- Large community experience and collective wisdom.
This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
[1] - http://openjdk.java.net/projects/code-tools/jmh/
[2] - http://hg.openjdk.java.net/code-tools/jmh/
[3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
[4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
[5] - https://github.com/melix/jmh-gradle-plugin
Diffs (updated)
-----
build.gradle fb729c5096108c535229e266fa9649f997e6da37
config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
Diff: https://reviews.apache.org/r/28710/diff/
Testing
-------
$ ./gradlew jmh
Sample results generated:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 1 s each
# Measurement: 3 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
# Run progress: 0.00% complete, ETA 00:00:04
# Fork: 1 of 1
# Warmup Iteration 1: 3156839103.911 ops/s
Iteration 1: 3333544897.411 ops/s
Iteration 2: 3357230627.218 ops/s
Iteration 3: 3461073727.560 ops/s
Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
Confidence interval (99.9%): [2146420835.212, 4621478666.247]
# Run complete. Total time: 00:00:05
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 4, 2014, 8:12 p.m., Kevin Sweeney wrote:
> > build.gradle, line 23
> > <https://reviews.apache.org/r/28710/diff/1/?file=782698#file782698line23>
> >
> > Looks like you want the new-style block here instead:
> >
> > https://plugins.gradle.org/plugin/me.champeau.gradle.jmh
Great suggestion. Done.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review63896
-----------------------------------------------------------
On Dec. 4, 2014, 7:36 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2014, 7:36 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle fb729c5096108c535229e266fa9649f997e6da37
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28710: Adding JMH framework support for scheduler
performance analysis.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28710/#review63896
-----------------------------------------------------------
build.gradle
<https://reviews.apache.org/r/28710/#comment106209>
Looks like you want the new-style block here instead:
https://plugins.gradle.org/plugin/me.champeau.gradle.jmh
- Kevin Sweeney
On Dec. 4, 2014, 11:36 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28710/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2014, 11:36 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This RB is superseding the https://reviews.apache.org/r/28474/.
>
>
> I have spent some time researching the available microbenchmark frameworks and JMH [1] came as a clear winner:
> - Active development trail [2]
> - Advanced featureset and built-in optimizations improving accuracy and consistency [3]
> - Well documented set of examples [4]
> - Large community experience and collective wisdom.
>
>
> This RB adds gradle support for running JMH benchmarks and is relying on JMH gradle plugin [5]. The benchmarks are run via `./gradlew jmh` command.
>
> [1] - http://openjdk.java.net/projects/code-tools/jmh/
> [2] - http://hg.openjdk.java.net/code-tools/jmh/
> [3] - https://groups.google.com/forum/#!msg/mechanical-sympathy/m4opvy4xq3U/7lY8x8SvHgwJ
> [4] - http://hg.openjdk.java.net/code-tools/jmh/file/adb6047266d8/jmh-samples/src/main/java/org/openjdk/jmh/samples
> [5] - https://github.com/melix/jmh-gradle-plugin
>
>
> Diffs
> -----
>
> build.gradle fb729c5096108c535229e266fa9649f997e6da37
> config/findbugs/excludeFilter.xml d6c1b1681c2d8505a088f9fb082ce11ac400126f
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28710/diff/
>
>
> Testing
> -------
>
> $ ./gradlew jmh
>
> Sample results generated:
>
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 1 s each
> # Measurement: 3 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SchedulerBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:00:04
> # Fork: 1 of 1
> # Warmup Iteration 1: 3156839103.911 ops/s
> Iteration 1: 3333544897.411 ops/s
> Iteration 2: 3357230627.218 ops/s
> Iteration 3: 3461073727.560 ops/s
>
>
> Result: 3383949750.729 ±(99.9%) 1237528915.517 ops/s [Average]
> Statistics: (min, avg, max) = (3333544897.411, 3383949750.729, 3461073727.560), stdev = 67833135.714
> Confidence interval (99.9%): [2146420835.212, 4621478666.247]
>
>
> # Run complete. Total time: 00:00:05
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulerBenchmark.example thrpt 3 3383949750.729 ± 1237528915.517 ops/s
>
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>