You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Bruno P. Kinoshita (JIRA)" <ji...@apache.org> on 2016/12/16 02:45:58 UTC
[jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using
JMH
[ https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753237#comment-15753237 ]
Bruno P. Kinoshita commented on LANG-1110:
------------------------------------------
As I never heard of JMH before, decided to have fun checking out the pull request locally, and changing the HashSetvBitSetTest to use JMH and run it.
Will attach the HashSetvBitSetTest.java here. It took 54:09 minutes to run, without changing any settings in the pom.xml (Ubuntu 16.04, Macbook i5 2.7, 8 cores, 16 GB, not virtualized). Here's the benchmark results that appear in the console. Not sure if we could add it to the project web site in some way. There may have some maven plug-in for that, but I couldn't find it by just googling JMH and maven plug-in.
{noformat}
...
Iteration 19: 183.914 ns/op
Iteration 20: 184.460 ns/op
# Run progress: 97.50% complete, ETA 00:00:40
# Fork: 10 of 10
# Warmup Iteration 1: 214.977 ns/op
# Warmup Iteration 2: 189.999 ns/op
# Warmup Iteration 3: 193.562 ns/op
# Warmup Iteration 4: 182.608 ns/op
# Warmup Iteration 5: 188.630 ns/op
# Warmup Iteration 6: 181.914 ns/op
# Warmup Iteration 7: 183.420 ns/op
# Warmup Iteration 8: 184.043 ns/op
# Warmup Iteration 9: 183.083 ns/op
# Warmup Iteration 10: 181.218 ns/op
# Warmup Iteration 11: 183.786 ns/op
# Warmup Iteration 12: 182.048 ns/op
# Warmup Iteration 13: 189.763 ns/op
# Warmup Iteration 14: 182.147 ns/op
# Warmup Iteration 15: 183.410 ns/op
# Warmup Iteration 16: 187.146 ns/op
# Warmup Iteration 17: 184.401 ns/op
# Warmup Iteration 18: 183.841 ns/op
# Warmup Iteration 19: 185.179 ns/op
# Warmup Iteration 20: 182.114 ns/op
Iteration 1: 183.059 ns/op
Iteration 2: 185.323 ns/op
Iteration 3: 184.455 ns/op
Iteration 4: 186.051 ns/op
Iteration 5: 187.268 ns/op
Iteration 6: 190.621 ns/op
Iteration 7: 184.044 ns/op
Iteration 8: 183.438 ns/op
Iteration 9: 183.282 ns/op
Iteration 10: 182.034 ns/op
Iteration 11: 183.265 ns/op
Iteration 12: 183.481 ns/op
Iteration 13: 183.454 ns/op
Iteration 14: 184.774 ns/op
Iteration 15: 189.690 ns/op
Iteration 16: 188.991 ns/op
Iteration 17: 183.585 ns/op
Iteration 18: 181.612 ns/op
Iteration 19: 182.598 ns/op
Iteration 20: 184.105 ns/op
Result "timeExtractRemoveAll":
196.003 ±(99.9%) 9.039 ns/op [Average]
(min, avg, max) = (164.015, 196.003, 432.640), stdev = 38.270
CI (99.9%): [186.965, 205.042] (assumes normal distribution)
# Run complete. Total time: 00:26:55
Benchmark Mode Cnt Score Error Units
HashSetvBitSetTest.testBitSet avgt 200 53.333 ± 2.263 ns/op
HashSetvBitSetTest.testHashSet avgt 200 212.623 ± 4.806 ns/op
HashSetvBitSetTest.timeBitSetRemoveAll avgt 200 133.441 ± 7.006 ns/op
HashSetvBitSetTest.timeExtractRemoveAll avgt 200 196.003 ± 9.039 ns/op
Benchmark result is saved to target/jmh-result.org.apache.commons.lang3.HashSetvBitSetTest.json
[INFO]
[INFO] --- maven-jar-plugin:2.6:jar (default-jar) @ commons-lang3 ---
[INFO] Building jar: /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ commons-lang3 ---
[INFO]
[INFO] --- maven-jar-plugin:2.6:test-jar (default) @ commons-lang3 ---
[INFO] Building jar: /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ commons-lang3 ---
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT.jar to /home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT.jar
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/pom.xml to /home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT.pom
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT-tests.jar to /home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT-tests.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 54:09 min
[INFO] Finished at: 2016-12-16T15:14:27+13:00
[INFO] Final Memory: 44M/720M
[INFO] ------------------------------------------------------------------------
{noformat}
I added the JMH annotations that I found in the [project examples|http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/] and that seemed to make sense to me. Someone with experience on JMH tests can probably suggest a better set up.
Hope that helps
Bruno
> Implement HashSetvBitSetTest using JMH
> --------------------------------------
>
> Key: LANG-1110
> URL: https://issues.apache.org/jira/browse/LANG-1110
> Project: Commons Lang
> Issue Type: Test
> Components: lang.*
> Reporter: Benedikt Ritter
> Fix For: Patch Needed, 3.6
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for unit tests. Since HashSetvBitSetTest is really a performance test and not a unit test, we should reimplement it using OpenJMH. The results could then be published on our website.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)