You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "jeqo (via GitHub)" <gi...@apache.org> on 2023/01/31 16:45:16 UTC
[GitHub] [kafka] jeqo opened a new pull request, #13177: [KAFKA-14441] Benchmark performance impact of metrics library
jeqo opened a new pull request, #13177:
URL: https://github.com/apache/kafka/pull/13177
https://issues.apache.org/jira/browse/KAFKA-14441
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] github-actions[bot] commented on pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13177:
URL: https://github.com/apache/kafka/pull/13177#issuecomment-1585813259
This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)
If this PR is no longer valid or desired, please feel free to close it. If no activity occurrs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jeqo commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "jeqo (via GitHub)" <gi...@apache.org>.
jeqo commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1096076032
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
+@BenchmarkMode(Mode.AverageTime)
Review Comment:
Sure! I have added SampleTime, and got some interesting stats:
```
Benchmark (numSensors) Mode Cnt Score Error Units
SensorBenchmark.recordBenchmark 1 avgt 4 122.285 ± 3.831 ns/op
SensorBenchmark.recordBenchmark 5 avgt 4 535.055 ± 10.002 ns/op
SensorBenchmark.recordBenchmark 20 avgt 4 2153.720 ± 562.511 ns/op
SensorBenchmark.recordWithAvgBenchmark 1 avgt 4 148.655 ± 7.245 ns/op
SensorBenchmark.recordWithAvgBenchmark 5 avgt 4 801.836 ± 25.243 ns/op
SensorBenchmark.recordWithAvgBenchmark 20 avgt 4 5934.782 ± 463.325 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 1 avgt 4 153.731 ± 8.982 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 5 avgt 4 1067.862 ± 68.656 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 20 avgt 4 10486.323 ± 209.097 ns/op
SensorBenchmark.recordWithMaxBenchmark 1 avgt 4 148.492 ± 20.097 ns/op
SensorBenchmark.recordWithMaxBenchmark 5 avgt 4 807.746 ± 300.785 ns/op
SensorBenchmark.recordWithMaxBenchmark 20 avgt 4 6361.096 ± 1160.907 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 1 avgt 4 178.676 ± 12.371 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 5 avgt 4 1749.455 ± 204.069 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 20 avgt 4 25027.613 ± 1407.740 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 1 avgt 4 158.032 ± 9.249 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 5 avgt 4 1001.310 ± 48.258 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 20 avgt 4 8958.896 ± 264.169 ns/op
SensorBenchmark.recordWithPercentileBenchmark 1 avgt 4 133.637 ± 7.675 ns/op
SensorBenchmark.recordWithPercentileBenchmark 5 avgt 4 766.876 ± 14.212 ns/op
SensorBenchmark.recordWithPercentileBenchmark 20 avgt 4 5661.163 ± 463.749 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 1 avgt 4 119.896 ± 13.392 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 5 avgt 4 663.232 ± 9.704 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 20 avgt 4 4976.913 ± 430.296 ns/op
SensorBenchmark.recordBenchmark 1 sample 1306062 176.717 ± 2.809 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.00 1 sample 92.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.50 1 sample 135.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.90 1 sample 150.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.95 1 sample 156.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.99 1 sample 181.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.999 1 sample 19325.984 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.9999 1 sample 32108.598 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p1.00 1 sample 195328.000 ns/op
SensorBenchmark.recordBenchmark 5 sample 1023159 694.906 ± 7.662 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.00 5 sample 455.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.50 5 sample 565.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.90 5 sample 780.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.95 5 sample 847.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.99 5 sample 1214.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.999 5 sample 24192.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.9999 5 sample 60338.432 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p1.00 5 sample 716800.000 ns/op
SensorBenchmark.recordBenchmark 20 sample 1271143 2011.945 ± 7.842 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.00 20 sample 1610.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.50 20 sample 1750.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.90 20 sample 2100.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.95 20 sample 2892.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.99 20 sample 3616.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.999 20 sample 28576.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.9999 20 sample 77425.357 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p1.00 20 sample 1118208.000 ns/op
SensorBenchmark.recordWithAvgBenchmark 1 sample 1262271 179.238 ± 13.287 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.00 1 sample 102.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.50 1 sample 140.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.90 1 sample 149.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.95 1 sample 155.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.99 1 sample 174.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.999 1 sample 16272.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.9999 1 sample 29304.730 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p1.00 1 sample 4165632.000 ns/op
SensorBenchmark.recordWithAvgBenchmark 5 sample 933071 719.230 ± 4.662 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.00 5 sample 582.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.50 5 sample 633.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.90 5 sample 681.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.95 5 sample 718.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.99 5 sample 763.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.999 5 sample 24544.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.9999 5 sample 36588.339 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p1.00 5 sample 178176.000 ns/op
SensorBenchmark.recordWithAvgBenchmark 20 sample 1042420 4797.208 ± 8.407 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.00 20 sample 4052.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.50 20 sample 4496.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.90 20 sample 4712.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.95 20 sample 4976.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.99 20 sample 20864.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.999 20 sample 32480.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.9999 20 sample 64432.506 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p1.00 20 sample 113280.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 1 sample 1205175 182.177 ± 2.624 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.00 1 sample 68.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.50 1 sample 148.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.90 1 sample 159.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.95 1 sample 169.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.99 1 sample 180.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.999 1 sample 16730.368 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.9999 1 sample 29728.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p1.00 1 sample 114944.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 5 sample 1445629 899.259 ± 3.420 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.00 5 sample 746.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.50 5 sample 828.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.90 5 sample 870.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.95 5 sample 918.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.99 5 sample 994.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.999 5 sample 23296.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.9999 5 sample 33792.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p1.00 5 sample 206848.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 20 sample 1212757 8237.676 ± 10.315 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.00 20 sample 7000.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.50 20 sample 7608.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.90 20 sample 8192.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.95 20 sample 8672.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.99 20 sample 28672.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.999 20 sample 38336.000 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p0.9999 20 sample 71388.698 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark:recordWithMaxAvgBenchmark·p1.00 20 sample 172544.000 ns/op
SensorBenchmark.recordWithMaxBenchmark 1 sample 1260135 171.441 ± 2.398 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.00 1 sample 48.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.50 1 sample 141.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.90 1 sample 149.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.95 1 sample 156.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.99 1 sample 166.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.999 1 sample 16112.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.9999 1 sample 29344.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p1.00 1 sample 105856.000 ns/op
SensorBenchmark.recordWithMaxBenchmark 5 sample 927800 722.861 ± 4.686 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.00 5 sample 572.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.50 5 sample 648.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.90 5 sample 686.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.95 5 sample 723.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.99 5 sample 780.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.999 5 sample 24384.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.9999 5 sample 37070.074 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p1.00 5 sample 213760.000 ns/op
SensorBenchmark.recordWithMaxBenchmark 20 sample 1010754 4959.078 ± 9.340 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.00 20 sample 4248.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.50 20 sample 4600.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.90 20 sample 4832.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.95 20 sample 5200.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.99 20 sample 21440.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.999 20 sample 34496.000 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p0.9999 20 sample 83446.336 ns/op
SensorBenchmark.recordWithMaxBenchmark:recordWithMaxBenchmark·p1.00 20 sample 215040.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 1 sample 1061095 200.444 ± 2.934 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.00 1 sample 73.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.50 1 sample 163.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.90 1 sample 172.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.95 1 sample 181.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.99 1 sample 195.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.999 1 sample 17696.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.9999 1 sample 30140.493 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p1.00 1 sample 146688.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 5 sample 860679 1491.948 ± 6.049 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.00 5 sample 1264.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.50 5 sample 1372.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.90 5 sample 1434.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.95 5 sample 1522.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.99 5 sample 1660.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.999 5 sample 27722.240 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.9999 5 sample 44462.592 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p1.00 5 sample 236800.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 20 sample 1061996 18787.282 ± 19.680 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.00 20 sample 15936.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.50 20 sample 17472.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.90 20 sample 19040.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.95 20 sample 25376.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.99 20 sample 52928.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.999 20 sample 69760.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p0.9999 20 sample 100096.000 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark:recordWithMinMaxAvgBenchmark·p1.00 20 sample 203008.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 1 sample 1163816 211.907 ± 4.970 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.00 1 sample 96.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.50 1 sample 146.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.90 1 sample 156.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.95 1 sample 165.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.99 1 sample 210.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.999 1 sample 22208.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.9999 1 sample 67535.142 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p1.00 1 sample 349696.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 5 sample 1350850 965.156 ± 4.210 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.00 5 sample 770.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.50 5 sample 852.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.90 5 sample 959.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.95 5 sample 986.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.99 5 sample 1504.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.999 5 sample 24288.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.9999 5 sample 43712.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p1.00 5 sample 168192.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 20 sample 1211711 8249.891 ± 10.594 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.00 20 sample 7096.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.50 20 sample 7664.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.90 20 sample 8144.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.95 20 sample 8752.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.99 20 sample 28672.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.999 20 sample 38528.000 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p0.9999 20 sample 81298.432 ns/op
SensorBenchmark.recordWithMinMaxBenchmark:recordWithMinMaxBenchmark·p1.00 20 sample 325632.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark 1 sample 1160640 183.623 ± 2.605 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.00 1 sample 109.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.50 1 sample 150.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.90 1 sample 159.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.95 1 sample 166.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.99 1 sample 178.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.999 1 sample 16800.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.9999 1 sample 29245.949 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p1.00 1 sample 119040.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark 5 sample 838436 796.833 ± 5.200 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.00 5 sample 590.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.50 5 sample 705.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.90 5 sample 774.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.95 5 sample 802.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.99 5 sample 871.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.999 5 sample 25554.016 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.9999 5 sample 37258.003 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p1.00 5 sample 256256.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark 20 sample 926469 5392.666 ± 10.367 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.00 20 sample 4632.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.50 20 sample 5008.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.90 20 sample 5304.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.95 20 sample 5704.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.99 20 sample 22432.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.999 20 sample 34752.000 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p0.9999 20 sample 73780.736 ns/op
SensorBenchmark.recordWithPercentileBenchmark:recordWithPercentileBenchmark·p1.00 20 sample 699392.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 1 sample 1282204 169.071 ± 2.370 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.00 1 sample 105.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.50 1 sample 138.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.90 1 sample 149.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.95 1 sample 155.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.99 1 sample 167.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.999 1 sample 15840.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.9999 1 sample 29560.944 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p1.00 1 sample 168192.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 5 sample 961144 698.277 ± 4.367 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.00 5 sample 574.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.50 5 sample 630.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.90 5 sample 662.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.95 5 sample 698.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.99 5 sample 755.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.999 5 sample 23680.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.9999 5 sample 36578.688 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p1.00 5 sample 204288.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 20 sample 1056588 4733.302 ± 8.413 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.00 20 sample 4052.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.50 20 sample 4416.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.90 20 sample 4656.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.95 20 sample 4976.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.99 20 sample 20672.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.999 20 sample 33088.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.9999 20 sample 64064.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p1.00 20 sample 160000.000 ns/op
JMH benchmarks done
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jeqo commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "jeqo (via GitHub)" <gi...@apache.org>.
jeqo commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1096076032
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
+@BenchmarkMode(Mode.AverageTime)
Review Comment:
Sure! I have added SampleTime, and got some interesting stats:
```
Benchmark (numSensors) Mode Cnt Score Error Units
SensorBenchmark.recordBenchmark 1 avgt 4 122.285 ± 3.831 ns/op
SensorBenchmark.recordBenchmark 5 avgt 4 535.055 ± 10.002 ns/op
SensorBenchmark.recordBenchmark 20 avgt 4 2153.720 ± 562.511 ns/op
SensorBenchmark.recordWithAvgBenchmark 1 avgt 4 148.655 ± 7.245 ns/op
SensorBenchmark.recordWithAvgBenchmark 5 avgt 4 801.836 ± 25.243 ns/op
SensorBenchmark.recordWithAvgBenchmark 20 avgt 4 5934.782 ± 463.325 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 1 avgt 4 153.731 ± 8.982 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 5 avgt 4 1067.862 ± 68.656 ns/op
SensorBenchmark.recordWithMaxAvgBenchmark 20 avgt 4 10486.323 ± 209.097 ns/op
SensorBenchmark.recordWithMaxBenchmark 1 avgt 4 148.492 ± 20.097 ns/op
SensorBenchmark.recordWithMaxBenchmark 5 avgt 4 807.746 ± 300.785 ns/op
SensorBenchmark.recordWithMaxBenchmark 20 avgt 4 6361.096 ± 1160.907 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 1 avgt 4 178.676 ± 12.371 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 5 avgt 4 1749.455 ± 204.069 ns/op
SensorBenchmark.recordWithMinMaxAvgBenchmark 20 avgt 4 25027.613 ± 1407.740 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 1 avgt 4 158.032 ± 9.249 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 5 avgt 4 1001.310 ± 48.258 ns/op
SensorBenchmark.recordWithMinMaxBenchmark 20 avgt 4 8958.896 ± 264.169 ns/op
SensorBenchmark.recordWithPercentileBenchmark 1 avgt 4 133.637 ± 7.675 ns/op
SensorBenchmark.recordWithPercentileBenchmark 5 avgt 4 766.876 ± 14.212 ns/op
SensorBenchmark.recordWithPercentileBenchmark 20 avgt 4 5661.163 ± 463.749 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 1 avgt 4 119.896 ± 13.392 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 5 avgt 4 663.232 ± 9.704 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark 20 avgt 4 4976.913 ± 430.296 ns/op
SensorBenchmark.recordBenchmark 1 sample 1306062 176.717 ± 2.809 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.00 1 sample 92.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.50 1 sample 135.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.90 1 sample 150.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.95 1 sample 156.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.99 1 sample 181.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.999 1 sample 19325.984 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.9999 1 sample 32108.598 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p1.00 1 sample 195328.000 ns/op
SensorBenchmark.recordBenchmark 5 sample 1023159 694.906 ± 7.662 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.00 5 sample 455.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.50 5 sample 565.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.90 5 sample 780.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.95 5 sample 847.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.99 5 sample 1214.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.999 5 sample 24192.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.9999 5 sample 60338.432 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p1.00 5 sample 716800.000 ns/op
SensorBenchmark.recordBenchmark 20 sample 1271143 2011.945 ± 7.842 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.00 20 sample 1610.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.50 20 sample 1750.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.90 20 sample 2100.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.95 20 sample 2892.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.99 20 sample 3616.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.999 20 sample 28576.000 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p0.9999 20 sample 77425.357 ns/op
SensorBenchmark.recordBenchmark:recordBenchmark·p1.00 20 sample 1118208.000 ns/op
SensorBenchmark.recordWithAvgBenchmark 1 sample 1262271 179.238 ± 13.287 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.00 1 sample 102.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.50 1 sample 140.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.90 1 sample 149.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.95 1 sample 155.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.99 1 sample 174.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.999 1 sample 16272.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.9999 1 sample 29304.730 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p1.00 1 sample 4165632.000 ns/op
SensorBenchmark.recordWithAvgBenchmark 5 sample 933071 719.230 ± 4.662 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.00 5 sample 582.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.50 5 sample 633.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.90 5 sample 681.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.95 5 sample 718.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.99 5 sample 763.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.999 5 sample 24544.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.9999 5 sample 36588.339 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p1.00 5 sample 178176.000 ns/op
SensorBenchmark.recordWithAvgBenchmark 20 sample 1042420 4797.208 ± 8.407 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.00 20 sample 4052.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.50 20 sample 4496.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.90 20 sample 4712.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.95 20 sample 4976.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.99 20 sample 20864.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.999 20 sample 32480.000 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p0.9999 20 sample 64432.506 ns/op
SensorBenchmark.recordWithAvgBenchmark:recordWithAvgBenchmark·p1.00 20 sample 113280.000 ns/op
sample 204288.000 ns/op
...
SensorBenchmark.recordWithWindowedSumBenchmark 20 sample 1056588 4733.302 ± 8.413 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.00 20 sample 4052.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.50 20 sample 4416.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.90 20 sample 4656.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.95 20 sample 4976.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.99 20 sample 20672.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.999 20 sample 33088.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p0.9999 20 sample 64064.000 ns/op
SensorBenchmark.recordWithWindowedSumBenchmark:recordWithWindowedSumBenchmark·p1.00 20 sample 160000.000 ns/op
JMH benchmarks done
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jeqo commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "jeqo (via GitHub)" <gi...@apache.org>.
jeqo commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1096068517
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+public class SensorBenchmark {
+
+ @State(Scope.Benchmark)
+ public static class SensorWithNoStats {
+
+ Metrics metrics = new Metrics();
+
+ List<Sensor> sensors = new ArrayList<>();
+ @Param({"1", "5", "20"})
Review Comment:
Yes, e.g Sink task iteration has around 5 sensor records. Adding a comment to clarify this. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jeqo commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "jeqo (via GitHub)" <gi...@apache.org>.
jeqo commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1096074802
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
Review Comment:
> Unsure how representative the benchmark is for high-throughput invocations in production environments.
Yeah, I don't think these are highly representative, but a starting point to compare stats and how adding either sensors or stats affect latency (which is the concern from KIP-864).
Though, happy to hear thoughts on how to make these more solid. e.g. I have an idea to also add a benchmark of Yammer metrics to compare the impact of these two libraries.
Also, found there was some work around it:
- [MetricsBench](https://github.com/apache/kafka/blob/a8aedc85ebfadcf1472acafe2e0311a73d3040be/clients/src/test/java/org/apache/kafka/test/MetricsBench.java): simple average time execution of a sensor + a bunch of stats.
- [Microbenchmarks](https://github.com/apache/kafka/blob/00e5803cd3af89011254e734232308618403309d/clients/src/test/java/org/apache/kafka/test/Microbenchmarks.java) which seems more elaborated, but haven't looked into details yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] clolov commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "clolov (via GitHub)" <gi...@apache.org>.
clolov commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1094319881
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+public class SensorBenchmark {
+
+ @State(Scope.Benchmark)
+ public static class SensorWithNoStats {
+
+ Metrics metrics = new Metrics();
+
+ List<Sensor> sensors = new ArrayList<>();
+ @Param({"1", "5", "20"})
Review Comment:
In general I am more than happy with this benchmark, but where did these numbers come from? Are they roughly the number of sensors you have seen registered per Connector task?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] Hangleton commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "Hangleton (via GitHub)" <gi...@apache.org>.
Hangleton commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1094530705
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
Review Comment:
Is the number of warm-up and iterations really enough? 5 may be a bit low for the code path exercised. Unsure how representative the benchmark is for high-throughput invocations in production environments.
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
+@BenchmarkMode(Mode.AverageTime)
Review Comment:
What is the variance of the distribution? `SampleTime` may be useful to review the resulting percentiles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jeqo commented on a diff in pull request #13177: [KAFKA-14441] Benchmark performance impact of metrics library
Posted by "jeqo (via GitHub)" <gi...@apache.org>.
jeqo commented on code in PR #13177:
URL: https://github.com/apache/kafka/pull/13177#discussion_r1096069586
##########
jmh-benchmarks/src/main/java/org/apache/kafka/jmh/common/metrics/SensorBenchmark.java:
##########
@@ -0,0 +1,145 @@
+package org.apache.kafka.jmh.common.metrics;
+
+import org.apache.kafka.common.metrics.Metrics;
+import org.apache.kafka.common.metrics.Sensor;
+import org.apache.kafka.common.metrics.stats.Avg;
+import org.apache.kafka.common.metrics.stats.Max;
+import org.apache.kafka.common.metrics.stats.Min;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Param;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+@State(Scope.Benchmark)
+@Fork(value = 1)
+@Warmup(iterations = 5)
+@Measurement(iterations = 5)
Review Comment:
Sure, I reduced it for testing purpose. Increasing to 15, similar to other benchmarks — as I realize it's possible to override these via args.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org