You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Reza Motamedi <re...@gmail.com> on 2017/01/10 09:00:52 UTC
Re: Review Request 55284: Benchmarking SLA Algorithms
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55284/
-----------------------------------------------------------
(Updated Jan. 10, 2017, 9 a.m.)
Review request for Aurora, Joshua Cohen, Mehrdad Nurolahzade, and Santhosh Kumar Shanmugham.
Summary (updated)
-----------------
Benchmarking SLA Algorithms
Repository: aurora
Description
-------
# Adding SLA benchmarks benchmarks
This RB is related to [AURORA-1854](https://issues.apache.org/jira/browse/AURORA-1854).
The goal is to add JMH benchmarks for SlaCalculation and then other metric calculators.
Diffs
-----
src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java 3aabc5ef222a97d8ebde693c52030a66bca340d2
src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java 5d8d5bd8f705770979f284d26d2e932aabe707e5
Diff: https://reviews.apache.org/r/55284/diff/
Testing
-------
the content of <./aurora/dist/reports/jmh/human.txt>
```
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
# Parameters: (numTasks = 1000)
# Run progress: 0.00% complete, ETA 00:07:00
# Fork: 1 of 1
# Warmup Iteration 1: 5141.403 ops/s
Iteration 1: 5514.591 ops/s
Iteration 2: 5508.875 ops/s
Iteration 3: 5594.224 ops/s
Iteration 4: 5467.605 ops/s
Iteration 5: 5619.229 ops/s
Result "measureMTTA":
5540.905 �(99.9%) 244.059 ops/s [Average]
(min, avg, max) = (5467.605, 5540.905, 5619.229), stdev = 63.381
CI (99.9%): [5296.846, 5784.963] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
# Parameters: (numTasks = 10000)
# Run progress: 8.33% complete, ETA 00:06:34
# Fork: 1 of 1
# Warmup Iteration 1: 64.993 ops/s
Iteration 1: 61.762 ops/s
Iteration 2: 64.468 ops/s
Iteration 3: 65.047 ops/s
Iteration 4: 63.199 ops/s
Iteration 5: 64.412 ops/s
Result "measureMTTA":
63.778 �(99.9%) 5.053 ops/s [Average]
(min, avg, max) = (61.762, 63.778, 65.047), stdev = 1.312
CI (99.9%): [58.725, 68.830] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
# Parameters: (numTasks = 100000)
# Run progress: 16.67% complete, ETA 00:06:00
# Fork: 1 of 1
# Warmup Iteration 1: 0.490 ops/s
Iteration 1: 0.556 ops/s
Iteration 2: 0.503 ops/s
Iteration 3: 0.502 ops/s
Iteration 4: 0.576 ops/s
Iteration 5: 0.591 ops/s
Result "measureMTTA":
0.546 �(99.9%) 0.159 ops/s [Average]
(min, avg, max) = (0.502, 0.546, 0.591), stdev = 0.041
CI (99.9%): [0.387, 0.705] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
# Parameters: (numTasks = 200000)
# Run progress: 25.00% complete, ETA 00:05:40
# Fork: 1 of 1
# Warmup Iteration 1: 0.099 ops/s
Iteration 1: 0.130 ops/s
Iteration 2: 0.146 ops/s
Iteration 3: 0.117 ops/s
Iteration 4: 0.145 ops/s
Iteration 5: 0.144 ops/s
Result "measureMTTA":
0.137 �(99.9%) 0.049 ops/s [Average]
(min, avg, max) = (0.117, 0.137, 0.146), stdev = 0.013
CI (99.9%): [0.088, 0.185] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
# Parameters: (numTasks = 1000)
# Run progress: 33.33% complete, ETA 00:05:28
# Fork: 1 of 1
# Warmup Iteration 1: 4184.816 ops/s
Iteration 1: 4274.550 ops/s
Iteration 2: 2630.657 ops/s
Iteration 3: 3168.746 ops/s
Iteration 4: 4054.913 ops/s
Iteration 5: 3653.909 ops/s
Result "measureMTTR":
3556.555 �(99.9%) 2569.449 ops/s [Average]
(min, avg, max) = (2630.657, 3556.555, 4274.550), stdev = 667.278
CI (99.9%): [987.106, 6126.004] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
# Parameters: (numTasks = 10000)
# Run progress: 41.67% complete, ETA 00:04:40
# Fork: 1 of 1
# Warmup Iteration 1: 39.150 ops/s
Iteration 1: 56.293 ops/s
Iteration 2: 59.521 ops/s
Iteration 3: 46.747 ops/s
Iteration 4: 55.249 ops/s
Iteration 5: 52.481 ops/s
Result "measureMTTR":
54.058 �(99.9%) 18.486 ops/s [Average]
(min, avg, max) = (46.747, 54.058, 59.521), stdev = 4.801
CI (99.9%): [35.572, 72.544] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
# Parameters: (numTasks = 100000)
# Run progress: 50.00% complete, ETA 00:03:57
# Fork: 1 of 1
# Warmup Iteration 1: 0.448 ops/s
Iteration 1: 0.482 ops/s
Iteration 2: 0.475 ops/s
Iteration 3: 0.353 ops/s
Iteration 4: 0.478 ops/s
Iteration 5: 0.513 ops/s
Result "measureMTTR":
0.460 �(99.9%) 0.238 ops/s [Average]
(min, avg, max) = (0.353, 0.460, 0.513), stdev = 0.062
CI (99.9%): [0.223, 0.698] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
# Parameters: (numTasks = 200000)
# Run progress: 58.33% complete, ETA 00:03:23
# Fork: 1 of 1
# Warmup Iteration 1: 0.085 ops/s
Iteration 1: 0.125 ops/s
Iteration 2: 0.133 ops/s
Iteration 3: 0.131 ops/s
Iteration 4: 0.129 ops/s
Iteration 5: 0.117 ops/s
Result "measureMTTR":
0.127 �(99.9%) 0.025 ops/s [Average]
(min, avg, max) = (0.117, 0.127, 0.133), stdev = 0.006
CI (99.9%): [0.102, 0.152] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
# Parameters: (numTasks = 1000)
# Run progress: 66.67% complete, ETA 00:02:50
# Fork: 1 of 1
# Warmup Iteration 1: 3914.893 ops/s
Iteration 1: 4447.483 ops/s
Iteration 2: 1089.132 ops/s
Iteration 3: 1799.594 ops/s
Iteration 4: 4622.600 ops/s
Iteration 5: 4259.013 ops/s
Result "measureMTTS":
3243.565 �(99.9%) 6417.109 ops/s [Average]
(min, avg, max) = (1089.132, 3243.565, 4622.600), stdev = 1666.503
CI (99.9%): [? 0, 9660.673] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
# Parameters: (numTasks = 10000)
# Run progress: 75.00% complete, ETA 00:02:05
# Fork: 1 of 1
# Warmup Iteration 1: 55.750 ops/s
Iteration 1: 56.535 ops/s
Iteration 2: 57.740 ops/s
Iteration 3: 59.033 ops/s
Iteration 4: 55.974 ops/s
Iteration 5: 49.698 ops/s
Result "measureMTTS":
55.796 �(99.9%) 13.886 ops/s [Average]
(min, avg, max) = (49.698, 55.796, 59.033), stdev = 3.606
CI (99.9%): [41.910, 69.682] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
# Parameters: (numTasks = 100000)
# Run progress: 83.33% complete, ETA 00:01:22
# Fork: 1 of 1
# Warmup Iteration 1: 0.484 ops/s
Iteration 1: 0.523 ops/s
Iteration 2: 0.544 ops/s
Iteration 3: 0.550 ops/s
Iteration 4: 0.518 ops/s
Iteration 5: 0.506 ops/s
Result "measureMTTS":
0.528 �(99.9%) 0.071 ops/s [Average]
(min, avg, max) = (0.506, 0.528, 0.550), stdev = 0.018
CI (99.9%): [0.458, 0.599] (assumes normal distribution)
# JMH 1.15 (released 102 days ago)
# VM version: JDK 1.8.0_111, VM 25.111-b14
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
# VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 1 iterations, 10 s each
# Measurement: 5 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
# Parameters: (numTasks = 200000)
# Run progress: 91.67% complete, ETA 00:00:41
# Fork: 1 of 1
# Warmup Iteration 1: 0.099 ops/s
Iteration 1: 0.131 ops/s
Iteration 2: 0.116 ops/s
Iteration 3: 0.114 ops/s
Iteration 4: 0.107 ops/s
Iteration 5: 0.105 ops/s
Result "measureMTTS":
0.115 �(99.9%) 0.039 ops/s [Average]
(min, avg, max) = (0.105, 0.115, 0.131), stdev = 0.010
CI (99.9%): [0.076, 0.154] (assumes normal distribution)
# Run complete. Total time: 00:08:45
Benchmark (numTasks) Mode Cnt Score Error Units
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 1000 thrpt 5 5540.905 � 244.059 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 10000 thrpt 5 63.778 � 5.053 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 100000 thrpt 5 0.546 � 0.159 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 200000 thrpt 5 0.137 � 0.049 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 1000 thrpt 5 3556.555 � 2569.449 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 10000 thrpt 5 54.058 � 18.486 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 100000 thrpt 5 0.460 � 0.238 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 200000 thrpt 5 0.127 � 0.025 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 1000 thrpt 5 3243.565 � 6417.109 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 10000 thrpt 5 55.796 � 13.886 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 100000 thrpt 5 0.528 � 0.071 ops/s
SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 200000 thrpt 5 0.115 � 0.039 ops/s
Benchmark result is saved to dist/reports/jmh/results.txt
```
Thanks,
Reza Motamedi
Re: Review Request 55284: Benchmarking SLA Algorithms
Posted by Mehrdad Nurolahzade <me...@nurolahzade.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55284/#review161084
-----------------------------------------------------------
src/jmh/java/org/apache/aurora/benchmark/Tasks.java (lines 98 - 105)
<https://reviews.apache.org/r/55284/#comment232326>
I see what you are trying to keep task status and events consistant. But if I set the task events first and then set the status then I'll lose the events. Also setting status based on event timestamps is rather shaky (all those events are likely to have the same timestamp).
So, I would suggest moving the consistency checking logic to `build()` method and only verifying that the last event (last element of the `events` list) matches the status.
- Mehrdad Nurolahzade
On Jan. 10, 2017, 1 a.m., Reza Motamedi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55284/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 1 a.m.)
>
>
> Review request for Aurora, Joshua Cohen, Mehrdad Nurolahzade, and Santhosh Kumar Shanmugham.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> # Adding SLA benchmarks benchmarks
> This RB is related to [AURORA-1854](https://issues.apache.org/jira/browse/AURORA-1854).
> The goal is to add JMH benchmarks for SlaCalculation and then other metric calculators.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 3aabc5ef222a97d8ebde693c52030a66bca340d2
> src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java 5d8d5bd8f705770979f284d26d2e932aabe707e5
>
> Diff: https://reviews.apache.org/r/55284/diff/
>
>
> Testing
> -------
>
> the content of <./aurora/dist/reports/jmh/human.txt>
>
> ```
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 1000)
>
> # Run progress: 0.00% complete, ETA 00:07:00
> # Fork: 1 of 1
> # Warmup Iteration 1: 5141.403 ops/s
> Iteration 1: 5514.591 ops/s
> Iteration 2: 5508.875 ops/s
> Iteration 3: 5594.224 ops/s
> Iteration 4: 5467.605 ops/s
> Iteration 5: 5619.229 ops/s
>
>
> Result "measureMTTA":
> 5540.905 �(99.9%) 244.059 ops/s [Average]
> (min, avg, max) = (5467.605, 5540.905, 5619.229), stdev = 63.381
> CI (99.9%): [5296.846, 5784.963] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 10000)
>
> # Run progress: 8.33% complete, ETA 00:06:34
> # Fork: 1 of 1
> # Warmup Iteration 1: 64.993 ops/s
> Iteration 1: 61.762 ops/s
> Iteration 2: 64.468 ops/s
> Iteration 3: 65.047 ops/s
> Iteration 4: 63.199 ops/s
> Iteration 5: 64.412 ops/s
>
>
> Result "measureMTTA":
> 63.778 �(99.9%) 5.053 ops/s [Average]
> (min, avg, max) = (61.762, 63.778, 65.047), stdev = 1.312
> CI (99.9%): [58.725, 68.830] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 100000)
>
> # Run progress: 16.67% complete, ETA 00:06:00
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.490 ops/s
> Iteration 1: 0.556 ops/s
> Iteration 2: 0.503 ops/s
> Iteration 3: 0.502 ops/s
> Iteration 4: 0.576 ops/s
> Iteration 5: 0.591 ops/s
>
>
> Result "measureMTTA":
> 0.546 �(99.9%) 0.159 ops/s [Average]
> (min, avg, max) = (0.502, 0.546, 0.591), stdev = 0.041
> CI (99.9%): [0.387, 0.705] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 200000)
>
> # Run progress: 25.00% complete, ETA 00:05:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.099 ops/s
> Iteration 1: 0.130 ops/s
> Iteration 2: 0.146 ops/s
> Iteration 3: 0.117 ops/s
> Iteration 4: 0.145 ops/s
> Iteration 5: 0.144 ops/s
>
>
> Result "measureMTTA":
> 0.137 �(99.9%) 0.049 ops/s [Average]
> (min, avg, max) = (0.117, 0.137, 0.146), stdev = 0.013
> CI (99.9%): [0.088, 0.185] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 1000)
>
> # Run progress: 33.33% complete, ETA 00:05:28
> # Fork: 1 of 1
> # Warmup Iteration 1: 4184.816 ops/s
> Iteration 1: 4274.550 ops/s
> Iteration 2: 2630.657 ops/s
> Iteration 3: 3168.746 ops/s
> Iteration 4: 4054.913 ops/s
> Iteration 5: 3653.909 ops/s
>
>
> Result "measureMTTR":
> 3556.555 �(99.9%) 2569.449 ops/s [Average]
> (min, avg, max) = (2630.657, 3556.555, 4274.550), stdev = 667.278
> CI (99.9%): [987.106, 6126.004] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 10000)
>
> # Run progress: 41.67% complete, ETA 00:04:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 39.150 ops/s
> Iteration 1: 56.293 ops/s
> Iteration 2: 59.521 ops/s
> Iteration 3: 46.747 ops/s
> Iteration 4: 55.249 ops/s
> Iteration 5: 52.481 ops/s
>
>
> Result "measureMTTR":
> 54.058 �(99.9%) 18.486 ops/s [Average]
> (min, avg, max) = (46.747, 54.058, 59.521), stdev = 4.801
> CI (99.9%): [35.572, 72.544] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 100000)
>
> # Run progress: 50.00% complete, ETA 00:03:57
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.448 ops/s
> Iteration 1: 0.482 ops/s
> Iteration 2: 0.475 ops/s
> Iteration 3: 0.353 ops/s
> Iteration 4: 0.478 ops/s
> Iteration 5: 0.513 ops/s
>
>
> Result "measureMTTR":
> 0.460 �(99.9%) 0.238 ops/s [Average]
> (min, avg, max) = (0.353, 0.460, 0.513), stdev = 0.062
> CI (99.9%): [0.223, 0.698] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 200000)
>
> # Run progress: 58.33% complete, ETA 00:03:23
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.085 ops/s
> Iteration 1: 0.125 ops/s
> Iteration 2: 0.133 ops/s
> Iteration 3: 0.131 ops/s
> Iteration 4: 0.129 ops/s
> Iteration 5: 0.117 ops/s
>
>
> Result "measureMTTR":
> 0.127 �(99.9%) 0.025 ops/s [Average]
> (min, avg, max) = (0.117, 0.127, 0.133), stdev = 0.006
> CI (99.9%): [0.102, 0.152] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 1000)
>
> # Run progress: 66.67% complete, ETA 00:02:50
> # Fork: 1 of 1
> # Warmup Iteration 1: 3914.893 ops/s
> Iteration 1: 4447.483 ops/s
> Iteration 2: 1089.132 ops/s
> Iteration 3: 1799.594 ops/s
> Iteration 4: 4622.600 ops/s
> Iteration 5: 4259.013 ops/s
>
>
> Result "measureMTTS":
> 3243.565 �(99.9%) 6417.109 ops/s [Average]
> (min, avg, max) = (1089.132, 3243.565, 4622.600), stdev = 1666.503
> CI (99.9%): [? 0, 9660.673] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 10000)
>
> # Run progress: 75.00% complete, ETA 00:02:05
> # Fork: 1 of 1
> # Warmup Iteration 1: 55.750 ops/s
> Iteration 1: 56.535 ops/s
> Iteration 2: 57.740 ops/s
> Iteration 3: 59.033 ops/s
> Iteration 4: 55.974 ops/s
> Iteration 5: 49.698 ops/s
>
>
> Result "measureMTTS":
> 55.796 �(99.9%) 13.886 ops/s [Average]
> (min, avg, max) = (49.698, 55.796, 59.033), stdev = 3.606
> CI (99.9%): [41.910, 69.682] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 100000)
>
> # Run progress: 83.33% complete, ETA 00:01:22
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.484 ops/s
> Iteration 1: 0.523 ops/s
> Iteration 2: 0.544 ops/s
> Iteration 3: 0.550 ops/s
> Iteration 4: 0.518 ops/s
> Iteration 5: 0.506 ops/s
>
>
> Result "measureMTTS":
> 0.528 �(99.9%) 0.071 ops/s [Average]
> (min, avg, max) = (0.506, 0.528, 0.550), stdev = 0.018
> CI (99.9%): [0.458, 0.599] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 200000)
>
> # Run progress: 91.67% complete, ETA 00:00:41
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.099 ops/s
> Iteration 1: 0.131 ops/s
> Iteration 2: 0.116 ops/s
> Iteration 3: 0.114 ops/s
> Iteration 4: 0.107 ops/s
> Iteration 5: 0.105 ops/s
>
>
> Result "measureMTTS":
> 0.115 �(99.9%) 0.039 ops/s [Average]
> (min, avg, max) = (0.105, 0.115, 0.131), stdev = 0.010
> CI (99.9%): [0.076, 0.154] (assumes normal distribution)
>
>
> # Run complete. Total time: 00:08:45
>
> Benchmark (numTasks) Mode Cnt Score Error Units
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 1000 thrpt 5 5540.905 � 244.059 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 10000 thrpt 5 63.778 � 5.053 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 100000 thrpt 5 0.546 � 0.159 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 200000 thrpt 5 0.137 � 0.049 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 1000 thrpt 5 3556.555 � 2569.449 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 10000 thrpt 5 54.058 � 18.486 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 100000 thrpt 5 0.460 � 0.238 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 200000 thrpt 5 0.127 � 0.025 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 1000 thrpt 5 3243.565 � 6417.109 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 10000 thrpt 5 55.796 � 13.886 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 100000 thrpt 5 0.528 � 0.071 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 200000 thrpt 5 0.115 � 0.039 ops/s
>
> Benchmark result is saved to dist/reports/jmh/results.txt
>
>
> ```
>
>
> Thanks,
>
> Reza Motamedi
>
>
Re: Review Request 55284: Benchmarking SLA Algorithms
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55284/#review161046
-----------------------------------------------------------
Master (d4ebb56) is red with this patch.
./build-support/jenkins/build.sh
:commons:generateThriftResources
:commons:processResources
:commons:classes
:commons:jar
:compileJava/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:74: Note: Wrote forwarder org.apache.aurora.scheduler.storage.log.WriteAheadStorageForwarder
@Forward({
^
Note: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/org/apache/aurora/common/args/apt/cmdline.arg.info.txt.2
Note: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/META-INF/compiler/resource-mappings/org.apache.aurora.common.args.apt.CmdLineProcessor
:generateBuildProperties
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/Tasks.java:18:8: Unused import - java.util.Optional. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/Tasks.java:107:54: '{' is not preceded with whitespace. [WhitespaceAround]
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkstyleJmh'.
> Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/jmh.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1 mins 8.49 secs
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 10, 2017, 9 a.m., Reza Motamedi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55284/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 9 a.m.)
>
>
> Review request for Aurora, Joshua Cohen, Mehrdad Nurolahzade, and Santhosh Kumar Shanmugham.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> # Adding SLA benchmarks benchmarks
> This RB is related to [AURORA-1854](https://issues.apache.org/jira/browse/AURORA-1854).
> The goal is to add JMH benchmarks for SlaCalculation and then other metric calculators.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 3aabc5ef222a97d8ebde693c52030a66bca340d2
> src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java 5d8d5bd8f705770979f284d26d2e932aabe707e5
>
> Diff: https://reviews.apache.org/r/55284/diff/
>
>
> Testing
> -------
>
> the content of <./aurora/dist/reports/jmh/human.txt>
>
> ```
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 1000)
>
> # Run progress: 0.00% complete, ETA 00:07:00
> # Fork: 1 of 1
> # Warmup Iteration 1: 5141.403 ops/s
> Iteration 1: 5514.591 ops/s
> Iteration 2: 5508.875 ops/s
> Iteration 3: 5594.224 ops/s
> Iteration 4: 5467.605 ops/s
> Iteration 5: 5619.229 ops/s
>
>
> Result "measureMTTA":
> 5540.905 �(99.9%) 244.059 ops/s [Average]
> (min, avg, max) = (5467.605, 5540.905, 5619.229), stdev = 63.381
> CI (99.9%): [5296.846, 5784.963] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 10000)
>
> # Run progress: 8.33% complete, ETA 00:06:34
> # Fork: 1 of 1
> # Warmup Iteration 1: 64.993 ops/s
> Iteration 1: 61.762 ops/s
> Iteration 2: 64.468 ops/s
> Iteration 3: 65.047 ops/s
> Iteration 4: 63.199 ops/s
> Iteration 5: 64.412 ops/s
>
>
> Result "measureMTTA":
> 63.778 �(99.9%) 5.053 ops/s [Average]
> (min, avg, max) = (61.762, 63.778, 65.047), stdev = 1.312
> CI (99.9%): [58.725, 68.830] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 100000)
>
> # Run progress: 16.67% complete, ETA 00:06:00
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.490 ops/s
> Iteration 1: 0.556 ops/s
> Iteration 2: 0.503 ops/s
> Iteration 3: 0.502 ops/s
> Iteration 4: 0.576 ops/s
> Iteration 5: 0.591 ops/s
>
>
> Result "measureMTTA":
> 0.546 �(99.9%) 0.159 ops/s [Average]
> (min, avg, max) = (0.502, 0.546, 0.591), stdev = 0.041
> CI (99.9%): [0.387, 0.705] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 200000)
>
> # Run progress: 25.00% complete, ETA 00:05:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.099 ops/s
> Iteration 1: 0.130 ops/s
> Iteration 2: 0.146 ops/s
> Iteration 3: 0.117 ops/s
> Iteration 4: 0.145 ops/s
> Iteration 5: 0.144 ops/s
>
>
> Result "measureMTTA":
> 0.137 �(99.9%) 0.049 ops/s [Average]
> (min, avg, max) = (0.117, 0.137, 0.146), stdev = 0.013
> CI (99.9%): [0.088, 0.185] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 1000)
>
> # Run progress: 33.33% complete, ETA 00:05:28
> # Fork: 1 of 1
> # Warmup Iteration 1: 4184.816 ops/s
> Iteration 1: 4274.550 ops/s
> Iteration 2: 2630.657 ops/s
> Iteration 3: 3168.746 ops/s
> Iteration 4: 4054.913 ops/s
> Iteration 5: 3653.909 ops/s
>
>
> Result "measureMTTR":
> 3556.555 �(99.9%) 2569.449 ops/s [Average]
> (min, avg, max) = (2630.657, 3556.555, 4274.550), stdev = 667.278
> CI (99.9%): [987.106, 6126.004] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 10000)
>
> # Run progress: 41.67% complete, ETA 00:04:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 39.150 ops/s
> Iteration 1: 56.293 ops/s
> Iteration 2: 59.521 ops/s
> Iteration 3: 46.747 ops/s
> Iteration 4: 55.249 ops/s
> Iteration 5: 52.481 ops/s
>
>
> Result "measureMTTR":
> 54.058 �(99.9%) 18.486 ops/s [Average]
> (min, avg, max) = (46.747, 54.058, 59.521), stdev = 4.801
> CI (99.9%): [35.572, 72.544] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 100000)
>
> # Run progress: 50.00% complete, ETA 00:03:57
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.448 ops/s
> Iteration 1: 0.482 ops/s
> Iteration 2: 0.475 ops/s
> Iteration 3: 0.353 ops/s
> Iteration 4: 0.478 ops/s
> Iteration 5: 0.513 ops/s
>
>
> Result "measureMTTR":
> 0.460 �(99.9%) 0.238 ops/s [Average]
> (min, avg, max) = (0.353, 0.460, 0.513), stdev = 0.062
> CI (99.9%): [0.223, 0.698] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 200000)
>
> # Run progress: 58.33% complete, ETA 00:03:23
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.085 ops/s
> Iteration 1: 0.125 ops/s
> Iteration 2: 0.133 ops/s
> Iteration 3: 0.131 ops/s
> Iteration 4: 0.129 ops/s
> Iteration 5: 0.117 ops/s
>
>
> Result "measureMTTR":
> 0.127 �(99.9%) 0.025 ops/s [Average]
> (min, avg, max) = (0.117, 0.127, 0.133), stdev = 0.006
> CI (99.9%): [0.102, 0.152] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 1000)
>
> # Run progress: 66.67% complete, ETA 00:02:50
> # Fork: 1 of 1
> # Warmup Iteration 1: 3914.893 ops/s
> Iteration 1: 4447.483 ops/s
> Iteration 2: 1089.132 ops/s
> Iteration 3: 1799.594 ops/s
> Iteration 4: 4622.600 ops/s
> Iteration 5: 4259.013 ops/s
>
>
> Result "measureMTTS":
> 3243.565 �(99.9%) 6417.109 ops/s [Average]
> (min, avg, max) = (1089.132, 3243.565, 4622.600), stdev = 1666.503
> CI (99.9%): [? 0, 9660.673] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 10000)
>
> # Run progress: 75.00% complete, ETA 00:02:05
> # Fork: 1 of 1
> # Warmup Iteration 1: 55.750 ops/s
> Iteration 1: 56.535 ops/s
> Iteration 2: 57.740 ops/s
> Iteration 3: 59.033 ops/s
> Iteration 4: 55.974 ops/s
> Iteration 5: 49.698 ops/s
>
>
> Result "measureMTTS":
> 55.796 �(99.9%) 13.886 ops/s [Average]
> (min, avg, max) = (49.698, 55.796, 59.033), stdev = 3.606
> CI (99.9%): [41.910, 69.682] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 100000)
>
> # Run progress: 83.33% complete, ETA 00:01:22
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.484 ops/s
> Iteration 1: 0.523 ops/s
> Iteration 2: 0.544 ops/s
> Iteration 3: 0.550 ops/s
> Iteration 4: 0.518 ops/s
> Iteration 5: 0.506 ops/s
>
>
> Result "measureMTTS":
> 0.528 �(99.9%) 0.071 ops/s [Average]
> (min, avg, max) = (0.506, 0.528, 0.550), stdev = 0.018
> CI (99.9%): [0.458, 0.599] (assumes normal distribution)
>
>
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 200000)
>
> # Run progress: 91.67% complete, ETA 00:00:41
> # Fork: 1 of 1
> # Warmup Iteration 1: 0.099 ops/s
> Iteration 1: 0.131 ops/s
> Iteration 2: 0.116 ops/s
> Iteration 3: 0.114 ops/s
> Iteration 4: 0.107 ops/s
> Iteration 5: 0.105 ops/s
>
>
> Result "measureMTTS":
> 0.115 �(99.9%) 0.039 ops/s [Average]
> (min, avg, max) = (0.105, 0.115, 0.131), stdev = 0.010
> CI (99.9%): [0.076, 0.154] (assumes normal distribution)
>
>
> # Run complete. Total time: 00:08:45
>
> Benchmark (numTasks) Mode Cnt Score Error Units
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 1000 thrpt 5 5540.905 � 244.059 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 10000 thrpt 5 63.778 � 5.053 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 100000 thrpt 5 0.546 � 0.159 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 200000 thrpt 5 0.137 � 0.049 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 1000 thrpt 5 3556.555 � 2569.449 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 10000 thrpt 5 54.058 � 18.486 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 100000 thrpt 5 0.460 � 0.238 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 200000 thrpt 5 0.127 � 0.025 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 1000 thrpt 5 3243.565 � 6417.109 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 10000 thrpt 5 55.796 � 13.886 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 100000 thrpt 5 0.528 � 0.071 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 200000 thrpt 5 0.115 � 0.039 ops/s
>
> Benchmark result is saved to dist/reports/jmh/results.txt
>
>
> ```
>
>
> Thanks,
>
> Reza Motamedi
>
>