You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2014/12/05 02:27:29 UTC
Review Request 28731: Implemented TaskScheduler benchmarks.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
Review request for Aurora, Kevin Sweeney and Bill Farner.
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
This is branched off of https://reviews.apache.org/r/28710
Diffs
-----
build.gradle fb729c5096108c535229e266fa9649f997e6da37
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeEventSink.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
# Run progress: 0.00% complete, ETA 00:03:40
# Fork: 1 of 1
# Warmup Iteration 1: 104725900.000 ns/op
# Warmup Iteration 2: 20439888.889 ns/op
# Warmup Iteration 3: 13612185.185 ns/op
# Warmup Iteration 4: 5856248.677 ns/op
# Warmup Iteration 5: 5148014.019 ns/op
# Warmup Iteration 6: 4972004.505 ns/op
# Warmup Iteration 7: 4595562.500 ns/op
# Warmup Iteration 8: 4639226.891 ns/op
# Warmup Iteration 9: 4639521.008 ns/op
# Warmup Iteration 10: 4660248.945 ns/op
Iteration 1: 4718478.632 ns/op
Iteration 2: 4800686.957 ns/op
Iteration 3: 4438253.012 ns/op
Iteration 4: 4935376.682 ns/op
Iteration 5: 5236895.735 ns/op
Iteration 6: 6439345.029 ns/op
Iteration 7: 4792708.696 ns/op
Iteration 8: 4579008.299 ns/op
Iteration 9: 4681224.576 ns/op
Iteration 10: 4651143.460 ns/op
Iteration 11: 4394079.681 ns/op
Iteration 12: 4390573.705 ns/op
Iteration 13: 4363304.348 ns/op
Iteration 14: 4419488.000 ns/op
Iteration 15: 4869718.062 ns/op
Iteration 16: 4600612.500 ns/op
Iteration 17: 4475906.883 ns/op
Iteration 18: 4710388.889 ns/op
Iteration 19: 6137000.000 ns/op
Iteration 20: 4467425.101 ns/op
Iteration 21: 4348877.953 ns/op
Iteration 22: 4666907.173 ns/op
Iteration 23: 4431325.301 ns/op
Iteration 24: 4463186.235 ns/op
Iteration 25: 4580074.689 ns/op
Iteration 26: 4467805.668 ns/op
Iteration 27: 4886960.177 ns/op
Iteration 28: 4858515.419 ns/op
Iteration 29: 4734025.751 ns/op
Iteration 30: 4718081.197 ns/op
Iteration 31: 4711688.034 ns/op
Iteration 32: 4462392.713 ns/op
Iteration 33: 4683597.458 ns/op
Iteration 34: 4755176.724 ns/op
Iteration 35: 4530278.689 ns/op
Iteration 36: 4392119.048 ns/op
Iteration 37: 4659722.689 ns/op
Iteration 38: 4805869.565 ns/op
Iteration 39: 4398573.705 ns/op
Iteration 40: 4572547.718 ns/op
Iteration 41: 4643478.992 ns/op
Iteration 42: 4545699.588 ns/op
Iteration 43: 4670266.949 ns/op
Iteration 44: 4506677.551 ns/op
Iteration 45: 4574149.378 ns/op
Iteration 46: 4743154.506 ns/op
Iteration 47: 4777606.061 ns/op
Iteration 48: 4481987.805 ns/op
Iteration 49: 4840921.053 ns/op
Iteration 50: 4637231.092 ns/op
Iteration 51: 4515286.885 ns/op
Iteration 52: 4372884.921 ns/op
Iteration 53: 4385003.968 ns/op
Iteration 54: 4619648.536 ns/op
Iteration 55: 4557822.314 ns/op
Iteration 56: 4594362.500 ns/op
Iteration 57: 7870319.149 ns/op
Iteration 58: 9764495.575 ns/op
Iteration 59: 10302028.037 ns/op
Iteration 60: 6043169.399 ns/op
Iteration 61: 5523472.362 ns/op
Iteration 62: 4513685.714 ns/op
Iteration 63: 4361806.324 ns/op
Iteration 64: 4388615.079 ns/op
Iteration 65: 4548264.463 ns/op
Iteration 66: 4710752.137 ns/op
Iteration 67: 5688128.866 ns/op
Iteration 68: 6109292.818 ns/op
Iteration 69: 5896914.439 ns/op
Iteration 70: 5986548.913 ns/op
Iteration 71: 6093618.785 ns/op
Iteration 72: 6051450.549 ns/op
Iteration 73: 6237870.056 ns/op
Iteration 74: 5954345.946 ns/op
Iteration 75: 6460748.538 ns/op
Iteration 76: 5929037.634 ns/op
Iteration 77: 6170748.603 ns/op
Iteration 78: 6037382.514 ns/op
Iteration 79: 6156776.536 ns/op
Iteration 80: 5987103.261 ns/op
Iteration 81: 5927080.645 ns/op
Iteration 82: 5914165.775 ns/op
Iteration 83: 6039852.459 ns/op
Iteration 84: 6156798.883 ns/op
Iteration 85: 5902329.787 ns/op
Iteration 86: 5731875.648 ns/op
Iteration 87: 6137622.222 ns/op
Iteration 88: 5926424.731 ns/op
Iteration 89: 5959324.324 ns/op
Iteration 90: 6269630.682 ns/op
Iteration 91: 5720062.176 ns/op
Iteration 92: 5938537.634 ns/op
Iteration 93: 4430361.446 ns/op
Iteration 94: 4498792.683 ns/op
Iteration 95: 5454004.950 ns/op
Iteration 96: 4516479.508 ns/op
Iteration 97: 4382456.349 ns/op
Iteration 98: 4410548.000 ns/op
Iteration 99: 4301657.588 ns/op
Iteration 100: 4438582.329 ns/op
Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
Confidence interval (99.9%): [4820809.970, 5509963.825]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
# Run progress: 50.00% complete, ETA 00:02:14
# Fork: 1 of 1
# Warmup Iteration 1: 34499258.065 ns/op
# Warmup Iteration 2: 9519784.483 ns/op
# Warmup Iteration 3: 7017031.847 ns/op
# Warmup Iteration 4: 4898586.667 ns/op
# Warmup Iteration 5: 4924111.607 ns/op
# Warmup Iteration 6: 5391946.341 ns/op
# Warmup Iteration 7: 5405700.980 ns/op
# Warmup Iteration 8: 5138181.395 ns/op
# Warmup Iteration 9: 4860863.436 ns/op
# Warmup Iteration 10: 5076539.171 ns/op
Iteration 1: 4693804.255 ns/op
Iteration 2: 5147288.372 ns/op
Iteration 3: 4809647.826 ns/op
Iteration 4: 6034147.541 ns/op
Iteration 5: 6675740.964 ns/op
Iteration 6: 5156481.308 ns/op
Iteration 7: 4678330.508 ns/op
Iteration 8: 4775528.139 ns/op
Iteration 9: 4800773.913 ns/op
Iteration 10: 4623054.393 ns/op
Iteration 11: 4816021.834 ns/op
Iteration 12: 4597212.500 ns/op
Iteration 13: 4691842.553 ns/op
Iteration 14: 4720820.513 ns/op
Iteration 15: 4662227.848 ns/op
Iteration 16: 5261447.619 ns/op
Iteration 17: 4840087.719 ns/op
Iteration 18: 4726081.197 ns/op
Iteration 19: 4673974.576 ns/op
Iteration 20: 5129837.209 ns/op
Iteration 21: 4724444.444 ns/op
Iteration 22: 4740678.112 ns/op
Iteration 23: 4710260.684 ns/op
Iteration 24: 4789688.312 ns/op
Iteration 25: 4589266.667 ns/op
Iteration 26: 4675394.068 ns/op
Iteration 27: 4594887.500 ns/op
Iteration 28: 4642579.832 ns/op
Iteration 29: 4666215.190 ns/op
Iteration 30: 4916044.444 ns/op
Iteration 31: 4734978.541 ns/op
Iteration 32: 4527586.066 ns/op
Iteration 33: 4695753.191 ns/op
Iteration 34: 4675682.203 ns/op
Iteration 35: 4598608.333 ns/op
Iteration 36: 4616338.912 ns/op
Iteration 37: 4722192.308 ns/op
Iteration 38: 5385243.902 ns/op
Iteration 39: 5183159.624 ns/op
Iteration 40: 4802855.895 ns/op
Iteration 41: 5393404.878 ns/op
Iteration 42: 7052592.357 ns/op
Iteration 43: 6087237.569 ns/op
Iteration 44: 6829685.185 ns/op
Iteration 45: 6105154.696 ns/op
Iteration 46: 5331869.565 ns/op
Iteration 47: 4765870.690 ns/op
Iteration 48: 4635117.155 ns/op
Iteration 49: 4882526.549 ns/op
Iteration 50: 4883402.655 ns/op
Iteration 51: 5018100.000 ns/op
Iteration 52: 4899177.778 ns/op
Iteration 53: 5568404.040 ns/op
Iteration 54: 4912208.889 ns/op
Iteration 55: 5073077.982 ns/op
Iteration 56: 5441458.128 ns/op
Iteration 57: 4964382.883 ns/op
Iteration 58: 5379580.488 ns/op
Iteration 59: 4925330.357 ns/op
Iteration 60: 5111805.556 ns/op
Iteration 61: 5226265.403 ns/op
Iteration 62: 5304846.154 ns/op
Iteration 63: 4920982.222 ns/op
Iteration 64: 4905186.667 ns/op
Iteration 65: 4970238.739 ns/op
Iteration 66: 5002425.339 ns/op
Iteration 67: 4842846.491 ns/op
Iteration 68: 5543889.447 ns/op
Iteration 69: 4828794.760 ns/op
Iteration 70: 4809737.991 ns/op
Iteration 71: 5151565.421 ns/op
Iteration 72: 5077170.507 ns/op
Iteration 73: 4987425.339 ns/op
Iteration 74: 4888584.071 ns/op
Iteration 75: 4980319.820 ns/op
Iteration 76: 5075321.101 ns/op
Iteration 77: 4719508.547 ns/op
Iteration 78: 4932200.893 ns/op
Iteration 79: 5046264.840 ns/op
Iteration 80: 4835530.702 ns/op
Iteration 81: 4855162.996 ns/op
Iteration 82: 4814104.803 ns/op
Iteration 83: 4650232.068 ns/op
Iteration 84: 5211075.472 ns/op
Iteration 85: 4970504.505 ns/op
Iteration 86: 4776272.727 ns/op
Iteration 87: 4929089.286 ns/op
Iteration 88: 5063995.413 ns/op
Iteration 89: 5059706.422 ns/op
Iteration 90: 5042342.466 ns/op
Iteration 91: 4760637.931 ns/op
Iteration 92: 5009213.636 ns/op
Iteration 93: 5985075.676 ns/op
Iteration 94: 4735798.283 ns/op
Iteration 95: 4893389.381 ns/op
Iteration 96: 6055005.495 ns/op
Iteration 97: 4684194.915 ns/op
Iteration 98: 4649130.252 ns/op
Iteration 99: 4619602.510 ns/op
Iteration 100: 4593595.833 ns/op
Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
Confidence interval (99.9%): [4851488.689, 5171987.930]
# Run complete. Total time: 00:04:29
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review64792
-----------------------------------------------------------
Master (07bc3ab) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 11, 2014, 8:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2014, 8:45 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:03:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 104725900.000 ns/op
> # Warmup Iteration 2: 20439888.889 ns/op
> # Warmup Iteration 3: 13612185.185 ns/op
> # Warmup Iteration 4: 5856248.677 ns/op
> # Warmup Iteration 5: 5148014.019 ns/op
> # Warmup Iteration 6: 4972004.505 ns/op
> # Warmup Iteration 7: 4595562.500 ns/op
> # Warmup Iteration 8: 4639226.891 ns/op
> # Warmup Iteration 9: 4639521.008 ns/op
> # Warmup Iteration 10: 4660248.945 ns/op
> Iteration 1: 4718478.632 ns/op
> Iteration 2: 4800686.957 ns/op
> Iteration 3: 4438253.012 ns/op
> Iteration 4: 4935376.682 ns/op
> Iteration 5: 5236895.735 ns/op
> Iteration 6: 6439345.029 ns/op
> Iteration 7: 4792708.696 ns/op
> Iteration 8: 4579008.299 ns/op
> Iteration 9: 4681224.576 ns/op
> Iteration 10: 4651143.460 ns/op
> Iteration 11: 4394079.681 ns/op
> Iteration 12: 4390573.705 ns/op
> Iteration 13: 4363304.348 ns/op
> Iteration 14: 4419488.000 ns/op
> Iteration 15: 4869718.062 ns/op
> Iteration 16: 4600612.500 ns/op
> Iteration 17: 4475906.883 ns/op
> Iteration 18: 4710388.889 ns/op
> Iteration 19: 6137000.000 ns/op
> Iteration 20: 4467425.101 ns/op
> Iteration 21: 4348877.953 ns/op
> Iteration 22: 4666907.173 ns/op
> Iteration 23: 4431325.301 ns/op
> Iteration 24: 4463186.235 ns/op
> Iteration 25: 4580074.689 ns/op
> Iteration 26: 4467805.668 ns/op
> Iteration 27: 4886960.177 ns/op
> Iteration 28: 4858515.419 ns/op
> Iteration 29: 4734025.751 ns/op
> Iteration 30: 4718081.197 ns/op
> Iteration 31: 4711688.034 ns/op
> Iteration 32: 4462392.713 ns/op
> Iteration 33: 4683597.458 ns/op
> Iteration 34: 4755176.724 ns/op
> Iteration 35: 4530278.689 ns/op
> Iteration 36: 4392119.048 ns/op
> Iteration 37: 4659722.689 ns/op
> Iteration 38: 4805869.565 ns/op
> Iteration 39: 4398573.705 ns/op
> Iteration 40: 4572547.718 ns/op
> Iteration 41: 4643478.992 ns/op
> Iteration 42: 4545699.588 ns/op
> Iteration 43: 4670266.949 ns/op
> Iteration 44: 4506677.551 ns/op
> Iteration 45: 4574149.378 ns/op
> Iteration 46: 4743154.506 ns/op
> Iteration 47: 4777606.061 ns/op
> Iteration 48: 4481987.805 ns/op
> Iteration 49: 4840921.053 ns/op
> Iteration 50: 4637231.092 ns/op
> Iteration 51: 4515286.885 ns/op
> Iteration 52: 4372884.921 ns/op
> Iteration 53: 4385003.968 ns/op
> Iteration 54: 4619648.536 ns/op
> Iteration 55: 4557822.314 ns/op
> Iteration 56: 4594362.500 ns/op
> Iteration 57: 7870319.149 ns/op
> Iteration 58: 9764495.575 ns/op
> Iteration 59: 10302028.037 ns/op
> Iteration 60: 6043169.399 ns/op
> Iteration 61: 5523472.362 ns/op
> Iteration 62: 4513685.714 ns/op
> Iteration 63: 4361806.324 ns/op
> Iteration 64: 4388615.079 ns/op
> Iteration 65: 4548264.463 ns/op
> Iteration 66: 4710752.137 ns/op
> Iteration 67: 5688128.866 ns/op
> Iteration 68: 6109292.818 ns/op
> Iteration 69: 5896914.439 ns/op
> Iteration 70: 5986548.913 ns/op
> Iteration 71: 6093618.785 ns/op
> Iteration 72: 6051450.549 ns/op
> Iteration 73: 6237870.056 ns/op
> Iteration 74: 5954345.946 ns/op
> Iteration 75: 6460748.538 ns/op
> Iteration 76: 5929037.634 ns/op
> Iteration 77: 6170748.603 ns/op
> Iteration 78: 6037382.514 ns/op
> Iteration 79: 6156776.536 ns/op
> Iteration 80: 5987103.261 ns/op
> Iteration 81: 5927080.645 ns/op
> Iteration 82: 5914165.775 ns/op
> Iteration 83: 6039852.459 ns/op
> Iteration 84: 6156798.883 ns/op
> Iteration 85: 5902329.787 ns/op
> Iteration 86: 5731875.648 ns/op
> Iteration 87: 6137622.222 ns/op
> Iteration 88: 5926424.731 ns/op
> Iteration 89: 5959324.324 ns/op
> Iteration 90: 6269630.682 ns/op
> Iteration 91: 5720062.176 ns/op
> Iteration 92: 5938537.634 ns/op
> Iteration 93: 4430361.446 ns/op
> Iteration 94: 4498792.683 ns/op
> Iteration 95: 5454004.950 ns/op
> Iteration 96: 4516479.508 ns/op
> Iteration 97: 4382456.349 ns/op
> Iteration 98: 4410548.000 ns/op
> Iteration 99: 4301657.588 ns/op
> Iteration 100: 4438582.329 ns/op
>
>
> Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
> Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
> Confidence interval (99.9%): [4820809.970, 5509963.825]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
>
> # Run progress: 50.00% complete, ETA 00:02:14
> # Fork: 1 of 1
> # Warmup Iteration 1: 34499258.065 ns/op
> # Warmup Iteration 2: 9519784.483 ns/op
> # Warmup Iteration 3: 7017031.847 ns/op
> # Warmup Iteration 4: 4898586.667 ns/op
> # Warmup Iteration 5: 4924111.607 ns/op
> # Warmup Iteration 6: 5391946.341 ns/op
> # Warmup Iteration 7: 5405700.980 ns/op
> # Warmup Iteration 8: 5138181.395 ns/op
> # Warmup Iteration 9: 4860863.436 ns/op
> # Warmup Iteration 10: 5076539.171 ns/op
> Iteration 1: 4693804.255 ns/op
> Iteration 2: 5147288.372 ns/op
> Iteration 3: 4809647.826 ns/op
> Iteration 4: 6034147.541 ns/op
> Iteration 5: 6675740.964 ns/op
> Iteration 6: 5156481.308 ns/op
> Iteration 7: 4678330.508 ns/op
> Iteration 8: 4775528.139 ns/op
> Iteration 9: 4800773.913 ns/op
> Iteration 10: 4623054.393 ns/op
> Iteration 11: 4816021.834 ns/op
> Iteration 12: 4597212.500 ns/op
> Iteration 13: 4691842.553 ns/op
> Iteration 14: 4720820.513 ns/op
> Iteration 15: 4662227.848 ns/op
> Iteration 16: 5261447.619 ns/op
> Iteration 17: 4840087.719 ns/op
> Iteration 18: 4726081.197 ns/op
> Iteration 19: 4673974.576 ns/op
> Iteration 20: 5129837.209 ns/op
> Iteration 21: 4724444.444 ns/op
> Iteration 22: 4740678.112 ns/op
> Iteration 23: 4710260.684 ns/op
> Iteration 24: 4789688.312 ns/op
> Iteration 25: 4589266.667 ns/op
> Iteration 26: 4675394.068 ns/op
> Iteration 27: 4594887.500 ns/op
> Iteration 28: 4642579.832 ns/op
> Iteration 29: 4666215.190 ns/op
> Iteration 30: 4916044.444 ns/op
> Iteration 31: 4734978.541 ns/op
> Iteration 32: 4527586.066 ns/op
> Iteration 33: 4695753.191 ns/op
> Iteration 34: 4675682.203 ns/op
> Iteration 35: 4598608.333 ns/op
> Iteration 36: 4616338.912 ns/op
> Iteration 37: 4722192.308 ns/op
> Iteration 38: 5385243.902 ns/op
> Iteration 39: 5183159.624 ns/op
> Iteration 40: 4802855.895 ns/op
> Iteration 41: 5393404.878 ns/op
> Iteration 42: 7052592.357 ns/op
> Iteration 43: 6087237.569 ns/op
> Iteration 44: 6829685.185 ns/op
> Iteration 45: 6105154.696 ns/op
> Iteration 46: 5331869.565 ns/op
> Iteration 47: 4765870.690 ns/op
> Iteration 48: 4635117.155 ns/op
> Iteration 49: 4882526.549 ns/op
> Iteration 50: 4883402.655 ns/op
> Iteration 51: 5018100.000 ns/op
> Iteration 52: 4899177.778 ns/op
> Iteration 53: 5568404.040 ns/op
> Iteration 54: 4912208.889 ns/op
> Iteration 55: 5073077.982 ns/op
> Iteration 56: 5441458.128 ns/op
> Iteration 57: 4964382.883 ns/op
> Iteration 58: 5379580.488 ns/op
> Iteration 59: 4925330.357 ns/op
> Iteration 60: 5111805.556 ns/op
> Iteration 61: 5226265.403 ns/op
> Iteration 62: 5304846.154 ns/op
> Iteration 63: 4920982.222 ns/op
> Iteration 64: 4905186.667 ns/op
> Iteration 65: 4970238.739 ns/op
> Iteration 66: 5002425.339 ns/op
> Iteration 67: 4842846.491 ns/op
> Iteration 68: 5543889.447 ns/op
> Iteration 69: 4828794.760 ns/op
> Iteration 70: 4809737.991 ns/op
> Iteration 71: 5151565.421 ns/op
> Iteration 72: 5077170.507 ns/op
> Iteration 73: 4987425.339 ns/op
> Iteration 74: 4888584.071 ns/op
> Iteration 75: 4980319.820 ns/op
> Iteration 76: 5075321.101 ns/op
> Iteration 77: 4719508.547 ns/op
> Iteration 78: 4932200.893 ns/op
> Iteration 79: 5046264.840 ns/op
> Iteration 80: 4835530.702 ns/op
> Iteration 81: 4855162.996 ns/op
> Iteration 82: 4814104.803 ns/op
> Iteration 83: 4650232.068 ns/op
> Iteration 84: 5211075.472 ns/op
> Iteration 85: 4970504.505 ns/op
> Iteration 86: 4776272.727 ns/op
> Iteration 87: 4929089.286 ns/op
> Iteration 88: 5063995.413 ns/op
> Iteration 89: 5059706.422 ns/op
> Iteration 90: 5042342.466 ns/op
> Iteration 91: 4760637.931 ns/op
> Iteration 92: 5009213.636 ns/op
> Iteration 93: 5985075.676 ns/op
> Iteration 94: 4735798.283 ns/op
> Iteration 95: 4893389.381 ns/op
> Iteration 96: 6055005.495 ns/op
> Iteration 97: 4684194.915 ns/op
> Iteration 98: 4649130.252 ns/op
> Iteration 99: 4619602.510 ns/op
> Iteration 100: 4593595.833 ns/op
>
>
> Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
> Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
> Confidence interval (99.9%): [4851488.689, 5171987.930]
>
>
> # Run complete. Total time: 00:04:29
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 11, 2014, 10:28 p.m., Bill Farner wrote:
> > Please document new classes (excluding fakes, since they're obvious).
Done.
> On Dec. 11, 2014, 10:28 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 96
> > <https://reviews.apache.org/r/28731/diff/3/?file=789688#file789688line96>
> >
> > Presumably this is an artificially high number. Since a 1 hr benchmark is not unrealistic, how about something obviously high like 1 month?
Done.
> On Dec. 11, 2014, 10:28 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 101
> > <https://reviews.apache.org/r/28731/diff/3/?file=789688#file789688line101>
> >
> > line break after the paren instead
Done.
> On Dec. 11, 2014, 10:28 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 106
> > <https://reviews.apache.org/r/28731/diff/3/?file=789688#file789688line106>
> >
> > You should be able to omit this if you bind to `SchedulingFilter.class`.
Done.
> On Dec. 11, 2014, 10:28 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 190
> > <https://reviews.apache.org/r/28731/diff/3/?file=789688#file789688line190>
> >
> > Can you add another where we have to fall back the preemptor, and there's a large number of tasks/slaves for the preemptor to work through?
That required some additional changes including opening up the PreemptorModule to override ClusterState. Done.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review64799
-----------------------------------------------------------
On Dec. 11, 2014, 8:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2014, 8:45 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:03:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 104725900.000 ns/op
> # Warmup Iteration 2: 20439888.889 ns/op
> # Warmup Iteration 3: 13612185.185 ns/op
> # Warmup Iteration 4: 5856248.677 ns/op
> # Warmup Iteration 5: 5148014.019 ns/op
> # Warmup Iteration 6: 4972004.505 ns/op
> # Warmup Iteration 7: 4595562.500 ns/op
> # Warmup Iteration 8: 4639226.891 ns/op
> # Warmup Iteration 9: 4639521.008 ns/op
> # Warmup Iteration 10: 4660248.945 ns/op
> Iteration 1: 4718478.632 ns/op
> Iteration 2: 4800686.957 ns/op
> Iteration 3: 4438253.012 ns/op
> Iteration 4: 4935376.682 ns/op
> Iteration 5: 5236895.735 ns/op
> Iteration 6: 6439345.029 ns/op
> Iteration 7: 4792708.696 ns/op
> Iteration 8: 4579008.299 ns/op
> Iteration 9: 4681224.576 ns/op
> Iteration 10: 4651143.460 ns/op
> Iteration 11: 4394079.681 ns/op
> Iteration 12: 4390573.705 ns/op
> Iteration 13: 4363304.348 ns/op
> Iteration 14: 4419488.000 ns/op
> Iteration 15: 4869718.062 ns/op
> Iteration 16: 4600612.500 ns/op
> Iteration 17: 4475906.883 ns/op
> Iteration 18: 4710388.889 ns/op
> Iteration 19: 6137000.000 ns/op
> Iteration 20: 4467425.101 ns/op
> Iteration 21: 4348877.953 ns/op
> Iteration 22: 4666907.173 ns/op
> Iteration 23: 4431325.301 ns/op
> Iteration 24: 4463186.235 ns/op
> Iteration 25: 4580074.689 ns/op
> Iteration 26: 4467805.668 ns/op
> Iteration 27: 4886960.177 ns/op
> Iteration 28: 4858515.419 ns/op
> Iteration 29: 4734025.751 ns/op
> Iteration 30: 4718081.197 ns/op
> Iteration 31: 4711688.034 ns/op
> Iteration 32: 4462392.713 ns/op
> Iteration 33: 4683597.458 ns/op
> Iteration 34: 4755176.724 ns/op
> Iteration 35: 4530278.689 ns/op
> Iteration 36: 4392119.048 ns/op
> Iteration 37: 4659722.689 ns/op
> Iteration 38: 4805869.565 ns/op
> Iteration 39: 4398573.705 ns/op
> Iteration 40: 4572547.718 ns/op
> Iteration 41: 4643478.992 ns/op
> Iteration 42: 4545699.588 ns/op
> Iteration 43: 4670266.949 ns/op
> Iteration 44: 4506677.551 ns/op
> Iteration 45: 4574149.378 ns/op
> Iteration 46: 4743154.506 ns/op
> Iteration 47: 4777606.061 ns/op
> Iteration 48: 4481987.805 ns/op
> Iteration 49: 4840921.053 ns/op
> Iteration 50: 4637231.092 ns/op
> Iteration 51: 4515286.885 ns/op
> Iteration 52: 4372884.921 ns/op
> Iteration 53: 4385003.968 ns/op
> Iteration 54: 4619648.536 ns/op
> Iteration 55: 4557822.314 ns/op
> Iteration 56: 4594362.500 ns/op
> Iteration 57: 7870319.149 ns/op
> Iteration 58: 9764495.575 ns/op
> Iteration 59: 10302028.037 ns/op
> Iteration 60: 6043169.399 ns/op
> Iteration 61: 5523472.362 ns/op
> Iteration 62: 4513685.714 ns/op
> Iteration 63: 4361806.324 ns/op
> Iteration 64: 4388615.079 ns/op
> Iteration 65: 4548264.463 ns/op
> Iteration 66: 4710752.137 ns/op
> Iteration 67: 5688128.866 ns/op
> Iteration 68: 6109292.818 ns/op
> Iteration 69: 5896914.439 ns/op
> Iteration 70: 5986548.913 ns/op
> Iteration 71: 6093618.785 ns/op
> Iteration 72: 6051450.549 ns/op
> Iteration 73: 6237870.056 ns/op
> Iteration 74: 5954345.946 ns/op
> Iteration 75: 6460748.538 ns/op
> Iteration 76: 5929037.634 ns/op
> Iteration 77: 6170748.603 ns/op
> Iteration 78: 6037382.514 ns/op
> Iteration 79: 6156776.536 ns/op
> Iteration 80: 5987103.261 ns/op
> Iteration 81: 5927080.645 ns/op
> Iteration 82: 5914165.775 ns/op
> Iteration 83: 6039852.459 ns/op
> Iteration 84: 6156798.883 ns/op
> Iteration 85: 5902329.787 ns/op
> Iteration 86: 5731875.648 ns/op
> Iteration 87: 6137622.222 ns/op
> Iteration 88: 5926424.731 ns/op
> Iteration 89: 5959324.324 ns/op
> Iteration 90: 6269630.682 ns/op
> Iteration 91: 5720062.176 ns/op
> Iteration 92: 5938537.634 ns/op
> Iteration 93: 4430361.446 ns/op
> Iteration 94: 4498792.683 ns/op
> Iteration 95: 5454004.950 ns/op
> Iteration 96: 4516479.508 ns/op
> Iteration 97: 4382456.349 ns/op
> Iteration 98: 4410548.000 ns/op
> Iteration 99: 4301657.588 ns/op
> Iteration 100: 4438582.329 ns/op
>
>
> Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
> Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
> Confidence interval (99.9%): [4820809.970, 5509963.825]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
>
> # Run progress: 50.00% complete, ETA 00:02:14
> # Fork: 1 of 1
> # Warmup Iteration 1: 34499258.065 ns/op
> # Warmup Iteration 2: 9519784.483 ns/op
> # Warmup Iteration 3: 7017031.847 ns/op
> # Warmup Iteration 4: 4898586.667 ns/op
> # Warmup Iteration 5: 4924111.607 ns/op
> # Warmup Iteration 6: 5391946.341 ns/op
> # Warmup Iteration 7: 5405700.980 ns/op
> # Warmup Iteration 8: 5138181.395 ns/op
> # Warmup Iteration 9: 4860863.436 ns/op
> # Warmup Iteration 10: 5076539.171 ns/op
> Iteration 1: 4693804.255 ns/op
> Iteration 2: 5147288.372 ns/op
> Iteration 3: 4809647.826 ns/op
> Iteration 4: 6034147.541 ns/op
> Iteration 5: 6675740.964 ns/op
> Iteration 6: 5156481.308 ns/op
> Iteration 7: 4678330.508 ns/op
> Iteration 8: 4775528.139 ns/op
> Iteration 9: 4800773.913 ns/op
> Iteration 10: 4623054.393 ns/op
> Iteration 11: 4816021.834 ns/op
> Iteration 12: 4597212.500 ns/op
> Iteration 13: 4691842.553 ns/op
> Iteration 14: 4720820.513 ns/op
> Iteration 15: 4662227.848 ns/op
> Iteration 16: 5261447.619 ns/op
> Iteration 17: 4840087.719 ns/op
> Iteration 18: 4726081.197 ns/op
> Iteration 19: 4673974.576 ns/op
> Iteration 20: 5129837.209 ns/op
> Iteration 21: 4724444.444 ns/op
> Iteration 22: 4740678.112 ns/op
> Iteration 23: 4710260.684 ns/op
> Iteration 24: 4789688.312 ns/op
> Iteration 25: 4589266.667 ns/op
> Iteration 26: 4675394.068 ns/op
> Iteration 27: 4594887.500 ns/op
> Iteration 28: 4642579.832 ns/op
> Iteration 29: 4666215.190 ns/op
> Iteration 30: 4916044.444 ns/op
> Iteration 31: 4734978.541 ns/op
> Iteration 32: 4527586.066 ns/op
> Iteration 33: 4695753.191 ns/op
> Iteration 34: 4675682.203 ns/op
> Iteration 35: 4598608.333 ns/op
> Iteration 36: 4616338.912 ns/op
> Iteration 37: 4722192.308 ns/op
> Iteration 38: 5385243.902 ns/op
> Iteration 39: 5183159.624 ns/op
> Iteration 40: 4802855.895 ns/op
> Iteration 41: 5393404.878 ns/op
> Iteration 42: 7052592.357 ns/op
> Iteration 43: 6087237.569 ns/op
> Iteration 44: 6829685.185 ns/op
> Iteration 45: 6105154.696 ns/op
> Iteration 46: 5331869.565 ns/op
> Iteration 47: 4765870.690 ns/op
> Iteration 48: 4635117.155 ns/op
> Iteration 49: 4882526.549 ns/op
> Iteration 50: 4883402.655 ns/op
> Iteration 51: 5018100.000 ns/op
> Iteration 52: 4899177.778 ns/op
> Iteration 53: 5568404.040 ns/op
> Iteration 54: 4912208.889 ns/op
> Iteration 55: 5073077.982 ns/op
> Iteration 56: 5441458.128 ns/op
> Iteration 57: 4964382.883 ns/op
> Iteration 58: 5379580.488 ns/op
> Iteration 59: 4925330.357 ns/op
> Iteration 60: 5111805.556 ns/op
> Iteration 61: 5226265.403 ns/op
> Iteration 62: 5304846.154 ns/op
> Iteration 63: 4920982.222 ns/op
> Iteration 64: 4905186.667 ns/op
> Iteration 65: 4970238.739 ns/op
> Iteration 66: 5002425.339 ns/op
> Iteration 67: 4842846.491 ns/op
> Iteration 68: 5543889.447 ns/op
> Iteration 69: 4828794.760 ns/op
> Iteration 70: 4809737.991 ns/op
> Iteration 71: 5151565.421 ns/op
> Iteration 72: 5077170.507 ns/op
> Iteration 73: 4987425.339 ns/op
> Iteration 74: 4888584.071 ns/op
> Iteration 75: 4980319.820 ns/op
> Iteration 76: 5075321.101 ns/op
> Iteration 77: 4719508.547 ns/op
> Iteration 78: 4932200.893 ns/op
> Iteration 79: 5046264.840 ns/op
> Iteration 80: 4835530.702 ns/op
> Iteration 81: 4855162.996 ns/op
> Iteration 82: 4814104.803 ns/op
> Iteration 83: 4650232.068 ns/op
> Iteration 84: 5211075.472 ns/op
> Iteration 85: 4970504.505 ns/op
> Iteration 86: 4776272.727 ns/op
> Iteration 87: 4929089.286 ns/op
> Iteration 88: 5063995.413 ns/op
> Iteration 89: 5059706.422 ns/op
> Iteration 90: 5042342.466 ns/op
> Iteration 91: 4760637.931 ns/op
> Iteration 92: 5009213.636 ns/op
> Iteration 93: 5985075.676 ns/op
> Iteration 94: 4735798.283 ns/op
> Iteration 95: 4893389.381 ns/op
> Iteration 96: 6055005.495 ns/op
> Iteration 97: 4684194.915 ns/op
> Iteration 98: 4649130.252 ns/op
> Iteration 99: 4619602.510 ns/op
> Iteration 100: 4593595.833 ns/op
>
>
> Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
> Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
> Confidence interval (99.9%): [4851488.689, 5171987.930]
>
>
> # Run complete. Total time: 00:04:29
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review64799
-----------------------------------------------------------
Please document new classes (excluding fakes, since they're obvious).
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment107563>
Presumably this is an artificially high number. Since a 1 hr benchmark is not unrealistic, how about something obviously high like 1 month?
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment107562>
line break after the paren instead
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment107564>
You should be able to omit this if you bind to `SchedulingFilter.class`.
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment107566>
Can you add another where we have to fall back the preemptor, and there's a large number of tasks/slaves for the preemptor to work through?
- Bill Farner
On Dec. 11, 2014, 8:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2014, 8:45 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:03:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 104725900.000 ns/op
> # Warmup Iteration 2: 20439888.889 ns/op
> # Warmup Iteration 3: 13612185.185 ns/op
> # Warmup Iteration 4: 5856248.677 ns/op
> # Warmup Iteration 5: 5148014.019 ns/op
> # Warmup Iteration 6: 4972004.505 ns/op
> # Warmup Iteration 7: 4595562.500 ns/op
> # Warmup Iteration 8: 4639226.891 ns/op
> # Warmup Iteration 9: 4639521.008 ns/op
> # Warmup Iteration 10: 4660248.945 ns/op
> Iteration 1: 4718478.632 ns/op
> Iteration 2: 4800686.957 ns/op
> Iteration 3: 4438253.012 ns/op
> Iteration 4: 4935376.682 ns/op
> Iteration 5: 5236895.735 ns/op
> Iteration 6: 6439345.029 ns/op
> Iteration 7: 4792708.696 ns/op
> Iteration 8: 4579008.299 ns/op
> Iteration 9: 4681224.576 ns/op
> Iteration 10: 4651143.460 ns/op
> Iteration 11: 4394079.681 ns/op
> Iteration 12: 4390573.705 ns/op
> Iteration 13: 4363304.348 ns/op
> Iteration 14: 4419488.000 ns/op
> Iteration 15: 4869718.062 ns/op
> Iteration 16: 4600612.500 ns/op
> Iteration 17: 4475906.883 ns/op
> Iteration 18: 4710388.889 ns/op
> Iteration 19: 6137000.000 ns/op
> Iteration 20: 4467425.101 ns/op
> Iteration 21: 4348877.953 ns/op
> Iteration 22: 4666907.173 ns/op
> Iteration 23: 4431325.301 ns/op
> Iteration 24: 4463186.235 ns/op
> Iteration 25: 4580074.689 ns/op
> Iteration 26: 4467805.668 ns/op
> Iteration 27: 4886960.177 ns/op
> Iteration 28: 4858515.419 ns/op
> Iteration 29: 4734025.751 ns/op
> Iteration 30: 4718081.197 ns/op
> Iteration 31: 4711688.034 ns/op
> Iteration 32: 4462392.713 ns/op
> Iteration 33: 4683597.458 ns/op
> Iteration 34: 4755176.724 ns/op
> Iteration 35: 4530278.689 ns/op
> Iteration 36: 4392119.048 ns/op
> Iteration 37: 4659722.689 ns/op
> Iteration 38: 4805869.565 ns/op
> Iteration 39: 4398573.705 ns/op
> Iteration 40: 4572547.718 ns/op
> Iteration 41: 4643478.992 ns/op
> Iteration 42: 4545699.588 ns/op
> Iteration 43: 4670266.949 ns/op
> Iteration 44: 4506677.551 ns/op
> Iteration 45: 4574149.378 ns/op
> Iteration 46: 4743154.506 ns/op
> Iteration 47: 4777606.061 ns/op
> Iteration 48: 4481987.805 ns/op
> Iteration 49: 4840921.053 ns/op
> Iteration 50: 4637231.092 ns/op
> Iteration 51: 4515286.885 ns/op
> Iteration 52: 4372884.921 ns/op
> Iteration 53: 4385003.968 ns/op
> Iteration 54: 4619648.536 ns/op
> Iteration 55: 4557822.314 ns/op
> Iteration 56: 4594362.500 ns/op
> Iteration 57: 7870319.149 ns/op
> Iteration 58: 9764495.575 ns/op
> Iteration 59: 10302028.037 ns/op
> Iteration 60: 6043169.399 ns/op
> Iteration 61: 5523472.362 ns/op
> Iteration 62: 4513685.714 ns/op
> Iteration 63: 4361806.324 ns/op
> Iteration 64: 4388615.079 ns/op
> Iteration 65: 4548264.463 ns/op
> Iteration 66: 4710752.137 ns/op
> Iteration 67: 5688128.866 ns/op
> Iteration 68: 6109292.818 ns/op
> Iteration 69: 5896914.439 ns/op
> Iteration 70: 5986548.913 ns/op
> Iteration 71: 6093618.785 ns/op
> Iteration 72: 6051450.549 ns/op
> Iteration 73: 6237870.056 ns/op
> Iteration 74: 5954345.946 ns/op
> Iteration 75: 6460748.538 ns/op
> Iteration 76: 5929037.634 ns/op
> Iteration 77: 6170748.603 ns/op
> Iteration 78: 6037382.514 ns/op
> Iteration 79: 6156776.536 ns/op
> Iteration 80: 5987103.261 ns/op
> Iteration 81: 5927080.645 ns/op
> Iteration 82: 5914165.775 ns/op
> Iteration 83: 6039852.459 ns/op
> Iteration 84: 6156798.883 ns/op
> Iteration 85: 5902329.787 ns/op
> Iteration 86: 5731875.648 ns/op
> Iteration 87: 6137622.222 ns/op
> Iteration 88: 5926424.731 ns/op
> Iteration 89: 5959324.324 ns/op
> Iteration 90: 6269630.682 ns/op
> Iteration 91: 5720062.176 ns/op
> Iteration 92: 5938537.634 ns/op
> Iteration 93: 4430361.446 ns/op
> Iteration 94: 4498792.683 ns/op
> Iteration 95: 5454004.950 ns/op
> Iteration 96: 4516479.508 ns/op
> Iteration 97: 4382456.349 ns/op
> Iteration 98: 4410548.000 ns/op
> Iteration 99: 4301657.588 ns/op
> Iteration 100: 4438582.329 ns/op
>
>
> Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
> Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
> Confidence interval (99.9%): [4820809.970, 5509963.825]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
>
> # Run progress: 50.00% complete, ETA 00:02:14
> # Fork: 1 of 1
> # Warmup Iteration 1: 34499258.065 ns/op
> # Warmup Iteration 2: 9519784.483 ns/op
> # Warmup Iteration 3: 7017031.847 ns/op
> # Warmup Iteration 4: 4898586.667 ns/op
> # Warmup Iteration 5: 4924111.607 ns/op
> # Warmup Iteration 6: 5391946.341 ns/op
> # Warmup Iteration 7: 5405700.980 ns/op
> # Warmup Iteration 8: 5138181.395 ns/op
> # Warmup Iteration 9: 4860863.436 ns/op
> # Warmup Iteration 10: 5076539.171 ns/op
> Iteration 1: 4693804.255 ns/op
> Iteration 2: 5147288.372 ns/op
> Iteration 3: 4809647.826 ns/op
> Iteration 4: 6034147.541 ns/op
> Iteration 5: 6675740.964 ns/op
> Iteration 6: 5156481.308 ns/op
> Iteration 7: 4678330.508 ns/op
> Iteration 8: 4775528.139 ns/op
> Iteration 9: 4800773.913 ns/op
> Iteration 10: 4623054.393 ns/op
> Iteration 11: 4816021.834 ns/op
> Iteration 12: 4597212.500 ns/op
> Iteration 13: 4691842.553 ns/op
> Iteration 14: 4720820.513 ns/op
> Iteration 15: 4662227.848 ns/op
> Iteration 16: 5261447.619 ns/op
> Iteration 17: 4840087.719 ns/op
> Iteration 18: 4726081.197 ns/op
> Iteration 19: 4673974.576 ns/op
> Iteration 20: 5129837.209 ns/op
> Iteration 21: 4724444.444 ns/op
> Iteration 22: 4740678.112 ns/op
> Iteration 23: 4710260.684 ns/op
> Iteration 24: 4789688.312 ns/op
> Iteration 25: 4589266.667 ns/op
> Iteration 26: 4675394.068 ns/op
> Iteration 27: 4594887.500 ns/op
> Iteration 28: 4642579.832 ns/op
> Iteration 29: 4666215.190 ns/op
> Iteration 30: 4916044.444 ns/op
> Iteration 31: 4734978.541 ns/op
> Iteration 32: 4527586.066 ns/op
> Iteration 33: 4695753.191 ns/op
> Iteration 34: 4675682.203 ns/op
> Iteration 35: 4598608.333 ns/op
> Iteration 36: 4616338.912 ns/op
> Iteration 37: 4722192.308 ns/op
> Iteration 38: 5385243.902 ns/op
> Iteration 39: 5183159.624 ns/op
> Iteration 40: 4802855.895 ns/op
> Iteration 41: 5393404.878 ns/op
> Iteration 42: 7052592.357 ns/op
> Iteration 43: 6087237.569 ns/op
> Iteration 44: 6829685.185 ns/op
> Iteration 45: 6105154.696 ns/op
> Iteration 46: 5331869.565 ns/op
> Iteration 47: 4765870.690 ns/op
> Iteration 48: 4635117.155 ns/op
> Iteration 49: 4882526.549 ns/op
> Iteration 50: 4883402.655 ns/op
> Iteration 51: 5018100.000 ns/op
> Iteration 52: 4899177.778 ns/op
> Iteration 53: 5568404.040 ns/op
> Iteration 54: 4912208.889 ns/op
> Iteration 55: 5073077.982 ns/op
> Iteration 56: 5441458.128 ns/op
> Iteration 57: 4964382.883 ns/op
> Iteration 58: 5379580.488 ns/op
> Iteration 59: 4925330.357 ns/op
> Iteration 60: 5111805.556 ns/op
> Iteration 61: 5226265.403 ns/op
> Iteration 62: 5304846.154 ns/op
> Iteration 63: 4920982.222 ns/op
> Iteration 64: 4905186.667 ns/op
> Iteration 65: 4970238.739 ns/op
> Iteration 66: 5002425.339 ns/op
> Iteration 67: 4842846.491 ns/op
> Iteration 68: 5543889.447 ns/op
> Iteration 69: 4828794.760 ns/op
> Iteration 70: 4809737.991 ns/op
> Iteration 71: 5151565.421 ns/op
> Iteration 72: 5077170.507 ns/op
> Iteration 73: 4987425.339 ns/op
> Iteration 74: 4888584.071 ns/op
> Iteration 75: 4980319.820 ns/op
> Iteration 76: 5075321.101 ns/op
> Iteration 77: 4719508.547 ns/op
> Iteration 78: 4932200.893 ns/op
> Iteration 79: 5046264.840 ns/op
> Iteration 80: 4835530.702 ns/op
> Iteration 81: 4855162.996 ns/op
> Iteration 82: 4814104.803 ns/op
> Iteration 83: 4650232.068 ns/op
> Iteration 84: 5211075.472 ns/op
> Iteration 85: 4970504.505 ns/op
> Iteration 86: 4776272.727 ns/op
> Iteration 87: 4929089.286 ns/op
> Iteration 88: 5063995.413 ns/op
> Iteration 89: 5059706.422 ns/op
> Iteration 90: 5042342.466 ns/op
> Iteration 91: 4760637.931 ns/op
> Iteration 92: 5009213.636 ns/op
> Iteration 93: 5985075.676 ns/op
> Iteration 94: 4735798.283 ns/op
> Iteration 95: 4893389.381 ns/op
> Iteration 96: 6055005.495 ns/op
> Iteration 97: 4684194.915 ns/op
> Iteration 98: 4649130.252 ns/op
> Iteration 99: 4619602.510 ns/op
> Iteration 100: 4593595.833 ns/op
>
>
> Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
> Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
> Confidence interval (99.9%): [4851488.689, 5171987.930]
>
>
> # Run complete. Total time: 00:04:29
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review65140
-----------------------------------------------------------
Master (52efc93) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 15, 2014, 10:15 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 15, 2014, 10:15 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 21, 2015, 11:44 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 101
> > <https://reviews.apache.org/r/28731/diff/5/?file=808351#file808351line101>
> >
> > I think we should proceed one of two ways here:
> >
> > 1. whittle this down significantly to focus on a minimal set of classes related to the critical path when scheduling
> >
> > 2.accept that we're setting up ~the whole app, and do something more like what `SchedulerIT` does with modules.
> >
> > The concern i have is this class will drift from production code and rot, or just be an ongoing maintenance burden.
> >
> > Do you think either of those paths are realistic?
The current approach is on the lines of #1. The setup is crafted with classes participating in the scheduling loop. Do you have observations of the contrary?
I think approach #2 would introduce too much variablity into the JVM perf pass due to multiple scheduler sub-systems attempting to do their housekeeping. I doubt it will be any better in terms of rot either as it will require even more substantial setup and will not fully eliminate fakes/mocks.
To address your code rot concern, I think we should add './gradlew jmh' into our jenkins script to make sure it's on the critical change path.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review69051
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review69051
-----------------------------------------------------------
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment113621>
I think we should proceed one of two ways here:
1. whittle this down significantly to focus on a minimal set of classes related to the critical path when scheduling
2.accept that we're setting up ~the whole app, and do something more like what `SchedulerIT` does with modules.
The concern i have is this class will drift from production code and rot, or just be an ongoing maintenance burden.
Do you think either of those paths are realistic?
- Bill Farner
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 21, 2015, 10:27 p.m., Bill Farner wrote:
> > Meta question - how do you envision us using this? When a perf issue is discovered, should we generally push for a test case to be added here to validate the perf fix?
I doubt we can build a reliable/automated way to catch regressions due to perf. There is too much randomness coming from JVM and machine load for the scattered test runs to be comparable. These bencmarks are most useful when run back to back against different builds. The way I see it is more of a fix validation testbed than a comprehensive perf regression framework.
To your question, new tests may be required if the fix validation requires cluster/data setup significantly different from the existing tests. Otherwise, running the existing tests before and after the fix should be sufficient.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review69047
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
> On Jan. 21, 2015, 10:27 p.m., Bill Farner wrote:
> > Meta question - how do you envision us using this? When a perf issue is discovered, should we generally push for a test case to be added here to validate the perf fix?
>
> Maxim Khutornenko wrote:
> I doubt we can build a reliable/automated way to catch regressions due to perf. There is too much randomness coming from JVM and machine load for the scattered test runs to be comparable. These bencmarks are most useful when run back to back against different builds. The way I see it is more of a fix validation testbed than a comprehensive perf regression framework.
>
> To your question, new tests may be required if the fix validation requires cluster/data setup significantly different from the existing tests. Otherwise, running the existing tests before and after the fix should be sufficient.
Agreed - absolute figures are not necessary useful such that we will institute thresholds for passage.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review69047
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review69047
-----------------------------------------------------------
Meta question - how do you envision us using this? When a perf issue is discovered, should we generally push for a test case to be added here to validate the perf fix?
- Bill Farner
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66972
-----------------------------------------------------------
Master (1089413) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Kevin Sweeney <ke...@apache.org>.
> On Jan. 21, 2015, 10:19 a.m., Bill Farner wrote:
> > Kevin - any chance you're available to take a pass at this soon? My attempts to free up a chunk of time for a thoughtful review have so far failed, and this is blocking other work of Maxim's.
I'm afraid my review plate is full for this week, is anyone else able to step in?
- Kevin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review68950
-----------------------------------------------------------
On Jan. 21, 2015, 5:54 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2015, 5:54 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review68950
-----------------------------------------------------------
Kevin - any chance you're available to take a pass at this soon? My attempts to free up a chunk of time for a thoughtful review have so far failed, and this is blocking other work of Maxim's.
- Bill Farner
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review69116
-----------------------------------------------------------
Master (116ee2d) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 22, 2015, 1:54 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2015, 1:54 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70587
-----------------------------------------------------------
@ReviewBot retry
- Maxim Khutornenko
On Feb. 2, 2015, 5:39 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 5:39 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-969
> https://issues.apache.org/jira/browse/AURORA-969
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 10c4f06e3fde91d5ed80da3381caa9a3a6a0af33
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70586
-----------------------------------------------------------
Master (9233abd) is red with this patch.
./build-support/jenkins/build.sh
:checkstyleJmh
:jsHint
:checkstyleMain
:compileTestJava
:processTestResources
:testClasses
:checkstyleTest
:findbugsJmh
:findbugsMain
:findbugsTest
:licenseJmh UP-TO-DATE
:licenseMain UP-TO-DATE
:licenseTest UP-TO-DATE
:license UP-TO-DATE
:pmdMain
:test
org.apache.aurora.scheduler.http.ServletFilterTest > testGzipEncoding FAILED
java.lang.AssertionError at ServletFilterTest.java:49
774 tests completed, 1 failed, 1 skipped
:test FAILED
:jacocoTestReport
Coverage report generated: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/jacoco/test/html/index.html
:analyzeReport
Instruction coverage of 0.891844551356906 exceeds minimum coverage of 0.89.
Branch coverage of 0.8365580448065173 exceeds minimum coverage of 0.835.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/tests/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 3 mins 55.251 secs
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 2, 2015, 5:39 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 5:39 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-969
> https://issues.apache.org/jira/browse/AURORA-969
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 10c4f06e3fde91d5ed80da3381caa9a3a6a0af33
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70591
-----------------------------------------------------------
Master (9233abd) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 2, 2015, 5:39 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 5:39 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-969
> https://issues.apache.org/jira/browse/AURORA-969
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 10c4f06e3fde91d5ed80da3381caa9a3a6a0af33
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Feb. 2, 2015, 5:39 p.m.)
Review request for Aurora, Bill Farner and Zameer Manji.
Changes
-------
Rebasing.
Bugs: AURORA-969
https://issues.apache.org/jira/browse/AURORA-969
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
Diffs (updated)
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 10c4f06e3fde91d5ed80da3381caa9a3a6a0af33
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
# Run progress: 0.00% complete, ETA 00:05:30
# Fork: 1 of 1
# Warmup Iteration 1: 278284250.000 ns/op
# Warmup Iteration 2: 70294312.500 ns/op
# Warmup Iteration 3: 19293379.310 ns/op
# Warmup Iteration 4: 11945387.097 ns/op
# Warmup Iteration 5: 10725388.350 ns/op
# Warmup Iteration 6: 13043282.353 ns/op
# Warmup Iteration 7: 9233083.333 ns/op
# Warmup Iteration 8: 9521051.724 ns/op
# Warmup Iteration 9: 10750854.369 ns/op
# Warmup Iteration 10: 7460243.243 ns/op
Iteration 1: 7885364.286 ns/op
Iteration 2: 7735139.860 ns/op
Iteration 3: 7660208.333 ns/op
Iteration 4: 7761204.225 ns/op
Iteration 5: 7868907.143 ns/op
Iteration 6: 7567404.110 ns/op
Iteration 7: 7611000.000 ns/op
Iteration 8: 7766154.930 ns/op
Iteration 9: 7669344.828 ns/op
Iteration 10: 7707783.217 ns/op
Iteration 11: 7435651.007 ns/op
Iteration 12: 7697631.944 ns/op
Iteration 13: 7712531.469 ns/op
Iteration 14: 7899407.143 ns/op
Iteration 15: 7448472.973 ns/op
Iteration 16: 7791521.127 ns/op
Iteration 17: 7612213.793 ns/op
Iteration 18: 7710867.133 ns/op
Iteration 19: 7649296.552 ns/op
Iteration 20: 7768309.859 ns/op
Iteration 21: 7688666.667 ns/op
Iteration 22: 7531557.823 ns/op
Iteration 23: 7381193.333 ns/op
Iteration 24: 7726006.993 ns/op
Iteration 25: 7603358.621 ns/op
Iteration 26: 7653631.944 ns/op
Iteration 27: 7442275.168 ns/op
Iteration 28: 7613186.207 ns/op
Iteration 29: 7765823.944 ns/op
Iteration 30: 7489687.075 ns/op
Iteration 31: 7811443.662 ns/op
Iteration 32: 8015007.246 ns/op
Iteration 33: 8192392.593 ns/op
Iteration 34: 8040335.766 ns/op
Iteration 35: 7584212.329 ns/op
Iteration 36: 8001934.783 ns/op
Iteration 37: 9744815.789 ns/op
Iteration 38: 11688284.211 ns/op
Iteration 39: 8661406.250 ns/op
Iteration 40: 7678413.793 ns/op
Iteration 41: 8502223.077 ns/op
Iteration 42: 7640820.690 ns/op
Iteration 43: 7875624.113 ns/op
Iteration 44: 7506809.524 ns/op
Iteration 45: 8005431.655 ns/op
Iteration 46: 8081664.234 ns/op
Iteration 47: 7579438.356 ns/op
Iteration 48: 7993405.797 ns/op
Iteration 49: 7571958.904 ns/op
Iteration 50: 8116463.235 ns/op
Iteration 51: 7941330.935 ns/op
Iteration 52: 7687145.833 ns/op
Iteration 53: 8082554.745 ns/op
Iteration 54: 7597889.655 ns/op
Iteration 55: 7299907.285 ns/op
Iteration 56: 7992789.855 ns/op
Iteration 57: 7648268.966 ns/op
Iteration 58: 7570863.014 ns/op
Iteration 59: 7885078.571 ns/op
Iteration 60: 7647158.621 ns/op
Iteration 61: 7830858.156 ns/op
Iteration 62: 7773690.141 ns/op
Iteration 63: 7905850.000 ns/op
Iteration 64: 7653800.000 ns/op
Iteration 65: 7408248.322 ns/op
Iteration 66: 7961352.518 ns/op
Iteration 67: 7879785.714 ns/op
Iteration 68: 8308616.541 ns/op
Iteration 69: 7396852.349 ns/op
Iteration 70: 8203214.815 ns/op
Iteration 71: 7548821.918 ns/op
Iteration 72: 7834631.206 ns/op
Iteration 73: 7451594.595 ns/op
Iteration 74: 7633262.069 ns/op
Iteration 75: 8182348.148 ns/op
Iteration 76: 7412646.667 ns/op
Iteration 77: 7662375.000 ns/op
Iteration 78: 7714825.175 ns/op
Iteration 79: 7726195.804 ns/op
Iteration 80: 7552568.493 ns/op
Iteration 81: 7642655.172 ns/op
Iteration 82: 7632737.931 ns/op
Iteration 83: 7567616.438 ns/op
Iteration 84: 7641517.241 ns/op
Iteration 85: 7664152.778 ns/op
Iteration 86: 8082532.847 ns/op
Iteration 87: 8027427.536 ns/op
Iteration 88: 7837191.489 ns/op
Iteration 89: 7587938.356 ns/op
Iteration 90: 7631262.069 ns/op
Iteration 91: 7672423.611 ns/op
Iteration 92: 7505258.503 ns/op
Iteration 93: 7410449.664 ns/op
Iteration 94: 8164419.118 ns/op
Iteration 95: 7712930.070 ns/op
Iteration 96: 7934287.770 ns/op
Iteration 97: 8007239.130 ns/op
Iteration 98: 8250343.284 ns/op
Iteration 99: 8482853.846 ns/op
Iteration 100: 8138595.588 ns/op
Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
Confidence interval (99.9%): [7659162.555, 8003522.350]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
# Run progress: 33.33% complete, ETA 00:04:43
# Fork: 1 of 1
# Warmup Iteration 1: 130891625.000 ns/op
# Warmup Iteration 2: 18382600.000 ns/op
# Warmup Iteration 3: 8235328.358 ns/op
# Warmup Iteration 4: 8724039.370 ns/op
# Warmup Iteration 5: 5823163.158 ns/op
# Warmup Iteration 6: 5425686.275 ns/op
# Warmup Iteration 7: 5722766.839 ns/op
# Warmup Iteration 8: 5716419.689 ns/op
# Warmup Iteration 9: 6278869.318 ns/op
# Warmup Iteration 10: 5738276.042 ns/op
Iteration 1: 5761427.083 ns/op
Iteration 2: 5946838.710 ns/op
Iteration 3: 6091066.298 ns/op
Iteration 4: 5721787.565 ns/op
Iteration 5: 6785852.761 ns/op
Iteration 6: 5813373.684 ns/op
Iteration 7: 5866824.468 ns/op
Iteration 8: 6948911.950 ns/op
Iteration 9: 6042060.109 ns/op
Iteration 10: 5633122.449 ns/op
Iteration 11: 5753838.542 ns/op
Iteration 12: 5853433.862 ns/op
Iteration 13: 5752322.917 ns/op
Iteration 14: 5332053.140 ns/op
Iteration 15: 5318278.846 ns/op
Iteration 16: 5260276.190 ns/op
Iteration 17: 5700340.206 ns/op
Iteration 18: 5767635.417 ns/op
Iteration 19: 5367514.563 ns/op
Iteration 20: 5721994.819 ns/op
Iteration 21: 5711243.523 ns/op
Iteration 22: 6276647.727 ns/op
Iteration 23: 5934473.118 ns/op
Iteration 24: 5821626.316 ns/op
Iteration 25: 5533788.945 ns/op
Iteration 26: 7226183.007 ns/op
Iteration 27: 5448920.792 ns/op
Iteration 28: 5749453.125 ns/op
Iteration 29: 5906727.273 ns/op
Iteration 30: 5702489.691 ns/op
Iteration 31: 5355660.194 ns/op
Iteration 32: 7413523.490 ns/op
Iteration 33: 7467831.081 ns/op
Iteration 34: 8952487.805 ns/op
Iteration 35: 8635742.188 ns/op
Iteration 36: 7981898.551 ns/op
Iteration 37: 7761732.394 ns/op
Iteration 38: 7130438.710 ns/op
Iteration 39: 8349406.015 ns/op
Iteration 40: 7980985.612 ns/op
Iteration 41: 8037847.826 ns/op
Iteration 42: 6315828.571 ns/op
Iteration 43: 5375892.683 ns/op
Iteration 44: 5897005.348 ns/op
Iteration 45: 5568793.970 ns/op
Iteration 46: 5273685.714 ns/op
Iteration 47: 8879392.000 ns/op
Iteration 48: 8641343.750 ns/op
Iteration 49: 10007136.364 ns/op
Iteration 50: 8484908.397 ns/op
Iteration 51: 7659333.333 ns/op
Iteration 52: 8661968.750 ns/op
Iteration 53: 8923153.226 ns/op
Iteration 54: 8746527.559 ns/op
Iteration 55: 8981138.211 ns/op
Iteration 56: 5631193.878 ns/op
Iteration 57: 5945069.892 ns/op
Iteration 58: 5610161.616 ns/op
Iteration 59: 5348665.049 ns/op
Iteration 60: 5429064.039 ns/op
Iteration 61: 5432418.719 ns/op
Iteration 62: 5621515.306 ns/op
Iteration 63: 5268630.332 ns/op
Iteration 64: 5849894.180 ns/op
Iteration 65: 5418126.829 ns/op
Iteration 66: 6133177.778 ns/op
Iteration 67: 6195095.506 ns/op
Iteration 68: 6111276.243 ns/op
Iteration 69: 5886244.681 ns/op
Iteration 70: 6430941.860 ns/op
Iteration 71: 5783329.843 ns/op
Iteration 72: 6507141.176 ns/op
Iteration 73: 10092783.784 ns/op
Iteration 74: 11873978.495 ns/op
Iteration 75: 8876424.000 ns/op
Iteration 76: 5878223.404 ns/op
Iteration 77: 7130651.613 ns/op
Iteration 78: 5508910.000 ns/op
Iteration 79: 5481338.308 ns/op
Iteration 80: 5999592.391 ns/op
Iteration 81: 6065917.582 ns/op
Iteration 82: 6087812.155 ns/op
Iteration 83: 5653512.821 ns/op
Iteration 84: 5528065.000 ns/op
Iteration 85: 5599314.721 ns/op
Iteration 86: 5300822.115 ns/op
Iteration 87: 6341833.333 ns/op
Iteration 88: 5511190.000 ns/op
Iteration 89: 5813026.316 ns/op
Iteration 90: 5914112.299 ns/op
Iteration 91: 5663928.205 ns/op
Iteration 92: 5723891.192 ns/op
Iteration 93: 5580843.434 ns/op
Iteration 94: 5990407.609 ns/op
Iteration 95: 5656717.949 ns/op
Iteration 96: 5432532.020 ns/op
Iteration 97: 5797827.225 ns/op
Iteration 98: 5647410.256 ns/op
Iteration 99: 5250814.286 ns/op
Iteration 100: 5602177.665 ns/op
Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
Confidence interval (99.9%): [5971349.307, 6865454.172]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
# Run progress: 66.67% complete, ETA 00:02:19
# Fork: 1 of 1
# Warmup Iteration 1: 7780760000.000 ns/op
# Warmup Iteration 2: 165927428.571 ns/op
# Warmup Iteration 3: 94242076.923 ns/op
# Warmup Iteration 4: 66061000.000 ns/op
# Warmup Iteration 5: 70444500.000 ns/op
# Warmup Iteration 6: 62350000.000 ns/op
# Warmup Iteration 7: 63746555.556 ns/op
# Warmup Iteration 8: 62712111.111 ns/op
# Warmup Iteration 9: 62776388.889 ns/op
# Warmup Iteration 10: 62455277.778 ns/op
Iteration 1: 65696764.706 ns/op
Iteration 2: 66255823.529 ns/op
Iteration 3: 68231647.059 ns/op
Iteration 4: 71606750.000 ns/op
Iteration 5: 66780235.294 ns/op
Iteration 6: 69914750.000 ns/op
Iteration 7: 74681333.333 ns/op
Iteration 8: 64133944.444 ns/op
Iteration 9: 60831736.842 ns/op
Iteration 10: 63261444.444 ns/op
Iteration 11: 60814105.263 ns/op
Iteration 12: 67882941.176 ns/op
Iteration 13: 62512888.889 ns/op
Iteration 14: 65551823.529 ns/op
Iteration 15: 65574294.118 ns/op
Iteration 16: 67928352.941 ns/op
Iteration 17: 65510588.235 ns/op
Iteration 18: 62637055.556 ns/op
Iteration 19: 62900222.222 ns/op
Iteration 20: 63715944.444 ns/op
Iteration 21: 62284611.111 ns/op
Iteration 22: 68174235.294 ns/op
Iteration 23: 69029187.500 ns/op
Iteration 24: 68584764.706 ns/op
Iteration 25: 63332777.778 ns/op
Iteration 26: 64316000.000 ns/op
Iteration 27: 64219444.444 ns/op
Iteration 28: 64219888.889 ns/op
Iteration 29: 63113888.889 ns/op
Iteration 30: 65802117.647 ns/op
Iteration 31: 63661277.778 ns/op
Iteration 32: 63322000.000 ns/op
Iteration 33: 63770000.000 ns/op
Iteration 34: 63920333.333 ns/op
Iteration 35: 62403500.000 ns/op
Iteration 36: 64137166.667 ns/op
Iteration 37: 64725444.444 ns/op
Iteration 38: 63145166.667 ns/op
Iteration 39: 64183944.444 ns/op
Iteration 40: 65873666.667 ns/op
Iteration 41: 65648529.412 ns/op
Iteration 42: 66487176.471 ns/op
Iteration 43: 61702888.889 ns/op
Iteration 44: 63267666.667 ns/op
Iteration 45: 69614125.000 ns/op
Iteration 46: 61574222.222 ns/op
Iteration 47: 62678611.111 ns/op
Iteration 48: 62792277.778 ns/op
Iteration 49: 65355352.941 ns/op
Iteration 50: 65206823.529 ns/op
Iteration 51: 64246722.222 ns/op
Iteration 52: 68074823.529 ns/op
Iteration 53: 67314823.529 ns/op
Iteration 54: 65511882.353 ns/op
Iteration 55: 63607722.222 ns/op
Iteration 56: 66646529.412 ns/op
Iteration 57: 67675764.706 ns/op
Iteration 58: 62502388.889 ns/op
Iteration 59: 70476625.000 ns/op
Iteration 60: 97138500.000 ns/op
Iteration 61: 93276461.538 ns/op
Iteration 62: 103508818.182 ns/op
Iteration 63: 96725916.667 ns/op
Iteration 64: 97539500.000 ns/op
Iteration 65: 93591461.538 ns/op
Iteration 66: 95847083.333 ns/op
Iteration 67: 101147666.667 ns/op
Iteration 68: 91144230.769 ns/op
Iteration 69: 69291375.000 ns/op
Iteration 70: 67021941.176 ns/op
Iteration 71: 64101166.667 ns/op
Iteration 72: 66573823.529 ns/op
Iteration 73: 66116235.294 ns/op
Iteration 74: 62875611.111 ns/op
Iteration 75: 72765812.500 ns/op
Iteration 76: 65663176.471 ns/op
Iteration 77: 64187000.000 ns/op
Iteration 78: 64425444.444 ns/op
Iteration 79: 63835111.111 ns/op
Iteration 80: 62942166.667 ns/op
Iteration 81: 69006500.000 ns/op
Iteration 82: 64798176.471 ns/op
Iteration 83: 67823117.647 ns/op
Iteration 84: 77090866.667 ns/op
Iteration 85: 71548625.000 ns/op
Iteration 86: 63718333.333 ns/op
Iteration 87: 67034764.706 ns/op
Iteration 88: 61701833.333 ns/op
Iteration 89: 65456529.412 ns/op
Iteration 90: 63462388.889 ns/op
Iteration 91: 61797555.556 ns/op
Iteration 92: 66772823.529 ns/op
Iteration 93: 66240117.647 ns/op
Iteration 94: 63701444.444 ns/op
Iteration 95: 64047888.889 ns/op
Iteration 96: 62162777.778 ns/op
Iteration 97: 64846588.235 ns/op
Iteration 98: 61865666.667 ns/op
Iteration 99: 61965222.222 ns/op
Iteration 100: 64089333.333 ns/op
Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
Confidence interval (99.9%): [64952939.106, 71404582.386]
# Run complete. Total time: 00:07:18
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Zameer Manji <zm...@apache.org>.
> On Jan. 30, 2015, 4:16 p.m., Zameer Manji wrote:
> > Ship It!
My comment got chompted, I'm expecting these tests to be run on Jenkins for every commit.
- Zameer
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70458
-----------------------------------------------------------
On Jan. 29, 2015, 12:11 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2015, 12:11 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 31, 2015, 12:16 a.m., Zameer Manji wrote:
> > Ship It!
>
> Zameer Manji wrote:
> My comment got chompted, I'm expecting these tests to be run on Jenkins for every commit.
If you check the RB history this has been discussed earlier. The outcome: perf testing is too machine state sensitive to block the build on its results. That said, we may run them in jenkins just to prevent bit rot.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70458
-----------------------------------------------------------
On Jan. 29, 2015, 8:11 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2015, 8:11 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70458
-----------------------------------------------------------
Ship it!
Ship It!
- Zameer Manji
On Jan. 29, 2015, 12:11 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2015, 12:11 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Jan. 29, 2015, 8:11 p.m.)
Review request for Aurora, Bill Farner and Zameer Manji.
Changes
-------
People -= kevints
People += zmanji
-wfarner
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
Diffs
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
# Run progress: 0.00% complete, ETA 00:05:30
# Fork: 1 of 1
# Warmup Iteration 1: 278284250.000 ns/op
# Warmup Iteration 2: 70294312.500 ns/op
# Warmup Iteration 3: 19293379.310 ns/op
# Warmup Iteration 4: 11945387.097 ns/op
# Warmup Iteration 5: 10725388.350 ns/op
# Warmup Iteration 6: 13043282.353 ns/op
# Warmup Iteration 7: 9233083.333 ns/op
# Warmup Iteration 8: 9521051.724 ns/op
# Warmup Iteration 9: 10750854.369 ns/op
# Warmup Iteration 10: 7460243.243 ns/op
Iteration 1: 7885364.286 ns/op
Iteration 2: 7735139.860 ns/op
Iteration 3: 7660208.333 ns/op
Iteration 4: 7761204.225 ns/op
Iteration 5: 7868907.143 ns/op
Iteration 6: 7567404.110 ns/op
Iteration 7: 7611000.000 ns/op
Iteration 8: 7766154.930 ns/op
Iteration 9: 7669344.828 ns/op
Iteration 10: 7707783.217 ns/op
Iteration 11: 7435651.007 ns/op
Iteration 12: 7697631.944 ns/op
Iteration 13: 7712531.469 ns/op
Iteration 14: 7899407.143 ns/op
Iteration 15: 7448472.973 ns/op
Iteration 16: 7791521.127 ns/op
Iteration 17: 7612213.793 ns/op
Iteration 18: 7710867.133 ns/op
Iteration 19: 7649296.552 ns/op
Iteration 20: 7768309.859 ns/op
Iteration 21: 7688666.667 ns/op
Iteration 22: 7531557.823 ns/op
Iteration 23: 7381193.333 ns/op
Iteration 24: 7726006.993 ns/op
Iteration 25: 7603358.621 ns/op
Iteration 26: 7653631.944 ns/op
Iteration 27: 7442275.168 ns/op
Iteration 28: 7613186.207 ns/op
Iteration 29: 7765823.944 ns/op
Iteration 30: 7489687.075 ns/op
Iteration 31: 7811443.662 ns/op
Iteration 32: 8015007.246 ns/op
Iteration 33: 8192392.593 ns/op
Iteration 34: 8040335.766 ns/op
Iteration 35: 7584212.329 ns/op
Iteration 36: 8001934.783 ns/op
Iteration 37: 9744815.789 ns/op
Iteration 38: 11688284.211 ns/op
Iteration 39: 8661406.250 ns/op
Iteration 40: 7678413.793 ns/op
Iteration 41: 8502223.077 ns/op
Iteration 42: 7640820.690 ns/op
Iteration 43: 7875624.113 ns/op
Iteration 44: 7506809.524 ns/op
Iteration 45: 8005431.655 ns/op
Iteration 46: 8081664.234 ns/op
Iteration 47: 7579438.356 ns/op
Iteration 48: 7993405.797 ns/op
Iteration 49: 7571958.904 ns/op
Iteration 50: 8116463.235 ns/op
Iteration 51: 7941330.935 ns/op
Iteration 52: 7687145.833 ns/op
Iteration 53: 8082554.745 ns/op
Iteration 54: 7597889.655 ns/op
Iteration 55: 7299907.285 ns/op
Iteration 56: 7992789.855 ns/op
Iteration 57: 7648268.966 ns/op
Iteration 58: 7570863.014 ns/op
Iteration 59: 7885078.571 ns/op
Iteration 60: 7647158.621 ns/op
Iteration 61: 7830858.156 ns/op
Iteration 62: 7773690.141 ns/op
Iteration 63: 7905850.000 ns/op
Iteration 64: 7653800.000 ns/op
Iteration 65: 7408248.322 ns/op
Iteration 66: 7961352.518 ns/op
Iteration 67: 7879785.714 ns/op
Iteration 68: 8308616.541 ns/op
Iteration 69: 7396852.349 ns/op
Iteration 70: 8203214.815 ns/op
Iteration 71: 7548821.918 ns/op
Iteration 72: 7834631.206 ns/op
Iteration 73: 7451594.595 ns/op
Iteration 74: 7633262.069 ns/op
Iteration 75: 8182348.148 ns/op
Iteration 76: 7412646.667 ns/op
Iteration 77: 7662375.000 ns/op
Iteration 78: 7714825.175 ns/op
Iteration 79: 7726195.804 ns/op
Iteration 80: 7552568.493 ns/op
Iteration 81: 7642655.172 ns/op
Iteration 82: 7632737.931 ns/op
Iteration 83: 7567616.438 ns/op
Iteration 84: 7641517.241 ns/op
Iteration 85: 7664152.778 ns/op
Iteration 86: 8082532.847 ns/op
Iteration 87: 8027427.536 ns/op
Iteration 88: 7837191.489 ns/op
Iteration 89: 7587938.356 ns/op
Iteration 90: 7631262.069 ns/op
Iteration 91: 7672423.611 ns/op
Iteration 92: 7505258.503 ns/op
Iteration 93: 7410449.664 ns/op
Iteration 94: 8164419.118 ns/op
Iteration 95: 7712930.070 ns/op
Iteration 96: 7934287.770 ns/op
Iteration 97: 8007239.130 ns/op
Iteration 98: 8250343.284 ns/op
Iteration 99: 8482853.846 ns/op
Iteration 100: 8138595.588 ns/op
Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
Confidence interval (99.9%): [7659162.555, 8003522.350]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
# Run progress: 33.33% complete, ETA 00:04:43
# Fork: 1 of 1
# Warmup Iteration 1: 130891625.000 ns/op
# Warmup Iteration 2: 18382600.000 ns/op
# Warmup Iteration 3: 8235328.358 ns/op
# Warmup Iteration 4: 8724039.370 ns/op
# Warmup Iteration 5: 5823163.158 ns/op
# Warmup Iteration 6: 5425686.275 ns/op
# Warmup Iteration 7: 5722766.839 ns/op
# Warmup Iteration 8: 5716419.689 ns/op
# Warmup Iteration 9: 6278869.318 ns/op
# Warmup Iteration 10: 5738276.042 ns/op
Iteration 1: 5761427.083 ns/op
Iteration 2: 5946838.710 ns/op
Iteration 3: 6091066.298 ns/op
Iteration 4: 5721787.565 ns/op
Iteration 5: 6785852.761 ns/op
Iteration 6: 5813373.684 ns/op
Iteration 7: 5866824.468 ns/op
Iteration 8: 6948911.950 ns/op
Iteration 9: 6042060.109 ns/op
Iteration 10: 5633122.449 ns/op
Iteration 11: 5753838.542 ns/op
Iteration 12: 5853433.862 ns/op
Iteration 13: 5752322.917 ns/op
Iteration 14: 5332053.140 ns/op
Iteration 15: 5318278.846 ns/op
Iteration 16: 5260276.190 ns/op
Iteration 17: 5700340.206 ns/op
Iteration 18: 5767635.417 ns/op
Iteration 19: 5367514.563 ns/op
Iteration 20: 5721994.819 ns/op
Iteration 21: 5711243.523 ns/op
Iteration 22: 6276647.727 ns/op
Iteration 23: 5934473.118 ns/op
Iteration 24: 5821626.316 ns/op
Iteration 25: 5533788.945 ns/op
Iteration 26: 7226183.007 ns/op
Iteration 27: 5448920.792 ns/op
Iteration 28: 5749453.125 ns/op
Iteration 29: 5906727.273 ns/op
Iteration 30: 5702489.691 ns/op
Iteration 31: 5355660.194 ns/op
Iteration 32: 7413523.490 ns/op
Iteration 33: 7467831.081 ns/op
Iteration 34: 8952487.805 ns/op
Iteration 35: 8635742.188 ns/op
Iteration 36: 7981898.551 ns/op
Iteration 37: 7761732.394 ns/op
Iteration 38: 7130438.710 ns/op
Iteration 39: 8349406.015 ns/op
Iteration 40: 7980985.612 ns/op
Iteration 41: 8037847.826 ns/op
Iteration 42: 6315828.571 ns/op
Iteration 43: 5375892.683 ns/op
Iteration 44: 5897005.348 ns/op
Iteration 45: 5568793.970 ns/op
Iteration 46: 5273685.714 ns/op
Iteration 47: 8879392.000 ns/op
Iteration 48: 8641343.750 ns/op
Iteration 49: 10007136.364 ns/op
Iteration 50: 8484908.397 ns/op
Iteration 51: 7659333.333 ns/op
Iteration 52: 8661968.750 ns/op
Iteration 53: 8923153.226 ns/op
Iteration 54: 8746527.559 ns/op
Iteration 55: 8981138.211 ns/op
Iteration 56: 5631193.878 ns/op
Iteration 57: 5945069.892 ns/op
Iteration 58: 5610161.616 ns/op
Iteration 59: 5348665.049 ns/op
Iteration 60: 5429064.039 ns/op
Iteration 61: 5432418.719 ns/op
Iteration 62: 5621515.306 ns/op
Iteration 63: 5268630.332 ns/op
Iteration 64: 5849894.180 ns/op
Iteration 65: 5418126.829 ns/op
Iteration 66: 6133177.778 ns/op
Iteration 67: 6195095.506 ns/op
Iteration 68: 6111276.243 ns/op
Iteration 69: 5886244.681 ns/op
Iteration 70: 6430941.860 ns/op
Iteration 71: 5783329.843 ns/op
Iteration 72: 6507141.176 ns/op
Iteration 73: 10092783.784 ns/op
Iteration 74: 11873978.495 ns/op
Iteration 75: 8876424.000 ns/op
Iteration 76: 5878223.404 ns/op
Iteration 77: 7130651.613 ns/op
Iteration 78: 5508910.000 ns/op
Iteration 79: 5481338.308 ns/op
Iteration 80: 5999592.391 ns/op
Iteration 81: 6065917.582 ns/op
Iteration 82: 6087812.155 ns/op
Iteration 83: 5653512.821 ns/op
Iteration 84: 5528065.000 ns/op
Iteration 85: 5599314.721 ns/op
Iteration 86: 5300822.115 ns/op
Iteration 87: 6341833.333 ns/op
Iteration 88: 5511190.000 ns/op
Iteration 89: 5813026.316 ns/op
Iteration 90: 5914112.299 ns/op
Iteration 91: 5663928.205 ns/op
Iteration 92: 5723891.192 ns/op
Iteration 93: 5580843.434 ns/op
Iteration 94: 5990407.609 ns/op
Iteration 95: 5656717.949 ns/op
Iteration 96: 5432532.020 ns/op
Iteration 97: 5797827.225 ns/op
Iteration 98: 5647410.256 ns/op
Iteration 99: 5250814.286 ns/op
Iteration 100: 5602177.665 ns/op
Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
Confidence interval (99.9%): [5971349.307, 6865454.172]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
# Run progress: 66.67% complete, ETA 00:02:19
# Fork: 1 of 1
# Warmup Iteration 1: 7780760000.000 ns/op
# Warmup Iteration 2: 165927428.571 ns/op
# Warmup Iteration 3: 94242076.923 ns/op
# Warmup Iteration 4: 66061000.000 ns/op
# Warmup Iteration 5: 70444500.000 ns/op
# Warmup Iteration 6: 62350000.000 ns/op
# Warmup Iteration 7: 63746555.556 ns/op
# Warmup Iteration 8: 62712111.111 ns/op
# Warmup Iteration 9: 62776388.889 ns/op
# Warmup Iteration 10: 62455277.778 ns/op
Iteration 1: 65696764.706 ns/op
Iteration 2: 66255823.529 ns/op
Iteration 3: 68231647.059 ns/op
Iteration 4: 71606750.000 ns/op
Iteration 5: 66780235.294 ns/op
Iteration 6: 69914750.000 ns/op
Iteration 7: 74681333.333 ns/op
Iteration 8: 64133944.444 ns/op
Iteration 9: 60831736.842 ns/op
Iteration 10: 63261444.444 ns/op
Iteration 11: 60814105.263 ns/op
Iteration 12: 67882941.176 ns/op
Iteration 13: 62512888.889 ns/op
Iteration 14: 65551823.529 ns/op
Iteration 15: 65574294.118 ns/op
Iteration 16: 67928352.941 ns/op
Iteration 17: 65510588.235 ns/op
Iteration 18: 62637055.556 ns/op
Iteration 19: 62900222.222 ns/op
Iteration 20: 63715944.444 ns/op
Iteration 21: 62284611.111 ns/op
Iteration 22: 68174235.294 ns/op
Iteration 23: 69029187.500 ns/op
Iteration 24: 68584764.706 ns/op
Iteration 25: 63332777.778 ns/op
Iteration 26: 64316000.000 ns/op
Iteration 27: 64219444.444 ns/op
Iteration 28: 64219888.889 ns/op
Iteration 29: 63113888.889 ns/op
Iteration 30: 65802117.647 ns/op
Iteration 31: 63661277.778 ns/op
Iteration 32: 63322000.000 ns/op
Iteration 33: 63770000.000 ns/op
Iteration 34: 63920333.333 ns/op
Iteration 35: 62403500.000 ns/op
Iteration 36: 64137166.667 ns/op
Iteration 37: 64725444.444 ns/op
Iteration 38: 63145166.667 ns/op
Iteration 39: 64183944.444 ns/op
Iteration 40: 65873666.667 ns/op
Iteration 41: 65648529.412 ns/op
Iteration 42: 66487176.471 ns/op
Iteration 43: 61702888.889 ns/op
Iteration 44: 63267666.667 ns/op
Iteration 45: 69614125.000 ns/op
Iteration 46: 61574222.222 ns/op
Iteration 47: 62678611.111 ns/op
Iteration 48: 62792277.778 ns/op
Iteration 49: 65355352.941 ns/op
Iteration 50: 65206823.529 ns/op
Iteration 51: 64246722.222 ns/op
Iteration 52: 68074823.529 ns/op
Iteration 53: 67314823.529 ns/op
Iteration 54: 65511882.353 ns/op
Iteration 55: 63607722.222 ns/op
Iteration 56: 66646529.412 ns/op
Iteration 57: 67675764.706 ns/op
Iteration 58: 62502388.889 ns/op
Iteration 59: 70476625.000 ns/op
Iteration 60: 97138500.000 ns/op
Iteration 61: 93276461.538 ns/op
Iteration 62: 103508818.182 ns/op
Iteration 63: 96725916.667 ns/op
Iteration 64: 97539500.000 ns/op
Iteration 65: 93591461.538 ns/op
Iteration 66: 95847083.333 ns/op
Iteration 67: 101147666.667 ns/op
Iteration 68: 91144230.769 ns/op
Iteration 69: 69291375.000 ns/op
Iteration 70: 67021941.176 ns/op
Iteration 71: 64101166.667 ns/op
Iteration 72: 66573823.529 ns/op
Iteration 73: 66116235.294 ns/op
Iteration 74: 62875611.111 ns/op
Iteration 75: 72765812.500 ns/op
Iteration 76: 65663176.471 ns/op
Iteration 77: 64187000.000 ns/op
Iteration 78: 64425444.444 ns/op
Iteration 79: 63835111.111 ns/op
Iteration 80: 62942166.667 ns/op
Iteration 81: 69006500.000 ns/op
Iteration 82: 64798176.471 ns/op
Iteration 83: 67823117.647 ns/op
Iteration 84: 77090866.667 ns/op
Iteration 85: 71548625.000 ns/op
Iteration 86: 63718333.333 ns/op
Iteration 87: 67034764.706 ns/op
Iteration 88: 61701833.333 ns/op
Iteration 89: 65456529.412 ns/op
Iteration 90: 63462388.889 ns/op
Iteration 91: 61797555.556 ns/op
Iteration 92: 66772823.529 ns/op
Iteration 93: 66240117.647 ns/op
Iteration 94: 63701444.444 ns/op
Iteration 95: 64047888.889 ns/op
Iteration 96: 62162777.778 ns/op
Iteration 97: 64846588.235 ns/op
Iteration 98: 61865666.667 ns/op
Iteration 99: 61965222.222 ns/op
Iteration 100: 64089333.333 ns/op
Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
Confidence interval (99.9%): [64952939.106, 71404582.386]
# Run complete. Total time: 00:07:18
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review70095
-----------------------------------------------------------
Ship it!
LGTM. I suspect we will learn more about how this scaffolding should look as we start submitting perf improvement patches with test cases here.
- Bill Farner
On Jan. 22, 2015, 1:54 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2015, 1:54 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Jan. 22, 2015, 1:54 a.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Replaced FakeClusterState with CachedClusterState.
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
Diffs (updated)
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java 03c2a8fde4a3fe5ac73f44da2cbe227995501c46
src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
# Run progress: 0.00% complete, ETA 00:05:30
# Fork: 1 of 1
# Warmup Iteration 1: 278284250.000 ns/op
# Warmup Iteration 2: 70294312.500 ns/op
# Warmup Iteration 3: 19293379.310 ns/op
# Warmup Iteration 4: 11945387.097 ns/op
# Warmup Iteration 5: 10725388.350 ns/op
# Warmup Iteration 6: 13043282.353 ns/op
# Warmup Iteration 7: 9233083.333 ns/op
# Warmup Iteration 8: 9521051.724 ns/op
# Warmup Iteration 9: 10750854.369 ns/op
# Warmup Iteration 10: 7460243.243 ns/op
Iteration 1: 7885364.286 ns/op
Iteration 2: 7735139.860 ns/op
Iteration 3: 7660208.333 ns/op
Iteration 4: 7761204.225 ns/op
Iteration 5: 7868907.143 ns/op
Iteration 6: 7567404.110 ns/op
Iteration 7: 7611000.000 ns/op
Iteration 8: 7766154.930 ns/op
Iteration 9: 7669344.828 ns/op
Iteration 10: 7707783.217 ns/op
Iteration 11: 7435651.007 ns/op
Iteration 12: 7697631.944 ns/op
Iteration 13: 7712531.469 ns/op
Iteration 14: 7899407.143 ns/op
Iteration 15: 7448472.973 ns/op
Iteration 16: 7791521.127 ns/op
Iteration 17: 7612213.793 ns/op
Iteration 18: 7710867.133 ns/op
Iteration 19: 7649296.552 ns/op
Iteration 20: 7768309.859 ns/op
Iteration 21: 7688666.667 ns/op
Iteration 22: 7531557.823 ns/op
Iteration 23: 7381193.333 ns/op
Iteration 24: 7726006.993 ns/op
Iteration 25: 7603358.621 ns/op
Iteration 26: 7653631.944 ns/op
Iteration 27: 7442275.168 ns/op
Iteration 28: 7613186.207 ns/op
Iteration 29: 7765823.944 ns/op
Iteration 30: 7489687.075 ns/op
Iteration 31: 7811443.662 ns/op
Iteration 32: 8015007.246 ns/op
Iteration 33: 8192392.593 ns/op
Iteration 34: 8040335.766 ns/op
Iteration 35: 7584212.329 ns/op
Iteration 36: 8001934.783 ns/op
Iteration 37: 9744815.789 ns/op
Iteration 38: 11688284.211 ns/op
Iteration 39: 8661406.250 ns/op
Iteration 40: 7678413.793 ns/op
Iteration 41: 8502223.077 ns/op
Iteration 42: 7640820.690 ns/op
Iteration 43: 7875624.113 ns/op
Iteration 44: 7506809.524 ns/op
Iteration 45: 8005431.655 ns/op
Iteration 46: 8081664.234 ns/op
Iteration 47: 7579438.356 ns/op
Iteration 48: 7993405.797 ns/op
Iteration 49: 7571958.904 ns/op
Iteration 50: 8116463.235 ns/op
Iteration 51: 7941330.935 ns/op
Iteration 52: 7687145.833 ns/op
Iteration 53: 8082554.745 ns/op
Iteration 54: 7597889.655 ns/op
Iteration 55: 7299907.285 ns/op
Iteration 56: 7992789.855 ns/op
Iteration 57: 7648268.966 ns/op
Iteration 58: 7570863.014 ns/op
Iteration 59: 7885078.571 ns/op
Iteration 60: 7647158.621 ns/op
Iteration 61: 7830858.156 ns/op
Iteration 62: 7773690.141 ns/op
Iteration 63: 7905850.000 ns/op
Iteration 64: 7653800.000 ns/op
Iteration 65: 7408248.322 ns/op
Iteration 66: 7961352.518 ns/op
Iteration 67: 7879785.714 ns/op
Iteration 68: 8308616.541 ns/op
Iteration 69: 7396852.349 ns/op
Iteration 70: 8203214.815 ns/op
Iteration 71: 7548821.918 ns/op
Iteration 72: 7834631.206 ns/op
Iteration 73: 7451594.595 ns/op
Iteration 74: 7633262.069 ns/op
Iteration 75: 8182348.148 ns/op
Iteration 76: 7412646.667 ns/op
Iteration 77: 7662375.000 ns/op
Iteration 78: 7714825.175 ns/op
Iteration 79: 7726195.804 ns/op
Iteration 80: 7552568.493 ns/op
Iteration 81: 7642655.172 ns/op
Iteration 82: 7632737.931 ns/op
Iteration 83: 7567616.438 ns/op
Iteration 84: 7641517.241 ns/op
Iteration 85: 7664152.778 ns/op
Iteration 86: 8082532.847 ns/op
Iteration 87: 8027427.536 ns/op
Iteration 88: 7837191.489 ns/op
Iteration 89: 7587938.356 ns/op
Iteration 90: 7631262.069 ns/op
Iteration 91: 7672423.611 ns/op
Iteration 92: 7505258.503 ns/op
Iteration 93: 7410449.664 ns/op
Iteration 94: 8164419.118 ns/op
Iteration 95: 7712930.070 ns/op
Iteration 96: 7934287.770 ns/op
Iteration 97: 8007239.130 ns/op
Iteration 98: 8250343.284 ns/op
Iteration 99: 8482853.846 ns/op
Iteration 100: 8138595.588 ns/op
Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
Confidence interval (99.9%): [7659162.555, 8003522.350]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
# Run progress: 33.33% complete, ETA 00:04:43
# Fork: 1 of 1
# Warmup Iteration 1: 130891625.000 ns/op
# Warmup Iteration 2: 18382600.000 ns/op
# Warmup Iteration 3: 8235328.358 ns/op
# Warmup Iteration 4: 8724039.370 ns/op
# Warmup Iteration 5: 5823163.158 ns/op
# Warmup Iteration 6: 5425686.275 ns/op
# Warmup Iteration 7: 5722766.839 ns/op
# Warmup Iteration 8: 5716419.689 ns/op
# Warmup Iteration 9: 6278869.318 ns/op
# Warmup Iteration 10: 5738276.042 ns/op
Iteration 1: 5761427.083 ns/op
Iteration 2: 5946838.710 ns/op
Iteration 3: 6091066.298 ns/op
Iteration 4: 5721787.565 ns/op
Iteration 5: 6785852.761 ns/op
Iteration 6: 5813373.684 ns/op
Iteration 7: 5866824.468 ns/op
Iteration 8: 6948911.950 ns/op
Iteration 9: 6042060.109 ns/op
Iteration 10: 5633122.449 ns/op
Iteration 11: 5753838.542 ns/op
Iteration 12: 5853433.862 ns/op
Iteration 13: 5752322.917 ns/op
Iteration 14: 5332053.140 ns/op
Iteration 15: 5318278.846 ns/op
Iteration 16: 5260276.190 ns/op
Iteration 17: 5700340.206 ns/op
Iteration 18: 5767635.417 ns/op
Iteration 19: 5367514.563 ns/op
Iteration 20: 5721994.819 ns/op
Iteration 21: 5711243.523 ns/op
Iteration 22: 6276647.727 ns/op
Iteration 23: 5934473.118 ns/op
Iteration 24: 5821626.316 ns/op
Iteration 25: 5533788.945 ns/op
Iteration 26: 7226183.007 ns/op
Iteration 27: 5448920.792 ns/op
Iteration 28: 5749453.125 ns/op
Iteration 29: 5906727.273 ns/op
Iteration 30: 5702489.691 ns/op
Iteration 31: 5355660.194 ns/op
Iteration 32: 7413523.490 ns/op
Iteration 33: 7467831.081 ns/op
Iteration 34: 8952487.805 ns/op
Iteration 35: 8635742.188 ns/op
Iteration 36: 7981898.551 ns/op
Iteration 37: 7761732.394 ns/op
Iteration 38: 7130438.710 ns/op
Iteration 39: 8349406.015 ns/op
Iteration 40: 7980985.612 ns/op
Iteration 41: 8037847.826 ns/op
Iteration 42: 6315828.571 ns/op
Iteration 43: 5375892.683 ns/op
Iteration 44: 5897005.348 ns/op
Iteration 45: 5568793.970 ns/op
Iteration 46: 5273685.714 ns/op
Iteration 47: 8879392.000 ns/op
Iteration 48: 8641343.750 ns/op
Iteration 49: 10007136.364 ns/op
Iteration 50: 8484908.397 ns/op
Iteration 51: 7659333.333 ns/op
Iteration 52: 8661968.750 ns/op
Iteration 53: 8923153.226 ns/op
Iteration 54: 8746527.559 ns/op
Iteration 55: 8981138.211 ns/op
Iteration 56: 5631193.878 ns/op
Iteration 57: 5945069.892 ns/op
Iteration 58: 5610161.616 ns/op
Iteration 59: 5348665.049 ns/op
Iteration 60: 5429064.039 ns/op
Iteration 61: 5432418.719 ns/op
Iteration 62: 5621515.306 ns/op
Iteration 63: 5268630.332 ns/op
Iteration 64: 5849894.180 ns/op
Iteration 65: 5418126.829 ns/op
Iteration 66: 6133177.778 ns/op
Iteration 67: 6195095.506 ns/op
Iteration 68: 6111276.243 ns/op
Iteration 69: 5886244.681 ns/op
Iteration 70: 6430941.860 ns/op
Iteration 71: 5783329.843 ns/op
Iteration 72: 6507141.176 ns/op
Iteration 73: 10092783.784 ns/op
Iteration 74: 11873978.495 ns/op
Iteration 75: 8876424.000 ns/op
Iteration 76: 5878223.404 ns/op
Iteration 77: 7130651.613 ns/op
Iteration 78: 5508910.000 ns/op
Iteration 79: 5481338.308 ns/op
Iteration 80: 5999592.391 ns/op
Iteration 81: 6065917.582 ns/op
Iteration 82: 6087812.155 ns/op
Iteration 83: 5653512.821 ns/op
Iteration 84: 5528065.000 ns/op
Iteration 85: 5599314.721 ns/op
Iteration 86: 5300822.115 ns/op
Iteration 87: 6341833.333 ns/op
Iteration 88: 5511190.000 ns/op
Iteration 89: 5813026.316 ns/op
Iteration 90: 5914112.299 ns/op
Iteration 91: 5663928.205 ns/op
Iteration 92: 5723891.192 ns/op
Iteration 93: 5580843.434 ns/op
Iteration 94: 5990407.609 ns/op
Iteration 95: 5656717.949 ns/op
Iteration 96: 5432532.020 ns/op
Iteration 97: 5797827.225 ns/op
Iteration 98: 5647410.256 ns/op
Iteration 99: 5250814.286 ns/op
Iteration 100: 5602177.665 ns/op
Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
Confidence interval (99.9%): [5971349.307, 6865454.172]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
# Run progress: 66.67% complete, ETA 00:02:19
# Fork: 1 of 1
# Warmup Iteration 1: 7780760000.000 ns/op
# Warmup Iteration 2: 165927428.571 ns/op
# Warmup Iteration 3: 94242076.923 ns/op
# Warmup Iteration 4: 66061000.000 ns/op
# Warmup Iteration 5: 70444500.000 ns/op
# Warmup Iteration 6: 62350000.000 ns/op
# Warmup Iteration 7: 63746555.556 ns/op
# Warmup Iteration 8: 62712111.111 ns/op
# Warmup Iteration 9: 62776388.889 ns/op
# Warmup Iteration 10: 62455277.778 ns/op
Iteration 1: 65696764.706 ns/op
Iteration 2: 66255823.529 ns/op
Iteration 3: 68231647.059 ns/op
Iteration 4: 71606750.000 ns/op
Iteration 5: 66780235.294 ns/op
Iteration 6: 69914750.000 ns/op
Iteration 7: 74681333.333 ns/op
Iteration 8: 64133944.444 ns/op
Iteration 9: 60831736.842 ns/op
Iteration 10: 63261444.444 ns/op
Iteration 11: 60814105.263 ns/op
Iteration 12: 67882941.176 ns/op
Iteration 13: 62512888.889 ns/op
Iteration 14: 65551823.529 ns/op
Iteration 15: 65574294.118 ns/op
Iteration 16: 67928352.941 ns/op
Iteration 17: 65510588.235 ns/op
Iteration 18: 62637055.556 ns/op
Iteration 19: 62900222.222 ns/op
Iteration 20: 63715944.444 ns/op
Iteration 21: 62284611.111 ns/op
Iteration 22: 68174235.294 ns/op
Iteration 23: 69029187.500 ns/op
Iteration 24: 68584764.706 ns/op
Iteration 25: 63332777.778 ns/op
Iteration 26: 64316000.000 ns/op
Iteration 27: 64219444.444 ns/op
Iteration 28: 64219888.889 ns/op
Iteration 29: 63113888.889 ns/op
Iteration 30: 65802117.647 ns/op
Iteration 31: 63661277.778 ns/op
Iteration 32: 63322000.000 ns/op
Iteration 33: 63770000.000 ns/op
Iteration 34: 63920333.333 ns/op
Iteration 35: 62403500.000 ns/op
Iteration 36: 64137166.667 ns/op
Iteration 37: 64725444.444 ns/op
Iteration 38: 63145166.667 ns/op
Iteration 39: 64183944.444 ns/op
Iteration 40: 65873666.667 ns/op
Iteration 41: 65648529.412 ns/op
Iteration 42: 66487176.471 ns/op
Iteration 43: 61702888.889 ns/op
Iteration 44: 63267666.667 ns/op
Iteration 45: 69614125.000 ns/op
Iteration 46: 61574222.222 ns/op
Iteration 47: 62678611.111 ns/op
Iteration 48: 62792277.778 ns/op
Iteration 49: 65355352.941 ns/op
Iteration 50: 65206823.529 ns/op
Iteration 51: 64246722.222 ns/op
Iteration 52: 68074823.529 ns/op
Iteration 53: 67314823.529 ns/op
Iteration 54: 65511882.353 ns/op
Iteration 55: 63607722.222 ns/op
Iteration 56: 66646529.412 ns/op
Iteration 57: 67675764.706 ns/op
Iteration 58: 62502388.889 ns/op
Iteration 59: 70476625.000 ns/op
Iteration 60: 97138500.000 ns/op
Iteration 61: 93276461.538 ns/op
Iteration 62: 103508818.182 ns/op
Iteration 63: 96725916.667 ns/op
Iteration 64: 97539500.000 ns/op
Iteration 65: 93591461.538 ns/op
Iteration 66: 95847083.333 ns/op
Iteration 67: 101147666.667 ns/op
Iteration 68: 91144230.769 ns/op
Iteration 69: 69291375.000 ns/op
Iteration 70: 67021941.176 ns/op
Iteration 71: 64101166.667 ns/op
Iteration 72: 66573823.529 ns/op
Iteration 73: 66116235.294 ns/op
Iteration 74: 62875611.111 ns/op
Iteration 75: 72765812.500 ns/op
Iteration 76: 65663176.471 ns/op
Iteration 77: 64187000.000 ns/op
Iteration 78: 64425444.444 ns/op
Iteration 79: 63835111.111 ns/op
Iteration 80: 62942166.667 ns/op
Iteration 81: 69006500.000 ns/op
Iteration 82: 64798176.471 ns/op
Iteration 83: 67823117.647 ns/op
Iteration 84: 77090866.667 ns/op
Iteration 85: 71548625.000 ns/op
Iteration 86: 63718333.333 ns/op
Iteration 87: 67034764.706 ns/op
Iteration 88: 61701833.333 ns/op
Iteration 89: 65456529.412 ns/op
Iteration 90: 63462388.889 ns/op
Iteration 91: 61797555.556 ns/op
Iteration 92: 66772823.529 ns/op
Iteration 93: 66240117.647 ns/op
Iteration 94: 63701444.444 ns/op
Iteration 95: 64047888.889 ns/op
Iteration 96: 62162777.778 ns/op
Iteration 97: 64846588.235 ns/op
Iteration 98: 61865666.667 ns/op
Iteration 99: 61965222.222 ns/op
Iteration 100: 64089333.333 ns/op
Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
Confidence interval (99.9%): [64952939.106, 71404582.386]
# Run complete. Total time: 00:07:18
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Jan. 7, 2015, 1:34 a.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Bill's comments.
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
Diffs (updated)
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
# Run progress: 0.00% complete, ETA 00:05:30
# Fork: 1 of 1
# Warmup Iteration 1: 278284250.000 ns/op
# Warmup Iteration 2: 70294312.500 ns/op
# Warmup Iteration 3: 19293379.310 ns/op
# Warmup Iteration 4: 11945387.097 ns/op
# Warmup Iteration 5: 10725388.350 ns/op
# Warmup Iteration 6: 13043282.353 ns/op
# Warmup Iteration 7: 9233083.333 ns/op
# Warmup Iteration 8: 9521051.724 ns/op
# Warmup Iteration 9: 10750854.369 ns/op
# Warmup Iteration 10: 7460243.243 ns/op
Iteration 1: 7885364.286 ns/op
Iteration 2: 7735139.860 ns/op
Iteration 3: 7660208.333 ns/op
Iteration 4: 7761204.225 ns/op
Iteration 5: 7868907.143 ns/op
Iteration 6: 7567404.110 ns/op
Iteration 7: 7611000.000 ns/op
Iteration 8: 7766154.930 ns/op
Iteration 9: 7669344.828 ns/op
Iteration 10: 7707783.217 ns/op
Iteration 11: 7435651.007 ns/op
Iteration 12: 7697631.944 ns/op
Iteration 13: 7712531.469 ns/op
Iteration 14: 7899407.143 ns/op
Iteration 15: 7448472.973 ns/op
Iteration 16: 7791521.127 ns/op
Iteration 17: 7612213.793 ns/op
Iteration 18: 7710867.133 ns/op
Iteration 19: 7649296.552 ns/op
Iteration 20: 7768309.859 ns/op
Iteration 21: 7688666.667 ns/op
Iteration 22: 7531557.823 ns/op
Iteration 23: 7381193.333 ns/op
Iteration 24: 7726006.993 ns/op
Iteration 25: 7603358.621 ns/op
Iteration 26: 7653631.944 ns/op
Iteration 27: 7442275.168 ns/op
Iteration 28: 7613186.207 ns/op
Iteration 29: 7765823.944 ns/op
Iteration 30: 7489687.075 ns/op
Iteration 31: 7811443.662 ns/op
Iteration 32: 8015007.246 ns/op
Iteration 33: 8192392.593 ns/op
Iteration 34: 8040335.766 ns/op
Iteration 35: 7584212.329 ns/op
Iteration 36: 8001934.783 ns/op
Iteration 37: 9744815.789 ns/op
Iteration 38: 11688284.211 ns/op
Iteration 39: 8661406.250 ns/op
Iteration 40: 7678413.793 ns/op
Iteration 41: 8502223.077 ns/op
Iteration 42: 7640820.690 ns/op
Iteration 43: 7875624.113 ns/op
Iteration 44: 7506809.524 ns/op
Iteration 45: 8005431.655 ns/op
Iteration 46: 8081664.234 ns/op
Iteration 47: 7579438.356 ns/op
Iteration 48: 7993405.797 ns/op
Iteration 49: 7571958.904 ns/op
Iteration 50: 8116463.235 ns/op
Iteration 51: 7941330.935 ns/op
Iteration 52: 7687145.833 ns/op
Iteration 53: 8082554.745 ns/op
Iteration 54: 7597889.655 ns/op
Iteration 55: 7299907.285 ns/op
Iteration 56: 7992789.855 ns/op
Iteration 57: 7648268.966 ns/op
Iteration 58: 7570863.014 ns/op
Iteration 59: 7885078.571 ns/op
Iteration 60: 7647158.621 ns/op
Iteration 61: 7830858.156 ns/op
Iteration 62: 7773690.141 ns/op
Iteration 63: 7905850.000 ns/op
Iteration 64: 7653800.000 ns/op
Iteration 65: 7408248.322 ns/op
Iteration 66: 7961352.518 ns/op
Iteration 67: 7879785.714 ns/op
Iteration 68: 8308616.541 ns/op
Iteration 69: 7396852.349 ns/op
Iteration 70: 8203214.815 ns/op
Iteration 71: 7548821.918 ns/op
Iteration 72: 7834631.206 ns/op
Iteration 73: 7451594.595 ns/op
Iteration 74: 7633262.069 ns/op
Iteration 75: 8182348.148 ns/op
Iteration 76: 7412646.667 ns/op
Iteration 77: 7662375.000 ns/op
Iteration 78: 7714825.175 ns/op
Iteration 79: 7726195.804 ns/op
Iteration 80: 7552568.493 ns/op
Iteration 81: 7642655.172 ns/op
Iteration 82: 7632737.931 ns/op
Iteration 83: 7567616.438 ns/op
Iteration 84: 7641517.241 ns/op
Iteration 85: 7664152.778 ns/op
Iteration 86: 8082532.847 ns/op
Iteration 87: 8027427.536 ns/op
Iteration 88: 7837191.489 ns/op
Iteration 89: 7587938.356 ns/op
Iteration 90: 7631262.069 ns/op
Iteration 91: 7672423.611 ns/op
Iteration 92: 7505258.503 ns/op
Iteration 93: 7410449.664 ns/op
Iteration 94: 8164419.118 ns/op
Iteration 95: 7712930.070 ns/op
Iteration 96: 7934287.770 ns/op
Iteration 97: 8007239.130 ns/op
Iteration 98: 8250343.284 ns/op
Iteration 99: 8482853.846 ns/op
Iteration 100: 8138595.588 ns/op
Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
Confidence interval (99.9%): [7659162.555, 8003522.350]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
# Run progress: 33.33% complete, ETA 00:04:43
# Fork: 1 of 1
# Warmup Iteration 1: 130891625.000 ns/op
# Warmup Iteration 2: 18382600.000 ns/op
# Warmup Iteration 3: 8235328.358 ns/op
# Warmup Iteration 4: 8724039.370 ns/op
# Warmup Iteration 5: 5823163.158 ns/op
# Warmup Iteration 6: 5425686.275 ns/op
# Warmup Iteration 7: 5722766.839 ns/op
# Warmup Iteration 8: 5716419.689 ns/op
# Warmup Iteration 9: 6278869.318 ns/op
# Warmup Iteration 10: 5738276.042 ns/op
Iteration 1: 5761427.083 ns/op
Iteration 2: 5946838.710 ns/op
Iteration 3: 6091066.298 ns/op
Iteration 4: 5721787.565 ns/op
Iteration 5: 6785852.761 ns/op
Iteration 6: 5813373.684 ns/op
Iteration 7: 5866824.468 ns/op
Iteration 8: 6948911.950 ns/op
Iteration 9: 6042060.109 ns/op
Iteration 10: 5633122.449 ns/op
Iteration 11: 5753838.542 ns/op
Iteration 12: 5853433.862 ns/op
Iteration 13: 5752322.917 ns/op
Iteration 14: 5332053.140 ns/op
Iteration 15: 5318278.846 ns/op
Iteration 16: 5260276.190 ns/op
Iteration 17: 5700340.206 ns/op
Iteration 18: 5767635.417 ns/op
Iteration 19: 5367514.563 ns/op
Iteration 20: 5721994.819 ns/op
Iteration 21: 5711243.523 ns/op
Iteration 22: 6276647.727 ns/op
Iteration 23: 5934473.118 ns/op
Iteration 24: 5821626.316 ns/op
Iteration 25: 5533788.945 ns/op
Iteration 26: 7226183.007 ns/op
Iteration 27: 5448920.792 ns/op
Iteration 28: 5749453.125 ns/op
Iteration 29: 5906727.273 ns/op
Iteration 30: 5702489.691 ns/op
Iteration 31: 5355660.194 ns/op
Iteration 32: 7413523.490 ns/op
Iteration 33: 7467831.081 ns/op
Iteration 34: 8952487.805 ns/op
Iteration 35: 8635742.188 ns/op
Iteration 36: 7981898.551 ns/op
Iteration 37: 7761732.394 ns/op
Iteration 38: 7130438.710 ns/op
Iteration 39: 8349406.015 ns/op
Iteration 40: 7980985.612 ns/op
Iteration 41: 8037847.826 ns/op
Iteration 42: 6315828.571 ns/op
Iteration 43: 5375892.683 ns/op
Iteration 44: 5897005.348 ns/op
Iteration 45: 5568793.970 ns/op
Iteration 46: 5273685.714 ns/op
Iteration 47: 8879392.000 ns/op
Iteration 48: 8641343.750 ns/op
Iteration 49: 10007136.364 ns/op
Iteration 50: 8484908.397 ns/op
Iteration 51: 7659333.333 ns/op
Iteration 52: 8661968.750 ns/op
Iteration 53: 8923153.226 ns/op
Iteration 54: 8746527.559 ns/op
Iteration 55: 8981138.211 ns/op
Iteration 56: 5631193.878 ns/op
Iteration 57: 5945069.892 ns/op
Iteration 58: 5610161.616 ns/op
Iteration 59: 5348665.049 ns/op
Iteration 60: 5429064.039 ns/op
Iteration 61: 5432418.719 ns/op
Iteration 62: 5621515.306 ns/op
Iteration 63: 5268630.332 ns/op
Iteration 64: 5849894.180 ns/op
Iteration 65: 5418126.829 ns/op
Iteration 66: 6133177.778 ns/op
Iteration 67: 6195095.506 ns/op
Iteration 68: 6111276.243 ns/op
Iteration 69: 5886244.681 ns/op
Iteration 70: 6430941.860 ns/op
Iteration 71: 5783329.843 ns/op
Iteration 72: 6507141.176 ns/op
Iteration 73: 10092783.784 ns/op
Iteration 74: 11873978.495 ns/op
Iteration 75: 8876424.000 ns/op
Iteration 76: 5878223.404 ns/op
Iteration 77: 7130651.613 ns/op
Iteration 78: 5508910.000 ns/op
Iteration 79: 5481338.308 ns/op
Iteration 80: 5999592.391 ns/op
Iteration 81: 6065917.582 ns/op
Iteration 82: 6087812.155 ns/op
Iteration 83: 5653512.821 ns/op
Iteration 84: 5528065.000 ns/op
Iteration 85: 5599314.721 ns/op
Iteration 86: 5300822.115 ns/op
Iteration 87: 6341833.333 ns/op
Iteration 88: 5511190.000 ns/op
Iteration 89: 5813026.316 ns/op
Iteration 90: 5914112.299 ns/op
Iteration 91: 5663928.205 ns/op
Iteration 92: 5723891.192 ns/op
Iteration 93: 5580843.434 ns/op
Iteration 94: 5990407.609 ns/op
Iteration 95: 5656717.949 ns/op
Iteration 96: 5432532.020 ns/op
Iteration 97: 5797827.225 ns/op
Iteration 98: 5647410.256 ns/op
Iteration 99: 5250814.286 ns/op
Iteration 100: 5602177.665 ns/op
Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
Confidence interval (99.9%): [5971349.307, 6865454.172]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
# Run progress: 66.67% complete, ETA 00:02:19
# Fork: 1 of 1
# Warmup Iteration 1: 7780760000.000 ns/op
# Warmup Iteration 2: 165927428.571 ns/op
# Warmup Iteration 3: 94242076.923 ns/op
# Warmup Iteration 4: 66061000.000 ns/op
# Warmup Iteration 5: 70444500.000 ns/op
# Warmup Iteration 6: 62350000.000 ns/op
# Warmup Iteration 7: 63746555.556 ns/op
# Warmup Iteration 8: 62712111.111 ns/op
# Warmup Iteration 9: 62776388.889 ns/op
# Warmup Iteration 10: 62455277.778 ns/op
Iteration 1: 65696764.706 ns/op
Iteration 2: 66255823.529 ns/op
Iteration 3: 68231647.059 ns/op
Iteration 4: 71606750.000 ns/op
Iteration 5: 66780235.294 ns/op
Iteration 6: 69914750.000 ns/op
Iteration 7: 74681333.333 ns/op
Iteration 8: 64133944.444 ns/op
Iteration 9: 60831736.842 ns/op
Iteration 10: 63261444.444 ns/op
Iteration 11: 60814105.263 ns/op
Iteration 12: 67882941.176 ns/op
Iteration 13: 62512888.889 ns/op
Iteration 14: 65551823.529 ns/op
Iteration 15: 65574294.118 ns/op
Iteration 16: 67928352.941 ns/op
Iteration 17: 65510588.235 ns/op
Iteration 18: 62637055.556 ns/op
Iteration 19: 62900222.222 ns/op
Iteration 20: 63715944.444 ns/op
Iteration 21: 62284611.111 ns/op
Iteration 22: 68174235.294 ns/op
Iteration 23: 69029187.500 ns/op
Iteration 24: 68584764.706 ns/op
Iteration 25: 63332777.778 ns/op
Iteration 26: 64316000.000 ns/op
Iteration 27: 64219444.444 ns/op
Iteration 28: 64219888.889 ns/op
Iteration 29: 63113888.889 ns/op
Iteration 30: 65802117.647 ns/op
Iteration 31: 63661277.778 ns/op
Iteration 32: 63322000.000 ns/op
Iteration 33: 63770000.000 ns/op
Iteration 34: 63920333.333 ns/op
Iteration 35: 62403500.000 ns/op
Iteration 36: 64137166.667 ns/op
Iteration 37: 64725444.444 ns/op
Iteration 38: 63145166.667 ns/op
Iteration 39: 64183944.444 ns/op
Iteration 40: 65873666.667 ns/op
Iteration 41: 65648529.412 ns/op
Iteration 42: 66487176.471 ns/op
Iteration 43: 61702888.889 ns/op
Iteration 44: 63267666.667 ns/op
Iteration 45: 69614125.000 ns/op
Iteration 46: 61574222.222 ns/op
Iteration 47: 62678611.111 ns/op
Iteration 48: 62792277.778 ns/op
Iteration 49: 65355352.941 ns/op
Iteration 50: 65206823.529 ns/op
Iteration 51: 64246722.222 ns/op
Iteration 52: 68074823.529 ns/op
Iteration 53: 67314823.529 ns/op
Iteration 54: 65511882.353 ns/op
Iteration 55: 63607722.222 ns/op
Iteration 56: 66646529.412 ns/op
Iteration 57: 67675764.706 ns/op
Iteration 58: 62502388.889 ns/op
Iteration 59: 70476625.000 ns/op
Iteration 60: 97138500.000 ns/op
Iteration 61: 93276461.538 ns/op
Iteration 62: 103508818.182 ns/op
Iteration 63: 96725916.667 ns/op
Iteration 64: 97539500.000 ns/op
Iteration 65: 93591461.538 ns/op
Iteration 66: 95847083.333 ns/op
Iteration 67: 101147666.667 ns/op
Iteration 68: 91144230.769 ns/op
Iteration 69: 69291375.000 ns/op
Iteration 70: 67021941.176 ns/op
Iteration 71: 64101166.667 ns/op
Iteration 72: 66573823.529 ns/op
Iteration 73: 66116235.294 ns/op
Iteration 74: 62875611.111 ns/op
Iteration 75: 72765812.500 ns/op
Iteration 76: 65663176.471 ns/op
Iteration 77: 64187000.000 ns/op
Iteration 78: 64425444.444 ns/op
Iteration 79: 63835111.111 ns/op
Iteration 80: 62942166.667 ns/op
Iteration 81: 69006500.000 ns/op
Iteration 82: 64798176.471 ns/op
Iteration 83: 67823117.647 ns/op
Iteration 84: 77090866.667 ns/op
Iteration 85: 71548625.000 ns/op
Iteration 86: 63718333.333 ns/op
Iteration 87: 67034764.706 ns/op
Iteration 88: 61701833.333 ns/op
Iteration 89: 65456529.412 ns/op
Iteration 90: 63462388.889 ns/op
Iteration 91: 61797555.556 ns/op
Iteration 92: 66772823.529 ns/op
Iteration 93: 66240117.647 ns/op
Iteration 94: 63701444.444 ns/op
Iteration 95: 64047888.889 ns/op
Iteration 96: 62162777.778 ns/op
Iteration 97: 64846588.235 ns/op
Iteration 98: 61865666.667 ns/op
Iteration 99: 61965222.222 ns/op
Iteration 100: 64089333.333 ns/op
Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
Confidence interval (99.9%): [64952939.106, 71404582.386]
# Run complete. Total time: 00:07:18
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java, line 23
> > <https://reviews.apache.org/r/28731/diff/4/?file=792353#file792353line23>
> >
> > Did you consider using `CachedClusterState` instead? That might cause some other things to fall into place, and alleviate the manual maintenance of this mapping. AFAICT there's a small amount of wiring necessary to make this work (including a more real `EventSink`), but that might be a worthwhile investment if it simplifies the benchmark implementations.
>
> Maxim Khutornenko wrote:
> I am hesitant to make CachedClusterState public as it's meaningless outside of its package. There is not much happening in FakeClusterState to press for a real object use here.
>
> Bill Farner wrote:
> Isn't this stance contradictory with this from above: "I actually prefer relying on real objects as much as possible to get more realistic perf picture". Especially given that `CachedClusterState` was introduced for performance reasons, seems like it belongs here just as much if not more than `StateManagerImpl`.
>
> Maxim Khutornenko wrote:
> I was referring to fakes/mocks that replace a substantial chunk of functionality with some custom logic. This specific case does not quite fit into that picture as all it does is key/value mapping. I decided to leave the CachedClusterState out due to the reasons above. If you feel CachedClusterState is important I can reconsider. Though it will likely going to make wiring more complicated due to event bus support.
Tried to wire in CachedClusterState and ran into AURORA-1046.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66714
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java, line 23
> > <https://reviews.apache.org/r/28731/diff/4/?file=792353#file792353line23>
> >
> > Did you consider using `CachedClusterState` instead? That might cause some other things to fall into place, and alleviate the manual maintenance of this mapping. AFAICT there's a small amount of wiring necessary to make this work (including a more real `EventSink`), but that might be a worthwhile investment if it simplifies the benchmark implementations.
>
> Maxim Khutornenko wrote:
> I am hesitant to make CachedClusterState public as it's meaningless outside of its package. There is not much happening in FakeClusterState to press for a real object use here.
Isn't this stance contradictory with this from above: "I actually prefer relying on real objects as much as possible to get more realistic perf picture". Especially given that `CachedClusterState` was introduced for performance reasons, seems like it belongs here just as much if not more than `StateManagerImpl`.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66714
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java, line 23
> > <https://reviews.apache.org/r/28731/diff/4/?file=792353#file792353line23>
> >
> > Did you consider using `CachedClusterState` instead? That might cause some other things to fall into place, and alleviate the manual maintenance of this mapping. AFAICT there's a small amount of wiring necessary to make this work (including a more real `EventSink`), but that might be a worthwhile investment if it simplifies the benchmark implementations.
>
> Maxim Khutornenko wrote:
> I am hesitant to make CachedClusterState public as it's meaningless outside of its package. There is not much happening in FakeClusterState to press for a real object use here.
>
> Bill Farner wrote:
> Isn't this stance contradictory with this from above: "I actually prefer relying on real objects as much as possible to get more realistic perf picture". Especially given that `CachedClusterState` was introduced for performance reasons, seems like it belongs here just as much if not more than `StateManagerImpl`.
>
> Maxim Khutornenko wrote:
> I was referring to fakes/mocks that replace a substantial chunk of functionality with some custom logic. This specific case does not quite fit into that picture as all it does is key/value mapping. I decided to leave the CachedClusterState out due to the reasons above. If you feel CachedClusterState is important I can reconsider. Though it will likely going to make wiring more complicated due to event bus support.
>
> Maxim Khutornenko wrote:
> Tried to wire in CachedClusterState and ran into AURORA-1046.
Should rebase more often. This has apparently been fixed in AURORA-1016. Update is coming.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66714
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java, line 23
> > <https://reviews.apache.org/r/28731/diff/4/?file=792353#file792353line23>
> >
> > Did you consider using `CachedClusterState` instead? That might cause some other things to fall into place, and alleviate the manual maintenance of this mapping. AFAICT there's a small amount of wiring necessary to make this work (including a more real `EventSink`), but that might be a worthwhile investment if it simplifies the benchmark implementations.
>
> Maxim Khutornenko wrote:
> I am hesitant to make CachedClusterState public as it's meaningless outside of its package. There is not much happening in FakeClusterState to press for a real object use here.
>
> Bill Farner wrote:
> Isn't this stance contradictory with this from above: "I actually prefer relying on real objects as much as possible to get more realistic perf picture". Especially given that `CachedClusterState` was introduced for performance reasons, seems like it belongs here just as much if not more than `StateManagerImpl`.
I was referring to fakes/mocks that replace a substantial chunk of functionality with some custom logic. This specific case does not quite fit into that picture as all it does is key/value mapping. I decided to leave the CachedClusterState out due to the reasons above. If you feel CachedClusterState is important I can reconsider. Though it will likely going to make wiring more complicated due to event bus support.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66714
-----------------------------------------------------------
On Jan. 7, 2015, 1:34 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 1:34 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/Hosts.java, line 44
> > <https://reviews.apache.org/r/28731/diff/4/?file=792348#file792348line44>
> >
> > It would be nice to supply the minimal attributes we expect - host and rack designations. this will at least cause diversity computation to be realistic. Seems like all you need to accomplish that is a parameter for the number of hosts per rack.
Sure, added.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 123
> > <https://reviews.apache.org/r/28731/diff/4/?file=792351#file792351line123>
> >
> > Did you consider using a fake for `StateManager`? I consider its behavior out of scope for this benchmark, and cutting it out might remove the need for a bunch of additional binding glue. The same can be said (to a lesster extent) about `TaskAssigner`.
I actually prefer relying on real objects as much as possible to get more realistic perf picture. Excessive use of fakes may distort the perf results to the extent of hiding problems or missing important observations. In addition, faking out StateManager would require manual assignment of tasks to hosts when filling up cluster (see below).
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 166
> > <https://reviews.apache.org/r/28731/diff/4/?file=792351#file792351line166>
> >
> > Is this only to get the tasks out of PENDING? If so, would it make sense to instead initialize the storage with tasks in the state you want? Seems like that would reduce the setup complexity.
Not just that, this is filling up cluster with real task->host assignments populating AssignedTask slave fields. Doing this manually would require much more overhead.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 196
> > <https://reviews.apache.org/r/28731/diff/4/?file=792351#file792351line196>
> >
> > `getClusterUtilization`?
Changed.
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/Tasks.java, line 65
> > <https://reviews.apache.org/r/28731/diff/4/?file=792352#file792352line65>
> >
> > This might be worth omitting, i fear its prominent inclusion here could be misleading about how priority is used in scheduling.
It was used for preemptor path to work correctly. I have replaced it with "isProduction".
> On Jan. 5, 2015, 8:38 p.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java, line 23
> > <https://reviews.apache.org/r/28731/diff/4/?file=792353#file792353line23>
> >
> > Did you consider using `CachedClusterState` instead? That might cause some other things to fall into place, and alleviate the manual maintenance of this mapping. AFAICT there's a small amount of wiring necessary to make this work (including a more real `EventSink`), but that might be a worthwhile investment if it simplifies the benchmark implementations.
I am hesitant to make CachedClusterState public as it's meaningless outside of its package. There is not much happening in FakeClusterState to press for a real object use here.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66714
-----------------------------------------------------------
On Dec. 15, 2014, 10:15 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 15, 2014, 10:15 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review66714
-----------------------------------------------------------
src/jmh/java/org/apache/aurora/benchmark/Hosts.java
<https://reviews.apache.org/r/28731/#comment110299>
It would be nice to supply the minimal attributes we expect - host and rack designations. this will at least cause diversity computation to be realistic. Seems like all you need to accomplish that is a parameter for the number of hosts per rack.
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment110313>
Did you consider using a fake for `StateManager`? I consider its behavior out of scope for this benchmark, and cutting it out might remove the need for a bunch of additional binding glue. The same can be said (to a lesster extent) about `TaskAssigner`.
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment110307>
Is this only to get the tasks out of PENDING? If so, would it make sense to instead initialize the storage with tasks in the state you want? Seems like that would reduce the setup complexity.
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28731/#comment110300>
`getClusterUtilization`?
src/jmh/java/org/apache/aurora/benchmark/Tasks.java
<https://reviews.apache.org/r/28731/#comment110301>
This might be worth omitting, i fear its prominent inclusion here could be misleading about how priority is used in scheduling.
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java
<https://reviews.apache.org/r/28731/#comment110309>
Did you consider using `CachedClusterState` instead? That might cause some other things to fall into place, and alleviate the manual maintenance of this mapping. AFAICT there's a small amount of wiring necessary to make this work (including a more real `EventSink`), but that might be a worthwhile investment if it simplifies the benchmark implementations.
- Bill Farner
On Dec. 15, 2014, 10:15 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 15, 2014, 10:15 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
>
> Diffs
> -----
>
> build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
> src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>
> # Run progress: 0.00% complete, ETA 00:05:30
> # Fork: 1 of 1
> # Warmup Iteration 1: 278284250.000 ns/op
> # Warmup Iteration 2: 70294312.500 ns/op
> # Warmup Iteration 3: 19293379.310 ns/op
> # Warmup Iteration 4: 11945387.097 ns/op
> # Warmup Iteration 5: 10725388.350 ns/op
> # Warmup Iteration 6: 13043282.353 ns/op
> # Warmup Iteration 7: 9233083.333 ns/op
> # Warmup Iteration 8: 9521051.724 ns/op
> # Warmup Iteration 9: 10750854.369 ns/op
> # Warmup Iteration 10: 7460243.243 ns/op
> Iteration 1: 7885364.286 ns/op
> Iteration 2: 7735139.860 ns/op
> Iteration 3: 7660208.333 ns/op
> Iteration 4: 7761204.225 ns/op
> Iteration 5: 7868907.143 ns/op
> Iteration 6: 7567404.110 ns/op
> Iteration 7: 7611000.000 ns/op
> Iteration 8: 7766154.930 ns/op
> Iteration 9: 7669344.828 ns/op
> Iteration 10: 7707783.217 ns/op
> Iteration 11: 7435651.007 ns/op
> Iteration 12: 7697631.944 ns/op
> Iteration 13: 7712531.469 ns/op
> Iteration 14: 7899407.143 ns/op
> Iteration 15: 7448472.973 ns/op
> Iteration 16: 7791521.127 ns/op
> Iteration 17: 7612213.793 ns/op
> Iteration 18: 7710867.133 ns/op
> Iteration 19: 7649296.552 ns/op
> Iteration 20: 7768309.859 ns/op
> Iteration 21: 7688666.667 ns/op
> Iteration 22: 7531557.823 ns/op
> Iteration 23: 7381193.333 ns/op
> Iteration 24: 7726006.993 ns/op
> Iteration 25: 7603358.621 ns/op
> Iteration 26: 7653631.944 ns/op
> Iteration 27: 7442275.168 ns/op
> Iteration 28: 7613186.207 ns/op
> Iteration 29: 7765823.944 ns/op
> Iteration 30: 7489687.075 ns/op
> Iteration 31: 7811443.662 ns/op
> Iteration 32: 8015007.246 ns/op
> Iteration 33: 8192392.593 ns/op
> Iteration 34: 8040335.766 ns/op
> Iteration 35: 7584212.329 ns/op
> Iteration 36: 8001934.783 ns/op
> Iteration 37: 9744815.789 ns/op
> Iteration 38: 11688284.211 ns/op
> Iteration 39: 8661406.250 ns/op
> Iteration 40: 7678413.793 ns/op
> Iteration 41: 8502223.077 ns/op
> Iteration 42: 7640820.690 ns/op
> Iteration 43: 7875624.113 ns/op
> Iteration 44: 7506809.524 ns/op
> Iteration 45: 8005431.655 ns/op
> Iteration 46: 8081664.234 ns/op
> Iteration 47: 7579438.356 ns/op
> Iteration 48: 7993405.797 ns/op
> Iteration 49: 7571958.904 ns/op
> Iteration 50: 8116463.235 ns/op
> Iteration 51: 7941330.935 ns/op
> Iteration 52: 7687145.833 ns/op
> Iteration 53: 8082554.745 ns/op
> Iteration 54: 7597889.655 ns/op
> Iteration 55: 7299907.285 ns/op
> Iteration 56: 7992789.855 ns/op
> Iteration 57: 7648268.966 ns/op
> Iteration 58: 7570863.014 ns/op
> Iteration 59: 7885078.571 ns/op
> Iteration 60: 7647158.621 ns/op
> Iteration 61: 7830858.156 ns/op
> Iteration 62: 7773690.141 ns/op
> Iteration 63: 7905850.000 ns/op
> Iteration 64: 7653800.000 ns/op
> Iteration 65: 7408248.322 ns/op
> Iteration 66: 7961352.518 ns/op
> Iteration 67: 7879785.714 ns/op
> Iteration 68: 8308616.541 ns/op
> Iteration 69: 7396852.349 ns/op
> Iteration 70: 8203214.815 ns/op
> Iteration 71: 7548821.918 ns/op
> Iteration 72: 7834631.206 ns/op
> Iteration 73: 7451594.595 ns/op
> Iteration 74: 7633262.069 ns/op
> Iteration 75: 8182348.148 ns/op
> Iteration 76: 7412646.667 ns/op
> Iteration 77: 7662375.000 ns/op
> Iteration 78: 7714825.175 ns/op
> Iteration 79: 7726195.804 ns/op
> Iteration 80: 7552568.493 ns/op
> Iteration 81: 7642655.172 ns/op
> Iteration 82: 7632737.931 ns/op
> Iteration 83: 7567616.438 ns/op
> Iteration 84: 7641517.241 ns/op
> Iteration 85: 7664152.778 ns/op
> Iteration 86: 8082532.847 ns/op
> Iteration 87: 8027427.536 ns/op
> Iteration 88: 7837191.489 ns/op
> Iteration 89: 7587938.356 ns/op
> Iteration 90: 7631262.069 ns/op
> Iteration 91: 7672423.611 ns/op
> Iteration 92: 7505258.503 ns/op
> Iteration 93: 7410449.664 ns/op
> Iteration 94: 8164419.118 ns/op
> Iteration 95: 7712930.070 ns/op
> Iteration 96: 7934287.770 ns/op
> Iteration 97: 8007239.130 ns/op
> Iteration 98: 8250343.284 ns/op
> Iteration 99: 8482853.846 ns/op
> Iteration 100: 8138595.588 ns/op
>
>
> Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
> Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
> Confidence interval (99.9%): [7659162.555, 8003522.350]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>
> # Run progress: 33.33% complete, ETA 00:04:43
> # Fork: 1 of 1
> # Warmup Iteration 1: 130891625.000 ns/op
> # Warmup Iteration 2: 18382600.000 ns/op
> # Warmup Iteration 3: 8235328.358 ns/op
> # Warmup Iteration 4: 8724039.370 ns/op
> # Warmup Iteration 5: 5823163.158 ns/op
> # Warmup Iteration 6: 5425686.275 ns/op
> # Warmup Iteration 7: 5722766.839 ns/op
> # Warmup Iteration 8: 5716419.689 ns/op
> # Warmup Iteration 9: 6278869.318 ns/op
> # Warmup Iteration 10: 5738276.042 ns/op
> Iteration 1: 5761427.083 ns/op
> Iteration 2: 5946838.710 ns/op
> Iteration 3: 6091066.298 ns/op
> Iteration 4: 5721787.565 ns/op
> Iteration 5: 6785852.761 ns/op
> Iteration 6: 5813373.684 ns/op
> Iteration 7: 5866824.468 ns/op
> Iteration 8: 6948911.950 ns/op
> Iteration 9: 6042060.109 ns/op
> Iteration 10: 5633122.449 ns/op
> Iteration 11: 5753838.542 ns/op
> Iteration 12: 5853433.862 ns/op
> Iteration 13: 5752322.917 ns/op
> Iteration 14: 5332053.140 ns/op
> Iteration 15: 5318278.846 ns/op
> Iteration 16: 5260276.190 ns/op
> Iteration 17: 5700340.206 ns/op
> Iteration 18: 5767635.417 ns/op
> Iteration 19: 5367514.563 ns/op
> Iteration 20: 5721994.819 ns/op
> Iteration 21: 5711243.523 ns/op
> Iteration 22: 6276647.727 ns/op
> Iteration 23: 5934473.118 ns/op
> Iteration 24: 5821626.316 ns/op
> Iteration 25: 5533788.945 ns/op
> Iteration 26: 7226183.007 ns/op
> Iteration 27: 5448920.792 ns/op
> Iteration 28: 5749453.125 ns/op
> Iteration 29: 5906727.273 ns/op
> Iteration 30: 5702489.691 ns/op
> Iteration 31: 5355660.194 ns/op
> Iteration 32: 7413523.490 ns/op
> Iteration 33: 7467831.081 ns/op
> Iteration 34: 8952487.805 ns/op
> Iteration 35: 8635742.188 ns/op
> Iteration 36: 7981898.551 ns/op
> Iteration 37: 7761732.394 ns/op
> Iteration 38: 7130438.710 ns/op
> Iteration 39: 8349406.015 ns/op
> Iteration 40: 7980985.612 ns/op
> Iteration 41: 8037847.826 ns/op
> Iteration 42: 6315828.571 ns/op
> Iteration 43: 5375892.683 ns/op
> Iteration 44: 5897005.348 ns/op
> Iteration 45: 5568793.970 ns/op
> Iteration 46: 5273685.714 ns/op
> Iteration 47: 8879392.000 ns/op
> Iteration 48: 8641343.750 ns/op
> Iteration 49: 10007136.364 ns/op
> Iteration 50: 8484908.397 ns/op
> Iteration 51: 7659333.333 ns/op
> Iteration 52: 8661968.750 ns/op
> Iteration 53: 8923153.226 ns/op
> Iteration 54: 8746527.559 ns/op
> Iteration 55: 8981138.211 ns/op
> Iteration 56: 5631193.878 ns/op
> Iteration 57: 5945069.892 ns/op
> Iteration 58: 5610161.616 ns/op
> Iteration 59: 5348665.049 ns/op
> Iteration 60: 5429064.039 ns/op
> Iteration 61: 5432418.719 ns/op
> Iteration 62: 5621515.306 ns/op
> Iteration 63: 5268630.332 ns/op
> Iteration 64: 5849894.180 ns/op
> Iteration 65: 5418126.829 ns/op
> Iteration 66: 6133177.778 ns/op
> Iteration 67: 6195095.506 ns/op
> Iteration 68: 6111276.243 ns/op
> Iteration 69: 5886244.681 ns/op
> Iteration 70: 6430941.860 ns/op
> Iteration 71: 5783329.843 ns/op
> Iteration 72: 6507141.176 ns/op
> Iteration 73: 10092783.784 ns/op
> Iteration 74: 11873978.495 ns/op
> Iteration 75: 8876424.000 ns/op
> Iteration 76: 5878223.404 ns/op
> Iteration 77: 7130651.613 ns/op
> Iteration 78: 5508910.000 ns/op
> Iteration 79: 5481338.308 ns/op
> Iteration 80: 5999592.391 ns/op
> Iteration 81: 6065917.582 ns/op
> Iteration 82: 6087812.155 ns/op
> Iteration 83: 5653512.821 ns/op
> Iteration 84: 5528065.000 ns/op
> Iteration 85: 5599314.721 ns/op
> Iteration 86: 5300822.115 ns/op
> Iteration 87: 6341833.333 ns/op
> Iteration 88: 5511190.000 ns/op
> Iteration 89: 5813026.316 ns/op
> Iteration 90: 5914112.299 ns/op
> Iteration 91: 5663928.205 ns/op
> Iteration 92: 5723891.192 ns/op
> Iteration 93: 5580843.434 ns/op
> Iteration 94: 5990407.609 ns/op
> Iteration 95: 5656717.949 ns/op
> Iteration 96: 5432532.020 ns/op
> Iteration 97: 5797827.225 ns/op
> Iteration 98: 5647410.256 ns/op
> Iteration 99: 5250814.286 ns/op
> Iteration 100: 5602177.665 ns/op
>
>
> Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
> Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
> Confidence interval (99.9%): [5971349.307, 6865454.172]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
>
> # Run progress: 66.67% complete, ETA 00:02:19
> # Fork: 1 of 1
> # Warmup Iteration 1: 7780760000.000 ns/op
> # Warmup Iteration 2: 165927428.571 ns/op
> # Warmup Iteration 3: 94242076.923 ns/op
> # Warmup Iteration 4: 66061000.000 ns/op
> # Warmup Iteration 5: 70444500.000 ns/op
> # Warmup Iteration 6: 62350000.000 ns/op
> # Warmup Iteration 7: 63746555.556 ns/op
> # Warmup Iteration 8: 62712111.111 ns/op
> # Warmup Iteration 9: 62776388.889 ns/op
> # Warmup Iteration 10: 62455277.778 ns/op
> Iteration 1: 65696764.706 ns/op
> Iteration 2: 66255823.529 ns/op
> Iteration 3: 68231647.059 ns/op
> Iteration 4: 71606750.000 ns/op
> Iteration 5: 66780235.294 ns/op
> Iteration 6: 69914750.000 ns/op
> Iteration 7: 74681333.333 ns/op
> Iteration 8: 64133944.444 ns/op
> Iteration 9: 60831736.842 ns/op
> Iteration 10: 63261444.444 ns/op
> Iteration 11: 60814105.263 ns/op
> Iteration 12: 67882941.176 ns/op
> Iteration 13: 62512888.889 ns/op
> Iteration 14: 65551823.529 ns/op
> Iteration 15: 65574294.118 ns/op
> Iteration 16: 67928352.941 ns/op
> Iteration 17: 65510588.235 ns/op
> Iteration 18: 62637055.556 ns/op
> Iteration 19: 62900222.222 ns/op
> Iteration 20: 63715944.444 ns/op
> Iteration 21: 62284611.111 ns/op
> Iteration 22: 68174235.294 ns/op
> Iteration 23: 69029187.500 ns/op
> Iteration 24: 68584764.706 ns/op
> Iteration 25: 63332777.778 ns/op
> Iteration 26: 64316000.000 ns/op
> Iteration 27: 64219444.444 ns/op
> Iteration 28: 64219888.889 ns/op
> Iteration 29: 63113888.889 ns/op
> Iteration 30: 65802117.647 ns/op
> Iteration 31: 63661277.778 ns/op
> Iteration 32: 63322000.000 ns/op
> Iteration 33: 63770000.000 ns/op
> Iteration 34: 63920333.333 ns/op
> Iteration 35: 62403500.000 ns/op
> Iteration 36: 64137166.667 ns/op
> Iteration 37: 64725444.444 ns/op
> Iteration 38: 63145166.667 ns/op
> Iteration 39: 64183944.444 ns/op
> Iteration 40: 65873666.667 ns/op
> Iteration 41: 65648529.412 ns/op
> Iteration 42: 66487176.471 ns/op
> Iteration 43: 61702888.889 ns/op
> Iteration 44: 63267666.667 ns/op
> Iteration 45: 69614125.000 ns/op
> Iteration 46: 61574222.222 ns/op
> Iteration 47: 62678611.111 ns/op
> Iteration 48: 62792277.778 ns/op
> Iteration 49: 65355352.941 ns/op
> Iteration 50: 65206823.529 ns/op
> Iteration 51: 64246722.222 ns/op
> Iteration 52: 68074823.529 ns/op
> Iteration 53: 67314823.529 ns/op
> Iteration 54: 65511882.353 ns/op
> Iteration 55: 63607722.222 ns/op
> Iteration 56: 66646529.412 ns/op
> Iteration 57: 67675764.706 ns/op
> Iteration 58: 62502388.889 ns/op
> Iteration 59: 70476625.000 ns/op
> Iteration 60: 97138500.000 ns/op
> Iteration 61: 93276461.538 ns/op
> Iteration 62: 103508818.182 ns/op
> Iteration 63: 96725916.667 ns/op
> Iteration 64: 97539500.000 ns/op
> Iteration 65: 93591461.538 ns/op
> Iteration 66: 95847083.333 ns/op
> Iteration 67: 101147666.667 ns/op
> Iteration 68: 91144230.769 ns/op
> Iteration 69: 69291375.000 ns/op
> Iteration 70: 67021941.176 ns/op
> Iteration 71: 64101166.667 ns/op
> Iteration 72: 66573823.529 ns/op
> Iteration 73: 66116235.294 ns/op
> Iteration 74: 62875611.111 ns/op
> Iteration 75: 72765812.500 ns/op
> Iteration 76: 65663176.471 ns/op
> Iteration 77: 64187000.000 ns/op
> Iteration 78: 64425444.444 ns/op
> Iteration 79: 63835111.111 ns/op
> Iteration 80: 62942166.667 ns/op
> Iteration 81: 69006500.000 ns/op
> Iteration 82: 64798176.471 ns/op
> Iteration 83: 67823117.647 ns/op
> Iteration 84: 77090866.667 ns/op
> Iteration 85: 71548625.000 ns/op
> Iteration 86: 63718333.333 ns/op
> Iteration 87: 67034764.706 ns/op
> Iteration 88: 61701833.333 ns/op
> Iteration 89: 65456529.412 ns/op
> Iteration 90: 63462388.889 ns/op
> Iteration 91: 61797555.556 ns/op
> Iteration 92: 66772823.529 ns/op
> Iteration 93: 66240117.647 ns/op
> Iteration 94: 63701444.444 ns/op
> Iteration 95: 64047888.889 ns/op
> Iteration 96: 62162777.778 ns/op
> Iteration 97: 64846588.235 ns/op
> Iteration 98: 61865666.667 ns/op
> Iteration 99: 61965222.222 ns/op
> Iteration 100: 64089333.333 ns/op
>
>
> Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
> Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
> Confidence interval (99.9%): [64952939.106, 71404582.386]
>
>
> # Run complete. Total time: 00:07:18
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Dec. 15, 2014, 10:15 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
CR comments.
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
Diffs (updated)
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeClusterState.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java f7e157c890b5627411acd4bd5c2559ef4829147c
src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 0204d14b19ae412236f19ca274d81decb4eba12d
Diff: https://reviews.apache.org/r/28731/diff/
Testing (updated)
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
# Run progress: 0.00% complete, ETA 00:05:30
# Fork: 1 of 1
# Warmup Iteration 1: 278284250.000 ns/op
# Warmup Iteration 2: 70294312.500 ns/op
# Warmup Iteration 3: 19293379.310 ns/op
# Warmup Iteration 4: 11945387.097 ns/op
# Warmup Iteration 5: 10725388.350 ns/op
# Warmup Iteration 6: 13043282.353 ns/op
# Warmup Iteration 7: 9233083.333 ns/op
# Warmup Iteration 8: 9521051.724 ns/op
# Warmup Iteration 9: 10750854.369 ns/op
# Warmup Iteration 10: 7460243.243 ns/op
Iteration 1: 7885364.286 ns/op
Iteration 2: 7735139.860 ns/op
Iteration 3: 7660208.333 ns/op
Iteration 4: 7761204.225 ns/op
Iteration 5: 7868907.143 ns/op
Iteration 6: 7567404.110 ns/op
Iteration 7: 7611000.000 ns/op
Iteration 8: 7766154.930 ns/op
Iteration 9: 7669344.828 ns/op
Iteration 10: 7707783.217 ns/op
Iteration 11: 7435651.007 ns/op
Iteration 12: 7697631.944 ns/op
Iteration 13: 7712531.469 ns/op
Iteration 14: 7899407.143 ns/op
Iteration 15: 7448472.973 ns/op
Iteration 16: 7791521.127 ns/op
Iteration 17: 7612213.793 ns/op
Iteration 18: 7710867.133 ns/op
Iteration 19: 7649296.552 ns/op
Iteration 20: 7768309.859 ns/op
Iteration 21: 7688666.667 ns/op
Iteration 22: 7531557.823 ns/op
Iteration 23: 7381193.333 ns/op
Iteration 24: 7726006.993 ns/op
Iteration 25: 7603358.621 ns/op
Iteration 26: 7653631.944 ns/op
Iteration 27: 7442275.168 ns/op
Iteration 28: 7613186.207 ns/op
Iteration 29: 7765823.944 ns/op
Iteration 30: 7489687.075 ns/op
Iteration 31: 7811443.662 ns/op
Iteration 32: 8015007.246 ns/op
Iteration 33: 8192392.593 ns/op
Iteration 34: 8040335.766 ns/op
Iteration 35: 7584212.329 ns/op
Iteration 36: 8001934.783 ns/op
Iteration 37: 9744815.789 ns/op
Iteration 38: 11688284.211 ns/op
Iteration 39: 8661406.250 ns/op
Iteration 40: 7678413.793 ns/op
Iteration 41: 8502223.077 ns/op
Iteration 42: 7640820.690 ns/op
Iteration 43: 7875624.113 ns/op
Iteration 44: 7506809.524 ns/op
Iteration 45: 8005431.655 ns/op
Iteration 46: 8081664.234 ns/op
Iteration 47: 7579438.356 ns/op
Iteration 48: 7993405.797 ns/op
Iteration 49: 7571958.904 ns/op
Iteration 50: 8116463.235 ns/op
Iteration 51: 7941330.935 ns/op
Iteration 52: 7687145.833 ns/op
Iteration 53: 8082554.745 ns/op
Iteration 54: 7597889.655 ns/op
Iteration 55: 7299907.285 ns/op
Iteration 56: 7992789.855 ns/op
Iteration 57: 7648268.966 ns/op
Iteration 58: 7570863.014 ns/op
Iteration 59: 7885078.571 ns/op
Iteration 60: 7647158.621 ns/op
Iteration 61: 7830858.156 ns/op
Iteration 62: 7773690.141 ns/op
Iteration 63: 7905850.000 ns/op
Iteration 64: 7653800.000 ns/op
Iteration 65: 7408248.322 ns/op
Iteration 66: 7961352.518 ns/op
Iteration 67: 7879785.714 ns/op
Iteration 68: 8308616.541 ns/op
Iteration 69: 7396852.349 ns/op
Iteration 70: 8203214.815 ns/op
Iteration 71: 7548821.918 ns/op
Iteration 72: 7834631.206 ns/op
Iteration 73: 7451594.595 ns/op
Iteration 74: 7633262.069 ns/op
Iteration 75: 8182348.148 ns/op
Iteration 76: 7412646.667 ns/op
Iteration 77: 7662375.000 ns/op
Iteration 78: 7714825.175 ns/op
Iteration 79: 7726195.804 ns/op
Iteration 80: 7552568.493 ns/op
Iteration 81: 7642655.172 ns/op
Iteration 82: 7632737.931 ns/op
Iteration 83: 7567616.438 ns/op
Iteration 84: 7641517.241 ns/op
Iteration 85: 7664152.778 ns/op
Iteration 86: 8082532.847 ns/op
Iteration 87: 8027427.536 ns/op
Iteration 88: 7837191.489 ns/op
Iteration 89: 7587938.356 ns/op
Iteration 90: 7631262.069 ns/op
Iteration 91: 7672423.611 ns/op
Iteration 92: 7505258.503 ns/op
Iteration 93: 7410449.664 ns/op
Iteration 94: 8164419.118 ns/op
Iteration 95: 7712930.070 ns/op
Iteration 96: 7934287.770 ns/op
Iteration 97: 8007239.130 ns/op
Iteration 98: 8250343.284 ns/op
Iteration 99: 8482853.846 ns/op
Iteration 100: 8138595.588 ns/op
Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average]
Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), stdev = 507676.348
Confidence interval (99.9%): [7659162.555, 8003522.350]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
# Run progress: 33.33% complete, ETA 00:04:43
# Fork: 1 of 1
# Warmup Iteration 1: 130891625.000 ns/op
# Warmup Iteration 2: 18382600.000 ns/op
# Warmup Iteration 3: 8235328.358 ns/op
# Warmup Iteration 4: 8724039.370 ns/op
# Warmup Iteration 5: 5823163.158 ns/op
# Warmup Iteration 6: 5425686.275 ns/op
# Warmup Iteration 7: 5722766.839 ns/op
# Warmup Iteration 8: 5716419.689 ns/op
# Warmup Iteration 9: 6278869.318 ns/op
# Warmup Iteration 10: 5738276.042 ns/op
Iteration 1: 5761427.083 ns/op
Iteration 2: 5946838.710 ns/op
Iteration 3: 6091066.298 ns/op
Iteration 4: 5721787.565 ns/op
Iteration 5: 6785852.761 ns/op
Iteration 6: 5813373.684 ns/op
Iteration 7: 5866824.468 ns/op
Iteration 8: 6948911.950 ns/op
Iteration 9: 6042060.109 ns/op
Iteration 10: 5633122.449 ns/op
Iteration 11: 5753838.542 ns/op
Iteration 12: 5853433.862 ns/op
Iteration 13: 5752322.917 ns/op
Iteration 14: 5332053.140 ns/op
Iteration 15: 5318278.846 ns/op
Iteration 16: 5260276.190 ns/op
Iteration 17: 5700340.206 ns/op
Iteration 18: 5767635.417 ns/op
Iteration 19: 5367514.563 ns/op
Iteration 20: 5721994.819 ns/op
Iteration 21: 5711243.523 ns/op
Iteration 22: 6276647.727 ns/op
Iteration 23: 5934473.118 ns/op
Iteration 24: 5821626.316 ns/op
Iteration 25: 5533788.945 ns/op
Iteration 26: 7226183.007 ns/op
Iteration 27: 5448920.792 ns/op
Iteration 28: 5749453.125 ns/op
Iteration 29: 5906727.273 ns/op
Iteration 30: 5702489.691 ns/op
Iteration 31: 5355660.194 ns/op
Iteration 32: 7413523.490 ns/op
Iteration 33: 7467831.081 ns/op
Iteration 34: 8952487.805 ns/op
Iteration 35: 8635742.188 ns/op
Iteration 36: 7981898.551 ns/op
Iteration 37: 7761732.394 ns/op
Iteration 38: 7130438.710 ns/op
Iteration 39: 8349406.015 ns/op
Iteration 40: 7980985.612 ns/op
Iteration 41: 8037847.826 ns/op
Iteration 42: 6315828.571 ns/op
Iteration 43: 5375892.683 ns/op
Iteration 44: 5897005.348 ns/op
Iteration 45: 5568793.970 ns/op
Iteration 46: 5273685.714 ns/op
Iteration 47: 8879392.000 ns/op
Iteration 48: 8641343.750 ns/op
Iteration 49: 10007136.364 ns/op
Iteration 50: 8484908.397 ns/op
Iteration 51: 7659333.333 ns/op
Iteration 52: 8661968.750 ns/op
Iteration 53: 8923153.226 ns/op
Iteration 54: 8746527.559 ns/op
Iteration 55: 8981138.211 ns/op
Iteration 56: 5631193.878 ns/op
Iteration 57: 5945069.892 ns/op
Iteration 58: 5610161.616 ns/op
Iteration 59: 5348665.049 ns/op
Iteration 60: 5429064.039 ns/op
Iteration 61: 5432418.719 ns/op
Iteration 62: 5621515.306 ns/op
Iteration 63: 5268630.332 ns/op
Iteration 64: 5849894.180 ns/op
Iteration 65: 5418126.829 ns/op
Iteration 66: 6133177.778 ns/op
Iteration 67: 6195095.506 ns/op
Iteration 68: 6111276.243 ns/op
Iteration 69: 5886244.681 ns/op
Iteration 70: 6430941.860 ns/op
Iteration 71: 5783329.843 ns/op
Iteration 72: 6507141.176 ns/op
Iteration 73: 10092783.784 ns/op
Iteration 74: 11873978.495 ns/op
Iteration 75: 8876424.000 ns/op
Iteration 76: 5878223.404 ns/op
Iteration 77: 7130651.613 ns/op
Iteration 78: 5508910.000 ns/op
Iteration 79: 5481338.308 ns/op
Iteration 80: 5999592.391 ns/op
Iteration 81: 6065917.582 ns/op
Iteration 82: 6087812.155 ns/op
Iteration 83: 5653512.821 ns/op
Iteration 84: 5528065.000 ns/op
Iteration 85: 5599314.721 ns/op
Iteration 86: 5300822.115 ns/op
Iteration 87: 6341833.333 ns/op
Iteration 88: 5511190.000 ns/op
Iteration 89: 5813026.316 ns/op
Iteration 90: 5914112.299 ns/op
Iteration 91: 5663928.205 ns/op
Iteration 92: 5723891.192 ns/op
Iteration 93: 5580843.434 ns/op
Iteration 94: 5990407.609 ns/op
Iteration 95: 5656717.949 ns/op
Iteration 96: 5432532.020 ns/op
Iteration 97: 5797827.225 ns/op
Iteration 98: 5647410.256 ns/op
Iteration 99: 5250814.286 ns/op
Iteration 100: 5602177.665 ns/op
Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average]
Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), stdev = 1318144.279
Confidence interval (99.9%): [5971349.307, 6865454.172]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
# Run progress: 66.67% complete, ETA 00:02:19
# Fork: 1 of 1
# Warmup Iteration 1: 7780760000.000 ns/op
# Warmup Iteration 2: 165927428.571 ns/op
# Warmup Iteration 3: 94242076.923 ns/op
# Warmup Iteration 4: 66061000.000 ns/op
# Warmup Iteration 5: 70444500.000 ns/op
# Warmup Iteration 6: 62350000.000 ns/op
# Warmup Iteration 7: 63746555.556 ns/op
# Warmup Iteration 8: 62712111.111 ns/op
# Warmup Iteration 9: 62776388.889 ns/op
# Warmup Iteration 10: 62455277.778 ns/op
Iteration 1: 65696764.706 ns/op
Iteration 2: 66255823.529 ns/op
Iteration 3: 68231647.059 ns/op
Iteration 4: 71606750.000 ns/op
Iteration 5: 66780235.294 ns/op
Iteration 6: 69914750.000 ns/op
Iteration 7: 74681333.333 ns/op
Iteration 8: 64133944.444 ns/op
Iteration 9: 60831736.842 ns/op
Iteration 10: 63261444.444 ns/op
Iteration 11: 60814105.263 ns/op
Iteration 12: 67882941.176 ns/op
Iteration 13: 62512888.889 ns/op
Iteration 14: 65551823.529 ns/op
Iteration 15: 65574294.118 ns/op
Iteration 16: 67928352.941 ns/op
Iteration 17: 65510588.235 ns/op
Iteration 18: 62637055.556 ns/op
Iteration 19: 62900222.222 ns/op
Iteration 20: 63715944.444 ns/op
Iteration 21: 62284611.111 ns/op
Iteration 22: 68174235.294 ns/op
Iteration 23: 69029187.500 ns/op
Iteration 24: 68584764.706 ns/op
Iteration 25: 63332777.778 ns/op
Iteration 26: 64316000.000 ns/op
Iteration 27: 64219444.444 ns/op
Iteration 28: 64219888.889 ns/op
Iteration 29: 63113888.889 ns/op
Iteration 30: 65802117.647 ns/op
Iteration 31: 63661277.778 ns/op
Iteration 32: 63322000.000 ns/op
Iteration 33: 63770000.000 ns/op
Iteration 34: 63920333.333 ns/op
Iteration 35: 62403500.000 ns/op
Iteration 36: 64137166.667 ns/op
Iteration 37: 64725444.444 ns/op
Iteration 38: 63145166.667 ns/op
Iteration 39: 64183944.444 ns/op
Iteration 40: 65873666.667 ns/op
Iteration 41: 65648529.412 ns/op
Iteration 42: 66487176.471 ns/op
Iteration 43: 61702888.889 ns/op
Iteration 44: 63267666.667 ns/op
Iteration 45: 69614125.000 ns/op
Iteration 46: 61574222.222 ns/op
Iteration 47: 62678611.111 ns/op
Iteration 48: 62792277.778 ns/op
Iteration 49: 65355352.941 ns/op
Iteration 50: 65206823.529 ns/op
Iteration 51: 64246722.222 ns/op
Iteration 52: 68074823.529 ns/op
Iteration 53: 67314823.529 ns/op
Iteration 54: 65511882.353 ns/op
Iteration 55: 63607722.222 ns/op
Iteration 56: 66646529.412 ns/op
Iteration 57: 67675764.706 ns/op
Iteration 58: 62502388.889 ns/op
Iteration 59: 70476625.000 ns/op
Iteration 60: 97138500.000 ns/op
Iteration 61: 93276461.538 ns/op
Iteration 62: 103508818.182 ns/op
Iteration 63: 96725916.667 ns/op
Iteration 64: 97539500.000 ns/op
Iteration 65: 93591461.538 ns/op
Iteration 66: 95847083.333 ns/op
Iteration 67: 101147666.667 ns/op
Iteration 68: 91144230.769 ns/op
Iteration 69: 69291375.000 ns/op
Iteration 70: 67021941.176 ns/op
Iteration 71: 64101166.667 ns/op
Iteration 72: 66573823.529 ns/op
Iteration 73: 66116235.294 ns/op
Iteration 74: 62875611.111 ns/op
Iteration 75: 72765812.500 ns/op
Iteration 76: 65663176.471 ns/op
Iteration 77: 64187000.000 ns/op
Iteration 78: 64425444.444 ns/op
Iteration 79: 63835111.111 ns/op
Iteration 80: 62942166.667 ns/op
Iteration 81: 69006500.000 ns/op
Iteration 82: 64798176.471 ns/op
Iteration 83: 67823117.647 ns/op
Iteration 84: 77090866.667 ns/op
Iteration 85: 71548625.000 ns/op
Iteration 86: 63718333.333 ns/op
Iteration 87: 67034764.706 ns/op
Iteration 88: 61701833.333 ns/op
Iteration 89: 65456529.412 ns/op
Iteration 90: 63462388.889 ns/op
Iteration 91: 61797555.556 ns/op
Iteration 92: 66772823.529 ns/op
Iteration 93: 66240117.647 ns/op
Iteration 94: 63701444.444 ns/op
Iteration 95: 64047888.889 ns/op
Iteration 96: 62162777.778 ns/op
Iteration 97: 64846588.235 ns/op
Iteration 98: 61865666.667 ns/op
Iteration 99: 61965222.222 ns/op
Iteration 100: 64089333.333 ns/op
Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average]
Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), stdev = 9511408.567
Confidence interval (99.9%): [64952939.106, 71404582.386]
# Run complete. Total time: 00:07:18
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7831342.453 ± 172179.897 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6418401.739 ± 447052.433 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 68178760.746 ± 3225821.640 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Dec. 11, 2014, 8:45 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Repository: aurora
Description (updated)
-------
Added baseline benchmarks for a few static veto cases.
Diffs
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
# Run progress: 0.00% complete, ETA 00:03:40
# Fork: 1 of 1
# Warmup Iteration 1: 104725900.000 ns/op
# Warmup Iteration 2: 20439888.889 ns/op
# Warmup Iteration 3: 13612185.185 ns/op
# Warmup Iteration 4: 5856248.677 ns/op
# Warmup Iteration 5: 5148014.019 ns/op
# Warmup Iteration 6: 4972004.505 ns/op
# Warmup Iteration 7: 4595562.500 ns/op
# Warmup Iteration 8: 4639226.891 ns/op
# Warmup Iteration 9: 4639521.008 ns/op
# Warmup Iteration 10: 4660248.945 ns/op
Iteration 1: 4718478.632 ns/op
Iteration 2: 4800686.957 ns/op
Iteration 3: 4438253.012 ns/op
Iteration 4: 4935376.682 ns/op
Iteration 5: 5236895.735 ns/op
Iteration 6: 6439345.029 ns/op
Iteration 7: 4792708.696 ns/op
Iteration 8: 4579008.299 ns/op
Iteration 9: 4681224.576 ns/op
Iteration 10: 4651143.460 ns/op
Iteration 11: 4394079.681 ns/op
Iteration 12: 4390573.705 ns/op
Iteration 13: 4363304.348 ns/op
Iteration 14: 4419488.000 ns/op
Iteration 15: 4869718.062 ns/op
Iteration 16: 4600612.500 ns/op
Iteration 17: 4475906.883 ns/op
Iteration 18: 4710388.889 ns/op
Iteration 19: 6137000.000 ns/op
Iteration 20: 4467425.101 ns/op
Iteration 21: 4348877.953 ns/op
Iteration 22: 4666907.173 ns/op
Iteration 23: 4431325.301 ns/op
Iteration 24: 4463186.235 ns/op
Iteration 25: 4580074.689 ns/op
Iteration 26: 4467805.668 ns/op
Iteration 27: 4886960.177 ns/op
Iteration 28: 4858515.419 ns/op
Iteration 29: 4734025.751 ns/op
Iteration 30: 4718081.197 ns/op
Iteration 31: 4711688.034 ns/op
Iteration 32: 4462392.713 ns/op
Iteration 33: 4683597.458 ns/op
Iteration 34: 4755176.724 ns/op
Iteration 35: 4530278.689 ns/op
Iteration 36: 4392119.048 ns/op
Iteration 37: 4659722.689 ns/op
Iteration 38: 4805869.565 ns/op
Iteration 39: 4398573.705 ns/op
Iteration 40: 4572547.718 ns/op
Iteration 41: 4643478.992 ns/op
Iteration 42: 4545699.588 ns/op
Iteration 43: 4670266.949 ns/op
Iteration 44: 4506677.551 ns/op
Iteration 45: 4574149.378 ns/op
Iteration 46: 4743154.506 ns/op
Iteration 47: 4777606.061 ns/op
Iteration 48: 4481987.805 ns/op
Iteration 49: 4840921.053 ns/op
Iteration 50: 4637231.092 ns/op
Iteration 51: 4515286.885 ns/op
Iteration 52: 4372884.921 ns/op
Iteration 53: 4385003.968 ns/op
Iteration 54: 4619648.536 ns/op
Iteration 55: 4557822.314 ns/op
Iteration 56: 4594362.500 ns/op
Iteration 57: 7870319.149 ns/op
Iteration 58: 9764495.575 ns/op
Iteration 59: 10302028.037 ns/op
Iteration 60: 6043169.399 ns/op
Iteration 61: 5523472.362 ns/op
Iteration 62: 4513685.714 ns/op
Iteration 63: 4361806.324 ns/op
Iteration 64: 4388615.079 ns/op
Iteration 65: 4548264.463 ns/op
Iteration 66: 4710752.137 ns/op
Iteration 67: 5688128.866 ns/op
Iteration 68: 6109292.818 ns/op
Iteration 69: 5896914.439 ns/op
Iteration 70: 5986548.913 ns/op
Iteration 71: 6093618.785 ns/op
Iteration 72: 6051450.549 ns/op
Iteration 73: 6237870.056 ns/op
Iteration 74: 5954345.946 ns/op
Iteration 75: 6460748.538 ns/op
Iteration 76: 5929037.634 ns/op
Iteration 77: 6170748.603 ns/op
Iteration 78: 6037382.514 ns/op
Iteration 79: 6156776.536 ns/op
Iteration 80: 5987103.261 ns/op
Iteration 81: 5927080.645 ns/op
Iteration 82: 5914165.775 ns/op
Iteration 83: 6039852.459 ns/op
Iteration 84: 6156798.883 ns/op
Iteration 85: 5902329.787 ns/op
Iteration 86: 5731875.648 ns/op
Iteration 87: 6137622.222 ns/op
Iteration 88: 5926424.731 ns/op
Iteration 89: 5959324.324 ns/op
Iteration 90: 6269630.682 ns/op
Iteration 91: 5720062.176 ns/op
Iteration 92: 5938537.634 ns/op
Iteration 93: 4430361.446 ns/op
Iteration 94: 4498792.683 ns/op
Iteration 95: 5454004.950 ns/op
Iteration 96: 4516479.508 ns/op
Iteration 97: 4382456.349 ns/op
Iteration 98: 4410548.000 ns/op
Iteration 99: 4301657.588 ns/op
Iteration 100: 4438582.329 ns/op
Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
Confidence interval (99.9%): [4820809.970, 5509963.825]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
# Run progress: 50.00% complete, ETA 00:02:14
# Fork: 1 of 1
# Warmup Iteration 1: 34499258.065 ns/op
# Warmup Iteration 2: 9519784.483 ns/op
# Warmup Iteration 3: 7017031.847 ns/op
# Warmup Iteration 4: 4898586.667 ns/op
# Warmup Iteration 5: 4924111.607 ns/op
# Warmup Iteration 6: 5391946.341 ns/op
# Warmup Iteration 7: 5405700.980 ns/op
# Warmup Iteration 8: 5138181.395 ns/op
# Warmup Iteration 9: 4860863.436 ns/op
# Warmup Iteration 10: 5076539.171 ns/op
Iteration 1: 4693804.255 ns/op
Iteration 2: 5147288.372 ns/op
Iteration 3: 4809647.826 ns/op
Iteration 4: 6034147.541 ns/op
Iteration 5: 6675740.964 ns/op
Iteration 6: 5156481.308 ns/op
Iteration 7: 4678330.508 ns/op
Iteration 8: 4775528.139 ns/op
Iteration 9: 4800773.913 ns/op
Iteration 10: 4623054.393 ns/op
Iteration 11: 4816021.834 ns/op
Iteration 12: 4597212.500 ns/op
Iteration 13: 4691842.553 ns/op
Iteration 14: 4720820.513 ns/op
Iteration 15: 4662227.848 ns/op
Iteration 16: 5261447.619 ns/op
Iteration 17: 4840087.719 ns/op
Iteration 18: 4726081.197 ns/op
Iteration 19: 4673974.576 ns/op
Iteration 20: 5129837.209 ns/op
Iteration 21: 4724444.444 ns/op
Iteration 22: 4740678.112 ns/op
Iteration 23: 4710260.684 ns/op
Iteration 24: 4789688.312 ns/op
Iteration 25: 4589266.667 ns/op
Iteration 26: 4675394.068 ns/op
Iteration 27: 4594887.500 ns/op
Iteration 28: 4642579.832 ns/op
Iteration 29: 4666215.190 ns/op
Iteration 30: 4916044.444 ns/op
Iteration 31: 4734978.541 ns/op
Iteration 32: 4527586.066 ns/op
Iteration 33: 4695753.191 ns/op
Iteration 34: 4675682.203 ns/op
Iteration 35: 4598608.333 ns/op
Iteration 36: 4616338.912 ns/op
Iteration 37: 4722192.308 ns/op
Iteration 38: 5385243.902 ns/op
Iteration 39: 5183159.624 ns/op
Iteration 40: 4802855.895 ns/op
Iteration 41: 5393404.878 ns/op
Iteration 42: 7052592.357 ns/op
Iteration 43: 6087237.569 ns/op
Iteration 44: 6829685.185 ns/op
Iteration 45: 6105154.696 ns/op
Iteration 46: 5331869.565 ns/op
Iteration 47: 4765870.690 ns/op
Iteration 48: 4635117.155 ns/op
Iteration 49: 4882526.549 ns/op
Iteration 50: 4883402.655 ns/op
Iteration 51: 5018100.000 ns/op
Iteration 52: 4899177.778 ns/op
Iteration 53: 5568404.040 ns/op
Iteration 54: 4912208.889 ns/op
Iteration 55: 5073077.982 ns/op
Iteration 56: 5441458.128 ns/op
Iteration 57: 4964382.883 ns/op
Iteration 58: 5379580.488 ns/op
Iteration 59: 4925330.357 ns/op
Iteration 60: 5111805.556 ns/op
Iteration 61: 5226265.403 ns/op
Iteration 62: 5304846.154 ns/op
Iteration 63: 4920982.222 ns/op
Iteration 64: 4905186.667 ns/op
Iteration 65: 4970238.739 ns/op
Iteration 66: 5002425.339 ns/op
Iteration 67: 4842846.491 ns/op
Iteration 68: 5543889.447 ns/op
Iteration 69: 4828794.760 ns/op
Iteration 70: 4809737.991 ns/op
Iteration 71: 5151565.421 ns/op
Iteration 72: 5077170.507 ns/op
Iteration 73: 4987425.339 ns/op
Iteration 74: 4888584.071 ns/op
Iteration 75: 4980319.820 ns/op
Iteration 76: 5075321.101 ns/op
Iteration 77: 4719508.547 ns/op
Iteration 78: 4932200.893 ns/op
Iteration 79: 5046264.840 ns/op
Iteration 80: 4835530.702 ns/op
Iteration 81: 4855162.996 ns/op
Iteration 82: 4814104.803 ns/op
Iteration 83: 4650232.068 ns/op
Iteration 84: 5211075.472 ns/op
Iteration 85: 4970504.505 ns/op
Iteration 86: 4776272.727 ns/op
Iteration 87: 4929089.286 ns/op
Iteration 88: 5063995.413 ns/op
Iteration 89: 5059706.422 ns/op
Iteration 90: 5042342.466 ns/op
Iteration 91: 4760637.931 ns/op
Iteration 92: 5009213.636 ns/op
Iteration 93: 5985075.676 ns/op
Iteration 94: 4735798.283 ns/op
Iteration 95: 4893389.381 ns/op
Iteration 96: 6055005.495 ns/op
Iteration 97: 4684194.915 ns/op
Iteration 98: 4649130.252 ns/op
Iteration 99: 4619602.510 ns/op
Iteration 100: 4593595.833 ns/op
Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
Confidence interval (99.9%): [4851488.689, 5171987.930]
# Run complete. Total time: 00:04:29
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Dec. 11, 2014, 8:37 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Rebased.
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
This is branched off of https://reviews.apache.org/r/28710
Diffs (updated)
-----
build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java 5cecada93e4e04b689e826af49f691ed7e94ae49
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
# Run progress: 0.00% complete, ETA 00:03:40
# Fork: 1 of 1
# Warmup Iteration 1: 104725900.000 ns/op
# Warmup Iteration 2: 20439888.889 ns/op
# Warmup Iteration 3: 13612185.185 ns/op
# Warmup Iteration 4: 5856248.677 ns/op
# Warmup Iteration 5: 5148014.019 ns/op
# Warmup Iteration 6: 4972004.505 ns/op
# Warmup Iteration 7: 4595562.500 ns/op
# Warmup Iteration 8: 4639226.891 ns/op
# Warmup Iteration 9: 4639521.008 ns/op
# Warmup Iteration 10: 4660248.945 ns/op
Iteration 1: 4718478.632 ns/op
Iteration 2: 4800686.957 ns/op
Iteration 3: 4438253.012 ns/op
Iteration 4: 4935376.682 ns/op
Iteration 5: 5236895.735 ns/op
Iteration 6: 6439345.029 ns/op
Iteration 7: 4792708.696 ns/op
Iteration 8: 4579008.299 ns/op
Iteration 9: 4681224.576 ns/op
Iteration 10: 4651143.460 ns/op
Iteration 11: 4394079.681 ns/op
Iteration 12: 4390573.705 ns/op
Iteration 13: 4363304.348 ns/op
Iteration 14: 4419488.000 ns/op
Iteration 15: 4869718.062 ns/op
Iteration 16: 4600612.500 ns/op
Iteration 17: 4475906.883 ns/op
Iteration 18: 4710388.889 ns/op
Iteration 19: 6137000.000 ns/op
Iteration 20: 4467425.101 ns/op
Iteration 21: 4348877.953 ns/op
Iteration 22: 4666907.173 ns/op
Iteration 23: 4431325.301 ns/op
Iteration 24: 4463186.235 ns/op
Iteration 25: 4580074.689 ns/op
Iteration 26: 4467805.668 ns/op
Iteration 27: 4886960.177 ns/op
Iteration 28: 4858515.419 ns/op
Iteration 29: 4734025.751 ns/op
Iteration 30: 4718081.197 ns/op
Iteration 31: 4711688.034 ns/op
Iteration 32: 4462392.713 ns/op
Iteration 33: 4683597.458 ns/op
Iteration 34: 4755176.724 ns/op
Iteration 35: 4530278.689 ns/op
Iteration 36: 4392119.048 ns/op
Iteration 37: 4659722.689 ns/op
Iteration 38: 4805869.565 ns/op
Iteration 39: 4398573.705 ns/op
Iteration 40: 4572547.718 ns/op
Iteration 41: 4643478.992 ns/op
Iteration 42: 4545699.588 ns/op
Iteration 43: 4670266.949 ns/op
Iteration 44: 4506677.551 ns/op
Iteration 45: 4574149.378 ns/op
Iteration 46: 4743154.506 ns/op
Iteration 47: 4777606.061 ns/op
Iteration 48: 4481987.805 ns/op
Iteration 49: 4840921.053 ns/op
Iteration 50: 4637231.092 ns/op
Iteration 51: 4515286.885 ns/op
Iteration 52: 4372884.921 ns/op
Iteration 53: 4385003.968 ns/op
Iteration 54: 4619648.536 ns/op
Iteration 55: 4557822.314 ns/op
Iteration 56: 4594362.500 ns/op
Iteration 57: 7870319.149 ns/op
Iteration 58: 9764495.575 ns/op
Iteration 59: 10302028.037 ns/op
Iteration 60: 6043169.399 ns/op
Iteration 61: 5523472.362 ns/op
Iteration 62: 4513685.714 ns/op
Iteration 63: 4361806.324 ns/op
Iteration 64: 4388615.079 ns/op
Iteration 65: 4548264.463 ns/op
Iteration 66: 4710752.137 ns/op
Iteration 67: 5688128.866 ns/op
Iteration 68: 6109292.818 ns/op
Iteration 69: 5896914.439 ns/op
Iteration 70: 5986548.913 ns/op
Iteration 71: 6093618.785 ns/op
Iteration 72: 6051450.549 ns/op
Iteration 73: 6237870.056 ns/op
Iteration 74: 5954345.946 ns/op
Iteration 75: 6460748.538 ns/op
Iteration 76: 5929037.634 ns/op
Iteration 77: 6170748.603 ns/op
Iteration 78: 6037382.514 ns/op
Iteration 79: 6156776.536 ns/op
Iteration 80: 5987103.261 ns/op
Iteration 81: 5927080.645 ns/op
Iteration 82: 5914165.775 ns/op
Iteration 83: 6039852.459 ns/op
Iteration 84: 6156798.883 ns/op
Iteration 85: 5902329.787 ns/op
Iteration 86: 5731875.648 ns/op
Iteration 87: 6137622.222 ns/op
Iteration 88: 5926424.731 ns/op
Iteration 89: 5959324.324 ns/op
Iteration 90: 6269630.682 ns/op
Iteration 91: 5720062.176 ns/op
Iteration 92: 5938537.634 ns/op
Iteration 93: 4430361.446 ns/op
Iteration 94: 4498792.683 ns/op
Iteration 95: 5454004.950 ns/op
Iteration 96: 4516479.508 ns/op
Iteration 97: 4382456.349 ns/op
Iteration 98: 4410548.000 ns/op
Iteration 99: 4301657.588 ns/op
Iteration 100: 4438582.329 ns/op
Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
Confidence interval (99.9%): [4820809.970, 5509963.825]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
# Run progress: 50.00% complete, ETA 00:02:14
# Fork: 1 of 1
# Warmup Iteration 1: 34499258.065 ns/op
# Warmup Iteration 2: 9519784.483 ns/op
# Warmup Iteration 3: 7017031.847 ns/op
# Warmup Iteration 4: 4898586.667 ns/op
# Warmup Iteration 5: 4924111.607 ns/op
# Warmup Iteration 6: 5391946.341 ns/op
# Warmup Iteration 7: 5405700.980 ns/op
# Warmup Iteration 8: 5138181.395 ns/op
# Warmup Iteration 9: 4860863.436 ns/op
# Warmup Iteration 10: 5076539.171 ns/op
Iteration 1: 4693804.255 ns/op
Iteration 2: 5147288.372 ns/op
Iteration 3: 4809647.826 ns/op
Iteration 4: 6034147.541 ns/op
Iteration 5: 6675740.964 ns/op
Iteration 6: 5156481.308 ns/op
Iteration 7: 4678330.508 ns/op
Iteration 8: 4775528.139 ns/op
Iteration 9: 4800773.913 ns/op
Iteration 10: 4623054.393 ns/op
Iteration 11: 4816021.834 ns/op
Iteration 12: 4597212.500 ns/op
Iteration 13: 4691842.553 ns/op
Iteration 14: 4720820.513 ns/op
Iteration 15: 4662227.848 ns/op
Iteration 16: 5261447.619 ns/op
Iteration 17: 4840087.719 ns/op
Iteration 18: 4726081.197 ns/op
Iteration 19: 4673974.576 ns/op
Iteration 20: 5129837.209 ns/op
Iteration 21: 4724444.444 ns/op
Iteration 22: 4740678.112 ns/op
Iteration 23: 4710260.684 ns/op
Iteration 24: 4789688.312 ns/op
Iteration 25: 4589266.667 ns/op
Iteration 26: 4675394.068 ns/op
Iteration 27: 4594887.500 ns/op
Iteration 28: 4642579.832 ns/op
Iteration 29: 4666215.190 ns/op
Iteration 30: 4916044.444 ns/op
Iteration 31: 4734978.541 ns/op
Iteration 32: 4527586.066 ns/op
Iteration 33: 4695753.191 ns/op
Iteration 34: 4675682.203 ns/op
Iteration 35: 4598608.333 ns/op
Iteration 36: 4616338.912 ns/op
Iteration 37: 4722192.308 ns/op
Iteration 38: 5385243.902 ns/op
Iteration 39: 5183159.624 ns/op
Iteration 40: 4802855.895 ns/op
Iteration 41: 5393404.878 ns/op
Iteration 42: 7052592.357 ns/op
Iteration 43: 6087237.569 ns/op
Iteration 44: 6829685.185 ns/op
Iteration 45: 6105154.696 ns/op
Iteration 46: 5331869.565 ns/op
Iteration 47: 4765870.690 ns/op
Iteration 48: 4635117.155 ns/op
Iteration 49: 4882526.549 ns/op
Iteration 50: 4883402.655 ns/op
Iteration 51: 5018100.000 ns/op
Iteration 52: 4899177.778 ns/op
Iteration 53: 5568404.040 ns/op
Iteration 54: 4912208.889 ns/op
Iteration 55: 5073077.982 ns/op
Iteration 56: 5441458.128 ns/op
Iteration 57: 4964382.883 ns/op
Iteration 58: 5379580.488 ns/op
Iteration 59: 4925330.357 ns/op
Iteration 60: 5111805.556 ns/op
Iteration 61: 5226265.403 ns/op
Iteration 62: 5304846.154 ns/op
Iteration 63: 4920982.222 ns/op
Iteration 64: 4905186.667 ns/op
Iteration 65: 4970238.739 ns/op
Iteration 66: 5002425.339 ns/op
Iteration 67: 4842846.491 ns/op
Iteration 68: 5543889.447 ns/op
Iteration 69: 4828794.760 ns/op
Iteration 70: 4809737.991 ns/op
Iteration 71: 5151565.421 ns/op
Iteration 72: 5077170.507 ns/op
Iteration 73: 4987425.339 ns/op
Iteration 74: 4888584.071 ns/op
Iteration 75: 4980319.820 ns/op
Iteration 76: 5075321.101 ns/op
Iteration 77: 4719508.547 ns/op
Iteration 78: 4932200.893 ns/op
Iteration 79: 5046264.840 ns/op
Iteration 80: 4835530.702 ns/op
Iteration 81: 4855162.996 ns/op
Iteration 82: 4814104.803 ns/op
Iteration 83: 4650232.068 ns/op
Iteration 84: 5211075.472 ns/op
Iteration 85: 4970504.505 ns/op
Iteration 86: 4776272.727 ns/op
Iteration 87: 4929089.286 ns/op
Iteration 88: 5063995.413 ns/op
Iteration 89: 5059706.422 ns/op
Iteration 90: 5042342.466 ns/op
Iteration 91: 4760637.931 ns/op
Iteration 92: 5009213.636 ns/op
Iteration 93: 5985075.676 ns/op
Iteration 94: 4735798.283 ns/op
Iteration 95: 4893389.381 ns/op
Iteration 96: 6055005.495 ns/op
Iteration 97: 4684194.915 ns/op
Iteration 98: 4649130.252 ns/op
Iteration 99: 4619602.510 ns/op
Iteration 100: 4593595.833 ns/op
Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
Confidence interval (99.9%): [4851488.689, 5171987.930]
# Run complete. Total time: 00:04:29
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review64112
-----------------------------------------------------------
This patch does not apply cleanly on master (40d2718), do you need to rebase?
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 5, 2014, 11:02 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 11:02 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
> This is branched off of https://reviews.apache.org/r/28710
>
>
> Diffs
> -----
>
> build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeEventSink.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:03:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 104725900.000 ns/op
> # Warmup Iteration 2: 20439888.889 ns/op
> # Warmup Iteration 3: 13612185.185 ns/op
> # Warmup Iteration 4: 5856248.677 ns/op
> # Warmup Iteration 5: 5148014.019 ns/op
> # Warmup Iteration 6: 4972004.505 ns/op
> # Warmup Iteration 7: 4595562.500 ns/op
> # Warmup Iteration 8: 4639226.891 ns/op
> # Warmup Iteration 9: 4639521.008 ns/op
> # Warmup Iteration 10: 4660248.945 ns/op
> Iteration 1: 4718478.632 ns/op
> Iteration 2: 4800686.957 ns/op
> Iteration 3: 4438253.012 ns/op
> Iteration 4: 4935376.682 ns/op
> Iteration 5: 5236895.735 ns/op
> Iteration 6: 6439345.029 ns/op
> Iteration 7: 4792708.696 ns/op
> Iteration 8: 4579008.299 ns/op
> Iteration 9: 4681224.576 ns/op
> Iteration 10: 4651143.460 ns/op
> Iteration 11: 4394079.681 ns/op
> Iteration 12: 4390573.705 ns/op
> Iteration 13: 4363304.348 ns/op
> Iteration 14: 4419488.000 ns/op
> Iteration 15: 4869718.062 ns/op
> Iteration 16: 4600612.500 ns/op
> Iteration 17: 4475906.883 ns/op
> Iteration 18: 4710388.889 ns/op
> Iteration 19: 6137000.000 ns/op
> Iteration 20: 4467425.101 ns/op
> Iteration 21: 4348877.953 ns/op
> Iteration 22: 4666907.173 ns/op
> Iteration 23: 4431325.301 ns/op
> Iteration 24: 4463186.235 ns/op
> Iteration 25: 4580074.689 ns/op
> Iteration 26: 4467805.668 ns/op
> Iteration 27: 4886960.177 ns/op
> Iteration 28: 4858515.419 ns/op
> Iteration 29: 4734025.751 ns/op
> Iteration 30: 4718081.197 ns/op
> Iteration 31: 4711688.034 ns/op
> Iteration 32: 4462392.713 ns/op
> Iteration 33: 4683597.458 ns/op
> Iteration 34: 4755176.724 ns/op
> Iteration 35: 4530278.689 ns/op
> Iteration 36: 4392119.048 ns/op
> Iteration 37: 4659722.689 ns/op
> Iteration 38: 4805869.565 ns/op
> Iteration 39: 4398573.705 ns/op
> Iteration 40: 4572547.718 ns/op
> Iteration 41: 4643478.992 ns/op
> Iteration 42: 4545699.588 ns/op
> Iteration 43: 4670266.949 ns/op
> Iteration 44: 4506677.551 ns/op
> Iteration 45: 4574149.378 ns/op
> Iteration 46: 4743154.506 ns/op
> Iteration 47: 4777606.061 ns/op
> Iteration 48: 4481987.805 ns/op
> Iteration 49: 4840921.053 ns/op
> Iteration 50: 4637231.092 ns/op
> Iteration 51: 4515286.885 ns/op
> Iteration 52: 4372884.921 ns/op
> Iteration 53: 4385003.968 ns/op
> Iteration 54: 4619648.536 ns/op
> Iteration 55: 4557822.314 ns/op
> Iteration 56: 4594362.500 ns/op
> Iteration 57: 7870319.149 ns/op
> Iteration 58: 9764495.575 ns/op
> Iteration 59: 10302028.037 ns/op
> Iteration 60: 6043169.399 ns/op
> Iteration 61: 5523472.362 ns/op
> Iteration 62: 4513685.714 ns/op
> Iteration 63: 4361806.324 ns/op
> Iteration 64: 4388615.079 ns/op
> Iteration 65: 4548264.463 ns/op
> Iteration 66: 4710752.137 ns/op
> Iteration 67: 5688128.866 ns/op
> Iteration 68: 6109292.818 ns/op
> Iteration 69: 5896914.439 ns/op
> Iteration 70: 5986548.913 ns/op
> Iteration 71: 6093618.785 ns/op
> Iteration 72: 6051450.549 ns/op
> Iteration 73: 6237870.056 ns/op
> Iteration 74: 5954345.946 ns/op
> Iteration 75: 6460748.538 ns/op
> Iteration 76: 5929037.634 ns/op
> Iteration 77: 6170748.603 ns/op
> Iteration 78: 6037382.514 ns/op
> Iteration 79: 6156776.536 ns/op
> Iteration 80: 5987103.261 ns/op
> Iteration 81: 5927080.645 ns/op
> Iteration 82: 5914165.775 ns/op
> Iteration 83: 6039852.459 ns/op
> Iteration 84: 6156798.883 ns/op
> Iteration 85: 5902329.787 ns/op
> Iteration 86: 5731875.648 ns/op
> Iteration 87: 6137622.222 ns/op
> Iteration 88: 5926424.731 ns/op
> Iteration 89: 5959324.324 ns/op
> Iteration 90: 6269630.682 ns/op
> Iteration 91: 5720062.176 ns/op
> Iteration 92: 5938537.634 ns/op
> Iteration 93: 4430361.446 ns/op
> Iteration 94: 4498792.683 ns/op
> Iteration 95: 5454004.950 ns/op
> Iteration 96: 4516479.508 ns/op
> Iteration 97: 4382456.349 ns/op
> Iteration 98: 4410548.000 ns/op
> Iteration 99: 4301657.588 ns/op
> Iteration 100: 4438582.329 ns/op
>
>
> Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
> Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
> Confidence interval (99.9%): [4820809.970, 5509963.825]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
>
> # Run progress: 50.00% complete, ETA 00:02:14
> # Fork: 1 of 1
> # Warmup Iteration 1: 34499258.065 ns/op
> # Warmup Iteration 2: 9519784.483 ns/op
> # Warmup Iteration 3: 7017031.847 ns/op
> # Warmup Iteration 4: 4898586.667 ns/op
> # Warmup Iteration 5: 4924111.607 ns/op
> # Warmup Iteration 6: 5391946.341 ns/op
> # Warmup Iteration 7: 5405700.980 ns/op
> # Warmup Iteration 8: 5138181.395 ns/op
> # Warmup Iteration 9: 4860863.436 ns/op
> # Warmup Iteration 10: 5076539.171 ns/op
> Iteration 1: 4693804.255 ns/op
> Iteration 2: 5147288.372 ns/op
> Iteration 3: 4809647.826 ns/op
> Iteration 4: 6034147.541 ns/op
> Iteration 5: 6675740.964 ns/op
> Iteration 6: 5156481.308 ns/op
> Iteration 7: 4678330.508 ns/op
> Iteration 8: 4775528.139 ns/op
> Iteration 9: 4800773.913 ns/op
> Iteration 10: 4623054.393 ns/op
> Iteration 11: 4816021.834 ns/op
> Iteration 12: 4597212.500 ns/op
> Iteration 13: 4691842.553 ns/op
> Iteration 14: 4720820.513 ns/op
> Iteration 15: 4662227.848 ns/op
> Iteration 16: 5261447.619 ns/op
> Iteration 17: 4840087.719 ns/op
> Iteration 18: 4726081.197 ns/op
> Iteration 19: 4673974.576 ns/op
> Iteration 20: 5129837.209 ns/op
> Iteration 21: 4724444.444 ns/op
> Iteration 22: 4740678.112 ns/op
> Iteration 23: 4710260.684 ns/op
> Iteration 24: 4789688.312 ns/op
> Iteration 25: 4589266.667 ns/op
> Iteration 26: 4675394.068 ns/op
> Iteration 27: 4594887.500 ns/op
> Iteration 28: 4642579.832 ns/op
> Iteration 29: 4666215.190 ns/op
> Iteration 30: 4916044.444 ns/op
> Iteration 31: 4734978.541 ns/op
> Iteration 32: 4527586.066 ns/op
> Iteration 33: 4695753.191 ns/op
> Iteration 34: 4675682.203 ns/op
> Iteration 35: 4598608.333 ns/op
> Iteration 36: 4616338.912 ns/op
> Iteration 37: 4722192.308 ns/op
> Iteration 38: 5385243.902 ns/op
> Iteration 39: 5183159.624 ns/op
> Iteration 40: 4802855.895 ns/op
> Iteration 41: 5393404.878 ns/op
> Iteration 42: 7052592.357 ns/op
> Iteration 43: 6087237.569 ns/op
> Iteration 44: 6829685.185 ns/op
> Iteration 45: 6105154.696 ns/op
> Iteration 46: 5331869.565 ns/op
> Iteration 47: 4765870.690 ns/op
> Iteration 48: 4635117.155 ns/op
> Iteration 49: 4882526.549 ns/op
> Iteration 50: 4883402.655 ns/op
> Iteration 51: 5018100.000 ns/op
> Iteration 52: 4899177.778 ns/op
> Iteration 53: 5568404.040 ns/op
> Iteration 54: 4912208.889 ns/op
> Iteration 55: 5073077.982 ns/op
> Iteration 56: 5441458.128 ns/op
> Iteration 57: 4964382.883 ns/op
> Iteration 58: 5379580.488 ns/op
> Iteration 59: 4925330.357 ns/op
> Iteration 60: 5111805.556 ns/op
> Iteration 61: 5226265.403 ns/op
> Iteration 62: 5304846.154 ns/op
> Iteration 63: 4920982.222 ns/op
> Iteration 64: 4905186.667 ns/op
> Iteration 65: 4970238.739 ns/op
> Iteration 66: 5002425.339 ns/op
> Iteration 67: 4842846.491 ns/op
> Iteration 68: 5543889.447 ns/op
> Iteration 69: 4828794.760 ns/op
> Iteration 70: 4809737.991 ns/op
> Iteration 71: 5151565.421 ns/op
> Iteration 72: 5077170.507 ns/op
> Iteration 73: 4987425.339 ns/op
> Iteration 74: 4888584.071 ns/op
> Iteration 75: 4980319.820 ns/op
> Iteration 76: 5075321.101 ns/op
> Iteration 77: 4719508.547 ns/op
> Iteration 78: 4932200.893 ns/op
> Iteration 79: 5046264.840 ns/op
> Iteration 80: 4835530.702 ns/op
> Iteration 81: 4855162.996 ns/op
> Iteration 82: 4814104.803 ns/op
> Iteration 83: 4650232.068 ns/op
> Iteration 84: 5211075.472 ns/op
> Iteration 85: 4970504.505 ns/op
> Iteration 86: 4776272.727 ns/op
> Iteration 87: 4929089.286 ns/op
> Iteration 88: 5063995.413 ns/op
> Iteration 89: 5059706.422 ns/op
> Iteration 90: 5042342.466 ns/op
> Iteration 91: 4760637.931 ns/op
> Iteration 92: 5009213.636 ns/op
> Iteration 93: 5985075.676 ns/op
> Iteration 94: 4735798.283 ns/op
> Iteration 95: 4893389.381 ns/op
> Iteration 96: 6055005.495 ns/op
> Iteration 97: 4684194.915 ns/op
> Iteration 98: 4649130.252 ns/op
> Iteration 99: 4619602.510 ns/op
> Iteration 100: 4593595.833 ns/op
>
>
> Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
> Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
> Confidence interval (99.9%): [4851488.689, 5171987.930]
>
>
> # Run complete. Total time: 00:04:29
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/
-----------------------------------------------------------
(Updated Dec. 5, 2014, 11:02 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Adding cluster fillup to better simulate real cluster conditions.
Repository: aurora
Description
-------
Added baseline benchmarks for a few static veto cases.
This is branched off of https://reviews.apache.org/r/28710
Diffs (updated)
-----
build.gradle 152ba631e2dd07f0306e58e355274e10a4128140
src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeEventSink.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
Diff: https://reviews.apache.org/r/28731/diff/
Testing
-------
Sample run on a local box:
```
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
# Run progress: 0.00% complete, ETA 00:03:40
# Fork: 1 of 1
# Warmup Iteration 1: 104725900.000 ns/op
# Warmup Iteration 2: 20439888.889 ns/op
# Warmup Iteration 3: 13612185.185 ns/op
# Warmup Iteration 4: 5856248.677 ns/op
# Warmup Iteration 5: 5148014.019 ns/op
# Warmup Iteration 6: 4972004.505 ns/op
# Warmup Iteration 7: 4595562.500 ns/op
# Warmup Iteration 8: 4639226.891 ns/op
# Warmup Iteration 9: 4639521.008 ns/op
# Warmup Iteration 10: 4660248.945 ns/op
Iteration 1: 4718478.632 ns/op
Iteration 2: 4800686.957 ns/op
Iteration 3: 4438253.012 ns/op
Iteration 4: 4935376.682 ns/op
Iteration 5: 5236895.735 ns/op
Iteration 6: 6439345.029 ns/op
Iteration 7: 4792708.696 ns/op
Iteration 8: 4579008.299 ns/op
Iteration 9: 4681224.576 ns/op
Iteration 10: 4651143.460 ns/op
Iteration 11: 4394079.681 ns/op
Iteration 12: 4390573.705 ns/op
Iteration 13: 4363304.348 ns/op
Iteration 14: 4419488.000 ns/op
Iteration 15: 4869718.062 ns/op
Iteration 16: 4600612.500 ns/op
Iteration 17: 4475906.883 ns/op
Iteration 18: 4710388.889 ns/op
Iteration 19: 6137000.000 ns/op
Iteration 20: 4467425.101 ns/op
Iteration 21: 4348877.953 ns/op
Iteration 22: 4666907.173 ns/op
Iteration 23: 4431325.301 ns/op
Iteration 24: 4463186.235 ns/op
Iteration 25: 4580074.689 ns/op
Iteration 26: 4467805.668 ns/op
Iteration 27: 4886960.177 ns/op
Iteration 28: 4858515.419 ns/op
Iteration 29: 4734025.751 ns/op
Iteration 30: 4718081.197 ns/op
Iteration 31: 4711688.034 ns/op
Iteration 32: 4462392.713 ns/op
Iteration 33: 4683597.458 ns/op
Iteration 34: 4755176.724 ns/op
Iteration 35: 4530278.689 ns/op
Iteration 36: 4392119.048 ns/op
Iteration 37: 4659722.689 ns/op
Iteration 38: 4805869.565 ns/op
Iteration 39: 4398573.705 ns/op
Iteration 40: 4572547.718 ns/op
Iteration 41: 4643478.992 ns/op
Iteration 42: 4545699.588 ns/op
Iteration 43: 4670266.949 ns/op
Iteration 44: 4506677.551 ns/op
Iteration 45: 4574149.378 ns/op
Iteration 46: 4743154.506 ns/op
Iteration 47: 4777606.061 ns/op
Iteration 48: 4481987.805 ns/op
Iteration 49: 4840921.053 ns/op
Iteration 50: 4637231.092 ns/op
Iteration 51: 4515286.885 ns/op
Iteration 52: 4372884.921 ns/op
Iteration 53: 4385003.968 ns/op
Iteration 54: 4619648.536 ns/op
Iteration 55: 4557822.314 ns/op
Iteration 56: 4594362.500 ns/op
Iteration 57: 7870319.149 ns/op
Iteration 58: 9764495.575 ns/op
Iteration 59: 10302028.037 ns/op
Iteration 60: 6043169.399 ns/op
Iteration 61: 5523472.362 ns/op
Iteration 62: 4513685.714 ns/op
Iteration 63: 4361806.324 ns/op
Iteration 64: 4388615.079 ns/op
Iteration 65: 4548264.463 ns/op
Iteration 66: 4710752.137 ns/op
Iteration 67: 5688128.866 ns/op
Iteration 68: 6109292.818 ns/op
Iteration 69: 5896914.439 ns/op
Iteration 70: 5986548.913 ns/op
Iteration 71: 6093618.785 ns/op
Iteration 72: 6051450.549 ns/op
Iteration 73: 6237870.056 ns/op
Iteration 74: 5954345.946 ns/op
Iteration 75: 6460748.538 ns/op
Iteration 76: 5929037.634 ns/op
Iteration 77: 6170748.603 ns/op
Iteration 78: 6037382.514 ns/op
Iteration 79: 6156776.536 ns/op
Iteration 80: 5987103.261 ns/op
Iteration 81: 5927080.645 ns/op
Iteration 82: 5914165.775 ns/op
Iteration 83: 6039852.459 ns/op
Iteration 84: 6156798.883 ns/op
Iteration 85: 5902329.787 ns/op
Iteration 86: 5731875.648 ns/op
Iteration 87: 6137622.222 ns/op
Iteration 88: 5926424.731 ns/op
Iteration 89: 5959324.324 ns/op
Iteration 90: 6269630.682 ns/op
Iteration 91: 5720062.176 ns/op
Iteration 92: 5938537.634 ns/op
Iteration 93: 4430361.446 ns/op
Iteration 94: 4498792.683 ns/op
Iteration 95: 5454004.950 ns/op
Iteration 96: 4516479.508 ns/op
Iteration 97: 4382456.349 ns/op
Iteration 98: 4410548.000 ns/op
Iteration 99: 4301657.588 ns/op
Iteration 100: 4438582.329 ns/op
Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
Confidence interval (99.9%): [4820809.970, 5509963.825]
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
# Warmup: 10 iterations, 1 s each
# Measurement: 100 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
# Run progress: 50.00% complete, ETA 00:02:14
# Fork: 1 of 1
# Warmup Iteration 1: 34499258.065 ns/op
# Warmup Iteration 2: 9519784.483 ns/op
# Warmup Iteration 3: 7017031.847 ns/op
# Warmup Iteration 4: 4898586.667 ns/op
# Warmup Iteration 5: 4924111.607 ns/op
# Warmup Iteration 6: 5391946.341 ns/op
# Warmup Iteration 7: 5405700.980 ns/op
# Warmup Iteration 8: 5138181.395 ns/op
# Warmup Iteration 9: 4860863.436 ns/op
# Warmup Iteration 10: 5076539.171 ns/op
Iteration 1: 4693804.255 ns/op
Iteration 2: 5147288.372 ns/op
Iteration 3: 4809647.826 ns/op
Iteration 4: 6034147.541 ns/op
Iteration 5: 6675740.964 ns/op
Iteration 6: 5156481.308 ns/op
Iteration 7: 4678330.508 ns/op
Iteration 8: 4775528.139 ns/op
Iteration 9: 4800773.913 ns/op
Iteration 10: 4623054.393 ns/op
Iteration 11: 4816021.834 ns/op
Iteration 12: 4597212.500 ns/op
Iteration 13: 4691842.553 ns/op
Iteration 14: 4720820.513 ns/op
Iteration 15: 4662227.848 ns/op
Iteration 16: 5261447.619 ns/op
Iteration 17: 4840087.719 ns/op
Iteration 18: 4726081.197 ns/op
Iteration 19: 4673974.576 ns/op
Iteration 20: 5129837.209 ns/op
Iteration 21: 4724444.444 ns/op
Iteration 22: 4740678.112 ns/op
Iteration 23: 4710260.684 ns/op
Iteration 24: 4789688.312 ns/op
Iteration 25: 4589266.667 ns/op
Iteration 26: 4675394.068 ns/op
Iteration 27: 4594887.500 ns/op
Iteration 28: 4642579.832 ns/op
Iteration 29: 4666215.190 ns/op
Iteration 30: 4916044.444 ns/op
Iteration 31: 4734978.541 ns/op
Iteration 32: 4527586.066 ns/op
Iteration 33: 4695753.191 ns/op
Iteration 34: 4675682.203 ns/op
Iteration 35: 4598608.333 ns/op
Iteration 36: 4616338.912 ns/op
Iteration 37: 4722192.308 ns/op
Iteration 38: 5385243.902 ns/op
Iteration 39: 5183159.624 ns/op
Iteration 40: 4802855.895 ns/op
Iteration 41: 5393404.878 ns/op
Iteration 42: 7052592.357 ns/op
Iteration 43: 6087237.569 ns/op
Iteration 44: 6829685.185 ns/op
Iteration 45: 6105154.696 ns/op
Iteration 46: 5331869.565 ns/op
Iteration 47: 4765870.690 ns/op
Iteration 48: 4635117.155 ns/op
Iteration 49: 4882526.549 ns/op
Iteration 50: 4883402.655 ns/op
Iteration 51: 5018100.000 ns/op
Iteration 52: 4899177.778 ns/op
Iteration 53: 5568404.040 ns/op
Iteration 54: 4912208.889 ns/op
Iteration 55: 5073077.982 ns/op
Iteration 56: 5441458.128 ns/op
Iteration 57: 4964382.883 ns/op
Iteration 58: 5379580.488 ns/op
Iteration 59: 4925330.357 ns/op
Iteration 60: 5111805.556 ns/op
Iteration 61: 5226265.403 ns/op
Iteration 62: 5304846.154 ns/op
Iteration 63: 4920982.222 ns/op
Iteration 64: 4905186.667 ns/op
Iteration 65: 4970238.739 ns/op
Iteration 66: 5002425.339 ns/op
Iteration 67: 4842846.491 ns/op
Iteration 68: 5543889.447 ns/op
Iteration 69: 4828794.760 ns/op
Iteration 70: 4809737.991 ns/op
Iteration 71: 5151565.421 ns/op
Iteration 72: 5077170.507 ns/op
Iteration 73: 4987425.339 ns/op
Iteration 74: 4888584.071 ns/op
Iteration 75: 4980319.820 ns/op
Iteration 76: 5075321.101 ns/op
Iteration 77: 4719508.547 ns/op
Iteration 78: 4932200.893 ns/op
Iteration 79: 5046264.840 ns/op
Iteration 80: 4835530.702 ns/op
Iteration 81: 4855162.996 ns/op
Iteration 82: 4814104.803 ns/op
Iteration 83: 4650232.068 ns/op
Iteration 84: 5211075.472 ns/op
Iteration 85: 4970504.505 ns/op
Iteration 86: 4776272.727 ns/op
Iteration 87: 4929089.286 ns/op
Iteration 88: 5063995.413 ns/op
Iteration 89: 5059706.422 ns/op
Iteration 90: 5042342.466 ns/op
Iteration 91: 4760637.931 ns/op
Iteration 92: 5009213.636 ns/op
Iteration 93: 5985075.676 ns/op
Iteration 94: 4735798.283 ns/op
Iteration 95: 4893389.381 ns/op
Iteration 96: 6055005.495 ns/op
Iteration 97: 4684194.915 ns/op
Iteration 98: 4649130.252 ns/op
Iteration 99: 4619602.510 ns/op
Iteration 100: 4593595.833 ns/op
Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
Confidence interval (99.9%): [4851488.689, 5171987.930]
# Run complete. Total time: 00:04:29
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
```
Thanks,
Maxim Khutornenko
Re: Review Request 28731: Implemented TaskScheduler benchmarks.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28731/#review63947
-----------------------------------------------------------
This patch does not apply cleanly on master (fd37f0d), do you need to rebase?
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 5, 2014, 1:27 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28731/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 1:27 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Added baseline benchmarks for a few static veto cases.
>
> This is branched off of https://reviews.apache.org/r/28710
>
>
> Diffs
> -----
>
> build.gradle fb729c5096108c535229e266fa9649f997e6da37
> src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeEventSink.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java PRE-CREATION
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
>
> Diff: https://reviews.apache.org/r/28731/diff/
>
>
> Testing
> -------
>
> Sample run on a local box:
> ```
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example
>
> # Run progress: 0.00% complete, ETA 00:03:40
> # Fork: 1 of 1
> # Warmup Iteration 1: 104725900.000 ns/op
> # Warmup Iteration 2: 20439888.889 ns/op
> # Warmup Iteration 3: 13612185.185 ns/op
> # Warmup Iteration 4: 5856248.677 ns/op
> # Warmup Iteration 5: 5148014.019 ns/op
> # Warmup Iteration 6: 4972004.505 ns/op
> # Warmup Iteration 7: 4595562.500 ns/op
> # Warmup Iteration 8: 4639226.891 ns/op
> # Warmup Iteration 9: 4639521.008 ns/op
> # Warmup Iteration 10: 4660248.945 ns/op
> Iteration 1: 4718478.632 ns/op
> Iteration 2: 4800686.957 ns/op
> Iteration 3: 4438253.012 ns/op
> Iteration 4: 4935376.682 ns/op
> Iteration 5: 5236895.735 ns/op
> Iteration 6: 6439345.029 ns/op
> Iteration 7: 4792708.696 ns/op
> Iteration 8: 4579008.299 ns/op
> Iteration 9: 4681224.576 ns/op
> Iteration 10: 4651143.460 ns/op
> Iteration 11: 4394079.681 ns/op
> Iteration 12: 4390573.705 ns/op
> Iteration 13: 4363304.348 ns/op
> Iteration 14: 4419488.000 ns/op
> Iteration 15: 4869718.062 ns/op
> Iteration 16: 4600612.500 ns/op
> Iteration 17: 4475906.883 ns/op
> Iteration 18: 4710388.889 ns/op
> Iteration 19: 6137000.000 ns/op
> Iteration 20: 4467425.101 ns/op
> Iteration 21: 4348877.953 ns/op
> Iteration 22: 4666907.173 ns/op
> Iteration 23: 4431325.301 ns/op
> Iteration 24: 4463186.235 ns/op
> Iteration 25: 4580074.689 ns/op
> Iteration 26: 4467805.668 ns/op
> Iteration 27: 4886960.177 ns/op
> Iteration 28: 4858515.419 ns/op
> Iteration 29: 4734025.751 ns/op
> Iteration 30: 4718081.197 ns/op
> Iteration 31: 4711688.034 ns/op
> Iteration 32: 4462392.713 ns/op
> Iteration 33: 4683597.458 ns/op
> Iteration 34: 4755176.724 ns/op
> Iteration 35: 4530278.689 ns/op
> Iteration 36: 4392119.048 ns/op
> Iteration 37: 4659722.689 ns/op
> Iteration 38: 4805869.565 ns/op
> Iteration 39: 4398573.705 ns/op
> Iteration 40: 4572547.718 ns/op
> Iteration 41: 4643478.992 ns/op
> Iteration 42: 4545699.588 ns/op
> Iteration 43: 4670266.949 ns/op
> Iteration 44: 4506677.551 ns/op
> Iteration 45: 4574149.378 ns/op
> Iteration 46: 4743154.506 ns/op
> Iteration 47: 4777606.061 ns/op
> Iteration 48: 4481987.805 ns/op
> Iteration 49: 4840921.053 ns/op
> Iteration 50: 4637231.092 ns/op
> Iteration 51: 4515286.885 ns/op
> Iteration 52: 4372884.921 ns/op
> Iteration 53: 4385003.968 ns/op
> Iteration 54: 4619648.536 ns/op
> Iteration 55: 4557822.314 ns/op
> Iteration 56: 4594362.500 ns/op
> Iteration 57: 7870319.149 ns/op
> Iteration 58: 9764495.575 ns/op
> Iteration 59: 10302028.037 ns/op
> Iteration 60: 6043169.399 ns/op
> Iteration 61: 5523472.362 ns/op
> Iteration 62: 4513685.714 ns/op
> Iteration 63: 4361806.324 ns/op
> Iteration 64: 4388615.079 ns/op
> Iteration 65: 4548264.463 ns/op
> Iteration 66: 4710752.137 ns/op
> Iteration 67: 5688128.866 ns/op
> Iteration 68: 6109292.818 ns/op
> Iteration 69: 5896914.439 ns/op
> Iteration 70: 5986548.913 ns/op
> Iteration 71: 6093618.785 ns/op
> Iteration 72: 6051450.549 ns/op
> Iteration 73: 6237870.056 ns/op
> Iteration 74: 5954345.946 ns/op
> Iteration 75: 6460748.538 ns/op
> Iteration 76: 5929037.634 ns/op
> Iteration 77: 6170748.603 ns/op
> Iteration 78: 6037382.514 ns/op
> Iteration 79: 6156776.536 ns/op
> Iteration 80: 5987103.261 ns/op
> Iteration 81: 5927080.645 ns/op
> Iteration 82: 5914165.775 ns/op
> Iteration 83: 6039852.459 ns/op
> Iteration 84: 6156798.883 ns/op
> Iteration 85: 5902329.787 ns/op
> Iteration 86: 5731875.648 ns/op
> Iteration 87: 6137622.222 ns/op
> Iteration 88: 5926424.731 ns/op
> Iteration 89: 5959324.324 ns/op
> Iteration 90: 6269630.682 ns/op
> Iteration 91: 5720062.176 ns/op
> Iteration 92: 5938537.634 ns/op
> Iteration 93: 4430361.446 ns/op
> Iteration 94: 4498792.683 ns/op
> Iteration 95: 5454004.950 ns/op
> Iteration 96: 4516479.508 ns/op
> Iteration 97: 4382456.349 ns/op
> Iteration 98: 4410548.000 ns/op
> Iteration 99: 4301657.588 ns/op
> Iteration 100: 4438582.329 ns/op
>
>
> Result: 5165386.898 ±(99.9%) 344576.928 ns/op [Average]
> Statistics: (min, avg, max) = (4301657.588, 5165386.898, 10302028.037), stdev = 1015992.919
> Confidence interval (99.9%): [4820809.970, 5509963.825]
>
>
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java
> # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 10 iterations, 1 s each
> # Measurement: 100 iterations, 1 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Average time, time/op
> # Benchmark: org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example
>
> # Run progress: 50.00% complete, ETA 00:02:14
> # Fork: 1 of 1
> # Warmup Iteration 1: 34499258.065 ns/op
> # Warmup Iteration 2: 9519784.483 ns/op
> # Warmup Iteration 3: 7017031.847 ns/op
> # Warmup Iteration 4: 4898586.667 ns/op
> # Warmup Iteration 5: 4924111.607 ns/op
> # Warmup Iteration 6: 5391946.341 ns/op
> # Warmup Iteration 7: 5405700.980 ns/op
> # Warmup Iteration 8: 5138181.395 ns/op
> # Warmup Iteration 9: 4860863.436 ns/op
> # Warmup Iteration 10: 5076539.171 ns/op
> Iteration 1: 4693804.255 ns/op
> Iteration 2: 5147288.372 ns/op
> Iteration 3: 4809647.826 ns/op
> Iteration 4: 6034147.541 ns/op
> Iteration 5: 6675740.964 ns/op
> Iteration 6: 5156481.308 ns/op
> Iteration 7: 4678330.508 ns/op
> Iteration 8: 4775528.139 ns/op
> Iteration 9: 4800773.913 ns/op
> Iteration 10: 4623054.393 ns/op
> Iteration 11: 4816021.834 ns/op
> Iteration 12: 4597212.500 ns/op
> Iteration 13: 4691842.553 ns/op
> Iteration 14: 4720820.513 ns/op
> Iteration 15: 4662227.848 ns/op
> Iteration 16: 5261447.619 ns/op
> Iteration 17: 4840087.719 ns/op
> Iteration 18: 4726081.197 ns/op
> Iteration 19: 4673974.576 ns/op
> Iteration 20: 5129837.209 ns/op
> Iteration 21: 4724444.444 ns/op
> Iteration 22: 4740678.112 ns/op
> Iteration 23: 4710260.684 ns/op
> Iteration 24: 4789688.312 ns/op
> Iteration 25: 4589266.667 ns/op
> Iteration 26: 4675394.068 ns/op
> Iteration 27: 4594887.500 ns/op
> Iteration 28: 4642579.832 ns/op
> Iteration 29: 4666215.190 ns/op
> Iteration 30: 4916044.444 ns/op
> Iteration 31: 4734978.541 ns/op
> Iteration 32: 4527586.066 ns/op
> Iteration 33: 4695753.191 ns/op
> Iteration 34: 4675682.203 ns/op
> Iteration 35: 4598608.333 ns/op
> Iteration 36: 4616338.912 ns/op
> Iteration 37: 4722192.308 ns/op
> Iteration 38: 5385243.902 ns/op
> Iteration 39: 5183159.624 ns/op
> Iteration 40: 4802855.895 ns/op
> Iteration 41: 5393404.878 ns/op
> Iteration 42: 7052592.357 ns/op
> Iteration 43: 6087237.569 ns/op
> Iteration 44: 6829685.185 ns/op
> Iteration 45: 6105154.696 ns/op
> Iteration 46: 5331869.565 ns/op
> Iteration 47: 4765870.690 ns/op
> Iteration 48: 4635117.155 ns/op
> Iteration 49: 4882526.549 ns/op
> Iteration 50: 4883402.655 ns/op
> Iteration 51: 5018100.000 ns/op
> Iteration 52: 4899177.778 ns/op
> Iteration 53: 5568404.040 ns/op
> Iteration 54: 4912208.889 ns/op
> Iteration 55: 5073077.982 ns/op
> Iteration 56: 5441458.128 ns/op
> Iteration 57: 4964382.883 ns/op
> Iteration 58: 5379580.488 ns/op
> Iteration 59: 4925330.357 ns/op
> Iteration 60: 5111805.556 ns/op
> Iteration 61: 5226265.403 ns/op
> Iteration 62: 5304846.154 ns/op
> Iteration 63: 4920982.222 ns/op
> Iteration 64: 4905186.667 ns/op
> Iteration 65: 4970238.739 ns/op
> Iteration 66: 5002425.339 ns/op
> Iteration 67: 4842846.491 ns/op
> Iteration 68: 5543889.447 ns/op
> Iteration 69: 4828794.760 ns/op
> Iteration 70: 4809737.991 ns/op
> Iteration 71: 5151565.421 ns/op
> Iteration 72: 5077170.507 ns/op
> Iteration 73: 4987425.339 ns/op
> Iteration 74: 4888584.071 ns/op
> Iteration 75: 4980319.820 ns/op
> Iteration 76: 5075321.101 ns/op
> Iteration 77: 4719508.547 ns/op
> Iteration 78: 4932200.893 ns/op
> Iteration 79: 5046264.840 ns/op
> Iteration 80: 4835530.702 ns/op
> Iteration 81: 4855162.996 ns/op
> Iteration 82: 4814104.803 ns/op
> Iteration 83: 4650232.068 ns/op
> Iteration 84: 5211075.472 ns/op
> Iteration 85: 4970504.505 ns/op
> Iteration 86: 4776272.727 ns/op
> Iteration 87: 4929089.286 ns/op
> Iteration 88: 5063995.413 ns/op
> Iteration 89: 5059706.422 ns/op
> Iteration 90: 5042342.466 ns/op
> Iteration 91: 4760637.931 ns/op
> Iteration 92: 5009213.636 ns/op
> Iteration 93: 5985075.676 ns/op
> Iteration 94: 4735798.283 ns/op
> Iteration 95: 4893389.381 ns/op
> Iteration 96: 6055005.495 ns/op
> Iteration 97: 4684194.915 ns/op
> Iteration 98: 4649130.252 ns/op
> Iteration 99: 4619602.510 ns/op
> Iteration 100: 4593595.833 ns/op
>
>
> Result: 5011738.309 ±(99.9%) 160249.620 ns/op [Average]
> Statistics: (min, avg, max) = (4527586.066, 5011738.309, 7052592.357), stdev = 472499.654
> Confidence interval (99.9%): [4851488.689, 5171987.930]
>
>
> # Run complete. Total time: 00:04:29
>
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 5165386.898 ± 344576.928 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 5011738.309 ± 160249.620 ns/op
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>