You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2018/12/17 23:57:33 UTC
[GitHub] clintropolis opened a new pull request #6750: update roaring from
0.5.18 to 0.7.30
clintropolis opened a new pull request #6750: update roaring from 0.5.18 to 0.7.30
URL: https://github.com/apache/incubator-druid/pull/6750
There seem to be some performance improvements according to our `BitmapIterationBenchmark`. I read these results as iteration speed is largely unchanged, intersections and unions are faster in some cases, more notable when doing a lot of them. Intersection improvements seem mainly for sparsely set bitmaps, union improvements look to be across the board no matter the density.
Benchmark parameters are:
* n: number of operations (intersect, union) to perform
* prob: fraction of set bits in the bitmaps to iterate
* size: row count
and result sets are grouped by benchmark, with the first set of results when run with 0.5.18 and second set run with 0.7.30.
```
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.constructAndIter roaring N/A 0.0 1000000 avgt 25 125.671 ± 1.755 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.001 1000000 avgt 25 47787.364 ± 2063.837 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.1 1000000 avgt 25 1734515.411 ± 27482.425 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.5 1000000 avgt 25 6588106.966 ± 102822.821 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.99 1000000 avgt 25 12663113.733 ± 250103.338 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 1.0 1000000 avgt 25 12539263.269 ± 99253.206 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.0 1000000 avgt 25 121.804 ± 0.520 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.001 1000000 avgt 25 15054.748 ± 76.885 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.1 1000000 avgt 25 1754862.371 ± 6571.034 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.5 1000000 avgt 25 6937182.670 ± 40085.478 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 0.99 1000000 avgt 25 12959777.360 ± 48462.008 ns/op
BitmapIterationBenchmark.constructAndIter roaring N/A 1.0 1000000 avgt 25 13072005.032 ± 60920.976 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.0 1000000 avgt 25 32.938 ± 0.318 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.001 1000000 avgt 25 354768.196 ± 3049.721 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.1 1000000 avgt 25 438125.783 ± 3041.058 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.5 1000000 avgt 25 1341080.164 ± 14919.854 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.99 1000000 avgt 25 2372885.498 ± 25801.699 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 1.0 1000000 avgt 25 2349813.953 ± 46231.371 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.0 1000000 avgt 25 31.809 ± 0.245 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.001 1000000 avgt 25 331907.708 ± 1058.425 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.1 1000000 avgt 25 414956.564 ± 2287.364 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.5 1000000 avgt 25 1209335.039 ± 3482.080 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 0.99 1000000 avgt 25 2132848.548 ± 40491.084 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 2 1.0 1000000 avgt 25 2062765.131 ± 24929.425 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.0 1000000 avgt 25 40.967 ± 0.576 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.001 1000000 avgt 25 947893.519 ± 7485.068 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.1 1000000 avgt 25 720661.269 ± 10120.735 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.5 1000000 avgt 25 1410395.578 ± 12518.931 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.99 1000000 avgt 25 2498582.393 ± 23933.799 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 1.0 1000000 avgt 25 2478900.062 ± 56722.279 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.0 1000000 avgt 25 39.321 ± 0.180 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.001 1000000 avgt 25 415745.718 ± 2908.972 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.1 1000000 avgt 25 664133.812 ± 2454.430 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.5 1000000 avgt 25 1349669.646 ± 8874.467 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 0.99 1000000 avgt 25 2313150.242 ± 9862.472 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 10 1.0 1000000 avgt 25 2283366.761 ± 28321.616 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.0 1000000 avgt 25 132.252 ± 1.008 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.001 1000000 avgt 25 6172672.344 ± 53334.246 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.1 1000000 avgt 25 3651389.978 ± 34583.211 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.5 1000000 avgt 25 2828300.616 ± 30629.728 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.99 1000000 avgt 25 3866094.326 ± 40798.994 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 1.0 1000000 avgt 25 3843606.490 ± 58477.359 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.0 1000000 avgt 25 131.064 ± 0.569 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.001 1000000 avgt 25 4569199.142 ± 18282.734 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.1 1000000 avgt 25 3543556.679 ± 8817.585 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.5 1000000 avgt 25 2699918.284 ± 21189.257 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 0.99 1000000 avgt 25 3647781.995 ± 34562.551 ns/op
BitmapIterationBenchmark.intersectionAndIter roaring 100 1.0 1000000 avgt 25 3673044.196 ± 24049.630 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.iter roaring N/A 0.0 1000000 avgt 25 5.118 ± 0.039 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.001 1000000 avgt 25 3988.876 ± 29.215 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.1 1000000 avgt 25 419068.202 ± 8202.659 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.5 1000000 avgt 25 1310704.917 ± 21344.194 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.99 1000000 avgt 25 2356631.440 ± 42472.373 ns/op
BitmapIterationBenchmark.iter roaring N/A 1.0 1000000 avgt 25 2439613.934 ± 31652.651 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.0 1000000 avgt 25 4.997 ± 0.028 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.001 1000000 avgt 25 3895.576 ± 19.780 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.1 1000000 avgt 25 377123.898 ± 1458.387 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.5 1000000 avgt 25 1179947.970 ± 4328.062 ns/op
BitmapIterationBenchmark.iter roaring N/A 0.99 1000000 avgt 25 2179637.402 ± 8768.373 ns/op
BitmapIterationBenchmark.iter roaring N/A 1.0 1000000 avgt 25 2105264.731 ± 40574.565 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.unionAndIter roaring 2 0.0 1000000 avgt 25 32.926 ± 0.603 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.001 1000000 avgt 25 286128.194 ± 5812.936 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.1 1000000 avgt 25 946200.957 ± 16904.540 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.5 1000000 avgt 25 1808854.660 ± 33900.619 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.99 1000000 avgt 25 2369361.732 ± 58004.664 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 1.0 1000000 avgt 25 2376736.402 ± 57244.307 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.0 1000000 avgt 25 30.791 ± 0.148 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.001 1000000 avgt 25 288214.743 ± 2526.239 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.1 1000000 avgt 25 923202.840 ± 5579.664 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.5 1000000 avgt 25 1631831.813 ± 14506.614 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 0.99 1000000 avgt 25 2146568.780 ± 34116.435 ns/op
BitmapIterationBenchmark.unionAndIter roaring 2 1.0 1000000 avgt 25 2099013.263 ± 8448.716 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.unionAndIter roaring 10 0.0 1000000 avgt 25 42.881 ± 0.602 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.001 1000000 avgt 25 1990921.445 ± 38230.766 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.1 1000000 avgt 25 2132513.097 ± 81851.405 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.5 1000000 avgt 25 2334131.847 ± 36425.494 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.99 1000000 avgt 25 2384036.005 ± 62638.755 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 1.0 1000000 avgt 25 2395463.863 ± 63834.946 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.0 1000000 avgt 25 40.400 ± 0.414 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.001 1000000 avgt 25 1828705.202 ± 9140.295 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.1 1000000 avgt 25 1902512.804 ± 18344.209 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.5 1000000 avgt 25 2152630.488 ± 18157.638 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 0.99 1000000 avgt 25 2168880.995 ± 22036.277 ns/op
BitmapIterationBenchmark.unionAndIter roaring 10 1.0 1000000 avgt 25 2122416.570 ± 12085.826 ns/op
Benchmark (bitmapAlgo) (n) (prob) (size) Mode Cnt Score Error Units
BitmapIterationBenchmark.unionAndIter roaring 100 0.0 1000000 avgt 25 172.170 ± 2.549 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.001 1000000 avgt 25 4269440.344 ± 88460.265 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.1 1000000 avgt 25 4378616.293 ± 99312.551 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.5 1000000 avgt 25 4436662.011 ± 102928.713 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.99 1000000 avgt 25 4407234.736 ± 104321.848 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 1.0 1000000 avgt 25 4368125.841 ± 103162.143 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.0 1000000 avgt 25 169.291 ± 1.100 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.001 1000000 avgt 25 3176993.184 ± 29959.552 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.1 1000000 avgt 25 3263884.302 ± 13550.251 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.5 1000000 avgt 25 3322039.882 ± 22866.858 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 0.99 1000000 avgt 25 3429006.404 ± 160510.293 ns/op
BitmapIterationBenchmark.unionAndIter roaring 100 1.0 1000000 avgt 25 3429996.682 ± 65763.717 ns/op
```
For a sanity check, also tested locally by writing some segments with 0.5.18 and ensuring they were still chill to read with 0.7.30.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org