You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/03 15:33:38 UTC
[GitHub] [arrow] pitrou edited a comment on pull request #8823: ARROW-10790: [C++] Improve ChunkedArray and Table sort_indices performance
pitrou edited a comment on pull request #8823:
URL: https://github.com/apache/arrow/pull/8823#issuecomment-738034200
Micro-benchmarks (minus changes that are within +/-20%):
```
benchmark baseline contender change % counters
15 TableSortIndicesInt64Narrow/1048576/100/1/32/min_time:1.000 2.139m items/sec 30.607m items/sec 1330.734 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
12 ChunkedArraySortIndicesInt64Wide/32768/1/min_time:1.000 430.595 MiB/sec 4.860 GiB/sec 1055.753 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 19528, 'null_percent': 100.0}
4 TableSortIndicesInt64Narrow/1048576/0/1/32/min_time:1.000 2.781m items/sec 31.260m items/sec 1024.004 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
8 ChunkedArraySortIndicesInt64Narrow/32768/1/min_time:1.000 470.814 MiB/sec 4.875 GiB/sec 960.380 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 21121, 'null_percent': 100.0}
66 TableSortIndicesInt64Wide/1048576/100/1/32/min_time:1.000 1.253m items/sec 5.764m items/sec 360.030 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
27 ChunkedArraySortIndicesInt64Narrow/1048576/100/min_time:1.000 126.998 MiB/sec 477.777 MiB/sec 276.207 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/1048576/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 176, 'null_percent': 1.0}
11 TableSortIndicesInt64Wide/1048576/0/1/32/min_time:1.000 1.812m items/sec 5.806m items/sec 220.367 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
16 ChunkedArraySortIndicesInt64Narrow/8388608/100/min_time:1.000 107.544 MiB/sec 309.121 MiB/sec 187.436 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/8388608/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 19, 'null_percent': 1.0}
39 ChunkedArraySortIndicesInt64Narrow/32768/2/min_time:1.000 109.268 MiB/sec 259.067 MiB/sec 137.094 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/2/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4863, 'null_percent': 50.0}
37 ChunkedArraySortIndicesInt64Wide/32768/2/min_time:1.000 96.473 MiB/sec 215.956 MiB/sec 123.851 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/2/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4310, 'null_percent': 50.0}
1 ChunkedArraySortIndicesInt64Narrow/32768/10/min_time:1.000 73.125 MiB/sec 150.009 MiB/sec 105.140 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/10/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3269, 'null_percent': 10.0}
19 ChunkedArraySortIndicesInt64Wide/32768/10/min_time:1.000 58.555 MiB/sec 116.894 MiB/sec 99.630 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/10/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2670, 'null_percent': 10.0}
40 ChunkedArraySortIndicesInt64Narrow/32768/100/min_time:1.000 69.008 MiB/sec 136.962 MiB/sec 98.473 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3087, 'null_percent': 1.0}
48 ChunkedArraySortIndicesInt64Wide/32768/100/min_time:1.000 54.876 MiB/sec 105.713 MiB/sec 92.641 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2448, 'null_percent': 1.0}
79 ChunkedArraySortIndicesInt64Wide/1048576/100/min_time:1.000 45.457 MiB/sec 74.826 MiB/sec 64.608 {'run_name': 'ChunkedArraySortIndicesInt64Wide/1048576/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 64, 'null_percent': 1.0}
24 TableSortIndicesInt64Narrow/1048576/100/2/32/min_time:1.000 1.090m items/sec 1.721m items/sec 57.997 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
42 TableSortIndicesInt64Narrow/1048576/0/2/32/min_time:1.000 1.146m items/sec 1.784m items/sec 55.686 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
52 ChunkedArraySortIndicesInt64Narrow/32768/0/min_time:1.000 93.226 MiB/sec 144.796 MiB/sec 55.317 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/0/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4187, 'null_percent': 0.0}
46 TableSortIndicesInt64Wide/1048576/0/2/32/min_time:1.000 1.965m items/sec 3.003m items/sec 52.864 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
47 ChunkedArraySortIndicesInt64Narrow/32768/10000/min_time:1.000 94.026 MiB/sec 141.270 MiB/sec 50.246 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/10000/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4187, 'null_percent': 0.01}
0 TableSortIndicesInt64Wide/1048576/100/2/32/min_time:1.000 1.988m items/sec 2.966m items/sec 49.175 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
60 TableSortIndicesInt64Wide/1048576/0/8/32/min_time:1.000 2.023m items/sec 3.011m items/sec 48.839 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
43 TableSortIndicesInt64Narrow/1048576/0/16/32/min_time:1.000 804.026k items/sec 1.196m items/sec 48.721 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
35 TableSortIndicesInt64Wide/1048576/100/16/32/min_time:1.000 2.091m items/sec 3.078m items/sec 47.203 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
18 ChunkedArraySortIndicesInt64Wide/8388608/100/min_time:1.000 38.925 MiB/sec 57.272 MiB/sec 47.135 {'run_name': 'ChunkedArraySortIndicesInt64Wide/8388608/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 7, 'null_percent': 1.0}
33 TableSortIndicesInt64Narrow/1048576/100/1/4/min_time:1.000 41.094m items/sec 60.431m items/sec 47.056 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/1/4/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 56}
77 TableSortIndicesInt64Narrow/1048576/0/8/32/min_time:1.000 819.170k items/sec 1.200m items/sec 46.485 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
78 TableSortIndicesInt64Wide/1048576/100/8/32/min_time:1.000 2.042m items/sec 2.976m items/sec 45.745 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
53 TableSortIndicesInt64Narrow/1048576/100/8/32/min_time:1.000 788.089k items/sec 1.115m items/sec 41.420 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
54 TableSortIndicesInt64Narrow/1048576/100/16/32/min_time:1.000 792.230k items/sec 1.120m items/sec 41.413 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
56 TableSortIndicesInt64Wide/1048576/0/16/32/min_time:1.000 2.142m items/sec 3.024m items/sec 41.191 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
44 ChunkedArraySortIndicesInt64Wide/32768/0/min_time:1.000 78.245 MiB/sec 108.182 MiB/sec 38.260 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/0/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3503, 'null_percent': 0.0}
9 ChunkedArraySortIndicesInt64Wide/32768/10000/min_time:1.000 77.572 MiB/sec 107.186 MiB/sec 38.176 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/10000/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3477, 'null_percent': 0.01}
[ ... ]
57 TableSortIndicesInt64Wide/1048576/0/8/1/min_time:1.000 5.772m items/sec 4.334m items/sec -24.919 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/8/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
45 TableSortIndicesInt64Wide/1048576/0/16/1/min_time:1.000 5.801m items/sec 4.345m items/sec -25.100 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/16/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
62 TableSortIndicesInt64Wide/1048576/100/2/1/min_time:1.000 5.697m items/sec 4.257m items/sec -25.273 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/2/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
34 TableSortIndicesInt64Wide/1048576/0/2/1/min_time:1.000 5.817m items/sec 4.317m items/sec -25.789 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/2/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
38 TableSortIndicesInt64Wide/1048576/100/16/1/min_time:1.000 5.778m items/sec 4.273m items/sec -26.035 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/16/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
75 TableSortIndicesInt64Wide/1048576/100/8/1/min_time:1.000 5.739m items/sec 4.239m items/sec -26.142 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/8/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org