You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/03 14:37:59 UTC
[GitHub] [arrow] pitrou commented on pull request #8823: [WIP] ARROW-10790: [C++] Improve ChunkedArray and Table sort_indices performance
pitrou commented on pull request #8823:
URL: https://github.com/apache/arrow/pull/8823#issuecomment-738034200
Micro-benchmarks (without changes that are within +/-20%):
```
benchmark baseline contender change % counters
70 TableSortIndicesInt64Narrow/1048576/100/1/32/min_time:1.000 2.103m items/sec 29.517m items/sec 1303.373 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
42 TableSortIndicesInt64Narrow/1048576/0/1/32/min_time:1.000 2.632m items/sec 32.005m items/sec 1115.857 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
4 TableSortIndicesInt64Wide/1048576/100/1/32/min_time:1.000 1.310m items/sec 5.699m items/sec 334.947 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
45 ChunkedArraySortIndicesInt64Narrow/1048576/100/min_time:1.000 120.277 MiB/sec 457.146 MiB/sec 280.079 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/1048576/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 169, 'null_percent': 1.0}
74 TableSortIndicesInt64Wide/1048576/0/1/32/min_time:1.000 1.818m items/sec 5.713m items/sec 214.210 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/1/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
43 ChunkedArraySortIndicesInt64Narrow/8388608/100/min_time:1.000 100.449 MiB/sec 296.414 MiB/sec 195.090 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/8388608/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 18, 'null_percent': 1.0}
36 ChunkedArraySortIndicesInt64Wide/32768/1/min_time:1.000 436.969 MiB/sec 937.543 MiB/sec 114.556 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 19706, 'null_percent': 100.0}
67 ChunkedArraySortIndicesInt64Narrow/32768/2/min_time:1.000 103.001 MiB/sec 220.776 MiB/sec 114.344 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/2/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4599, 'null_percent': 50.0}
33 ChunkedArraySortIndicesInt64Narrow/32768/1/min_time:1.000 440.082 MiB/sec 936.393 MiB/sec 112.777 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 19740, 'null_percent': 100.0}
20 ChunkedArraySortIndicesInt64Narrow/32768/10/min_time:1.000 68.612 MiB/sec 141.755 MiB/sec 106.604 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/10/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3079, 'null_percent': 10.0}
26 ChunkedArraySortIndicesInt64Narrow/32768/100/min_time:1.000 64.937 MiB/sec 132.435 MiB/sec 103.942 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2904, 'null_percent': 1.0}
52 ChunkedArraySortIndicesInt64Wide/32768/2/min_time:1.000 92.572 MiB/sec 186.731 MiB/sec 101.715 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/2/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 4144, 'null_percent': 50.0}
27 ChunkedArraySortIndicesInt64Wide/32768/10/min_time:1.000 56.622 MiB/sec 108.760 MiB/sec 92.080 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/10/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2538, 'null_percent': 10.0}
60 ChunkedArraySortIndicesInt64Wide/32768/100/min_time:1.000 51.971 MiB/sec 99.685 MiB/sec 91.809 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 2313, 'null_percent': 1.0}
0 ChunkedArraySortIndicesInt64Wide/1048576/100/min_time:1.000 43.418 MiB/sec 71.135 MiB/sec 63.836 {'run_name': 'ChunkedArraySortIndicesInt64Wide/1048576/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 61, 'null_percent': 1.0}
72 TableSortIndicesInt64Narrow/1048576/0/2/32/min_time:1.000 1.034m items/sec 1.681m items/sec 62.568 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
11 ChunkedArraySortIndicesInt64Narrow/32768/0/min_time:1.000 88.918 MiB/sec 138.677 MiB/sec 55.960 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/0/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3976, 'null_percent': 0.0}
30 TableSortIndicesInt64Narrow/1048576/100/2/32/min_time:1.000 1.045m items/sec 1.618m items/sec 54.872 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
23 ChunkedArraySortIndicesInt64Narrow/32768/10000/min_time:1.000 88.881 MiB/sec 136.954 MiB/sec 54.086 {'run_name': 'ChunkedArraySortIndicesInt64Narrow/32768/10000/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3952, 'null_percent': 0.01}
41 ChunkedArraySortIndicesInt64Wide/8388608/100/min_time:1.000 36.545 MiB/sec 54.546 MiB/sec 49.259 {'run_name': 'ChunkedArraySortIndicesInt64Wide/8388608/100/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 6, 'null_percent': 1.0}
49 TableSortIndicesInt64Narrow/1048576/0/8/32/min_time:1.000 756.824k items/sec 1.110m items/sec 46.676 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
40 TableSortIndicesInt64Narrow/1048576/0/16/32/min_time:1.000 769.325k items/sec 1.111m items/sec 44.463 {'run_name': 'TableSortIndicesInt64Narrow/1048576/0/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
22 TableSortIndicesInt64Narrow/1048576/100/1/4/min_time:1.000 41.401m items/sec 59.226m items/sec 43.055 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/1/4/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 55}
64 TableSortIndicesInt64Wide/1048576/0/2/32/min_time:1.000 2.037m items/sec 2.896m items/sec 42.210 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
28 TableSortIndicesInt64Narrow/1048576/100/16/32/min_time:1.000 734.690k items/sec 1.037m items/sec 41.134 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
57 TableSortIndicesInt64Narrow/1048576/100/8/32/min_time:1.000 740.323k items/sec 1.040m items/sec 40.479 {'run_name': 'TableSortIndicesInt64Narrow/1048576/100/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
69 TableSortIndicesInt64Wide/1048576/100/2/32/min_time:1.000 2.052m items/sec 2.860m items/sec 39.332 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/2/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
56 TableSortIndicesInt64Wide/1048576/100/16/32/min_time:1.000 2.069m items/sec 2.874m items/sec 38.875 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
71 TableSortIndicesInt64Wide/1048576/100/8/32/min_time:1.000 2.067m items/sec 2.867m items/sec 38.699 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
63 ChunkedArraySortIndicesInt64Wide/32768/0/min_time:1.000 73.852 MiB/sec 102.317 MiB/sec 38.544 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/0/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3287, 'null_percent': 0.0}
16 ChunkedArraySortIndicesInt64Wide/32768/10000/min_time:1.000 73.234 MiB/sec 100.801 MiB/sec 37.642 {'run_name': 'ChunkedArraySortIndicesInt64Wide/32768/10000/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3273, 'null_percent': 0.01}
15 TableSortIndicesInt64Wide/1048576/0/8/32/min_time:1.000 2.115m items/sec 2.906m items/sec 37.416 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/8/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
46 TableSortIndicesInt64Wide/1048576/0/16/32/min_time:1.000 2.124m items/sec 2.882m items/sec 35.644 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/16/32/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
[ ... ]
76 TableSortIndicesInt64Wide/1048576/100/16/1/min_time:1.000 5.701m items/sec 4.260m items/sec -25.267 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/16/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
21 TableSortIndicesInt64Wide/1048576/0/16/1/min_time:1.000 6.023m items/sec 4.350m items/sec -27.777 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/16/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
50 TableSortIndicesInt64Wide/1048576/100/8/1/min_time:1.000 6.003m items/sec 4.277m items/sec -28.752 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/8/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
51 TableSortIndicesInt64Wide/1048576/0/8/1/min_time:1.000 6.120m items/sec 4.359m items/sec -28.776 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/8/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
39 TableSortIndicesInt64Wide/1048576/0/2/1/min_time:1.000 6.122m items/sec 4.358m items/sec -28.818 {'run_name': 'TableSortIndicesInt64Wide/1048576/0/2/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
6 TableSortIndicesInt64Wide/1048576/100/2/1/min_time:1.000 6.027m items/sec 4.285m items/sec -28.901 {'run_name': 'TableSortIndicesInt64Wide/1048576/100/2/1/min_time:1.000', 'run_type': 'iteration', 'repetitions': 0, 'repetition_index': 0, 'threads': 1, 'iterations': 8}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org