You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/14 01:46:52 UTC
[GitHub] [arrow] westonpace commented on pull request #12150: ARROW-15332: [C++] Add new cases and fix issues in IPC read/write benchmark
westonpace commented on pull request #12150:
URL: https://github.com/apache/arrow/pull/12150#issuecomment-1012674728
Sample Results:
```
2022-01-13T15:39:41-10:00
Running ./release/arrow-ipc-read-write-benchmark
Run on (16 X 5100 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x8)
L1 Instruction 32 KiB (x8)
L2 Unified 256 KiB (x8)
L3 Unified 16384 KiB (x1)
Load Average: 1.03, 1.04, 0.97
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
-------------------------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
-------------------------------------------------------------------------------------------------------------------------
ReadBuffer/num_cols:1/is_partial:0/real_time 2339 ns 2339 ns 299520 bytes_per_second=417.528G/s
ReadBuffer/num_cols:8/is_partial:0/real_time 5370 ns 5370 ns 129234 bytes_per_second=181.846G/s
ReadBuffer/num_cols:64/is_partial:0/real_time 31651 ns 31651 ns 22064 bytes_per_second=30.854G/s
ReadBuffer/num_cols:512/is_partial:0/real_time 281760 ns 281756 ns 2489 bytes_per_second=3.46593G/s
ReadBuffer/num_cols:4096/is_partial:0/real_time 2248611 ns 2248569 ns 311 bytes_per_second=444.719M/s
ReadBuffer/num_cols:1/is_partial:1/real_time 34997 ns 34996 ns 20005 bytes_per_second=27.9038G/s
ReadBuffer/num_cols:8/is_partial:1/real_time 10516 ns 10515 ns 65872 bytes_per_second=11.6082G/s
ReadBuffer/num_cols:64/is_partial:1/real_time 31562 ns 31561 ns 22004 bytes_per_second=3.86766G/s
ReadBuffer/num_cols:512/is_partial:1/real_time 214759 ns 214757 ns 3237 bytes_per_second=582.047M/s
ReadBuffer/num_cols:4096/is_partial:1/real_time 1671177 ns 1670968 ns 419 bytes_per_second=74.7976M/s
ReadBufferAsync/num_cols:1/is_partial:0/real_time 3855 ns 3855 ns 180777 bytes_per_second=253.309G/s
ReadBufferAsync/num_cols:8/is_partial:0/real_time 6970 ns 6970 ns 100649 bytes_per_second=140.104G/s
ReadBufferAsync/num_cols:64/is_partial:0/real_time 33747 ns 33746 ns 20721 bytes_per_second=28.9377G/s
ReadBufferAsync/num_cols:512/is_partial:0/real_time 284691 ns 284690 ns 2453 bytes_per_second=3.43026G/s
ReadBufferAsync/num_cols:4096/is_partial:0/real_time 2246321 ns 2246299 ns 312 bytes_per_second=445.172M/s
ReadBufferAsync/num_cols:1/is_partial:1/real_time 4442 ns 4442 ns 156962 bytes_per_second=219.86G/s
ReadBufferAsync/num_cols:8/is_partial:1/real_time 6304 ns 6304 ns 111473 bytes_per_second=19.3636G/s
ReadBufferAsync/num_cols:64/is_partial:1/real_time 23053 ns 23051 ns 29330 bytes_per_second=5.29527G/s
ReadBufferAsync/num_cols:512/is_partial:1/real_time 178310 ns 178299 ns 3981 bytes_per_second=701.027M/s
ReadBufferAsync/num_cols:4096/is_partial:1/real_time 1399513 ns 1399473 ns 498 bytes_per_second=89.3168M/s
ReadCachedFile/num_cols:1/is_partial:0/real_time 1215565 ns 1174926 ns 522 bytes_per_second=12.8541G/s
ReadCachedFile/num_cols:8/is_partial:0/real_time 1280821 ns 1238925 ns 432 bytes_per_second=12.1992G/s
ReadCachedFile/num_cols:64/is_partial:0/real_time 1493422 ns 1451935 ns 384 bytes_per_second=10.4625G/s
ReadCachedFile/num_cols:512/is_partial:0/real_time 4437113 ns 4374682 ns 165 bytes_per_second=3.52143G/s
ReadCachedFile/num_cols:4096/is_partial:0/real_time 26650682 ns 26419503 ns 25 bytes_per_second=600.36M/s
ReadCachedFile/num_cols:1/is_partial:1/real_time 1230819 ns 1189216 ns 450 bytes_per_second=12.6948G/s
ReadCachedFile/num_cols:8/is_partial:1/real_time 184017 ns 176714 ns 3274 bytes_per_second=10.6138G/s
ReadCachedFile/num_cols:64/is_partial:1/real_time 426931 ns 413851 ns 1542 bytes_per_second=4.5748G/s
ReadCachedFile/num_cols:512/is_partial:1/real_time 2278713 ns 2217240 ns 280 bytes_per_second=877.688M/s
ReadCachedFile/num_cols:4096/is_partial:1/real_time 18832777 ns 18614163 ns 39 bytes_per_second=106.198M/s
ReadCachedFileAsync/num_cols:1/is_partial:0/real_time 1410031 ns 116183 ns 410 bytes_per_second=11.0813G/s
ReadCachedFileAsync/num_cols:8/is_partial:0/real_time 1459886 ns 130222 ns 483 bytes_per_second=10.7029G/s
ReadCachedFileAsync/num_cols:64/is_partial:0/real_time 2330443 ns 321086 ns 221 bytes_per_second=6.70473G/s
ReadCachedFileAsync/num_cols:512/is_partial:0/real_time 5482006 ns 1257601 ns 103 bytes_per_second=2.85023G/s
ReadCachedFileAsync/num_cols:4096/is_partial:0/real_time 58510720 ns 20036778 ns 10 bytes_per_second=273.454M/s
ReadCachedFileAsync/num_cols:1/is_partial:1/real_time 1372293 ns 110795 ns 485 bytes_per_second=11.3861G/s
ReadCachedFileAsync/num_cols:8/is_partial:1/real_time 1405059 ns 118576 ns 511 bytes_per_second=1.39007G/s
ReadCachedFileAsync/num_cols:64/is_partial:1/real_time 1636863 ns 167263 ns 430 bytes_per_second=1.19321G/s
ReadCachedFileAsync/num_cols:512/is_partial:1/real_time 4227871 ns 643349 ns 259 bytes_per_second=473.051M/s
ReadCachedFileAsync/num_cols:4096/is_partial:1/real_time 15536425 ns 2950475 ns 44 bytes_per_second=128.73M/s
ReadUncachedFile/num_cols:1/is_partial:0/real_time 90759059 ns 21259292 ns 7 bytes_per_second=176.291M/s
ReadUncachedFile/num_cols:8/is_partial:0/real_time 869888780 ns 176945198 ns 1 bytes_per_second=147.145M/s
ReadUncachedFile/num_cols:64/is_partial:0/real_time 7444635509 ns 1366674351 ns 1 bytes_per_second=137.549M/s
ReadUncachedFile/num_cols:1/is_partial:1/real_time 62928170 ns 18097667 ns 9 bytes_per_second=254.258M/s
ReadUncachedFile/num_cols:8/is_partial:1/real_time 133338397 ns 33394170 ns 4 bytes_per_second=119.995M/s
ReadUncachedFile/num_cols:64/is_partial:1/real_time 1467858164 ns 284143075 ns 1 bytes_per_second=87.2019M/s
ReadUncachedFileAsync/num_cols:1/is_partial:0/real_time 74365294 ns 4432433 ns 8 bytes_per_second=215.154M/s
ReadUncachedFileAsync/num_cols:8/is_partial:0/real_time 1112630678 ns 15459211 ns 1 bytes_per_second=115.043M/s
ReadUncachedFileAsync/num_cols:64/is_partial:0/real_time 8005232159 ns 87626234 ns 1 bytes_per_second=127.916M/s
ReadUncachedFileAsync/num_cols:1/is_partial:1/real_time 82149012 ns 4770464 ns 8 bytes_per_second=194.768M/s
ReadUncachedFileAsync/num_cols:8/is_partial:1/real_time 999669229 ns 25252003 ns 1 bytes_per_second=16.0053M/s
ReadUncachedFileAsync/num_cols:64/is_partial:1/real_time 8953151862 ns 122078755 ns 1 bytes_per_second=14.2966M/s
ReadMmapCachedFile/num_cols:1/is_partial:0/real_time 125726 ns 125677 ns 4359 bytes_per_second=124.278G/s
ReadMmapCachedFile/num_cols:8/is_partial:0/real_time 152793 ns 152693 ns 4078 bytes_per_second=102.263G/s
ReadMmapCachedFile/num_cols:64/is_partial:0/real_time 448895 ns 448890 ns 1544 bytes_per_second=34.8077G/s
ReadMmapCachedFile/num_cols:512/is_partial:0/real_time 3245646 ns 3245563 ns 201 bytes_per_second=4.81414G/s
ReadMmapCachedFile/num_cols:4096/is_partial:0/real_time 24613857 ns 24613192 ns 26 bytes_per_second=650.04M/s
ReadMmapCachedFile/num_cols:1/is_partial:1/real_time 1404416 ns 1403848 ns 429 bytes_per_second=11.1256G/s
ReadMmapCachedFile/num_cols:8/is_partial:1/real_time 258900 ns 258655 ns 2438 bytes_per_second=7.54394G/s
ReadMmapCachedFile/num_cols:64/is_partial:1/real_time 637583 ns 637050 ns 1070 bytes_per_second=3.06333G/s
ReadMmapCachedFile/num_cols:512/is_partial:1/real_time 2433011 ns 2430223 ns 260 bytes_per_second=822.027M/s
ReadMmapCachedFile/num_cols:4096/is_partial:1/real_time 13873093 ns 13871727 ns 49 bytes_per_second=144.164M/s
ReadMmapCachedFileAsync/num_cols:1/is_partial:0/real_time 140850 ns 140795 ns 4303 bytes_per_second=110.933G/s
ReadMmapCachedFileAsync/num_cols:8/is_partial:0/real_time 179309 ns 179210 ns 3469 bytes_per_second=87.14G/s
ReadMmapCachedFileAsync/num_cols:64/is_partial:0/real_time 448913 ns 448908 ns 1446 bytes_per_second=34.8063G/s
ReadMmapCachedFileAsync/num_cols:512/is_partial:0/real_time 3254736 ns 3254735 ns 201 bytes_per_second=4.8007G/s
ReadMmapCachedFileAsync/num_cols:4096/is_partial:0/real_time 24568415 ns 24568065 ns 29 bytes_per_second=651.243M/s
ReadMmapCachedFileAsync/num_cols:1/is_partial:1/real_time 147544 ns 147505 ns 4174 bytes_per_second=105.901G/s
ReadMmapCachedFileAsync/num_cols:8/is_partial:1/real_time 156600 ns 156560 ns 3922 bytes_per_second=12.4721G/s
ReadMmapCachedFileAsync/num_cols:64/is_partial:1/real_time 279874 ns 279869 ns 2286 bytes_per_second=6.9786G/s
ReadMmapCachedFileAsync/num_cols:512/is_partial:1/real_time 1294640 ns 1294599 ns 485 bytes_per_second=1.50862G/s
ReadMmapCachedFileAsync/num_cols:4096/is_partial:1/real_time 9290458 ns 9290186 ns 61 bytes_per_second=215.275M/s
ReadMmapUncachedFile/num_cols:1/is_partial:0/real_time 14218441 ns 4163041 ns 37 bytes_per_second=1125.3M/s
ReadMmapUncachedFile/num_cols:8/is_partial:0/real_time 15186534 ns 5445685 ns 34 bytes_per_second=8.23098G/s
ReadMmapUncachedFile/num_cols:64/is_partial:0/real_time 18517013 ns 6723617 ns 40 bytes_per_second=54.0044G/s
ReadMmapUncachedFile/num_cols:1/is_partial:1/real_time 199373218 ns 25452922 ns 6 bytes_per_second=80.2515M/s
ReadMmapUncachedFile/num_cols:8/is_partial:1/real_time 89239301 ns 20759803 ns 6 bytes_per_second=179.293M/s
ReadMmapUncachedFile/num_cols:64/is_partial:1/real_time 1248248168 ns 341291147 ns 1 bytes_per_second=102.544M/s
ReadMmapUncachedFileAsync/num_cols:1/is_partial:0/real_time 18912041 ns 4831881 ns 30 bytes_per_second=846.022M/s
ReadMmapUncachedFileAsync/num_cols:8/is_partial:0/real_time 18659801 ns 5592895 ns 29 bytes_per_second=6.69889G/s
ReadMmapUncachedFileAsync/num_cols:64/is_partial:0/real_time 22052427 ns 8448925 ns 25 bytes_per_second=45.3465G/s
ReadMmapUncachedFileAsync/num_cols:1/is_partial:1/real_time 19152585 ns 4715533 ns 32 bytes_per_second=835.396M/s
ReadMmapUncachedFileAsync/num_cols:8/is_partial:1/real_time 31351166 ns 5621547 ns 24 bytes_per_second=510.348M/s
ReadMmapUncachedFileAsync/num_cols:64/is_partial:1/real_time 23942101 ns 9873209 ns 24 bytes_per_second=5.22093G/s
ReadCompressedBuffer/num_cols:1/is_partial:0/real_time 23104550 ns 1218454 ns 30 bytes_per_second=692.504M/s
ReadCompressedBuffer/num_cols:8/is_partial:0/real_time 4595707 ns 740912 ns 152 bytes_per_second=3.39991G/s
ReadCompressedBuffer/num_cols:64/is_partial:0/real_time 6477766 ns 5240771 ns 109 bytes_per_second=2.4121G/s
ReadCompressedBuffer/num_cols:512/is_partial:0/real_time 33545970 ns 31936587 ns 21 bytes_per_second=476.957M/s
ReadCompressedBuffer/num_cols:4096/is_partial:0/real_time 266192000 ns 253035894 ns 3 bytes_per_second=60.107M/s
ReadCompressedBuffer/num_cols:1/is_partial:1/real_time 58344718 ns 2052251 ns 10 bytes_per_second=274.232M/s
ReadCompressedBuffer/num_cols:8/is_partial:1/real_time 3345093 ns 227327 ns 212 bytes_per_second=597.891M/s
ReadCompressedBuffer/num_cols:64/is_partial:1/real_time 1353268 ns 974815 ns 521 bytes_per_second=1.44327G/s
ReadCompressedBuffer/num_cols:512/is_partial:1/real_time 5785294 ns 5541711 ns 117 bytes_per_second=345.704M/s
ReadCompressedBuffer/num_cols:4096/is_partial:1/real_time 44613066 ns 43177604 ns 16 bytes_per_second=44.8299M/s
ReadCompressedBufferAsync/num_cols:1/is_partial:0/real_time 62489273 ns 1824188 ns 16 bytes_per_second=256.044M/s
ReadCompressedBufferAsync/num_cols:8/is_partial:0/real_time 4611022 ns 774268 ns 150 bytes_per_second=3.38862G/s
ReadCompressedBufferAsync/num_cols:64/is_partial:0/real_time 6482862 ns 5258856 ns 108 bytes_per_second=2.4102G/s
ReadCompressedBufferAsync/num_cols:512/is_partial:0/real_time 33535205 ns 31985320 ns 21 bytes_per_second=477.111M/s
ReadCompressedBufferAsync/num_cols:4096/is_partial:0/real_time 261546151 ns 248884692 ns 3 bytes_per_second=61.1747M/s
ReadCompressedBufferAsync/num_cols:1/is_partial:1/real_time 54144682 ns 1612839 ns 10 bytes_per_second=295.505M/s
ReadCompressedBufferAsync/num_cols:8/is_partial:1/real_time 3189016 ns 219889 ns 214 bytes_per_second=627.153M/s
ReadCompressedBufferAsync/num_cols:64/is_partial:1/real_time 1249434 ns 871746 ns 549 bytes_per_second=1.56321G/s
ReadCompressedBufferAsync/num_cols:512/is_partial:1/real_time 5307896 ns 5074152 ns 131 bytes_per_second=376.797M/s
ReadCompressedBufferAsync/num_cols:4096/is_partial:1/real_time 39080839 ns 37670339 ns 18 bytes_per_second=51.176M/s
WriteRecordBatch/1/real_time 34772 ns 34772 ns 20300 bytes_per_second=28.0844G/s
WriteRecordBatch/4/real_time 34535 ns 34535 ns 20351 bytes_per_second=28.2771G/s
WriteRecordBatch/16/real_time 36643 ns 36642 ns 19124 bytes_per_second=26.6505G/s
WriteRecordBatch/64/real_time 45013 ns 45012 ns 15689 bytes_per_second=21.695G/s
WriteRecordBatch/256/real_time 78685 ns 78684 ns 8838 bytes_per_second=12.411G/s
WriteRecordBatch/1024/real_time 201633 ns 201631 ns 3471 bytes_per_second=4.84328G/s
WriteRecordBatch/4096/real_time 694521 ns 694524 ns 1005 bytes_per_second=1.4061G/s
WriteRecordBatch/8192/real_time 1575717 ns 1575656 ns 442 bytes_per_second=634.632M/s
ReadRecordBatch/1/real_time 979 ns 979 ns 701667 bytes_per_second=997.706G/s
ReadRecordBatch/4/real_time 1750 ns 1750 ns 395005 bytes_per_second=557.954G/s
ReadRecordBatch/16/real_time 5132 ns 5132 ns 135412 bytes_per_second=190.286G/s
ReadRecordBatch/64/real_time 18851 ns 18851 ns 37344 bytes_per_second=51.8031G/s
ReadRecordBatch/256/real_time 86498 ns 86496 ns 7923 bytes_per_second=11.29G/s
ReadRecordBatch/1024/real_time 351658 ns 351652 ns 1989 bytes_per_second=2.77702G/s
ReadRecordBatch/4096/real_time 1395187 ns 1395182 ns 498 bytes_per_second=716.75M/s
ReadRecordBatch/8192/real_time 3144721 ns 3144621 ns 222 bytes_per_second=317.993M/s
ReadStream/1/real_time 1910 ns 1910 ns 364013 bytes_per_second=511.201G/s
ReadStream/4/real_time 3464 ns 3464 ns 200990 bytes_per_second=281.943G/s
ReadStream/16/real_time 9450 ns 9450 ns 74020 bytes_per_second=103.339G/s
ReadStream/64/real_time 34052 ns 34050 ns 20488 bytes_per_second=28.6789G/s
ReadStream/256/real_time 157756 ns 157756 ns 4436 bytes_per_second=6.19032G/s
ReadStream/1024/real_time 619067 ns 619065 ns 1122 bytes_per_second=1.57748G/s
ReadStream/4096/real_time 2416601 ns 2416567 ns 289 bytes_per_second=413.804M/s
ReadStream/8192/real_time 4807772 ns 4807614 ns 147 bytes_per_second=207.997M/s
DecodeStream/1/real_time 2148 ns 2148 ns 325906 bytes_per_second=454.604G/s
DecodeStream/4/real_time 3680 ns 3680 ns 189757 bytes_per_second=265.384G/s
DecodeStream/16/real_time 9498 ns 9498 ns 73771 bytes_per_second=102.818G/s
DecodeStream/64/real_time 33636 ns 33635 ns 20995 bytes_per_second=29.0335G/s
DecodeStream/256/real_time 149850 ns 149850 ns 4676 bytes_per_second=6.51692G/s
DecodeStream/1024/real_time 592956 ns 592940 ns 1173 bytes_per_second=1.64694G/s
DecodeStream/4096/real_time 2362735 ns 2362619 ns 297 bytes_per_second=423.238M/s
DecodeStream/8192/real_time 5194284 ns 5193784 ns 134 bytes_per_second=192.519M/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org