You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "qidaye (via GitHub)" <gi...@apache.org> on 2024/01/25 04:17:27 UTC
[PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
qidaye opened a new pull request, #30352:
URL: https://github.com/apache/doris/pull/30352
## Proposed changes
pick #30326
`null_bitmap` file is not considered in index compaction process. This will lead wrong query result when doc is contain `NULL` values.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909392935
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909437133
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909405445
TeamCity be ut coverage result:
Function Coverage: 37.83% (8032/21232)
Line Coverage: 29.53% (65445/221588)
Region Coverage: 28.95% (33606/116076)
Branch Coverage: 24.82% (17245/69488)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bc90a78782f95aec934baf3fdf9a5b0011e976f9_bc90a78782f95aec934baf3fdf9a5b0011e976f9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909387617
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909488244
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909368097
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.3 seconds inserted 10000000 Rows, about 469K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909425554
<details>
<summary>TPC-DS: <b>Total hot run time: 239109 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9, data reload: false
query1 897 385 378 378
query2 6542 2207 2070 2070
query3 6916 195 194 194
query4 21425 17994 18045 17994
query5 19735 6272 6265 6265
query6 295 216 229 216
query7 4182 300 301 300
query8 273 258 239 239
query9 3156 2720 2630 2630
query10 427 277 300 277
query11 11370 10694 10719 10694
query12 122 76 77 76
query13 5651 637 629 629
query14 17877 13306 13214 13214
query15 378 232 247 232
query16 6431 260 263 260
query17 3031 878 880 878
query18 2247 413 404 404
query19 205 147 148 147
query20 78 74 76 74
query21 185 103 97 97
query22 5213 5264 5115 5115
query23 32427 31895 31952 31895
query24 6908 6552 6480 6480
query25 529 421 449 421
query26 524 166 161 161
query27 1816 301 298 298
query28 6106 2227 2218 2218
query29 3000 2692 2670 2670
query30 247 157 164 157
query31 896 705 745 705
query32 65 60 59 59
query33 382 249 242 242
query34 826 478 491 478
query35 1091 941 879 879
query36 1507 1740 1568 1568
query37 94 64 62 62
query38 3087 2933 2940 2933
query39 1375 1307 1306 1306
query40 208 95 97 95
query41 33 32 33 32
query42 85 94 90 90
query43 680 679 572 572
query44 1124 717 712 712
query45 245 229 235 229
query46 1230 955 963 955
query47 1882 1845 2084 1845
query48 987 683 647 647
query49 610 353 379 353
query50 855 610 600 600
query51 5636 5501 5547 5501
query52 95 79 85 79
query53 444 326 307 307
query54 2461 2230 2197 2197
query55 79 76 83 76
query56 228 205 200 200
query57 1138 1025 1097 1025
query58 222 209 190 190
query59 3441 3351 3001 3001
query60 223 202 199 199
query61 84 83 87 83
query62 776 473 504 473
query63 478 329 327 327
query64 2013 1303 1306 1303
query65 3696 3645 3618 3618
query66 810 364 359 359
query67 15727 15396 15177 15177
query68 8860 665 673 665
query69 570 337 344 337
query70 1916 1639 1849 1639
query71 413 298 307 298
query72 4616 3327 3334 3327
query73 730 320 314 314
query74 6273 5950 5838 5838
query75 4912 3545 3576 3545
query76 5467 1151 1181 1151
query77 882 250 244 244
query78 32388 53652 48883 48883
query79 15573 622 611 611
query80 4698 391 377 377
query81 565 227 230 227
query82 1573 95 95 95
query83 333 129 128 128
query84 259 69 69 69
query85 1829 280 278 278
query86 447 393 393 393
query87 3226 3003 3006 3003
query88 7057 2302 2315 2302
query89 453 282 285 282
query90 2503 203 210 203
query91 151 125 117 117
query92 63 55 52 52
query93 6463 559 573 559
query94 1670 198 206 198
query95 1123 1082 1075 1075
query96 658 320 323 320
query97 6488 6370 6336 6336
query98 198 172 176 172
query99 3688 911 1002 911
Total cold run time: 349051 ms
Total hot run time: 239109 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909458824
<details>
<summary>TPC-H: <b>Total hot run time: 49945 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 09065f82c1cfade627ec29116b4f69e85441f359, data reload: false
------ Round 1 ----------------------------------
q1 17532 4362 4314 4314
q2 2032 154 141 141
q3 10269 1916 1963 1916
q4 10110 1249 1419 1249
q5 8550 3960 4005 3960
q6 234 123 126 123
q7 2045 1602 1581 1581
q8 9265 2728 2739 2728
q9 10841 11224 10460 10460
q10 8656 3542 3524 3524
q11 432 240 233 233
q12 480 298 311 298
q13 18373 3959 4055 3959
q14 352 328 320 320
q15 514 456 457 456
q16 702 589 592 589
q17 1124 986 1001 986
q18 7327 6830 6865 6830
q19 1688 1594 1488 1488
q20 535 303 307 303
q21 4443 4101 4146 4101
q22 486 386 406 386
Total cold run time: 115990 ms
Total hot run time: 49945 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4322 4298 4314 4298
q2 318 226 232 226
q3 4222 4176 4198 4176
q4 2762 2755 2749 2749
q5 7274 7260 7179 7179
q6 236 124 122 122
q7 3255 2876 2828 2828
q8 4422 4510 4530 4510
q9 17145 17051 17007 17007
q10 4243 4264 4311 4264
q11 758 658 675 658
q12 1040 847 855 847
q13 6704 3752 3748 3748
q14 464 433 421 421
q15 494 449 447 447
q16 784 706 723 706
q17 3898 3852 3820 3820
q18 8856 8832 8973 8832
q19 1705 1705 1657 1657
q20 2426 2139 2111 2111
q21 8571 8565 8393 8393
q22 1073 951 961 951
Total cold run time: 84972 ms
Total hot run time: 79950 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909360978
<details>
<summary>TPC-DS: <b>Total hot run time: 241169 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e, data reload: false
query1 909 391 388 388
query2 6531 2141 2062 2062
query3 6913 201 197 197
query4 21236 17979 17899 17899
query5 19722 6288 6214 6214
query6 281 216 238 216
query7 4152 315 309 309
query8 274 247 227 227
query9 3127 2717 2670 2670
query10 411 285 303 285
query11 11270 10734 10674 10674
query12 120 70 75 70
query13 5664 635 648 635
query14 17492 13344 13656 13344
query15 381 234 239 234
query16 6449 267 265 265
query17 3175 863 887 863
query18 2274 413 406 406
query19 211 149 157 149
query20 76 74 75 74
query21 185 99 94 94
query22 5347 5149 5145 5145
query23 32555 31971 31931 31931
query24 7025 6553 6497 6497
query25 528 432 450 432
query26 535 166 163 163
query27 1870 296 299 296
query28 6123 2255 2229 2229
query29 2963 2715 2761 2715
query30 237 158 162 158
query31 890 769 746 746
query32 63 60 56 56
query33 391 264 250 250
query34 851 473 498 473
query35 1114 886 909 886
query36 1804 1396 1521 1396
query37 96 58 62 58
query38 3107 2927 2889 2889
query39 1359 1320 1333 1320
query40 210 96 99 96
query41 35 32 31 31
query42 86 84 86 84
query43 567 615 587 587
query44 1116 724 728 724
query45 243 227 229 227
query46 1230 982 956 956
query47 1804 1642 1838 1642
query48 1001 705 682 682
query49 598 388 374 374
query50 862 607 582 582
query51 5687 5525 5541 5525
query52 88 87 84 84
query53 443 332 312 312
query54 2410 2242 2257 2242
query55 80 75 74 74
query56 225 215 203 203
query57 1179 1130 1093 1093
query58 212 203 185 185
query59 3430 3229 3244 3229
query60 214 187 196 187
query61 86 82 88 82
query62 778 453 468 453
query63 474 341 340 340
query64 2061 1361 1281 1281
query65 3659 3604 3639 3604
query66 827 369 367 367
query67 16469 16803 15469 15469
query68 7656 640 661 640
query69 568 330 364 330
query70 1973 1786 1660 1660
query71 392 294 311 294
query72 4600 3357 3363 3357
query73 714 318 315 315
query74 6353 5881 5809 5809
query75 4395 3480 3528 3480
query76 4652 1178 1204 1178
query77 528 253 269 253
query78 32165 50480 53478 50480
query79 14932 602 625 602
query80 4285 383 382 382
query81 566 235 227 227
query82 1401 103 107 103
query83 324 140 134 134
query84 261 67 68 67
query85 1695 277 275 275
query86 451 368 404 368
query87 3236 3063 3001 3001
query88 7035 2323 2296 2296
query89 394 274 290 274
query90 2503 206 207 206
query91 148 114 115 114
query92 67 52 50 50
query93 5155 607 615 607
query94 1668 208 210 208
query95 1090 1055 1068 1055
query96 639 332 324 324
query97 6434 6389 6409 6389
query98 187 173 170 170
query99 3666 967 891 891
Total cold run time: 343918 ms
Total hot run time: 241169 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909324408
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909430737
<details>
<summary>ClickBench: <b>Total hot run time: 30.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9, data reload: false
query1 0.02 0.02 0.02
query2 0.07 0.02 0.02
query3 0.24 0.04 0.05
query4 1.83 0.06 0.07
query5 0.52 0.52 0.51
query6 1.24 0.62 0.61
query7 0.01 0.01 0.01
query8 0.04 0.02 0.02
query9 0.52 0.49 0.49
query10 0.55 0.54 0.53
query11 0.12 0.08 0.09
query12 0.11 0.09 0.09
query13 0.62 0.62 0.60
query14 0.77 0.79 0.77
query15 0.77 0.75 0.76
query16 0.39 0.36 0.37
query17 1.02 1.00 1.00
query18 0.24 0.25 0.24
query19 1.94 1.85 1.89
query20 0.01 0.01 0.01
query21 15.51 0.57 0.57
query22 1.80 2.16 1.20
query23 16.97 0.91 0.87
query24 4.11 3.35 1.42
query25 2.12 0.16 0.14
query26 0.15 0.14 0.15
query27 0.14 0.15 0.14
query28 7.22 0.72 0.70
query29 12.75 2.14 2.27
query30 0.54 0.51 0.51
query31 2.82 0.39 0.38
query32 3.40 0.50 0.48
query33 3.10 3.04 3.11
query34 15.26 4.81 4.80
query35 4.84 4.83 4.84
query36 1.05 1.01 1.01
query37 0.06 0.04 0.05
query38 0.03 0.02 0.02
query39 0.02 0.02 0.02
query40 0.16 0.14 0.14
query41 0.06 0.01 0.02
query42 0.01 0.02 0.02
query43 0.03 0.01 0.02
Total cold run time: 103.18 s
Total hot run time: 30.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909479990
<details>
<summary>ClickBench: <b>Total hot run time: 30.82 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 09065f82c1cfade627ec29116b4f69e85441f359, data reload: false
query1 0.02 0.02 0.01
query2 0.05 0.03 0.02
query3 0.25 0.05 0.04
query4 1.82 0.07 0.07
query5 0.53 0.52 0.52
query6 1.26 0.61 0.61
query7 0.01 0.01 0.01
query8 0.04 0.02 0.02
query9 0.53 0.48 0.47
query10 0.54 0.51 0.54
query11 0.12 0.09 0.10
query12 0.11 0.09 0.10
query13 0.61 0.61 0.60
query14 0.78 0.78 0.79
query15 0.79 0.75 0.77
query16 0.36 0.36 0.37
query17 1.04 1.02 1.00
query18 0.21 0.28 0.28
query19 1.95 1.77 1.85
query20 0.01 0.01 0.01
query21 15.48 0.62 0.57
query22 1.69 1.68 2.18
query23 17.12 1.10 0.84
query24 5.68 1.02 2.90
query25 2.28 0.16 0.14
query26 0.14 0.12 0.15
query27 0.14 0.15 0.14
query28 6.15 0.73 0.71
query29 12.62 2.24 2.37
query30 0.57 0.53 0.52
query31 2.82 0.38 0.37
query32 3.44 0.50 0.50
query33 3.09 3.05 3.06
query34 15.28 4.79 4.81
query35 4.85 4.87 4.84
query36 1.08 1.02 1.02
query37 0.05 0.05 0.05
query38 0.04 0.02 0.02
query39 0.01 0.02 0.01
query40 0.16 0.15 0.14
query41 0.06 0.02 0.02
query42 0.02 0.01 0.01
query43 0.02 0.02 0.01
Total cold run time: 103.82 s
Total hot run time: 30.82 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909365663
<details>
<summary>ClickBench: <b>Total hot run time: 30.6 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e, data reload: false
query1 0.02 0.02 0.02
query2 0.06 0.02 0.02
query3 0.24 0.05 0.04
query4 1.82 0.07 0.07
query5 0.53 0.52 0.52
query6 1.26 0.62 0.62
query7 0.02 0.01 0.01
query8 0.03 0.03 0.02
query9 0.50 0.50 0.49
query10 0.54 0.53 0.54
query11 0.11 0.09 0.08
query12 0.11 0.09 0.09
query13 0.62 0.62 0.61
query14 0.78 0.77 0.78
query15 0.78 0.76 0.75
query16 0.36 0.40 0.35
query17 1.01 1.00 1.00
query18 0.21 0.25 0.26
query19 1.92 1.84 1.85
query20 0.02 0.01 0.02
query21 15.49 0.56 0.57
query22 1.84 1.93 1.32
query23 17.45 0.95 0.95
query24 6.67 4.32 0.89
query25 2.38 0.16 0.16
query26 0.15 0.14 0.14
query27 0.15 0.16 0.14
query28 4.32 0.71 0.73
query29 12.78 2.42 2.30
query30 0.60 0.54 0.50
query31 2.82 0.39 0.38
query32 3.41 0.50 0.49
query33 3.08 3.09 3.10
query34 15.26 4.79 4.80
query35 4.85 4.83 4.85
query36 1.03 1.01 1.01
query37 0.06 0.05 0.05
query38 0.03 0.02 0.02
query39 0.01 0.02 0.02
query40 0.16 0.13 0.14
query41 0.07 0.01 0.02
query42 0.01 0.01 0.01
query43 0.02 0.01 0.02
Total cold run time: 103.58 s
Total hot run time: 30.6 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909474318
<details>
<summary>TPC-DS: <b>Total hot run time: 235540 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 09065f82c1cfade627ec29116b4f69e85441f359, data reload: false
query1 919 388 379 379
query2 6512 2203 1943 1943
query3 6908 203 197 197
query4 20183 18068 17880 17880
query5 19719 6265 6227 6227
query6 276 212 223 212
query7 4144 292 291 291
query8 262 252 247 247
query9 3141 2719 2659 2659
query10 403 286 285 285
query11 11374 10763 10714 10714
query12 122 75 72 72
query13 5657 665 636 636
query14 17658 13266 13480 13266
query15 400 231 246 231
query16 6444 266 267 266
query17 3218 853 877 853
query18 2266 400 408 400
query19 207 144 152 144
query20 74 76 75 75
query21 187 97 91 91
query22 5340 5199 5098 5098
query23 32415 31793 31992 31793
query24 6917 6560 6476 6476
query25 510 431 426 426
query26 521 159 157 157
query27 1875 291 293 291
query28 6124 2241 2224 2224
query29 2818 2754 2741 2741
query30 237 162 160 160
query31 889 725 791 725
query32 65 59 57 57
query33 393 249 252 249
query34 849 461 484 461
query35 1114 891 909 891
query36 1474 1555 1461 1461
query37 87 58 56 56
query38 3096 2917 2890 2890
query39 1365 1325 1321 1321
query40 199 98 91 91
query41 37 35 31 31
query42 88 87 90 87
query43 651 598 576 576
query44 1116 729 720 720
query45 241 227 230 227
query46 1234 993 968 968
query47 1775 1717 1754 1717
query48 1013 693 685 685
query49 621 379 368 368
query50 868 593 595 593
query51 5631 5531 5471 5471
query52 87 77 79 77
query53 450 331 321 321
query54 2448 2226 2279 2226
query55 82 78 72 72
query56 211 199 200 199
query57 1104 1100 1004 1004
query58 198 190 208 190
query59 3512 3089 3155 3089
query60 204 186 192 186
query61 83 84 80 80
query62 805 521 489 489
query63 479 341 336 336
query64 1998 1240 1258 1240
query65 3644 3636 3637 3636
query66 828 371 358 358
query67 17435 15777 15024 15024
query68 5155 634 653 634
query69 536 350 342 342
query70 1768 1625 2047 1625
query71 374 302 314 302
query72 4246 3270 3311 3270
query73 706 317 319 317
query74 6386 5898 5922 5898
query75 4079 3525 3450 3450
query76 2978 1218 1219 1218
query77 421 250 246 246
query78 30815 49004 46012 46012
query79 14364 616 614 614
query80 4582 375 374 374
query81 561 233 228 228
query82 467 101 92 92
query83 332 134 128 128
query84 257 66 67 66
query85 2533 272 271 271
query86 487 448 387 387
query87 3288 2996 3027 2996
query88 6185 2331 2307 2307
query89 370 302 272 272
query90 2484 207 209 207
query91 162 117 112 112
query92 66 52 51 51
query93 2737 590 584 584
query94 1862 208 207 207
query95 1097 1061 1052 1052
query96 629 332 324 324
query97 6504 6304 6403 6304
query98 185 178 166 166
query99 3891 878 941 878
Total cold run time: 333712 ms
Total hot run time: 235540 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909582433
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909329491
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909369797
TeamCity be ut coverage result:
Function Coverage: 37.83% (8033/21232)
Line Coverage: 29.54% (65453/221579)
Region Coverage: 28.95% (33607/116071)
Branch Coverage: 24.82% (17245/69484)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1265a89924ddd8bd97b892943416b4355f79e91e_1265a89924ddd8bd97b892943416b4355f79e91e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909442873
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909411724
<details>
<summary>TPC-H: <b>Total hot run time: 49824 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9, data reload: false
------ Round 1 ----------------------------------
q1 17611 4433 4402 4402
q2 2056 146 138 138
q3 10450 1884 1893 1884
q4 10392 1269 1327 1269
q5 8369 4302 3968 3968
q6 231 122 122 122
q7 2011 1620 1583 1583
q8 9533 2704 2710 2704
q9 13161 10492 10386 10386
q10 8654 3525 3480 3480
q11 407 235 247 235
q12 461 298 308 298
q13 18360 3984 4075 3984
q14 351 332 339 332
q15 497 454 463 454
q16 679 598 591 591
q17 1123 918 942 918
q18 7227 6792 6915 6792
q19 1686 1567 1482 1482
q20 527 308 315 308
q21 4497 4107 4100 4100
q22 498 399 394 394
Total cold run time: 118781 ms
Total hot run time: 49824 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4282 4315 4303 4303
q2 321 226 221 221
q3 4149 4193 4171 4171
q4 2763 2740 2761 2740
q5 7306 7252 7270 7252
q6 233 116 118 116
q7 3275 2896 2856 2856
q8 4408 4540 4524 4524
q9 17288 17131 17058 17058
q10 4278 4397 4436 4397
q11 795 722 722 722
q12 1074 922 932 922
q13 6452 3872 3926 3872
q14 477 444 454 444
q15 508 474 470 470
q16 807 765 799 765
q17 3847 3927 3879 3879
q18 8887 8842 8833 8833
q19 1752 1682 1672 1672
q20 2434 2132 2137 2132
q21 8601 8549 8554 8549
q22 1077 931 961 931
Total cold run time: 85014 ms
Total hot run time: 80829 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909466035
TeamCity be ut coverage result:
Function Coverage: 37.84% (8034/21232)
Line Coverage: 29.55% (65470/221588)
Region Coverage: 28.96% (33620/116076)
Branch Coverage: 24.83% (17254/69488)
Coverage Report: http://coverage.selectdb-in.cc/coverage/09065f82c1cfade627ec29116b4f69e85441f359_09065f82c1cfade627ec29116b4f69e85441f359/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909349623
<details>
<summary>TPC-H: <b>Total hot run time: 50059 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e, data reload: false
------ Round 1 ----------------------------------
q1 17967 4443 4353 4353
q2 2025 150 143 143
q3 10278 1919 1910 1910
q4 10097 1226 1316 1226
q5 8552 3974 4017 3974
q6 229 122 120 120
q7 2028 1598 1578 1578
q8 9313 2735 2745 2735
q9 11157 10557 10418 10418
q10 8683 3531 3529 3529
q11 416 233 235 233
q12 457 301 299 299
q13 18358 3955 4035 3955
q14 347 326 330 326
q15 500 446 457 446
q16 709 597 589 589
q17 1138 975 966 966
q18 7331 7070 6928 6928
q19 1687 1561 1559 1559
q20 550 303 302 302
q21 4446 4080 4121 4080
q22 498 396 390 390
Total cold run time: 116766 ms
Total hot run time: 50059 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4362 4286 4282 4282
q2 324 230 222 222
q3 4203 4171 4166 4166
q4 2779 2762 2799 2762
q5 7284 7240 7191 7191
q6 240 120 120 120
q7 3225 2861 2840 2840
q8 4373 4496 4466 4466
q9 17184 17069 16963 16963
q10 4250 4298 4298 4298
q11 749 676 692 676
q12 1028 851 852 851
q13 6804 3708 3764 3708
q14 450 424 414 414
q15 508 445 456 445
q16 765 701 716 701
q17 3921 3874 3889 3874
q18 8838 8725 8745 8725
q19 1722 1699 1649 1649
q20 2374 2130 2138 2130
q21 8526 8468 8435 8435
q22 1041 935 953 935
Total cold run time: 84950 ms
Total hot run time: 79853 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909433681
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9 with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](index compaction) Support merge null_bitmap during index compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30352:
URL: https://github.com/apache/doris/pull/30352#issuecomment-1909483317
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 09065f82c1cfade627ec29116b4f69e85441f359 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.3 seconds inserted 10000000 Rows, about 469K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org