You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/09/25 11:36:23 UTC
[PR] [improvement](binlog) filter dropped indexes #41246 [doris]
w41ter opened a new pull request, #41299:
URL: https://github.com/apache/doris/pull/41299
cherry pick from #41246
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373836057
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373941159
<details>
<summary>TPC-DS: <b>Total hot run time: 213869 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9f092c511e3cd8c318011c4211bcbb97df219bb8, data reload: false
query1 928 393 436 393
query2 6556 2257 2128 2128
query3 6923 202 208 202
query4 22874 21797 21455 21455
query5 19738 6534 6535 6534
query6 285 221 235 221
query7 4326 300 305 300
query8 261 270 251 251
query9 3060 2637 2582 2582
query10 468 292 310 292
query11 15406 15011 14766 14766
query12 121 76 72 72
query13 1019 445 435 435
query14 17488 13709 13349 13349
query15 387 230 225 225
query16 6493 291 262 262
query17 1762 921 930 921
query18 895 312 317 312
query19 207 150 151 150
query20 104 110 107 107
query21 187 102 93 93
query22 5198 5025 4967 4967
query23 34106 33376 33542 33376
query24 7912 6266 6313 6266
query25 526 442 431 431
query26 1272 166 171 166
query27 2528 296 293 293
query28 6113 2247 2220 2220
query29 2924 2808 2803 2803
query30 240 170 164 164
query31 955 710 738 710
query32 68 61 59 59
query33 450 268 270 268
query34 859 490 480 480
query35 1124 920 878 878
query36 1360 1238 1159 1159
query37 172 63 60 60
query38 3088 2891 2898 2891
query39 1372 1309 1318 1309
query40 309 95 98 95
query41 48 38 40 38
query42 93 99 81 81
query43 640 570 563 563
query44 1145 704 710 704
query45 248 227 232 227
query46 1235 960 974 960
query47 2289 1777 1804 1777
query48 534 411 416 411
query49 637 380 378 378
query50 844 612 635 612
query51 4806 4606 4640 4606
query52 79 75 73 73
query53 235 187 190 187
query54 2660 2510 2518 2510
query55 81 86 83 83
query56 237 207 208 207
query57 1343 1066 1096 1066
query58 223 215 222 215
query59 3469 3170 3414 3170
query60 212 204 191 191
query61 97 93 95 93
query62 785 496 456 456
query63 200 177 173 173
query64 3571 1594 1479 1479
query65 3600 3557 3515 3515
query66 767 411 398 398
query67 15783 17427 17798 17427
query68 9104 669 640 640
query69 477 258 276 258
query70 1556 1482 1383 1383
query71 359 312 303 303
query72 6828 4784 4823 4784
query73 754 321 331 321
query74 6262 5812 5816 5812
query75 4452 3651 3763 3651
query76 4671 1120 1191 1120
query77 645 253 247 247
query78 12564 11548 11800 11548
query79 8694 654 652 652
query80 1764 388 392 388
query81 497 229 236 229
query82 1699 102 96 96
query83 166 133 132 132
query84 261 71 71 71
query85 990 318 316 316
query86 329 322 293 293
query87 3245 3066 3005 3005
query88 4795 2300 2286 2286
query89 483 284 299 284
query90 1846 213 212 212
query91 158 122 125 122
query92 60 54 51 51
query93 6534 539 588 539
query94 772 215 207 207
query95 1929 1949 1982 1949
query96 655 326 321 321
query97 6466 6448 6358 6358
query98 231 206 213 206
query99 2955 840 871 840
Total cold run time: 318126 ms
Total hot run time: 213869 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373911062
<details>
<summary>TPC-H: <b>Total hot run time: 49272 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9f092c511e3cd8c318011c4211bcbb97df219bb8, data reload: false
------ Round 1 ----------------------------------
q1 17842 4442 4356 4356
q2 2090 154 141 141
q3 10469 1930 1929 1929
q4 10321 1243 1329 1243
q5 8619 3884 3928 3884
q6 236 124 128 124
q7 2031 1604 1585 1585
q8 9350 2768 2728 2728
q9 10222 9902 9801 9801
q10 8678 3572 3511 3511
q11 430 252 249 249
q12 474 302 303 302
q13 18364 3952 4079 3952
q14 358 328 323 323
q15 509 454 457 454
q16 550 464 450 450
q17 1154 1001 943 943
q18 7209 6934 6953 6934
q19 1705 1597 1544 1544
q20 542 320 294 294
q21 4496 4207 4139 4139
q22 488 386 394 386
Total cold run time: 116137 ms
Total hot run time: 49272 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4364 4324 4318 4318
q2 322 224 219 219
q3 4170 4131 4130 4130
q4 2790 2768 2756 2756
q5 7164 7158 7110 7110
q6 240 117 122 117
q7 3220 2846 2826 2826
q8 4419 4431 4457 4431
q9 13660 13651 13583 13583
q10 4248 4248 4259 4248
q11 748 726 668 668
q12 1029 855 845 845
q13 6541 3771 3761 3761
q14 442 419 436 419
q15 506 459 456 456
q16 620 586 564 564
q17 3811 3842 3852 3842
q18 8854 8729 8814 8729
q19 1733 1682 1689 1682
q20 2391 2103 2116 2103
q21 8394 8532 8419 8419
q22 1064 917 971 917
Total cold run time: 80730 ms
Total hot run time: 76143 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373852385
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373959305
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 9f092c511e3cd8c318011c4211bcbb97df219bb8 with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.0 seconds inserted 10000000 Rows, about 476K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373952524
<details>
<summary>ClickBench: <b>Total hot run time: 30.86 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9f092c511e3cd8c318011c4211bcbb97df219bb8, data reload: false
query1 0.02 0.02 0.02
query2 0.07 0.03 0.03
query3 0.26 0.05 0.04
query4 1.79 0.06 0.06
query5 0.55 0.52 0.52
query6 1.24 0.62 0.62
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.52 0.50 0.48
query10 0.54 0.53 0.54
query11 0.12 0.09 0.09
query12 0.12 0.09 0.08
query13 0.62 0.63 0.61
query14 0.80 0.78 0.79
query15 0.79 0.76 0.75
query16 0.36 0.36 0.37
query17 1.02 0.98 1.02
query18 0.23 0.23 0.26
query19 1.90 1.82 1.86
query20 0.01 0.01 0.01
query21 15.47 0.56 0.56
query22 1.85 1.76 1.83
query23 17.34 1.01 1.02
query24 6.48 0.81 1.18
query25 0.40 0.10 0.04
query26 0.64 0.15 0.15
query27 0.04 0.03 0.04
query28 6.72 0.74 0.78
query29 12.61 2.40 2.37
query30 0.54 0.53 0.54
query31 2.80 0.38 0.37
query32 3.39 0.50 0.50
query33 3.09 3.12 3.08
query34 15.24 4.80 4.79
query35 4.87 4.84 4.83
query36 1.06 1.02 1.00
query37 0.06 0.05 0.04
query38 0.04 0.02 0.02
query39 0.02 0.01 0.01
query40 0.17 0.14 0.14
query41 0.07 0.01 0.01
query42 0.02 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 103.95 s
Total hot run time: 30.86 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #41299:
URL: https://github.com/apache/doris/pull/41299#issuecomment-2373835956
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](binlog) filter dropped indexes #41246 [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter merged PR #41299:
URL: https://github.com/apache/doris/pull/41299
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org