You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/09 11:22:11 UTC
[PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Gabriel39 opened a new pull request, #29738:
URL: https://github.com/apache/doris/pull/29738
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882890479
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882953344
TeamCity be ut coverage result:
Function Coverage: 36.66% (8645/23581)
Line Coverage: 28.72% (70436/245273)
Region Coverage: 27.64% (36387/131637)
Branch Coverage: 24.33% (18600/76440)
Coverage Report: http://coverage.selectdb-in.cc/coverage/24ca4e12b9505098cb707d7c15b41a53756c0b8f_24ca4e12b9505098cb707d7c15b41a53756c0b8f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882943246
<details>
<summary>TPC-H: <b>Total hot run time: 40330 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f, data reload: false
------ Round 1 ----------------------------------
q1 18278 5579 5551 5551
q2 2037 163 146 146
q3 10777 1208 1211 1208
q4 10337 845 870 845
q5 7921 3066 3023 3023
q6 210 132 137 132
q7 958 528 518 518
q8 9422 2061 2057 2057
q9 6923 6629 6594 6594
q10 8393 3058 3061 3058
q11 448 215 217 215
q12 396 228 232 228
q13 18289 3467 3463 3463
q14 243 214 217 214
q15 560 532 514 514
q16 471 418 424 418
q17 956 791 664 664
q18 7607 6953 6887 6887
q19 1578 1540 1524 1524
q20 733 338 329 329
q21 2777 2483 2411 2411
q22 384 348 331 331
Total cold run time: 109698 ms
Total hot run time: 40330 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5515 5500 5524 5500
q2 357 258 261 258
q3 3424 3352 3319 3319
q4 2126 2071 2107 2071
q5 5888 5846 5912 5846
q6 208 124 129 124
q7 2350 1883 1936 1883
q8 3542 3622 3556 3556
q9 8981 8960 8919 8919
q10 3890 3920 3918 3918
q11 600 503 506 503
q12 817 641 647 641
q13 3950 3260 3238 3238
q14 299 261 270 261
q15 567 518 513 513
q16 550 482 491 482
q17 2458 2388 2387 2387
q18 9088 8817 8859 8817
q19 1594 1634 1636 1634
q20 2260 1987 1950 1950
q21 5791 5488 5249 5249
q22 602 539 520 520
Total cold run time: 64857 ms
Total hot run time: 61589 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882961962
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.26 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s
storage size: 17184032047 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882968994
<details>
<summary>ClickBench: <b>Total hot run time: 31.32 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f, data reload: false
query1 0.11 0.07 0.06
query2 0.07 0.02 0.02
query3 0.32 0.21 0.20
query4 1.71 0.19 0.19
query5 0.54 0.53 0.54
query6 1.58 0.68 0.70
query7 0.02 0.01 0.01
query8 0.04 0.02 0.03
query9 0.62 0.57 0.58
query10 0.63 0.63 0.64
query11 0.17 0.13 0.11
query12 0.15 0.13 0.11
query13 0.61 0.61 0.61
query14 0.80 0.79 0.79
query15 0.84 0.82 0.82
query16 0.35 0.34 0.35
query17 0.97 0.95 1.01
query18 0.27 0.26 0.27
query19 1.85 1.80 1.77
query20 0.01 0.01 0.01
query21 15.42 0.54 0.54
query22 2.93 3.25 0.73
query23 17.12 1.15 1.06
query24 10.03 0.34 0.98
query25 2.44 0.18 0.18
query26 0.16 0.15 0.16
query27 0.17 0.17 0.17
query28 5.32 0.94 0.87
query29 13.13 3.53 3.53
query30 0.58 0.50 0.49
query31 2.92 0.38 0.40
query32 3.27 0.49 0.50
query33 3.29 3.27 3.29
query34 16.19 4.73 4.66
query35 4.60 4.76 4.45
query36 1.17 1.09 1.11
query37 0.07 0.04 0.04
query38 0.03 0.03 0.02
query39 0.02 0.01 0.02
query40 0.16 0.13 0.14
query41 0.07 0.01 0.02
query42 0.02 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 110.8 s
Total hot run time: 31.32 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882973575
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f with default session variables
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 17.4 seconds inserted 10000000 Rows, about 574K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882901097
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29738:
URL: https://github.com/apache/doris/pull/29738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882894246
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882894282
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix runtime filter timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29738:
URL: https://github.com/apache/doris/pull/29738#issuecomment-1882960724
<details>
<summary>TPC-DS: <b>Total hot run time: 186608 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 24ca4e12b9505098cb707d7c15b41a53756c0b8f, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 955 351 339 339
query2 6839 2054 1990 1990
query3 6726 232 233 232
query4 26494 22594 22487 22487
query5 6164 628 583 583
query6 334 220 224 220
query7 4669 304 309 304
query8 262 243 224 224
query9 9720 3305 3340 3305
query10 589 355 307 307
query11 16411 15670 15623 15623
query12 173 116 112 112
query13 1681 333 336 333
query14 13966 7509 7668 7509
query15 295 246 240 240
query16 6384 362 357 357
query17 1684 563 530 530
query18 1956 343 332 332
query19 346 178 161 161
query20 134 118 119 118
query21 202 103 99 99
query22 5009 4710 4801 4710
query23 32187 31206 31575 31206
query24 12036 3069 3017 3017
query25 620 400 379 379
query26 1768 177 182 177
query27 2756 323 304 304
query28 7157 2162 2135 2135
query29 2029 458 434 434
query30 287 149 151 149
query31 1053 823 847 823
query32 142 91 91 91
query33 837 350 360 350
query34 887 490 508 490
query35 1040 847 847 847
query36 1368 1357 1391 1357
query37 228 130 107 107
query38 3516 3378 3333 3333
query39 1384 1298 1314 1298
query40 310 92 94 92
query41 41 42 40 40
query42 119 119 116 116
query43 620 563 583 563
query44 1307 826 852 826
query45 216 199 212 199
query46 1093 714 711 711
query47 1773 1740 1729 1729
query48 388 289 290 289
query49 1215 326 322 322
query50 799 342 369 342
query51 5494 5341 5339 5339
query52 125 115 112 112
query53 236 182 176 176
query54 1395 716 711 711
query55 111 101 102 101
query56 290 285 269 269
query57 1137 1041 1013 1013
query58 347 327 338 327
query59 3331 3142 3120 3120
query60 345 332 300 300
query61 152 154 150 150
query62 590 494 502 494
query63 213 203 205 203
query64 6119 1753 1768 1753
query65 3436 3377 3375 3375
query66 1349 427 393 393
query67 15832 15711 15481 15481
query68 12884 556 545 545
query69 616 373 367 367
query70 1818 1800 1738 1738
query71 618 306 271 271
query72 5482 3436 3551 3436
query73 3024 336 346 336
query74 7192 6586 6617 6586
query75 5330 2313 2333 2313
query76 6368 1049 1245 1049
query77 763 365 331 331
query78 9103 8849 8686 8686
query79 4590 548 547 547
query80 703 360 362 360
query81 467 220 232 220
query82 241 141 140 140
query83 178 147 147 147
query84 254 58 63 58
query85 968 280 284 280
query86 454 469 471 469
query87 3630 3496 3448 3448
query88 3644 2550 2537 2537
query89 415 321 327 321
query90 1943 273 284 273
query91 135 95 100 95
query92 116 91 85 85
query93 2025 496 426 426
query94 863 237 254 237
query95 603 530 519 519
query96 666 337 337 337
query97 4420 4287 4219 4219
query98 239 215 212 212
query99 1191 898 862 862
Total cold run time: 309614 ms
Total hot run time: 186608 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org