You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2024/03/28 11:37:04 UTC
[PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
englefly opened a new pull request, #32982:
URL: https://github.com/apache/doris/pull/32982
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029239680
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 4f4afc655ca3810e0312a27261f918354c301c66 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 15.5 seconds inserted 10000000 Rows, about 645K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026452405
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 4545a011b8cc6e80dc527a6904e9e8ddc3f08782 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029236450
<details>
<summary>ClickBench: <b>Total hot run time: 30.82 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4f4afc655ca3810e0312a27261f918354c301c66, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.24 0.05 0.04
query4 1.68 0.06 0.06
query5 0.49 0.48 0.49
query6 1.15 0.65 0.65
query7 0.02 0.01 0.01
query8 0.05 0.05 0.05
query9 0.54 0.51 0.50
query10 0.57 0.57 0.57
query11 0.15 0.11 0.11
query12 0.13 0.11 0.12
query13 0.63 0.60 0.59
query14 0.76 0.79 0.78
query15 0.85 0.82 0.83
query16 0.36 0.34 0.36
query17 0.98 0.99 0.97
query18 0.25 0.25 0.26
query19 1.82 1.71 1.78
query20 0.02 0.01 0.01
query21 15.40 0.74 0.70
query22 3.59 6.31 2.49
query23 17.79 1.25 1.09
query24 1.38 0.23 0.38
query25 0.13 0.08 0.08
query26 0.28 0.17 0.18
query27 0.08 0.09 0.08
query28 13.46 0.97 0.93
query29 12.60 3.53 3.34
query30 0.25 0.06 0.06
query31 2.87 0.38 0.38
query32 3.28 0.49 0.46
query33 2.90 2.87 2.93
query34 15.55 4.32 4.32
query35 4.38 4.37 4.37
query36 0.67 0.48 0.47
query37 0.20 0.17 0.18
query38 0.17 0.17 0.16
query39 0.05 0.04 0.05
query40 0.20 0.15 0.15
query41 0.10 0.05 0.05
query42 0.07 0.06 0.05
query43 0.05 0.04 0.04
Total cold run time: 106.26 s
Total hot run time: 30.82 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029231092
<details>
<summary>TPC-DS: <b>Total hot run time: 181948 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f4afc655ca3810e0312a27261f918354c301c66, data reload: false
query1 1226 1111 1118 1111
query2 6293 2074 1924 1924
query3 6664 214 204 204
query4 24865 21484 21515 21484
query5 4163 398 387 387
query6 284 190 180 180
query7 4599 301 305 301
query8 229 183 182 182
query9 8448 2244 2228 2228
query10 460 281 275 275
query11 15106 14508 14561 14508
query12 143 103 96 96
query13 1653 393 396 393
query14 8436 7008 6969 6969
query15 209 180 180 180
query16 6905 280 283 280
query17 971 593 596 593
query18 1867 289 294 289
query19 218 168 180 168
query20 109 97 94 94
query21 198 131 131 131
query22 4979 4775 4710 4710
query23 33614 32544 32552 32544
query24 11640 3241 3214 3214
query25 697 437 469 437
query26 1573 178 172 172
query27 3324 375 405 375
query28 6832 1927 1863 1863
query29 1202 627 607 607
query30 308 162 164 162
query31 1077 771 764 764
query32 98 68 65 65
query33 713 266 247 247
query34 1090 507 516 507
query35 881 752 741 741
query36 1021 883 885 883
query37 154 74 84 74
query38 3604 3584 3595 3584
query39 1675 1626 1617 1617
query40 246 142 142 142
query41 50 44 50 44
query42 133 116 122 116
query43 462 416 416 416
query44 1165 721 731 721
query45 295 251 268 251
query46 1129 802 797 797
query47 1998 1862 1900 1862
query48 399 313 310 310
query49 925 376 376 376
query50 830 416 419 416
query51 7061 6869 6884 6869
query52 115 97 101 97
query53 380 308 300 300
query54 318 237 244 237
query55 95 80 75 75
query56 251 239 233 233
query57 1248 1191 1169 1169
query58 250 241 231 231
query59 2764 2554 2398 2398
query60 250 222 232 222
query61 90 86 85 85
query62 642 453 450 450
query63 308 282 285 282
query64 5792 2980 3024 2980
query65 3030 3017 3009 3009
query66 1313 326 314 314
query67 15616 15043 14639 14639
query68 9342 575 576 575
query69 606 329 332 329
query70 1397 1113 1094 1094
query71 497 274 270 270
query72 6635 2574 2424 2424
query73 1566 323 334 323
query74 6687 6295 6317 6295
query75 3587 2280 2268 2268
query76 5456 1096 1189 1096
query77 558 253 248 248
query78 10801 10196 10043 10043
query79 10636 535 524 524
query80 1613 427 428 427
query81 493 231 221 221
query82 375 110 103 103
query83 209 164 164 164
query84 266 92 90 90
query85 940 325 283 283
query86 358 274 278 274
query87 3727 3453 3538 3453
query88 3810 2273 2292 2273
query89 532 374 367 367
query90 1898 178 182 178
query91 134 106 109 106
query92 66 54 56 54
query93 6541 545 538 538
query94 1216 192 196 192
query95 440 339 342 339
query96 602 277 264 264
query97 2660 2483 2499 2483
query98 228 215 212 212
query99 1182 879 854 854
Total cold run time: 299741 ms
Total hot run time: 181948 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2024985479
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029198845
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026838681
<details>
<summary>TPC-H: <b>Total hot run time: 38248 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 954fd48afbd8d6e8e6a737cbf16fda85884b6660, data reload: false
------ Round 1 ----------------------------------
q1 17610 4489 4236 4236
q2 2083 192 185 185
q3 10455 1124 1173 1124
q4 10190 794 797 794
q5 7463 2810 2701 2701
q6 214 134 134 134
q7 1021 610 581 581
q8 9211 2049 2026 2026
q9 7523 6575 6543 6543
q10 8479 3534 3539 3534
q11 456 254 241 241
q12 403 222 212 212
q13 17911 2937 2937 2937
q14 266 230 232 230
q15 518 464 461 461
q16 534 384 389 384
q17 968 548 561 548
q18 7367 6774 6563 6563
q19 1590 1437 1423 1423
q20 667 313 291 291
q21 3448 2798 2805 2798
q22 359 302 306 302
Total cold run time: 108736 ms
Total hot run time: 38248 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4207 4137 4167 4137
q2 378 257 269 257
q3 2993 2684 2628 2628
q4 1868 1492 1530 1492
q5 5239 5178 5199 5178
q6 210 124 125 124
q7 2223 1859 1860 1859
q8 3190 3295 3338 3295
q9 8524 8503 8542 8503
q10 3886 3693 3586 3586
q11 598 482 474 474
q12 722 583 586 583
q13 17775 2923 2973 2923
q14 306 263 283 263
q15 509 458 463 458
q16 488 423 414 414
q17 1794 1429 1408 1408
q18 7555 7273 7374 7273
q19 1607 1511 1513 1511
q20 1939 1766 1740 1740
q21 4790 4718 4790 4718
q22 531 455 449 449
Total cold run time: 71332 ms
Total hot run time: 53273 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026399948
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2024974247
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029220681
<details>
<summary>TPC-H: <b>Total hot run time: 38877 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f4afc655ca3810e0312a27261f918354c301c66, data reload: false
------ Round 1 ----------------------------------
q1 17607 4117 4111 4111
q2 2022 189 187 187
q3 10477 1213 1427 1213
q4 10207 869 1011 869
q5 7475 2960 2916 2916
q6 222 138 138 138
q7 1118 659 610 610
q8 9419 2080 2052 2052
q9 6678 6191 6138 6138
q10 8464 3510 3504 3504
q11 429 252 245 245
q12 382 228 217 217
q13 17790 2918 2936 2918
q14 274 253 247 247
q15 521 496 494 494
q16 523 381 383 381
q17 974 919 920 919
q18 7434 6509 6423 6423
q19 1608 1553 1550 1550
q20 585 326 322 322
q21 3635 3123 3130 3123
q22 371 300 313 300
Total cold run time: 108215 ms
Total hot run time: 38877 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4119 4043 4032 4032
q2 334 223 226 223
q3 2970 2965 2992 2965
q4 1899 1827 1842 1827
q5 5269 5229 5249 5229
q6 209 127 124 124
q7 2265 1803 1789 1789
q8 3208 3275 3282 3275
q9 8455 8499 8447 8447
q10 3750 4006 3997 3997
q11 580 471 465 465
q12 738 605 574 574
q13 16823 3090 3143 3090
q14 314 291 313 291
q15 546 509 500 500
q16 517 446 456 446
q17 1775 1753 1740 1740
q18 8264 7657 7663 7657
q19 1716 1714 1714 1714
q20 2030 1841 1845 1841
q21 5125 5010 4990 4990
q22 513 442 453 442
Total cold run time: 71419 ms
Total hot run time: 55658 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026852647
<details>
<summary>TPC-DS: <b>Total hot run time: 182445 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 954fd48afbd8d6e8e6a737cbf16fda85884b6660, data reload: false
query1 895 1123 1116 1116
query2 7321 1971 1899 1899
query3 6664 217 216 216
query4 25032 21563 21579 21563
query5 4236 403 408 403
query6 270 181 175 175
query7 4602 295 292 292
query8 224 175 180 175
query9 8430 2288 2279 2279
query10 537 240 250 240
query11 14770 14252 14285 14252
query12 137 92 89 89
query13 1642 371 381 371
query14 8796 6840 6887 6840
query15 208 180 185 180
query16 7051 265 268 265
query17 1370 604 559 559
query18 1471 289 283 283
query19 204 162 155 155
query20 93 94 94 94
query21 201 128 127 127
query22 5009 4807 4849 4807
query23 33522 32810 32829 32810
query24 12874 2870 2764 2764
query25 634 381 391 381
query26 1903 161 154 154
query27 3223 317 330 317
query28 7377 1846 1841 1841
query29 999 625 612 612
query30 303 152 150 150
query31 1004 732 714 714
query32 82 57 57 57
query33 705 256 259 256
query34 1135 484 479 479
query35 814 672 687 672
query36 1082 885 923 885
query37 274 67 69 67
query38 3570 3434 3404 3404
query39 1580 1608 1536 1536
query40 279 134 126 126
query41 47 44 43 43
query42 107 97 98 97
query43 482 461 452 452
query44 1214 749 705 705
query45 268 254 273 254
query46 1085 677 712 677
query47 1912 1845 1856 1845
query48 368 298 306 298
query49 1175 353 364 353
query50 783 385 380 380
query51 6795 6701 6669 6669
query52 105 90 91 90
query53 344 276 278 276
query54 281 231 228 228
query55 84 80 80 80
query56 244 219 220 219
query57 1228 1126 1121 1121
query58 222 195 206 195
query59 2968 2730 2541 2541
query60 266 239 233 233
query61 95 96 91 91
query62 663 431 428 428
query63 310 279 287 279
query64 6682 4070 4055 4055
query65 3160 3067 3017 3017
query66 1415 350 345 345
query67 15327 15240 15169 15169
query68 5570 520 536 520
query69 519 313 313 313
query70 1311 1187 1161 1161
query71 439 277 272 272
query72 6832 2742 2599 2599
query73 748 328 334 328
query74 6739 6410 6357 6357
query75 3100 2258 2253 2253
query76 3193 1051 1084 1051
query77 623 270 268 268
query78 11032 10192 10098 10098
query79 9120 525 540 525
query80 1952 447 459 447
query81 524 1085 226 226
query82 923 94 94 94
query83 345 170 176 170
query84 268 86 87 86
query85 1941 264 266 264
query86 477 275 285 275
query87 3721 3545 3521 3521
query88 5771 2395 2396 2395
query89 562 375 367 367
query90 1980 180 181 180
query91 128 100 96 96
query92 60 47 47 47
query93 6525 519 504 504
query94 1227 180 184 180
query95 400 318 306 306
query96 604 267 271 267
query97 2684 2479 2483 2479
query98 231 213 215 213
query99 1203 839 838 838
Total cold run time: 298905 ms
Total hot run time: 182445 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #32982:
URL: https://github.com/apache/doris/pull/32982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026433143
<details>
<summary>TPC-H: <b>Total hot run time: 37514 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4545a011b8cc6e80dc527a6904e9e8ddc3f08782, data reload: false
------ Round 1 ----------------------------------
q1 17621 4191 4133 4133
q2 2123 157 148 148
q3 10583 1095 1198 1095
q4 10228 763 744 744
q5 7450 2872 2871 2871
q6 200 122 123 122
q7 1030 588 566 566
q8 9342 1948 1963 1948
q9 6933 6320 6295 6295
q10 8444 3464 3596 3464
q11 443 240 212 212
q12 437 197 194 194
q13 17795 2863 2880 2863
q14 243 199 207 199
q15 504 470 460 460
q16 494 364 369 364
q17 950 631 554 554
q18 7029 6423 6416 6416
q19 1548 1426 1492 1426
q20 536 255 261 255
q21 3611 2908 2894 2894
q22 337 294 291 291
Total cold run time: 107881 ms
Total hot run time: 37514 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4132 4074 4069 4069
q2 326 232 230 230
q3 2970 2840 2831 2831
q4 1842 1520 1553 1520
q5 5201 5234 5268 5234
q6 195 116 117 116
q7 2189 1794 1810 1794
q8 3162 3310 3294 3294
q9 8437 8357 8402 8357
q10 3799 3777 3900 3777
q11 547 459 451 451
q12 779 576 595 576
q13 17029 3019 3081 3019
q14 303 280 272 272
q15 523 487 479 479
q16 494 457 455 455
q17 1763 1506 1531 1506
q18 8101 7670 7698 7670
q19 1690 1571 1569 1569
q20 2039 1895 1715 1715
q21 5015 4816 4833 4816
q22 541 440 445 440
Total cold run time: 71077 ms
Total hot run time: 54190 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026860381
<details>
<summary>ClickBench: <b>Total hot run time: 29.54 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 954fd48afbd8d6e8e6a737cbf16fda85884b6660, data reload: false
query1 0.04 0.04 0.02
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.47 0.47 0.49
query6 1.14 0.66 0.66
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.54 0.49 0.50
query10 0.55 0.55 0.55
query11 0.15 0.10 0.12
query12 0.14 0.12 0.11
query13 0.58 0.58 0.58
query14 0.76 0.76 0.76
query15 0.82 0.80 0.79
query16 0.37 0.37 0.37
query17 1.01 0.96 0.97
query18 0.21 0.21 0.26
query19 1.71 1.65 1.66
query20 0.01 0.01 0.01
query21 15.60 0.64 0.64
query22 2.91 3.09 1.80
query23 17.23 0.82 0.90
query24 1.16 0.25 0.23
query25 0.09 0.08 0.08
query26 0.23 0.18 0.17
query27 0.08 0.07 0.08
query28 14.02 0.93 0.93
query29 12.56 3.25 3.29
query30 0.28 0.09 0.07
query31 2.81 0.40 0.37
query32 3.27 0.45 0.44
query33 2.80 2.78 2.86
query34 16.85 4.43 4.42
query35 4.51 4.51 4.49
query36 0.61 0.46 0.46
query37 0.19 0.15 0.15
query38 0.15 0.14 0.13
query39 0.06 0.03 0.04
query40 0.18 0.14 0.15
query41 0.09 0.05 0.04
query42 0.06 0.05 0.06
query43 0.05 0.04 0.04
Total cold run time: 106.34 s
Total hot run time: 29.54 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029015609
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2029015576
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026866517
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 954fd48afbd8d6e8e6a737cbf16fda85884b6660 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.7 seconds inserted 10000000 Rows, about 729K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026820876
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2026445590
<details>
<summary>TPC-DS: <b>Total hot run time: 183048 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4545a011b8cc6e80dc527a6904e9e8ddc3f08782, data reload: false
query1 920 353 357 353
query2 6344 1990 2079 1990
query3 6702 227 216 216
query4 31388 21385 21484 21385
query5 4254 412 388 388
query6 266 183 174 174
query7 4627 284 290 284
query8 229 173 160 160
query9 9337 2275 2279 2275
query10 432 258 251 251
query11 15652 14238 14205 14205
query12 136 97 94 94
query13 1627 433 427 427
query14 9781 7716 7485 7485
query15 249 187 190 187
query16 8124 275 269 269
query17 1934 579 552 552
query18 2018 306 294 294
query19 263 166 158 158
query20 96 90 88 88
query21 197 129 132 129
query22 5002 4838 4815 4815
query23 33637 33116 33315 33116
query24 10968 2951 2933 2933
query25 621 401 418 401
query26 1188 168 162 162
query27 2570 375 378 375
query28 7344 1925 1943 1925
query29 933 657 628 628
query30 308 159 154 154
query31 993 768 740 740
query32 91 65 59 59
query33 757 245 247 245
query34 1444 503 526 503
query35 908 736 735 735
query36 1038 914 891 891
query37 117 63 63 63
query38 3746 3707 3613 3613
query39 1651 1615 1595 1595
query40 175 112 106 106
query41 49 48 50 48
query42 111 100 94 94
query43 472 469 468 468
query44 1274 753 769 753
query45 285 267 252 252
query46 1144 740 717 717
query47 2062 1958 1917 1917
query48 468 378 368 368
query49 945 355 345 345
query50 778 390 370 370
query51 6827 6813 6734 6734
query52 113 97 86 86
query53 348 269 284 269
query54 303 249 245 245
query55 81 85 79 79
query56 245 246 244 244
query57 1279 1189 1216 1189
query58 235 220 212 212
query59 2973 2757 2520 2520
query60 267 271 246 246
query61 116 114 110 110
query62 607 447 437 437
query63 302 275 272 272
query64 5536 4044 4134 4044
query65 3045 3052 3021 3021
query66 799 367 386 367
query67 15600 14926 14913 14913
query68 8559 541 538 538
query69 636 396 383 383
query70 1162 1172 1153 1153
query71 506 271 267 267
query72 6296 2532 2359 2359
query73 743 325 323 323
query74 8267 6449 6419 6419
query75 3864 2202 2192 2192
query76 5228 847 889 847
query77 609 268 259 259
query78 11004 10246 10170 10170
query79 11155 534 529 529
query80 2000 382 370 370
query81 523 217 213 213
query82 802 87 82 82
query83 202 146 147 146
query84 288 84 76 76
query85 1397 308 302 302
query86 401 303 293 293
query87 3741 3569 3498 3498
query88 4947 2417 2410 2410
query89 493 357 357 357
query90 1945 178 179 178
query91 162 134 138 134
query92 59 47 45 45
query93 6945 490 493 490
query94 1077 181 172 172
query95 407 312 313 312
query96 627 286 271 271
query97 2715 2475 2483 2475
query98 227 208 205 205
query99 1160 855 837 837
Total cold run time: 312478 ms
Total hot run time: 183048 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) push down runtime filter to all children of SetOperation [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32982:
URL: https://github.com/apache/doris/pull/32982#issuecomment-2030872469
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org