You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2024/01/07 13:03:00 UTC
[PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
englefly opened a new pull request, #29633:
URL: https://github.com/apache/doris/pull/29633
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884657475
<details>
<summary>TPC-DS: <b>Total hot run time: 179094 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2a58aa200f5f5b2044207be279fb145fdac45622, data reload: false
query1 935 344 335 335
query2 6690 1980 1868 1868
query3 6720 208 202 202
query4 26278 22309 22231 22231
query5 5123 526 487 487
query6 279 190 189 189
query7 4629 278 281 278
query8 238 207 203 203
query9 9122 2861 2776 2776
query10 574 242 235 235
query11 16192 15370 15380 15370
query12 136 83 80 80
query13 1624 335 352 335
query14 11942 7269 7225 7225
query15 295 201 201 201
query16 6528 276 265 265
query17 1909 489 488 488
query18 1970 282 279 279
query19 289 149 145 145
query20 88 81 81 81
query21 187 92 91 91
query22 5122 4838 4712 4712
query23 32181 31327 31331 31327
query24 11146 2839 2805 2805
query25 569 350 341 341
query26 1380 157 155 155
query27 2856 282 290 282
query28 6881 1927 1934 1927
query29 1125 395 405 395
query30 282 149 144 144
query31 1001 792 758 758
query32 101 61 60 60
query33 741 282 271 271
query34 872 441 442 441
query35 855 754 726 726
query36 1431 1282 1296 1282
query37 103 74 75 74
query38 3410 3261 3264 3261
query39 1339 1288 1287 1287
query40 195 93 101 93
query41 38 37 36 36
query42 98 96 96 96
query43 548 502 542 502
query44 1147 724 737 724
query45 213 189 179 179
query46 1056 656 674 656
query47 1674 1586 1601 1586
query48 409 320 333 320
query49 1130 317 314 314
query50 789 363 334 334
query51 5377 5359 5317 5317
query52 97 88 93 88
query53 224 163 156 156
query54 1416 587 608 587
query55 95 87 87 87
query56 226 206 198 198
query57 999 915 947 915
query58 243 218 225 218
query59 2872 2635 2702 2635
query60 252 242 252 242
query61 83 85 83 83
query62 653 478 446 446
query63 171 153 149 149
query64 4886 1801 1664 1664
query65 3382 3268 3305 3268
query66 1080 334 341 334
query67 15698 15161 15008 15008
query68 12005 540 516 516
query69 567 282 286 282
query70 1750 1626 1617 1617
query71 514 238 241 238
query72 4996 2823 2846 2823
query73 2302 326 314 314
query74 6954 6428 6502 6428
query75 4961 2287 2301 2287
query76 6315 1083 1134 1083
query77 691 282 293 282
query78 9727 8719 8539 8539
query79 1025 512 512 512
query80 585 346 372 346
query81 460 210 226 210
query82 214 91 91 91
query83 173 139 140 139
query84 266 59 58 58
query85 974 291 284 284
query86 393 386 403 386
query87 3539 3442 3356 3356
query88 2937 2234 2230 2230
query89 341 266 261 261
query90 1867 207 197 197
query91 165 132 143 132
query92 65 63 54 54
query93 1130 453 502 453
query94 772 196 191 191
query95 494 443 440 440
query96 622 319 327 319
query97 4310 4191 4186 4186
query98 217 194 185 185
query99 1108 870 808 808
Total cold run time: 290633 ms
Total hot run time: 179094 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882593691
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880659680
<details>
<summary>TPC-H: <b>Total hot run time: 38664 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1c6d66b32efa23fe3a0b7973ea7f07634aa0974f, data reload: false
------ Round 1 ----------------------------------
q1 17652 4913 4904 4904
q2 2032 153 142 142
q3 10837 1113 1128 1113
q4 10521 776 797 776
q5 7809 2977 2955 2955
q6 203 129 130 129
q7 955 528 499 499
q8 9256 2033 2017 2017
q9 6560 6428 6403 6403
q10 8312 3071 2994 2994
q11 448 234 218 218
q12 396 242 248 242
q13 18013 3423 3382 3382
q14 235 218 207 207
q15 544 509 503 503
q16 472 425 389 389
q17 975 724 604 604
q18 7382 6806 6696 6696
q19 1611 1561 1528 1528
q20 722 329 332 329
q21 2788 2504 2329 2329
q22 358 305 316 305
Total cold run time: 108081 ms
Total hot run time: 38664 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4933 4875 4941 4875
q2 341 232 235 232
q3 3287 3304 3302 3302
q4 2106 2011 2104 2011
q5 5818 5811 5789 5789
q6 206 119 120 119
q7 2309 1899 1920 1899
q8 3400 3455 3497 3455
q9 8826 8843 8777 8777
q10 3783 3825 3857 3825
q11 548 437 444 437
q12 806 642 632 632
q13 9562 3199 3204 3199
q14 290 253 264 253
q15 546 508 504 504
q16 549 514 506 506
q17 1900 1897 1871 1871
q18 8677 8408 8347 8347
q19 1650 1653 1657 1653
q20 2193 1970 1929 1929
q21 5560 5247 5226 5226
q22 510 438 462 438
Total cold run time: 67800 ms
Total hot run time: 59279 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1885034811
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 417e0397e2a4f1c9485e146d4ce2c1e3886a544c with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.2 seconds inserted 10000000 Rows, about 704K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1885918645
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880485981
<details>
<summary>TPC-DS: <b>Total hot run time: 184285 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f336b7704ecb359867bf433b8181c89d0c345a91, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 946 346 341 341
query2 6580 2225 1982 1982
query3 6725 229 217 217
query4 26011 22539 22494 22494
query5 5744 580 548 548
query6 345 196 201 196
query7 4641 308 317 308
query8 251 221 217 217
query9 8609 2957 2975 2957
query10 549 328 317 317
query11 16376 15697 15671 15671
query12 174 107 99 99
query13 1685 343 334 334
query14 12421 7444 7596 7444
query15 275 221 204 204
query16 6426 338 329 329
query17 1713 516 510 510
query18 1970 314 300 300
query19 243 168 162 162
query20 106 109 98 98
query21 189 101 91 91
query22 5102 4765 4941 4765
query23 32163 31299 31491 31299
query24 12245 2928 2934 2928
query25 604 388 357 357
query26 1804 179 168 168
query27 3007 313 308 308
query28 6992 2190 2181 2181
query29 2053 427 432 427
query30 287 145 150 145
query31 1009 831 803 803
query32 137 77 74 74
query33 848 345 332 332
query34 861 455 466 455
query35 994 901 862 862
query36 1395 1242 1278 1242
query37 214 93 95 93
query38 3535 3388 3335 3335
query39 1369 1302 1299 1299
query40 310 94 88 88
query41 40 38 39 38
query42 124 113 101 101
query43 565 515 531 515
query44 1218 803 817 803
query45 212 211 217 211
query46 1096 695 721 695
query47 1703 1673 1728 1673
query48 374 271 305 271
query49 1214 310 309 309
query50 752 355 319 319
query51 5515 5243 5331 5243
query52 109 95 106 95
query53 232 167 169 167
query54 1389 691 672 672
query55 110 98 100 98
query56 308 253 264 253
query57 1046 972 1011 972
query58 290 259 278 259
query59 3141 2947 2896 2896
query60 349 294 300 294
query61 157 153 130 130
query62 579 497 465 465
query63 194 183 177 177
query64 5856 1775 1719 1719
query65 3418 3317 3362 3317
query66 1396 393 385 385
query67 15827 15235 15741 15235
query68 12614 550 573 550
query69 634 299 316 299
query70 1712 1583 1564 1564
query71 580 295 294 294
query72 5332 3447 3464 3447
query73 2500 338 342 338
query74 7055 6595 6549 6549
query75 5225 2296 2279 2279
query76 6329 1246 1217 1217
query77 874 314 317 314
query78 9086 8603 8733 8603
query79 1066 523 534 523
query80 563 342 339 339
query81 484 217 216 216
query82 235 105 107 105
query83 199 145 141 141
query84 255 57 58 57
query85 964 293 274 274
query86 444 431 414 414
query87 3619 3600 3493 3493
query88 3028 2536 2507 2507
query89 376 281 284 281
query90 1906 253 254 253
query91 126 95 98 95
query92 89 69 67 67
query93 1228 450 492 450
query94 897 241 248 241
query95 594 506 494 494
query96 643 336 330 330
query97 4366 4216 4247 4216
query98 243 222 220 220
query99 1160 823 844 823
Total cold run time: 298548 ms
Total hot run time: 184285 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884665618
<details>
<summary>ClickBench: <b>Total hot run time: 30.68 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2a58aa200f5f5b2044207be279fb145fdac45622, data reload: false
query1 0.06 0.06 0.05
query2 0.07 0.03 0.02
query3 0.26 0.11 0.10
query4 1.76 0.11 0.11
query5 0.52 0.51 0.50
query6 1.37 0.64 0.63
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.57 0.50 0.49
query10 0.55 0.55 0.55
query11 0.13 0.09 0.09
query12 0.11 0.09 0.10
query13 0.61 0.61 0.61
query14 0.77 0.80 0.78
query15 0.82 0.82 0.81
query16 0.35 0.36 0.35
query17 0.99 0.96 1.00
query18 0.26 0.24 0.24
query19 1.82 1.80 1.72
query20 0.02 0.01 0.01
query21 15.39 0.60 0.57
query22 2.79 2.31 1.68
query23 17.41 0.84 0.79
query24 5.29 1.08 1.01
query25 2.02 0.15 0.14
query26 0.14 0.14 0.13
query27 0.15 0.15 0.15
query28 9.16 0.80 0.81
query29 12.55 3.23 3.20
query30 0.51 0.49 0.46
query31 2.78 0.35 0.36
query32 3.40 0.48 0.48
query33 3.23 3.22 3.24
query34 15.71 4.21 4.12
query35 4.16 4.13 4.20
query36 1.11 1.05 1.05
query37 0.06 0.04 0.04
query38 0.04 0.03 0.03
query39 0.02 0.01 0.01
query40 0.19 0.13 0.13
query41 0.07 0.02 0.01
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 107.32 s
Total hot run time: 30.68 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884997617
<details>
<summary>TPC-H: <b>Total hot run time: 38582 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 417e0397e2a4f1c9485e146d4ce2c1e3886a544c, data reload: false
------ Round 1 ----------------------------------
q1 17959 5031 4957 4957
q2 2038 155 136 136
q3 10674 1135 1118 1118
q4 10460 803 810 803
q5 7868 3107 3141 3107
q6 202 124 125 124
q7 871 503 497 497
q8 9284 1968 2017 1968
q9 7391 6501 6429 6429
q10 8235 3068 3031 3031
q11 408 200 219 200
q12 355 197 192 192
q13 18118 3441 3434 3434
q14 245 214 218 214
q15 559 522 504 504
q16 446 384 388 384
q17 943 556 504 504
q18 7253 6663 6624 6624
q19 1575 1330 1465 1330
q20 583 311 285 285
q21 2752 2419 2442 2419
q22 386 332 322 322
Total cold run time: 108605 ms
Total hot run time: 38582 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5009 5086 4998 4998
q2 312 203 202 202
q3 3369 3317 3291 3291
q4 2249 2246 2252 2246
q5 5874 5804 5788 5788
q6 191 116 118 116
q7 2323 1870 1882 1870
q8 3449 3565 3587 3565
q9 8895 8826 8796 8796
q10 3734 3839 3814 3814
q11 533 435 427 427
q12 798 641 612 612
q13 4097 3283 3222 3222
q14 290 265 277 265
q15 574 506 503 503
q16 510 456 463 456
q17 2054 2030 2001 2001
q18 8733 8409 8331 8331
q19 1607 1623 1597 1597
q20 2178 1950 1916 1916
q21 6118 5634 5734 5634
q22 598 507 512 507
Total cold run time: 63495 ms
Total hot run time: 60157 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #29633:
URL: https://github.com/apache/doris/pull/29633
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880607243
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882345052
<details>
<summary>TPC-DS: <b>Total hot run time: 184827 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f6314b56c02b8cd19bb7d48e23979812dcaf426b, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 925 343 330 330
query2 6570 2114 2034 2034
query3 6718 224 223 223
query4 25594 22444 22344 22344
query5 5314 587 569 569
query6 336 222 211 211
query7 4632 306 306 306
query8 258 220 232 220
query9 8840 2925 2963 2925
query10 542 323 353 323
query11 15826 15616 15592 15592
query12 168 115 99 99
query13 1672 350 335 335
query14 13510 7572 7483 7483
query15 269 227 216 216
query16 6509 359 325 325
query17 1682 532 518 518
query18 1952 301 303 301
query19 241 167 170 167
query20 106 104 98 98
query21 191 95 91 91
query22 5100 4934 4899 4899
query23 32169 31602 31812 31602
query24 11676 2940 2893 2893
query25 609 382 371 371
query26 1822 167 164 164
query27 2785 305 302 302
query28 6801 2140 2160 2140
query29 2082 427 430 427
query30 290 142 149 142
query31 1015 809 820 809
query32 122 88 92 88
query33 842 349 339 339
query34 845 465 465 465
query35 1031 878 870 870
query36 1425 1376 1341 1341
query37 198 97 100 97
query38 3489 3421 3323 3323
query39 1350 1300 1298 1298
query40 298 92 89 89
query41 39 38 40 38
query42 114 105 103 103
query43 581 524 531 524
query44 1156 809 827 809
query45 220 203 211 203
query46 1059 698 691 691
query47 1700 1622 1633 1622
query48 368 294 284 284
query49 1212 314 323 314
query50 732 360 323 323
query51 5370 5391 5288 5288
query52 102 107 101 101
query53 225 167 169 167
query54 1414 682 739 682
query55 106 92 97 92
query56 265 272 252 252
query57 1040 1006 1009 1006
query58 318 290 265 265
query59 3058 2954 2973 2954
query60 345 307 298 298
query61 154 157 148 148
query62 570 517 474 474
query63 196 186 172 172
query64 5838 1758 1691 1691
query65 3429 3358 3349 3349
query66 1390 387 383 383
query67 15695 15269 15376 15269
query68 13562 544 559 544
query69 640 312 321 312
query70 1807 1470 1591 1470
query71 605 289 296 289
query72 5380 3523 3500 3500
query73 3251 334 381 334
query74 7154 6650 6643 6643
query75 5246 2256 2282 2256
query76 6412 1170 1153 1153
query77 793 347 353 347
query78 9074 8501 8579 8501
query79 1978 533 531 531
query80 575 340 350 340
query81 469 214 210 210
query82 240 111 107 107
query83 194 142 142 142
query84 254 58 58 58
query85 945 281 274 274
query86 425 448 445 445
query87 3597 3474 3501 3474
query88 3009 2506 2511 2506
query89 370 288 290 288
query90 2021 264 249 249
query91 127 96 95 95
query92 99 82 90 82
query93 1221 495 477 477
query94 919 243 245 243
query95 569 505 503 503
query96 636 344 344 344
query97 4331 4327 4184 4184
query98 248 229 211 211
query99 1151 904 844 844
Total cold run time: 299802 ms
Total hot run time: 184827 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882368950
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.83 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.3 seconds inserted 10000000 Rows, about 366K ops/s
storage size: 17188524322 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880054451
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884947732
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880683125
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.71 seconds
stream load tsv: 572 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.6 seconds inserted 10000000 Rows, about 362K ops/s
storage size: 17189056282 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882513854
<details>
<summary>TPC-DS: <b>Total hot run time: 184383 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41e1e6baffd9436a19fe012b9a4121bf77a96769, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 916 339 340 339
query2 6389 1977 1922 1922
query3 6670 226 221 221
query4 26032 22462 22490 22462
query5 5581 572 554 554
query6 344 207 206 206
query7 4639 293 315 293
query8 251 238 211 211
query9 8680 2898 2778 2778
query10 552 355 355 355
query11 16372 15598 15745 15598
query12 177 96 108 96
query13 1671 336 334 334
query14 13110 7584 7673 7584
query15 245 217 215 215
query16 5979 356 325 325
query17 1635 537 503 503
query18 1958 302 287 287
query19 238 163 158 158
query20 106 103 105 103
query21 188 97 89 89
query22 5108 4771 4911 4771
query23 32236 31559 31414 31414
query24 11936 3023 2965 2965
query25 552 365 368 365
query26 1591 167 177 167
query27 2967 308 306 306
query28 6804 2132 2102 2102
query29 1255 410 412 410
query30 284 145 148 145
query31 1001 833 823 823
query32 111 78 79 78
query33 813 332 350 332
query34 944 465 469 465
query35 1000 880 797 797
query36 1431 1200 1273 1200
query37 212 81 85 81
query38 3477 3396 3349 3349
query39 1354 1464 1306 1306
query40 291 89 87 87
query41 38 35 38 35
query42 103 110 103 103
query43 549 506 521 506
query44 1260 799 807 799
query45 222 201 204 201
query46 1154 701 705 701
query47 1674 1714 1669 1669
query48 382 284 287 284
query49 1137 326 317 317
query50 815 364 352 352
query51 5573 5279 5249 5249
query52 113 100 104 100
query53 240 173 163 163
query54 1389 698 709 698
query55 104 101 100 100
query56 271 259 237 237
query57 1020 971 953 953
query58 303 278 277 277
query59 3045 2911 2755 2755
query60 345 304 300 300
query61 152 151 154 151
query62 567 484 478 478
query63 195 171 174 171
query64 4831 1671 1689 1671
query65 3432 3377 3380 3377
query66 1205 399 368 368
query67 15638 15817 15622 15622
query68 12893 560 554 554
query69 645 290 295 290
query70 1682 1609 1615 1609
query71 611 281 286 281
query72 5876 3503 3469 3469
query73 2780 325 344 325
query74 7364 6457 6499 6457
query75 5556 2293 2278 2278
query76 6612 1226 1305 1226
query77 770 343 347 343
query78 9146 8655 8655 8655
query79 1190 531 564 531
query80 589 341 343 341
query81 461 210 216 210
query82 229 112 118 112
query83 190 142 145 142
query84 257 58 57 57
query85 943 277 269 269
query86 447 408 445 408
query87 3715 3405 3510 3405
query88 3110 2539 2521 2521
query89 405 303 295 295
query90 1988 256 257 256
query91 124 98 95 95
query92 90 78 68 68
query93 1210 430 424 424
query94 925 249 218 218
query95 606 495 491 491
query96 700 357 350 350
query97 4349 4212 4243 4212
query98 235 208 209 208
query99 1174 850 896 850
Total cold run time: 297729 ms
Total hot run time: 184383 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884655611
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.63 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s
storage size: 17183989232 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1885028701
<details>
<summary>ClickBench: <b>Total hot run time: 30.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 417e0397e2a4f1c9485e146d4ce2c1e3886a544c, data reload: false
query1 0.09 0.06 0.06
query2 0.06 0.02 0.02
query3 0.25 0.12 0.10
query4 1.77 0.11 0.11
query5 0.53 0.52 0.51
query6 1.41 0.65 0.66
query7 0.02 0.02 0.01
query8 0.03 0.02 0.02
query9 0.55 0.51 0.49
query10 0.55 0.54 0.55
query11 0.14 0.10 0.09
query12 0.12 0.09 0.09
query13 0.61 0.60 0.59
query14 0.78 0.79 0.78
query15 0.81 0.80 0.80
query16 0.36 0.37 0.35
query17 0.95 0.94 0.97
query18 0.27 0.25 0.26
query19 1.80 1.78 1.75
query20 0.02 0.01 0.01
query21 15.39 0.58 0.56
query22 2.51 2.82 1.77
query23 17.08 0.71 0.81
query24 6.23 1.11 1.03
query25 2.02 0.14 0.14
query26 0.14 0.13 0.14
query27 0.15 0.14 0.14
query28 9.44 0.83 0.81
query29 12.53 3.22 3.13
query30 0.63 0.51 0.46
query31 2.78 0.36 0.36
query32 3.38 0.48 0.48
query33 3.20 3.20 3.21
query34 15.87 4.18 4.15
query35 4.16 4.15 4.17
query36 1.10 1.05 1.05
query37 0.06 0.05 0.05
query38 0.04 0.03 0.03
query39 0.03 0.01 0.02
query40 0.17 0.13 0.14
query41 0.07 0.02 0.01
query42 0.02 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 108.15 s
Total hot run time: 30.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880310632
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880401458
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.88 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.6 seconds inserted 10000000 Rows, about 362K ops/s
storage size: 17188543202 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882265898
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1885018592
<details>
<summary>TPC-DS: <b>Total hot run time: 178313 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 417e0397e2a4f1c9485e146d4ce2c1e3886a544c, data reload: false
query1 928 340 334 334
query2 6704 1815 1935 1815
query3 6706 210 203 203
query4 26031 22239 22171 22171
query5 5131 522 496 496
query6 277 194 190 190
query7 4629 270 273 270
query8 240 205 211 205
query9 9249 2794 2795 2794
query10 558 264 233 233
query11 16057 15349 15249 15249
query12 141 78 77 77
query13 1619 360 336 336
query14 12118 6964 7241 6964
query15 245 194 203 194
query16 6486 274 269 269
query17 1931 486 480 480
query18 1936 280 262 262
query19 292 143 138 138
query20 82 83 80 80
query21 189 91 89 89
query22 5042 4777 4822 4777
query23 31893 31208 31569 31208
query24 11115 2857 2829 2829
query25 561 344 337 337
query26 1356 160 148 148
query27 2885 272 280 272
query28 6968 1915 1903 1903
query29 1050 388 385 385
query30 281 142 144 142
query31 980 782 791 782
query32 95 64 54 54
query33 729 277 274 274
query34 882 431 431 431
query35 888 760 756 756
query36 1382 1283 1179 1179
query37 102 66 72 66
query38 3383 3320 3290 3290
query39 1319 1288 1287 1287
query40 193 99 93 93
query41 38 36 35 35
query42 101 88 93 88
query43 546 498 513 498
query44 1076 715 725 715
query45 210 181 187 181
query46 1068 655 655 655
query47 1658 1527 1557 1527
query48 410 314 313 313
query49 1133 331 318 318
query50 723 342 365 342
query51 5352 5320 5231 5231
query52 107 83 83 83
query53 217 153 153 153
query54 1377 576 565 565
query55 98 88 85 85
query56 220 192 196 192
query57 977 975 979 975
query58 233 217 215 215
query59 2797 2510 2515 2510
query60 255 245 231 231
query61 81 79 77 77
query62 674 463 462 462
query63 170 149 157 149
query64 4896 1712 1685 1685
query65 3352 3276 3277 3276
query66 1066 334 329 329
query67 15587 15387 15173 15173
query68 12737 513 519 513
query69 576 281 278 278
query70 1634 1599 1566 1566
query71 511 225 234 225
query72 4904 2830 2836 2830
query73 2917 329 314 314
query74 7082 6509 6445 6445
query75 5295 2327 2278 2278
query76 6371 1059 1053 1053
query77 683 278 290 278
query78 9889 8565 8612 8565
query79 1037 486 494 486
query80 578 345 359 345
query81 472 214 220 214
query82 205 95 95 95
query83 163 141 132 132
query84 262 58 56 56
query85 1009 282 280 280
query86 379 362 370 362
query87 3553 3438 3400 3400
query88 3094 2216 2219 2216
query89 352 269 263 263
query90 1953 200 202 200
query91 166 138 142 138
query92 62 54 50 50
query93 1474 490 487 487
query94 774 194 188 188
query95 477 434 429 429
query96 628 322 315 315
query97 4290 4121 4191 4121
query98 218 193 187 187
query99 1149 879 891 879
Total cold run time: 291969 ms
Total hot run time: 178313 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882593636
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880476165
<details>
<summary>TPC-H: <b>Total hot run time: 38411 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f336b7704ecb359867bf433b8181c89d0c345a91, data reload: false
------ Round 1 ----------------------------------
q1 17706 4955 4957 4955
q2 2009 150 142 142
q3 10595 1156 1165 1156
q4 10434 793 824 793
q5 7802 2926 2925 2925
q6 203 122 123 122
q7 907 507 492 492
q8 9248 2004 1990 1990
q9 6414 6359 6341 6341
q10 8289 3058 3048 3048
q11 440 211 215 211
q12 388 227 226 226
q13 18006 3383 3371 3371
q14 241 227 218 218
q15 544 508 504 504
q16 456 411 407 407
q17 943 603 488 488
q18 7278 6644 6640 6640
q19 1532 1515 1502 1502
q20 753 305 311 305
q21 2702 2384 2270 2270
q22 368 312 305 305
Total cold run time: 107258 ms
Total hot run time: 38411 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4936 4889 4939 4889
q2 339 242 251 242
q3 3297 3307 3280 3280
q4 2120 2029 2030 2029
q5 5784 5797 5794 5794
q6 202 116 117 116
q7 2297 1920 1879 1879
q8 3368 3416 3450 3416
q9 8783 8724 8714 8714
q10 3783 3842 3826 3826
q11 571 439 448 439
q12 807 610 623 610
q13 5469 3198 3147 3147
q14 299 264 274 264
q15 537 502 503 502
q16 561 510 489 489
q17 1877 1843 1831 1831
q18 8660 8263 8171 8171
q19 1612 1613 1610 1610
q20 2166 1954 1959 1954
q21 5561 5335 5284 5284
q22 515 457 453 453
Total cold run time: 63544 ms
Total hot run time: 58939 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1880676125
<details>
<summary>TPC-DS: <b>Total hot run time: 183709 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1c6d66b32efa23fe3a0b7973ea7f07634aa0974f, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 832 352 340 340
query2 4038 2040 2096 2040
query3 4227 231 219 219
query4 25149 22456 22416 22416
query5 4406 592 637 592
query6 318 222 196 196
query7 3955 308 307 307
query8 232 230 219 219
query9 7155 2905 2823 2823
query10 527 333 308 308
query11 16288 15548 15723 15548
query12 150 118 101 101
query13 1428 344 343 343
query14 11275 7711 7571 7571
query15 255 212 207 207
query16 6085 342 329 329
query17 1347 516 512 512
query18 1841 311 302 302
query19 259 169 159 159
query20 102 96 97 96
query21 167 94 94 94
query22 4934 4874 5000 4874
query23 31721 31527 31099 31099
query24 11343 2975 2932 2932
query25 586 366 356 356
query26 1463 159 167 159
query27 2787 315 313 313
query28 6655 2129 2119 2119
query29 1511 402 399 399
query30 266 143 148 143
query31 1035 797 828 797
query32 143 90 71 71
query33 782 348 335 335
query34 872 462 459 459
query35 975 891 856 856
query36 1413 1285 1207 1207
query37 180 83 81 81
query38 3460 3361 3357 3357
query39 1350 1296 1327 1296
query40 268 90 88 88
query41 39 36 35 35
query42 112 103 105 103
query43 545 509 542 509
query44 1178 818 815 815
query45 216 209 204 204
query46 1101 706 697 697
query47 1729 1675 1677 1675
query48 370 284 295 284
query49 960 308 324 308
query50 739 363 315 315
query51 5409 5324 5257 5257
query52 118 96 99 96
query53 234 166 160 160
query54 1069 674 732 674
query55 102 94 103 94
query56 295 244 277 244
query57 1051 987 1004 987
query58 295 287 284 284
query59 3104 2829 2896 2829
query60 327 307 299 299
query61 151 154 155 154
query62 567 457 456 456
query63 185 174 189 174
query64 5199 1784 1736 1736
query65 3391 3315 3328 3315
query66 1324 383 409 383
query67 15783 15426 15255 15255
query68 12237 547 561 547
query69 615 320 302 302
query70 1706 1638 1693 1638
query71 554 293 279 279
query72 5712 3466 3493 3466
query73 2429 338 335 335
query74 7418 6527 6531 6527
query75 5237 2302 2274 2274
query76 6318 1115 1162 1115
query77 773 300 332 300
query78 9143 8581 8641 8581
query79 1073 529 526 526
query80 563 345 338 338
query81 467 211 214 211
query82 230 101 105 101
query83 189 139 138 138
query84 255 56 55 55
query85 931 293 272 272
query86 409 399 389 389
query87 3644 3484 3428 3428
query88 3231 2518 2502 2502
query89 359 276 279 276
query90 2006 253 242 242
query91 122 96 99 96
query92 91 79 80 79
query93 1155 430 484 430
query94 952 239 223 223
query95 577 508 452 452
query96 633 329 337 329
query97 4345 4222 4192 4192
query98 237 219 220 219
query99 1197 842 883 842
Total cold run time: 282181 ms
Total hot run time: 183709 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882502660
<details>
<summary>TPC-H: <b>Total hot run time: 38618 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41e1e6baffd9436a19fe012b9a4121bf77a96769, data reload: false
------ Round 1 ----------------------------------
q1 17736 4953 4920 4920
q2 2025 158 143 143
q3 10804 1146 1112 1112
q4 10528 737 751 737
q5 7822 2971 2971 2971
q6 202 123 123 123
q7 909 496 507 496
q8 9477 2037 2019 2019
q9 8290 6390 6371 6371
q10 8295 3080 3004 3004
q11 425 219 205 205
q12 395 224 237 224
q13 18003 3379 3347 3347
q14 243 209 204 204
q15 554 508 500 500
q16 514 428 408 408
q17 963 748 646 646
q18 7329 6748 6697 6697
q19 1593 1524 1519 1519
q20 699 333 312 312
q21 2763 2381 2357 2357
q22 359 303 304 303
Total cold run time: 109928 ms
Total hot run time: 38618 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4958 4885 4907 4885
q2 337 227 244 227
q3 3274 3301 3270 3270
q4 2098 2035 2002 2002
q5 5778 5771 5769 5769
q6 205 115 112 112
q7 2312 1866 1900 1866
q8 3368 3454 3432 3432
q9 8830 8737 8719 8719
q10 3802 3826 3834 3826
q11 540 443 459 443
q12 780 631 638 631
q13 3860 3168 3160 3160
q14 294 269 298 269
q15 543 511 497 497
q16 542 481 523 481
q17 1863 1870 1845 1845
q18 8637 8277 8296 8277
q19 1587 1638 1624 1624
q20 2223 1964 1932 1932
q21 5560 5155 5218 5155
q22 502 450 434 434
Total cold run time: 61893 ms
Total hot run time: 58856 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1886005429
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884669777
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 2a58aa200f5f5b2044207be279fb145fdac45622 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.4 seconds inserted 10000000 Rows, about 694K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1885026489
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.44 seconds
stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
storage size: 17184208160 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882320981
<details>
<summary>TPC-H: <b>Total hot run time: 38367 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f6314b56c02b8cd19bb7d48e23979812dcaf426b, data reload: false
------ Round 1 ----------------------------------
q1 17718 4913 4916 4913
q2 2022 153 141 141
q3 10613 1167 1188 1167
q4 10542 784 801 784
q5 7796 2931 2842 2842
q6 213 122 121 121
q7 899 500 480 480
q8 9278 1988 1980 1980
q9 6403 6379 6365 6365
q10 8215 2999 3011 2999
q11 434 220 204 204
q12 386 228 237 228
q13 17997 3375 3371 3371
q14 233 213 208 208
q15 565 509 509 509
q16 443 380 406 380
q17 942 708 599 599
q18 7232 6615 6600 6600
q19 1566 1533 1518 1518
q20 753 332 297 297
q21 2731 2364 2383 2364
q22 346 297 314 297
Total cold run time: 107327 ms
Total hot run time: 38367 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4907 4901 4884 4884
q2 341 230 246 230
q3 3283 3300 3280 3280
q4 2103 2027 2027 2027
q5 5786 5753 5754 5753
q6 207 115 116 115
q7 2290 1873 1869 1869
q8 3312 3422 3421 3421
q9 8746 8684 8711 8684
q10 3761 3841 3817 3817
q11 549 449 448 448
q12 796 633 627 627
q13 5572 3195 3151 3151
q14 294 276 260 260
q15 547 500 509 500
q16 552 479 489 479
q17 1858 1843 1857 1843
q18 8586 8368 8237 8237
q19 1575 1627 1618 1618
q20 2199 1954 1966 1954
q21 5552 5054 5151 5054
q22 513 446 487 446
Total cold run time: 63329 ms
Total hot run time: 58697 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882485075
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884585980
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1882525862
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.31 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 26.6 seconds inserted 10000000 Rows, about 375K ops/s
storage size: 17183858279 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [runtimeFilter](nereids) expand runtime filter by join condition by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29633:
URL: https://github.com/apache/doris/pull/29633#issuecomment-1884640869
<details>
<summary>TPC-H: <b>Total hot run time: 38477 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2a58aa200f5f5b2044207be279fb145fdac45622, data reload: false
------ Round 1 ----------------------------------
q1 17937 5024 4958 4958
q2 2040 151 138 138
q3 10695 1054 1169 1054
q4 10241 815 805 805
q5 7812 3191 3145 3145
q6 201 124 126 124
q7 872 499 501 499
q8 9248 1933 1996 1933
q9 7377 6478 6422 6422
q10 8234 3099 3048 3048
q11 422 212 213 212
q12 355 196 196 196
q13 18113 3440 3475 3440
q14 237 217 210 210
q15 551 519 501 501
q16 436 379 403 379
q17 937 533 474 474
q18 7288 6680 6630 6630
q19 1602 1406 1316 1316
q20 573 301 282 282
q21 2794 2429 2402 2402
q22 375 332 309 309
Total cold run time: 108340 ms
Total hot run time: 38477 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4941 5008 4988 4988
q2 320 200 199 199
q3 3335 3313 3324 3313
q4 2241 2240 2240 2240
q5 5815 5797 5820 5797
q6 195 117 119 117
q7 2319 1839 1850 1839
q8 3461 3551 3575 3551
q9 8935 8907 8899 8899
q10 3762 3847 3832 3832
q11 551 430 436 430
q12 789 592 603 592
q13 5731 3244 3189 3189
q14 283 253 261 253
q15 565 522 509 509
q16 512 465 477 465
q17 2057 2033 2071 2033
q18 8831 8420 8319 8319
q19 1636 1618 1588 1588
q20 2170 1939 1931 1931
q21 6070 5634 5746 5634
q22 593 507 493 493
Total cold run time: 65112 ms
Total hot run time: 60211 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org