You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2023/11/16 07:27:11 UTC
[PR] [feature](nereids) runtime filter prune when column stats are not available #26963 [doris]
englefly opened a new pull request, #27099:
URL: https://github.com/apache/doris/pull/27099
## Proposed changes
prune rf when column stats are not available
print rf in "explain shape plan", both join side and apply side
add regression case to check plan shape/rf/rf prune for tpch_sf1000 (stats are not available)
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1814156148
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 16328717575faa013a51d8e5ea8342c59a756686, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4949 4636 4632 4632
q2 366 184 159 159
q3 2034 1927 1876 1876
q4 1390 1270 1249 1249
q5 3952 3979 4033 3979
q6 247 132 131 131
q7 1437 867 866 866
q8 2751 2783 2778 2778
q9 9841 9663 9598 9598
q10 3471 3508 3536 3508
q11 378 249 247 247
q12 437 294 293 293
q13 4553 3825 3817 3817
q14 316 302 282 282
q15 582 532 525 525
q16 666 591 585 585
q17 1197 992 928 928
q18 7870 7324 7539 7324
q19 1670 1678 1687 1678
q20 527 287 311 287
q21 4411 3910 4021 3910
q22 481 370 382 370
Total cold run time: 53526 ms
Total hot run time: 49022 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4570 4578 4567 4567
q2 343 238 270 238
q3 4022 4018 4001 4001
q4 2711 2695 2690 2690
q5 9636 9572 9659 9572
q6 245 124 124 124
q7 2575 2258 2280 2258
q8 4446 4458 4443 4443
q9 13237 13154 13183 13154
q10 4108 4186 4179 4179
q11 773 644 709 644
q12 976 807 833 807
q13 4274 3568 3579 3568
q14 385 362 340 340
q15 585 527 530 527
q16 752 667 676 667
q17 3871 3821 3863 3821
q18 9607 9121 9207 9121
q19 1835 1773 1750 1750
q20 2403 2068 2038 2038
q21 8882 8440 8676 8440
q22 843 767 786 767
Total cold run time: 81079 ms
Total hot run time: 77716 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1813925500
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1814598385
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1815792721
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.38 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17098950752 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1814834572
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 07f30ad00b971eee4c67c2403d067a634e979fb0, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4883 4655 4619 4619
q2 354 162 158 158
q3 2036 1877 1869 1869
q4 1376 1258 1226 1226
q5 3955 3947 4032 3947
q6 251 131 129 129
q7 1385 876 871 871
q8 2759 2796 2765 2765
q9 9743 9617 9635 9617
q10 3498 3501 3526 3501
q11 370 246 248 246
q12 438 298 299 298
q13 4599 3818 3827 3818
q14 320 300 299 299
q15 593 538 529 529
q16 658 580 585 580
q17 1127 945 910 910
q18 7897 7326 7340 7326
q19 1668 1681 1659 1659
q20 528 330 287 287
q21 4388 3926 4004 3926
q22 481 378 388 378
Total cold run time: 53307 ms
Total hot run time: 48958 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4593 4598 4590 4590
q2 332 231 255 231
q3 4013 4022 4033 4022
q4 2710 2697 2695 2695
q5 9602 9578 9692 9578
q6 248 125 126 125
q7 2583 2266 2308 2266
q8 4397 4437 4440 4437
q9 13191 13104 13221 13104
q10 4069 4177 4165 4165
q11 791 756 654 654
q12 992 814 812 812
q13 4271 3522 3574 3522
q14 379 345 350 345
q15 582 524 527 524
q16 743 670 676 670
q17 3870 3892 3861 3861
q18 9548 9002 8973 8973
q19 1827 1772 1763 1763
q20 2393 2045 2046 2045
q21 8757 8593 8385 8385
q22 869 828 743 743
Total cold run time: 80760 ms
Total hot run time: 77510 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1817315166
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1817352236
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit c51c559e7316c6021edac1eec14307558dead679, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4951 4688 4684 4684
q2 358 158 157 157
q3 2036 1905 1859 1859
q4 1382 1224 1235 1224
q5 3917 3898 4062 3898
q6 257 129 134 129
q7 1430 898 894 894
q8 2762 2790 2763 2763
q9 9687 9500 9423 9423
q10 3444 3515 3514 3514
q11 383 248 256 248
q12 433 295 301 295
q13 4533 3819 3800 3800
q14 320 293 285 285
q15 588 524 522 522
q16 668 594 587 587
q17 1130 974 935 935
q18 7972 7406 7504 7406
q19 1674 1688 1647 1647
q20 539 324 310 310
q21 4400 3984 4028 3984
q22 474 371 372 371
Total cold run time: 53338 ms
Total hot run time: 48935 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4581 4582 4609 4582
q2 338 239 298 239
q3 4019 4046 4010 4010
q4 2712 2710 2743 2710
q5 9584 9598 9567 9567
q6 249 120 126 120
q7 3041 2503 2452 2452
q8 4442 4430 4412 4412
q9 12956 12905 12731 12731
q10 4079 4147 4195 4147
q11 753 699 674 674
q12 978 826 823 823
q13 4301 3597 3571 3571
q14 391 347 357 347
q15 570 521 523 521
q16 730 705 693 693
q17 3925 3842 3907 3842
q18 9804 9206 9325 9206
q19 1811 1779 1788 1779
q20 2383 2076 2052 2052
q21 8960 8711 8524 8524
q22 890 821 799 799
Total cold run time: 81497 ms
Total hot run time: 77801 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1817474609
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1816494701
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1816106649
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.87 seconds
stream load tsv: 587 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17100978901 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1815711788
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1814094284
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.37 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17099541044 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1816668717
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit c500eaadf1ac46050667583455383d96b21d6fc2, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4907 4648 4631 4631
q2 358 153 140 140
q3 2029 1974 1942 1942
q4 1401 1287 1218 1218
q5 3963 3913 3993 3913
q6 254 127 132 127
q7 1444 874 883 874
q8 2761 2771 2761 2761
q9 9574 9385 9436 9385
q10 3438 3495 3521 3495
q11 383 238 237 237
q12 439 301 292 292
q13 4576 3789 3801 3789
q14 324 278 287 278
q15 582 542 524 524
q16 677 585 581 581
q17 1137 947 866 866
q18 7865 7454 7370 7370
q19 1675 1688 1668 1668
q20 547 319 305 305
q21 4423 4014 4009 4009
q22 480 379 381 379
Total cold run time: 53237 ms
Total hot run time: 48784 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4566 4547 4529 4529
q2 336 227 250 227
q3 4003 3988 3991 3988
q4 2712 2697 2700 2697
q5 9539 9605 9524 9524
q6 243 125 123 123
q7 3022 2569 2505 2505
q8 4455 4448 4489 4448
q9 12913 12783 12877 12783
q10 4063 4184 4171 4171
q11 785 662 715 662
q12 991 808 820 808
q13 4295 3566 3514 3514
q14 388 341 346 341
q15 555 514 515 514
q16 721 667 681 667
q17 3843 3812 3825 3812
q18 9510 9147 9076 9076
q19 1789 1762 1805 1762
q20 2388 2065 2039 2039
q21 8861 8614 8635 8614
q22 861 796 766 766
Total cold run time: 80839 ms
Total hot run time: 77570 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1816126422
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit af9393f1a65a85e2b46d22696583eaa2ec596417, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4878 4603 4607 4603
q2 357 147 136 136
q3 2019 1872 1879 1872
q4 1383 1238 1257 1238
q5 3975 3916 4039 3916
q6 247 135 131 131
q7 1386 872 881 872
q8 2769 2809 2777 2777
q9 9761 9723 9607 9607
q10 3466 3524 3534 3524
q11 388 257 254 254
q12 443 288 292 288
q13 4584 3817 3792 3792
q14 323 280 283 280
q15 582 531 523 523
q16 663 575 579 575
q17 1148 976 952 952
q18 7857 7528 7502 7502
q19 1682 1689 1674 1674
q20 574 312 309 309
q21 4348 3972 3993 3972
q22 469 369 374 369
Total cold run time: 53302 ms
Total hot run time: 49166 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4608 4545 4556 4545
q2 344 211 269 211
q3 4023 3982 3993 3982
q4 2706 2695 2694 2694
q5 9654 9618 9652 9618
q6 242 120 124 120
q7 2588 2238 2261 2238
q8 4499 4485 4477 4477
q9 13203 13101 13093 13093
q10 4130 4226 4214 4214
q11 737 635 692 635
q12 981 820 824 820
q13 4273 3529 3579 3529
q14 376 353 351 351
q15 582 508 519 508
q16 724 680 670 670
q17 3850 3841 3769 3769
q18 9593 9156 9169 9156
q19 1847 1778 1784 1778
q20 2390 2057 2021 2021
q21 8770 8628 8546 8546
q22 870 794 772 772
Total cold run time: 80990 ms
Total hot run time: 77747 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #27099:
URL: https://github.com/apache/doris/pull/27099
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1816084758
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1816686174
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.26 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17100040409 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1815764250
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1815821261
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 9423f987d8d8cc83e8eee6aa0a6a17b83f7ee4db, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4914 4696 4658 4658
q2 359 158 154 154
q3 2053 1992 1940 1940
q4 1390 1291 1255 1255
q5 3983 3963 4006 3963
q6 246 130 130 130
q7 1408 875 882 875
q8 2743 2801 2777 2777
q9 9739 9893 9589 9589
q10 3483 3532 3581 3532
q11 377 252 235 235
q12 439 294 300 294
q13 4607 3817 3812 3812
q14 334 293 289 289
q15 587 537 524 524
q16 667 595 588 588
q17 1141 969 982 969
q18 7761 7391 7519 7391
q19 1684 1712 1673 1673
q20 563 337 291 291
q21 4458 3945 3955 3945
q22 469 371 383 371
Total cold run time: 53405 ms
Total hot run time: 49255 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4640 4588 4597 4588
q2 342 231 252 231
q3 4055 4011 4024 4011
q4 2712 2714 2705 2705
q5 9676 9792 9863 9792
q6 241 124 125 124
q7 2585 2282 2256 2256
q8 4460 4478 4498 4478
q9 13315 13225 13164 13164
q10 4118 4182 4192 4182
q11 784 659 692 659
q12 974 809 799 799
q13 4317 3574 3568 3568
q14 385 341 362 341
q15 596 518 515 515
q16 739 665 664 664
q17 3963 3846 3851 3846
q18 9613 9094 9084 9084
q19 1828 1797 1801 1797
q20 2407 2067 2042 2042
q21 8944 8761 8772 8761
q22 881 781 818 781
Total cold run time: 81575 ms
Total hot run time: 78388 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1815930803
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1814844181
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.11 seconds
stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.8 seconds inserted 10000000 Rows, about 335K ops/s
storage size: 17098927560 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1817359347
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.2 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17103598973 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) runtime filter prune when column stats are not available [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27099:
URL: https://github.com/apache/doris/pull/27099#issuecomment-1817474599
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org