You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2024/06/13 11:46:59 UTC
[PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
cambyzju opened a new pull request, #36270:
URL: https://github.com/apache/doris/pull/36270
## Proposed changes
Issue Number: close https://github.com/apache/doris/issues/36072
<!--Describe your changes.-->
In pr https://github.com/apache/doris/pull/33630, we already support auto aggregation for random distributed table.
But if all keys are `NOT NULL`, the aggregation will be eliminated by the rule: `ELIMINATE_GROUP_BY`
This pr we prevent `ELIMINATE_GROUP_BY` do not work for random distributed table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2165429661
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju merged PR #36270:
URL: https://github.com/apache/doris/pull/36270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2165539312
<details>
<summary>TPC-DS: <b>Total hot run time: 173333 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 276cade9c5ef56de34feb10f991e467dbeaa23aa, data reload: false
query1 808 381 383 381
query2 6396 2500 2410 2410
query3 6402 209 210 209
query4 21504 17109 17377 17109
query5 3780 487 470 470
query6 248 167 156 156
query7 4454 311 304 304
query8 338 297 292 292
query9 8373 2453 2456 2453
query10 581 306 274 274
query11 10528 10142 9964 9964
query12 139 83 84 83
query13 1566 365 368 365
query14 8759 6974 6916 6916
query15 244 200 184 184
query16 7239 287 278 278
query17 1397 564 566 564
query18 1913 294 281 281
query19 217 168 169 168
query20 100 91 89 89
query21 206 139 136 136
query22 4566 4352 3994 3994
query23 33590 33068 32986 32986
query24 11109 2877 2808 2808
query25 645 396 379 379
query26 1242 157 157 157
query27 2650 326 336 326
query28 7278 2070 2083 2070
query29 962 681 637 637
query30 274 150 155 150
query31 921 751 746 746
query32 96 57 60 57
query33 774 296 291 291
query34 902 476 475 475
query35 744 626 630 626
query36 1087 918 939 918
query37 164 73 73 73
query38 2868 2732 2737 2732
query39 845 801 813 801
query40 210 139 133 133
query41 56 53 53 53
query42 121 97 101 97
query43 597 581 536 536
query44 1252 751 731 731
query45 199 163 165 163
query46 1075 747 725 725
query47 1868 1754 1801 1754
query48 402 307 308 307
query49 1010 447 426 426
query50 780 403 388 388
query51 6762 6614 6569 6569
query52 104 97 93 93
query53 360 296 293 293
query54 901 475 460 460
query55 76 74 74 74
query56 301 280 286 280
query57 1177 1064 1065 1064
query58 268 262 246 246
query59 3865 3869 3461 3461
query60 316 291 291 291
query61 112 113 109 109
query62 639 443 465 443
query63 321 288 294 288
query64 9068 2408 1851 1851
query65 3259 3095 3076 3076
query66 1117 335 333 333
query67 15572 15007 14874 14874
query68 8396 565 566 565
query69 701 482 546 482
query70 1237 1059 1108 1059
query71 524 274 276 274
query72 8941 5801 5333 5333
query73 1117 333 332 332
query74 5951 5502 5528 5502
query75 4999 2704 2701 2701
query76 4585 930 956 930
query77 811 297 294 294
query78 10581 9752 9646 9646
query79 8523 556 539 539
query80 1098 476 472 472
query81 561 223 221 221
query82 727 107 105 105
query83 343 171 168 168
query84 270 85 86 85
query85 1300 278 318 278
query86 397 323 336 323
query87 3301 3091 3078 3078
query88 5213 2474 2476 2474
query89 554 394 376 376
query90 1989 193 192 192
query91 131 99 100 99
query92 61 51 50 50
query93 7020 519 514 514
query94 1265 194 191 191
query95 403 327 317 317
query96 648 270 270 270
query97 3236 2997 3009 2997
query98 231 202 195 195
query99 1242 872 841 841
Total cold run time: 294053 ms
Total hot run time: 173333 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2165425845
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2167283186
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2165513470
<details>
<summary>TPC-H: <b>Total hot run time: 40079 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 276cade9c5ef56de34feb10f991e467dbeaa23aa, data reload: false
------ Round 1 ----------------------------------
q1 17642 4323 4273 4273
q2 2030 187 188 187
q3 10471 1127 1050 1050
q4 10191 853 770 770
q5 7447 2679 2671 2671
q6 221 137 136 136
q7 967 610 606 606
q8 9219 2056 2029 2029
q9 8921 6461 6462 6461
q10 8963 3698 3700 3698
q11 453 240 231 231
q12 446 224 225 224
q13 17779 2963 2955 2955
q14 275 215 224 215
q15 516 479 475 475
q16 503 376 388 376
q17 960 634 680 634
q18 8364 8104 7759 7759
q19 7176 1401 1430 1401
q20 695 333 355 333
q21 5072 3250 4052 3250
q22 409 345 355 345
Total cold run time: 118720 ms
Total hot run time: 40079 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4577 4377 4437 4377
q2 370 261 270 261
q3 3075 2894 2866 2866
q4 1979 1688 1694 1688
q5 5501 5523 5487 5487
q6 222 129 130 129
q7 2173 1816 1843 1816
q8 3195 3396 3407 3396
q9 8655 8720 8769 8720
q10 4103 3854 3878 3854
q11 605 510 483 483
q12 827 691 630 630
q13 16994 3020 3158 3020
q14 344 258 264 258
q15 521 477 490 477
q16 487 424 445 424
q17 1791 1526 1490 1490
q18 8117 7665 7752 7665
q19 1774 1473 1709 1473
q20 3039 1845 1847 1845
q21 5122 4842 4849 4842
q22 644 532 536 532
Total cold run time: 74115 ms
Total hot run time: 55733 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2165551111
<details>
<summary>ClickBench: <b>Total hot run time: 30.42 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 276cade9c5ef56de34feb10f991e467dbeaa23aa, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.50 0.48 0.47
query6 1.12 0.73 0.72
query7 0.01 0.01 0.02
query8 0.05 0.04 0.04
query9 0.54 0.48 0.49
query10 0.53 0.53 0.56
query11 0.16 0.11 0.12
query12 0.14 0.12 0.12
query13 0.60 0.59 0.60
query14 0.76 0.79 0.77
query15 0.86 0.81 0.82
query16 0.37 0.35 0.36
query17 1.00 0.98 1.02
query18 0.21 0.24 0.24
query19 1.75 1.71 1.78
query20 0.02 0.01 0.01
query21 15.42 0.66 0.65
query22 4.01 7.64 1.89
query23 18.22 1.38 1.26
query24 2.14 0.22 0.22
query25 0.15 0.08 0.08
query26 0.27 0.18 0.18
query27 0.08 0.07 0.08
query28 13.22 1.02 0.99
query29 12.57 3.27 3.22
query30 0.27 0.06 0.07
query31 2.84 0.40 0.39
query32 3.26 0.46 0.47
query33 2.83 2.90 2.88
query34 17.00 4.42 4.43
query35 4.44 4.47 4.60
query36 0.65 0.46 0.47
query37 0.20 0.15 0.15
query38 0.15 0.15 0.15
query39 0.05 0.04 0.03
query40 0.17 0.15 0.14
query41 0.10 0.04 0.04
query42 0.06 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 108.79 s
Total hot run time: 30.42 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](plan) wrong result for random distributed agg table with all keys not null [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36270:
URL: https://github.com/apache/doris/pull/36270#issuecomment-2167283147
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org