You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2024/01/10 07:38:43 UTC
[PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Jibing-Li opened a new pull request, #29786:
URL: https://github.com/apache/doris/pull/29786
Alter column stats operation need to write bdbje, so it should be forwarded to master to execute.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884368860
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.91 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
storage size: 17183859464 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884333791
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884393587
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit ce18e8e6f70912c2ff7b58676de0b485dbf595e2 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.0 seconds inserted 10000000 Rows, about 714K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884338762
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884338796
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884367799
<details>
<summary>TPC-H: <b>Total hot run time: 38592 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ce18e8e6f70912c2ff7b58676de0b485dbf595e2, data reload: false
------ Round 1 ----------------------------------
q1 18053 5039 5020 5020
q2 2041 144 136 136
q3 10680 1109 1130 1109
q4 10544 767 839 767
q5 7862 3080 3165 3080
q6 201 125 125 125
q7 868 500 488 488
q8 9255 1964 2001 1964
q9 6543 6467 6412 6412
q10 8246 3070 3036 3036
q11 428 212 218 212
q12 363 198 207 198
q13 18121 3398 3416 3398
q14 244 213 210 210
q15 552 519 505 505
q16 432 384 388 384
q17 995 499 540 499
q18 7300 6688 6680 6680
q19 1565 1360 1383 1360
q20 566 299 316 299
q21 2782 2384 2394 2384
q22 358 328 326 326
Total cold run time: 107999 ms
Total hot run time: 38592 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4973 4947 4975 4947
q2 309 199 203 199
q3 3338 3297 3292 3292
q4 2238 2247 2230 2230
q5 5778 5758 5784 5758
q6 192 118 121 118
q7 2290 1884 1839 1839
q8 3442 3559 3563 3559
q9 8855 8872 8794 8794
q10 3738 3783 3811 3783
q11 554 449 417 417
q12 766 597 606 597
q13 5597 3190 3185 3185
q14 292 260 274 260
q15 573 508 512 508
q16 514 466 470 466
q17 2043 2032 2073 2032
q18 8754 8313 8372 8313
q19 1608 1615 1614 1614
q20 2152 1947 1917 1917
q21 6060 5658 5716 5658
q22 590 487 500 487
Total cold run time: 64656 ms
Total hot run time: 59973 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884389535
<details>
<summary>ClickBench: <b>Total hot run time: 30.9 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ce18e8e6f70912c2ff7b58676de0b485dbf595e2, data reload: false
query1 0.07 0.06 0.06
query2 0.06 0.03 0.02
query3 0.25 0.12 0.12
query4 1.75 0.13 0.12
query5 0.53 0.51 0.53
query6 1.40 0.64 0.62
query7 0.01 0.01 0.01
query8 0.04 0.03 0.03
query9 0.56 0.49 0.50
query10 0.56 0.55 0.54
query11 0.13 0.10 0.09
query12 0.12 0.09 0.10
query13 0.60 0.60 0.60
query14 0.79 0.78 0.81
query15 0.83 0.80 0.80
query16 0.36 0.36 0.34
query17 0.98 0.98 0.96
query18 0.25 0.26 0.24
query19 1.81 1.80 1.73
query20 0.01 0.00 0.01
query21 15.40 0.57 0.58
query22 3.13 2.88 1.97
query23 17.51 0.77 0.70
query24 2.49 3.01 1.00
query25 2.00 0.15 0.15
query26 0.14 0.14 0.14
query27 0.14 0.15 0.15
query28 11.01 0.81 0.79
query29 12.66 3.15 3.15
query30 0.60 0.50 0.50
query31 2.77 0.35 0.35
query32 3.38 0.48 0.50
query33 3.25 3.19 3.23
query34 15.77 4.20 4.16
query35 4.18 4.15 4.15
query36 1.11 1.03 1.05
query37 0.07 0.05 0.05
query38 0.03 0.02 0.02
query39 0.02 0.01 0.02
query40 0.16 0.14 0.14
query41 0.07 0.02 0.01
query42 0.03 0.02 0.02
query43 0.03 0.02 0.02
Total cold run time: 107.06 s
Total hot run time: 30.9 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29786:
URL: https://github.com/apache/doris/pull/29786#issuecomment-1884382211
<details>
<summary>TPC-DS: <b>Total hot run time: 179123 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ce18e8e6f70912c2ff7b58676de0b485dbf595e2, data reload: false
query1 942 342 331 331
query2 6708 1889 1766 1766
query3 6704 219 212 212
query4 26294 22160 22164 22160
query5 5111 528 523 523
query6 280 185 187 185
query7 4620 279 285 279
query8 241 206 208 206
query9 9192 2753 2834 2753
query10 483 246 276 246
query11 16223 15404 15285 15285
query12 136 80 78 78
query13 1632 344 349 344
query14 12056 7294 7313 7294
query15 257 189 194 189
query16 6574 283 270 270
query17 1904 502 484 484
query18 1930 302 267 267
query19 301 148 148 148
query20 86 81 78 78
query21 189 98 87 87
query22 5239 4867 5140 4867
query23 32136 31194 31301 31194
query24 12085 2883 2849 2849
query25 565 338 340 338
query26 1563 156 165 156
query27 2788 279 286 279
query28 7006 1910 1902 1902
query29 1612 407 398 398
query30 281 145 147 145
query31 969 782 787 782
query32 97 69 55 55
query33 732 270 279 270
query34 893 448 450 448
query35 876 776 747 747
query36 1421 1372 1273 1273
query37 103 75 72 72
query38 3426 3356 3289 3289
query39 1328 1290 1289 1289
query40 303 94 94 94
query41 39 35 35 35
query42 98 92 101 92
query43 515 519 509 509
query44 1088 706 715 706
query45 215 187 186 186
query46 1072 667 680 667
query47 1749 1648 1541 1541
query48 406 321 325 321
query49 1237 326 334 326
query50 756 337 388 337
query51 5478 5249 5259 5249
query52 90 88 85 85
query53 218 151 154 151
query54 1383 585 609 585
query55 104 92 86 86
query56 219 196 197 196
query57 1000 971 993 971
query58 235 209 220 209
query59 2749 2450 2431 2431
query60 268 238 237 237
query61 80 80 80 80
query62 638 449 467 449
query63 179 162 150 150
query64 5826 1661 1725 1661
query65 3350 3280 3266 3266
query66 1323 344 326 326
query67 15852 15427 15643 15427
query68 10457 548 518 518
query69 574 286 297 286
query70 1770 1592 1624 1592
query71 500 220 234 220
query72 4712 2813 2854 2813
query73 2107 322 316 316
query74 7028 6481 6573 6481
query75 4729 2274 2327 2274
query76 6366 1029 1135 1029
query77 687 281 258 258
query78 9614 8775 8524 8524
query79 1032 505 497 497
query80 576 344 359 344
query81 453 213 223 213
query82 227 88 97 88
query83 165 145 137 137
query84 267 57 58 57
query85 975 291 283 283
query86 392 383 369 369
query87 3569 3444 3435 3435
query88 3162 2319 2316 2316
query89 321 279 252 252
query90 1879 203 203 203
query91 166 134 139 134
query92 68 58 51 51
query93 1433 502 431 431
query94 805 187 188 187
query95 463 435 411 411
query96 632 332 331 331
query97 4320 4182 4174 4174
query98 212 189 190 189
query99 1139 799 818 799
Total cold run time: 292253 ms
Total hot run time: 179123 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Fix alter column stats not forward to master bug. [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #29786:
URL: https://github.com/apache/doris/pull/29786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org