You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "gavinchou (via GitHub)" <gi...@apache.org> on 2024/06/06 15:21:08 UTC
[PR] [opt][ddl] Do not check number replicas when create table in cloud mode [doris]
gavinchou opened a new pull request, #36002:
URL: https://github.com/apache/doris/pull/36002
When creating OLAP tables, this is a check `getReplicaQuotaLeftWithLock` which acquires the locks of the entire database all contained tables consuming a lot of time, however, **cloud mode does not rely on alive replicas when creating OLAP tables**, we can eliminate the costly operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt][ddl] Do not check number replicas when create table in cloud mode [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2152804790
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt][ddl] Do not check number replicas when create table in cloud mode [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2152803341
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt][ddl] Do not check number replicas when create table in cloud mode [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2152945059
<details>
<summary>TPC-DS: <b>Total hot run time: 172449 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ebdc1f0e3d42af51107f2028aa86cdd148046534, data reload: false
query1 927 388 373 373
query2 6445 2408 2328 2328
query3 6641 203 204 203
query4 19173 17172 17316 17172
query5 4103 444 452 444
query6 247 160 156 156
query7 4606 307 287 287
query8 320 276 285 276
query9 8411 2380 2340 2340
query10 454 306 268 268
query11 10549 10276 10039 10039
query12 128 95 84 84
query13 1629 358 354 354
query14 9392 6825 7520 6825
query15 230 182 186 182
query16 7808 250 260 250
query17 1527 549 526 526
query18 1913 284 279 279
query19 202 155 154 154
query20 95 89 85 85
query21 219 142 129 129
query22 4426 4010 3968 3968
query23 33860 33080 33182 33080
query24 11141 2858 2861 2858
query25 639 370 379 370
query26 1150 155 156 155
query27 2309 321 330 321
query28 6487 2035 2036 2035
query29 899 617 625 617
query30 293 151 157 151
query31 977 764 731 731
query32 100 55 57 55
query33 761 297 283 283
query34 920 467 474 467
query35 824 611 617 611
query36 1073 964 917 917
query37 134 68 72 68
query38 2883 2775 2737 2737
query39 867 778 786 778
query40 214 122 125 122
query41 56 52 56 52
query42 116 95 97 95
query43 576 539 521 521
query44 1199 731 750 731
query45 201 168 161 161
query46 1059 718 726 718
query47 1894 1780 1830 1780
query48 395 301 292 292
query49 1010 399 416 399
query50 779 381 391 381
query51 6695 6710 6682 6682
query52 110 90 93 90
query53 351 292 279 279
query54 860 455 441 441
query55 74 72 71 71
query56 273 250 265 250
query57 1122 1057 1073 1057
query58 267 232 240 232
query59 3318 3224 3159 3159
query60 285 290 270 270
query61 91 87 88 87
query62 621 451 444 444
query63 322 286 283 283
query64 8930 2206 1696 1696
query65 3163 3145 3104 3104
query66 863 333 331 331
query67 15392 15007 15113 15007
query68 4558 555 521 521
query69 568 360 332 332
query70 1145 1115 1098 1098
query71 415 273 271 271
query72 7050 5486 5567 5486
query73 755 337 324 324
query74 6154 5540 5693 5540
query75 3339 2661 2620 2620
query76 2637 967 897 897
query77 652 296 287 287
query78 10440 9861 9899 9861
query79 2637 524 509 509
query80 1050 465 461 461
query81 561 220 221 220
query82 1397 105 110 105
query83 255 169 174 169
query84 260 88 86 86
query85 1531 284 257 257
query86 506 315 300 300
query87 3293 3044 3073 3044
query88 4215 2431 2417 2417
query89 488 394 374 374
query90 1842 183 188 183
query91 128 103 97 97
query92 64 49 47 47
query93 2372 515 496 496
query94 1226 195 185 185
query95 407 305 303 303
query96 595 267 267 267
query97 3163 2990 3024 2990
query98 220 204 200 200
query99 1185 861 811 811
Total cold run time: 270106 ms
Total hot run time: 172449 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](ddl) Do not check number replicas when create table in cloud mode [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2153694166
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](ddl) Do not check number replicas when create table in cloud mode [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2153694208
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt][ddl] Do not check number replicas when create table in cloud mode [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2152954519
<details>
<summary>ClickBench: <b>Total hot run time: 30.74 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ebdc1f0e3d42af51107f2028aa86cdd148046534, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.65 0.08 0.08
query5 0.51 0.50 0.49
query6 1.12 0.73 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.54 0.50 0.50
query10 0.55 0.53 0.55
query11 0.15 0.11 0.11
query12 0.15 0.12 0.12
query13 0.59 0.58 0.60
query14 0.78 0.77 0.78
query15 0.84 0.83 0.81
query16 0.34 0.37 0.37
query17 0.96 1.05 0.96
query18 0.20 0.27 0.22
query19 1.86 1.75 1.69
query20 0.02 0.01 0.01
query21 15.73 0.67 0.66
query22 4.34 6.88 2.06
query23 18.32 1.35 1.29
query24 2.13 0.23 0.23
query25 0.16 0.07 0.09
query26 0.25 0.17 0.18
query27 0.09 0.08 0.08
query28 13.21 1.02 1.01
query29 13.38 3.34 3.27
query30 0.24 0.06 0.05
query31 2.86 0.39 0.38
query32 3.30 0.47 0.46
query33 2.85 2.93 2.94
query34 17.08 4.42 4.42
query35 4.47 4.47 4.57
query36 0.70 0.46 0.46
query37 0.18 0.15 0.15
query38 0.15 0.14 0.15
query39 0.04 0.04 0.03
query40 0.16 0.14 0.15
query41 0.10 0.05 0.05
query42 0.06 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 110.53 s
Total hot run time: 30.74 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt][ddl] Do not check number replicas when create table in cloud mode [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36002:
URL: https://github.com/apache/doris/pull/36002#issuecomment-2152922675
<details>
<summary>TPC-H: <b>Total hot run time: 41604 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ebdc1f0e3d42af51107f2028aa86cdd148046534, data reload: false
------ Round 1 ----------------------------------
q1 17628 4397 4268 4268
q2 2030 196 194 194
q3 10449 1269 1084 1084
q4 10183 829 787 787
q5 7491 2664 2733 2664
q6 222 136 139 136
q7 987 617 628 617
q8 9249 2130 2111 2111
q9 9419 6681 6742 6681
q10 9219 3841 3883 3841
q11 470 237 248 237
q12 500 226 221 221
q13 18398 3194 3228 3194
q14 262 217 222 217
q15 514 467 464 464
q16 488 386 391 386
q17 1001 642 670 642
q18 8390 8016 7813 7813
q19 5871 1433 1519 1433
q20 651 335 308 308
q21 5500 3977 4060 3977
q22 393 329 334 329
Total cold run time: 119315 ms
Total hot run time: 41604 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4630 4448 4429 4429
q2 368 272 258 258
q3 3130 3007 2886 2886
q4 1911 1642 1661 1642
q5 5503 5521 5555 5521
q6 226 127 125 125
q7 2163 1845 1825 1825
q8 3221 3396 3373 3373
q9 8600 8688 8766 8688
q10 4081 3818 3659 3659
q11 593 487 507 487
q12 811 634 646 634
q13 17228 3109 3117 3109
q14 331 258 279 258
q15 513 490 463 463
q16 515 435 415 415
q17 1847 1530 1500 1500
q18 8108 7526 7341 7341
q19 1692 1614 1485 1485
q20 2100 1804 1800 1800
q21 8766 4780 4732 4732
q22 598 546 544 544
Total cold run time: 76935 ms
Total hot run time: 55174 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](ddl) Do not check number replicas when create table in cloud mode [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #36002:
URL: https://github.com/apache/doris/pull/36002
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org