You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "deardeng (via GitHub)" <gi...@apache.org> on 2024/09/04 08:20:47 UTC
[PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
deardeng opened a new pull request, #40371:
URL: https://github.com/apache/doris/pull/40371
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2328220519
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2333575756
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2333534534
<details>
<summary>TPC-H: <b>Total hot run time: 38015 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2b0075b9affa992b7f04eb8f40df88ab4f55ed8, data reload: false
------ Round 1 ----------------------------------
q1 17612 4548 4366 4366
q2 2031 191 185 185
q3 11713 933 1084 933
q4 10466 797 722 722
q5 7721 2858 2806 2806
q6 226 139 140 139
q7 968 631 605 605
q8 9314 2059 2090 2059
q9 6954 6555 6536 6536
q10 7012 2274 2253 2253
q11 461 249 255 249
q12 394 220 224 220
q13 18933 3066 3077 3066
q14 272 247 255 247
q15 529 489 489 489
q16 530 442 435 435
q17 985 737 760 737
q18 7352 6907 6888 6888
q19 1405 976 949 949
q20 689 358 334 334
q21 4017 3040 2789 2789
q22 1117 1008 1031 1008
Total cold run time: 110701 ms
Total hot run time: 38015 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4447 4343 4461 4343
q2 382 280 279 279
q3 2909 2727 2654 2654
q4 1905 1689 1655 1655
q5 5752 5721 5783 5721
q6 242 148 144 144
q7 2243 1836 1852 1836
q8 3332 3502 3507 3502
q9 8957 8868 8989 8868
q10 3674 3511 3414 3414
q11 620 525 536 525
q12 870 693 716 693
q13 14411 3360 3487 3360
q14 343 303 306 303
q15 554 509 509 509
q16 582 541 501 501
q17 1858 1607 1546 1546
q18 8335 7887 8040 7887
q19 1812 1457 1548 1457
q20 2190 1903 1947 1903
q21 5674 5500 5498 5498
q22 1198 1112 1030 1030
Total cold run time: 72290 ms
Total hot run time: 57628 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou merged PR #40371:
URL: https://github.com/apache/doris/pull/40371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2333575664
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #40371:
URL: https://github.com/apache/doris/pull/40371#discussion_r1743692877
##########
fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudTabletRebalancer.java:
##########
@@ -502,7 +502,7 @@ private void completeRouteInfo() {
Map<String, List<Long>> clusterToBackends =
((CloudReplica) replica).getClusterToBackends();
if (!clusterToBackends.containsKey(cluster)) {
- long beId = ((CloudReplica) replica).hashReplicaToBe(cluster, true);
+ long beId = ((CloudReplica) replica).hashReplicaToBe(cluster, true, true);
Review Comment:
add comment to (`true`, `true`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #40371:
URL: https://github.com/apache/doris/pull/40371#discussion_r1743790863
##########
fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudReplica.java:
##########
@@ -216,20 +219,40 @@ private long getBackendIdImpl(String cluster) {
long backendId = clusterToBackends.get(clusterId).get(0);
Backend be = Env.getCurrentSystemInfo().getBackend(backendId);
if (be != null && be.isQueryAvailable()) {
+ // be normal
if (LOG.isDebugEnabled()) {
LOG.debug("backendId={} ", backendId);
}
+ secondaryBeAndChangeTimeWithCluster.remove(clusterId);
return backendId;
}
+ // be abnormal but use secondary
+ if (!Config.enable_immediate_be_assign && be != null) {
+ Long secondaryBe = -1L;
+ if (!secondaryBeAndChangeTimeWithCluster.containsKey(clusterId)) {
+ secondaryBe = hashReplicaToBe(clusterId, false, false);
+ secondaryBeAndChangeTimeWithCluster.put(clusterId,
+ Pair.of(secondaryBe, System.currentTimeMillis()));
+ } else {
+ secondaryBe = secondaryBeAndChangeTimeWithCluster.get(clusterId).first;
+ Long changeTimestamp = secondaryBeAndChangeTimeWithCluster.get(clusterId).second;
+ if (System.currentTimeMillis() - changeTimestamp > Config.secondary_be_validity_seconds * 1000L) {
+ // set secondary be to tablet after secondary_be_validity_seconds
+ setBeToTablet(clusterId, secondaryBe);
+ secondaryBeAndChangeTimeWithCluster.remove(clusterId);
+ }
+ }
Review Comment:
seems unnecessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "deardeng (via GitHub)" <gi...@apache.org>.
deardeng commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2333482782
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #40371:
URL: https://github.com/apache/doris/pull/40371#discussion_r1743794389
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -3030,6 +3030,12 @@ public static int metaServiceRpcRetryTimes() {
@ConfField(mutable = true, description = {"存算分离模式下是否开启大事务提交，默认false"})
public static boolean enable_cloud_txn_lazy_commit = false;
+ @ConfField(mutable = true, description = {"存算分离模式下，当tablet分布的be异常，是否立即映射tablet到新的be上，默认true"})
+ public static boolean enable_immediate_be_assign = true;
Review Comment:
when should we set it false? what's the benefit and penalty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2333556285
<details>
<summary>TPC-DS: <b>Total hot run time: 192006 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2b0075b9affa992b7f04eb8f40df88ab4f55ed8, data reload: false
query1 1256 902 922 902
query2 6315 1859 1942 1859
query3 10601 4002 3704 3704
query4 59454 23849 23203 23203
query5 5503 500 480 480
query6 416 177 158 158
query7 5953 301 281 281
query8 313 217 223 217
query9 9121 2479 2480 2479
query10 508 260 275 260
query11 18192 15089 15224 15089
query12 156 102 107 102
query13 1624 372 378 372
query14 11205 7266 7739 7266
query15 245 184 175 175
query16 7510 465 465 465
query17 1140 565 560 560
query18 2064 309 296 296
query19 283 149 154 149
query20 129 121 112 112
query21 209 113 106 106
query22 4832 4359 4363 4359
query23 34175 33355 33366 33355
query24 5909 2839 2823 2823
query25 564 408 399 399
query26 692 160 158 158
query27 1782 280 279 279
query28 3603 2038 2007 2007
query29 712 433 425 425
query30 234 157 154 154
query31 925 763 764 763
query32 90 54 57 54
query33 417 296 289 289
query34 870 478 465 465
query35 837 719 730 719
query36 1088 930 937 930
query37 151 87 84 84
query38 3982 3867 3890 3867
query39 1455 1397 1404 1397
query40 194 119 117 117
query41 49 50 44 44
query42 122 96 95 95
query43 541 475 469 469
query44 1082 756 745 745
query45 198 168 167 167
query46 1105 799 757 757
query47 1916 1788 1820 1788
query48 371 294 301 294
query49 778 447 456 447
query50 819 412 423 412
query51 7120 6813 6854 6813
query52 98 89 88 88
query53 252 179 181 179
query54 568 482 465 465
query55 81 73 78 73
query56 303 298 262 262
query57 1182 1093 1079 1079
query58 228 234 230 230
query59 3055 2813 2727 2727
query60 296 281 287 281
query61 151 141 204 141
query62 743 673 623 623
query63 222 179 184 179
query64 2798 709 655 655
query65 3193 3109 3171 3109
query66 636 330 345 330
query67 15286 15295 15240 15240
query68 4387 567 562 562
query69 414 276 270 270
query70 1189 1087 1043 1043
query71 354 279 276 276
query72 6559 4023 4033 4023
query73 742 323 324 323
query74 9262 8826 8789 8789
query75 3359 2689 2714 2689
query76 1668 1062 1062 1062
query77 570 323 317 317
query78 9739 8967 9869 8967
query79 1034 533 519 519
query80 921 520 528 520
query81 565 232 226 226
query82 241 142 140 140
query83 225 147 150 147
query84 265 75 74 74
query85 974 282 282 282
query86 372 304 292 292
query87 4424 4222 4160 4160
query88 3285 2285 2274 2274
query89 385 289 283 283
query90 1907 188 188 188
query91 119 99 101 99
query92 57 49 48 48
query93 1074 529 535 529
query94 798 297 300 297
query95 342 248 253 248
query96 583 259 260 259
query97 3213 3068 3101 3068
query98 211 199 189 189
query99 1636 1272 1290 1272
Total cold run time: 308776 ms
Total hot run time: 192006 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](cloud) Add a policy when be abnormal, tablet delay switch be [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40371:
URL: https://github.com/apache/doris/pull/40371#issuecomment-2333566153
<details>
<summary>ClickBench: <b>Total hot run time: 31.2 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2b0075b9affa992b7f04eb8f40df88ab4f55ed8, data reload: false
query1 0.05 0.05 0.04
query2 0.07 0.03 0.04
query3 0.23 0.05 0.05
query4 1.69 0.08 0.07
query5 0.51 0.50 0.49
query6 1.14 0.73 0.72
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.54 0.49 0.48
query10 0.54 0.56 0.55
query11 0.15 0.12 0.12
query12 0.15 0.13 0.12
query13 0.60 0.58 0.59
query14 1.42 1.43 1.47
query15 0.84 0.82 0.80
query16 0.36 0.36 0.37
query17 1.03 1.08 0.96
query18 0.20 0.20 0.20
query19 1.84 1.78 1.78
query20 0.01 0.01 0.01
query21 15.43 0.69 0.68
query22 4.19 7.39 1.79
query23 18.28 1.37 1.27
query24 2.20 0.22 0.23
query25 0.15 0.09 0.08
query26 0.26 0.18 0.17
query27 0.08 0.08 0.07
query28 13.17 1.01 1.00
query29 12.63 3.31 3.29
query30 0.24 0.06 0.05
query31 2.89 0.40 0.38
query32 3.26 0.47 0.48
query33 3.01 3.01 3.00
query34 16.99 4.38 4.44
query35 4.43 4.44 4.48
query36 0.66 0.47 0.47
query37 0.18 0.15 0.15
query38 0.16 0.15 0.14
query39 0.05 0.04 0.04
query40 0.16 0.13 0.12
query41 0.09 0.05 0.05
query42 0.06 0.06 0.05
query43 0.04 0.04 0.05
Total cold run time: 110.05 s
Total hot run time: 31.2 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org