You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/02/29 11:46:12 UTC
[PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
w41ter opened a new pull request, #31620:
URL: https://github.com/apache/doris/pull/31620
## Proposed changes
Issue Number: close #xxx
An patch for brpc is added, to maintain compatibility between the old and new versions of the service, when modifying the package name of the proto file.
This patch adds a service option 'secondary_package_name', and the bRPC will add the service with the two package names.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1972329587
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1972682054
<details>
<summary>ClickBench: <b>Total hot run time: 26.52 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a984ae8ee453d684ee0b538418d2b4bec1c24d43, data reload: false
query1 0.05 0.03 0.04
query2 0.10 0.04 0.04
query3 0.26 0.07 0.06
query4 1.59 0.08 0.08
query5 0.54 0.51 0.49
query6 1.52 0.96 0.98
query7 0.03 0.02 0.02
query8 0.06 0.04 0.04
query9 0.68 0.63 0.64
query10 0.68 0.69 0.68
query11 0.21 0.14 0.15
query12 0.19 0.15 0.15
query13 0.48 0.46 0.46
query14 0.71 0.74 0.68
query15 0.92 0.86 0.87
query16 0.38 0.44 0.42
query17 1.12 1.18 1.20
query18 0.35 0.34 0.33
query19 2.17 2.02 2.05
query20 0.02 0.02 0.02
query21 15.37 0.96 0.96
query22 0.57 0.59 0.56
query23 13.65 0.58 0.63
query24 0.61 0.18 0.20
query25 0.04 0.05 0.05
query26 0.19 0.20 0.20
query27 0.05 0.05 0.04
query28 1.31 0.74 0.69
query29 12.89 4.56 4.66
query30 0.83 0.83 0.73
query31 0.86 0.39 0.39
query32 3.23 0.43 0.40
query33 2.43 2.30 2.10
query34 2.24 2.24 2.23
query35 2.24 2.24 2.23
query36 0.97 0.97 0.98
query37 0.07 0.06 0.06
query38 0.05 0.05 0.04
query39 0.03 0.02 0.03
query40 0.19 0.17 0.15
query41 0.09 0.02 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 70.05 s
Total hot run time: 26.52 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1983135430
TeamCity be ut coverage result:
Function Coverage: 35.40% (8561/24184)
Line Coverage: 27.15% (69521/256026)
Region Coverage: 26.39% (36075/136725)
Branch Coverage: 23.22% (18435/79402)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8f71b769abc5f9115f28a69c5b7cccf29014643f_8f71b769abc5f9115f28a69c5b7cccf29014643f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1978168461
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1970965342
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1972676957
<details>
<summary>TPC-DS: <b>Total hot run time: 189218 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a984ae8ee453d684ee0b538418d2b4bec1c24d43, data reload: false
query1 1024 387 421 387
query2 7903 2315 2340 2315
query3 6730 296 294 294
query4 26613 24243 23634 23634
query5 5479 856 1442 856
query6 369 292 291 291
query7 4922 444 436 436
query8 370 250 260 250
query9 8701 3387 3355 3355
query10 545 339 344 339
query11 18860 19141 20121 19141
query12 340 301 190 190
query13 1695 638 671 638
query14 11441 7844 7842 7842
query15 282 256 253 253
query16 8314 394 376 376
query17 2102 670 663 663
query18 2203 410 410 410
query19 485 246 234 234
query20 149 135 135 135
query21 229 179 159 159
query22 5384 5095 5081 5081
query23 24854 24619 24742 24619
query24 6782 2072 2086 2072
query25 603 507 522 507
query26 774 225 228 225
query27 2347 474 490 474
query28 3225 2784 2807 2784
query29 954 752 745 745
query30 320 192 190 190
query31 1355 833 844 833
query32 130 97 92 92
query33 627 379 393 379
query34 899 648 669 648
query35 939 915 862 862
query36 1274 1163 1175 1163
query37 137 117 108 108
query38 3240 3197 3085 3085
query39 1192 1125 1115 1115
query40 252 165 162 162
query41 87 85 85 85
query42 162 155 169 155
query43 591 573 582 573
query44 1250 1066 1033 1033
query45 220 216 211 211
query46 877 771 787 771
query47 1718 1728 1703 1703
query48 598 516 533 516
query49 732 476 457 457
query50 652 553 547 547
query51 9312 8945 9012 8945
query52 149 147 146 146
query53 329 326 323 323
query54 368 362 351 351
query55 166 125 135 125
query56 354 334 340 334
query57 1088 1068 1049 1049
query58 352 330 326 326
query59 3687 3395 3412 3395
query60 450 403 377 377
query61 165 161 164 161
query62 540 412 419 412
query63 318 317 332 317
query64 4213 1842 1833 1833
query65 2362 2384 2299 2299
query66 898 393 383 383
query67 15999 15162 15235 15162
query68 4185 713 703 703
query69 651 572 559 559
query70 1588 1594 1597 1594
query71 450 413 413 413
query72 2869 2336 2364 2336
query73 656 476 482 476
query74 7487 7165 7438 7165
query75 3088 2758 2681 2681
query76 2113 659 673 659
query77 493 411 402 402
query78 6939 6420 6451 6420
query79 1097 749 781 749
query80 739 596 574 574
query81 481 255 257 255
query82 168 139 135 135
query83 240 226 227 226
query84 273 126 126 126
query85 841 521 515 515
query86 419 427 459 427
query87 3552 3235 3238 3235
query88 3688 3429 3408 3408
query89 438 422 425 422
query90 956 222 235 222
query91 211 204 205 204
query92 86 83 81 81
query93 1029 728 712 712
query94 490 268 263 263
query95 566 528 518 518
query96 625 388 391 388
query97 3052 3030 2994 2994
query98 307 303 292 292
query99 851 733 734 733
Total cold run time: 262309 ms
Total hot run time: 189218 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #31620:
URL: https://github.com/apache/doris/pull/31620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1972664485
<details>
<summary>TPC-H: <b>Total hot run time: 32356 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a984ae8ee453d684ee0b538418d2b4bec1c24d43, data reload: false
------ Round 1 ----------------------------------
q1 17890 4531 4515 4515
q2 2165 228 224 224
q3 10389 1121 1128 1121
q4 4707 1224 1223 1223
q5 7700 2049 2062 2049
q6 151 111 108 108
q7 1189 1103 1113 1103
q8 9507 1040 1129 1040
q9 6408 4976 4903 4903
q10 7863 2006 1947 1947
q11 513 317 310 310
q12 868 425 420 420
q13 18090 3045 2945 2945
q14 312 297 291 291
q15 391 361 356 356
q16 558 514 520 514
q17 754 320 311 311
q18 5811 4851 5161 4851
q19 1177 406 407 406
q20 591 436 427 427
q21 6122 2900 2842 2842
q22 506 456 450 450
Total cold run time: 103662 ms
Total hot run time: 32356 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4637 4631 4592 4592
q2 285 225 222 222
q3 2201 1877 1829 1829
q4 1680 1287 1350 1287
q5 3309 2991 3031 2991
q6 151 105 102 102
q7 1971 1798 1774 1774
q8 1743 1648 1682 1648
q9 5440 5235 5301 5235
q10 2660 2348 2332 2332
q11 479 409 405 405
q12 757 663 652 652
q13 3184 2709 2635 2635
q14 292 286 281 281
q15 367 348 352 348
q16 490 476 480 476
q17 1106 759 755 755
q18 5848 5497 5532 5497
q19 1091 610 614 610
q20 1634 1486 1433 1433
q21 4727 4373 4135 4135
q22 620 598 586 586
Total cold run time: 44672 ms
Total hot run time: 39825 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1972609153
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1972629609
TeamCity be ut coverage result:
Function Coverage: 35.64% (8549/23989)
Line Coverage: 27.44% (69381/252821)
Region Coverage: 26.62% (36006/135282)
Branch Coverage: 23.43% (18395/78514)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a984ae8ee453d684ee0b538418d2b4bec1c24d43_a984ae8ee453d684ee0b538418d2b4bec1c24d43/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1983018004
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #31620:
URL: https://github.com/apache/doris/pull/31620#discussion_r1510574223
##########
cloud/src/common/config.h:
##########
@@ -145,4 +145,7 @@ CONF_Bool(focus_add_kms_data_key, "false");
CONF_Bool(enable_txn_store_retry, "false");
CONF_Int32(txn_store_retry_times, "20");
+// The secondary package name of the MetaService.
+CONF_String(secondary_package_name, "selectdb");
Review Comment:
the default should be empty.
And better to explain why need this config and how to use, for other opensource user who want to expand this service
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31620:
URL: https://github.com/apache/doris/pull/31620#issuecomment-1978953735
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #31620:
URL: https://github.com/apache/doris/pull/31620#discussion_r1507456814
##########
cloud/src/meta-service/meta_server.cpp:
##########
@@ -75,7 +75,11 @@ int MetaServer::start(brpc::Server* server) {
// Add service
auto meta_service = std::make_unique<MetaServiceImpl>(txn_kv_, rc_mgr, rate_limiter);
auto meta_service_proxy = new MetaServiceProxy(std::move(meta_service));
- server->AddService(meta_service_proxy, brpc::SERVER_OWNS_SERVICE);
+
+ brpc::ServiceOptions options;
+ options.ownership = brpc::SERVER_OWNS_SERVICE;
+ options.secondary_package_name = "selectdb"; // the package name of selectdb cloud.
Review Comment:
make it an config option
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org