You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "freemandealer (via GitHub)" <gi...@apache.org> on 2026/03/27 14:00:53 UTC
[PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
freemandealer opened a new pull request, #61833:
URL: https://github.com/apache/doris/pull/61833
TabletHotspot memory kept growing by about 1.8-2.7 GB per day because counters for cold tablets remained resident even after they no longer contributed to the hour/day/week hotspot windows. Their history deques kept consuming memory, and the sharded unordered_maps continued to hold expanded bucket arrays without a reclamation path.
Fix this by running a maintenance pass after each hourly rollup to evict counters whose last access is older than the one-week window and whose current/day/week contributions are all zero. When a shard becomes sparse after eviction, rebuild the map to release bucket memory. Also add GC-focused diagnostics and unit tests to verify eviction, recreation, and shard compaction behavior.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4142839016
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4143077091
<details>
<summary>TPC-H: <b>Total hot run time: 26349 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 320b17ddc7576a107c4e90ba67df123e099b6d7d, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17640 4492 4281 4281
q2 q3 10649 778 513 513
q4 4675 348 256 256
q5 7561 1192 1017 1017
q6 183 172 148 148
q7 780 836 674 674
q8 9314 1435 1307 1307
q9 4884 4738 4677 4677
q10 6326 1916 1643 1643
q11 457 261 244 244
q12 765 585 466 466
q13 18028 2758 1956 1956
q14 228 237 207 207
q15 q16 723 744 668 668
q17 716 805 456 456
q18 6272 5415 5175 5175
q19 1110 978 578 578
q20 533 473 378 378
q21 4495 1828 1414 1414
q22 331 291 420 291
Total cold run time: 95670 ms
Total hot run time: 26349 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4675 4735 4709 4709
q2 q3 3955 4411 3888 3888
q4 851 1207 790 790
q5 4051 4346 4281 4281
q6 187 182 144 144
q7 1751 1664 1524 1524
q8 2487 2734 2589 2589
q9 7702 7319 7262 7262
q10 3766 3929 3711 3711
q11 507 440 419 419
q12 542 603 458 458
q13 2581 2890 2013 2013
q14 287 320 287 287
q15 q16 716 781 720 720
q17 1180 1350 1335 1335
q18 7074 6719 6724 6719
q19 933 887 890 887
q20 2142 2166 2069 2069
q21 3966 3554 3465 3465
q22 472 439 405 405
Total cold run time: 49825 ms
Total hot run time: 47675 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4265314477
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen merged PR #61833:
URL: https://github.com/apache/doris/pull/61833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4143788085
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61833_320b17ddc7576a107c4e90ba67df123e099b6d7d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61833_320b17ddc7576a107c4e90ba67df123e099b6d7d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.56% (27158/36920) |
| Line Coverage | 57.12% (292260/511646) |
| Region Coverage | 54.45% (243884/447913) |
| Branch Coverage | 56.14% (105625/188154) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4142839302
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4142874852
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4143149085
<details>
<summary>TPC-DS: <b>Total hot run time: 168768 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 320b17ddc7576a107c4e90ba67df123e099b6d7d, data reload: false
query5 4324 622 514 514
query6 332 216 200 200
query7 4215 475 259 259
query8 349 233 226 226
query9 8708 2721 2735 2721
query10 508 396 321 321
query11 6977 5109 4856 4856
query12 189 125 129 125
query13 1297 447 327 327
query14 5705 3652 3409 3409
query14_1 2823 2773 2745 2745
query15 211 186 172 172
query16 971 502 430 430
query17 995 693 586 586
query18 2444 438 334 334
query19 218 201 178 178
query20 145 127 123 123
query21 220 130 108 108
query22 13184 13862 14654 13862
query23 16601 16334 16124 16124
query23_1 16231 15888 15663 15663
query24 7201 1647 1211 1211
query24_1 1210 1220 1230 1220
query25 576 486 431 431
query26 1246 259 156 156
query27 2767 479 291 291
query28 4533 1858 1843 1843
query29 854 581 504 504
query30 297 231 195 195
query31 1013 943 869 869
query32 82 76 71 71
query33 524 347 295 295
query34 885 865 510 510
query35 632 671 626 626
query36 1109 1144 1003 1003
query37 141 100 81 81
query38 2913 2918 2949 2918
query39 864 838 809 809
query39_1 801 812 789 789
query40 248 161 142 142
query41 69 65 65 65
query42 266 261 263 261
query43 262 260 228 228
query44
query45 202 214 190 190
query46 931 1012 635 635
query47 2146 2205 2080 2080
query48 305 316 227 227
query49 629 458 376 376
query50 698 279 230 230
query51 4211 4056 4058 4056
query52 270 266 260 260
query53 291 346 294 294
query54 312 269 305 269
query55 97 89 86 86
query56 327 331 323 323
query57 1933 1900 1833 1833
query58 292 280 269 269
query59 2827 2997 2773 2773
query60 333 341 325 325
query61 162 153 161 153
query62 629 593 544 544
query63 308 284 282 282
query64 5084 1319 1022 1022
query65
query66 1473 479 358 358
query67 24182 24328 24240 24240
query68
query69 396 315 287 287
query70 991 975 929 929
query71 326 300 290 290
query72 2804 2665 2481 2481
query73 550 536 307 307
query74 9594 9548 9382 9382
query75 2826 2723 2446 2446
query76 2304 1014 673 673
query77 387 397 325 325
query78 10927 11023 10441 10441
query79 3236 745 564 564
query80 1720 626 543 543
query81 584 260 225 225
query82 985 159 119 119
query83 333 267 241 241
query84 276 116 97 97
query85 938 492 454 454
query86 528 302 290 290
query87 3132 3102 3001 3001
query88 3903 2645 2632 2632
query89 428 364 338 338
query90 2093 179 166 166
query91 167 177 137 137
query92 81 76 76 76
query93 2823 814 504 504
query94 647 299 298 298
query95 598 336 386 336
query96 649 508 230 230
query97 2442 2492 2418 2418
query98 240 219 234 219
query99 989 1014 908 908
Total cold run time: 254227 ms
Total hot run time: 168768 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4142829009
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61833:
URL: https://github.com/apache/doris/pull/61833#issuecomment-4143391577
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/320b17ddc7576a107c4e90ba67df123e099b6d7d_320b17ddc7576a107c4e90ba67df123e099b6d7d/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/320b17ddc7576a107c4e90ba67df123e099b6d7d_320b17ddc7576a107c4e90ba67df123e099b6d7d/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.93% (19954/37698) |
| Line Coverage | 36.45% (187065/513194) |
| Region Coverage | 32.72% (145192/443791) |
| Branch Coverage | 33.89% (63575/187588) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61833:
URL: https://github.com/apache/doris/pull/61833#discussion_r3097971264
##########
be/src/cloud/cloud_tablet_hotspot.cpp:
##########
@@ -184,6 +316,103 @@ uint64_t HotspotCounter::qpw() {
return week_history_counter + cur_counter.load();
}
+bool TabletHotspot::is_gc_eligible(const HotspotCounter& counter, SystemTimePoint now) {
+ const auto week_window = std::chrono::seconds((HotspotCounter::week_counters_size + 1) *
+ HotspotCounter::time_interval);
+ return counter.last_access_time < now - week_window &&
+ counter.cur_counter.load(std::memory_order_relaxed) == 0 &&
+ counter.day_history_counter.load(std::memory_order_relaxed) == 0 &&
+ counter.week_history_counter.load(std::memory_order_relaxed) == 0;
+}
+
+TabletHotspot::MaintenanceStats TabletHotspot::run_maintenance_once(SystemTimePoint now) {
+ MaintenanceStats stats;
+ auto gc_elapsed = SteadyClock::duration::zero();
+
+ for (size_t slot_idx = 0; slot_idx < s_slot_size; ++slot_idx) {
+ auto& slot = _tablets_hotspot[slot_idx];
+ std::vector<HotspotCounterPtr> counters;
+ size_t slot_size_before = 0;
+ size_t slot_bucket_count_before = 0;
+
+ {
+ std::lock_guard lock(slot.mtx);
+ slot_size_before = slot.map.size();
+ slot_bucket_count_before = slot.map.bucket_count();
+ stats.total_counters_before_gc += slot_size_before;
+ if (slot_size_before > 0) {
+ ++stats.non_empty_slots_before_gc;
+ }
+ stats.max_slot_size_before_gc =
+ std::max<uint64_t>(stats.max_slot_size_before_gc, slot_size_before);
+ stats.sum_bucket_count_before_gc += slot_bucket_count_before;
+ counters.reserve(slot_size_before);
+ for (auto& [_, counter] : slot.map) {
+ counters.push_back(counter);
+ }
+ }
+ stats.copied_counters += counters.size();
+ std::for_each(counters.begin(), counters.end(),
+ [](HotspotCounterPtr& counter) { counter->make_dot_point(); });
+
+ size_t erased_count = 0;
+ bool compacted = false;
+ size_t slot_size_after = 0;
+ size_t slot_bucket_count_after = 0;
+ auto gc_start = SteadyClock::now();
+ {
+ std::lock_guard lock(slot.mtx);
+ for (auto iter = slot.map.begin(); iter != slot.map.end();) {
+ if (is_gc_eligible(*iter->second, now)) {
+ iter = slot.map.erase(iter);
+ ++erased_count;
+ } else {
+ ++iter;
+ }
Review Comment:
Rebuilding the shard inside this `slot.mtx` critical section will block `TabletHotspot::count()` for every tablet hashing into the same slot. `count()` sits on the cloud scan preparation path (`parallel_scanner_builder.cpp`, `olap_scanner.cpp`, `olap_scan_operator.cpp`), so in the exact sparse-shard scenario this PR is targeting, an hourly GC round can turn into visible query-latency spikes. Please avoid doing the full rehash under the shard mutex and keep the critical section to the minimum state swap/erase work.
##########
be/src/cloud/cloud_tablet_hotspot.cpp:
##########
@@ -184,6 +316,103 @@ uint64_t HotspotCounter::qpw() {
return week_history_counter + cur_counter.load();
}
+bool TabletHotspot::is_gc_eligible(const HotspotCounter& counter, SystemTimePoint now) {
+ const auto week_window = std::chrono::seconds((HotspotCounter::week_counters_size + 1) *
+ HotspotCounter::time_interval);
+ return counter.last_access_time < now - week_window &&
+ counter.cur_counter.load(std::memory_order_relaxed) == 0 &&
+ counter.day_history_counter.load(std::memory_order_relaxed) == 0 &&
+ counter.week_history_counter.load(std::memory_order_relaxed) == 0;
+}
+
+TabletHotspot::MaintenanceStats TabletHotspot::run_maintenance_once(SystemTimePoint now) {
+ MaintenanceStats stats;
+ auto gc_elapsed = SteadyClock::duration::zero();
+
+ for (size_t slot_idx = 0; slot_idx < s_slot_size; ++slot_idx) {
+ auto& slot = _tablets_hotspot[slot_idx];
+ std::vector<HotspotCounterPtr> counters;
+ size_t slot_size_before = 0;
+ size_t slot_bucket_count_before = 0;
+
+ {
+ std::lock_guard lock(slot.mtx);
+ slot_size_before = slot.map.size();
+ slot_bucket_count_before = slot.map.bucket_count();
+ stats.total_counters_before_gc += slot_size_before;
+ if (slot_size_before > 0) {
+ ++stats.non_empty_slots_before_gc;
+ }
+ stats.max_slot_size_before_gc =
+ std::max<uint64_t>(stats.max_slot_size_before_gc, slot_size_before);
+ stats.sum_bucket_count_before_gc += slot_bucket_count_before;
+ counters.reserve(slot_size_before);
+ for (auto& [_, counter] : slot.map) {
+ counters.push_back(counter);
+ }
+ }
+ stats.copied_counters += counters.size();
+ std::for_each(counters.begin(), counters.end(),
+ [](HotspotCounterPtr& counter) { counter->make_dot_point(); });
+
+ size_t erased_count = 0;
+ bool compacted = false;
+ size_t slot_size_after = 0;
+ size_t slot_bucket_count_after = 0;
+ auto gc_start = SteadyClock::now();
+ {
+ std::lock_guard lock(slot.mtx);
+ for (auto iter = slot.map.begin(); iter != slot.map.end();) {
+ if (is_gc_eligible(*iter->second, now)) {
+ iter = slot.map.erase(iter);
+ ++erased_count;
+ } else {
+ ++iter;
+ }
+ }
+
Review Comment:
This new compaction branch allocates a second `unordered_map` for all surviving entries, but there is no `try_reserve()`/fallback handling and `make_dot_point()` has no catch boundary above it. When a hotspot shard is already large enough to need compaction, `reserve()`/`emplace()` here can fail under memory pressure; an uncaught allocation exception on `_counter_thread` will call `std::terminate` and take down the BE. Please make this path fail-safe (reserve with rollback/fallback, or catch and skip compaction/log instead of letting the background thread abort the process).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen closed pull request #61833: [fix](filecache) reclaim expired tablet hotspot counters and compact sparse shards
URL: https://github.com/apache/doris/pull/61833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org