You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2026/10/12 08:55:03 UTC
[PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
HappenLee opened a new pull request, #68961:
URL: https://github.com/apache/doris/pull/68961
### What problem does this PR solve?
Related PR: #63258
Remote broadcast channels share the same `PBlock`, but brpc normally serializes its large `column_values` into a separate request buffer for every destination. Broadcasting an S-byte block to N destinations therefore copies roughly N*S bytes while preparing the protobuf requests.
Encode the `block` or repeated `blocks` field once in `BroadcastPBlockHolder`, before publishing it to the destination queues. Each RPC serializes its routing fields and appends references to the cached encoding through `brpc::SerializedRequest`. The receiver continues to parse the existing `PTransmitDataParams` wire format. Queue order, packet sequences, EOS, and per-RPC callback ownership are preserved.
The original PBlock remains available to the existing send paths. Queue admission accounts for both it and the cache once per holder. Single-destination broadcasts and oversized fragments retain the original path; assembled requests retain required-field validation and the protobuf size limit. The HTTP borrowed-data fix remains in #63258.
The earlier standalone prototype measured packing CPU at 1 MiB x 8 destinations dropping from 3370.81 us to 402.43 us. This is motivation for the change, not an end-to-end measurement of this production patch. Network bytes and receiver processing are unchanged.
### Release note
Reduce repeated protobuf serialization and payload copies when broadcasting exchange blocks to multiple remote destinations.
### Check List (For Author)
- Test
- [x] Unit Test: `broadcast_protobuf_test.cpp` covers wire equivalence, shared backing buffers, batching/EOS, the actual send branch, queue accounting, size limits, and asynchronous RPC completion/cancellation.
- ASAN: `ENABLE_PCH=OFF ENABLE_UNITY_BUILD=ON ./run-be-ut.sh -j 48 --run --filter='*BroadcastProtobufTest*:*ExchangeSinkOperatorTest*:*ExchangeSinkTest*'` — 24 tests passed (14 new and 10 existing).
- `build-support/clang-format.sh` / `build-support/check-format.sh` with clang-format 16.0.6; `build-support/check-build-hygiene.sh` and `git diff --check` passed.
- `build-support/run-clang-tidy.sh`: all changed production lines and the entire new test file passed. The local LLVM 20 package needed an explicit resource-directory argument.
- Behavior changed:
- [x] No. SQL results and the exchange wire protocol are unchanged.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6120977098
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6124985179
<details>
<summary>TPC-DS: <b>Total hot run time: 151861 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 70c4f63fe54db120cbc8dc482baa9887d527f2bb, data reload: false
query5 4315 604 444 444
query6 432 228 200 200
query7 4811 443 239 239
query8 325 184 166 166
query9 8782 4024 4022 4022
query10 465 311 242 242
query11 6004 3410 3190 3190
query12 144 87 83 83
query13 1247 441 317 317
query14 6528 4781 4506 4506
query14_1 4324 4231 4299 4231
query15 208 195 180 180
query16 937 429 413 413
query17 851 656 542 542
query18 2465 423 318 318
query19 194 172 133 133
query20 83 78 79 78
query21 208 142 117 117
query22 13055 13006 12867 12867
query23 13156 12593 12233 12233
query23_1 12224 12355 12267 12267
query24 7116 859 460 460
query24_1 481 486 478 478
query25 540 418 364 364
query26 1247 262 139 139
query27 2775 439 277 277
query28 4636 1950 1939 1939
query29 1555 590 467 467
query30 300 218 186 186
query31 894 771 664 664
query32 138 94 105 94
query33 520 306 259 259
query34 1001 849 502 502
query35 749 765 657 657
query36 820 799 762 762
query37 132 107 94 94
query38 1805 1763 1702 1702
query39 704 724 657 657
query39_1 652 671 651 651
query40 222 118 97 97
query41 69 72 67 67
query42 99 82 84 82
query43 366 377 325 325
query44 1289 729 702 702
query45 183 176 167 167
query46 849 975 573 573
query47 2935 2961 2854 2854
query48 305 312 225 225
query49 582 410 302 302
query50 710 286 215 215
query51 10461 10125 10141 10125
query52 81 89 75 75
query53 196 208 155 155
query54 251 203 192 192
query55 75 72 68 68
query56 244 224 226 224
query57 1624 1582 1548 1548
query58 299 267 252 252
query59 2360 2376 2142 2142
query60 316 242 229 229
query61 172 168 199 168
query62 386 343 284 284
query63 196 162 165 162
query64 2770 931 779 779
query65 3404 3362 3355 3355
query66 1801 412 299 299
query67 20038 20317 19932 19932
query68 3008 1012 631 631
query69 384 285 249 249
query70 920 819 823 819
query71 291 233 206 206
query72 2762 2562 2061 2061
query73 521 526 290 290
query74 4587 4460 4335 4335
query75 2342 2300 1957 1957
query76 2294 998 593 593
query77 364 390 300 300
query78 9039 8971 8439 8439
query79 993 916 524 524
query80 767 431 387 387
query81 584 325 276 276
query82 579 143 114 114
query83 291 198 174 174
query84 322 129 99 99
query85 970 424 414 414
query86 341 241 234 234
query87 1996 1991 1830 1830
query88 3623 2712 2690 2690
query89 339 301 253 253
query90 1696 188 178 178
query91 165 147 124 124
query92 98 84 85 84
query93 1010 984 571 571
query94 503 314 284 284
query95 580 343 384 343
query96 680 524 235 235
query97 2459 2446 2296 2296
query98 161 148 140 140
query99 781 762 631 631
Total cold run time: 234252 ms
Total hot run time: 151861 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6123202119
The earlier standalone prototype reported in this PR measured sender-side protobuf request packing with a **1 MiB payload broadcast to 8 destinations**:
| Measurement | Before | Shared encoded payload |
| --- | ---: | ---: |
| Packing CPU time | 3370.81 µs | 402.43 µs |
This is approximately **8.38× faster**, or an **88.06% reduction** in packing CPU time. The mechanism is to encode the block field once and append shared buffer references for each destination, avoiding repeated payload encoding/copies.
These numbers come from the earlier standalone prototype; they are not a benchmark of the production patch at `70c4f63fe54db120cbc8dc482baa9887d527f2bb`. End-to-end query throughput/latency gains have not yet been measured. Network bytes and receiver processing are unchanged, so the packing speedup should not be interpreted as an equivalent query speedup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange merged PR #68961:
URL: https://github.com/apache/doris/pull/68961
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6166748481
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6123184652
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6127640969
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (10/10)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/68961_70c4f63fe54db120cbc8dc482baa9887d527f2bb_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/68961_70c4f63fe54db120cbc8dc482baa9887d527f2bb_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 76.90% (35455/46103) |
| Line Coverage | 62.12% (400854/645261) |
| Region Coverage | 58.47% (337957/577973) |
| Branch Coverage | 59.41% (155702/262079) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6124815018
<details>
<summary>TPC-H: <b>Total hot run time: 29282 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 70c4f63fe54db120cbc8dc482baa9887d527f2bb, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17609 5766 5588 5588
q2 2098 311 253 253
q3 10253 1482 846 846
q4 4679 489 345 345
q5 7439 816 550 550
q6 188 191 158 158
q7 776 801 619 619
q8 9252 1312 1095 1095
q9 5699 4589 4471 4471
q10 6849 1283 1012 1012
q11 421 259 239 239
q12 631 399 297 297
q13 18046 3060 2379 2379
q14 282 266 244 244
q15 q16 729 729 680 680
q17 1195 734 580 580
q18 7083 6221 6177 6177
q19 1109 982 609 609
q20 405 343 227 227
q21 5709 2590 2653 2590
q22 421 391 323 323
Total cold run time: 100873 ms
Total hot run time: 29282 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6648 6640 6766 6640
q2 731 629 552 552
q3 5542 5766 5234 5234
q4 2144 2184 1639 1639
q5 5845 5699 5990 5699
q6 243 196 147 147
q7 2212 1998 1812 1812
q8 3210 2780 2756 2756
q9 8236 8153 8051 8051
q10 3658 3601 3245 3245
q11 604 419 384 384
q12 670 696 504 504
q13 2754 3079 2371 2371
q14 282 281 258 258
q15 q16 669 691 644 644
q17 7813 6957 6856 6856
q18 13121 12260 13140 12260
q19 924 810 804 804
q20 2177 2168 1952 1952
q21 6068 4983 4986 4983
q22 510 453 399 399
Total cold run time: 74061 ms
Total hot run time: 67190 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6125066216
<details>
<summary>ClickBench: <b>Total hot run time: 24.73 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 70c4f63fe54db120cbc8dc482baa9887d527f2bb, data reload: false
query1 0.01 0.01 0.01
query2 0.10 0.05 0.05
query3 0.27 0.12 0.12
query4 1.62 0.12 0.12
query5 0.27 0.25 0.25
query6 1.16 0.69 0.67
query7 0.04 0.01 0.01
query8 0.06 0.04 0.04
query9 0.42 0.36 0.35
query10 0.57 0.59 0.56
query11 0.21 0.14 0.15
query12 0.18 0.14 0.15
query13 0.50 0.50 0.51
query14 0.98 0.97 0.95
query15 0.64 0.62 0.60
query16 0.34 0.36 0.36
query17 1.20 1.15 1.15
query18 0.22 0.20 0.20
query19 2.08 2.01 2.06
query20 0.02 0.01 0.01
query21 15.49 0.30 0.15
query22 4.48 0.05 0.04
query23 16.22 0.32 0.12
query24 3.12 0.65 0.56
query25 0.11 0.07 0.06
query26 1.86 0.24 0.14
query27 0.05 0.04 0.03
query28 2.50 0.63 0.39
query29 12.51 4.14 3.31
query30 0.28 0.13 0.13
query31 2.78 0.57 0.35
query32 3.22 0.60 0.51
query33 3.22 3.20 3.25
query34 15.61 3.98 3.43
query35 3.35 3.31 3.37
query36 0.55 0.43 0.48
query37 0.10 0.07 0.06
query38 0.04 0.04 0.03
query39 0.03 0.02 0.03
query40 0.17 0.14 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.02
query43 0.04 0.04 0.03
Total cold run time: 96.74 s
Total hot run time: 24.73 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6121075684
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6166749241
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org