You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2026/10/12 08:55:03 UTC

[PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

HappenLee opened a new pull request, #68961:
URL: https://github.com/apache/doris/pull/68961

   ### What problem does this PR solve?
   
   Related PR: #63258
   
   Remote broadcast channels share the same `PBlock`, but brpc normally serializes its large `column_values` into a separate request buffer for every destination. Broadcasting an S-byte block to N destinations therefore copies roughly N*S bytes while preparing the protobuf requests.
   
   Encode the `block` or repeated `blocks` field once in `BroadcastPBlockHolder`, before publishing it to the destination queues. Each RPC serializes its routing fields and appends references to the cached encoding through `brpc::SerializedRequest`. The receiver continues to parse the existing `PTransmitDataParams` wire format. Queue order, packet sequences, EOS, and per-RPC callback ownership are preserved.
   
   The original PBlock remains available to the existing send paths. Queue admission accounts for both it and the cache once per holder. Single-destination broadcasts and oversized fragments retain the original path; assembled requests retain required-field validation and the protobuf size limit. The HTTP borrowed-data fix remains in #63258.
   
   The earlier standalone prototype measured packing CPU at 1 MiB x 8 destinations dropping from 3370.81 us to 402.43 us. This is motivation for the change, not an end-to-end measurement of this production patch. Network bytes and receiver processing are unchanged.
   
   ### Release note
   
   Reduce repeated protobuf serialization and payload copies when broadcasting exchange blocks to multiple remote destinations.
   
   ### Check List (For Author)
   
   - Test
       - [x] Unit Test: `broadcast_protobuf_test.cpp` covers wire equivalence, shared backing buffers, batching/EOS, the actual send branch, queue accounting, size limits, and asynchronous RPC completion/cancellation.
       - ASAN: `ENABLE_PCH=OFF ENABLE_UNITY_BUILD=ON ./run-be-ut.sh -j 48 --run --filter='*BroadcastProtobufTest*:*ExchangeSinkOperatorTest*:*ExchangeSinkTest*'` — 24 tests passed (14 new and 10 existing).
       - `build-support/clang-format.sh` / `build-support/check-format.sh` with clang-format 16.0.6; `build-support/check-build-hygiene.sh` and `git diff --check` passed.
       - `build-support/run-clang-tidy.sh`: all changed production lines and the entire new test file passed. The local LLVM 20 package needed an explicit resource-directory argument.
   - Behavior changed:
       - [x] No. SQL results and the exchange wire protocol are unchanged.
   - Does this need documentation?
       - [x] No.
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6120977098

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6124985179

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 151861 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 70c4f63fe54db120cbc8dc482baa9887d527f2bb, data reload: false
   
   query5	4315	604	444	444
   query6	432	228	200	200
   query7	4811	443	239	239
   query8	325	184	166	166
   query9	8782	4024	4022	4022
   query10	465	311	242	242
   query11	6004	3410	3190	3190
   query12	144	87	83	83
   query13	1247	441	317	317
   query14	6528	4781	4506	4506
   query14_1	4324	4231	4299	4231
   query15	208	195	180	180
   query16	937	429	413	413
   query17	851	656	542	542
   query18	2465	423	318	318
   query19	194	172	133	133
   query20	83	78	79	78
   query21	208	142	117	117
   query22	13055	13006	12867	12867
   query23	13156	12593	12233	12233
   query23_1	12224	12355	12267	12267
   query24	7116	859	460	460
   query24_1	481	486	478	478
   query25	540	418	364	364
   query26	1247	262	139	139
   query27	2775	439	277	277
   query28	4636	1950	1939	1939
   query29	1555	590	467	467
   query30	300	218	186	186
   query31	894	771	664	664
   query32	138	94	105	94
   query33	520	306	259	259
   query34	1001	849	502	502
   query35	749	765	657	657
   query36	820	799	762	762
   query37	132	107	94	94
   query38	1805	1763	1702	1702
   query39	704	724	657	657
   query39_1	652	671	651	651
   query40	222	118	97	97
   query41	69	72	67	67
   query42	99	82	84	82
   query43	366	377	325	325
   query44	1289	729	702	702
   query45	183	176	167	167
   query46	849	975	573	573
   query47	2935	2961	2854	2854
   query48	305	312	225	225
   query49	582	410	302	302
   query50	710	286	215	215
   query51	10461	10125	10141	10125
   query52	81	89	75	75
   query53	196	208	155	155
   query54	251	203	192	192
   query55	75	72	68	68
   query56	244	224	226	224
   query57	1624	1582	1548	1548
   query58	299	267	252	252
   query59	2360	2376	2142	2142
   query60	316	242	229	229
   query61	172	168	199	168
   query62	386	343	284	284
   query63	196	162	165	162
   query64	2770	931	779	779
   query65	3404	3362	3355	3355
   query66	1801	412	299	299
   query67	20038	20317	19932	19932
   query68	3008	1012	631	631
   query69	384	285	249	249
   query70	920	819	823	819
   query71	291	233	206	206
   query72	2762	2562	2061	2061
   query73	521	526	290	290
   query74	4587	4460	4335	4335
   query75	2342	2300	1957	1957
   query76	2294	998	593	593
   query77	364	390	300	300
   query78	9039	8971	8439	8439
   query79	993	916	524	524
   query80	767	431	387	387
   query81	584	325	276	276
   query82	579	143	114	114
   query83	291	198	174	174
   query84	322	129	99	99
   query85	970	424	414	414
   query86	341	241	234	234
   query87	1996	1991	1830	1830
   query88	3623	2712	2690	2690
   query89	339	301	253	253
   query90	1696	188	178	178
   query91	165	147	124	124
   query92	98	84	85	84
   query93	1010	984	571	571
   query94	503	314	284	284
   query95	580	343	384	343
   query96	680	524	235	235
   query97	2459	2446	2296	2296
   query98	161	148	140	140
   query99	781	762	631	631
   Total cold run time: 234252 ms
   Total hot run time: 151861 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6123202119

   The earlier standalone prototype reported in this PR measured sender-side protobuf request packing with a **1 MiB payload broadcast to 8 destinations**:
   
   | Measurement | Before | Shared encoded payload |
   | --- | ---: | ---: |
   | Packing CPU time | 3370.81 µs | 402.43 µs |
   
   This is approximately **8.38× faster**, or an **88.06% reduction** in packing CPU time. The mechanism is to encode the block field once and append shared buffer references for each destination, avoiding repeated payload encoding/copies.
   
   These numbers come from the earlier standalone prototype; they are not a benchmark of the production patch at `70c4f63fe54db120cbc8dc482baa9887d527f2bb`. End-to-end query throughput/latency gains have not yet been measured. Network bytes and receiver processing are unchanged, so the packing speedup should not be interpreted as an equivalent query speedup.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange merged PR #68961:
URL: https://github.com/apache/doris/pull/68961


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6166748481

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6123184652

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6127640969

   # BE Regression && UT Coverage Report
   Increment line coverage `100.00% (10/10)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/68961_70c4f63fe54db120cbc8dc482baa9887d527f2bb_merge/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/68961_70c4f63fe54db120cbc8dc482baa9887d527f2bb_merge/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 76.90% (35455/46103) |
   | Line Coverage     | 62.12% (400854/645261) |
   | Region Coverage   | 58.47% (337957/577973) |
   | Branch Coverage   | 59.41% (155702/262079) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6124815018

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 29282 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 70c4f63fe54db120cbc8dc482baa9887d527f2bb, data reload: false
   
   ------ Round 1 ----------------------------------
   ============================================
   q1	17609	5766	5588	5588
   q2	2098	311	253	253
   q3	10253	1482	846	846
   q4	4679	489	345	345
   q5	7439	816	550	550
   q6	188	191	158	158
   q7	776	801	619	619
   q8	9252	1312	1095	1095
   q9	5699	4589	4471	4471
   q10	6849	1283	1012	1012
   q11	421	259	239	239
   q12	631	399	297	297
   q13	18046	3060	2379	2379
   q14	282	266	244	244
   q15	q16	729	729	680	680
   q17	1195	734	580	580
   q18	7083	6221	6177	6177
   q19	1109	982	609	609
   q20	405	343	227	227
   q21	5709	2590	2653	2590
   q22	421	391	323	323
   Total cold run time: 100873 ms
   Total hot run time: 29282 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   ============================================
   q1	6648	6640	6766	6640
   q2	731	629	552	552
   q3	5542	5766	5234	5234
   q4	2144	2184	1639	1639
   q5	5845	5699	5990	5699
   q6	243	196	147	147
   q7	2212	1998	1812	1812
   q8	3210	2780	2756	2756
   q9	8236	8153	8051	8051
   q10	3658	3601	3245	3245
   q11	604	419	384	384
   q12	670	696	504	504
   q13	2754	3079	2371	2371
   q14	282	281	258	258
   q15	q16	669	691	644	644
   q17	7813	6957	6856	6856
   q18	13121	12260	13140	12260
   q19	924	810	804	804
   q20	2177	2168	1952	1952
   q21	6068	4983	4986	4983
   q22	510	453	399	399
   Total cold run time: 74061 ms
   Total hot run time: 67190 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6125066216

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 24.73 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 70c4f63fe54db120cbc8dc482baa9887d527f2bb, data reload: false
   
   query1	0.01	0.01	0.01
   query2	0.10	0.05	0.05
   query3	0.27	0.12	0.12
   query4	1.62	0.12	0.12
   query5	0.27	0.25	0.25
   query6	1.16	0.69	0.67
   query7	0.04	0.01	0.01
   query8	0.06	0.04	0.04
   query9	0.42	0.36	0.35
   query10	0.57	0.59	0.56
   query11	0.21	0.14	0.15
   query12	0.18	0.14	0.15
   query13	0.50	0.50	0.51
   query14	0.98	0.97	0.95
   query15	0.64	0.62	0.60
   query16	0.34	0.36	0.36
   query17	1.20	1.15	1.15
   query18	0.22	0.20	0.20
   query19	2.08	2.01	2.06
   query20	0.02	0.01	0.01
   query21	15.49	0.30	0.15
   query22	4.48	0.05	0.04
   query23	16.22	0.32	0.12
   query24	3.12	0.65	0.56
   query25	0.11	0.07	0.06
   query26	1.86	0.24	0.14
   query27	0.05	0.04	0.03
   query28	2.50	0.63	0.39
   query29	12.51	4.14	3.31
   query30	0.28	0.13	0.13
   query31	2.78	0.57	0.35
   query32	3.22	0.60	0.51
   query33	3.22	3.20	3.25
   query34	15.61	3.98	3.43
   query35	3.35	3.31	3.37
   query36	0.55	0.43	0.48
   query37	0.10	0.07	0.06
   query38	0.04	0.04	0.03
   query39	0.03	0.02	0.03
   query40	0.17	0.14	0.14
   query41	0.08	0.03	0.03
   query42	0.04	0.03	0.02
   query43	0.04	0.04	0.03
   Total cold run time: 96.74 s
   Total hot run time: 24.73 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6121075684

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](be) Share encoded protobuf blocks across broadcast RPCs [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68961:
URL: https://github.com/apache/doris/pull/68961#issuecomment-6166749241

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org