You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zclllyybb (via GitHub)" <gi...@apache.org> on 2026/03/01 14:47:20 UTC
[PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
zclllyybb opened a new pull request, #60936:
URL: https://github.com/apache/doris/pull/60936
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
When filtering or clearing block columns, the code incorrectly assumed exclusive ownership and directly mutated shared columns via `assume_mutable()`. This violates Copy-On-Write (COW) semantics and can cause data corruption when columns are shared across multiple operators (e.g., in CTE MultiCast scenarios).
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb merged PR #60936:
URL: https://github.com/apache/doris/pull/60936
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3982251370
# BE Regression && UT Coverage Report
Increment line coverage `86.67% (26/30)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60936_7465baf2080e19c968eba6529013874cdcffaba2_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60936_7465baf2080e19c968eba6529013874cdcffaba2_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.17% (26752/36559) |
| Line Coverage | 56.46% (285025/504825) |
| Region Coverage | 53.82% (237740/441732) |
| Branch Coverage | 55.47% (102593/184942) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3980307236
<details>
<summary>TPC-H: <b>Total hot run time: 28631 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7465baf2080e19c968eba6529013874cdcffaba2, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17640 4458 4278 4278
q2 q3 10646 788 525 525
q4 4685 364 253 253
q5 7535 1202 998 998
q6 173 179 147 147
q7 789 825 672 672
q8 9294 1437 1342 1342
q9 4897 4731 4653 4653
q10 6829 1883 1625 1625
q11 481 269 245 245
q12 750 564 460 460
q13 17774 4211 3393 3393
q14 223 229 204 204
q15 965 792 793 792
q16 746 736 673 673
q17 724 854 420 420
q18 6019 5381 5309 5309
q19 1189 965 603 603
q20 500 488 389 389
q21 4446 1867 1384 1384
q22 336 288 266 266
Total cold run time: 96641 ms
Total hot run time: 28631 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4406 5393 4344 4344
q2 q3 1763 2167 1709 1709
q4 837 1137 768 768
q5 4011 4294 4313 4294
q6 173 169 138 138
q7 1706 1586 1439 1439
q8 2387 2631 2505 2505
q9 7470 7280 7519 7280
q10 2732 2871 2415 2415
q11 526 445 412 412
q12 483 579 461 461
q13 4045 4524 3564 3564
q14 284 328 297 297
q15 864 821 820 820
q16 700 771 697 697
q17 1158 1532 1331 1331
q18 7164 6761 6608 6608
q19 921 891 918 891
q20 2092 2163 2041 2041
q21 3993 3711 3487 3487
q22 441 445 366 366
Total cold run time: 48156 ms
Total hot run time: 45867 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3982316595
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3980131072
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3980369813
# BE UT Coverage Report
Increment line coverage `56.67% (17/30)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/7465baf2080e19c968eba6529013874cdcffaba2_7465baf2080e19c968eba6529013874cdcffaba2/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/7465baf2080e19c968eba6529013874cdcffaba2_7465baf2080e19c968eba6529013874cdcffaba2/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.57% (19628/37337) |
| Line Coverage | 36.18% (183212/506378) |
| Region Coverage | 32.47% (142095/437595) |
| Branch Coverage | 33.43% (61643/184378) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3980327588
<details>
<summary>TPC-DS: <b>Total hot run time: 183904 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7465baf2080e19c968eba6529013874cdcffaba2, data reload: false
query5 4837 645 523 523
query6 324 213 211 211
query7 4213 479 277 277
query8 351 245 237 237
query9 8764 2728 2739 2728
query10 544 387 362 362
query11 17019 16884 16538 16538
query12 193 126 127 126
query13 1274 458 349 349
query14 6566 3203 3022 3022
query14_1 2828 2809 2822 2809
query15 203 189 176 176
query16 991 481 461 461
query17 1073 744 619 619
query18 2640 444 350 350
query19 210 207 186 186
query20 144 143 130 130
query21 222 148 121 121
query22 5303 5709 5520 5520
query23 17574 17183 16996 16996
query23_1 17164 17058 17170 17058
query24 7218 1614 1230 1230
query24_1 1238 1245 1243 1243
query25 580 486 429 429
query26 1238 257 152 152
query27 2776 518 291 291
query28 4477 1899 1863 1863
query29 787 564 472 472
query30 312 242 202 202
query31 872 734 643 643
query32 80 70 66 66
query33 504 319 285 285
query34 914 903 576 576
query35 627 669 596 596
query36 1128 1128 920 920
query37 141 93 85 85
query38 2950 2886 2922 2886
query39 897 853 843 843
query39_1 829 839 813 813
query40 230 154 136 136
query41 65 59 59 59
query42 109 101 100 100
query43 369 381 366 366
query44
query45 204 192 186 186
query46 863 982 598 598
query47 2092 2116 2037 2037
query48 309 310 224 224
query49 654 475 373 373
query50 670 275 207 207
query51 4214 4094 4020 4020
query52 106 105 95 95
query53 290 340 284 284
query54 290 264 262 262
query55 87 82 79 79
query56 337 316 305 305
query57 1359 1336 1290 1290
query58 287 275 273 273
query59 2535 2760 2553 2553
query60 335 337 313 313
query61 144 148 143 143
query62 623 592 538 538
query63 307 274 271 271
query64 4818 1243 970 970
query65
query66 1405 454 358 358
query67 16367 16379 16271 16271
query68
query69 392 307 280 280
query70 1008 985 935 935
query71 340 305 295 295
query72 2728 2638 2400 2400
query73 540 541 317 317
query74 10005 9942 9740 9740
query75 2824 2754 2432 2432
query76 2289 1039 672 672
query77 366 376 305 305
query78 11072 11309 10633 10633
query79 2340 792 605 605
query80 1620 626 535 535
query81 559 280 259 259
query82 991 149 112 112
query83 340 263 240 240
query84 251 121 93 93
query85 879 488 421 421
query86 422 305 307 305
query87 3087 3102 2962 2962
query88 3561 2636 2641 2636
query89 432 368 347 347
query90 2033 178 174 174
query91 162 162 132 132
query92 76 72 68 68
query93 988 846 515 515
query94 642 316 304 304
query95 576 337 319 319
query96 635 507 224 224
query97 2448 2473 2424 2424
query98 252 223 214 214
query99 1003 995 908 908
Total cold run time: 255791 ms
Total hot run time: 183904 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3980131146
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) Fix COW violation in block column operations when filtering and clearing (PART I) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60936:
URL: https://github.com/apache/doris/pull/60936#issuecomment-3983102810
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org