You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2026/10/16 06:51:40 UTC
[PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Gabriel39 opened a new pull request, #69181:
URL: https://github.com/apache/doris/pull/69181
### What problem does this PR solve?
A queued scan task can retain `ScannerContext` after its scan operator has been destroyed. During adaptive scan cleanup, `debug_string()` dereferences the operator-owned `_shared_scan_limit`, causing an ASAN heap-use-after-free. The same diagnostic is also used when the scheduler cannot lock the task execution context.
Remove the borrowed `remaining_limit` value from diagnostics so both paths remain safe after the operator is released. Scan limit enforcement and memory quota cleanup are unchanged. Add unit tests for diagnostics and scheduler submission after owner destruction, and for adaptive cleanup with verbose logging enabled.
### Release note
Fix a use-after-free in scan diagnostics during task cleanup.
### Check List (For Author)
- Test
- [x] Unit Test: add two lifecycle regression cases and update the existing diagnostic-format assertion.
- [x] Manual test: a focused ASAN harness extracting the actual `debug_string()` method reproduces the invalid read before the fix and passes all four combinations of adaptive scanning enabled/disabled and limited/unlimited scans after the fix. The surrounding fields are test stand-ins; this is not a full BE UT run.
- Validation
- Clang syntax checks passed for the modified source and test translation units using regenerated protocol headers and BE test flags.
- clang-format 16 and build hygiene checks passed.
- The full BE UT invocation was blocked before execution by the local third-party installation missing `libsimdutf.a`; the new BE unit tests still require CI execution.
- Behavior changed:
- [x] Yes: diagnostic output no longer includes `remaining_limit`.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6209859958
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6210688777
<details>
<summary>ClickBench: <b>Total hot run time: 24.55 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5, data reload: false
query1 0.00 0.00 0.00
query2 0.09 0.04 0.05
query3 0.27 0.12 0.12
query4 1.61 0.12 0.13
query5 0.27 0.26 0.24
query6 1.17 0.69 0.68
query7 0.04 0.01 0.00
query8 0.05 0.04 0.03
query9 0.42 0.35 0.36
query10 0.56 0.56 0.55
query11 0.20 0.13 0.13
query12 0.18 0.14 0.15
query13 0.49 0.50 0.49
query14 1.03 1.00 1.01
query15 0.63 0.60 0.63
query16 0.34 0.31 0.36
query17 1.14 1.09 1.12
query18 0.22 0.21 0.21
query19 2.09 1.95 2.04
query20 0.02 0.02 0.01
query21 15.52 0.25 0.15
query22 4.50 0.06 0.05
query23 16.25 0.31 0.12
query24 3.31 0.58 0.45
query25 0.11 0.07 0.06
query26 1.54 0.24 0.15
query27 0.04 0.03 0.05
query28 2.31 0.60 0.38
query29 12.49 4.20 3.31
query30 0.29 0.15 0.14
query31 2.78 0.56 0.34
query32 3.23 0.61 0.54
query33 3.26 3.35 3.19
query34 15.56 4.01 3.42
query35 3.39 3.36 3.36
query36 0.55 0.43 0.43
query37 0.10 0.07 0.06
query38 0.05 0.04 0.04
query39 0.04 0.03 0.04
query40 0.17 0.16 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 96.47 s
Total hot run time: 24.55 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6210611700
<details>
<summary>TPC-DS: <b>Total hot run time: 151657 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5, data reload: false
query5 4330 578 453 453
query6 426 197 187 187
query7 4796 451 220 220
query8 321 171 153 153
query9 8818 3919 3924 3919
query10 473 313 255 255
query11 5939 3778 3508 3508
query12 143 90 85 85
query13 1244 459 326 326
query14 6144 4057 3795 3795
query14_1 3549 3507 3502 3502
query15 210 202 189 189
query16 969 445 432 432
query17 869 675 539 539
query18 2430 434 307 307
query19 206 166 134 134
query20 85 95 82 82
query21 214 135 114 114
query22 12985 12979 12792 12792
query23 13657 12982 12653 12653
query23_1 12576 12757 12622 12622
query24 7083 803 464 464
query24_1 464 470 460 460
query25 509 386 333 333
query26 1288 258 143 143
query27 2763 431 263 263
query28 4646 1962 1960 1960
query29 1537 573 426 426
query30 287 218 184 184
query31 1014 915 801 801
query32 139 86 91 86
query33 497 295 220 220
query34 906 846 489 489
query35 735 740 628 628
query36 837 838 751 751
query37 126 94 85 85
query38 1792 1741 1698 1698
query39 806 789 757 757
query39_1 735 723 755 723
query40 217 110 99 99
query41 69 66 65 65
query42 94 87 85 85
query43 367 379 328 328
query44 1303 689 706 689
query45 181 188 165 165
query46 844 946 557 557
query47 3004 2996 2939 2939
query48 294 307 215 215
query49 592 392 297 297
query50 699 278 205 205
query51 10607 10521 10598 10521
query52 76 82 71 71
query53 189 205 151 151
query54 239 191 170 170
query55 68 73 62 62
query56 227 227 197 197
query57 1602 1564 1587 1564
query58 286 267 253 253
query59 2125 2171 1984 1984
query60 276 241 225 225
query61 135 144 143 143
query62 391 339 289 289
query63 199 164 170 164
query64 2702 897 791 791
query65 3295 3245 3250 3245
query66 1778 401 299 299
query67 20323 20290 19947 19947
query68 3272 1000 607 607
query69 386 293 229 229
query70 867 806 755 755
query71 294 226 228 226
query72 2800 2597 1850 1850
query73 537 526 293 293
query74 4607 4483 4289 4289
query75 2402 2276 1923 1923
query76 2395 1022 612 612
query77 357 380 275 275
query78 9681 9589 9118 9118
query79 870 832 535 535
query80 484 402 327 327
query81 523 328 270 270
query82 209 129 103 103
query83 193 194 177 177
query84 292 120 101 101
query85 770 410 365 365
query86 264 242 237 237
query87 2015 1979 1808 1808
query88 3575 2659 2659 2659
query89 329 296 253 253
query90 2161 176 179 176
query91 160 146 127 127
query92 99 89 90 89
query93 995 1001 576 576
query94 450 297 272 272
query95 579 393 317 317
query96 664 541 234 234
query97 2451 2437 2335 2335
query98 163 155 149 149
query99 773 768 657 657
Total cold run time: 234050 ms
Total hot run time: 151657 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6210461174
<details>
<summary>TPC-H: <b>Total hot run time: 30151 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17592 5775 5679 5679
q2 2101 311 260 260
q3 10247 1523 862 862
q4 4683 512 339 339
q5 7426 830 545 545
q6 206 208 156 156
q7 819 818 607 607
q8 9245 1529 1226 1226
q9 5692 4454 4465 4454
q10 6861 1299 1004 1004
q11 465 271 240 240
q12 639 448 285 285
q13 18095 3784 2993 2993
q14 281 265 249 249
q15 q16 746 736 685 685
q17 1346 768 581 581
q18 7086 6249 6218 6218
q19 1248 1096 637 637
q20 406 350 232 232
q21 6041 2586 2688 2586
q22 409 356 313 313
Total cold run time: 101634 ms
Total hot run time: 30151 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6905 6843 6968 6843
q2 764 665 581 581
q3 5695 5737 5294 5294
q4 2128 2340 1582 1582
q5 5776 5572 5953 5572
q6 301 209 148 148
q7 2308 2072 1782 1782
q8 3333 2841 2857 2841
q9 8025 8062 7914 7914
q10 3777 3662 3248 3248
q11 619 411 371 371
q12 677 719 499 499
q13 3331 3683 3037 3037
q14 301 301 260 260
q15 q16 679 685 613 613
q17 8071 7075 6928 6928
q18 13170 12304 13138 12304
q19 1034 846 835 835
q20 2217 2188 1924 1924
q21 6176 4874 4980 4874
q22 521 443 412 412
Total cold run time: 75808 ms
Total hot run time: 67862 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6212596688
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5_49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5_49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 66.34% (31684/47760) |
| Line Coverage | 51.07% (333503/652991) |
| Region Coverage | 46.43% (269461/580390) |
| Branch Coverage | 48.14% (126874/263530) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #69181:
URL: https://github.com/apache/doris/pull/69181
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6212671761
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69181_49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69181_49fb5d3a41de9b4b8df044e0b7ff44b5215e84b5_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 77.19% (35725/46284) |
| Line Coverage | 62.37% (402931/646016) |
| Region Coverage | 58.66% (339223/578306) |
| Branch Coverage | 59.70% (156948/262889) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6209861177
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scan) Avoid reading freed operator state in scanner diagnostics [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #69181:
URL: https://github.com/apache/doris/pull/69181#issuecomment-6209867201
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org