You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2026/01/26 09:14:47 UTC
[PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
liaoxin01 opened a new pull request, #60238:
URL: https://github.com/apache/doris/pull/60238
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Change ProducerType from SINGLE to MULTI in TaskDisruptor to fix concurrent event publishing issue. Multiple thrift-server-pool threads can submit export tasks simultaneously, but SINGLE producer mode is not thread-safe and causes events to be lost when concurrent publishing occurs.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60238:
URL: https://github.com/apache/doris/pull/60238#issuecomment-3798665104
<details>
<summary>TPC-H: <b>Total hot run time: 32659 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9525312d7f7cce9f10db4d673b07ca6bfd2a900, data reload: false
------ Round 1 ----------------------------------
q1 17649 5234 5050 5050
q2 2030 308 228 228
q3 10196 1291 744 744
q4 10204 875 322 322
q5 7572 2168 1901 1901
q6 200 180 151 151
q7 913 706 625 625
q8 9258 1375 1059 1059
q9 5169 4815 4812 4812
q10 6778 1934 1524 1524
q11 521 292 277 277
q12 335 385 222 222
q13 17765 4058 3250 3250
q14 235 234 213 213
q15 887 830 810 810
q16 688 683 624 624
q17 637 761 508 508
q18 6712 6407 7319 6407
q19 1253 1033 680 680
q20 431 353 243 243
q21 2937 2425 2007 2007
q22 1138 1100 1002 1002
Total cold run time: 103508 ms
Total hot run time: 32659 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5569 5569 5562 5562
q2 275 358 257 257
q3 2360 2835 2727 2727
q4 1415 1833 1415 1415
q5 4626 4479 4597 4479
q6 230 193 142 142
q7 2176 1904 1818 1818
q8 2513 2577 2456 2456
q9 7509 7470 7440 7440
q10 2853 3066 2583 2583
q11 556 472 453 453
q12 670 733 622 622
q13 3996 4245 3227 3227
q14 286 280 267 267
q15 834 804 794 794
q16 655 743 633 633
q17 1066 1291 1271 1271
q18 7623 7364 7464 7364
q19 821 743 769 743
q20 1975 2054 1901 1901
q21 4436 4269 3993 3993
q22 1067 1028 962 962
Total cold run time: 53511 ms
Total hot run time: 51109 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60238:
URL: https://github.com/apache/doris/pull/60238#issuecomment-3815937976
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60238:
URL: https://github.com/apache/doris/pull/60238#issuecomment-3799214156
# FE Regression Coverage Report
Increment line coverage ` 100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60238_c9525312d7f7cce9f10db4d673b07ca6bfd2a900_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60238_c9525312d7f7cce9f10db4d673b07ca6bfd2a900_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #60238:
URL: https://github.com/apache/doris/pull/60238#issuecomment-3798585094
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60238:
URL: https://github.com/apache/doris/pull/60238#issuecomment-3798732688
<details>
<summary>ClickBench: <b>Total hot run time: 28.19 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9525312d7f7cce9f10db4d673b07ca6bfd2a900, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.04 0.04
query3 0.25 0.08 0.08
query4 1.60 0.11 0.11
query5 0.27 0.26 0.24
query6 1.18 0.69 0.66
query7 0.04 0.03 0.02
query8 0.05 0.04 0.04
query9 0.58 0.50 0.50
query10 0.56 0.55 0.54
query11 0.14 0.10 0.10
query12 0.14 0.11 0.10
query13 0.63 0.61 0.63
query14 1.05 1.05 1.05
query15 0.87 0.86 0.88
query16 0.37 0.39 0.39
query17 1.12 1.12 1.13
query18 0.23 0.22 0.21
query19 2.10 1.92 2.04
query20 0.02 0.02 0.02
query21 15.41 0.27 0.14
query22 5.23 0.05 0.05
query23 16.22 0.29 0.11
query24 1.51 0.23 1.78
query25 0.12 0.06 0.20
query26 0.16 0.15 0.14
query27 0.07 0.06 0.06
query28 5.82 1.12 0.97
query29 12.57 4.01 3.19
query30 0.28 0.13 0.11
query31 2.81 0.63 0.41
query32 3.26 0.59 0.49
query33 3.21 3.19 3.28
query34 16.33 5.38 4.72
query35 4.79 4.82 4.83
query36 0.65 0.50 0.48
query37 0.11 0.07 0.06
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.19 0.17 0.15
query41 0.09 0.04 0.03
query42 0.04 0.03 0.02
query43 0.05 0.04 0.03
Total cold run time: 100.39 s
Total hot run time: 28.19 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60238:
URL: https://github.com/apache/doris/pull/60238#issuecomment-3815938165
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](scheduler) Fix TaskDisruptor event loss due to wrong ProducerType [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 merged PR #60238:
URL: https://github.com/apache/doris/pull/60238
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org