You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "bobhan1 (via GitHub)" <gi...@apache.org> on 2025/05/13 07:45:15 UTC
[PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
bobhan1 opened a new pull request, #50847:
URL: https://github.com/apache/doris/pull/50847
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #50847:
URL: https://github.com/apache/doris/pull/50847#issuecomment-2875406426
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50847:
URL: https://github.com/apache/doris/pull/50847#issuecomment-2875652993
<details>
<summary>TPC-H: <b>Total hot run time: 33931 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6cd698212f2152264f45a2475c2f642fcd4edb01, data reload: false
------ Round 1 ----------------------------------
q1 26338 5048 4990 4990
q2 2068 291 179 179
q3 10390 1269 722 722
q4 10215 1002 509 509
q5 7530 2387 2361 2361
q6 187 164 132 132
q7 919 758 621 621
q8 9310 1335 1119 1119
q9 6844 5062 5133 5062
q10 6825 2339 1926 1926
q11 466 292 275 275
q12 354 355 215 215
q13 17760 3657 3132 3132
q14 231 227 215 215
q15 532 486 491 486
q16 425 430 395 395
q17 606 876 382 382
q18 7847 7129 6999 6999
q19 1512 951 555 555
q20 347 336 231 231
q21 4084 2726 2447 2447
q22 1039 1029 978 978
Total cold run time: 115829 ms
Total hot run time: 33931 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5147 5049 5064 5049
q2 242 325 229 229
q3 2181 2614 2269 2269
q4 1304 1799 1369 1369
q5 4455 4384 4428 4384
q6 221 169 127 127
q7 2027 1938 1776 1776
q8 2615 2673 2504 2504
q9 7259 7146 6955 6955
q10 3074 3187 2757 2757
q11 585 522 494 494
q12 656 775 606 606
q13 3519 3872 3308 3308
q14 312 301 291 291
q15 525 489 489 489
q16 453 512 437 437
q17 1129 1495 1424 1424
q18 7760 7565 7461 7461
q19 803 805 1127 805
q20 1981 2058 1873 1873
q21 5064 4831 4775 4775
q22 1045 1057 1028 1028
Total cold run time: 52357 ms
Total hot run time: 50410 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50847:
URL: https://github.com/apache/doris/pull/50847#issuecomment-2875711537
<details>
<summary>ClickBench: <b>Total hot run time: 29.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6cd698212f2152264f45a2475c2f642fcd4edb01, data reload: false
query1 0.03 0.04 0.03
query2 0.12 0.11 0.12
query3 0.25 0.19 0.20
query4 1.60 0.19 0.20
query5 0.60 0.60 0.58
query6 1.20 0.72 0.73
query7 0.03 0.02 0.02
query8 0.04 0.04 0.04
query9 0.57 0.51 0.51
query10 0.55 0.58 0.57
query11 0.15 0.11 0.11
query12 0.15 0.12 0.12
query13 0.62 0.60 0.60
query14 0.80 0.81 0.82
query15 0.87 0.85 0.87
query16 0.38 0.39 0.40
query17 1.06 1.04 1.03
query18 0.23 0.21 0.21
query19 1.89 1.83 1.83
query20 0.01 0.01 0.01
query21 15.39 0.92 0.56
query22 0.76 1.41 0.97
query23 14.73 1.37 0.65
query24 7.72 1.39 0.68
query25 0.44 0.19 0.16
query26 0.72 0.17 0.13
query27 0.06 0.05 0.05
query28 8.80 0.89 0.47
query29 12.53 4.01 3.29
query30 0.26 0.10 0.07
query31 2.82 0.61 0.39
query32 3.23 0.56 0.48
query33 3.01 3.10 3.07
query34 15.81 5.09 4.47
query35 4.55 4.50 4.52
query36 0.66 0.49 0.49
query37 0.09 0.07 0.06
query38 0.05 0.04 0.04
query39 0.03 0.02 0.02
query40 0.18 0.15 0.14
query41 0.08 0.03 0.03
query42 0.04 0.03 0.02
query43 0.04 0.04 0.03
Total cold run time: 103.15 s
Total hot run time: 29.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50847:
URL: https://github.com/apache/doris/pull/50847#issuecomment-2875693534
<details>
<summary>TPC-DS: <b>Total hot run time: 193988 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6cd698212f2152264f45a2475c2f642fcd4edb01, data reload: false
query1 1406 1066 1041 1041
query2 6148 1852 1889 1852
query3 11085 4580 4475 4475
query4 54058 25003 23547 23547
query5 5002 583 470 470
query6 364 222 210 210
query7 4960 515 297 297
query8 323 275 263 263
query9 5769 2617 2613 2613
query10 448 323 268 268
query11 15179 15087 14858 14858
query12 174 114 108 108
query13 1083 527 422 422
query14 10306 6471 6419 6419
query15 207 214 184 184
query16 7101 675 490 490
query17 1083 753 600 600
query18 1539 428 324 324
query19 212 208 179 179
query20 131 128 121 121
query21 210 136 111 111
query22 4329 4395 4341 4341
query23 34535 33650 33665 33650
query24 6661 2427 2428 2427
query25 451 469 407 407
query26 731 277 164 164
query27 2307 523 346 346
query28 2986 2174 2163 2163
query29 592 560 440 440
query30 284 224 194 194
query31 869 862 822 822
query32 70 61 64 61
query33 445 377 337 337
query34 795 875 546 546
query35 802 826 750 750
query36 966 993 896 896
query37 119 102 78 78
query38 4190 4324 4127 4127
query39 1502 1467 1500 1467
query40 235 142 107 107
query41 56 55 62 55
query42 129 122 110 110
query43 530 523 508 508
query44 1408 853 854 853
query45 187 197 170 170
query46 875 1052 661 661
query47 1878 1872 1792 1792
query48 418 459 352 352
query49 698 532 409 409
query50 685 710 423 423
query51 4276 4287 4209 4209
query52 117 118 111 111
query53 229 262 197 197
query54 618 591 518 518
query55 84 85 87 85
query56 314 302 285 285
query57 1202 1200 1151 1151
query58 282 263 262 262
query59 2791 2865 2705 2705
query60 331 325 320 320
query61 131 133 120 120
query62 730 752 673 673
query63 227 196 205 196
query64 2000 1049 720 720
query65 4460 4366 4304 4304
query66 733 411 301 301
query67 16096 15592 15478 15478
query68 7920 895 523 523
query69 560 306 269 269
query70 1226 1114 1100 1100
query71 521 324 304 304
query72 5858 4937 4887 4887
query73 1238 683 356 356
query74 8979 9363 8622 8622
query75 4142 3242 2720 2720
query76 4226 1206 784 784
query77 677 397 292 292
query78 10220 10235 9267 9267
query79 2613 790 575 575
query80 604 505 441 441
query81 495 248 217 217
query82 419 124 98 98
query83 350 249 227 227
query84 296 101 83 83
query85 781 361 316 316
query86 366 318 280 280
query87 4462 4482 4517 4482
query88 3788 2329 2301 2301
query89 415 318 287 287
query90 1847 218 211 211
query91 143 152 110 110
query92 74 65 61 61
query93 2124 957 580 580
query94 668 416 310 310
query95 377 303 286 286
query96 501 586 285 285
query97 3197 3218 3108 3108
query98 243 200 201 200
query99 1443 1426 1263 1263
Total cold run time: 301915 ms
Total hot run time: 193988 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50847:
URL: https://github.com/apache/doris/pull/50847#issuecomment-2875518403
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](cloud-mow) avoid calc delete bitmap tasks on same (txn_id, tablet_id) being executed concurrently [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50847:
URL: https://github.com/apache/doris/pull/50847#issuecomment-2876013830
# BE UT Coverage Report
Increment line coverage `0.00% (0/22)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6cd698212f2152264f45a2475c2f642fcd4edb01_6cd698212f2152264f45a2475c2f642fcd4edb01/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6cd698212f2152264f45a2475c2f642fcd4edb01_6cd698212f2152264f45a2475c2f642fcd4edb01/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 55.78% (14894/26700) |
| Line Coverage | 44.59% (131794/295599) |
| Region Coverage | 43.64% (66270/151854) |
| Branch Coverage | 38.26% (33963/88762) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org