You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "sollhui (via GitHub)" <gi...@apache.org> on 2025/04/15 13:52:24 UTC
[PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
sollhui opened a new pull request, #50066:
URL: https://github.com/apache/doris/pull/50066
### What problem does this PR solve?
Repeatedly reporting errors in the log:
```
W20250415 17:54:01.151019 218169 storage_engine.cpp:785] failed to clear transaction. txn_id=1002, partition_id=1744709709398, tablet_id=1744709709485, status=[E-228]could not delete transaction from engine, just remove it from memory not delete from disk, because related rowset already published. partition_id: 1744709709398, transaction_id: 1002, tablet: 1744709709485.fb491f2a6f29dad0-28fe3fdfba3272b5, rowset id: 020000000000001fc6430de53121366b7d7bc36d82a1ae92, version: [2-2], state: VISIBLE
W20250415 17:54:01.152154 218169 storage_engine.cpp:785] failed to clear transaction. txn_id=1002, partition_id=1744709709398, tablet_id=1744709709493, status=[E-228]could not delete transaction from engine, just remove it from memory not delete from disk, because related rowset already published. partition_id: 1744709709398, transaction_id: 1002, tablet: 1744709709493.7a47a3bf7dcc70f0-353230158f6c2390, rowset id: 0200000000000019c6430de53121366b7d7bc36d82a1ae92, version: [2-2], state: VISIBLE
W20250415 17:54:01.152177 218169 storage_engine.cpp:785] failed to clear transaction. txn_id=1002, partition_id=1744709709398, tablet_id=1744709709509, status=[E-228]could not delete transaction from engine, just remove it from memory not delete from disk, because related rowset already published. partition_id: 1744709709398, transaction_id: 1002, tablet: 1744709709509.7f49efdb3b6c2c1d-ed627da9d22c9884, rowset id: 020000000000001bc6430de53121366b7d7bc36d82a1ae92, version: [2-2], state: VISIBLE
```
This bug can occur in the following scenarios:
1. After load the three replicas of the table, the publish task failed after making the rowset as visible in one of the replica.
2. The transactions became visible. Then FE clear visible transaction.
3. The BE node that fails to publish will not delete the memory transaction and report it to FE. FE cannot find the transaction and issues a clear command. However, the deletion fails due to the rowset already published.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805451088
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805455736
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805135023
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808239253
<details>
<summary>TPC-H: <b>Total hot run time: 33771 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6648d75ef7b15da8e0037f725735fed22ec1350, data reload: false
------ Round 1 ----------------------------------
q1 25847 5070 4990 4990
q2 2065 283 186 186
q3 10379 1284 719 719
q4 10222 998 541 541
q5 7508 2443 2290 2290
q6 185 167 132 132
q7 943 732 623 623
q8 9316 1300 1071 1071
q9 6795 5089 5122 5089
q10 6818 2299 1893 1893
q11 481 282 271 271
q12 353 354 217 217
q13 17781 3649 3081 3081
q14 246 222 202 202
q15 536 495 500 495
q16 444 445 394 394
q17 615 867 375 375
q18 7590 7217 7040 7040
q19 1230 942 559 559
q20 345 331 223 223
q21 4258 3422 2424 2424
q22 1017 1011 956 956
Total cold run time: 114974 ms
Total hot run time: 33771 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5075 5084 5074 5074
q2 245 332 226 226
q3 2155 2691 2318 2318
q4 1430 1798 1397 1397
q5 4441 4385 4404 4385
q6 218 172 130 130
q7 2012 1907 1781 1781
q8 2629 2585 2483 2483
q9 7304 7242 7057 7057
q10 3025 3188 2697 2697
q11 589 529 503 503
q12 689 783 614 614
q13 3519 3935 3270 3270
q14 287 313 294 294
q15 532 496 467 467
q16 465 502 464 464
q17 1152 1562 1379 1379
q18 7828 7620 7616 7616
q19 840 820 889 820
q20 2022 1990 1817 1817
q21 5205 4894 4817 4817
q22 1118 1040 996 996
Total cold run time: 52780 ms
Total hot run time: 50605 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #50066:
URL: https://github.com/apache/doris/pull/50066
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808144130
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805134721
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806625935
<details>
<summary>ClickBench: <b>Total hot run time: 31.24 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5cbeb96c2c72a3af21e380789445b82184ac231a, data reload: false
query1 0.04 0.03 0.03
query2 0.12 0.10 0.11
query3 0.24 0.20 0.19
query4 1.60 0.19 0.19
query5 0.58 0.60 0.58
query6 1.20 0.71 0.71
query7 0.03 0.02 0.01
query8 0.05 0.03 0.03
query9 0.58 0.54 0.52
query10 0.58 0.57 0.56
query11 0.15 0.11 0.11
query12 0.15 0.12 0.11
query13 0.62 0.62 0.59
query14 2.68 2.84 2.80
query15 0.95 0.86 0.86
query16 0.38 0.37 0.38
query17 1.04 1.05 1.02
query18 0.21 0.20 0.20
query19 1.86 1.97 1.86
query20 0.01 0.02 0.01
query21 15.91 0.87 0.56
query22 0.90 1.14 0.65
query23 15.12 1.40 0.60
query24 7.04 1.81 0.83
query25 0.56 0.20 0.19
query26 0.63 0.15 0.13
query27 0.05 0.04 0.05
query28 9.38 0.90 0.41
query29 12.56 3.98 3.30
query30 0.25 0.09 0.06
query31 2.84 0.59 0.39
query32 3.22 0.58 0.48
query33 3.00 2.99 3.06
query34 15.85 5.07 4.46
query35 4.47 4.51 4.52
query36 0.65 0.49 0.49
query37 0.09 0.06 0.06
query38 0.05 0.04 0.03
query39 0.04 0.02 0.03
query40 0.15 0.14 0.12
query41 0.08 0.03 0.02
query42 0.04 0.03 0.02
query43 0.03 0.03 0.03
Total cold run time: 105.98 s
Total hot run time: 31.24 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808283433
<details>
<summary>TPC-DS: <b>Total hot run time: 192575 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6648d75ef7b15da8e0037f725735fed22ec1350, data reload: false
query1 1427 1080 1050 1050
query2 6453 1833 1811 1811
query3 10982 4507 4405 4405
query4 54905 24849 23801 23801
query5 4986 555 482 482
query6 356 206 191 191
query7 4935 522 292 292
query8 346 248 254 248
query9 5373 2568 2572 2568
query10 445 305 249 249
query11 15315 14986 14768 14768
query12 158 112 110 110
query13 1030 550 394 394
query14 10250 6305 6303 6303
query15 211 191 180 180
query16 7105 664 463 463
query17 1086 701 577 577
query18 1588 430 295 295
query19 195 186 156 156
query20 128 130 115 115
query21 205 127 127 127
query22 4375 4475 4236 4236
query23 34197 33502 33325 33325
query24 6763 2369 2378 2369
query25 449 464 405 405
query26 679 276 161 161
query27 2389 509 334 334
query28 3032 2172 2167 2167
query29 578 561 424 424
query30 268 224 185 185
query31 860 883 771 771
query32 76 65 64 64
query33 454 367 320 320
query34 769 874 519 519
query35 815 841 775 775
query36 941 1001 895 895
query37 111 103 81 81
query38 4192 4230 4185 4185
query39 1489 1420 1435 1420
query40 226 126 106 106
query41 53 53 53 53
query42 123 110 108 108
query43 495 510 487 487
query44 1303 816 812 812
query45 178 177 169 169
query46 833 1031 648 648
query47 1876 1857 1760 1760
query48 386 413 318 318
query49 690 528 425 425
query50 658 704 417 417
query51 4248 4195 4277 4195
query52 109 109 99 99
query53 231 261 192 192
query54 622 578 516 516
query55 83 81 81 81
query56 319 296 299 296
query57 1151 1177 1153 1153
query58 273 292 258 258
query59 2780 2817 2723 2723
query60 319 311 319 311
query61 130 131 128 128
query62 746 757 699 699
query63 235 191 187 187
query64 1524 1044 730 730
query65 4431 4358 4219 4219
query66 724 415 360 360
query67 15954 15382 15344 15344
query68 6893 876 521 521
query69 532 289 261 261
query70 1197 1131 1124 1124
query71 470 318 280 280
query72 5892 4814 4843 4814
query73 1288 646 339 339
query74 8929 9141 8753 8753
query75 3659 3209 2693 2693
query76 4072 1189 747 747
query77 626 368 272 272
query78 9993 10039 9284 9284
query79 2666 801 554 554
query80 665 503 447 447
query81 475 256 216 216
query82 601 125 94 94
query83 247 244 227 227
query84 287 102 89 89
query85 780 358 315 315
query86 367 316 275 275
query87 4405 4472 4370 4370
query88 3219 2233 2193 2193
query89 387 322 276 276
query90 1761 213 215 213
query91 146 143 114 114
query92 78 61 55 55
query93 2425 960 592 592
query94 689 420 313 313
query95 366 294 283 283
query96 484 568 266 266
query97 3163 3189 3098 3098
query98 229 211 206 206
query99 1440 1425 1276 1276
Total cold run time: 299047 ms
Total hot run time: 192575 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805420442
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805563700
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808154379
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808289833
<details>
<summary>ClickBench: <b>Total hot run time: 29.5 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6648d75ef7b15da8e0037f725735fed22ec1350, data reload: false
query1 0.03 0.03 0.03
query2 0.12 0.10 0.10
query3 0.24 0.19 0.19
query4 1.59 0.19 0.19
query5 0.59 0.58 0.58
query6 1.16 0.72 0.71
query7 0.02 0.01 0.01
query8 0.04 0.04 0.03
query9 0.58 0.52 0.51
query10 0.55 0.56 0.58
query11 0.16 0.11 0.11
query12 0.14 0.11 0.12
query13 0.62 0.59 0.60
query14 1.14 1.18 1.18
query15 0.86 0.83 0.85
query16 0.36 0.38 0.38
query17 1.01 1.03 1.03
query18 0.21 0.21 0.20
query19 1.86 1.83 1.77
query20 0.01 0.00 0.01
query21 15.42 0.91 0.55
query22 0.74 1.09 0.65
query23 15.10 1.37 0.63
query24 6.92 1.98 0.81
query25 0.45 0.23 0.15
query26 0.62 0.17 0.13
query27 0.06 0.05 0.05
query28 9.31 0.81 0.42
query29 12.54 4.04 3.39
query30 0.26 0.10 0.06
query31 2.84 0.58 0.38
query32 3.22 0.54 0.47
query33 3.06 3.02 3.08
query34 15.74 5.09 4.42
query35 4.52 4.52 4.47
query36 0.66 0.49 0.48
query37 0.09 0.06 0.06
query38 0.04 0.03 0.03
query39 0.03 0.03 0.02
query40 0.17 0.13 0.13
query41 0.07 0.02 0.02
query42 0.03 0.03 0.02
query43 0.04 0.03 0.03
Total cold run time: 103.22 s
Total hot run time: 29.5 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808142210
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808430868
# BE UT Coverage Report
Increment line coverage `100.00% (3/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/d6648d75ef7b15da8e0037f725735fed22ec1350_d6648d75ef7b15da8e0037f725735fed22ec1350/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/d6648d75ef7b15da8e0037f725735fed22ec1350_d6648d75ef7b15da8e0037f725735fed22ec1350/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.83% (14222/26921) |
| Line Coverage | 41.72% (123433/295891) |
| Region Coverage | 40.44% (62856/155413) |
| Branch Coverage | 35.07% (31524/89892) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806451093
<details>
<summary>TPC-H: <b>Total hot run time: 34447 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5cbeb96c2c72a3af21e380789445b82184ac231a, data reload: false
------ Round 1 ----------------------------------
q1 26614 5028 5058 5028
q2 2108 291 189 189
q3 10366 1245 696 696
q4 10217 1060 541 541
q5 7547 3085 2416 2416
q6 197 165 132 132
q7 908 771 616 616
q8 9316 1393 1163 1163
q9 6899 5097 5185 5097
q10 6800 2321 1893 1893
q11 485 300 275 275
q12 351 356 219 219
q13 17763 3679 3136 3136
q14 226 222 213 213
q15 523 492 479 479
q16 631 613 576 576
q17 619 887 406 406
q18 7609 7370 7100 7100
q19 1229 953 580 580
q20 344 335 226 226
q21 4398 3382 2490 2490
q22 1034 976 979 976
Total cold run time: 116184 ms
Total hot run time: 34447 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5101 5089 5075 5075
q2 242 330 224 224
q3 2189 2656 2317 2317
q4 1458 1947 1508 1508
q5 4555 4434 4439 4434
q6 216 165 124 124
q7 1984 1910 1775 1775
q8 2578 2447 2574 2447
q9 7296 7272 7184 7184
q10 2968 3171 2703 2703
q11 575 510 485 485
q12 656 766 579 579
q13 3588 3903 3274 3274
q14 297 303 273 273
q15 517 485 475 475
q16 644 684 668 668
q17 1161 1598 1364 1364
q18 7724 7444 7499 7444
q19 835 841 972 841
q20 1988 2062 1842 1842
q21 5214 4638 4566 4566
q22 1066 1047 982 982
Total cold run time: 52852 ms
Total hot run time: 50584 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806582083
<details>
<summary>TPC-DS: <b>Total hot run time: 186345 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5cbeb96c2c72a3af21e380789445b82184ac231a, data reload: false
query1 1016 479 505 479
query2 6547 1946 1931 1931
query3 6761 220 218 218
query4 26197 23399 23415 23399
query5 4376 628 481 481
query6 305 193 198 193
query7 4639 488 277 277
query8 280 232 237 232
query9 8624 2566 2585 2566
query10 483 303 260 260
query11 15605 15120 14813 14813
query12 156 108 107 107
query13 1646 510 376 376
query14 9193 6039 5995 5995
query15 205 182 182 182
query16 7177 635 487 487
query17 1193 712 573 573
query18 1958 402 313 313
query19 191 181 157 157
query20 120 117 156 117
query21 211 122 102 102
query22 3971 4136 3973 3973
query23 33924 33099 33045 33045
query24 8461 2381 2394 2381
query25 524 444 410 410
query26 1236 267 155 155
query27 2748 513 335 335
query28 4354 2446 2399 2399
query29 807 573 456 456
query30 287 225 194 194
query31 918 854 756 756
query32 75 71 64 64
query33 564 388 336 336
query34 795 874 511 511
query35 828 836 724 724
query36 946 988 871 871
query37 124 102 78 78
query38 4108 4165 4159 4159
query39 1477 1404 1419 1404
query40 219 125 117 117
query41 62 62 62 62
query42 122 106 109 106
query43 517 515 479 479
query44 1313 812 791 791
query45 181 173 171 171
query46 846 1012 616 616
query47 1745 1804 1773 1773
query48 381 409 304 304
query49 810 553 447 447
query50 668 687 406 406
query51 4180 4167 4057 4057
query52 107 100 97 97
query53 230 248 188 188
query54 571 569 494 494
query55 80 79 79 79
query56 290 309 277 277
query57 1135 1156 1067 1067
query58 264 264 243 243
query59 2620 2815 2799 2799
query60 326 317 296 296
query61 133 131 132 131
query62 803 741 649 649
query63 231 190 187 187
query64 4469 1028 707 707
query65 4367 4290 4239 4239
query66 1140 431 311 311
query67 15841 15423 15199 15199
query68 8349 901 515 515
query69 473 309 261 261
query70 1231 1090 1112 1090
query71 444 337 290 290
query72 5296 4803 4726 4726
query73 712 598 334 334
query74 9224 9091 8740 8740
query75 3832 3200 2689 2689
query76 3663 1200 769 769
query77 786 376 289 289
query78 10061 10128 9270 9270
query79 2295 807 576 576
query80 606 497 434 434
query81 537 252 225 225
query82 443 123 96 96
query83 287 265 238 238
query84 290 98 84 84
query85 793 354 313 313
query86 338 294 273 273
query87 4616 4483 4362 4362
query88 3330 2187 2237 2187
query89 389 320 285 285
query90 1931 215 208 208
query91 146 142 110 110
query92 77 61 63 61
query93 1300 952 595 595
query94 677 403 293 293
query95 411 299 287 287
query96 489 561 279 279
query97 3219 3234 3166 3166
query98 232 202 207 202
query99 1499 1427 1297 1297
Total cold run time: 274657 ms
Total hot run time: 186345 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806987756
# BE UT Coverage Report
Increment line coverage `66.67% (2/3)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/5cbeb96c2c72a3af21e380789445b82184ac231a_5cbeb96c2c72a3af21e380789445b82184ac231a/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/5cbeb96c2c72a3af21e380789445b82184ac231a_5cbeb96c2c72a3af21e380789445b82184ac231a/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.43% (14109/26908) |
| Line Coverage | 41.35% (122333/295846) |
| Region Coverage | 40.07% (62252/155346) |
| Branch Coverage | 34.78% (31253/89858) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805447380
<details>
<summary>TPC-H: <b>Total hot run time: 35119 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8ac61e231b42553617ea09143f9c784871b368c3, data reload: false
------ Round 1 ----------------------------------
q1 26157 5132 5094 5094
q2 2055 298 179 179
q3 10378 1225 701 701
q4 10216 1001 553 553
q5 7521 2419 2287 2287
q6 185 170 133 133
q7 916 755 625 625
q8 9325 1263 1105 1105
q9 6817 5047 5057 5047
q10 6863 2277 1875 1875
q11 480 296 280 280
q12 369 362 227 227
q13 17777 3695 3098 3098
q14 223 226 233 226
q15 531 489 485 485
q16 626 604 569 569
q17 619 852 365 365
q18 7761 7445 7280 7280
q19 1396 953 563 563
q20 337 356 228 228
q21 4460 3609 3207 3207
q22 1062 1019 992 992
Total cold run time: 116074 ms
Total hot run time: 35119 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5309 5109 5099 5099
q2 242 329 229 229
q3 2154 2651 2226 2226
q4 1419 1924 1466 1466
q5 4458 4441 4370 4370
q6 215 171 133 133
q7 1985 1896 1792 1792
q8 2556 2550 2593 2550
q9 7207 6988 7230 6988
q10 2976 3147 2768 2768
q11 583 516 498 498
q12 683 803 601 601
q13 3496 3859 3402 3402
q14 273 298 279 279
q15 513 475 482 475
q16 659 683 672 672
q17 1157 1557 1395 1395
q18 7732 7604 7413 7413
q19 847 842 868 842
q20 1935 1988 1856 1856
q21 5299 4879 4728 4728
q22 1063 1039 1026 1026
Total cold run time: 52761 ms
Total hot run time: 50808 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805420906
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui closed pull request #50066: [fix](transaction) remove visible rowset from memory during deletion transaction
URL: https://github.com/apache/doris/pull/50066
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org