You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mymeiyi (via GitHub)" <gi...@apache.org> on 2026/01/12 06:54:38 UTC
[PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
mymeiyi opened a new pull request, #59757:
URL: https://github.com/apache/doris/pull/59757
### What problem does this PR solve?
1. `CloudGlobalTransactionMgr` does not remove `waitToCommitTxnCountMap` if table is dropped.
2. simplify `lastTxnIdMap` and `txnLastSignatureMap`
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #59757:
URL: https://github.com/apache/doris/pull/59757
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #59757:
URL: https://github.com/apache/doris/pull/59757#discussion_r2681075163
##########
fe/fe-core/src/main/java/org/apache/doris/cloud/transaction/CloudGlobalTransactionMgr.java:
##########
@@ -2589,55 +2589,30 @@ private void decreaseWaitingLockCount(List<Table> tableList) {
}
}
- public long getTableLastTxnId(long dbId, long tableId) {
- Map<Long, Long> tabletIdToTxnId = lastTxnIdMap.get(dbId);
- if (tabletIdToTxnId == null) {
- return -1;
- }
- return tabletIdToTxnId.getOrDefault(tableId, -1L);
+ private long getTableLastTxnId(long dbId, long tableId) {
+ return lastTxnIdMap.getOrDefault(tableId, -1L);
}
- public void setTableLastTxnId(long dbId, long tableId, long txnId) {
- lastTxnIdMap.compute(dbId, (k, v) -> {
- if (v == null) {
- v = Maps.newConcurrentMap();
- }
- LOG.debug("setTableLastTxnId dbId: {}, tableId: {}, txnId: {}", dbId, tableId, txnId);
- v.put(tableId, txnId);
- return v;
- });
+ private void setTableLastTxnId(long dbId, long tableId, long txnId) {
+ lastTxnIdMap.put(tableId, txnId);
+ LOG.debug("setTableLastTxnId dbId: {}, tableId: {}, txnId: {}", dbId, tableId, txnId);
}
- public void clearTableLastTxnId(long dbId, long tableId) {
- lastTxnIdMap.computeIfPresent(dbId, (k, v) -> {
- v.remove(tableId);
- return v.isEmpty() ? null : v;
- });
+ public void afterDropTable(long dbId, long tableId) {
+ lastTxnIdMap.remove(tableId);
+ waitToCommitTxnCountMap.remove(tableId);
}
Review Comment:
The new afterDropTable method lacks test coverage to verify that both lastTxnIdMap and waitToCommitTxnCountMap are properly cleaned up when a table is dropped. Consider adding a unit test in CloudGlobalTransactionMgrTest to verify this cleanup behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59757:
URL: https://github.com/apache/doris/pull/59757#issuecomment-3741570191
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59757:
URL: https://github.com/apache/doris/pull/59757#issuecomment-3737823226
<details>
<summary>TPC-DS: <b>Total hot run time: 173303 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 56fd5d84438a5aa9691b3855f82c678e4c055f44, data reload: false
query5 4864 590 431 431
query6 357 234 210 210
query7 4219 458 256 256
query8 329 243 227 227
query9 8775 2647 2654 2647
query10 484 375 320 320
query11 15411 15168 15052 15052
query12 190 115 115 115
query13 1258 502 365 365
query14 6458 3006 2733 2733
query14_1 2692 2622 2646 2622
query15 198 196 169 169
query16 979 459 479 459
query17 1045 665 558 558
query18 2655 424 326 326
query19 218 216 192 192
query20 124 118 117 117
query21 216 150 119 119
query22 3839 4041 4008 4008
query23 16012 15644 15327 15327
query23_1 15318 15778 15418 15418
query24 7405 1552 1202 1202
query24_1 1231 1210 1208 1208
query25 563 480 429 429
query26 1246 271 167 167
query27 2744 450 295 295
query28 4561 2151 2139 2139
query29 829 575 464 464
query30 308 242 211 211
query31 770 637 569 569
query32 83 71 81 71
query33 560 359 300 300
query34 915 884 524 524
query35 728 761 690 690
query36 868 904 832 832
query37 125 100 78 78
query38 2708 2703 2642 2642
query39 771 792 771 771
query39_1 718 721 710 710
query40 222 142 123 123
query41 76 69 70 69
query42 111 108 108 108
query43 485 468 439 439
query44 1300 726 728 726
query45 190 185 178 178
query46 856 961 595 595
query47 1343 1443 1362 1362
query48 311 343 245 245
query49 629 423 369 369
query50 647 283 210 210
query51 3780 3805 3745 3745
query52 107 111 97 97
query53 310 332 282 282
query54 297 275 260 260
query55 84 78 73 73
query56 312 304 320 304
query57 995 1044 889 889
query58 281 267 259 259
query59 2048 2103 2175 2103
query60 332 360 286 286
query61 167 167 160 160
query62 411 353 298 298
query63 310 269 274 269
query64 4915 1305 998 998
query65 3814 3796 3704 3704
query66 1384 411 299 299
query67 15210 14991 14995 14991
query68 6440 965 714 714
query69 500 340 316 316
query70 1067 937 978 937
query71 368 300 273 273
query72 6070 3251 3372 3251
query73 732 653 303 303
query74 8915 8736 8622 8622
query75 2798 2794 2427 2427
query76 3487 1061 644 644
query77 508 371 283 283
query78 9684 9867 9219 9219
query79 1229 894 583 583
query80 622 570 498 498
query81 518 265 232 232
query82 209 142 112 112
query83 265 258 240 240
query84 256 118 102 102
query85 895 527 457 457
query86 394 326 328 326
query87 2879 2877 2767 2767
query88 3131 2223 2203 2203
query89 382 356 324 324
query90 2164 149 142 142
query91 177 161 143 143
query92 84 68 66 66
query93 1369 939 531 531
query94 577 333 314 314
query95 576 392 303 303
query96 589 469 204 204
query97 2327 2401 2315 2315
query98 210 199 203 199
query99 572 569 509 509
Total cold run time: 252447 ms
Total hot run time: 173303 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59757:
URL: https://github.com/apache/doris/pull/59757#issuecomment-3737127537
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59757:
URL: https://github.com/apache/doris/pull/59757#issuecomment-3737779734
<details>
<summary>TPC-H: <b>Total hot run time: 32227 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 56fd5d84438a5aa9691b3855f82c678e4c055f44, data reload: false
------ Round 1 ----------------------------------
q1 17630 4202 4063 4063
q2 2041 371 252 252
q3 10131 1300 733 733
q4 10217 897 323 323
q5 7499 2099 1868 1868
q6 191 168 141 141
q7 929 788 666 666
q8 9284 1427 1153 1153
q9 5009 4668 4685 4668
q10 6818 1843 1407 1407
q11 527 298 285 285
q12 706 725 566 566
q13 17858 3853 3098 3098
q14 301 290 287 287
q15 602 509 495 495
q16 691 693 626 626
q17 687 825 511 511
q18 6649 6459 7180 6459
q19 1170 1007 626 626
q20 419 390 272 272
q21 3292 2717 2671 2671
q22 1189 1121 1057 1057
Total cold run time: 103840 ms
Total hot run time: 32227 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4500 4258 4271 4258
q2 319 432 309 309
q3 2262 2892 2352 2352
q4 1356 1941 1460 1460
q5 4486 4332 4331 4331
q6 214 168 133 133
q7 2018 1923 1806 1806
q8 2530 2726 2431 2431
q9 7124 7046 7150 7046
q10 2484 2681 2359 2359
q11 570 467 447 447
q12 768 860 633 633
q13 3605 4126 3293 3293
q14 317 328 282 282
q15 552 510 502 502
q16 661 724 655 655
q17 1147 1279 1325 1279
q18 7462 7219 7087 7087
q19 820 772 797 772
q20 1889 1945 1814 1814
q21 4502 4251 4165 4165
q22 1035 1025 969 969
Total cold run time: 50621 ms
Total hot run time: 48383 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59757:
URL: https://github.com/apache/doris/pull/59757#issuecomment-3741854475
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](fe) remove table from waitToCommitTxnCountMap in CloudGlobalTransactionMgr when drop table [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #59757:
URL: https://github.com/apache/doris/pull/59757#issuecomment-3737442520
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org