You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2025/12/26 12:49:36 UTC
[PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
w41ter opened a new pull request, #59413:
URL: https://github.com/apache/doris/pull/59413
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3788045790
<details>
<summary>TPC-DS: <b>Total hot run time: 172831 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2, data reload: false
query5 4382 620 501 501
query6 337 210 202 202
query7 4216 447 258 258
query8 324 252 241 241
query9 8667 2850 2834 2834
query10 483 314 269 269
query11 15283 15149 14916 14916
query12 180 117 116 116
query13 1248 463 372 372
query14 6236 2997 2767 2767
query14_1 2708 2667 2635 2635
query15 204 188 179 179
query16 979 478 477 477
query17 1096 674 578 578
query18 2501 434 336 336
query19 203 181 153 153
query20 125 118 116 116
query21 217 132 123 123
query22 3914 4100 4055 4055
query23 16083 15735 15349 15349
query23_1 15394 15554 15514 15514
query24 6963 1526 1168 1168
query24_1 1146 1173 1198 1173
query25 543 449 408 408
query26 1238 268 152 152
query27 2773 443 277 277
query28 4420 2174 2163 2163
query29 819 539 450 450
query30 314 251 204 204
query31 790 630 564 564
query32 92 77 73 73
query33 523 368 317 317
query34 897 882 536 536
query35 727 747 682 682
query36 874 925 850 850
query37 139 102 89 89
query38 2720 2706 2710 2706
query39 785 749 748 748
query39_1 727 710 717 710
query40 222 139 122 122
query41 72 74 71 71
query42 97 93 90 90
query43 439 444 424 424
query44 1324 753 748 748
query45 188 189 223 189
query46 830 949 575 575
query47 1434 1464 1273 1273
query48 311 313 247 247
query49 615 417 339 339
query50 675 266 205 205
query51 3741 3754 3785 3754
query52 93 94 80 80
query53 207 223 166 166
query54 280 261 238 238
query55 85 81 74 74
query56 294 289 298 289
query57 966 998 902 902
query58 273 256 251 251
query59 2006 2073 1985 1985
query60 333 326 308 308
query61 155 147 147 147
query62 390 355 312 312
query63 197 171 170 170
query64 4762 1174 852 852
query65 3839 3690 3736 3690
query66 1426 417 308 308
query67 15461 15572 15427 15427
query68 2693 1066 730 730
query69 385 310 273 273
query70 965 949 938 938
query71 299 296 271 271
query72 5294 3081 3221 3081
query73 594 715 308 308
query74 8749 8786 8520 8520
query75 2322 2321 1916 1916
query76 2056 1042 665 665
query77 342 366 312 312
query78 9744 9913 9115 9115
query79 2010 835 583 583
query80 1500 526 430 430
query81 534 263 229 229
query82 943 148 117 117
query83 321 259 242 242
query84 260 116 100 100
query85 877 460 410 410
query86 420 298 316 298
query87 2831 2848 2755 2755
query88 3473 2579 2573 2573
query89 309 259 234 234
query90 1994 169 162 162
query91 165 152 129 129
query92 87 72 72 72
query93 1187 999 656 656
query94 625 334 282 282
query95 566 323 312 312
query96 631 507 226 226
query97 2371 2406 2344 2344
query98 214 198 203 198
query99 611 564 495 495
Total cold run time: 246532 ms
Total hot run time: 172831 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3788409004
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59413_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59413_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.42% (25512/35719) |
| Line Coverage | 54.00% (267361/495087) |
| Region Coverage | 51.65% (221810/429476) |
| Branch Coverage | 53.04% (95236/179564) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3692972743
<details>
<summary>TPC-H: <b>Total hot run time: 35148 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e, data reload: false
------ Round 1 ----------------------------------
q1 17623 4155 4021 4021
q2 1998 351 225 225
q3 10208 1316 748 748
q4 10238 933 315 315
q5 7747 2162 1968 1968
q6 248 167 135 135
q7 1003 853 701 701
q8 9358 1454 1147 1147
q9 7226 5299 5322 5299
q10 6909 2383 2003 2003
q11 525 324 296 296
q12 673 724 583 583
q13 17786 3717 3073 3073
q14 287 294 270 270
q15 611 516 516 516
q16 678 672 644 644
q17 704 804 535 535
q18 7760 7155 7118 7118
q19 1113 990 652 652
q20 402 359 250 250
q21 4261 3942 3679 3679
q22 1077 989 970 970
Total cold run time: 108435 ms
Total hot run time: 35148 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4131 4034 4073 4034
q2 329 405 323 323
q3 2218 2694 2333 2333
q4 1359 1774 1306 1306
q5 4248 4761 4643 4643
q6 235 180 130 130
q7 2101 1985 1853 1853
q8 2686 2539 2580 2539
q9 7659 7489 7550 7489
q10 3137 3236 2806 2806
q11 624 527 513 513
q12 741 842 586 586
q13 3563 4025 3389 3389
q14 297 320 296 296
q15 563 519 513 513
q16 643 710 632 632
q17 1281 1483 1418 1418
q18 7832 7529 7536 7529
q19 854 807 846 807
q20 1926 1927 1802 1802
q21 4575 4180 4220 4180
q22 1094 1021 989 989
Total cold run time: 52096 ms
Total hot run time: 50110 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3693046956
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.40% (18952/35489) |
| Line Coverage | 39.27% (175780/447674) |
| Region Coverage | 33.84% (136057/402007) |
| Branch Coverage | 34.76% (58722/168938) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #59413:
URL: https://github.com/apache/doris/pull/59413#discussion_r2699127675
##########
cloud/src/meta-service/txn_lazy_committer.cpp:
##########
@@ -640,6 +659,7 @@ void TxnLazyCommitTask::commit() {
return;
}
+ bool defer_deleting_pending_delete_bitmaps = txn_info.defer_deleting_pending_delete_bitmaps();
Review Comment:
should we also check `has_xxx`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #59413:
URL: https://github.com/apache/doris/pull/59413#discussion_r2699082004
##########
cloud/src/common/config.h:
##########
@@ -320,6 +320,7 @@ CONF_Int32(parallel_txn_lazy_commit_num_threads, "0"); // hardware concurrency i
CONF_mInt64(txn_lazy_max_rowsets_per_batch, "1000");
CONF_mBool(txn_lazy_commit_shuffle_partitions, "true");
CONF_Int64(txn_lazy_commit_shuffle_seed, "0"); // 0 means generate a random seed
+CONF_mBool(txn_lazy_commit_defer_deleting_pending_delete_bitmaps, "false");
Review Comment:
seems all meta-service should be upgraded before changing it to true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3787828373
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3783429215
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou merged PR #59413:
URL: https://github.com/apache/doris/pull/59413
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3693149907
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59413_a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59413_a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.83% (25614/34694) |
| Line Coverage | 61.24% (273423/446482) |
| Region Coverage | 56.17% (228155/406172) |
| Branch Coverage | 58.02% (98353/169502) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3740273416
# Cloud UT Coverage Report
Increment line coverage `16.49% (16/97)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/a70e424c29ed255aa36023ad69858e1d9344e737_a70e424c29ed255aa36023ad69858e1d9344e737_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/a70e424c29ed255aa36023ad69858e1d9344e737_a70e424c29ed255aa36023ad69858e1d9344e737_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.54% (1784/2243) |
| Line Coverage | 64.66% (31738/49084) |
| Region Coverage | 65.33% (15778/24150) |
| Branch Coverage | 55.89% (8380/14994) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3788012281
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59413_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59413_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3788040807
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.46% (19116/36440) |
| Line Coverage | 35.83% (177572/495619) |
| Region Coverage | 32.29% (137208/424963) |
| Branch Coverage | 33.23% (59408/178793) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3692877268
# Cloud UT Coverage Report
Increment line coverage `16.49% (16/97)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.49% (1771/2228) |
| Line Coverage | 64.72% (31315/48386) |
| Region Coverage | 65.28% (15577/23861) |
| Branch Coverage | 55.90% (8286/14822) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3692986048
<details>
<summary>TPC-DS: <b>Total hot run time: 179303 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e, data reload: false
query5 4890 601 459 459
query6 341 244 235 235
query7 4216 465 276 276
query8 311 256 250 250
query9 8780 2547 2565 2547
query10 559 377 336 336
query11 15380 15245 14901 14901
query12 184 123 122 122
query13 1266 507 400 400
query14 6477 3026 2799 2799
query14_1 2647 2635 2679 2635
query15 212 194 178 178
query16 882 474 443 443
query17 1122 730 593 593
query18 2685 452 348 348
query19 229 227 200 200
query20 122 118 120 118
query21 226 142 122 122
query22 3890 4099 3893 3893
query23 16663 16332 16016 16016
query23_1 16028 16248 16321 16248
query24 7311 1631 1221 1221
query24_1 1220 1244 1240 1240
query25 543 477 450 450
query26 1228 267 162 162
query27 2764 478 299 299
query28 4498 2132 2134 2132
query29 798 544 438 438
query30 312 245 214 214
query31 824 719 611 611
query32 115 68 74 68
query33 534 347 269 269
query34 889 897 540 540
query35 750 798 723 723
query36 858 917 832 832
query37 132 96 73 73
query38 3042 3017 2954 2954
query39 781 731 736 731
query39_1 701 714 718 714
query40 228 142 119 119
query41 66 63 61 61
query42 110 103 104 103
query43 426 434 395 395
query44 1323 736 739 736
query45 192 186 185 185
query46 868 978 609 609
query47 1691 1697 1660 1660
query48 314 325 240 240
query49 646 435 349 349
query50 666 299 229 229
query51 3772 3847 3820 3820
query52 106 115 97 97
query53 322 347 289 289
query54 285 252 241 241
query55 78 82 73 73
query56 292 302 302 302
query57 1177 1151 1119 1119
query58 272 249 257 249
query59 2415 2461 2410 2410
query60 316 311 290 290
query61 160 160 164 160
query62 727 704 651 651
query63 327 294 303 294
query64 4965 1279 988 988
query65 4015 3942 3937 3937
query66 1388 439 338 338
query67 15317 15020 14875 14875
query68 3368 1026 721 721
query69 476 363 320 320
query70 1065 993 986 986
query71 356 297 292 292
query72 6282 4985 4866 4866
query73 640 587 305 305
query74 8562 8867 8656 8656
query75 3156 3154 2804 2804
query76 3710 1172 728 728
query77 517 402 302 302
query78 9409 9610 8888 8888
query79 1544 918 634 634
query80 1514 645 563 563
query81 558 272 229 229
query82 405 134 99 99
query83 366 252 241 241
query84 256 125 106 106
query85 935 512 460 460
query86 385 305 305 305
query87 3069 3281 3201 3201
query88 3242 2306 2297 2297
query89 458 417 396 396
query90 1847 163 150 150
query91 178 165 141 141
query92 73 71 67 67
query93 1087 936 557 557
query94 528 322 289 289
query95 571 375 292 292
query96 595 471 212 212
query97 2278 2367 2235 2235
query98 208 194 194 194
query99 1334 1339 1385 1339
Total cold run time: 255854 ms
Total hot run time: 179303 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #59413:
URL: https://github.com/apache/doris/pull/59413#discussion_r2699072446
##########
gensrc/proto/cloud.proto:
##########
@@ -506,6 +506,7 @@ message TxnInfoPB {
// TODO: There are more fields TBD
optional bool versioned_write = 19; // versioned write, don't need to write RecycleTxnPB again
optional bool versioned_read = 20; // whether to read versioned keys
+ optional bool defer_deleting_pending_delete_bitmaps = 21;
Review Comment:
add comment it is for lazy commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3784076185
<details>
<summary>ClickBench: <b>Total hot run time: 28.01 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ca9cb16116909f29dead316c7750b14214057d55, data reload: false
query1 0.06 0.05 0.04
query2 0.14 0.07 0.06
query3 0.34 0.08 0.08
query4 1.61 0.10 0.10
query5 0.26 0.25 0.26
query6 1.15 0.68 0.65
query7 0.04 0.03 0.03
query8 0.07 0.06 0.06
query9 0.60 0.51 0.49
query10 0.54 0.55 0.56
query11 0.26 0.14 0.14
query12 0.27 0.15 0.14
query13 0.63 0.63 0.59
query14 0.99 0.98 0.98
query15 0.92 0.83 0.83
query16 0.40 0.40 0.39
query17 1.01 1.07 1.06
query18 0.25 0.23 0.23
query19 2.00 1.88 1.74
query20 0.02 0.01 0.02
query21 15.40 0.29 0.24
query22 5.00 0.10 0.10
query23 15.38 0.40 0.22
query24 2.33 0.53 0.35
query25 0.10 0.10 0.10
query26 0.18 0.17 0.18
query27 0.10 0.09 0.10
query28 3.73 1.14 0.97
query29 12.51 4.03 3.27
query30 0.32 0.12 0.11
query31 2.79 0.69 0.46
query32 3.24 0.62 0.51
query33 3.08 3.06 3.04
query34 15.92 5.12 4.45
query35 4.43 4.58 4.45
query36 0.62 0.50 0.50
query37 0.25 0.09 0.09
query38 0.20 0.06 0.06
query39 0.08 0.05 0.05
query40 0.21 0.17 0.17
query41 0.14 0.07 0.07
query42 0.08 0.05 0.05
query43 0.09 0.07 0.07
Total cold run time: 97.74 s
Total hot run time: 28.01 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3740819427
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/a70e424c29ed255aa36023ad69858e1d9344e737_a70e424c29ed255aa36023ad69858e1d9344e737/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/a70e424c29ed255aa36023ad69858e1d9344e737_a70e424c29ed255aa36023ad69858e1d9344e737/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.01% (18978/35803) |
| Line Coverage | 39.05% (175861/450326) |
| Region Coverage | 33.61% (136052/404762) |
| Branch Coverage | 34.66% (58856/169809) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3740375332
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59413_a70e424c29ed255aa36023ad69858e1d9344e737/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59413_a70e424c29ed255aa36023ad69858e1d9344e737/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3740538869
<details>
<summary>TPC-H: <b>Total hot run time: 31855 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a70e424c29ed255aa36023ad69858e1d9344e737, data reload: false
------ Round 1 ----------------------------------
q1 17648 4300 4066 4066
q2 2027 359 245 245
q3 10155 1283 728 728
q4 10207 863 330 330
q5 7819 2047 1904 1904
q6 213 177 148 148
q7 929 802 681 681
q8 9275 1449 1132 1132
q9 5121 4687 4617 4617
q10 6855 1824 1427 1427
q11 515 288 273 273
q12 749 738 564 564
q13 17796 3801 3111 3111
q14 283 291 277 277
q15 595 519 519 519
q16 706 719 632 632
q17 682 794 583 583
q18 6772 6575 6493 6493
q19 1203 972 623 623
q20 398 360 258 258
q21 3054 2409 2265 2265
q22 1072 1004 979 979
Total cold run time: 104074 ms
Total hot run time: 31855 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4211 4070 4085 4070
q2 317 402 320 320
q3 2076 2626 2241 2241
q4 1353 1798 1342 1342
q5 4085 3999 4026 3999
q6 209 170 130 130
q7 1850 1769 2065 1769
q8 2592 2442 2559 2442
q9 7241 7217 7178 7178
q10 2548 2684 2276 2276
q11 584 489 457 457
q12 689 792 622 622
q13 3581 4095 3354 3354
q14 400 349 291 291
q15 540 523 497 497
q16 651 681 616 616
q17 1211 1342 1373 1342
q18 7784 7618 7733 7618
q19 918 820 861 820
q20 2030 2120 2103 2103
q21 4776 4439 4303 4303
q22 1118 1030 972 972
Total cold run time: 50764 ms
Total hot run time: 48762 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on code in PR #59413:
URL: https://github.com/apache/doris/pull/59413#discussion_r2703092800
##########
cloud/src/meta-service/txn_lazy_committer.cpp:
##########
@@ -563,6 +573,15 @@ void make_committed_txn_visible(const std::string& instance_id, int64_t db_id, i
LOG(INFO) << "remove running_key=" << hex(running_key) << " txn_id=" << txn_id;
txn->remove(running_key);
+ if (defer_deleting_pending_delete_bitmaps) {
+ for (int64_t table_id : txn_info.table_ids()) {
+ std::string lock_key =
+ meta_delete_bitmap_update_lock_key({instance_id, table_id, -1});
Review Comment:
should we check the lock_id == txn_id again to guarantee the correctness?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3784054160
<details>
<summary>TPC-DS: <b>Total hot run time: 172991 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ca9cb16116909f29dead316c7750b14214057d55, data reload: false
query5 4401 633 472 472
query6 327 213 214 213
query7 4219 449 261 261
query8 336 245 235 235
query9 8671 2843 2860 2843
query10 463 309 279 279
query11 15279 15187 14841 14841
query12 172 118 118 118
query13 1248 466 365 365
query14 6011 2986 2794 2794
query14_1 2776 2629 2641 2629
query15 200 191 172 172
query16 981 480 468 468
query17 1076 656 553 553
query18 2409 427 322 322
query19 193 168 145 145
query20 121 115 113 113
query21 208 139 117 117
query22 3869 4151 4022 4022
query23 16059 15668 15198 15198
query23_1 15363 15289 15435 15289
query24 7179 1538 1163 1163
query24_1 1153 1181 1174 1174
query25 551 452 414 414
query26 1245 274 152 152
query27 2784 453 281 281
query28 4579 2166 2141 2141
query29 792 559 489 489
query30 308 245 204 204
query31 800 636 575 575
query32 88 89 83 83
query33 553 364 326 326
query34 907 869 546 546
query35 742 752 701 701
query36 867 931 872 872
query37 148 100 91 91
query38 2749 2727 2645 2645
query39 782 763 741 741
query39_1 715 741 708 708
query40 222 136 117 117
query41 74 68 66 66
query42 96 95 93 93
query43 428 475 407 407
query44 1332 750 748 748
query45 193 191 193 191
query46 843 954 586 586
query47 1441 1500 1422 1422
query48 320 334 247 247
query49 625 444 358 358
query50 672 268 209 209
query51 3803 3808 3849 3808
query52 97 94 84 84
query53 214 220 172 172
query54 301 276 268 268
query55 86 87 79 79
query56 316 333 316 316
query57 1049 1018 936 936
query58 287 268 262 262
query59 2156 2134 2146 2134
query60 345 357 330 330
query61 176 172 165 165
query62 401 392 333 333
query63 197 166 160 160
query64 5071 1145 844 844
query65 3780 3795 3774 3774
query66 1446 421 316 316
query67 15528 15573 15438 15438
query68 2403 1073 724 724
query69 390 317 284 284
query70 1020 876 944 876
query71 297 287 270 270
query72 5326 3138 3316 3138
query73 593 717 310 310
query74 8688 8751 8554 8554
query75 2304 2363 1888 1888
query76 2281 1051 647 647
query77 359 376 310 310
query78 9709 9904 9233 9233
query79 1496 886 595 595
query80 1266 531 455 455
query81 541 262 235 235
query82 1015 150 117 117
query83 328 259 238 238
query84 265 123 97 97
query85 871 488 416 416
query86 417 305 281 281
query87 2896 2873 2789 2789
query88 3497 2596 2575 2575
query89 299 260 238 238
query90 2016 173 170 170
query91 166 162 171 162
query92 75 72 69 69
query93 1150 1013 647 647
query94 636 323 298 298
query95 585 401 316 316
query96 637 502 229 229
query97 2360 2393 2326 2326
query98 214 208 198 198
query99 598 595 508 508
Total cold run time: 246381 ms
Total hot run time: 172991 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3692994521
<details>
<summary>ClickBench: <b>Total hot run time: 28.31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a09864c1c9720d5a5eec5ad9548d9d9cf56fa14e, data reload: false
query1 0.05 0.05 0.05
query2 0.13 0.07 0.07
query3 0.31 0.08 0.08
query4 1.60 0.10 0.10
query5 0.26 0.25 0.24
query6 1.18 0.64 0.63
query7 0.03 0.02 0.02
query8 0.07 0.05 0.06
query9 0.57 0.49 0.49
query10 0.56 0.57 0.54
query11 0.26 0.14 0.13
query12 0.26 0.15 0.15
query13 0.63 0.62 0.61
query14 1.01 1.01 1.01
query15 0.88 0.81 0.82
query16 0.40 0.38 0.39
query17 0.98 1.05 1.06
query18 0.24 0.22 0.22
query19 1.87 1.80 1.75
query20 0.01 0.02 0.02
query21 15.39 0.28 0.24
query22 4.98 0.09 0.10
query23 15.42 0.41 0.22
query24 2.34 0.49 0.32
query25 0.11 0.10 0.09
query26 0.17 0.18 0.17
query27 0.10 0.09 0.09
query28 3.86 1.37 1.16
query29 12.60 4.17 3.37
query30 0.32 0.12 0.10
query31 2.80 0.66 0.42
query32 3.24 0.61 0.49
query33 3.03 3.03 3.01
query34 16.76 5.17 4.57
query35 4.73 4.58 4.69
query36 0.61 0.49 0.49
query37 0.24 0.09 0.08
query38 0.20 0.06 0.05
query39 0.06 0.04 0.04
query40 0.20 0.17 0.16
query41 0.12 0.06 0.06
query42 0.08 0.05 0.05
query43 0.06 0.06 0.06
Total cold run time: 98.72 s
Total hot run time: 28.31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3692825709
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3740126893
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3821704731
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3821590173
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3788075669
<details>
<summary>ClickBench: <b>Total hot run time: 27.93 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2, data reload: false
query1 0.05 0.04 0.04
query2 0.14 0.07 0.06
query3 0.32 0.08 0.08
query4 1.61 0.10 0.10
query5 0.29 0.24 0.26
query6 1.14 0.64 0.64
query7 0.03 0.02 0.02
query8 0.07 0.05 0.06
query9 0.59 0.51 0.49
query10 0.55 0.54 0.55
query11 0.25 0.13 0.13
query12 0.26 0.14 0.14
query13 0.63 0.63 0.61
query14 0.98 0.97 0.98
query15 0.92 0.81 0.83
query16 0.39 0.42 0.39
query17 1.04 1.03 1.06
query18 0.26 0.22 0.22
query19 1.92 1.84 1.90
query20 0.02 0.01 0.02
query21 15.39 0.27 0.24
query22 4.97 0.09 0.10
query23 15.40 0.38 0.22
query24 2.45 0.48 0.31
query25 0.11 0.10 0.10
query26 0.18 0.17 0.18
query27 0.10 0.10 0.09
query28 3.61 1.12 0.97
query29 12.51 3.96 3.25
query30 0.35 0.14 0.12
query31 2.80 0.68 0.44
query32 3.25 0.61 0.50
query33 3.01 3.10 3.04
query34 16.33 5.11 4.53
query35 4.52 4.56 4.42
query36 0.61 0.49 0.51
query37 0.26 0.09 0.08
query38 0.20 0.07 0.06
query39 0.08 0.05 0.05
query40 0.20 0.17 0.17
query41 0.13 0.07 0.07
query42 0.07 0.05 0.05
query43 0.08 0.07 0.06
Total cold run time: 98.07 s
Total hot run time: 27.93 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3783479082
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3783643349
# Cloud UT Coverage Report
Increment line coverage `14.04% (16/114)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/ca9cb16116909f29dead316c7750b14214057d55_ca9cb16116909f29dead316c7750b14214057d55_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/ca9cb16116909f29dead316c7750b14214057d55_ca9cb16116909f29dead316c7750b14214057d55_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.28% (1787/2254) |
| Line Coverage | 64.64% (31824/49233) |
| Region Coverage | 65.40% (15861/24254) |
| Branch Coverage | 55.95% (8429/15066) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3784008845
<details>
<summary>TPC-H: <b>Total hot run time: 30884 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ca9cb16116909f29dead316c7750b14214057d55, data reload: false
------ Round 1 ----------------------------------
q1 17688 4814 4573 4573
q2 1997 309 193 193
q3 10283 1303 744 744
q4 10321 864 313 313
q5 8831 2119 1839 1839
q6 220 171 136 136
q7 859 713 586 586
q8 9284 1393 1180 1180
q9 5212 4571 4671 4571
q10 6822 1653 1254 1254
q11 527 285 279 279
q12 376 381 222 222
q13 17807 3853 3116 3116
q14 232 236 208 208
q15 607 521 522 521
q16 633 638 593 593
q17 729 767 514 514
q18 7238 6351 6491 6351
q19 1379 978 614 614
q20 400 354 241 241
q21 2601 2083 1852 1852
q22 1032 1015 984 984
Total cold run time: 105078 ms
Total hot run time: 30884 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4855 4712 4742 4712
q2 337 400 315 315
q3 2164 2668 2256 2256
q4 1313 1804 1307 1307
q5 4109 4067 4018 4018
q6 220 171 129 129
q7 1947 2180 1948 1948
q8 2597 2555 2449 2449
q9 7200 7147 7301 7147
q10 2501 2799 2267 2267
q11 575 496 457 457
q12 744 795 614 614
q13 3729 4258 3318 3318
q14 298 318 294 294
q15 557 513 525 513
q16 683 700 754 700
q17 1243 1394 1370 1370
q18 8223 7906 7723 7723
q19 857 829 847 829
q20 2082 2289 1886 1886
q21 5063 4323 4127 4127
q22 1086 1026 985 985
Total cold run time: 52383 ms
Total hot run time: 49364 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3783752808
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59413_ca9cb16116909f29dead316c7750b14214057d55/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59413_ca9cb16116909f29dead316c7750b14214057d55/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3787897604
# Cloud UT Coverage Report
Increment line coverage `14.04% (16/114)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.28% (1787/2254) |
| Line Coverage | 64.68% (31844/49233) |
| Region Coverage | 65.44% (15871/24254) |
| Branch Coverage | 55.97% (8432/15066) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3788027223
<details>
<summary>TPC-H: <b>Total hot run time: 31332 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b2f5dadef2f24f08fc6d6f2c513aa15a103b20e2, data reload: false
------ Round 1 ----------------------------------
q1 16470 4655 4548 4548
q2 2020 317 226 226
q3 9780 1259 731 731
q4 9874 838 298 298
q5 7403 2064 1830 1830
q6 193 176 140 140
q7 885 722 604 604
q8 9159 1389 1074 1074
q9 4834 4637 4538 4538
q10 6746 1669 1278 1278
q11 491 299 279 279
q12 338 364 219 219
q13 17567 3803 3101 3101
q14 233 230 231 230
q15 580 522 521 521
q16 621 633 566 566
q17 643 755 542 542
q18 6694 6415 6402 6402
q19 1099 1115 781 781
q20 427 390 281 281
q21 2957 2412 2074 2074
q22 1156 1069 1100 1069
Total cold run time: 100170 ms
Total hot run time: 31332 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4953 5042 4849 4849
q2 323 414 317 317
q3 2446 2909 2438 2438
q4 1418 1843 1364 1364
q5 4508 4433 4348 4348
q6 224 167 132 132
q7 2007 1915 2044 1915
q8 2511 2356 2405 2356
q9 7085 7322 7153 7153
q10 2500 2789 2263 2263
q11 521 478 448 448
q12 685 745 619 619
q13 3660 4059 3130 3130
q14 265 283 283 283
q15 544 495 498 495
q16 615 638 599 599
q17 1087 1254 1312 1254
q18 7553 7261 7255 7255
q19 814 765 764 764
q20 1870 1966 1850 1850
q21 4505 4300 4085 4085
q22 1114 1027 969 969
Total cold run time: 51208 ms
Total hot run time: 48886 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3692829915
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](cloud) Txn lazy commit defers deleting pending delete bitmaps [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59413:
URL: https://github.com/apache/doris/pull/59413#issuecomment-3740573429
<details>
<summary>TPC-DS: <b>Total hot run time: 172427 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a70e424c29ed255aa36023ad69858e1d9344e737, data reload: false
query5 4384 599 432 432
query6 326 224 204 204
query7 4225 452 258 258
query8 351 267 240 240
query9 8752 2662 2704 2662
query10 519 392 326 326
query11 15347 15252 15034 15034
query12 172 116 115 115
query13 1264 486 404 404
query14 6219 3023 2775 2775
query14_1 2684 2644 2687 2644
query15 200 192 173 173
query16 981 474 468 468
query17 1113 681 583 583
query18 2486 444 341 341
query19 231 232 206 206
query20 126 116 115 115
query21 219 138 120 120
query22 3956 3928 3937 3928
query23 15946 15615 15244 15244
query23_1 15563 15445 15269 15269
query24 7457 1555 1185 1185
query24_1 1183 1165 1214 1165
query25 560 473 424 424
query26 1242 261 163 163
query27 2761 463 290 290
query28 4530 2142 2120 2120
query29 787 561 461 461
query30 308 247 212 212
query31 803 633 569 569
query32 77 71 68 68
query33 560 361 295 295
query34 899 900 524 524
query35 737 766 694 694
query36 912 905 746 746
query37 122 88 77 77
query38 2808 2752 2673 2673
query39 774 752 737 737
query39_1 696 705 711 705
query40 217 130 118 118
query41 66 60 60 60
query42 107 107 108 107
query43 475 476 432 432
query44 1335 711 715 711
query45 186 183 175 175
query46 859 962 581 581
query47 1375 1422 1372 1372
query48 310 317 244 244
query49 604 419 323 323
query50 640 274 205 205
query51 3808 3725 3760 3725
query52 110 109 97 97
query53 308 328 277 277
query54 286 254 245 245
query55 80 78 75 75
query56 285 300 277 277
query57 1039 1021 952 952
query58 270 243 251 243
query59 2142 2014 2048 2014
query60 320 303 294 294
query61 202 160 155 155
query62 392 338 295 295
query63 305 273 276 273
query64 4900 1273 972 972
query65 3852 3691 3649 3649
query66 1439 417 291 291
query67 15373 14766 15224 14766
query68 6549 988 701 701
query69 493 344 320 320
query70 1082 959 861 861
query71 365 301 280 280
query72 6027 3545 3377 3377
query73 764 727 297 297
query74 8817 8821 8560 8560
query75 2832 2824 2454 2454
query76 3458 1045 626 626
query77 513 371 305 305
query78 9675 9779 9222 9222
query79 1249 902 579 579
query80 605 565 480 480
query81 529 258 225 225
query82 209 147 107 107
query83 258 249 236 236
query84 253 125 115 115
query85 938 499 461 461
query86 395 301 288 288
query87 2867 2833 2780 2780
query88 3223 2248 2229 2229
query89 399 349 326 326
query90 2215 154 148 148
query91 170 157 142 142
query92 81 65 63 63
query93 1691 874 529 529
query94 572 284 291 284
query95 583 324 299 299
query96 576 465 205 205
query97 2331 2394 2322 2322
query98 230 211 204 204
query99 577 552 529 529
Total cold run time: 252924 ms
Total hot run time: 172427 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org