You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "hust-hhb (via GitHub)" <gi...@apache.org> on 2024/09/03 11:57:42 UTC
[PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
hust-hhb opened a new pull request, #40333:
URL: https://github.com/apache/doris/pull/40333
If the publish_status of txn_delete_bitmap_cache isn't SUCCEED, it should not be removed because this cache still being used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326353261
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2348625064
TeamCity be ut coverage result:
Function Coverage: 36.89% (9465/25654)
Line Coverage: 28.26% (77848/275433)
Region Coverage: 27.67% (40198/145296)
Branch Coverage: 24.28% (20429/84138)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fc00ceb4217f1ec8fbbc1943f0a8c175d3e5fc1e_fc00ceb4217f1ec8fbbc1943f0a8c175d3e5fc1e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen merged PR #40333:
URL: https://github.com/apache/doris/pull/40333
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326402951
<details>
<summary>TPC-DS: <b>Total hot run time: 188201 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1ce3e56e944dface7106077a5a1b3c2661492eba, data reload: false
query1 903 369 363 363
query2 6457 2004 2030 2004
query3 6647 213 209 209
query4 30985 23203 23395 23203
query5 4136 485 469 469
query6 250 156 158 156
query7 4597 293 283 283
query8 257 200 221 200
query9 8617 2478 2473 2473
query10 443 272 272 272
query11 17960 15074 15020 15020
query12 153 98 99 98
query13 1629 376 359 359
query14 9856 7120 7201 7120
query15 256 171 176 171
query16 8099 455 455 455
query17 1579 546 541 541
query18 2123 277 274 274
query19 282 138 143 138
query20 118 108 111 108
query21 212 100 101 100
query22 4398 4061 4327 4061
query23 34106 33439 34036 33439
query24 11098 2859 2829 2829
query25 630 378 386 378
query26 1177 150 151 150
query27 2842 278 275 275
query28 7616 2062 2032 2032
query29 815 411 408 408
query30 296 163 154 154
query31 1001 760 807 760
query32 96 54 55 54
query33 747 279 283 279
query34 998 470 489 470
query35 821 714 713 713
query36 1076 922 931 922
query37 155 89 87 87
query38 4029 3840 3877 3840
query39 1468 1540 1377 1377
query40 201 113 113 113
query41 49 49 45 45
query42 112 92 95 92
query43 534 483 501 483
query44 1181 739 721 721
query45 194 170 173 170
query46 1103 732 754 732
query47 1893 1829 1851 1829
query48 362 281 295 281
query49 1047 440 431 431
query50 802 402 418 402
query51 6992 6850 6934 6850
query52 96 85 85 85
query53 247 185 175 175
query54 865 452 456 452
query55 79 76 72 72
query56 264 262 248 248
query57 1198 1092 1070 1070
query58 237 259 265 259
query59 3054 2955 2921 2921
query60 306 267 262 262
query61 122 96 97 96
query62 819 662 673 662
query63 219 183 181 181
query64 4359 685 627 627
query65 3207 3133 3153 3133
query66 1218 332 331 331
query67 15542 15266 15129 15129
query68 3135 574 560 560
query69 389 280 270 270
query70 1182 1132 1134 1132
query71 340 271 262 262
query72 6328 4088 4020 4020
query73 742 326 377 326
query74 9136 8938 8951 8938
query75 3390 2685 2647 2647
query76 2040 1014 979 979
query77 489 327 310 310
query78 11313 9404 9713 9404
query79 1129 535 518 518
query80 755 527 491 491
query81 578 230 233 230
query82 753 143 147 143
query83 238 152 150 150
query84 229 80 79 79
query85 700 288 276 276
query86 324 300 305 300
query87 4348 4227 4296 4227
query88 2959 2363 2418 2363
query89 381 287 288 287
query90 1811 188 192 188
query91 125 100 101 100
query92 55 49 50 49
query93 1043 533 521 521
query94 516 279 299 279
query95 335 251 247 247
query96 580 267 265 265
query97 3204 3100 3063 3063
query98 216 199 219 199
query99 1495 1314 1282 1282
Total cold run time: 285488 ms
Total hot run time: 188201 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2350789674
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2348649247
<details>
<summary>TPC-H: <b>Total hot run time: 42866 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc00ceb4217f1ec8fbbc1943f0a8c175d3e5fc1e, data reload: false
------ Round 1 ----------------------------------
q1 17602 7339 7265 7265
q2 2032 181 175 175
q3 10501 1283 1381 1283
q4 10510 944 1043 944
q5 7737 3126 3151 3126
q6 239 156 153 153
q7 1030 636 622 622
q8 9453 1992 2049 1992
q9 6796 6310 6301 6301
q10 7009 2553 2530 2530
q11 425 251 248 248
q12 407 219 219 219
q13 17765 3016 3027 3016
q14 283 251 248 248
q15 577 535 542 535
q16 532 427 419 419
q17 979 950 946 946
q18 7768 6769 6791 6769
q19 1382 1228 1236 1228
q20 602 343 328 328
q21 3906 3530 3520 3520
q22 1091 999 1022 999
Total cold run time: 108626 ms
Total hot run time: 42866 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7186 7171 7182 7171
q2 354 235 245 235
q3 3057 3106 3043 3043
q4 2072 2111 2055 2055
q5 5686 5602 5669 5602
q6 243 148 156 148
q7 2135 1761 1786 1761
q8 3367 3430 3400 3400
q9 8840 8990 8733 8733
q10 3501 3548 3593 3548
q11 586 481 473 473
q12 867 625 595 595
q13 8535 3184 3192 3184
q14 310 280 286 280
q15 584 547 533 533
q16 501 480 474 474
q17 1789 1772 1754 1754
q18 8496 8091 7938 7938
q19 1774 1779 1753 1753
q20 2117 1875 1870 1870
q21 5938 5503 5598 5503
q22 1104 1022 1016 1016
Total cold run time: 69042 ms
Total hot run time: 61069 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #40333:
URL: https://github.com/apache/doris/pull/40333#discussion_r1742930518
##########
be/src/cloud/cloud_txn_delete_bitmap_cache.cpp:
##########
@@ -184,26 +184,34 @@ void CloudTxnDeleteBitmapCache::update_tablet_txn_info(TTransactionId transactio
LOG_INFO("update txn related delete bitmap")
.tag("txn_id", transaction_id)
.tag("tablt_id", tablet_id)
- .tag("delete_bitmap_size", charge);
+ .tag("delete_bitmap_size", charge)
+ .tag("publish_status", static_cast<int>(publish_status));
}
void CloudTxnDeleteBitmapCache::remove_expired_tablet_txn_info() {
TEST_SYNC_POINT_RETURN_WITH_VOID("CloudTxnDeleteBitmapCache::remove_expired_tablet_txn_info");
std::unique_lock<std::shared_mutex> wlock(_rwlock);
- while (!_expiration_txn.empty()) {
- auto iter = _expiration_txn.begin();
- if (_txn_map.find(iter->second) == _txn_map.end()) {
- _expiration_txn.erase(iter);
+ for (auto it = _expiration_txn.begin(); it != _expiration_txn.end();) {
+ if (_txn_map.find(it->second) == _txn_map.end()) {
+ it = _expiration_txn.erase(it);
continue;
}
int64_t current_time = duration_cast<std::chrono::seconds>(
std::chrono::system_clock::now().time_since_epoch())
.count();
- if (iter->first > current_time) {
+ if (it->first > current_time) {
break;
}
- auto txn_iter = _txn_map.find(iter->second);
- if ((txn_iter != _txn_map.end()) && (iter->first == txn_iter->second.txn_expiration)) {
+ if (*(_txn_map.find(it->second)->second.publish_status) != PublishStatus::SUCCEED) {
Review Comment:
If you don't clear the txn_info if it's not in SUCCEED status, the expiration time is use less
And if the txn failed, when would it be cleared?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2332536687
TeamCity be ut coverage result:
Function Coverage: 36.78% (9381/25507)
Line Coverage: 28.19% (77313/274212)
Region Coverage: 27.62% (39937/144588)
Branch Coverage: 24.26% (20322/83756)
Coverage Report: http://coverage.selectdb-in.cc/coverage/80a168255fcfd6f2091e6e545633736862062368_80a168255fcfd6f2091e6e545633736862062368/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2348670382
<details>
<summary>TPC-DS: <b>Total hot run time: 199980 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc00ceb4217f1ec8fbbc1943f0a8c175d3e5fc1e, data reload: false
query1 1240 882 875 875
query2 6404 1772 1758 1758
query3 10665 3808 3719 3719
query4 54653 25355 24231 24231
query5 5012 548 559 548
query6 310 173 172 172
query7 5498 311 307 307
query8 300 230 223 223
query9 5898 2603 2599 2599
query10 397 292 277 277
query11 16028 15579 15468 15468
query12 165 103 102 102
query13 1382 391 376 376
query14 10364 7381 7471 7381
query15 205 176 169 169
query16 6973 485 502 485
query17 1145 602 589 589
query18 1846 293 305 293
query19 200 164 151 151
query20 123 115 115 115
query21 214 105 106 105
query22 4871 4718 4322 4322
query23 34577 34049 33656 33656
query24 6084 3084 3107 3084
query25 525 398 406 398
query26 609 156 155 155
query27 1581 279 279 279
query28 2846 2110 2076 2076
query29 660 406 418 406
query30 226 152 157 152
query31 935 756 761 756
query32 77 51 52 51
query33 421 282 288 282
query34 904 494 495 494
query35 867 732 736 732
query36 1059 903 907 903
query37 141 80 81 80
query38 4038 3944 3914 3914
query39 1448 1394 1444 1394
query40 203 111 117 111
query41 47 49 43 43
query42 124 102 95 95
query43 488 455 444 444
query44 1206 780 768 768
query45 198 169 175 169
query46 1101 830 798 798
query47 1940 1764 1779 1764
query48 377 288 291 288
query49 720 447 475 447
query50 919 444 445 444
query51 7035 6951 7063 6951
query52 99 87 86 86
query53 254 183 180 180
query54 565 465 463 463
query55 79 85 77 77
query56 288 266 260 260
query57 1206 1103 1092 1092
query58 255 244 240 240
query59 2707 2459 2429 2429
query60 295 275 267 267
query61 103 100 101 100
query62 802 670 669 669
query63 215 189 193 189
query64 1607 700 722 700
query65 3261 3152 3194 3152
query66 695 310 306 306
query67 16088 15467 15631 15467
query68 2010 878 867 867
query69 437 331 337 331
query70 1148 1161 1149 1149
query71 369 352 348 348
query72 4642 3486 3743 3486
query73 585 587 585 585
query74 9181 9243 9076 9076
query75 3107 3049 3041 3041
query76 933 850 848 848
query77 495 415 412 412
query78 9460 9292 9275 9275
query79 930 889 862 862
query80 832 866 836 836
query81 444 271 271 271
query82 267 267 265 265
query83 194 193 191 191
query84 209 116 113 113
query85 694 572 397 397
query86 303 316 310 310
query87 4441 4378 4414 4378
query88 4167 4044 4037 4037
query89 377 364 368 364
query90 1348 309 302 302
query91 121 121 121 121
query92 80 78 82 78
query93 1027 1018 1037 1018
query94 542 375 387 375
query95 453 422 425 422
query96 468 465 468 465
query97 3170 3141 3161 3141
query98 239 232 238 232
query99 1326 1312 1308 1308
Total cold run time: 287767 ms
Total hot run time: 199980 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2332354755
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2332536023
<details>
<summary>ClickBench: <b>Total hot run time: 31.27 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 80a168255fcfd6f2091e6e545633736862062368, data reload: false
query1 0.04 0.04 0.05
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.52 0.49 0.49
query6 1.13 0.72 0.73
query7 0.02 0.01 0.02
query8 0.05 0.04 0.05
query9 0.54 0.51 0.49
query10 0.54 0.55 0.54
query11 0.16 0.12 0.12
query12 0.15 0.12 0.12
query13 0.59 0.59 0.59
query14 1.40 1.44 1.42
query15 0.82 0.82 0.81
query16 0.38 0.37 0.37
query17 1.03 1.04 1.04
query18 0.22 0.20 0.21
query19 1.92 1.84 1.83
query20 0.02 0.01 0.01
query21 15.40 0.65 0.64
query22 4.40 7.98 1.76
query23 18.31 1.32 1.24
query24 2.08 0.21 0.21
query25 0.15 0.08 0.08
query26 0.27 0.19 0.18
query27 0.08 0.08 0.07
query28 13.29 1.02 0.98
query29 12.59 3.32 3.33
query30 0.24 0.05 0.05
query31 2.87 0.40 0.39
query32 3.24 0.47 0.48
query33 2.99 3.00 3.00
query34 17.08 4.39 4.40
query35 4.44 4.42 4.43
query36 0.65 0.48 0.49
query37 0.19 0.15 0.16
query38 0.15 0.15 0.15
query39 0.04 0.04 0.04
query40 0.16 0.13 0.12
query41 0.10 0.05 0.05
query42 0.05 0.04 0.04
query43 0.05 0.05 0.04
Total cold run time: 110.33 s
Total hot run time: 31.27 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #40333:
URL: https://github.com/apache/doris/pull/40333#discussion_r1742930518
##########
be/src/cloud/cloud_txn_delete_bitmap_cache.cpp:
##########
@@ -184,26 +184,34 @@ void CloudTxnDeleteBitmapCache::update_tablet_txn_info(TTransactionId transactio
LOG_INFO("update txn related delete bitmap")
.tag("txn_id", transaction_id)
.tag("tablt_id", tablet_id)
- .tag("delete_bitmap_size", charge);
+ .tag("delete_bitmap_size", charge)
+ .tag("publish_status", static_cast<int>(publish_status));
}
void CloudTxnDeleteBitmapCache::remove_expired_tablet_txn_info() {
TEST_SYNC_POINT_RETURN_WITH_VOID("CloudTxnDeleteBitmapCache::remove_expired_tablet_txn_info");
std::unique_lock<std::shared_mutex> wlock(_rwlock);
- while (!_expiration_txn.empty()) {
- auto iter = _expiration_txn.begin();
- if (_txn_map.find(iter->second) == _txn_map.end()) {
- _expiration_txn.erase(iter);
+ for (auto it = _expiration_txn.begin(); it != _expiration_txn.end();) {
+ if (_txn_map.find(it->second) == _txn_map.end()) {
+ it = _expiration_txn.erase(it);
continue;
}
int64_t current_time = duration_cast<std::chrono::seconds>(
std::chrono::system_clock::now().time_since_epoch())
.count();
- if (iter->first > current_time) {
+ if (it->first > current_time) {
break;
}
- auto txn_iter = _txn_map.find(iter->second);
- if ((txn_iter != _txn_map.end()) && (iter->first == txn_iter->second.txn_expiration)) {
+ if (*(_txn_map.find(it->second)->second.publish_status) != PublishStatus::SUCCEED) {
Review Comment:
If you don't clear the txn_info when it's not in SUCCEED status, the expiration time is use less
And if the txn failed, when would it be cleared?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326340282
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326380143
<details>
<summary>TPC-H: <b>Total hot run time: 38387 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1ce3e56e944dface7106077a5a1b3c2661492eba, data reload: false
------ Round 1 ----------------------------------
q1 17862 4499 4460 4460
q2 2611 180 182 180
q3 11071 1133 1121 1121
q4 11211 739 713 713
q5 7824 2915 2830 2830
q6 231 139 148 139
q7 967 620 604 604
q8 9523 2058 2036 2036
q9 7288 6507 6529 6507
q10 7011 2199 2233 2199
q11 466 238 231 231
q12 400 227 220 220
q13 17940 3100 3097 3097
q14 281 246 235 235
q15 532 487 489 487
q16 598 501 508 501
q17 985 734 677 677
q18 7460 6874 6932 6874
q19 1390 1075 1038 1038
q20 681 332 334 332
q21 4289 2884 3061 2884
q22 1125 1022 1025 1022
Total cold run time: 111746 ms
Total hot run time: 38387 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4375 4353 4314 4314
q2 372 277 272 272
q3 2868 2656 2683 2656
q4 1941 1668 1711 1668
q5 5378 5391 5425 5391
q6 220 130 133 130
q7 2089 1715 1750 1715
q8 3187 3328 3346 3328
q9 8423 8429 8403 8403
q10 3465 3255 3230 3230
q11 588 516 498 498
q12 786 607 627 607
q13 11874 3099 3081 3081
q14 321 288 276 276
q15 520 479 485 479
q16 598 567 596 567
q17 1799 1477 1476 1476
q18 7740 7578 7539 7539
q19 1671 1617 1571 1571
q20 2052 1837 1800 1800
q21 5551 5280 5277 5277
q22 1110 1038 1027 1027
Total cold run time: 66928 ms
Total hot run time: 55305 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2348569423
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2332527427
<details>
<summary>TPC-DS: <b>Total hot run time: 192083 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 80a168255fcfd6f2091e6e545633736862062368, data reload: false
query1 1255 876 902 876
query2 6377 1877 1850 1850
query3 10611 4050 3927 3927
query4 59657 25196 23287 23287
query5 5396 505 494 494
query6 420 167 161 161
query7 5871 303 297 297
query8 323 239 234 234
query9 9214 2509 2502 2502
query10 489 264 256 256
query11 18103 14925 15349 14925
query12 169 109 107 107
query13 1582 431 395 395
query14 11178 7997 7042 7042
query15 241 180 179 179
query16 7616 473 500 473
query17 1149 581 604 581
query18 1954 303 308 303
query19 304 152 152 152
query20 125 111 112 111
query21 210 103 107 103
query22 4845 4489 4577 4489
query23 34702 33446 33401 33401
query24 5939 2821 2884 2821
query25 517 398 401 398
query26 683 159 155 155
query27 1775 279 286 279
query28 3679 2070 2042 2042
query29 644 425 418 418
query30 239 152 169 152
query31 923 754 772 754
query32 89 57 59 57
query33 482 286 287 286
query34 891 473 465 465
query35 820 708 714 708
query36 1083 935 960 935
query37 148 91 83 83
query38 3920 3909 3949 3909
query39 1437 1419 1374 1374
query40 197 118 112 112
query41 49 48 47 47
query42 116 96 97 96
query43 515 490 490 490
query44 1073 750 738 738
query45 200 172 169 169
query46 1077 754 735 735
query47 1901 1759 1814 1759
query48 382 300 305 300
query49 775 451 455 451
query50 840 417 424 417
query51 7014 6764 6882 6764
query52 99 90 88 88
query53 251 182 180 180
query54 566 448 465 448
query55 77 75 75 75
query56 280 270 273 270
query57 1193 1059 1066 1059
query58 308 235 225 225
query59 3002 2826 2780 2780
query60 290 263 254 254
query61 100 100 129 100
query62 739 652 623 623
query63 214 181 186 181
query64 2816 657 661 657
query65 3231 3104 3158 3104
query66 688 332 335 332
query67 15495 15262 15219 15219
query68 4391 578 559 559
query69 419 276 271 271
query70 1185 1111 1142 1111
query71 419 266 266 266
query72 6292 3953 3932 3932
query73 797 322 323 322
query74 9243 8828 8759 8759
query75 3358 2663 2672 2663
query76 1827 960 1006 960
query77 552 325 317 317
query78 9697 9060 9264 9060
query79 1019 551 537 537
query80 691 484 511 484
query81 535 234 230 230
query82 229 139 139 139
query83 173 150 146 146
query84 264 74 78 74
query85 843 286 323 286
query86 387 283 312 283
query87 4405 4351 4292 4292
query88 3367 2336 2313 2313
query89 392 287 280 280
query90 2140 186 183 183
query91 127 102 92 92
query92 58 50 48 48
query93 1094 531 540 531
query94 717 285 288 285
query95 339 249 251 249
query96 583 262 263 262
query97 3170 3061 3093 3061
query98 218 202 200 200
query99 1496 1234 1261 1234
Total cold run time: 308975 ms
Total hot run time: 192083 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2348679300
<details>
<summary>ClickBench: <b>Total hot run time: 30.94 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc00ceb4217f1ec8fbbc1943f0a8c175d3e5fc1e, data reload: false
query1 0.05 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.50 0.51 0.50
query6 1.14 0.73 0.73
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.58 0.55 0.52
query10 0.58 0.58 0.56
query11 0.16 0.12 0.12
query12 0.15 0.13 0.13
query13 0.63 0.62 0.60
query14 1.45 1.47 1.47
query15 0.90 0.88 0.86
query16 0.36 0.36 0.38
query17 1.04 1.04 1.02
query18 0.17 0.17 0.17
query19 1.93 1.86 1.79
query20 0.01 0.02 0.01
query21 15.39 0.70 0.68
query22 4.04 7.90 1.11
query23 18.11 1.34 1.33
query24 2.22 0.22 0.22
query25 0.18 0.09 0.08
query26 0.30 0.19 0.18
query27 0.08 0.08 0.09
query28 13.18 1.17 1.11
query29 12.51 3.32 3.32
query30 0.25 0.05 0.05
query31 2.91 0.42 0.41
query32 3.23 0.50 0.50
query33 3.02 3.09 3.06
query34 15.44 4.30 4.33
query35 4.36 4.34 4.34
query36 0.70 0.48 0.49
query37 0.19 0.16 0.16
query38 0.16 0.14 0.15
query39 0.05 0.04 0.04
query40 0.17 0.13 0.14
query41 0.10 0.05 0.05
query42 0.06 0.04 0.05
query43 0.05 0.04 0.04
Total cold run time: 108.39 s
Total hot run time: 30.94 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326339691
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326385347
TeamCity be ut coverage result:
Function Coverage: 36.81% (9382/25485)
Line Coverage: 28.26% (77416/273894)
Region Coverage: 27.66% (39950/144413)
Branch Coverage: 24.31% (20331/83624)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1ce3e56e944dface7106077a5a1b3c2661492eba_1ce3e56e944dface7106077a5a1b3c2661492eba/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2332508817
<details>
<summary>TPC-H: <b>Total hot run time: 38297 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 80a168255fcfd6f2091e6e545633736862062368, data reload: false
------ Round 1 ----------------------------------
q1 17616 5009 4478 4478
q2 2019 186 187 186
q3 11713 1018 1144 1018
q4 10519 748 741 741
q5 7753 2834 2866 2834
q6 232 138 138 138
q7 970 628 595 595
q8 9323 2084 2106 2084
q9 7242 6525 6531 6525
q10 7002 2154 2259 2154
q11 484 240 242 240
q12 392 219 220 219
q13 17757 3048 3060 3048
q14 288 244 237 237
q15 538 481 501 481
q16 536 420 426 420
q17 986 689 664 664
q18 7311 7038 6947 6947
q19 1394 1035 1061 1035
q20 708 340 337 337
q21 4105 2886 2943 2886
q22 1127 1030 1031 1030
Total cold run time: 110015 ms
Total hot run time: 38297 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4342 4308 4289 4289
q2 376 281 271 271
q3 2905 2681 2636 2636
q4 1898 1657 1595 1595
q5 5679 5704 5713 5704
q6 224 133 137 133
q7 2279 1891 1813 1813
q8 3284 3447 3488 3447
q9 8797 8849 8856 8849
q10 3652 3402 3361 3361
q11 591 496 505 496
q12 840 671 643 643
q13 13057 3213 3266 3213
q14 323 287 289 287
q15 550 491 492 491
q16 553 488 492 488
q17 1861 1586 1535 1535
q18 8202 7831 7845 7831
q19 1727 1581 1592 1581
q20 2208 1898 1867 1867
q21 5748 5501 5295 5295
q22 1129 1051 1050 1050
Total cold run time: 70225 ms
Total hot run time: 56875 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) check cloud_txn_delete_bitmap_cache's status before removing expired one [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2326413741
<details>
<summary>ClickBench: <b>Total hot run time: 31.99 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1ce3e56e944dface7106077a5a1b3c2661492eba, data reload: false
query1 0.05 0.04 0.04
query2 0.07 0.04 0.04
query3 0.22 0.04 0.05
query4 1.68 0.07 0.07
query5 0.51 0.50 0.49
query6 1.13 0.74 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.05
query9 0.55 0.48 0.47
query10 0.56 0.55 0.54
query11 0.16 0.12 0.11
query12 0.16 0.12 0.12
query13 0.61 0.58 0.59
query14 2.05 2.09 2.05
query15 0.89 0.84 0.82
query16 0.36 0.38 0.39
query17 0.99 1.05 1.02
query18 0.21 0.20 0.20
query19 1.95 1.81 1.77
query20 0.01 0.01 0.01
query21 15.41 0.66 0.65
query22 4.24 6.66 1.73
query23 18.25 1.51 1.31
query24 2.15 0.23 0.22
query25 0.16 0.09 0.08
query26 0.28 0.18 0.18
query27 0.08 0.07 0.07
query28 13.21 1.02 1.01
query29 12.61 3.38 3.37
query30 0.24 0.05 0.05
query31 2.87 0.40 0.39
query32 3.30 0.48 0.50
query33 3.05 3.02 3.00
query34 17.18 4.40 4.42
query35 4.52 4.56 4.48
query36 0.66 0.50 0.47
query37 0.18 0.15 0.15
query38 0.15 0.16 0.14
query39 0.05 0.04 0.04
query40 0.16 0.13 0.13
query41 0.10 0.05 0.04
query42 0.05 0.05 0.05
query43 0.04 0.04 0.03
Total cold run time: 111.17 s
Total hot run time: 31.99 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2350789686
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud-mow) make cloud_txn_delete_bitmap_cache's expired time more reasonable [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #40333:
URL: https://github.com/apache/doris/pull/40333#issuecomment-2332369296
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org