You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Lchangliang (via GitHub)" <gi...@apache.org> on 2024/04/25 14:04:38 UTC
[PR] (cloud-merge) Support to validate TTL keys [doris]
Lchangliang opened a new pull request, #34142:
URL: https://github.com/apache/doris/pull/34142
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2813
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2085006715
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "Lchangliang (via GitHub)" <gi...@apache.org>.
Lchangliang commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2077281537
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2085208491
TeamCity be ut coverage result:
Function Coverage: 35.61% (8935/25091)
Line Coverage: 27.24% (73744/270733)
Region Coverage: 26.42% (38093/144158)
Branch Coverage: 23.19% (19413/83696)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a674a2eb1a67274821eb8097102bd0160a622535_a674a2eb1a67274821eb8097102bd0160a622535/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #34142:
URL: https://github.com/apache/doris/pull/34142
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2084408437
<details>
<summary>TPC-DS: <b>Total hot run time: 186482 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1c65e4e8deb05c9e60878043ceb9ecc7ddad0172, data reload: false
query1 911 355 355 355
query2 6847 2395 2413 2395
query3 6653 205 205 205
query4 22869 21276 21147 21147
query5 4179 427 418 418
query6 279 178 183 178
query7 4589 290 287 287
query8 241 181 205 181
query9 8493 2334 2330 2330
query10 418 256 239 239
query11 14629 14117 14139 14117
query12 136 88 90 88
query13 1632 353 367 353
query14 10032 8298 8267 8267
query15 264 179 176 176
query16 8174 254 256 254
query17 1863 548 558 548
query18 2112 282 273 273
query19 306 152 154 152
query20 92 88 86 86
query21 198 126 124 124
query22 5078 4879 4884 4879
query23 33844 33107 32986 32986
query24 11103 2957 2914 2914
query25 640 391 376 376
query26 1540 157 152 152
query27 2918 325 346 325
query28 7038 2050 2034 2034
query29 951 627 616 616
query30 301 148 153 148
query31 998 737 711 711
query32 96 56 60 56
query33 758 249 255 249
query34 1022 476 482 476
query35 817 659 659 659
query36 1099 902 907 902
query37 140 69 69 69
query38 3151 2983 2961 2961
query39 1578 1577 1532 1532
query40 278 131 128 128
query41 42 41 42 41
query42 103 97 95 95
query43 585 531 526 526
query44 1224 720 739 720
query45 279 261 252 252
query46 1077 706 696 696
query47 1970 1891 1920 1891
query48 372 294 297 294
query49 1173 407 407 407
query50 791 382 379 379
query51 6723 6614 6552 6552
query52 103 93 91 91
query53 356 285 276 276
query54 313 248 241 241
query55 78 71 71 71
query56 247 228 229 228
query57 1245 1136 1126 1126
query58 261 208 207 207
query59 3287 3157 3326 3157
query60 262 247 241 241
query61 178 85 85 85
query62 661 453 436 436
query63 305 287 282 282
query64 9614 7190 7275 7190
query65 3063 3020 3030 3020
query66 1048 349 321 321
query67 15822 15083 14948 14948
query68 9681 565 572 565
query69 590 304 298 298
query70 1415 1130 1096 1096
query71 511 264 261 261
query72 8182 2511 2325 2325
query73 1559 321 331 321
query74 6552 6085 6136 6085
query75 4486 2656 2636 2636
query76 5819 1061 980 980
query77 669 266 267 266
query78 10951 10100 10251 10100
query79 11273 531 523 523
query80 2162 424 428 424
query81 502 218 223 218
query82 233 101 110 101
query83 220 166 165 165
query84 263 82 82 82
query85 1145 260 261 260
query86 344 315 292 292
query87 3342 3117 3073 3073
query88 5114 2339 2347 2339
query89 522 386 369 369
query90 2448 182 192 182
query91 127 97 100 97
query92 59 46 47 46
query93 6740 518 499 499
query94 1613 187 183 183
query95 409 302 311 302
query96 619 272 265 265
query97 3151 2943 2984 2943
query98 241 217 217 217
query99 1066 883 850 850
Total cold run time: 311457 ms
Total hot run time: 186482 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "Lchangliang (via GitHub)" <gi...@apache.org>.
Lchangliang commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2084330994
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "Lchangliang (via GitHub)" <gi...@apache.org>.
Lchangliang commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2077313589
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2081312204
<details>
<summary>TPC-DS: <b>Total hot run time: 187636 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4dc469fbd475737f33631e37c35be58ee712e3c9, data reload: false
query1 913 358 337 337
query2 6251 2467 2343 2343
query3 6636 203 212 203
query4 25909 21839 21769 21769
query5 3785 425 410 410
query6 264 194 178 178
query7 4568 301 286 286
query8 248 202 208 202
query9 8666 2395 2389 2389
query10 428 264 255 255
query11 15337 14768 14868 14768
query12 124 89 88 88
query13 1653 369 366 366
query14 10431 8589 7472 7472
query15 245 170 161 161
query16 8217 256 262 256
query17 1897 572 556 556
query18 2102 279 272 272
query19 338 150 152 150
query20 89 83 86 83
query21 195 120 129 120
query22 5027 4845 4842 4842
query23 34333 33395 33512 33395
query24 10848 2923 2956 2923
query25 603 364 364 364
query26 1107 149 146 146
query27 2342 316 312 312
query28 7115 2039 2023 2023
query29 874 601 604 601
query30 245 156 151 151
query31 969 732 746 732
query32 91 51 50 50
query33 764 245 250 245
query34 1008 498 467 467
query35 816 672 665 665
query36 1063 888 899 888
query37 137 69 73 69
query38 3135 3006 3030 3006
query39 1568 1538 1536 1536
query40 198 123 138 123
query41 40 37 37 37
query42 100 92 97 92
query43 561 546 570 546
query44 1167 735 750 735
query45 274 256 254 254
query46 1068 720 708 708
query47 1907 1834 1851 1834
query48 366 289 296 289
query49 848 402 401 401
query50 777 372 367 367
query51 6919 6716 6674 6674
query52 103 85 90 85
query53 347 272 300 272
query54 305 240 238 238
query55 79 72 75 72
query56 243 228 217 217
query57 1210 1137 1130 1130
query58 224 195 205 195
query59 3466 3082 3024 3024
query60 257 244 243 243
query61 98 90 90 90
query62 654 452 446 446
query63 301 274 278 274
query64 8485 7227 7144 7144
query65 3077 3060 3051 3051
query66 827 324 340 324
query67 15693 15090 15066 15066
query68 7522 541 552 541
query69 552 302 314 302
query70 1255 1167 1149 1149
query71 514 309 258 258
query72 8160 2615 2438 2438
query73 729 317 316 316
query74 6617 6070 6145 6070
query75 4261 2723 2695 2695
query76 5144 980 964 964
query77 625 262 270 262
query78 11101 10278 10271 10271
query79 8488 519 523 519
query80 1543 431 433 431
query81 498 225 226 225
query82 850 91 89 89
query83 199 165 162 162
query84 262 86 88 86
query85 1394 268 272 268
query86 451 300 273 273
query87 3292 3071 3124 3071
query88 4847 2335 2323 2323
query89 538 371 362 362
query90 2015 186 180 180
query91 124 100 100 100
query92 64 47 47 47
query93 6993 516 503 503
query94 1096 175 179 175
query95 396 299 298 298
query96 614 261 260 260
query97 3129 2919 2958 2919
query98 229 219 216 216
query99 1231 853 877 853
Total cold run time: 304668 ms
Total hot run time: 187636 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2085211650
<details>
<summary>TPC-DS: <b>Total hot run time: 188196 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a674a2eb1a67274821eb8097102bd0160a622535, data reload: false
query1 904 362 340 340
query2 6160 2321 2410 2321
query3 6658 204 209 204
query4 24150 21750 21818 21750
query5 3868 437 431 431
query6 261 172 181 172
query7 4550 302 299 299
query8 240 187 200 187
query9 8487 2414 2388 2388
query10 414 240 245 240
query11 15365 14834 14696 14696
query12 116 83 86 83
query13 1626 375 360 360
query14 8956 8261 8568 8261
query15 284 188 186 186
query16 8493 284 259 259
query17 1997 570 550 550
query18 2141 289 269 269
query19 333 157 158 157
query20 90 89 81 81
query21 204 124 124 124
query22 5209 4770 4910 4770
query23 33796 33329 33299 33299
query24 10620 2870 2864 2864
query25 585 344 353 344
query26 1212 151 148 148
query27 2337 313 316 313
query28 7232 2051 2057 2051
query29 846 624 596 596
query30 228 147 150 147
query31 918 731 725 725
query32 87 50 51 50
query33 735 244 236 236
query34 1003 473 481 473
query35 833 682 669 669
query36 1088 920 912 912
query37 129 67 65 65
query38 3130 2993 3026 2993
query39 1607 1530 1526 1526
query40 210 125 124 124
query41 40 37 37 37
query42 101 100 97 97
query43 599 536 554 536
query44 1141 728 744 728
query45 257 250 255 250
query46 1073 727 694 694
query47 1953 1849 1861 1849
query48 367 295 293 293
query49 837 393 376 376
query50 753 394 379 379
query51 6700 6587 6672 6587
query52 106 91 92 91
query53 350 280 277 277
query54 301 233 233 233
query55 76 71 73 71
query56 237 213 222 213
query57 1192 1133 1102 1102
query58 228 194 206 194
query59 3463 3126 3127 3126
query60 254 238 241 238
query61 107 103 106 103
query62 661 443 448 443
query63 317 289 291 289
query64 8617 7330 7259 7259
query65 3118 3095 3058 3058
query66 811 335 342 335
query67 15443 15013 14988 14988
query68 9273 561 555 555
query69 567 319 315 315
query70 1146 1144 1093 1093
query71 528 276 277 276
query72 8554 2662 2529 2529
query73 1104 325 324 324
query74 6511 6157 6065 6065
query75 4637 2637 2620 2620
query76 5429 1003 1057 1003
query77 682 265 282 265
query78 11129 10216 10213 10213
query79 11113 520 515 515
query80 1747 437 436 436
query81 489 216 224 216
query82 235 93 91 91
query83 216 173 166 166
query84 273 85 82 82
query85 1204 275 260 260
query86 344 313 301 301
query87 3341 3101 3056 3056
query88 5587 2334 2329 2329
query89 512 391 378 378
query90 2414 183 183 183
query91 124 99 98 98
query92 55 46 46 46
query93 7108 528 512 512
query94 1586 180 184 180
query95 407 305 300 300
query96 616 267 262 262
query97 3163 2966 2931 2931
query98 228 221 218 218
query99 1103 869 886 869
Total cold run time: 307629 ms
Total hot run time: 188196 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2084415706
TeamCity be ut coverage result:
Function Coverage: 35.57% (8935/25119)
Line Coverage: 27.21% (73735/270991)
Region Coverage: 26.40% (38089/144257)
Branch Coverage: 23.18% (19414/83744)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1c65e4e8deb05c9e60878043ceb9ecc7ddad0172_1c65e4e8deb05c9e60878043ceb9ecc7ddad0172/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2084396491
<details>
<summary>TPC-H: <b>Total hot run time: 40716 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1c65e4e8deb05c9e60878043ceb9ecc7ddad0172, data reload: false
------ Round 1 ----------------------------------
q1 18147 4663 4428 4428
q2 2567 191 194 191
q3 10971 1198 1179 1179
q4 10579 883 916 883
q5 7754 2767 2728 2728
q6 217 131 130 130
q7 1001 612 590 590
q8 9220 2138 2080 2080
q9 9286 6670 6684 6670
q10 8951 3763 3850 3763
q11 463 232 231 231
q12 427 221 222 221
q13 17755 3045 2945 2945
q14 268 218 211 211
q15 510 479 488 479
q16 520 389 378 378
q17 985 665 750 665
q18 8076 7507 7478 7478
q19 4382 1546 1536 1536
q20 642 304 310 304
q21 5082 3350 3937 3350
q22 351 286 276 276
Total cold run time: 118154 ms
Total hot run time: 40716 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4365 4238 4234 4234
q2 389 279 269 269
q3 3080 2706 2767 2706
q4 1863 1626 1614 1614
q5 5297 5309 5335 5309
q6 212 122 123 122
q7 2275 1891 1929 1891
q8 3221 3389 3329 3329
q9 8499 8507 8494 8494
q10 3972 3732 3729 3729
q11 604 501 499 499
q12 762 577 582 577
q13 17273 2984 2932 2932
q14 289 276 280 276
q15 520 481 473 473
q16 484 426 440 426
q17 1782 1490 1465 1465
q18 7650 7605 7458 7458
q19 2340 1543 1609 1543
q20 2004 1756 1752 1752
q21 5067 4815 4826 4815
q22 576 476 497 476
Total cold run time: 72524 ms
Total hot run time: 54389 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2085003452
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "Lchangliang (via GitHub)" <gi...@apache.org>.
Lchangliang commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2084332186
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2079434506
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2081316950
TeamCity be ut coverage result:
Function Coverage: 35.52% (8927/25134)
Line Coverage: 27.15% (73490/270722)
Region Coverage: 26.34% (37965/144155)
Branch Coverage: 23.13% (19343/83626)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4dc469fbd475737f33631e37c35be58ee712e3c9_4dc469fbd475737f33631e37c35be58ee712e3c9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #34142:
URL: https://github.com/apache/doris/pull/34142#discussion_r1579549430
##########
be/src/cloud/cloud_storage_engine.cpp:
##########
@@ -262,11 +264,44 @@ Status CloudStorageEngine::start_bg_threads() {
RETURN_IF_ERROR(Thread::create(
"StorageEngine", "lease_compaction_thread",
[this]() { this->_lease_compaction_thread_callback(); }, &_bg_threads.emplace_back()));
+
+ if (config::file_cache_ttl_valid_check_interval_second != 0) {
+ RETURN_IF_ERROR(Thread::create(
+ "StorageEngine", "check_file_cache_ttl_block_valid_thread",
+ [this]() { this->_check_file_cache_ttl_block_valid(); },
+ &_bg_threads.emplace_back()));
+ LOG(INFO) << "check file cache ttl block valid thread started";
+ }
+
LOG(INFO) << "lease compaction thread started";
return Status::OK();
}
+void CloudStorageEngine::_check_file_cache_ttl_block_valid() {
+ int64_t interval_seconds = config::file_cache_ttl_valid_check_interval_second / 2;
+ auto check_ttl = [](const std::weak_ptr<CloudTablet>& tablet_wk) {
+ auto tablet = tablet_wk.lock();
+ if (!tablet) return;
Review Comment:
warning: statement should be inside braces [readability-braces-around-statements]
```suggestion
if (!tablet) { return;
}
```
##########
be/test/io/cache/block_file_cache_test.cpp:
##########
@@ -3646,4 +3646,68 @@ TEST_F(BlockFileCacheTest, remove_if_cached_when_isnt_releasable) {
ASSERT_TRUE(blocks[0]->finalize().ok());
}
+TEST_F(BlockFileCacheTest, recyle_unvalid_ttl_async) {
+ config::file_cache_ttl_valid_check_interval_second = 4;
+ if (fs::exists(cache_base_path)) {
+ fs::remove_all(cache_base_path);
+ }
+ fs::create_directories(cache_base_path);
+ TUniqueId query_id;
+ query_id.hi = 1;
+ query_id.lo = 1;
+ io::FileCacheSettings settings;
+ settings.query_queue_size = 30;
+ settings.query_queue_elements = 5;
+ settings.index_queue_size = 30;
+ settings.index_queue_elements = 5;
+ settings.disposable_queue_size = 30;
+ settings.disposable_queue_elements = 5;
+ settings.capacity = 90;
+ settings.max_file_block_size = 30;
+ settings.max_query_cache_size = 30;
+ io::CacheContext context;
+ context.query_id = query_id;
+ auto key = io::BlockFileCache::hash("key1");
+ io::BlockFileCache cache(cache_base_path, settings);
+ context.cache_type = io::FileCacheType::TTL;
+ context.expiration_time = UnixSeconds() + 3600;
+ auto sp = SyncPoint::get_instance();
Review Comment:
warning: 'auto sp' can be declared as 'auto *sp' [readability-qualified-auto]
```suggestion
00;auto *
```
##########
be/src/cloud/cloud_storage_engine.cpp:
##########
@@ -262,11 +264,44 @@
RETURN_IF_ERROR(Thread::create(
"StorageEngine", "lease_compaction_thread",
[this]() { this->_lease_compaction_thread_callback(); }, &_bg_threads.emplace_back()));
+
+ if (config::file_cache_ttl_valid_check_interval_second != 0) {
+ RETURN_IF_ERROR(Thread::create(
+ "StorageEngine", "check_file_cache_ttl_block_valid_thread",
+ [this]() { this->_check_file_cache_ttl_block_valid(); },
+ &_bg_threads.emplace_back()));
+ LOG(INFO) << "check file cache ttl block valid thread started";
+ }
+
LOG(INFO) << "lease compaction thread started";
return Status::OK();
}
+void CloudStorageEngine::_check_file_cache_ttl_block_valid() {
+ int64_t interval_seconds = config::file_cache_ttl_valid_check_interval_second / 2;
+ auto check_ttl = [](const std::weak_ptr<CloudTablet>& tablet_wk) {
+ auto tablet = tablet_wk.lock();
+ if (!tablet) return;
+ if (tablet->tablet_meta()->ttl_seconds() == 0) return;
Review Comment:
warning: statement should be inside braces [readability-braces-around-statements]
```suggestion
if (tablet->tablet_meta()->ttl_seconds() == 0) { return;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2085078709
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2077281435
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "Lchangliang (via GitHub)" <gi...@apache.org>.
Lchangliang commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2081291959
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] (cloud-merge) Support to validate TTL keys [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34142:
URL: https://github.com/apache/doris/pull/34142#issuecomment-2084351896
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org