You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zddr (via GitHub)" <gi...@apache.org> on 2025/05/07 12:18:47 UTC
[PR] [enhance](catalog)add remove listener for catalog cache [doris]
zddr opened a new pull request, #50673:
URL: https://github.com/apache/doris/pull/50673
### What problem does this PR solve?
Print logs when cache is deleted
```
2025-05-07 20:15:33,071 WARN (NotCheckpointCommonRefreshExecutor-19|203) [CacheLogRemovalListener.onRemoval():38] [ExternalSchemaCache schemaCache]Key: SchemaCacheKey{dbName='tpcds10000_oss', tblName='call_center'} was removed. Cause: SIZE
```
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
add remove listener for catalog cache
### Release note
add remove listener for catalog cache
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858530709
<details>
<summary>ClickBench: <b>Total hot run time: 29.32 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8560f77b30838cf68e3a7698a66bd8af03576555, data reload: false
query1 0.03 0.04 0.03
query2 0.14 0.10 0.11
query3 0.26 0.18 0.19
query4 1.59 0.19 0.19
query5 0.59 0.59 0.61
query6 1.18 0.72 0.73
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.57 0.52 0.51
query10 0.56 0.59 0.58
query11 0.15 0.11 0.12
query12 0.14 0.12 0.11
query13 0.61 0.60 0.60
query14 0.80 0.78 0.81
query15 0.89 0.84 0.85
query16 0.40 0.38 0.38
query17 1.05 1.05 1.02
query18 0.20 0.19 0.19
query19 1.88 1.75 1.80
query20 0.01 0.01 0.02
query21 15.39 0.92 0.54
query22 0.75 1.12 0.64
query23 15.03 1.38 0.62
query24 8.06 1.06 0.99
query25 0.49 0.10 0.14
query26 0.58 0.17 0.15
query27 0.05 0.05 0.05
query28 9.51 0.85 0.44
query29 12.52 3.88 3.24
query30 0.26 0.09 0.06
query31 2.81 0.60 0.39
query32 3.22 0.55 0.47
query33 3.07 3.17 3.07
query34 15.72 5.10 4.50
query35 4.52 4.53 4.52
query36 0.66 0.50 0.48
query37 0.09 0.06 0.06
query38 0.06 0.04 0.04
query39 0.03 0.02 0.03
query40 0.17 0.14 0.14
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.03 0.02
Total cold run time: 104.26 s
Total hot run time: 29.32 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr closed pull request #50673: [enhance](catalog)add remove listener for catalog cache
URL: https://github.com/apache/doris/pull/50673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858467189
<details>
<summary>TPC-H: <b>Total hot run time: 33705 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8560f77b30838cf68e3a7698a66bd8af03576555, data reload: false
------ Round 1 ----------------------------------
q1 26437 5062 4988 4988
q2 2082 280 198 198
q3 10441 1256 717 717
q4 10259 1018 534 534
q5 8114 2411 2350 2350
q6 184 161 129 129
q7 918 731 612 612
q8 9313 1281 1104 1104
q9 6825 5033 5089 5033
q10 6864 2316 1860 1860
q11 492 281 269 269
q12 333 350 209 209
q13 17769 3673 3037 3037
q14 228 221 211 211
q15 534 484 480 480
q16 417 425 370 370
q17 601 864 357 357
q18 7613 7284 7086 7086
q19 1497 956 547 547
q20 328 331 220 220
q21 3947 3395 2425 2425
q22 1049 1001 969 969
Total cold run time: 116245 ms
Total hot run time: 33705 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5105 5083 5062 5062
q2 233 323 230 230
q3 2197 2666 2275 2275
q4 1423 1800 1425 1425
q5 4502 4445 4437 4437
q6 206 171 126 126
q7 1979 1902 1725 1725
q8 2565 2485 2490 2485
q9 7180 7201 7070 7070
q10 3017 3201 2748 2748
q11 566 496 488 488
q12 653 757 631 631
q13 3463 3945 3318 3318
q14 287 308 267 267
q15 524 470 473 470
q16 422 481 445 445
q17 1179 1488 1402 1402
q18 7730 7550 7467 7467
q19 822 839 833 833
q20 2005 2016 1910 1910
q21 5100 4659 4477 4477
q22 1041 1025 978 978
Total cold run time: 52199 ms
Total hot run time: 50269 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858507011
<details>
<summary>TPC-DS: <b>Total hot run time: 185690 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8560f77b30838cf68e3a7698a66bd8af03576555, data reload: false
query1 974 498 498 498
query2 6597 1845 1807 1807
query3 6753 219 219 219
query4 26123 23220 23008 23008
query5 4296 612 447 447
query6 293 193 202 193
query7 4606 490 289 289
query8 288 233 246 233
query9 8624 2537 2541 2537
query10 451 316 253 253
query11 15895 15148 14824 14824
query12 158 106 102 102
query13 1635 503 390 390
query14 9044 6216 6197 6197
query15 205 188 172 172
query16 7188 636 485 485
query17 1211 727 569 569
query18 1987 407 320 320
query19 197 192 164 164
query20 124 114 116 114
query21 219 121 109 109
query22 4022 4271 4196 4196
query23 34472 33116 33005 33005
query24 8380 2395 2376 2376
query25 559 492 423 423
query26 1251 270 172 172
query27 2734 497 377 377
query28 4290 2068 2049 2049
query29 777 547 414 414
query30 285 216 183 183
query31 947 854 751 751
query32 71 67 69 67
query33 541 376 295 295
query34 775 835 504 504
query35 785 796 763 763
query36 945 1024 903 903
query37 112 101 85 85
query38 4113 4173 4122 4122
query39 1446 1394 1391 1391
query40 209 121 103 103
query41 56 56 56 56
query42 124 103 103 103
query43 498 506 472 472
query44 1278 797 799 797
query45 174 169 169 169
query46 831 1025 625 625
query47 1757 1758 1755 1755
query48 379 408 299 299
query49 779 519 428 428
query50 628 672 393 393
query51 4252 4108 4049 4049
query52 98 110 94 94
query53 219 244 186 186
query54 588 568 504 504
query55 83 79 85 79
query56 308 320 295 295
query57 1132 1110 1091 1091
query58 268 250 262 250
query59 2607 2618 2513 2513
query60 311 314 294 294
query61 135 129 129 129
query62 814 726 652 652
query63 224 181 182 181
query64 4357 1051 654 654
query65 4331 4252 4264 4252
query66 1189 403 308 308
query67 15679 15435 15275 15275
query68 8417 876 503 503
query69 474 300 261 261
query70 1228 1113 1132 1113
query71 458 311 286 286
query72 5245 4641 4615 4615
query73 707 566 346 346
query74 9004 9117 8944 8944
query75 3868 3189 2721 2721
query76 3608 1228 760 760
query77 800 371 292 292
query78 10085 10144 9336 9336
query79 2239 816 558 558
query80 621 514 422 422
query81 485 264 220 220
query82 471 127 94 94
query83 283 244 226 226
query84 311 109 86 86
query85 808 357 313 313
query86 394 319 308 308
query87 4438 4500 4344 4344
query88 3590 2232 2215 2215
query89 388 314 285 285
query90 1828 205 210 205
query91 145 141 119 119
query92 71 59 57 57
query93 1668 941 574 574
query94 673 401 309 309
query95 381 293 288 288
query96 473 557 274 274
query97 3155 3264 3110 3110
query98 227 210 206 206
query99 1457 1424 1288 1288
Total cold run time: 274681 ms
Total hot run time: 185690 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858379365
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2861292182
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2861292423
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858370705
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org