You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yujun777 (via GitHub)" <gi...@apache.org> on 2026/05/08 12:36:29 UTC
[PR] [fix](statistics) Skip statistics cache for system dbs [doris]
yujun777 opened a new pull request, #63089:
URL: https://github.com/apache/doris/pull/63089
### What problem does this PR solve?
Statistics cache may still try to load column statistics, partition column statistics, or histograms for system databases. This is not expected for information_schema, mysql, or the dynamically assigned __internal_schema database.
### Solution
- Add a system database check in StatisticConstants.
- Reuse the check in StatisticsCache for column statistics, partition statistics, and histograms.
- Resolve __internal_schema by looking up the internal catalog DB object so the dynamic db id is covered.
### Check List
- [x] FE UT
### Test
CUSTOM_MVN=/home/yujun/.sdkman/candidates/maven/current/bin/mvn bash ./run-fe-ut.sh --coverage --run org.apache.doris.statistics.StatisticsCacheTest
Result: BUILD SUCCESS. StatisticsCacheTest ran 6 tests with 0 failures and 0 errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #63089:
URL: https://github.com/apache/doris/pull/63089
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411308692
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411409425
# FE UT Coverage Report
Increment line coverage ` 57.14% (8/14)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63089_111668f7cc2528b3f7a933cb107eadc4bb645b0b/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63089_111668f7cc2528b3f7a933cb107eadc4bb645b0b/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411602699
<details>
<summary>TPC-H: <b>Total hot run time: 29524 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55f4e284ce15ef4c45adf826f54492a8214cc3ab, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17712 3885 3899 3885
q2 q3 10716 887 611 611
q4 4663 466 354 354
q5 7450 1312 1148 1148
q6 184 169 141 141
q7 908 956 754 754
q8 9310 1397 1247 1247
q9 5722 5384 5351 5351
q10 6296 2086 1831 1831
q11 466 271 255 255
q12 688 426 301 301
q13 18209 3354 2740 2740
q14 295 286 263 263
q15 q16 903 889 789 789
q17 1021 1102 697 697
q18 6502 5661 5612 5612
q19 1187 1140 1011 1011
q20 524 384 263 263
q21 4705 2387 1937 1937
q22 471 395 334 334
Total cold run time: 97932 ms
Total hot run time: 29524 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4605 4605 4516 4516
q2 q3 4694 4809 4215 4215
q4 2118 2203 1425 1425
q5 5030 5097 5267 5097
q6 207 172 144 144
q7 2054 1793 1675 1675
q8 3374 3105 3081 3081
q9 8418 8638 8470 8470
q10 4477 4464 4263 4263
q11 644 413 387 387
q12 707 767 528 528
q13 3263 3637 2965 2965
q14 316 324 271 271
q15 q16 784 796 695 695
q17 1340 1302 1247 1247
q18 8074 7160 7193 7160
q19 1159 1140 1170 1140
q20 2264 2234 1982 1982
q21 6154 5443 4888 4888
q22 518 470 396 396
Total cold run time: 60200 ms
Total hot run time: 54545 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411169595
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4406447761
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411264750
<details>
<summary>TPC-DS: <b>Total hot run time: 172037 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 111668f7cc2528b3f7a933cb107eadc4bb645b0b, data reload: false
query5 4289 685 512 512
query6 339 223 201 201
query7 4227 565 302 302
query8 324 228 212 212
query9 8821 3963 3999 3963
query10 494 341 297 297
query11 6045 2396 2158 2158
query12 211 137 130 130
query13 1289 583 431 431
query14 6932 5351 5095 5095
query14_1 4376 4370 4379 4370
query15 217 204 189 189
query16 1009 469 429 429
query17 1403 740 604 604
query18 2725 483 349 349
query19 329 204 166 166
query20 140 133 131 131
query21 229 149 114 114
query22 13588 14098 14392 14098
query23 17286 16540 16320 16320
query23_1 16323 16303 16299 16299
query24 7964 1802 1373 1373
query24_1 1384 1350 1401 1350
query25 566 481 456 456
query26 1298 301 170 170
query27 2670 598 338 338
query28 4336 1951 1967 1951
query29 1012 655 532 532
query30 306 237 194 194
query31 1118 1084 948 948
query32 84 72 73 72
query33 552 357 298 298
query34 1145 1127 640 640
query35 785 782 674 674
query36 1362 1368 1162 1162
query37 150 108 88 88
query38 3300 3219 3040 3040
query39 932 899 902 899
query39_1 882 916 873 873
query40 233 160 138 138
query41 67 66 64 64
query42 112 113 111 111
query43 318 323 281 281
query44
query45 215 209 191 191
query46 1083 1239 713 713
query47 2330 2348 2204 2204
query48 391 405 304 304
query49 650 553 449 449
query50 702 288 222 222
query51 4357 4426 4154 4154
query52 105 107 96 96
query53 266 285 208 208
query54 322 303 266 266
query55 94 90 87 87
query56 327 315 319 315
query57 1465 1413 1338 1338
query58 306 317 267 267
query59 1618 1709 1447 1447
query60 353 334 326 326
query61 157 158 155 155
query62 708 638 556 556
query63 239 205 210 205
query64 2365 822 685 685
query65
query66 1682 513 390 390
query67 30241 30013 29839 29839
query68
query69 460 342 308 308
query70 992 974 974 974
query71 307 277 270 270
query72 2917 2748 2481 2481
query73 823 756 437 437
query74 5093 4928 4710 4710
query75 2772 2656 2335 2335
query76 2297 1160 747 747
query77 415 407 345 345
query78 12977 12972 12337 12337
query79 1525 1023 754 754
query80 1387 580 485 485
query81 522 279 259 259
query82 936 161 123 123
query83 352 273 246 246
query84 269 138 108 108
query85 919 538 442 442
query86 461 354 338 338
query87 3390 3361 3211 3211
query88 3535 2671 2667 2667
query89 439 386 335 335
query90 1949 185 174 174
query91 181 177 146 146
query92 75 72 68 68
query93 1078 961 558 558
query94 724 343 305 305
query95 651 463 369 369
query96 1065 781 358 358
query97 2694 2758 2557 2557
query98 240 226 248 226
query99 1124 1136 991 991
Total cold run time: 256442 ms
Total hot run time: 172037 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411803955
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411623458
<details>
<summary>TPC-DS: <b>Total hot run time: 171110 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55f4e284ce15ef4c45adf826f54492a8214cc3ab, data reload: false
query5 4352 663 529 529
query6 337 228 208 208
query7 4311 570 317 317
query8 337 244 231 231
query9 8863 4108 4102 4102
query10 475 353 310 310
query11 5912 2397 2224 2224
query12 183 135 140 135
query13 1284 621 425 425
query14 6774 5377 5061 5061
query14_1 4404 4396 4363 4363
query15 211 201 189 189
query16 1007 450 458 450
query17 1236 803 620 620
query18 2713 476 355 355
query19 222 202 161 161
query20 141 136 130 130
query21 212 140 118 118
query22 13668 13645 13410 13410
query23 17094 16418 15923 15923
query23_1 16254 16204 16225 16204
query24 7410 1754 1371 1371
query24_1 1360 1351 1353 1351
query25 562 493 449 449
query26 1316 324 168 168
query27 2656 614 349 349
query28 4389 1984 1968 1968
query29 996 634 520 520
query30 302 246 199 199
query31 1125 1076 953 953
query32 96 74 75 74
query33 533 397 286 286
query34 1148 1148 639 639
query35 768 791 670 670
query36 1367 1305 1111 1111
query37 148 101 93 93
query38 3208 3162 3028 3028
query39 931 928 892 892
query39_1 875 871 877 871
query40 235 163 135 135
query41 64 64 60 60
query42 114 114 112 112
query43 320 324 294 294
query44
query45 240 200 196 196
query46 1059 1220 744 744
query47 2235 2367 2211 2211
query48 386 426 300 300
query49 643 530 426 426
query50 725 283 225 225
query51 4372 4281 4286 4281
query52 104 107 94 94
query53 263 282 208 208
query54 311 267 261 261
query55 96 88 86 86
query56 308 299 295 295
query57 1436 1432 1326 1326
query58 326 288 264 264
query59 1608 1641 1456 1456
query60 352 348 334 334
query61 154 149 151 149
query62 679 633 580 580
query63 246 209 215 209
query64 2246 831 672 672
query65
query66 1665 514 400 400
query67 29989 30021 29871 29871
query68
query69 450 350 345 345
query70 1059 985 990 985
query71 297 278 266 266
query72 2863 2784 2417 2417
query73 859 726 440 440
query74 5051 4898 4717 4717
query75 2782 2649 2329 2329
query76 2318 1139 814 814
query77 409 422 349 349
query78 12833 12781 12346 12346
query79 1461 975 745 745
query80 674 586 511 511
query81 449 279 247 247
query82 1329 158 123 123
query83 355 272 246 246
query84 265 144 112 112
query85 861 497 434 434
query86 411 366 339 339
query87 3387 3361 3226 3226
query88 3563 2703 2676 2676
query89 436 384 339 339
query90 1884 181 179 179
query91 174 167 141 141
query92 87 79 74 74
query93 956 969 571 571
query94 519 346 264 264
query95 672 397 359 359
query96 1022 734 345 345
query97 2699 2696 2562 2562
query98 241 236 240 236
query99 1125 1143 986 986
Total cold run time: 253637 ms
Total hot run time: 171110 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411240309
<details>
<summary>TPC-H: <b>Total hot run time: 29393 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 111668f7cc2528b3f7a933cb107eadc4bb645b0b, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17792 3971 3927 3927
q2 q3 10733 868 600 600
q4 4656 461 343 343
q5 7456 1306 1112 1112
q6 190 163 136 136
q7 918 929 758 758
q8 9410 1354 1257 1257
q9 6111 5330 5294 5294
q10 6312 2056 1783 1783
q11 470 269 251 251
q12 698 424 288 288
q13 18221 3290 2759 2759
q14 297 285 261 261
q15 q16 896 865 782 782
q17 1032 1073 734 734
q18 6459 5670 5641 5641
q19 1387 1227 1060 1060
q20 504 394 260 260
q21 4573 2262 1850 1850
q22 409 352 297 297
Total cold run time: 98524 ms
Total hot run time: 29393 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4178 4057 4063 4057
q2 q3 4629 4755 4190 4190
q4 2082 2161 1375 1375
q5 4983 4985 5261 4985
q6 189 162 130 130
q7 2041 1753 2027 1753
q8 3367 3172 3128 3128
q9 8366 8419 8471 8419
q10 4460 4492 4278 4278
q11 620 433 407 407
q12 681 737 519 519
q13 3159 3638 2879 2879
q14 307 309 269 269
q15 q16 766 785 674 674
q17 1395 1314 1426 1314
q18 8086 7166 7024 7024
q19 1133 1155 1179 1155
q20 2220 2222 1939 1939
q21 6043 5332 4918 4918
q22 540 519 428 428
Total cold run time: 59245 ms
Total hot run time: 53841 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics) Skip statistics cache for system dbs [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #63089:
URL: https://github.com/apache/doris/pull/63089#issuecomment-4411803866
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org