You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2024/06/03 11:35:52 UTC
[PR] [feature](statistics)Remove not exist partitino stats while analyzing. [doris]
Jibing-Li opened a new pull request, #35814:
URL: https://github.com/apache/doris/pull/35814
## Proposed changes
Remove not exist partitino stats while analyzing.
<!--Describe your changes.-->
When doing full analyze, we need to drop not existing partition's statistics before analyzing. Because user may drop partition at any time. If we don't drop expired partition stats, we may get wrong table level stats by merging all partition stats.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146852191
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partitino stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2145036881
<details>
<summary>ClickBench: <b>Total hot run time: 29.81 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6acee37cf05109121f1d639ee2941aee12e80667, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.65 0.09 0.09
query5 0.49 0.49 0.50
query6 1.13 0.74 0.73
query7 0.02 0.02 0.01
query8 0.05 0.05 0.04
query9 0.54 0.49 0.48
query10 0.52 0.55 0.54
query11 0.16 0.12 0.11
query12 0.15 0.11 0.12
query13 0.60 0.59 0.60
query14 0.80 0.76 0.77
query15 0.83 0.80 0.81
query16 0.37 0.37 0.37
query17 1.03 0.94 0.97
query18 0.22 0.22 0.27
query19 1.76 1.69 1.80
query20 0.02 0.02 0.02
query21 15.63 0.65 0.65
query22 4.64 8.11 1.23
query23 18.19 1.39 1.26
query24 1.97 0.20 0.21
query25 0.13 0.08 0.08
query26 0.28 0.18 0.17
query27 0.08 0.09 0.08
query28 13.34 1.03 1.00
query29 13.20 3.35 3.30
query30 0.24 0.06 0.07
query31 2.84 0.39 0.39
query32 3.25 0.47 0.48
query33 2.86 2.91 2.92
query34 17.11 4.40 4.45
query35 4.57 4.44 4.52
query36 0.66 0.46 0.46
query37 0.18 0.16 0.16
query38 0.15 0.14 0.14
query39 0.04 0.03 0.03
query40 0.17 0.15 0.15
query41 0.09 0.05 0.04
query42 0.06 0.05 0.05
query43 0.04 0.03 0.03
Total cold run time: 110.41 s
Total hot run time: 29.81 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li merged PR #35814:
URL: https://github.com/apache/doris/pull/35814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146989260
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partitino stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2145005525
<details>
<summary>TPC-H: <b>Total hot run time: 39953 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6acee37cf05109121f1d639ee2941aee12e80667, data reload: false
------ Round 1 ----------------------------------
q1 18064 4476 4337 4337
q2 2024 194 189 189
q3 10478 1222 1085 1085
q4 10140 819 735 735
q5 7473 2664 2674 2664
q6 219 137 144 137
q7 943 621 596 596
q8 9223 2098 2079 2079
q9 8715 6501 6446 6446
q10 8884 3721 3771 3721
q11 481 251 260 251
q12 418 232 228 228
q13 17762 3027 2999 2999
q14 263 233 215 215
q15 516 490 474 474
q16 506 377 386 377
q17 969 653 677 653
q18 8008 7466 7478 7466
q19 3800 1459 1547 1459
q20 658 326 303 303
q21 5013 3204 3203 3203
q22 399 340 336 336
Total cold run time: 114956 ms
Total hot run time: 39953 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4430 4257 4260 4257
q2 372 258 260 258
q3 2987 2812 2717 2717
q4 1905 1679 1611 1611
q5 5281 5296 5285 5285
q6 221 131 132 131
q7 2130 1749 1764 1749
q8 3203 3335 3321 3321
q9 8304 8312 8299 8299
q10 3868 3659 3676 3659
q11 590 499 487 487
q12 787 595 608 595
q13 17445 2991 3018 2991
q14 307 264 254 254
q15 522 490 478 478
q16 474 411 413 411
q17 1811 1505 1481 1481
q18 7641 7456 7447 7447
q19 1703 1574 1535 1535
q20 2009 1810 1786 1786
q21 4875 4940 4691 4691
q22 645 552 555 552
Total cold run time: 71510 ms
Total hot run time: 53995 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on code in PR #35814:
URL: https://github.com/apache/doris/pull/35814#discussion_r1625444901
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java:
##########
@@ -346,22 +348,29 @@ public void setJob(AnalysisJob job) {
}
/**
- * 1. Get stats of each partition
- * 2. insert partition in batch
- * 3. calculate column stats based on partition stats
+ * 1. Remove not exist partition stats
+ * 2. Get stats of each partition
+ * 3. insert partition in batch
+ * 4. calculate column stats based on partition stats
*/
protected void doPartitionTable() throws Exception {
+ deleteNotExistPartitionStats();
Map<String, String> params = buildSqlParams();
params.put("dataSizeFunction", getDataSizeFunction(col, false));
Set<String> partitionNames = tbl.getPartitionNames();
List<String> sqls = Lists.newArrayList();
int count = 0;
TableStatsMeta tableStatsStatus = Env.getServingEnv().getAnalysisManager().findTableStatsStatus(tbl.getId());
for (String part : partitionNames) {
+ params.put("partId", "-1");
Partition partition = tbl.getPartition(part);
+ if (partition != null) {
+ params.put("partId", String.valueOf(partition.getId()));
+ }
// Skip partitions that not changed after last analyze.
// External table getPartition always return null. So external table doesn't skip any partitions.
if (partition != null && tableStatsStatus != null && tableStatsStatus.partitionUpdateRows != null) {
+ params.put("partId", String.valueOf(partition.getId()));
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146927072
<details>
<summary>TPC-H: <b>Total hot run time: 41237 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 974db4f19f932ce72865ee9dee2f7fd8432e3746, data reload: false
------ Round 1 ----------------------------------
q1 17622 4314 4249 4249
q2 2043 191 191 191
q3 10469 1201 1217 1201
q4 10182 908 834 834
q5 7469 2771 2701 2701
q6 223 138 140 138
q7 985 605 618 605
q8 9215 2125 2096 2096
q9 9199 6675 6662 6662
q10 10118 3889 3852 3852
q11 472 252 235 235
q12 426 249 256 249
q13 17431 3224 3281 3224
q14 265 218 219 218
q15 511 463 480 463
q16 494 417 407 407
q17 964 641 653 641
q18 8383 7793 7763 7763
q19 7527 1462 1494 1462
q20 652 327 321 321
q21 5104 3378 3977 3378
q22 419 351 347 347
Total cold run time: 120173 ms
Total hot run time: 41237 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4574 4451 4528 4451
q2 377 279 278 278
q3 3143 3005 2972 2972
q4 1924 1582 1611 1582
q5 5440 5515 5489 5489
q6 230 126 125 125
q7 2167 1802 1784 1784
q8 3213 3392 3361 3361
q9 8643 8609 8680 8609
q10 4062 3890 3681 3681
q11 571 484 498 484
q12 772 610 625 610
q13 17123 3174 3118 3118
q14 320 263 270 263
q15 513 475 495 475
q16 481 436 427 427
q17 1817 1485 1499 1485
q18 8168 7290 7357 7290
q19 1751 1659 1586 1586
q20 3036 1784 1806 1784
q21 7059 4827 4617 4617
q22 744 557 542 542
Total cold run time: 76128 ms
Total hot run time: 55013 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2147129920
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partitino stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2144969383
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #35814:
URL: https://github.com/apache/doris/pull/35814#discussion_r1625325245
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java:
##########
@@ -346,22 +348,29 @@ public void setJob(AnalysisJob job) {
}
/**
- * 1. Get stats of each partition
- * 2. insert partition in batch
- * 3. calculate column stats based on partition stats
+ * 1. Remove not exist partition stats
+ * 2. Get stats of each partition
+ * 3. insert partition in batch
+ * 4. calculate column stats based on partition stats
*/
protected void doPartitionTable() throws Exception {
+ deleteNotExistPartitionStats();
Map<String, String> params = buildSqlParams();
params.put("dataSizeFunction", getDataSizeFunction(col, false));
Set<String> partitionNames = tbl.getPartitionNames();
List<String> sqls = Lists.newArrayList();
int count = 0;
TableStatsMeta tableStatsStatus = Env.getServingEnv().getAnalysisManager().findTableStatsStatus(tbl.getId());
for (String part : partitionNames) {
+ params.put("partId", "-1");
Partition partition = tbl.getPartition(part);
+ if (partition != null) {
+ params.put("partId", String.valueOf(partition.getId()));
+ }
Review Comment:
```java
params.put("partId", partition == null ? "-1" : String.valueOf(partition.getId()));
```
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java:
##########
@@ -346,22 +348,29 @@ public void setJob(AnalysisJob job) {
}
/**
- * 1. Get stats of each partition
- * 2. insert partition in batch
- * 3. calculate column stats based on partition stats
+ * 1. Remove not exist partition stats
+ * 2. Get stats of each partition
+ * 3. insert partition in batch
+ * 4. calculate column stats based on partition stats
*/
protected void doPartitionTable() throws Exception {
+ deleteNotExistPartitionStats();
Map<String, String> params = buildSqlParams();
params.put("dataSizeFunction", getDataSizeFunction(col, false));
Set<String> partitionNames = tbl.getPartitionNames();
List<String> sqls = Lists.newArrayList();
int count = 0;
TableStatsMeta tableStatsStatus = Env.getServingEnv().getAnalysisManager().findTableStatsStatus(tbl.getId());
for (String part : partitionNames) {
+ params.put("partId", "-1");
Partition partition = tbl.getPartition(part);
+ if (partition != null) {
+ params.put("partId", String.valueOf(partition.getId()));
+ }
// Skip partitions that not changed after last analyze.
// External table getPartition always return null. So external table doesn't skip any partitions.
if (partition != null && tableStatsStatus != null && tableStatsStatus.partitionUpdateRows != null) {
+ params.put("partId", String.valueOf(partition.getId()));
Review Comment:
why add again?
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/BaseAnalysisTask.java:
##########
@@ -400,6 +409,10 @@ protected void doPartitionTable() throws Exception {
runQuery(stringSubstitutor.replace(MERGE_PARTITION_TEMPLATE));
}
+ protected void deleteNotExistPartitionStats() throws DdlException {
Review Comment:
abstract is better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2147230402
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146949158
<details>
<summary>TPC-DS: <b>Total hot run time: 172255 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 974db4f19f932ce72865ee9dee2f7fd8432e3746, data reload: false
query1 937 383 369 369
query2 6479 2394 2416 2394
query3 6652 211 212 211
query4 19671 17445 17318 17318
query5 4150 448 451 448
query6 238 155 150 150
query7 4592 295 286 286
query8 322 278 289 278
query9 8728 2456 2439 2439
query10 430 293 284 284
query11 10477 10089 10105 10089
query12 135 88 87 87
query13 1639 353 366 353
query14 8477 6386 7619 6386
query15 228 192 188 188
query16 7284 275 261 261
query17 1296 510 518 510
query18 1912 291 267 267
query19 213 158 155 155
query20 92 87 84 84
query21 207 132 127 127
query22 4313 4112 4016 4016
query23 33702 33072 33331 33072
query24 11257 2782 2758 2758
query25 645 353 357 353
query26 1502 155 152 152
query27 3142 324 324 324
query28 7603 2093 2063 2063
query29 985 632 588 588
query30 285 150 151 150
query31 949 742 769 742
query32 94 54 56 54
query33 785 291 267 267
query34 998 478 480 478
query35 725 621 611 611
query36 1129 940 959 940
query37 153 66 69 66
query38 2856 2750 2690 2690
query39 884 802 800 800
query40 204 127 121 121
query41 54 52 56 52
query42 118 95 99 95
query43 606 542 554 542
query44 1291 723 741 723
query45 197 167 161 161
query46 1072 728 691 691
query47 1814 1762 1803 1762
query48 371 314 298 298
query49 1002 391 397 391
query50 779 380 386 380
query51 6926 6827 6797 6797
query52 101 93 87 87
query53 353 280 283 280
query54 846 446 436 436
query55 73 75 72 72
query56 267 254 251 251
query57 1173 1080 1034 1034
query58 253 257 236 236
query59 3336 3460 3307 3307
query60 286 270 271 270
query61 109 118 85 85
query62 651 454 447 447
query63 324 284 292 284
query64 9602 2200 1729 1729
query65 3208 3113 3130 3113
query66 1487 331 330 330
query67 15469 14984 14839 14839
query68 4504 526 520 520
query69 456 301 306 301
query70 1180 1038 1064 1038
query71 388 277 272 272
query72 7103 5614 5671 5614
query73 752 328 324 324
query74 5936 5477 5410 5410
query75 3436 2654 2657 2654
query76 2302 905 859 859
query77 451 296 292 292
query78 10417 9888 9782 9782
query79 2383 517 523 517
query80 813 460 450 450
query81 555 215 223 215
query82 1031 100 100 100
query83 237 168 177 168
query84 258 91 90 90
query85 1282 317 255 255
query86 435 301 320 301
query87 3225 3102 3082 3082
query88 3513 2430 2411 2411
query89 469 389 374 374
query90 1777 185 185 185
query91 122 94 96 94
query92 65 50 49 49
query93 1697 517 491 491
query94 1284 189 185 185
query95 403 314 314 314
query96 585 270 273 270
query97 3162 3018 2980 2980
query98 249 216 219 216
query99 1112 821 824 821
Total cold run time: 269720 ms
Total hot run time: 172255 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146832699
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146961510
<details>
<summary>ClickBench: <b>Total hot run time: 30.73 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 974db4f19f932ce72865ee9dee2f7fd8432e3746, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.68 0.10 0.10
query5 0.50 0.47 0.50
query6 1.12 0.73 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.05
query9 0.54 0.49 0.56
query10 0.55 0.56 0.55
query11 0.15 0.11 0.11
query12 0.15 0.11 0.12
query13 0.60 0.59 0.59
query14 0.79 0.79 0.77
query15 0.83 0.82 0.80
query16 0.36 0.34 0.35
query17 0.96 0.96 0.99
query18 0.21 0.26 0.26
query19 1.80 1.72 1.81
query20 0.02 0.02 0.02
query21 15.77 0.68 0.66
query22 4.52 7.24 1.90
query23 18.34 1.45 1.29
query24 1.90 0.25 0.21
query25 0.15 0.08 0.09
query26 0.28 0.17 0.17
query27 0.08 0.08 0.08
query28 13.32 1.10 1.08
query29 13.11 3.36 3.34
query30 0.26 0.05 0.05
query31 2.87 0.40 0.38
query32 3.29 0.48 0.46
query33 2.90 2.87 2.92
query34 17.25 4.42 4.41
query35 4.56 4.57 4.51
query36 0.64 0.45 0.46
query37 0.17 0.15 0.16
query38 0.15 0.15 0.14
query39 0.04 0.04 0.03
query40 0.16 0.15 0.15
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.03 0.04
Total cold run time: 110.63 s
Total hot run time: 30.73 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partitino stats while analyzing. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2145025772
<details>
<summary>TPC-DS: <b>Total hot run time: 170151 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6acee37cf05109121f1d639ee2941aee12e80667, data reload: false
query1 924 375 366 366
query2 6460 2440 2317 2317
query3 6653 204 205 204
query4 19488 17493 17314 17314
query5 4122 450 440 440
query6 256 161 155 155
query7 4586 302 284 284
query8 332 289 293 289
query9 8610 2412 2380 2380
query10 451 321 271 271
query11 10396 9997 9852 9852
query12 132 91 84 84
query13 1630 359 364 359
query14 10172 7586 7513 7513
query15 228 187 185 185
query16 7833 264 266 264
query17 1847 519 544 519
query18 1977 265 269 265
query19 204 157 162 157
query20 97 86 83 83
query21 211 138 128 128
query22 4343 4030 4037 4030
query23 33643 33270 33166 33166
query24 11908 2814 2794 2794
query25 662 415 361 361
query26 1650 156 155 155
query27 3020 311 323 311
query28 7656 2049 2042 2042
query29 1030 613 598 598
query30 276 150 154 150
query31 978 759 726 726
query32 95 51 56 51
query33 764 303 279 279
query34 970 468 473 468
query35 761 651 607 607
query36 1089 953 901 901
query37 145 67 71 67
query38 2880 2741 2740 2740
query39 861 808 802 802
query40 276 124 124 124
query41 53 51 52 51
query42 118 96 96 96
query43 590 555 553 553
query44 1336 727 741 727
query45 206 166 166 166
query46 1073 721 699 699
query47 1835 1758 1822 1758
query48 374 293 291 291
query49 1131 405 409 405
query50 786 391 391 391
query51 6803 6902 6787 6787
query52 105 90 87 87
query53 354 284 291 284
query54 1028 441 431 431
query55 73 74 76 74
query56 313 246 258 246
query57 1126 1050 1065 1050
query58 265 248 245 245
query59 3309 3142 3240 3142
query60 298 276 265 265
query61 92 90 84 84
query62 647 443 441 441
query63 329 293 290 290
query64 9826 2267 1710 1710
query65 3184 3102 3136 3102
query66 1366 340 329 329
query67 15469 15042 14797 14797
query68 4586 538 547 538
query69 466 293 307 293
query70 1143 1109 1103 1103
query71 402 289 274 274
query72 7120 5730 2707 2707
query73 756 321 326 321
query74 5937 5502 5460 5460
query75 3403 2713 2705 2705
query76 2572 941 845 845
query77 442 303 287 287
query78 10149 9874 9707 9707
query79 2437 513 508 508
query80 1003 448 461 448
query81 578 221 220 220
query82 685 103 108 103
query83 233 167 166 166
query84 242 81 84 81
query85 1979 273 323 273
query86 489 309 313 309
query87 3315 3172 3098 3098
query88 3748 2356 2344 2344
query89 476 384 389 384
query90 1810 188 183 183
query91 122 96 97 96
query92 63 50 47 47
query93 2338 510 502 502
query94 1198 195 185 185
query95 395 308 316 308
query96 594 265 263 263
query97 3152 3078 3044 3044
query98 235 227 215 215
query99 1190 832 843 832
Total cold run time: 274963 ms
Total hot run time: 170151 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partitino stats while analyzing. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2144972056
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](statistics)Remove not exist partition stats while analyzing. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35814:
URL: https://github.com/apache/doris/pull/35814#issuecomment-2146832756
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org