You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zy-kkk (via GitHub)" <gi...@apache.org> on 2025/12/09 13:43:53 UTC
[PR] [opt](hive) use binary search to prune hive partitions [doris]
zy-kkk opened a new pull request, #58877:
URL: https://github.com/apache/doris/pull/58877
Enable binary search partition pruning optimization for Hive external tables.
This PR implements the `SupportBinarySearchFilteringPartitions` interface for `HMSExternalTable`,
allowing Hive tables to benefit from the binary search partition pruning optimization introduced in #44586.
**Performance improvement (20000 partitions, 1000 queries):**
- Binary search enabled: **4.548 seconds**
- Binary search disabled: **12.849 seconds**
- **~2.8x faster**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635230855
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3633371743
# FE Regression Coverage Report
Increment line coverage ` 55.00% (11/20)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635943900
<details>
<summary>TPC-H: <b>Total hot run time: 35130 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
------ Round 1 ----------------------------------
q1 17613 4183 4012 4012
q2 2023 363 236 236
q3 10190 1317 717 717
q4 10215 832 313 313
q5 7532 2133 1863 1863
q6 182 171 138 138
q7 984 833 710 710
q8 9345 1452 1099 1099
q9 6923 5327 5264 5264
q10 6845 2391 1969 1969
q11 547 318 289 289
q12 666 728 599 599
q13 17797 3652 2989 2989
q14 292 285 276 276
q15 582 518 517 517
q16 929 907 864 864
q17 707 793 546 546
q18 8471 7441 7212 7212
q19 1208 987 615 615
q20 391 367 260 260
q21 4344 4101 3680 3680
q22 1067 976 962 962
Total cold run time: 108853 ms
Total hot run time: 35130 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4345 4020 4003 4003
q2 337 398 326 326
q3 2146 2655 2282 2282
q4 1332 1760 1313 1313
q5 4224 4738 4550 4550
q6 244 186 144 144
q7 2088 2046 1823 1823
q8 2672 2475 2463 2463
q9 7630 7424 7572 7424
q10 3070 3297 2816 2816
q11 583 549 557 549
q12 698 817 608 608
q13 3609 3878 3418 3418
q14 299 298 290 290
q15 546 509 531 509
q16 922 909 884 884
q17 1181 1365 1433 1365
q18 8044 7419 7606 7419
q19 879 855 830 830
q20 1893 1962 1829 1829
q21 4605 4201 4130 4130
q22 1096 1028 974 974
Total cold run time: 52443 ms
Total hot run time: 49949 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632457513
<details>
<summary>TPC-H: <b>Total hot run time: 36473 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb2c282b47308a9f2d5b9277650b93fd59d0180e, data reload: false
------ Round 1 ----------------------------------
q1 17609 5019 4922 4922
q2 2039 364 253 253
q3 10148 1335 761 761
q4 10212 842 321 321
q5 7500 2172 1994 1994
q6 193 170 134 134
q7 995 873 729 729
q8 9360 1443 1175 1175
q9 7026 5347 5426 5347
q10 6791 2389 1952 1952
q11 514 313 306 306
q12 677 732 595 595
q13 17793 3692 3049 3049
q14 303 307 274 274
q15 593 514 511 511
q16 932 930 882 882
q17 716 799 560 560
q18 7467 7082 6997 6997
q19 1225 953 612 612
q20 391 380 263 263
q21 4330 4036 3866 3866
q22 1079 983 970 970
Total cold run time: 107893 ms
Total hot run time: 36473 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4985 4903 4913 4903
q2 332 419 328 328
q3 2191 2739 2302 2302
q4 1338 1760 1346 1346
q5 4991 4672 4520 4520
q6 261 178 131 131
q7 2018 1974 1825 1825
q8 2681 2562 2712 2562
q9 7552 7560 7483 7483
q10 3099 3248 2797 2797
q11 596 535 506 506
q12 694 733 644 644
q13 3647 3902 3332 3332
q14 309 317 354 317
q15 555 530 513 513
q16 902 921 894 894
q17 1187 1488 1430 1430
q18 7851 7786 7575 7575
q19 892 842 904 842
q20 2023 2078 1968 1968
q21 4693 4433 4194 4194
q22 1050 1059 1013 1013
Total cold run time: 53847 ms
Total hot run time: 51425 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635491839
<details>
<summary>TPC-H: <b>Total hot run time: 36931 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 22e3e49a305a54a8ec210d4545be2253a54128ab, data reload: false
------ Round 1 ----------------------------------
q1 17659 4257 4064 4064
q2 2035 353 236 236
q3 10155 1370 750 750
q4 10222 845 316 316
q5 7546 2175 1963 1963
q6 193 173 141 141
q7 1008 876 723 723
q8 9372 1477 1199 1199
q9 7004 5362 5374 5362
q10 6872 2396 1960 1960
q11 544 319 312 312
q12 724 737 578 578
q13 17806 3708 3095 3095
q14 287 295 273 273
q15 601 527 519 519
q16 944 912 889 889
q17 713 854 568 568
q18 8197 8077 7762 7762
q19 1245 1027 668 668
q20 430 381 266 266
q21 4653 4365 4249 4249
q22 1147 1157 1038 1038
Total cold run time: 109357 ms
Total hot run time: 36931 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4268 4252 4268 4252
q2 344 417 397 397
q3 2401 2812 2524 2524
q4 1496 2179 1437 1437
q5 4498 4595 4458 4458
q6 201 168 129 129
q7 2028 1934 1924 1924
q8 2716 2551 2504 2504
q9 7441 7474 7396 7396
q10 2926 3082 2680 2680
q11 571 491 471 471
q12 610 697 559 559
q13 3245 3668 3053 3053
q14 281 283 268 268
q15 534 494 493 493
q16 870 896 850 850
q17 1140 1377 1378 1377
q18 7351 7213 7031 7031
q19 864 793 823 793
q20 1924 1960 1816 1816
q21 4672 4328 4223 4223
q22 1110 1017 1004 1004
Total cold run time: 51491 ms
Total hot run time: 49639 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on code in PR #58877:
URL: https://github.com/apache/doris/pull/58877#discussion_r2605099037
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java:
##########
@@ -1031,54 +976,90 @@ public static class HiveFileStatus {
@Data
public static class HivePartitionValues {
private BiMap<String, Long> partitionNameToIdMap;
- private Map<Long, List<UniqueId>> idToUniqueIdsMap;
private Map<Long, PartitionItem> idToPartitionItem;
private Map<Long, List<String>> partitionValuesMap;
- //multi pair
- private Map<UniqueId, Range<PartitionKey>> uidToPartitionRange;
- private Map<Range<PartitionKey>, UniqueId> rangeToId;
- //single pair
- private RangeMap<ColumnBound, UniqueId> singleColumnRangeMap;
- private Map<UniqueId, Range<ColumnBound>> singleUidToColumnRangeMap;
+
+ // Sorted partition ranges for binary search filtering.
+ // Built at construction time, shares the same lifecycle with HivePartitionValues.
+ private SortedPartitionRanges<String> sortedPartitionRanges;
public HivePartitionValues() {
}
public HivePartitionValues(Map<Long, PartitionItem> idToPartitionItem,
- Map<UniqueId, Range<PartitionKey>> uidToPartitionRange,
- Map<Range<PartitionKey>, UniqueId> rangeToId,
- RangeMap<ColumnBound, UniqueId> singleColumnRangeMap,
BiMap<String, Long> partitionNameToIdMap,
- Map<Long, List<UniqueId>> idToUniqueIdsMap,
- Map<UniqueId, Range<ColumnBound>> singleUidToColumnRangeMap,
Map<Long, List<String>> partitionValuesMap) {
this.idToPartitionItem = idToPartitionItem;
- this.uidToPartitionRange = uidToPartitionRange;
- this.rangeToId = rangeToId;
- this.singleColumnRangeMap = singleColumnRangeMap;
this.partitionNameToIdMap = partitionNameToIdMap;
- this.idToUniqueIdsMap = idToUniqueIdsMap;
- this.singleUidToColumnRangeMap = singleUidToColumnRangeMap;
this.partitionValuesMap = partitionValuesMap;
+ this.sortedPartitionRanges = buildSortedPartitionRanges();
}
+ /**
+ * Create a copy for incremental updates (add/drop partitions).
+ * The sortedPartitionRanges will be rebuilt after the caller modifies the partition data.
+ */
public HivePartitionValues copy() {
HivePartitionValues copy = new HivePartitionValues();
copy.setPartitionNameToIdMap(partitionNameToIdMap == null ? null : HashBiMap.create(partitionNameToIdMap));
- copy.setIdToUniqueIdsMap(idToUniqueIdsMap == null ? null : Maps.newHashMap(idToUniqueIdsMap));
copy.setIdToPartitionItem(idToPartitionItem == null ? null : Maps.newHashMap(idToPartitionItem));
copy.setPartitionValuesMap(partitionValuesMap == null ? null : Maps.newHashMap(partitionValuesMap));
- copy.setUidToPartitionRange(uidToPartitionRange == null ? null : Maps.newHashMap(uidToPartitionRange));
- copy.setRangeToId(rangeToId == null ? null : Maps.newHashMap(rangeToId));
- copy.setSingleUidToColumnRangeMap(
- singleUidToColumnRangeMap == null ? null : Maps.newHashMap(singleUidToColumnRangeMap));
- if (singleColumnRangeMap != null) {
- RangeMap<ColumnBound, UniqueId> copySingleColumnRangeMap = TreeRangeMap.create();
- copySingleColumnRangeMap.putAll(singleColumnRangeMap);
- copy.setSingleColumnRangeMap(copySingleColumnRangeMap);
- }
+ // sortedPartitionRanges is not copied here, caller should call rebuildSortedPartitionRanges()
+ // after modifying partition data
return copy;
}
+
+ /**
+ * Rebuild sorted partition ranges after incremental updates.
+ * Should be called after add/drop partitions.
+ */
+ public void rebuildSortedPartitionRanges() {
+ this.sortedPartitionRanges = buildSortedPartitionRanges();
+ }
+
+ /**
+ * Get sorted partition ranges for binary search filtering.
+ */
+ public Optional<SortedPartitionRanges<String>> getSortedPartitionRanges() {
+ return Optional.ofNullable(sortedPartitionRanges);
+ }
+
+ private SortedPartitionRanges<String> buildSortedPartitionRanges() {
+ if (partitionNameToIdMap == null || partitionNameToIdMap.isEmpty()) {
+ return null;
+ }
+
+ BiMap<Long, String> idToName = partitionNameToIdMap.inverse();
+ List<PartitionItemAndRange<String>> sortedRanges = Lists.newArrayListWithCapacity(
+ idToPartitionItem.size());
+ List<PartitionItemAndId<String>> defaultPartitions = Lists.newArrayList();
+
+ for (Map.Entry<Long, PartitionItem> entry : idToPartitionItem.entrySet()) {
+ PartitionItem partitionItem = entry.getValue();
+ String partitionName = idToName.get(entry.getKey());
+ if (!partitionItem.isDefaultPartition()) {
+ List<Range<MultiColumnBound>> ranges = PartitionItemToRange.toRanges(partitionItem);
+ for (Range<MultiColumnBound> range : ranges) {
+ sortedRanges.add(new PartitionItemAndRange<>(partitionName, partitionItem, range));
+ }
+ } else {
+ defaultPartitions.add(new PartitionItemAndId<>(partitionName, partitionItem));
+ }
+ }
+
+ // Sort by range bounds
+ sortedRanges.sort((o1, o2) -> {
+ Range<MultiColumnBound> span1 = o1.range;
+ Range<MultiColumnBound> span2 = o2.range;
+ int result = span1.lowerEndpoint().compareTo(span2.lowerEndpoint());
+ if (result != 0) {
+ return result;
+ }
+ return span1.upperEndpoint().compareTo(span2.upperEndpoint());
+ });
+
+ return new SortedPartitionRanges<>(sortedRanges, defaultPartitions);
+ }
Review Comment:
I think you should extract code from `NereidsSortedPartitionsCacheManager.loadCache` and reuse the same utility function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635748024
<details>
<summary>ClickBench: <b>Total hot run time: 27.41 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
query1 0.05 0.04 0.05
query2 0.10 0.05 0.05
query3 0.25 0.09 0.09
query4 1.61 0.11 0.11
query5 0.28 0.26 0.26
query6 1.16 0.64 0.62
query7 0.03 0.03 0.02
query8 0.05 0.04 0.04
query9 0.56 0.52 0.51
query10 0.57 0.56 0.55
query11 0.15 0.11 0.12
query12 0.14 0.12 0.11
query13 0.61 0.60 0.59
query14 0.97 0.99 0.98
query15 0.82 0.79 0.81
query16 0.41 0.40 0.40
query17 1.03 1.02 1.03
query18 0.23 0.21 0.22
query19 1.90 1.85 1.76
query20 0.01 0.01 0.01
query21 15.44 0.29 0.14
query22 4.76 0.04 0.04
query23 15.98 0.28 0.10
query24 1.40 0.64 0.43
query25 0.08 0.06 0.07
query26 0.14 0.14 0.14
query27 0.06 0.04 0.07
query28 4.40 1.22 1.02
query29 12.57 4.05 3.20
query30 0.27 0.13 0.12
query31 2.83 0.64 0.40
query32 3.23 0.55 0.46
query33 2.93 3.08 3.03
query34 16.97 5.17 4.56
query35 4.56 4.57 4.63
query36 0.68 0.50 0.49
query37 0.10 0.06 0.07
query38 0.07 0.05 0.04
query39 0.05 0.03 0.04
query40 0.17 0.14 0.13
query41 0.08 0.03 0.04
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 97.78 s
Total hot run time: 27.41 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632690380
<details>
<summary>TPC-H: <b>Total hot run time: 36315 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a46590096eaff7a5cf99d33963a32066f635462, data reload: false
------ Round 1 ----------------------------------
q1 17616 5051 4934 4934
q2 2050 366 245 245
q3 10148 1295 739 739
q4 10239 870 333 333
q5 7537 2182 1900 1900
q6 195 171 138 138
q7 1006 873 723 723
q8 9361 1434 1142 1142
q9 6995 5312 5386 5312
q10 6787 2393 1999 1999
q11 553 316 304 304
q12 661 740 588 588
q13 17767 3704 3075 3075
q14 292 297 278 278
q15 588 515 521 515
q16 947 910 876 876
q17 695 817 533 533
q18 7922 7163 7137 7137
q19 956 964 612 612
q20 413 372 252 252
q21 4246 3918 3714 3714
q22 1058 978 966 966
Total cold run time: 108032 ms
Total hot run time: 36315 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4947 4953 4889 4889
q2 334 388 319 319
q3 2129 2707 2284 2284
q4 1308 1746 1273 1273
q5 4965 4673 4537 4537
q6 221 191 130 130
q7 2097 1962 1824 1824
q8 2690 2702 2520 2520
q9 7974 7991 7855 7855
q10 3213 3654 2940 2940
q11 658 519 517 517
q12 788 774 572 572
q13 3908 3979 3453 3453
q14 292 405 297 297
q15 586 581 520 520
q16 913 957 890 890
q17 1227 1499 1424 1424
q18 7859 7685 7843 7685
q19 919 871 866 866
q20 2018 2080 1850 1850
q21 4543 4307 4161 4161
q22 1069 1085 969 969
Total cold run time: 54658 ms
Total hot run time: 51775 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632749115
<details>
<summary>TPC-DS: <b>Total hot run time: 182033 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a46590096eaff7a5cf99d33963a32066f635462, data reload: false
query5 5898 677 503 503
query6 436 248 230 230
query7 5254 475 292 292
query8 353 261 248 248
query9 8773 2636 2662 2636
query10 598 366 337 337
query11 15689 14804 15133 14804
query12 197 120 115 115
query13 1242 503 371 371
query14 7749 3328 3114 3114
query14_1 2936 3014 2963 2963
query15 273 202 184 184
query16 907 526 479 479
query17 1154 698 608 608
query18 2733 444 341 341
query19 314 228 210 210
query20 127 116 113 113
query21 219 146 114 114
query22 3820 3866 3943 3866
query23 16803 16088 15928 15928
query23_1 16065 16033 15930 15930
query24 6698 1673 1246 1246
query24_1 1249 1257 1271 1257
query25 552 499 418 418
query26 1224 271 170 170
query27 2702 481 312 312
query28 4516 2172 2142 2142
query29 797 564 452 452
query30 322 249 226 226
query31 814 719 643 643
query32 82 68 73 68
query33 521 334 313 313
query34 886 911 547 547
query35 781 807 742 742
query36 873 914 846 846
query37 141 105 87 87
query38 3954 3844 3771 3771
query39 753 741 734 734
query39_1 706 704 703 703
query40 227 142 128 128
query41 68 62 65 62
query42 110 105 105 105
query43 427 447 392 392
query44 1393 758 771 758
query45 194 193 186 186
query46 898 1002 636 636
query47 1634 1717 1600 1600
query48 312 350 254 254
query49 636 438 378 378
query50 685 312 231 231
query51 3852 3921 3812 3812
query52 109 113 104 104
query53 325 363 305 305
query54 307 290 277 277
query55 80 82 78 78
query56 340 345 339 339
query57 1159 1143 1087 1087
query58 277 274 261 261
query59 2380 2446 2348 2348
query60 361 335 321 321
query61 198 188 191 188
query62 696 692 636 636
query63 340 301 312 301
query64 5075 1410 1138 1138
query65 4058 3982 3978 3978
query66 1395 476 334 334
query67 15201 14838 14688 14688
query68 6470 1027 747 747
query69 504 356 330 330
query70 1061 1034 985 985
query71 410 325 300 300
query72 6096 5010 4831 4831
query73 692 617 306 306
query74 8905 8873 8661 8661
query75 3566 3545 3184 3184
query76 3791 1177 791 791
query77 513 397 304 304
query78 9496 9845 8832 8832
query79 1602 943 615 615
query80 1365 661 562 562
query81 564 269 238 238
query82 420 127 105 105
query83 269 256 243 243
query84 265 114 98 98
query85 911 509 456 456
query86 376 321 280 280
query87 4002 4097 3978 3978
query88 3218 2318 2261 2261
query89 476 434 386 386
query90 2014 159 157 157
query91 178 167 141 141
query92 73 67 62 62
query93 1279 935 576 576
query94 536 305 277 277
query95 552 326 366 326
query96 606 455 211 211
query97 2623 2632 2605 2605
query98 213 198 193 193
query99 1291 1312 1221 1221
Total cold run time: 266033 ms
Total hot run time: 182033 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635634670
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3636000410
<details>
<summary>ClickBench: <b>Total hot run time: 27.64 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
query1 0.06 0.05 0.06
query2 0.10 0.05 0.04
query3 0.25 0.08 0.09
query4 1.60 0.11 0.11
query5 0.27 0.25 0.26
query6 1.17 0.64 0.62
query7 0.03 0.03 0.02
query8 0.05 0.04 0.05
query9 0.58 0.51 0.51
query10 0.54 0.55 0.55
query11 0.16 0.11 0.11
query12 0.15 0.11 0.12
query13 0.62 0.60 0.60
query14 0.98 0.99 0.98
query15 0.83 0.80 0.81
query16 0.40 0.40 0.40
query17 1.07 1.04 1.05
query18 0.24 0.22 0.22
query19 1.90 1.82 1.74
query20 0.01 0.02 0.02
query21 15.47 0.26 0.14
query22 4.76 0.05 0.05
query23 15.94 0.28 0.10
query24 0.95 0.57 1.09
query25 0.11 0.06 0.07
query26 0.14 0.13 0.13
query27 0.06 0.05 0.05
query28 5.43 1.23 1.02
query29 12.58 4.15 3.31
query30 0.31 0.14 0.12
query31 2.83 0.64 0.40
query32 3.23 0.55 0.46
query33 3.05 3.05 3.08
query34 16.83 5.18 4.49
query35 4.60 4.59 4.52
query36 0.64 0.51 0.49
query37 0.11 0.07 0.07
query38 0.08 0.04 0.04
query39 0.05 0.03 0.04
query40 0.18 0.15 0.15
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.05 0.04 0.04
Total cold run time: 98.54 s
Total hot run time: 27.64 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635702053
<details>
<summary>TPC-H: <b>Total hot run time: 36566 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
------ Round 1 ----------------------------------
q1 17671 4245 4088 4088
q2 2031 367 239 239
q3 10184 1309 745 745
q4 10227 869 311 311
q5 7593 2104 1924 1924
q6 184 166 136 136
q7 1029 884 717 717
q8 9353 1475 1110 1110
q9 6991 5326 5339 5326
q10 6791 2388 2010 2010
q11 525 324 296 296
q12 671 768 584 584
q13 17796 3638 3034 3034
q14 291 301 288 288
q15 576 521 515 515
q16 964 927 889 889
q17 718 807 534 534
q18 7385 7749 7823 7749
q19 1132 1016 665 665
q20 438 375 262 262
q21 4687 4276 4123 4123
q22 1097 1067 1021 1021
Total cold run time: 108334 ms
Total hot run time: 36566 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4379 4233 4298 4233
q2 351 390 340 340
q3 2377 2943 2558 2558
q4 1408 1896 1380 1380
q5 4591 4721 4428 4428
q6 205 171 128 128
q7 2021 1943 1770 1770
q8 2676 2556 2495 2495
q9 7591 7542 7486 7486
q10 3038 3243 2792 2792
q11 574 494 471 471
q12 669 685 570 570
q13 3297 3691 3041 3041
q14 265 294 266 266
q15 533 489 485 485
q16 872 870 835 835
q17 1161 1465 1338 1338
q18 7262 7084 7031 7031
q19 851 830 840 830
q20 1916 1960 1812 1812
q21 4626 4327 4250 4250
q22 1045 1039 984 984
Total cold run time: 51708 ms
Total hot run time: 49523 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635733660
<details>
<summary>TPC-DS: <b>Total hot run time: 180827 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
query5 4841 623 467 467
query6 340 231 215 215
query7 4211 463 272 272
query8 293 253 226 226
query9 8760 2579 2557 2557
query10 534 410 347 347
query11 15507 14793 14556 14556
query12 176 113 111 111
query13 1265 502 391 391
query14 6061 3212 2943 2943
query14_1 2841 2889 2874 2874
query15 216 194 189 189
query16 904 470 450 450
query17 1085 694 591 591
query18 2478 435 331 331
query19 242 232 204 204
query20 129 111 111 111
query21 216 135 117 117
query22 4031 4127 3903 3903
query23 16662 16381 15949 15949
query23_1 16064 16103 15999 15999
query24 7322 1659 1210 1210
query24_1 1222 1234 1212 1212
query25 546 463 437 437
query26 1261 306 162 162
query27 2719 473 304 304
query28 4472 2199 2177 2177
query29 791 543 428 428
query30 312 244 210 210
query31 808 710 654 654
query32 80 69 68 68
query33 533 330 275 275
query34 899 954 544 544
query35 800 805 724 724
query36 861 894 820 820
query37 127 92 79 79
query38 3935 3803 3748 3748
query39 757 730 726 726
query39_1 704 712 695 695
query40 223 150 129 129
query41 68 64 65 64
query42 106 103 100 100
query43 415 427 401 401
query44 1323 760 757 757
query45 192 186 183 183
query46 883 976 605 605
query47 1715 1715 1638 1638
query48 335 330 258 258
query49 622 434 377 377
query50 670 286 213 213
query51 3801 3816 3823 3816
query52 109 115 112 112
query53 322 351 295 295
query54 314 271 272 271
query55 84 74 72 72
query56 306 333 292 292
query57 1139 1151 1072 1072
query58 264 252 269 252
query59 2359 2524 2356 2356
query60 326 314 291 291
query61 165 161 154 154
query62 710 669 618 618
query63 329 293 297 293
query64 5036 1314 992 992
query65 3999 3956 3960 3956
query66 1474 439 306 306
query67 15233 14801 14706 14706
query68 2789 1059 764 764
query69 458 365 333 333
query70 1026 1011 981 981
query71 336 311 286 286
query72 5868 5109 5017 5017
query73 483 549 309 309
query74 8627 8980 8695 8695
query75 3483 3554 3206 3206
query76 2903 1140 731 731
query77 367 444 292 292
query78 9507 9588 8834 8834
query79 2399 891 627 627
query80 1692 654 562 562
query81 564 264 234 234
query82 1017 131 102 102
query83 348 249 241 241
query84 253 124 102 102
query85 952 501 464 464
query86 396 321 276 276
query87 4106 4036 4000 4000
query88 3301 2302 2291 2291
query89 482 429 386 386
query90 1991 156 151 151
query91 170 164 147 147
query92 70 69 65 65
query93 1134 916 581 581
query94 530 310 263 263
query95 573 380 313 313
query96 597 471 205 205
query97 2592 2627 2579 2579
query98 201 200 190 190
query99 1262 1314 1201 1201
Total cold run time: 257668 ms
Total hot run time: 180827 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635983473
<details>
<summary>TPC-DS: <b>Total hot run time: 181501 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
query5 5082 620 467 467
query6 331 243 212 212
query7 4234 466 274 274
query8 299 246 243 243
query9 8759 2532 2577 2532
query10 547 370 338 338
query11 15139 14863 14524 14524
query12 173 120 111 111
query13 1280 496 395 395
query14 5852 3218 3007 3007
query14_1 2890 2983 2871 2871
query15 222 202 180 180
query16 885 480 454 454
query17 1126 718 614 614
query18 2545 447 359 359
query19 238 246 214 214
query20 123 120 114 114
query21 220 147 120 120
query22 3858 3913 3913 3913
query23 16436 16199 15903 15903
query23_1 16101 16161 16065 16065
query24 7399 1662 1254 1254
query24_1 1249 1204 1259 1204
query25 588 506 471 471
query26 1255 290 170 170
query27 2729 477 315 315
query28 4426 2190 2182 2182
query29 852 582 483 483
query30 326 249 214 214
query31 837 707 629 629
query32 79 74 68 68
query33 547 343 302 302
query34 884 902 545 545
query35 809 828 744 744
query36 876 894 835 835
query37 130 142 84 84
query38 3883 3870 3860 3860
query39 751 731 709 709
query39_1 705 706 701 701
query40 222 136 121 121
query41 69 62 63 62
query42 104 105 108 105
query43 431 436 408 408
query44 1320 767 750 750
query45 190 192 180 180
query46 872 982 616 616
query47 1635 1706 1629 1629
query48 315 323 247 247
query49 631 434 346 346
query50 661 290 222 222
query51 3779 3855 3783 3783
query52 105 108 100 100
query53 320 346 289 289
query54 311 258 249 249
query55 79 73 72 72
query56 298 319 304 304
query57 1138 1165 1069 1069
query58 273 253 248 248
query59 2342 2386 2449 2386
query60 311 316 288 288
query61 159 155 158 155
query62 680 656 619 619
query63 332 301 299 299
query64 5005 1350 998 998
query65 4047 3938 3981 3938
query66 1427 451 334 334
query67 15070 14985 14806 14806
query68 8104 1013 783 783
query69 497 340 309 309
query70 1056 1014 1028 1014
query71 377 305 283 283
query72 6077 4909 5021 4909
query73 660 583 309 309
query74 8763 8808 8653 8653
query75 3549 3512 3167 3167
query76 3934 1141 728 728
query77 545 396 294 294
query78 9378 9640 8997 8997
query79 1737 889 628 628
query80 736 649 565 565
query81 544 268 236 236
query82 203 133 104 104
query83 263 255 247 247
query84 267 120 101 101
query85 895 507 463 463
query86 387 309 273 273
query87 4063 4114 3951 3951
query88 3227 2327 2313 2313
query89 476 431 414 414
query90 2237 161 157 157
query91 175 168 169 168
query92 86 70 68 68
query93 2115 915 575 575
query94 490 306 275 275
query95 567 332 303 303
query96 587 484 211 211
query97 2616 2624 2568 2568
query98 217 201 190 190
query99 1258 1284 1196 1196
Total cold run time: 262615 ms
Total hot run time: 181501 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635817412
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632532719
<details>
<summary>ClickBench: <b>Total hot run time: 27.37 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eb2c282b47308a9f2d5b9277650b93fd59d0180e, data reload: false
query1 0.06 0.05 0.05
query2 0.09 0.05 0.05
query3 0.25 0.09 0.09
query4 1.61 0.12 0.11
query5 0.28 0.26 0.27
query6 1.15 0.65 0.62
query7 0.03 0.02 0.02
query8 0.06 0.04 0.04
query9 0.58 0.50 0.51
query10 0.56 0.56 0.55
query11 0.15 0.11 0.12
query12 0.15 0.12 0.12
query13 0.62 0.60 0.60
query14 0.98 0.98 0.97
query15 0.81 0.80 0.79
query16 0.38 0.39 0.40
query17 1.01 1.02 0.98
query18 0.24 0.22 0.22
query19 1.95 1.88 1.86
query20 0.02 0.01 0.01
query21 15.43 0.26 0.14
query22 4.84 0.06 0.05
query23 16.21 0.28 0.10
query24 1.72 0.25 0.33
query25 0.07 0.11 0.06
query26 0.14 0.14 0.13
query27 0.05 0.06 0.05
query28 4.63 1.23 1.01
query29 12.59 4.02 3.25
query30 0.29 0.16 0.12
query31 2.83 0.61 0.40
query32 3.24 0.55 0.46
query33 3.10 3.08 3.15
query34 16.67 5.24 4.52
query35 4.56 4.58 4.60
query36 0.67 0.50 0.49
query37 0.11 0.07 0.07
query38 0.07 0.05 0.04
query39 0.04 0.03 0.03
query40 0.17 0.16 0.13
query41 0.09 0.03 0.02
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 98.58 s
Total hot run time: 27.37 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632533929
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632777024
<details>
<summary>ClickBench: <b>Total hot run time: 27.63 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a46590096eaff7a5cf99d33963a32066f635462, data reload: false
query1 0.05 0.04 0.05
query2 0.10 0.04 0.04
query3 0.26 0.09 0.08
query4 1.60 0.12 0.11
query5 0.27 0.27 0.26
query6 1.17 0.64 0.63
query7 0.03 0.02 0.02
query8 0.05 0.04 0.04
query9 0.58 0.52 0.52
query10 0.56 0.55 0.55
query11 0.15 0.11 0.11
query12 0.15 0.11 0.11
query13 0.63 0.61 0.59
query14 0.99 0.98 0.99
query15 0.82 0.79 0.81
query16 0.41 0.41 0.41
query17 1.02 1.06 1.01
query18 0.22 0.21 0.21
query19 1.94 1.80 1.84
query20 0.02 0.01 0.01
query21 15.47 0.29 0.14
query22 4.88 0.05 0.05
query23 16.07 0.28 0.10
query24 1.48 1.10 0.62
query25 0.09 0.08 0.05
query26 0.15 0.14 0.14
query27 0.05 0.05 0.06
query28 5.04 1.22 1.03
query29 12.61 4.05 3.19
query30 0.29 0.14 0.13
query31 2.81 0.62 0.38
query32 3.23 0.56 0.46
query33 3.05 3.03 3.03
query34 16.84 5.22 4.55
query35 4.55 4.64 4.56
query36 0.66 0.49 0.49
query37 0.11 0.07 0.06
query38 0.07 0.05 0.04
query39 0.04 0.03 0.04
query40 0.17 0.14 0.13
query41 0.08 0.03 0.03
query42 0.05 0.03 0.03
query43 0.05 0.03 0.03
Total cold run time: 98.86 s
Total hot run time: 27.63 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632509001
<details>
<summary>TPC-DS: <b>Total hot run time: 182268 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eb2c282b47308a9f2d5b9277650b93fd59d0180e, data reload: false
query5 5065 609 469 469
query6 328 231 221 221
query7 4214 488 281 281
query8 324 265 249 249
query9 8810 2656 2679 2656
query10 565 392 337 337
query11 15506 14854 14835 14835
query12 179 122 127 122
query13 1257 529 404 404
query14 6327 3280 3054 3054
query14_1 2945 2954 2932 2932
query15 219 196 186 186
query16 787 473 476 473
query17 1142 711 618 618
query18 2722 453 362 362
query19 247 236 212 212
query20 119 116 113 113
query21 222 142 119 119
query22 3914 3996 3762 3762
query23 16552 16208 16048 16048
query23_1 16159 16046 16046 16046
query24 7361 1674 1254 1254
query24_1 1248 1235 1246 1235
query25 576 502 458 458
query26 1249 287 167 167
query27 2736 460 310 310
query28 4431 2164 2176 2164
query29 850 580 469 469
query30 321 245 219 219
query31 827 728 646 646
query32 82 75 73 73
query33 555 364 311 311
query34 914 908 539 539
query35 793 862 737 737
query36 891 916 833 833
query37 130 91 75 75
query38 3853 3852 3801 3801
query39 759 731 711 711
query39_1 701 697 705 697
query40 221 143 123 123
query41 67 62 63 62
query42 106 107 105 105
query43 433 424 390 390
query44 1317 758 758 758
query45 192 194 185 185
query46 872 989 615 615
query47 1682 1742 1638 1638
query48 328 325 248 248
query49 649 444 353 353
query50 658 298 230 230
query51 3911 3874 3876 3874
query52 108 112 98 98
query53 329 350 297 297
query54 286 261 249 249
query55 78 78 74 74
query56 306 308 301 301
query57 1162 1156 1086 1086
query58 278 257 260 257
query59 2400 2475 2346 2346
query60 321 311 292 292
query61 162 155 156 155
query62 695 679 629 629
query63 325 298 297 297
query64 4997 1307 1017 1017
query65 4075 3977 3998 3977
query66 1385 433 321 321
query67 15293 14904 14766 14766
query68 8415 1026 728 728
query69 505 343 320 320
query70 1068 967 996 967
query71 381 331 289 289
query72 6121 5187 5275 5187
query73 701 645 311 311
query74 8924 8846 8827 8827
query75 3600 3538 3165 3165
query76 4047 1153 760 760
query77 592 409 293 293
query78 9417 9645 8845 8845
query79 1627 875 628 628
query80 723 640 566 566
query81 497 273 242 242
query82 200 131 106 106
query83 263 253 234 234
query84 256 115 96 96
query85 899 521 463 463
query86 388 305 288 288
query87 4012 4025 4002 4002
query88 3890 2282 2277 2277
query89 474 423 404 404
query90 2198 160 152 152
query91 175 167 143 143
query92 83 69 61 61
query93 1992 916 563 563
query94 472 313 292 292
query95 583 386 303 303
query96 595 467 211 211
query97 2603 2624 2566 2566
query98 221 193 193 193
query99 1316 1316 1244 1244
Total cold run time: 265575 ms
Total hot run time: 182268 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632368641
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635525653
<details>
<summary>ClickBench: <b>Total hot run time: 27.33 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 22e3e49a305a54a8ec210d4545be2253a54128ab, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.04 0.05
query3 0.26 0.09 0.09
query4 1.60 0.11 0.10
query5 0.30 0.29 0.26
query6 1.17 0.64 0.64
query7 0.04 0.03 0.02
query8 0.06 0.05 0.04
query9 0.57 0.50 0.51
query10 0.56 0.55 0.54
query11 0.16 0.11 0.11
query12 0.15 0.12 0.12
query13 0.63 0.60 0.61
query14 0.99 0.98 0.98
query15 0.81 0.80 0.81
query16 0.39 0.40 0.39
query17 1.05 1.05 1.03
query18 0.23 0.23 0.22
query19 1.96 1.85 1.85
query20 0.02 0.02 0.01
query21 15.43 0.27 0.14
query22 5.03 0.05 0.05
query23 16.08 0.29 0.10
query24 0.95 0.42 0.18
query25 0.06 0.06 0.06
query26 0.14 0.14 0.13
query27 0.07 0.06 0.05
query28 3.61 1.22 1.03
query29 12.62 4.07 3.26
query30 0.28 0.13 0.11
query31 2.82 0.63 0.40
query32 3.22 0.54 0.45
query33 2.98 3.08 2.99
query34 16.90 5.21 4.60
query35 4.57 4.54 4.54
query36 0.66 0.49 0.51
query37 0.10 0.06 0.07
query38 0.08 0.04 0.04
query39 0.04 0.03 0.03
query40 0.16 0.14 0.14
query41 0.09 0.03 0.04
query42 0.04 0.03 0.03
query43 0.05 0.04 0.03
Total cold run time: 97.09 s
Total hot run time: 27.33 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635634752
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635644671
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3636882899
# FE Regression Coverage Report
Increment line coverage ` 76.36% (42/55)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/58877_8a5da2d7af4be796b451d720b4ec119da10d7b46_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/58877_8a5da2d7af4be796b451d720b4ec119da10d7b46_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #58877:
URL: https://github.com/apache/doris/pull/58877
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632988002
# FE UT Coverage Report
Increment line coverage ` 10.00% (2/20)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635515339
<details>
<summary>TPC-DS: <b>Total hot run time: 181153 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 22e3e49a305a54a8ec210d4545be2253a54128ab, data reload: false
query5 4841 602 464 464
query6 325 231 217 217
query7 4221 464 279 279
query8 298 253 234 234
query9 8767 2560 2592 2560
query10 510 381 324 324
query11 15449 14859 14916 14859
query12 168 114 114 114
query13 1258 505 398 398
query14 6030 3306 2970 2970
query14_1 2876 2850 2872 2850
query15 211 198 197 197
query16 887 495 422 422
query17 1146 674 578 578
query18 2462 439 338 338
query19 229 239 204 204
query20 122 113 110 110
query21 217 136 113 113
query22 3950 4008 3827 3827
query23 16659 16175 15901 15901
query23_1 16007 16132 16098 16098
query24 7340 1625 1242 1242
query24_1 1228 1226 1238 1226
query25 556 506 445 445
query26 1249 269 162 162
query27 2776 484 310 310
query28 4486 2169 2140 2140
query29 793 549 438 438
query30 315 247 222 222
query31 811 708 634 634
query32 78 71 65 65
query33 523 341 285 285
query34 898 889 544 544
query35 789 797 716 716
query36 872 905 833 833
query37 132 95 74 74
query38 3971 3865 3806 3806
query39 752 742 747 742
query39_1 695 686 709 686
query40 225 142 120 120
query41 66 62 62 62
query42 109 106 105 105
query43 433 436 405 405
query44 1355 760 755 755
query45 193 189 184 184
query46 871 986 614 614
query47 1652 1698 1624 1624
query48 320 331 245 245
query49 635 435 350 350
query50 655 303 226 226
query51 3975 3849 3827 3827
query52 112 107 102 102
query53 317 341 300 300
query54 289 274 261 261
query55 82 74 71 71
query56 291 294 293 293
query57 1165 1142 1068 1068
query58 268 249 261 249
query59 2311 2410 2321 2321
query60 344 324 315 315
query61 211 184 179 179
query62 726 692 646 646
query63 336 303 305 303
query64 5170 1329 1000 1000
query65 4049 3926 3993 3926
query66 1452 437 322 322
query67 15275 14983 14648 14648
query68 4656 1037 751 751
query69 494 353 315 315
query70 1068 965 981 965
query71 371 325 285 285
query72 6185 4972 5066 4972
query73 692 619 308 308
query74 8897 8779 8570 8570
query75 3577 3527 3227 3227
query76 3800 1135 753 753
query77 525 402 310 310
query78 9492 9555 8811 8811
query79 1043 887 643 643
query80 1153 691 591 591
query81 553 276 237 237
query82 411 134 108 108
query83 341 259 240 240
query84 256 130 105 105
query85 1047 563 464 464
query86 381 297 284 284
query87 4066 4178 3960 3960
query88 3195 2308 2278 2278
query89 469 423 404 404
query90 1940 166 156 156
query91 181 168 145 145
query92 73 68 66 66
query93 1022 944 577 577
query94 528 294 282 282
query95 592 340 394 340
query96 594 467 223 223
query97 2608 2634 2588 2588
query98 210 192 202 192
query99 1274 1294 1186 1186
Total cold run time: 259789 ms
Total hot run time: 181153 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632344311
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org