You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zy-kkk (via GitHub)" <gi...@apache.org> on 2025/12/09 13:43:53 UTC

[PR] [opt](hive) use binary search to prune hive partitions [doris]

zy-kkk opened a new pull request, #58877:
URL: https://github.com/apache/doris/pull/58877

   Enable binary search partition pruning optimization for Hive external tables.
   
     This PR implements the `SupportBinarySearchFilteringPartitions` interface for `HMSExternalTable`,
     allowing Hive tables to benefit from the binary search partition pruning optimization introduced in #44586.
   
     **Performance improvement (20000 partitions, 1000 queries):**
     - Binary search enabled: **4.548 seconds**
     - Binary search disabled: **12.849 seconds**
     - **~2.8x faster**
     


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635230855

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3633371743

   # FE Regression Coverage Report
   Increment line coverage `  55.00% (11/20)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635943900

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 35130 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17613	4183	4012	4012
   q2	2023	363	236	236
   q3	10190	1317	717	717
   q4	10215	832	313	313
   q5	7532	2133	1863	1863
   q6	182	171	138	138
   q7	984	833	710	710
   q8	9345	1452	1099	1099
   q9	6923	5327	5264	5264
   q10	6845	2391	1969	1969
   q11	547	318	289	289
   q12	666	728	599	599
   q13	17797	3652	2989	2989
   q14	292	285	276	276
   q15	582	518	517	517
   q16	929	907	864	864
   q17	707	793	546	546
   q18	8471	7441	7212	7212
   q19	1208	987	615	615
   q20	391	367	260	260
   q21	4344	4101	3680	3680
   q22	1067	976	962	962
   Total cold run time: 108853 ms
   Total hot run time: 35130 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4345	4020	4003	4003
   q2	337	398	326	326
   q3	2146	2655	2282	2282
   q4	1332	1760	1313	1313
   q5	4224	4738	4550	4550
   q6	244	186	144	144
   q7	2088	2046	1823	1823
   q8	2672	2475	2463	2463
   q9	7630	7424	7572	7424
   q10	3070	3297	2816	2816
   q11	583	549	557	549
   q12	698	817	608	608
   q13	3609	3878	3418	3418
   q14	299	298	290	290
   q15	546	509	531	509
   q16	922	909	884	884
   q17	1181	1365	1433	1365
   q18	8044	7419	7606	7419
   q19	879	855	830	830
   q20	1893	1962	1829	1829
   q21	4605	4201	4130	4130
   q22	1096	1028	974	974
   Total cold run time: 52443 ms
   Total hot run time: 49949 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632457513

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36473 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit eb2c282b47308a9f2d5b9277650b93fd59d0180e, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17609	5019	4922	4922
   q2	2039	364	253	253
   q3	10148	1335	761	761
   q4	10212	842	321	321
   q5	7500	2172	1994	1994
   q6	193	170	134	134
   q7	995	873	729	729
   q8	9360	1443	1175	1175
   q9	7026	5347	5426	5347
   q10	6791	2389	1952	1952
   q11	514	313	306	306
   q12	677	732	595	595
   q13	17793	3692	3049	3049
   q14	303	307	274	274
   q15	593	514	511	511
   q16	932	930	882	882
   q17	716	799	560	560
   q18	7467	7082	6997	6997
   q19	1225	953	612	612
   q20	391	380	263	263
   q21	4330	4036	3866	3866
   q22	1079	983	970	970
   Total cold run time: 107893 ms
   Total hot run time: 36473 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4985	4903	4913	4903
   q2	332	419	328	328
   q3	2191	2739	2302	2302
   q4	1338	1760	1346	1346
   q5	4991	4672	4520	4520
   q6	261	178	131	131
   q7	2018	1974	1825	1825
   q8	2681	2562	2712	2562
   q9	7552	7560	7483	7483
   q10	3099	3248	2797	2797
   q11	596	535	506	506
   q12	694	733	644	644
   q13	3647	3902	3332	3332
   q14	309	317	354	317
   q15	555	530	513	513
   q16	902	921	894	894
   q17	1187	1488	1430	1430
   q18	7851	7786	7575	7575
   q19	892	842	904	842
   q20	2023	2078	1968	1968
   q21	4693	4433	4194	4194
   q22	1050	1059	1013	1013
   Total cold run time: 53847 ms
   Total hot run time: 51425 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635491839

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36931 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 22e3e49a305a54a8ec210d4545be2253a54128ab, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17659	4257	4064	4064
   q2	2035	353	236	236
   q3	10155	1370	750	750
   q4	10222	845	316	316
   q5	7546	2175	1963	1963
   q6	193	173	141	141
   q7	1008	876	723	723
   q8	9372	1477	1199	1199
   q9	7004	5362	5374	5362
   q10	6872	2396	1960	1960
   q11	544	319	312	312
   q12	724	737	578	578
   q13	17806	3708	3095	3095
   q14	287	295	273	273
   q15	601	527	519	519
   q16	944	912	889	889
   q17	713	854	568	568
   q18	8197	8077	7762	7762
   q19	1245	1027	668	668
   q20	430	381	266	266
   q21	4653	4365	4249	4249
   q22	1147	1157	1038	1038
   Total cold run time: 109357 ms
   Total hot run time: 36931 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4268	4252	4268	4252
   q2	344	417	397	397
   q3	2401	2812	2524	2524
   q4	1496	2179	1437	1437
   q5	4498	4595	4458	4458
   q6	201	168	129	129
   q7	2028	1934	1924	1924
   q8	2716	2551	2504	2504
   q9	7441	7474	7396	7396
   q10	2926	3082	2680	2680
   q11	571	491	471	471
   q12	610	697	559	559
   q13	3245	3668	3053	3053
   q14	281	283	268	268
   q15	534	494	493	493
   q16	870	896	850	850
   q17	1140	1377	1378	1377
   q18	7351	7213	7031	7031
   q19	864	793	823	793
   q20	1924	1960	1816	1816
   q21	4672	4328	4223	4223
   q22	1110	1017	1004	1004
   Total cold run time: 51491 ms
   Total hot run time: 49639 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on code in PR #58877:
URL: https://github.com/apache/doris/pull/58877#discussion_r2605099037


##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java:
##########
@@ -1031,54 +976,90 @@ public static class HiveFileStatus {
     @Data
     public static class HivePartitionValues {
         private BiMap<String, Long> partitionNameToIdMap;
-        private Map<Long, List<UniqueId>> idToUniqueIdsMap;
         private Map<Long, PartitionItem> idToPartitionItem;
         private Map<Long, List<String>> partitionValuesMap;
-        //multi pair
-        private Map<UniqueId, Range<PartitionKey>> uidToPartitionRange;
-        private Map<Range<PartitionKey>, UniqueId> rangeToId;
-        //single pair
-        private RangeMap<ColumnBound, UniqueId> singleColumnRangeMap;
-        private Map<UniqueId, Range<ColumnBound>> singleUidToColumnRangeMap;
+
+        // Sorted partition ranges for binary search filtering.
+        // Built at construction time, shares the same lifecycle with HivePartitionValues.
+        private SortedPartitionRanges<String> sortedPartitionRanges;
 
         public HivePartitionValues() {
         }
 
         public HivePartitionValues(Map<Long, PartitionItem> idToPartitionItem,
-                Map<UniqueId, Range<PartitionKey>> uidToPartitionRange,
-                Map<Range<PartitionKey>, UniqueId> rangeToId,
-                RangeMap<ColumnBound, UniqueId> singleColumnRangeMap,
                 BiMap<String, Long> partitionNameToIdMap,
-                Map<Long, List<UniqueId>> idToUniqueIdsMap,
-                Map<UniqueId, Range<ColumnBound>> singleUidToColumnRangeMap,
                 Map<Long, List<String>> partitionValuesMap) {
             this.idToPartitionItem = idToPartitionItem;
-            this.uidToPartitionRange = uidToPartitionRange;
-            this.rangeToId = rangeToId;
-            this.singleColumnRangeMap = singleColumnRangeMap;
             this.partitionNameToIdMap = partitionNameToIdMap;
-            this.idToUniqueIdsMap = idToUniqueIdsMap;
-            this.singleUidToColumnRangeMap = singleUidToColumnRangeMap;
             this.partitionValuesMap = partitionValuesMap;
+            this.sortedPartitionRanges = buildSortedPartitionRanges();
         }
 
+        /**
+         * Create a copy for incremental updates (add/drop partitions).
+         * The sortedPartitionRanges will be rebuilt after the caller modifies the partition data.
+         */
         public HivePartitionValues copy() {
             HivePartitionValues copy = new HivePartitionValues();
             copy.setPartitionNameToIdMap(partitionNameToIdMap == null ? null : HashBiMap.create(partitionNameToIdMap));
-            copy.setIdToUniqueIdsMap(idToUniqueIdsMap == null ? null : Maps.newHashMap(idToUniqueIdsMap));
             copy.setIdToPartitionItem(idToPartitionItem == null ? null : Maps.newHashMap(idToPartitionItem));
             copy.setPartitionValuesMap(partitionValuesMap == null ? null : Maps.newHashMap(partitionValuesMap));
-            copy.setUidToPartitionRange(uidToPartitionRange == null ? null : Maps.newHashMap(uidToPartitionRange));
-            copy.setRangeToId(rangeToId == null ? null : Maps.newHashMap(rangeToId));
-            copy.setSingleUidToColumnRangeMap(
-                    singleUidToColumnRangeMap == null ? null : Maps.newHashMap(singleUidToColumnRangeMap));
-            if (singleColumnRangeMap != null) {
-                RangeMap<ColumnBound, UniqueId> copySingleColumnRangeMap = TreeRangeMap.create();
-                copySingleColumnRangeMap.putAll(singleColumnRangeMap);
-                copy.setSingleColumnRangeMap(copySingleColumnRangeMap);
-            }
+            // sortedPartitionRanges is not copied here, caller should call rebuildSortedPartitionRanges()
+            // after modifying partition data
             return copy;
         }
+
+        /**
+         * Rebuild sorted partition ranges after incremental updates.
+         * Should be called after add/drop partitions.
+         */
+        public void rebuildSortedPartitionRanges() {
+            this.sortedPartitionRanges = buildSortedPartitionRanges();
+        }
+
+        /**
+         * Get sorted partition ranges for binary search filtering.
+         */
+        public Optional<SortedPartitionRanges<String>> getSortedPartitionRanges() {
+            return Optional.ofNullable(sortedPartitionRanges);
+        }
+
+        private SortedPartitionRanges<String> buildSortedPartitionRanges() {
+            if (partitionNameToIdMap == null || partitionNameToIdMap.isEmpty()) {
+                return null;
+            }
+
+            BiMap<Long, String> idToName = partitionNameToIdMap.inverse();
+            List<PartitionItemAndRange<String>> sortedRanges = Lists.newArrayListWithCapacity(
+                    idToPartitionItem.size());
+            List<PartitionItemAndId<String>> defaultPartitions = Lists.newArrayList();
+
+            for (Map.Entry<Long, PartitionItem> entry : idToPartitionItem.entrySet()) {
+                PartitionItem partitionItem = entry.getValue();
+                String partitionName = idToName.get(entry.getKey());
+                if (!partitionItem.isDefaultPartition()) {
+                    List<Range<MultiColumnBound>> ranges = PartitionItemToRange.toRanges(partitionItem);
+                    for (Range<MultiColumnBound> range : ranges) {
+                        sortedRanges.add(new PartitionItemAndRange<>(partitionName, partitionItem, range));
+                    }
+                } else {
+                    defaultPartitions.add(new PartitionItemAndId<>(partitionName, partitionItem));
+                }
+            }
+
+            // Sort by range bounds
+            sortedRanges.sort((o1, o2) -> {
+                Range<MultiColumnBound> span1 = o1.range;
+                Range<MultiColumnBound> span2 = o2.range;
+                int result = span1.lowerEndpoint().compareTo(span2.lowerEndpoint());
+                if (result != 0) {
+                    return result;
+                }
+                return span1.upperEndpoint().compareTo(span2.upperEndpoint());
+            });
+
+            return new SortedPartitionRanges<>(sortedRanges, defaultPartitions);
+        }

Review Comment:
   I think you should extract code from `NereidsSortedPartitionsCacheManager.loadCache` and reuse the same utility function



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635748024

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.41 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
   
   query1	0.05	0.04	0.05
   query2	0.10	0.05	0.05
   query3	0.25	0.09	0.09
   query4	1.61	0.11	0.11
   query5	0.28	0.26	0.26
   query6	1.16	0.64	0.62
   query7	0.03	0.03	0.02
   query8	0.05	0.04	0.04
   query9	0.56	0.52	0.51
   query10	0.57	0.56	0.55
   query11	0.15	0.11	0.12
   query12	0.14	0.12	0.11
   query13	0.61	0.60	0.59
   query14	0.97	0.99	0.98
   query15	0.82	0.79	0.81
   query16	0.41	0.40	0.40
   query17	1.03	1.02	1.03
   query18	0.23	0.21	0.22
   query19	1.90	1.85	1.76
   query20	0.01	0.01	0.01
   query21	15.44	0.29	0.14
   query22	4.76	0.04	0.04
   query23	15.98	0.28	0.10
   query24	1.40	0.64	0.43
   query25	0.08	0.06	0.07
   query26	0.14	0.14	0.14
   query27	0.06	0.04	0.07
   query28	4.40	1.22	1.02
   query29	12.57	4.05	3.20
   query30	0.27	0.13	0.12
   query31	2.83	0.64	0.40
   query32	3.23	0.55	0.46
   query33	2.93	3.08	3.03
   query34	16.97	5.17	4.56
   query35	4.56	4.57	4.63
   query36	0.68	0.50	0.49
   query37	0.10	0.06	0.07
   query38	0.07	0.05	0.04
   query39	0.05	0.03	0.04
   query40	0.17	0.14	0.13
   query41	0.08	0.03	0.04
   query42	0.04	0.02	0.02
   query43	0.04	0.04	0.03
   Total cold run time: 97.78 s
   Total hot run time: 27.41 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632690380

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36315 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 1a46590096eaff7a5cf99d33963a32066f635462, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17616	5051	4934	4934
   q2	2050	366	245	245
   q3	10148	1295	739	739
   q4	10239	870	333	333
   q5	7537	2182	1900	1900
   q6	195	171	138	138
   q7	1006	873	723	723
   q8	9361	1434	1142	1142
   q9	6995	5312	5386	5312
   q10	6787	2393	1999	1999
   q11	553	316	304	304
   q12	661	740	588	588
   q13	17767	3704	3075	3075
   q14	292	297	278	278
   q15	588	515	521	515
   q16	947	910	876	876
   q17	695	817	533	533
   q18	7922	7163	7137	7137
   q19	956	964	612	612
   q20	413	372	252	252
   q21	4246	3918	3714	3714
   q22	1058	978	966	966
   Total cold run time: 108032 ms
   Total hot run time: 36315 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4947	4953	4889	4889
   q2	334	388	319	319
   q3	2129	2707	2284	2284
   q4	1308	1746	1273	1273
   q5	4965	4673	4537	4537
   q6	221	191	130	130
   q7	2097	1962	1824	1824
   q8	2690	2702	2520	2520
   q9	7974	7991	7855	7855
   q10	3213	3654	2940	2940
   q11	658	519	517	517
   q12	788	774	572	572
   q13	3908	3979	3453	3453
   q14	292	405	297	297
   q15	586	581	520	520
   q16	913	957	890	890
   q17	1227	1499	1424	1424
   q18	7859	7685	7843	7685
   q19	919	871	866	866
   q20	2018	2080	1850	1850
   q21	4543	4307	4161	4161
   q22	1069	1085	969	969
   Total cold run time: 54658 ms
   Total hot run time: 51775 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632749115

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 182033 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 1a46590096eaff7a5cf99d33963a32066f635462, data reload: false
   
   query5	5898	677	503	503
   query6	436	248	230	230
   query7	5254	475	292	292
   query8	353	261	248	248
   query9	8773	2636	2662	2636
   query10	598	366	337	337
   query11	15689	14804	15133	14804
   query12	197	120	115	115
   query13	1242	503	371	371
   query14	7749	3328	3114	3114
   query14_1	2936	3014	2963	2963
   query15	273	202	184	184
   query16	907	526	479	479
   query17	1154	698	608	608
   query18	2733	444	341	341
   query19	314	228	210	210
   query20	127	116	113	113
   query21	219	146	114	114
   query22	3820	3866	3943	3866
   query23	16803	16088	15928	15928
   query23_1	16065	16033	15930	15930
   query24	6698	1673	1246	1246
   query24_1	1249	1257	1271	1257
   query25	552	499	418	418
   query26	1224	271	170	170
   query27	2702	481	312	312
   query28	4516	2172	2142	2142
   query29	797	564	452	452
   query30	322	249	226	226
   query31	814	719	643	643
   query32	82	68	73	68
   query33	521	334	313	313
   query34	886	911	547	547
   query35	781	807	742	742
   query36	873	914	846	846
   query37	141	105	87	87
   query38	3954	3844	3771	3771
   query39	753	741	734	734
   query39_1	706	704	703	703
   query40	227	142	128	128
   query41	68	62	65	62
   query42	110	105	105	105
   query43	427	447	392	392
   query44	1393	758	771	758
   query45	194	193	186	186
   query46	898	1002	636	636
   query47	1634	1717	1600	1600
   query48	312	350	254	254
   query49	636	438	378	378
   query50	685	312	231	231
   query51	3852	3921	3812	3812
   query52	109	113	104	104
   query53	325	363	305	305
   query54	307	290	277	277
   query55	80	82	78	78
   query56	340	345	339	339
   query57	1159	1143	1087	1087
   query58	277	274	261	261
   query59	2380	2446	2348	2348
   query60	361	335	321	321
   query61	198	188	191	188
   query62	696	692	636	636
   query63	340	301	312	301
   query64	5075	1410	1138	1138
   query65	4058	3982	3978	3978
   query66	1395	476	334	334
   query67	15201	14838	14688	14688
   query68	6470	1027	747	747
   query69	504	356	330	330
   query70	1061	1034	985	985
   query71	410	325	300	300
   query72	6096	5010	4831	4831
   query73	692	617	306	306
   query74	8905	8873	8661	8661
   query75	3566	3545	3184	3184
   query76	3791	1177	791	791
   query77	513	397	304	304
   query78	9496	9845	8832	8832
   query79	1602	943	615	615
   query80	1365	661	562	562
   query81	564	269	238	238
   query82	420	127	105	105
   query83	269	256	243	243
   query84	265	114	98	98
   query85	911	509	456	456
   query86	376	321	280	280
   query87	4002	4097	3978	3978
   query88	3218	2318	2261	2261
   query89	476	434	386	386
   query90	2014	159	157	157
   query91	178	167	141	141
   query92	73	67	62	62
   query93	1279	935	576	576
   query94	536	305	277	277
   query95	552	326	366	326
   query96	606	455	211	211
   query97	2623	2632	2605	2605
   query98	213	198	193	193
   query99	1291	1312	1221	1221
   Total cold run time: 266033 ms
   Total hot run time: 182033 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635634670

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3636000410

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.64 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
   
   query1	0.06	0.05	0.06
   query2	0.10	0.05	0.04
   query3	0.25	0.08	0.09
   query4	1.60	0.11	0.11
   query5	0.27	0.25	0.26
   query6	1.17	0.64	0.62
   query7	0.03	0.03	0.02
   query8	0.05	0.04	0.05
   query9	0.58	0.51	0.51
   query10	0.54	0.55	0.55
   query11	0.16	0.11	0.11
   query12	0.15	0.11	0.12
   query13	0.62	0.60	0.60
   query14	0.98	0.99	0.98
   query15	0.83	0.80	0.81
   query16	0.40	0.40	0.40
   query17	1.07	1.04	1.05
   query18	0.24	0.22	0.22
   query19	1.90	1.82	1.74
   query20	0.01	0.02	0.02
   query21	15.47	0.26	0.14
   query22	4.76	0.05	0.05
   query23	15.94	0.28	0.10
   query24	0.95	0.57	1.09
   query25	0.11	0.06	0.07
   query26	0.14	0.13	0.13
   query27	0.06	0.05	0.05
   query28	5.43	1.23	1.02
   query29	12.58	4.15	3.31
   query30	0.31	0.14	0.12
   query31	2.83	0.64	0.40
   query32	3.23	0.55	0.46
   query33	3.05	3.05	3.08
   query34	16.83	5.18	4.49
   query35	4.60	4.59	4.52
   query36	0.64	0.51	0.49
   query37	0.11	0.07	0.07
   query38	0.08	0.04	0.04
   query39	0.05	0.03	0.04
   query40	0.18	0.15	0.15
   query41	0.09	0.04	0.03
   query42	0.04	0.03	0.03
   query43	0.05	0.04	0.04
   Total cold run time: 98.54 s
   Total hot run time: 27.64 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635702053

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36566 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17671	4245	4088	4088
   q2	2031	367	239	239
   q3	10184	1309	745	745
   q4	10227	869	311	311
   q5	7593	2104	1924	1924
   q6	184	166	136	136
   q7	1029	884	717	717
   q8	9353	1475	1110	1110
   q9	6991	5326	5339	5326
   q10	6791	2388	2010	2010
   q11	525	324	296	296
   q12	671	768	584	584
   q13	17796	3638	3034	3034
   q14	291	301	288	288
   q15	576	521	515	515
   q16	964	927	889	889
   q17	718	807	534	534
   q18	7385	7749	7823	7749
   q19	1132	1016	665	665
   q20	438	375	262	262
   q21	4687	4276	4123	4123
   q22	1097	1067	1021	1021
   Total cold run time: 108334 ms
   Total hot run time: 36566 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4379	4233	4298	4233
   q2	351	390	340	340
   q3	2377	2943	2558	2558
   q4	1408	1896	1380	1380
   q5	4591	4721	4428	4428
   q6	205	171	128	128
   q7	2021	1943	1770	1770
   q8	2676	2556	2495	2495
   q9	7591	7542	7486	7486
   q10	3038	3243	2792	2792
   q11	574	494	471	471
   q12	669	685	570	570
   q13	3297	3691	3041	3041
   q14	265	294	266	266
   q15	533	489	485	485
   q16	872	870	835	835
   q17	1161	1465	1338	1338
   q18	7262	7084	7031	7031
   q19	851	830	840	830
   q20	1916	1960	1812	1812
   q21	4626	4327	4250	4250
   q22	1045	1039	984	984
   Total cold run time: 51708 ms
   Total hot run time: 49523 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635733660

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 180827 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
   
   query5	4841	623	467	467
   query6	340	231	215	215
   query7	4211	463	272	272
   query8	293	253	226	226
   query9	8760	2579	2557	2557
   query10	534	410	347	347
   query11	15507	14793	14556	14556
   query12	176	113	111	111
   query13	1265	502	391	391
   query14	6061	3212	2943	2943
   query14_1	2841	2889	2874	2874
   query15	216	194	189	189
   query16	904	470	450	450
   query17	1085	694	591	591
   query18	2478	435	331	331
   query19	242	232	204	204
   query20	129	111	111	111
   query21	216	135	117	117
   query22	4031	4127	3903	3903
   query23	16662	16381	15949	15949
   query23_1	16064	16103	15999	15999
   query24	7322	1659	1210	1210
   query24_1	1222	1234	1212	1212
   query25	546	463	437	437
   query26	1261	306	162	162
   query27	2719	473	304	304
   query28	4472	2199	2177	2177
   query29	791	543	428	428
   query30	312	244	210	210
   query31	808	710	654	654
   query32	80	69	68	68
   query33	533	330	275	275
   query34	899	954	544	544
   query35	800	805	724	724
   query36	861	894	820	820
   query37	127	92	79	79
   query38	3935	3803	3748	3748
   query39	757	730	726	726
   query39_1	704	712	695	695
   query40	223	150	129	129
   query41	68	64	65	64
   query42	106	103	100	100
   query43	415	427	401	401
   query44	1323	760	757	757
   query45	192	186	183	183
   query46	883	976	605	605
   query47	1715	1715	1638	1638
   query48	335	330	258	258
   query49	622	434	377	377
   query50	670	286	213	213
   query51	3801	3816	3823	3816
   query52	109	115	112	112
   query53	322	351	295	295
   query54	314	271	272	271
   query55	84	74	72	72
   query56	306	333	292	292
   query57	1139	1151	1072	1072
   query58	264	252	269	252
   query59	2359	2524	2356	2356
   query60	326	314	291	291
   query61	165	161	154	154
   query62	710	669	618	618
   query63	329	293	297	293
   query64	5036	1314	992	992
   query65	3999	3956	3960	3956
   query66	1474	439	306	306
   query67	15233	14801	14706	14706
   query68	2789	1059	764	764
   query69	458	365	333	333
   query70	1026	1011	981	981
   query71	336	311	286	286
   query72	5868	5109	5017	5017
   query73	483	549	309	309
   query74	8627	8980	8695	8695
   query75	3483	3554	3206	3206
   query76	2903	1140	731	731
   query77	367	444	292	292
   query78	9507	9588	8834	8834
   query79	2399	891	627	627
   query80	1692	654	562	562
   query81	564	264	234	234
   query82	1017	131	102	102
   query83	348	249	241	241
   query84	253	124	102	102
   query85	952	501	464	464
   query86	396	321	276	276
   query87	4106	4036	4000	4000
   query88	3301	2302	2291	2291
   query89	482	429	386	386
   query90	1991	156	151	151
   query91	170	164	147	147
   query92	70	69	65	65
   query93	1134	916	581	581
   query94	530	310	263	263
   query95	573	380	313	313
   query96	597	471	205	205
   query97	2592	2627	2579	2579
   query98	201	200	190	190
   query99	1262	1314	1201	1201
   Total cold run time: 257668 ms
   Total hot run time: 180827 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635983473

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 181501 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 8a5da2d7af4be796b451d720b4ec119da10d7b46, data reload: false
   
   query5	5082	620	467	467
   query6	331	243	212	212
   query7	4234	466	274	274
   query8	299	246	243	243
   query9	8759	2532	2577	2532
   query10	547	370	338	338
   query11	15139	14863	14524	14524
   query12	173	120	111	111
   query13	1280	496	395	395
   query14	5852	3218	3007	3007
   query14_1	2890	2983	2871	2871
   query15	222	202	180	180
   query16	885	480	454	454
   query17	1126	718	614	614
   query18	2545	447	359	359
   query19	238	246	214	214
   query20	123	120	114	114
   query21	220	147	120	120
   query22	3858	3913	3913	3913
   query23	16436	16199	15903	15903
   query23_1	16101	16161	16065	16065
   query24	7399	1662	1254	1254
   query24_1	1249	1204	1259	1204
   query25	588	506	471	471
   query26	1255	290	170	170
   query27	2729	477	315	315
   query28	4426	2190	2182	2182
   query29	852	582	483	483
   query30	326	249	214	214
   query31	837	707	629	629
   query32	79	74	68	68
   query33	547	343	302	302
   query34	884	902	545	545
   query35	809	828	744	744
   query36	876	894	835	835
   query37	130	142	84	84
   query38	3883	3870	3860	3860
   query39	751	731	709	709
   query39_1	705	706	701	701
   query40	222	136	121	121
   query41	69	62	63	62
   query42	104	105	108	105
   query43	431	436	408	408
   query44	1320	767	750	750
   query45	190	192	180	180
   query46	872	982	616	616
   query47	1635	1706	1629	1629
   query48	315	323	247	247
   query49	631	434	346	346
   query50	661	290	222	222
   query51	3779	3855	3783	3783
   query52	105	108	100	100
   query53	320	346	289	289
   query54	311	258	249	249
   query55	79	73	72	72
   query56	298	319	304	304
   query57	1138	1165	1069	1069
   query58	273	253	248	248
   query59	2342	2386	2449	2386
   query60	311	316	288	288
   query61	159	155	158	155
   query62	680	656	619	619
   query63	332	301	299	299
   query64	5005	1350	998	998
   query65	4047	3938	3981	3938
   query66	1427	451	334	334
   query67	15070	14985	14806	14806
   query68	8104	1013	783	783
   query69	497	340	309	309
   query70	1056	1014	1028	1014
   query71	377	305	283	283
   query72	6077	4909	5021	4909
   query73	660	583	309	309
   query74	8763	8808	8653	8653
   query75	3549	3512	3167	3167
   query76	3934	1141	728	728
   query77	545	396	294	294
   query78	9378	9640	8997	8997
   query79	1737	889	628	628
   query80	736	649	565	565
   query81	544	268	236	236
   query82	203	133	104	104
   query83	263	255	247	247
   query84	267	120	101	101
   query85	895	507	463	463
   query86	387	309	273	273
   query87	4063	4114	3951	3951
   query88	3227	2327	2313	2313
   query89	476	431	414	414
   query90	2237	161	157	157
   query91	175	168	169	168
   query92	86	70	68	68
   query93	2115	915	575	575
   query94	490	306	275	275
   query95	567	332	303	303
   query96	587	484	211	211
   query97	2616	2624	2568	2568
   query98	217	201	190	190
   query99	1258	1284	1196	1196
   Total cold run time: 262615 ms
   Total hot run time: 181501 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635817412

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632532719

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.37 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit eb2c282b47308a9f2d5b9277650b93fd59d0180e, data reload: false
   
   query1	0.06	0.05	0.05
   query2	0.09	0.05	0.05
   query3	0.25	0.09	0.09
   query4	1.61	0.12	0.11
   query5	0.28	0.26	0.27
   query6	1.15	0.65	0.62
   query7	0.03	0.02	0.02
   query8	0.06	0.04	0.04
   query9	0.58	0.50	0.51
   query10	0.56	0.56	0.55
   query11	0.15	0.11	0.12
   query12	0.15	0.12	0.12
   query13	0.62	0.60	0.60
   query14	0.98	0.98	0.97
   query15	0.81	0.80	0.79
   query16	0.38	0.39	0.40
   query17	1.01	1.02	0.98
   query18	0.24	0.22	0.22
   query19	1.95	1.88	1.86
   query20	0.02	0.01	0.01
   query21	15.43	0.26	0.14
   query22	4.84	0.06	0.05
   query23	16.21	0.28	0.10
   query24	1.72	0.25	0.33
   query25	0.07	0.11	0.06
   query26	0.14	0.14	0.13
   query27	0.05	0.06	0.05
   query28	4.63	1.23	1.01
   query29	12.59	4.02	3.25
   query30	0.29	0.16	0.12
   query31	2.83	0.61	0.40
   query32	3.24	0.55	0.46
   query33	3.10	3.08	3.15
   query34	16.67	5.24	4.52
   query35	4.56	4.58	4.60
   query36	0.67	0.50	0.49
   query37	0.11	0.07	0.07
   query38	0.07	0.05	0.04
   query39	0.04	0.03	0.03
   query40	0.17	0.16	0.13
   query41	0.09	0.03	0.02
   query42	0.04	0.02	0.02
   query43	0.04	0.04	0.03
   Total cold run time: 98.58 s
   Total hot run time: 27.37 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632533929

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632777024

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.63 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 1a46590096eaff7a5cf99d33963a32066f635462, data reload: false
   
   query1	0.05	0.04	0.05
   query2	0.10	0.04	0.04
   query3	0.26	0.09	0.08
   query4	1.60	0.12	0.11
   query5	0.27	0.27	0.26
   query6	1.17	0.64	0.63
   query7	0.03	0.02	0.02
   query8	0.05	0.04	0.04
   query9	0.58	0.52	0.52
   query10	0.56	0.55	0.55
   query11	0.15	0.11	0.11
   query12	0.15	0.11	0.11
   query13	0.63	0.61	0.59
   query14	0.99	0.98	0.99
   query15	0.82	0.79	0.81
   query16	0.41	0.41	0.41
   query17	1.02	1.06	1.01
   query18	0.22	0.21	0.21
   query19	1.94	1.80	1.84
   query20	0.02	0.01	0.01
   query21	15.47	0.29	0.14
   query22	4.88	0.05	0.05
   query23	16.07	0.28	0.10
   query24	1.48	1.10	0.62
   query25	0.09	0.08	0.05
   query26	0.15	0.14	0.14
   query27	0.05	0.05	0.06
   query28	5.04	1.22	1.03
   query29	12.61	4.05	3.19
   query30	0.29	0.14	0.13
   query31	2.81	0.62	0.38
   query32	3.23	0.56	0.46
   query33	3.05	3.03	3.03
   query34	16.84	5.22	4.55
   query35	4.55	4.64	4.56
   query36	0.66	0.49	0.49
   query37	0.11	0.07	0.06
   query38	0.07	0.05	0.04
   query39	0.04	0.03	0.04
   query40	0.17	0.14	0.13
   query41	0.08	0.03	0.03
   query42	0.05	0.03	0.03
   query43	0.05	0.03	0.03
   Total cold run time: 98.86 s
   Total hot run time: 27.63 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632509001

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 182268 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit eb2c282b47308a9f2d5b9277650b93fd59d0180e, data reload: false
   
   query5	5065	609	469	469
   query6	328	231	221	221
   query7	4214	488	281	281
   query8	324	265	249	249
   query9	8810	2656	2679	2656
   query10	565	392	337	337
   query11	15506	14854	14835	14835
   query12	179	122	127	122
   query13	1257	529	404	404
   query14	6327	3280	3054	3054
   query14_1	2945	2954	2932	2932
   query15	219	196	186	186
   query16	787	473	476	473
   query17	1142	711	618	618
   query18	2722	453	362	362
   query19	247	236	212	212
   query20	119	116	113	113
   query21	222	142	119	119
   query22	3914	3996	3762	3762
   query23	16552	16208	16048	16048
   query23_1	16159	16046	16046	16046
   query24	7361	1674	1254	1254
   query24_1	1248	1235	1246	1235
   query25	576	502	458	458
   query26	1249	287	167	167
   query27	2736	460	310	310
   query28	4431	2164	2176	2164
   query29	850	580	469	469
   query30	321	245	219	219
   query31	827	728	646	646
   query32	82	75	73	73
   query33	555	364	311	311
   query34	914	908	539	539
   query35	793	862	737	737
   query36	891	916	833	833
   query37	130	91	75	75
   query38	3853	3852	3801	3801
   query39	759	731	711	711
   query39_1	701	697	705	697
   query40	221	143	123	123
   query41	67	62	63	62
   query42	106	107	105	105
   query43	433	424	390	390
   query44	1317	758	758	758
   query45	192	194	185	185
   query46	872	989	615	615
   query47	1682	1742	1638	1638
   query48	328	325	248	248
   query49	649	444	353	353
   query50	658	298	230	230
   query51	3911	3874	3876	3874
   query52	108	112	98	98
   query53	329	350	297	297
   query54	286	261	249	249
   query55	78	78	74	74
   query56	306	308	301	301
   query57	1162	1156	1086	1086
   query58	278	257	260	257
   query59	2400	2475	2346	2346
   query60	321	311	292	292
   query61	162	155	156	155
   query62	695	679	629	629
   query63	325	298	297	297
   query64	4997	1307	1017	1017
   query65	4075	3977	3998	3977
   query66	1385	433	321	321
   query67	15293	14904	14766	14766
   query68	8415	1026	728	728
   query69	505	343	320	320
   query70	1068	967	996	967
   query71	381	331	289	289
   query72	6121	5187	5275	5187
   query73	701	645	311	311
   query74	8924	8846	8827	8827
   query75	3600	3538	3165	3165
   query76	4047	1153	760	760
   query77	592	409	293	293
   query78	9417	9645	8845	8845
   query79	1627	875	628	628
   query80	723	640	566	566
   query81	497	273	242	242
   query82	200	131	106	106
   query83	263	253	234	234
   query84	256	115	96	96
   query85	899	521	463	463
   query86	388	305	288	288
   query87	4012	4025	4002	4002
   query88	3890	2282	2277	2277
   query89	474	423	404	404
   query90	2198	160	152	152
   query91	175	167	143	143
   query92	83	69	61	61
   query93	1992	916	563	563
   query94	472	313	292	292
   query95	583	386	303	303
   query96	595	467	211	211
   query97	2603	2624	2566	2566
   query98	221	193	193	193
   query99	1316	1316	1244	1244
   Total cold run time: 265575 ms
   Total hot run time: 182268 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632368641

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635525653

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.33 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 22e3e49a305a54a8ec210d4545be2253a54128ab, data reload: false
   
   query1	0.06	0.05	0.05
   query2	0.10	0.04	0.05
   query3	0.26	0.09	0.09
   query4	1.60	0.11	0.10
   query5	0.30	0.29	0.26
   query6	1.17	0.64	0.64
   query7	0.04	0.03	0.02
   query8	0.06	0.05	0.04
   query9	0.57	0.50	0.51
   query10	0.56	0.55	0.54
   query11	0.16	0.11	0.11
   query12	0.15	0.12	0.12
   query13	0.63	0.60	0.61
   query14	0.99	0.98	0.98
   query15	0.81	0.80	0.81
   query16	0.39	0.40	0.39
   query17	1.05	1.05	1.03
   query18	0.23	0.23	0.22
   query19	1.96	1.85	1.85
   query20	0.02	0.02	0.01
   query21	15.43	0.27	0.14
   query22	5.03	0.05	0.05
   query23	16.08	0.29	0.10
   query24	0.95	0.42	0.18
   query25	0.06	0.06	0.06
   query26	0.14	0.14	0.13
   query27	0.07	0.06	0.05
   query28	3.61	1.22	1.03
   query29	12.62	4.07	3.26
   query30	0.28	0.13	0.11
   query31	2.82	0.63	0.40
   query32	3.22	0.54	0.45
   query33	2.98	3.08	2.99
   query34	16.90	5.21	4.60
   query35	4.57	4.54	4.54
   query36	0.66	0.49	0.51
   query37	0.10	0.06	0.07
   query38	0.08	0.04	0.04
   query39	0.04	0.03	0.03
   query40	0.16	0.14	0.14
   query41	0.09	0.03	0.04
   query42	0.04	0.03	0.03
   query43	0.05	0.04	0.03
   Total cold run time: 97.09 s
   Total hot run time: 27.33 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635634752

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635644671

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3636882899

   # FE Regression Coverage Report
   Increment line coverage `  76.36% (42/55)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/58877_8a5da2d7af4be796b451d720b4ec119da10d7b46_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/58877_8a5da2d7af4be796b451d720b4ec119da10d7b46_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #58877:
URL: https://github.com/apache/doris/pull/58877


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632988002

   # FE UT Coverage Report
   Increment line coverage `  10.00% (2/20)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462/fe_increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/58877_1a46590096eaff7a5cf99d33963a32066f635462/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3635515339

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 181153 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 22e3e49a305a54a8ec210d4545be2253a54128ab, data reload: false
   
   query5	4841	602	464	464
   query6	325	231	217	217
   query7	4221	464	279	279
   query8	298	253	234	234
   query9	8767	2560	2592	2560
   query10	510	381	324	324
   query11	15449	14859	14916	14859
   query12	168	114	114	114
   query13	1258	505	398	398
   query14	6030	3306	2970	2970
   query14_1	2876	2850	2872	2850
   query15	211	198	197	197
   query16	887	495	422	422
   query17	1146	674	578	578
   query18	2462	439	338	338
   query19	229	239	204	204
   query20	122	113	110	110
   query21	217	136	113	113
   query22	3950	4008	3827	3827
   query23	16659	16175	15901	15901
   query23_1	16007	16132	16098	16098
   query24	7340	1625	1242	1242
   query24_1	1228	1226	1238	1226
   query25	556	506	445	445
   query26	1249	269	162	162
   query27	2776	484	310	310
   query28	4486	2169	2140	2140
   query29	793	549	438	438
   query30	315	247	222	222
   query31	811	708	634	634
   query32	78	71	65	65
   query33	523	341	285	285
   query34	898	889	544	544
   query35	789	797	716	716
   query36	872	905	833	833
   query37	132	95	74	74
   query38	3971	3865	3806	3806
   query39	752	742	747	742
   query39_1	695	686	709	686
   query40	225	142	120	120
   query41	66	62	62	62
   query42	109	106	105	105
   query43	433	436	405	405
   query44	1355	760	755	755
   query45	193	189	184	184
   query46	871	986	614	614
   query47	1652	1698	1624	1624
   query48	320	331	245	245
   query49	635	435	350	350
   query50	655	303	226	226
   query51	3975	3849	3827	3827
   query52	112	107	102	102
   query53	317	341	300	300
   query54	289	274	261	261
   query55	82	74	71	71
   query56	291	294	293	293
   query57	1165	1142	1068	1068
   query58	268	249	261	249
   query59	2311	2410	2321	2321
   query60	344	324	315	315
   query61	211	184	179	179
   query62	726	692	646	646
   query63	336	303	305	303
   query64	5170	1329	1000	1000
   query65	4049	3926	3993	3926
   query66	1452	437	322	322
   query67	15275	14983	14648	14648
   query68	4656	1037	751	751
   query69	494	353	315	315
   query70	1068	965	981	965
   query71	371	325	285	285
   query72	6185	4972	5066	4972
   query73	692	619	308	308
   query74	8897	8779	8570	8570
   query75	3577	3527	3227	3227
   query76	3800	1135	753	753
   query77	525	402	310	310
   query78	9492	9555	8811	8811
   query79	1043	887	643	643
   query80	1153	691	591	591
   query81	553	276	237	237
   query82	411	134	108	108
   query83	341	259	240	240
   query84	256	130	105	105
   query85	1047	563	464	464
   query86	381	297	284	284
   query87	4066	4178	3960	3960
   query88	3195	2308	2278	2278
   query89	469	423	404	404
   query90	1940	166	156	156
   query91	181	168	145	145
   query92	73	68	66	66
   query93	1022	944	577	577
   query94	528	294	282	282
   query95	592	340	394	340
   query96	594	467	223	223
   query97	2608	2634	2588	2588
   query98	210	192	202	192
   query99	1274	1294	1186	1186
   Total cold run time: 259789 ms
   Total hot run time: 181153 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](hive) use binary search to prune hive partitions [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58877:
URL: https://github.com/apache/doris/pull/58877#issuecomment-3632344311

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org