You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2026/10/22 07:19:38 UTC

[PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

airborne12 opened a new pull request, #69441:
URL: https://github.com/apache/doris/pull/69441

   ### What problem does this PR solve?
   
   Issue Number: None
   
   Related PR: #68674
   
   Problem Summary:
   
   The light `DROP INDEX` pre-check added in #68674 (`SchemaChangeHandler#checkNoIndexChangeJobOnPartitions`) calls `hasIndexChangeJobOnPartition` once per partition, and every call copies and scans the whole retained `indexChangeJobs` history (finished jobs are kept for days). For a table with P partitions and a history of H jobs that is O(P * H) work while the table write lock is held, on top of the existing per-partition check in `buildOrDeleteTableInvertedIndices`. A recent `BUILD INDEX` on a table with many partitions leaves at least one record per partition, which makes it quadratic in the number of partitions.
   
   The pre-check also reported a conflict with the `BUILD INDEX` wording `partition xxx has been built specified index. please check your build stmt.`, which is misleading for a `DROP INDEX`.
   
   Fix: the pre-check takes one snapshot of the job history and scans it once, matching jobs on any of the table's partitions. The match condition is extracted into `isConflictingIndexChangeJob` and shared with `hasIndexChangeJobOnPartition`, so both checks keep the same rule. The pre-check reports `partition xxx of table t has an unfinished index change job, please retry after it finishes.` The `BUILD INDEX` / `DROP INDEX ... PARTITION` check and its message are unchanged.
   
   ### Release note
   
   A light DROP INDEX rejected because of an unfinished index change job now reports `partition xxx of table t has an unfinished index change job, please retry after it finishes.`
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [x] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
     `SchemaChangeHandlerTest#testDropIndexRejectedByIndexChangeJobKeepsMetadata` now expects the new message, and keeps checking the rejected DROP leaves the metadata unchanged and the drop succeeds once the job finishes.
   
   - Behavior changed:
       - [ ] No.
       - [x] Yes. <!-- Explain the behavior change -->
         Only the error message of a rejected light DROP INDEX changes.
   
   - Does this need documentation?
       - [x] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6354976212

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #69441:
URL: https://github.com/apache/doris/pull/69441#discussion_r4345804049


##########
fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java:
##########
@@ -3974,20 +3976,25 @@ public boolean hasIndexChangeJobOnPartition(
             List<Index> alterIndexes, boolean isDrop) {
         // TODO: this is temporary methods
         for (IndexChangeJob indexChangeJob : ImmutableList.copyOf(indexChangeJobs.values())) {
-            if (indexChangeJob.getOriginIndexId() == originIndexId
-                    && indexChangeJob.getDbId() == dbId
-                    && indexChangeJob.getTableId() == tableId
-                    && indexChangeJob.getPartitionName().equals(partitionName)
-                    && ((Config.isNotCloudMode() && indexChangeJob.hasSameAlterInvertedIndex(isDrop, alterIndexes))
-                    || Config.isCloudMode())
-                    && !indexChangeJob.isDone()) {
-                // if JobState is done (CANCELLED or FINISHED), also allow user to create job again
+            if (isConflictingIndexChangeJob(indexChangeJob, originIndexId, dbId, tableId, alterIndexes, isDrop)

Review Comment:
   [P2] Keep the partition guard ahead of index-list matching
   
   `hasIndexChangeJobOnPartition` is still called once per selected partition by `buildOrDeleteTableInvertedIndices`. Moving the partition comparison after `isConflictingIndexChangeJob` makes local mode run `hasSameAlterInvertedIndex`'s nested index scans for every retained job on the table, including finished jobs, for every partition. For example, after a batch partition DROP of 50 indexes across 1,000 partitions completes, dropping a disjoint batch of 50 indexes across those partitions now performs about 2.5 billion index-ID comparisons against that history instead of 2.5 million (in addition to the existing history scans). This work holds the table write lock and can substantially stall metadata access. Preserve the partition-name short circuit before invoking the shared conflict predicate.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6344852067

   # FE Regression Coverage Report
   Increment line coverage `  7.53% (7/93)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6350036463

   # FE Regression Coverage Report
   Increment line coverage `  50.00% (7/14)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6340169780

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6335117199

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6349898369

   # FE UT Coverage Report
   Increment line coverage `  57.14% (8/14)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d/fe_increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6350315155

   # FE Regression Coverage Report
   Increment line coverage `  50.00% (7/14)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 merged PR #69441:
URL: https://github.com/apache/doris/pull/69441


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6348470285

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 24.4 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 89555012934acca4763f0176649e72828b41496d, data reload: false
   
   query1	0.01	0.01	0.01
   query2	0.10	0.05	0.05
   query3	0.26	0.09	0.09
   query4	1.61	0.10	0.10
   query5	0.24	0.25	0.25
   query6	1.17	0.70	0.68
   query7	0.04	0.01	0.01
   query8	0.06	0.05	0.04
   query9	0.41	0.36	0.34
   query10	0.59	0.61	0.59
   query11	0.20	0.15	0.15
   query12	0.18	0.16	0.15
   query13	0.52	0.50	0.51
   query14	1.03	1.01	1.02
   query15	0.64	0.63	0.63
   query16	0.35	0.36	0.34
   query17	1.10	1.16	1.15
   query18	0.22	0.20	0.20
   query19	2.09	1.93	2.01
   query20	0.02	0.01	0.02
   query21	15.52	0.19	0.13
   query22	4.97	0.06	0.05
   query23	16.25	0.32	0.12
   query24	3.09	0.61	0.36
   query25	0.11	0.05	0.06
   query26	1.16	0.22	0.16
   query27	0.06	0.06	0.05
   query28	3.15	0.77	0.38
   query29	12.51	4.12	3.26
   query30	0.32	0.15	0.12
   query31	2.78	0.57	0.32
   query32	3.23	0.62	0.50
   query33	3.16	3.21	3.17
   query34	15.49	4.03	3.37
   query35	3.34	3.32	3.37
   query36	0.57	0.46	0.44
   query37	0.09	0.07	0.06
   query38	0.04	0.04	0.04
   query39	0.04	0.03	0.03
   query40	0.18	0.15	0.15
   query41	0.09	0.03	0.04
   query42	0.04	0.03	0.02
   query43	0.05	0.04	0.04
   Total cold run time: 97.08 s
   Total hot run time: 24.4 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on code in PR #69441:
URL: https://github.com/apache/doris/pull/69441#discussion_r4347840078


##########
fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java:
##########
@@ -3974,20 +3976,25 @@ public boolean hasIndexChangeJobOnPartition(
             List<Index> alterIndexes, boolean isDrop) {
         // TODO: this is temporary methods
         for (IndexChangeJob indexChangeJob : ImmutableList.copyOf(indexChangeJobs.values())) {
-            if (indexChangeJob.getOriginIndexId() == originIndexId
-                    && indexChangeJob.getDbId() == dbId
-                    && indexChangeJob.getTableId() == tableId
-                    && indexChangeJob.getPartitionName().equals(partitionName)
-                    && ((Config.isNotCloudMode() && indexChangeJob.hasSameAlterInvertedIndex(isDrop, alterIndexes))
-                    || Config.isCloudMode())
-                    && !indexChangeJob.isDone()) {
-                // if JobState is done (CANCELLED or FINISHED), also allow user to create job again
+            if (isConflictingIndexChangeJob(indexChangeJob, originIndexId, dbId, tableId, alterIndexes, isDrop)

Review Comment:
   Fixed in the latest commit: both the per-partition check (hasIndexChangeJobOnPartition) and the single-scan pre-check compare the partition first again and only then run the shared predicate with the index-list comparison.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6341649231

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 24.29 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 89555012934acca4763f0176649e72828b41496d, data reload: false
   
   query1	0.00	0.00	0.00
   query2	0.09	0.05	0.05
   query3	0.26	0.09	0.08
   query4	1.60	0.13	0.11
   query5	0.23	0.23	0.24
   query6	1.17	0.69	0.70
   query7	0.04	0.01	0.01
   query8	0.05	0.04	0.04
   query9	0.41	0.36	0.34
   query10	0.56	0.56	0.60
   query11	0.19	0.14	0.14
   query12	0.18	0.14	0.14
   query13	0.48	0.50	0.49
   query14	1.03	1.02	1.02
   query15	0.63	0.61	0.60
   query16	0.33	0.34	0.34
   query17	1.17	1.12	1.10
   query18	0.21	0.19	0.19
   query19	2.03	1.96	1.91
   query20	0.02	0.01	0.02
   query21	15.49	0.19	0.14
   query22	4.96	0.05	0.05
   query23	16.27	0.32	0.12
   query24	3.04	0.54	0.51
   query25	0.11	0.06	0.06
   query26	1.16	0.21	0.14
   query27	0.04	0.04	0.05
   query28	3.73	0.78	0.35
   query29	12.48	4.08	3.26
   query30	0.28	0.13	0.09
   query31	2.76	0.55	0.32
   query32	3.23	0.59	0.50
   query33	3.18	3.18	3.17
   query34	15.33	3.99	3.37
   query35	3.35	3.35	3.34
   query36	0.55	0.48	0.42
   query37	0.09	0.07	0.06
   query38	0.04	0.04	0.04
   query39	0.03	0.03	0.04
   query40	0.18	0.14	0.14
   query41	0.09	0.03	0.03
   query42	0.04	0.03	0.03
   query43	0.04	0.04	0.03
   Total cold run time: 97.15 s
   Total hot run time: 24.29 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6348362325

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 151565 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 89555012934acca4763f0176649e72828b41496d, data reload: false
   
   query5	4313	586	460	460
   query6	441	208	194	194
   query7	4806	451	224	224
   query8	327	179	171	171
   query9	8797	3110	3133	3110
   query10	455	318	261	261
   query11	5917	3912	3597	3597
   query12	145	95	86	86
   query13	1245	441	319	319
   query14	6135	3938	3718	3718
   query14_1	3518	3527	3527	3527
   query15	205	196	180	180
   query16	945	423	392	392
   query17	832	640	510	510
   query18	2407	417	303	303
   query19	197	158	135	135
   query20	84	81	78	78
   query21	207	132	108	108
   query22	12964	13039	12911	12911
   query23	13406	12813	12510	12510
   query23_1	12505	12524	12584	12524
   query24	7383	851	484	484
   query24_1	469	475	469	469
   query25	516	401	343	343
   query26	1260	260	140	140
   query27	2770	437	270	270
   query28	4625	1951	1951	1951
   query29	1567	607	460	460
   query30	295	222	184	184
   query31	1018	926	819	819
   query32	149	89	96	89
   query33	553	302	247	247
   query34	932	871	506	506
   query35	739	767	658	658
   query36	805	803	707	707
   query37	130	105	87	87
   query38	1798	1747	1679	1679
   query39	792	785	747	747
   query39_1	731	740	756	740
   query40	220	119	101	101
   query41	75	73	67	67
   query42	90	87	87	87
   query43	355	367	324	324
   query44	1378	686	693	686
   query45	183	178	174	174
   query46	842	981	588	588
   query47	2922	2747	2660	2660
   query48	300	330	233	233
   query49	587	409	293	293
   query50	678	282	219	219
   query51	10357	10497	10327	10327
   query52	81	88	74	74
   query53	193	222	162	162
   query54	269	212	189	189
   query55	75	70	63	63
   query56	267	246	219	219
   query57	1655	1636	1565	1565
   query58	290	279	275	275
   query59	2132	2164	1954	1954
   query60	307	260	238	238
   query61	174	166	167	166
   query62	405	339	287	287
   query63	196	168	169	168
   query64	2904	1076	768	768
   query65	3315	3287	3271	3271
   query66	1779	431	297	297
   query67	20059	20153	20026	20026
   query68	3017	1027	608	608
   query69	385	295	258	258
   query70	867	788	752	752
   query71	302	235	215	215
   query72	2706	2640	2398	2398
   query73	529	543	304	304
   query74	4644	4510	4377	4377
   query75	2355	2301	1921	1921
   query76	2320	1017	607	607
   query77	345	377	275	275
   query78	9744	9764	9174	9174
   query79	1027	852	519	519
   query80	1239	456	357	357
   query81	599	322	271	271
   query82	599	130	110	110
   query83	298	200	179	179
   query84	305	120	107	107
   query85	859	437	385	385
   query86	397	243	232	232
   query87	2031	1968	1812	1812
   query88	3771	2685	2672	2672
   query89	328	286	248	248
   query90	1860	177	172	172
   query91	169	156	130	130
   query92	102	89	83	83
   query93	1138	1030	565	565
   query94	628	324	277	277
   query95	571	338	359	338
   query96	675	507	235	235
   query97	2458	2454	2331	2331
   query98	161	171	149	149
   query99	761	758	638	638
   Total cold run time: 235436 ms
   Total hot run time: 151565 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6348158168

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 29029 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 89555012934acca4763f0176649e72828b41496d, data reload: false
   
   ------ Round 1 ----------------------------------
   ============================================
   q1	17602	5115	5097	5097
   q2	2145	346	279	279
   q3	10147	1391	843	843
   q4	4681	463	347	347
   q5	7448	811	541	541
   q6	184	188	159	159
   q7	753	812	606	606
   q8	9254	1360	1090	1090
   q9	5549	4199	4199	4199
   q10	6829	1293	1022	1022
   q11	430	253	240	240
   q12	639	408	303	303
   q13	18079	3702	3029	3029
   q14	276	273	255	255
   q15	q16	731	714	670	670
   q17	1198	762	583	583
   q18	7106	6176	6114	6114
   q19	1247	977	601	601
   q20	393	352	225	225
   q21	5918	2536	2533	2533
   q22	388	340	293	293
   Total cold run time: 100997 ms
   Total hot run time: 29029 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   ============================================
   q1	6309	5914	6113	5914
   q2	710	604	595	595
   q3	5477	5759	5242	5242
   q4	2041	2192	1649	1649
   q5	5801	5686	5872	5686
   q6	247	184	142	142
   q7	2215	2008	1806	1806
   q8	3076	2680	2662	2662
   q9	7881	7951	7681	7681
   q10	3642	3596	3158	3158
   q11	603	449	407	407
   q12	678	686	513	513
   q13	3365	3682	3073	3073
   q14	290	289	268	268
   q15	q16	654	700	633	633
   q17	7801	7050	6989	6989
   q18	12968	12182	12939	12182
   q19	934	840	820	820
   q20	2153	2157	1946	1946
   q21	5938	4935	4990	4935
   q22	492	487	404	404
   Total cold run time: 73275 ms
   Total hot run time: 66705 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6341349652

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 28191 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 89555012934acca4763f0176649e72828b41496d, data reload: false
   
   ------ Round 1 ----------------------------------
   ============================================
   q1	17572	5101	5016	5016
   q2	2123	298	247	247
   q3	10276	1340	818	818
   q4	4674	461	333	333
   q5	7416	791	523	523
   q6	180	186	153	153
   q7	754	783	608	608
   q8	9253	1336	1028	1028
   q9	5417	4149	4162	4149
   q10	6810	1293	1009	1009
   q11	425	249	234	234
   q12	652	394	279	279
   q13	18098	3522	2978	2978
   q14	278	276	245	245
   q15	q16	724	698	650	650
   q17	1176	737	568	568
   q18	6950	6157	6088	6088
   q19	1109	946	622	622
   q20	370	338	231	231
   q21	5166	2147	2292	2147
   q22	343	301	265	265
   Total cold run time: 99766 ms
   Total hot run time: 28191 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   ============================================
   q1	5551	5574	5571	5571
   q2	688	585	549	549
   q3	5185	5377	4933	4933
   q4	1985	2135	1547	1547
   q5	5250	5137	5442	5137
   q6	262	214	156	156
   q7	2383	2008	1786	1786
   q8	2960	2662	2696	2662
   q9	7845	8007	7860	7860
   q10	3682	3656	3222	3222
   q11	599	426	408	408
   q12	789	754	525	525
   q13	3585	3791	3239	3239
   q14	326	345	302	302
   q15	q16	717	734	648	648
   q17	8268	7333	6835	6835
   q18	13022	12225	12845	12225
   q19	876	758	782	758
   q20	2151	2131	1922	1922
   q21	5587	5142	4648	4648
   q22	484	447	398	398
   Total cold run time: 72195 ms
   Total hot run time: 65331 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6341550709

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 150195 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 89555012934acca4763f0176649e72828b41496d, data reload: false
   
   query5	4315	590	443	443
   query6	419	212	190	190
   query7	4809	436	228	228
   query8	332	162	150	150
   query9	8759	3061	3086	3061
   query10	474	303	252	252
   query11	5998	3842	3596	3596
   query12	149	92	90	90
   query13	1247	450	318	318
   query14	6053	3961	3642	3642
   query14_1	3421	3407	3423	3407
   query15	216	194	178	178
   query16	940	428	412	412
   query17	848	647	538	538
   query18	2424	444	312	312
   query19	197	167	158	158
   query20	83	85	83	83
   query21	213	133	107	107
   query22	12945	12892	12908	12892
   query23	13426	12867	12406	12406
   query23_1	12362	12547	12468	12468
   query24	7397	823	457	457
   query24_1	465	470	479	470
   query25	531	420	359	359
   query26	1256	256	142	142
   query27	2765	435	266	266
   query28	4635	1934	1935	1934
   query29	1587	623	458	458
   query30	295	221	188	188
   query31	997	927	812	812
   query32	145	98	93	93
   query33	532	302	242	242
   query34	881	849	497	497
   query35	725	756	668	668
   query36	805	787	710	710
   query37	124	100	83	83
   query38	1768	1729	1709	1709
   query39	791	809	745	745
   query39_1	735	727	738	727
   query40	219	117	96	96
   query41	72	66	66	66
   query42	90	88	83	83
   query43	343	382	320	320
   query44	1319	694	689	689
   query45	178	174	164	164
   query46	830	984	567	567
   query47	2829	2838	2641	2641
   query48	306	296	228	228
   query49	586	393	292	292
   query50	689	282	208	208
   query51	10561	10502	10469	10469
   query52	84	78	75	75
   query53	182	207	150	150
   query54	245	203	174	174
   query55	70	67	64	64
   query56	226	206	197	197
   query57	1676	1598	1570	1570
   query58	285	262	258	258
   query59	2136	2171	1910	1910
   query60	285	234	215	215
   query61	144	139	141	139
   query62	378	330	281	281
   query63	184	155	157	155
   query64	2733	912	737	737
   query65	3311	3253	3255	3253
   query66	1786	417	293	293
   query67	20027	20365	19780	19780
   query68	2869	1039	618	618
   query69	385	297	240	240
   query70	857	781	756	756
   query71	291	231	215	215
   query72	2672	2334	2273	2273
   query73	507	519	295	295
   query74	4617	4562	4343	4343
   query75	2313	2279	1918	1918
   query76	2296	1017	602	602
   query77	332	362	266	266
   query78	9596	9766	9120	9120
   query79	977	851	518	518
   query80	816	409	342	342
   query81	574	314	273	273
   query82	571	134	105	105
   query83	283	191	182	182
   query84	301	118	107	107
   query85	938	417	370	370
   query86	354	232	227	227
   query87	1989	1972	1790	1790
   query88	3626	2681	2651	2651
   query89	321	277	237	237
   query90	1718	168	161	161
   query91	160	151	129	129
   query92	93	89	91	89
   query93	985	989	568	568
   query94	497	305	265	265
   query95	562	329	296	296
   query96	651	471	229	229
   query97	2429	2448	2294	2294
   query98	161	150	148	148
   query99	774	758	639	639
   Total cold run time: 233085 ms
   Total hot run time: 150195 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6351215082

   run check_coverage


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6335118323

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6335213708

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](fe) Scan index change jobs once in the light DROP INDEX pre-check [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69441:
URL: https://github.com/apache/doris/pull/69441#issuecomment-6342759013

   # FE UT Coverage Report
   Increment line coverage `  57.14% (8/14)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d/fe_increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69441_89555012934acca4763f0176649e72828b41496d/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org