You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2026/10/23 14:06:00 UTC

[PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

morningman opened a new pull request, #69552:
URL: https://github.com/apache/doris/pull/69552

   ### What problem does this PR solve?
   
   Issue Number: None
   
   Related PR: #66246, #66007, #66112
   
   Problem Summary:
   
   **In short.** This ports test assertions from branch-4.1 #66246 that master never got. #66246 backported #66112, #66223 and #66007 to branch-4.1 and added checks for historical joins, branch reads and DESC comments on Iceberg tables. Master already behaves as those checks expect, but its suites either lack them or still assert the old, negative contract. No production code changes.
   
   **Background.** An audit of every block of #66246 that branch-4.2 still has, against master, found these test gaps. Everything else in #66246 is on master in another form (mostly in the connector plugins) or was replaced on master. One behavior differs on purpose: a plain Hudi read on master is not pinned to the instant seen at analysis (#64304). This PR leaves that alone.
   
   **The problem, and what it cost.** Without the checks, a regression in these paths would go unnoticed, and two suites still describe the current behavior as a defect:
   
   | Suite | Master today | After this PR |
   |---|---|---|
   | `test_iceberg_schema_dual_relation_matrix` | no check that a historical relation and the latest relation of the same table keep their own schemas in one join | both join orders checked: `old_name` on the snapshot, `new_name` on the latest table |
   | `test_iceberg_schema_time_travel_matrix` | branch reads checked only as "negative contracts": the old name is unknown, nested types are promoted | a branch read uses the table's current schema; renamed and re-added fields read NULL from the branch head (`qt_`, same rows as branch-4.2) |
   | `test_iceberg_schema_metadata_atomicity_matrix` | asserts that DESC omits Iceberg field comments, which only holds because `show_column_comment_in_describe` is off by default | enables the variable and asserts that DESC shows the top-level and nested comments |
   | `CreateTableCommandTest` | #67545 replaced the "existing CTAS target fails before any sink is built" case with the configuration preflight case | the existing-target case is back, next to the preflight case |
   
   **How this PR fixes it.**
   
   - `test_iceberg_schema_dual_relation_matrix` gets the two joins from branch-4.2, written with `assertEquals` like the rest of the master suite.
   - `test_iceberg_schema_time_travel_matrix` keeps master's checks, rewords their comments, and adds `qt_doris_nested_branch_current_schema` and `qt_top_branch_current_schema` from branch-4.2, with a new `.out`.
   - `test_iceberg_schema_metadata_atomicity_matrix` sets `show_column_comment_in_describe=true` and asserts that the comments are shown, as on branch-4.2.
   - `CreateTableCommandTest#existingCtasTargetPrecedesUnsupportedSinkValidation` comes from branch-4.2. On master the configuration preflight runs first, so the test stubs it to pass.
   
   **Results.** On a local cluster built from master with the Iceberg docker environment, the three suites pass. The `.out` is generated by the regression script and has the same rows as branch-4.2. All 4 cases of `CreateTableCommandTest` pass, and checkstyle passes.
   
   Original author: @Gabriel39
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [x] Regression test
       - [x] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [x] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [x] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6366779590

   /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6372282611

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 28375 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit c5ee12cd7332bd335830aa1049fe11e0af1648c9, data reload: false
   
   ------ Round 1 ----------------------------------
   ============================================
   q1	17589	5146	5047	5047
   q2	2092	311	253	253
   q3	10256	1342	818	818
   q4	4677	456	338	338
   q5	7427	790	519	519
   q6	181	186	158	158
   q7	748	787	600	600
   q8	9248	1326	1029	1029
   q9	5491	4164	4158	4158
   q10	6835	1289	1029	1029
   q11	442	249	228	228
   q12	634	395	284	284
   q13	18080	3604	2990	2990
   q14	269	276	247	247
   q15	q16	706	707	662	662
   q17	1185	752	569	569
   q18	7011	6153	6121	6121
   q19	1098	950	605	605
   q20	377	340	221	221
   q21	5166	2264	2250	2250
   q22	341	294	249	249
   Total cold run time: 99853 ms
   Total hot run time: 28375 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   ============================================
   q1	5587	5636	5588	5588
   q2	684	571	524	524
   q3	5205	5398	4920	4920
   q4	2024	2125	1744	1744
   q5	5252	5157	5515	5157
   q6	257	195	147	147
   q7	2344	1984	1840	1840
   q8	3060	2708	2746	2708
   q9	8021	7888	8007	7888
   q10	3706	3660	3214	3214
   q11	690	503	433	433
   q12	705	680	520	520
   q13	3622	3887	3226	3226
   q14	319	309	293	293
   q15	q16	699	753	648	648
   q17	8266	7465	6927	6927
   q18	13047	12216	12890	12216
   q19	892	772	780	772
   q20	2115	2130	1924	1924
   q21	5694	4697	4681	4681
   q22	505	456	430	430
   Total cold run time: 72694 ms
   Total hot run time: 65800 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6366766205

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6372458188

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 151063 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit c5ee12cd7332bd335830aa1049fe11e0af1648c9, data reload: false
   
   query5	4330	610	457	457
   query6	432	200	188	188
   query7	4789	450	235	235
   query8	320	167	150	150
   query9	8779	3099	3080	3080
   query10	461	318	254	254
   query11	6073	3874	3569	3569
   query12	162	93	87	87
   query13	1242	446	340	340
   query14	6093	3903	3638	3638
   query14_1	3411	3455	3409	3409
   query15	204	197	176	176
   query16	951	416	399	399
   query17	837	635	518	518
   query18	2424	421	302	302
   query19	203	165	131	131
   query20	84	82	80	80
   query21	208	130	108	108
   query22	12937	12940	12899	12899
   query23	13619	13057	12478	12478
   query23_1	12526	12473	12570	12473
   query24	7383	844	464	464
   query24_1	483	476	469	469
   query25	540	426	365	365
   query26	1273	256	143	143
   query27	2765	438	271	271
   query28	4671	1925	1941	1925
   query29	1564	606	461	461
   query30	300	226	184	184
   query31	1037	937	813	813
   query32	138	96	94	94
   query33	523	303	247	247
   query34	976	856	501	501
   query35	743	754	642	642
   query36	833	812	756	756
   query37	127	98	93	93
   query38	1793	1722	1646	1646
   query39	798	786	757	757
   query39_1	749	737	737	737
   query40	231	116	97	97
   query41	72	71	70	70
   query42	87	81	82	81
   query43	346	366	311	311
   query44	1311	689	683	683
   query45	182	178	165	165
   query46	810	965	570	570
   query47	2892	2773	2878	2773
   query48	302	339	222	222
   query49	577	395	303	303
   query50	713	284	215	215
   query51	10353	10655	10466	10466
   query52	89	83	74	74
   query53	197	211	162	162
   query54	240	201	189	189
   query55	70	73	64	64
   query56	245	239	246	239
   query57	1679	1642	1588	1588
   query58	299	274	259	259
   query59	2076	2090	1926	1926
   query60	314	260	235	235
   query61	169	167	167	167
   query62	384	340	301	301
   query63	195	163	166	163
   query64	2855	943	753	753
   query65	3311	3294	3249	3249
   query66	1759	413	299	299
   query67	20324	20164	19982	19982
   query68	3083	998	605	605
   query69	380	293	247	247
   query70	878	767	783	767
   query71	283	227	217	217
   query72	2688	2601	2302	2302
   query73	516	541	299	299
   query74	4650	4538	4373	4373
   query75	2345	2273	1906	1906
   query76	2326	1018	576	576
   query77	329	396	267	267
   query78	9841	9791	9225	9225
   query79	1012	864	522	522
   query80	1450	423	337	337
   query81	600	343	281	281
   query82	586	132	108	108
   query83	281	196	179	179
   query84	311	129	100	100
   query85	1092	426	374	374
   query86	403	235	228	228
   query87	2016	1937	1825	1825
   query88	3645	2674	2652	2652
   query89	332	284	240	240
   query90	1832	170	166	166
   query91	166	151	125	125
   query92	101	84	79	79
   query93	1026	1038	576	576
   query94	640	297	271	271
   query95	574	324	299	299
   query96	661	491	236	236
   query97	2468	2414	2339	2339
   query98	166	150	147	147
   query99	780	763	644	644
   Total cold run time: 236034 ms
   Total hot run time: 151063 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6372548236

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 24.35 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit c5ee12cd7332bd335830aa1049fe11e0af1648c9, data reload: false
   
   query1	0.01	0.01	0.01
   query2	0.10	0.05	0.05
   query3	0.26	0.09	0.08
   query4	1.62	0.10	0.10
   query5	0.24	0.24	0.24
   query6	1.20	0.68	0.66
   query7	0.04	0.02	0.00
   query8	0.06	0.04	0.04
   query9	0.40	0.34	0.36
   query10	0.58	0.61	0.61
   query11	0.20	0.15	0.15
   query12	0.17	0.15	0.15
   query13	0.51	0.50	0.49
   query14	1.02	1.02	1.01
   query15	0.62	0.63	0.62
   query16	0.34	0.36	0.31
   query17	1.11	1.11	1.15
   query18	0.22	0.20	0.22
   query19	2.08	1.95	1.97
   query20	0.02	0.01	0.02
   query21	15.49	0.19	0.13
   query22	4.85	0.06	0.05
   query23	16.28	0.31	0.12
   query24	2.99	0.41	0.41
   query25	0.11	0.06	0.06
   query26	0.91	0.21	0.16
   query27	0.05	0.05	0.04
   query28	3.79	0.79	0.37
   query29	12.48	4.08	3.21
   query30	0.28	0.13	0.10
   query31	2.77	0.55	0.32
   query32	3.22	0.59	0.50
   query33	3.27	3.22	3.29
   query34	15.44	4.03	3.36
   query35	3.39	3.35	3.34
   query36	0.56	0.49	0.47
   query37	0.09	0.06	0.06
   query38	0.05	0.04	0.03
   query39	0.04	0.03	0.03
   query40	0.17	0.16	0.15
   query41	0.08	0.03	0.03
   query42	0.04	0.03	0.03
   query43	0.04	0.03	0.04
   Total cold run time: 97.19 s
   Total hot run time: 24.35 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6375195344

   # FE Regression Coverage Report
   Increment line coverage `  100% (0/0)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/69552_c5ee12cd7332bd335830aa1049fe11e0af1648c9_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/69552_c5ee12cd7332bd335830aa1049fe11e0af1648c9_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6366778000

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #69552:
URL: https://github.com/apache/doris/pull/69552


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org