You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2026/10/23 14:06:00 UTC
[PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
morningman opened a new pull request, #69552:
URL: https://github.com/apache/doris/pull/69552
### What problem does this PR solve?
Issue Number: None
Related PR: #66246, #66007, #66112
Problem Summary:
**In short.** This ports test assertions from branch-4.1 #66246 that master never got. #66246 backported #66112, #66223 and #66007 to branch-4.1 and added checks for historical joins, branch reads and DESC comments on Iceberg tables. Master already behaves as those checks expect, but its suites either lack them or still assert the old, negative contract. No production code changes.
**Background.** An audit of every block of #66246 that branch-4.2 still has, against master, found these test gaps. Everything else in #66246 is on master in another form (mostly in the connector plugins) or was replaced on master. One behavior differs on purpose: a plain Hudi read on master is not pinned to the instant seen at analysis (#64304). This PR leaves that alone.
**The problem, and what it cost.** Without the checks, a regression in these paths would go unnoticed, and two suites still describe the current behavior as a defect:
| Suite | Master today | After this PR |
|---|---|---|
| `test_iceberg_schema_dual_relation_matrix` | no check that a historical relation and the latest relation of the same table keep their own schemas in one join | both join orders checked: `old_name` on the snapshot, `new_name` on the latest table |
| `test_iceberg_schema_time_travel_matrix` | branch reads checked only as "negative contracts": the old name is unknown, nested types are promoted | a branch read uses the table's current schema; renamed and re-added fields read NULL from the branch head (`qt_`, same rows as branch-4.2) |
| `test_iceberg_schema_metadata_atomicity_matrix` | asserts that DESC omits Iceberg field comments, which only holds because `show_column_comment_in_describe` is off by default | enables the variable and asserts that DESC shows the top-level and nested comments |
| `CreateTableCommandTest` | #67545 replaced the "existing CTAS target fails before any sink is built" case with the configuration preflight case | the existing-target case is back, next to the preflight case |
**How this PR fixes it.**
- `test_iceberg_schema_dual_relation_matrix` gets the two joins from branch-4.2, written with `assertEquals` like the rest of the master suite.
- `test_iceberg_schema_time_travel_matrix` keeps master's checks, rewords their comments, and adds `qt_doris_nested_branch_current_schema` and `qt_top_branch_current_schema` from branch-4.2, with a new `.out`.
- `test_iceberg_schema_metadata_atomicity_matrix` sets `show_column_comment_in_describe=true` and asserts that the comments are shown, as on branch-4.2.
- `CreateTableCommandTest#existingCtasTargetPrecedesUnsupportedSinkValidation` comes from branch-4.2. On master the configuration preflight runs first, so the test stubs it to pass.
**Results.** On a local cluster built from master with the Iceberg docker environment, the three suites pass. The `.out` is generated by the regression script and has the same rows as branch-4.2. All 4 cases of `CreateTableCommandTest` pass, and checkstyle passes.
Original author: @Gabriel39
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6366779590
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6372282611
<details>
<summary>TPC-H: <b>Total hot run time: 28375 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c5ee12cd7332bd335830aa1049fe11e0af1648c9, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17589 5146 5047 5047
q2 2092 311 253 253
q3 10256 1342 818 818
q4 4677 456 338 338
q5 7427 790 519 519
q6 181 186 158 158
q7 748 787 600 600
q8 9248 1326 1029 1029
q9 5491 4164 4158 4158
q10 6835 1289 1029 1029
q11 442 249 228 228
q12 634 395 284 284
q13 18080 3604 2990 2990
q14 269 276 247 247
q15 q16 706 707 662 662
q17 1185 752 569 569
q18 7011 6153 6121 6121
q19 1098 950 605 605
q20 377 340 221 221
q21 5166 2264 2250 2250
q22 341 294 249 249
Total cold run time: 99853 ms
Total hot run time: 28375 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 5587 5636 5588 5588
q2 684 571 524 524
q3 5205 5398 4920 4920
q4 2024 2125 1744 1744
q5 5252 5157 5515 5157
q6 257 195 147 147
q7 2344 1984 1840 1840
q8 3060 2708 2746 2708
q9 8021 7888 8007 7888
q10 3706 3660 3214 3214
q11 690 503 433 433
q12 705 680 520 520
q13 3622 3887 3226 3226
q14 319 309 293 293
q15 q16 699 753 648 648
q17 8266 7465 6927 6927
q18 13047 12216 12890 12216
q19 892 772 780 772
q20 2115 2130 1924 1924
q21 5694 4697 4681 4681
q22 505 456 430 430
Total cold run time: 72694 ms
Total hot run time: 65800 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6366766205
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6372458188
<details>
<summary>TPC-DS: <b>Total hot run time: 151063 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c5ee12cd7332bd335830aa1049fe11e0af1648c9, data reload: false
query5 4330 610 457 457
query6 432 200 188 188
query7 4789 450 235 235
query8 320 167 150 150
query9 8779 3099 3080 3080
query10 461 318 254 254
query11 6073 3874 3569 3569
query12 162 93 87 87
query13 1242 446 340 340
query14 6093 3903 3638 3638
query14_1 3411 3455 3409 3409
query15 204 197 176 176
query16 951 416 399 399
query17 837 635 518 518
query18 2424 421 302 302
query19 203 165 131 131
query20 84 82 80 80
query21 208 130 108 108
query22 12937 12940 12899 12899
query23 13619 13057 12478 12478
query23_1 12526 12473 12570 12473
query24 7383 844 464 464
query24_1 483 476 469 469
query25 540 426 365 365
query26 1273 256 143 143
query27 2765 438 271 271
query28 4671 1925 1941 1925
query29 1564 606 461 461
query30 300 226 184 184
query31 1037 937 813 813
query32 138 96 94 94
query33 523 303 247 247
query34 976 856 501 501
query35 743 754 642 642
query36 833 812 756 756
query37 127 98 93 93
query38 1793 1722 1646 1646
query39 798 786 757 757
query39_1 749 737 737 737
query40 231 116 97 97
query41 72 71 70 70
query42 87 81 82 81
query43 346 366 311 311
query44 1311 689 683 683
query45 182 178 165 165
query46 810 965 570 570
query47 2892 2773 2878 2773
query48 302 339 222 222
query49 577 395 303 303
query50 713 284 215 215
query51 10353 10655 10466 10466
query52 89 83 74 74
query53 197 211 162 162
query54 240 201 189 189
query55 70 73 64 64
query56 245 239 246 239
query57 1679 1642 1588 1588
query58 299 274 259 259
query59 2076 2090 1926 1926
query60 314 260 235 235
query61 169 167 167 167
query62 384 340 301 301
query63 195 163 166 163
query64 2855 943 753 753
query65 3311 3294 3249 3249
query66 1759 413 299 299
query67 20324 20164 19982 19982
query68 3083 998 605 605
query69 380 293 247 247
query70 878 767 783 767
query71 283 227 217 217
query72 2688 2601 2302 2302
query73 516 541 299 299
query74 4650 4538 4373 4373
query75 2345 2273 1906 1906
query76 2326 1018 576 576
query77 329 396 267 267
query78 9841 9791 9225 9225
query79 1012 864 522 522
query80 1450 423 337 337
query81 600 343 281 281
query82 586 132 108 108
query83 281 196 179 179
query84 311 129 100 100
query85 1092 426 374 374
query86 403 235 228 228
query87 2016 1937 1825 1825
query88 3645 2674 2652 2652
query89 332 284 240 240
query90 1832 170 166 166
query91 166 151 125 125
query92 101 84 79 79
query93 1026 1038 576 576
query94 640 297 271 271
query95 574 324 299 299
query96 661 491 236 236
query97 2468 2414 2339 2339
query98 166 150 147 147
query99 780 763 644 644
Total cold run time: 236034 ms
Total hot run time: 151063 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6372548236
<details>
<summary>ClickBench: <b>Total hot run time: 24.35 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c5ee12cd7332bd335830aa1049fe11e0af1648c9, data reload: false
query1 0.01 0.01 0.01
query2 0.10 0.05 0.05
query3 0.26 0.09 0.08
query4 1.62 0.10 0.10
query5 0.24 0.24 0.24
query6 1.20 0.68 0.66
query7 0.04 0.02 0.00
query8 0.06 0.04 0.04
query9 0.40 0.34 0.36
query10 0.58 0.61 0.61
query11 0.20 0.15 0.15
query12 0.17 0.15 0.15
query13 0.51 0.50 0.49
query14 1.02 1.02 1.01
query15 0.62 0.63 0.62
query16 0.34 0.36 0.31
query17 1.11 1.11 1.15
query18 0.22 0.20 0.22
query19 2.08 1.95 1.97
query20 0.02 0.01 0.02
query21 15.49 0.19 0.13
query22 4.85 0.06 0.05
query23 16.28 0.31 0.12
query24 2.99 0.41 0.41
query25 0.11 0.06 0.06
query26 0.91 0.21 0.16
query27 0.05 0.05 0.04
query28 3.79 0.79 0.37
query29 12.48 4.08 3.21
query30 0.28 0.13 0.10
query31 2.77 0.55 0.32
query32 3.22 0.59 0.50
query33 3.27 3.22 3.29
query34 15.44 4.03 3.36
query35 3.39 3.35 3.34
query36 0.56 0.49 0.47
query37 0.09 0.06 0.06
query38 0.05 0.04 0.03
query39 0.04 0.03 0.03
query40 0.17 0.16 0.15
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.03 0.04
Total cold run time: 97.19 s
Total hot run time: 24.35 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6375195344
# FE Regression Coverage Report
Increment line coverage ` 100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69552_c5ee12cd7332bd335830aa1049fe11e0af1648c9_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69552_c5ee12cd7332bd335830aa1049fe11e0af1648c9_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #69552:
URL: https://github.com/apache/doris/pull/69552#issuecomment-6366778000
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [test](iceberg) Port the branch-4.1 schema matrix assertions for joins, branch reads and DESC comments [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #69552:
URL: https://github.com/apache/doris/pull/69552
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org