You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "jacktengg (via GitHub)" <gi...@apache.org> on 2026/10/14 15:45:10 UTC
[PR] [fix](be) Handle nullable integer array_sort comparators [doris]
jacktengg opened a new pull request, #69078:
URL: https://github.com/apache/doris/pull/69078
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: SELECT ARRAY_SORT((x, y) -> y - x, ARRAY(1, 3, 2)) crashes BE and surfaces as an RPC failure. The comparator returns a nullable SMALLINT, while array_sort casts its result directly to ColumnInt8. Unwrapping nullable alone still leaves an incorrect integer-width assumption. Materialize constant results, reject actual NULL values, and compare against zero in the comparator's own integer type, preserving LARGEINT signs without narrowing. Reject non-integer comparator return types during preparation. Add unit and regression coverage, and keep nested binding tests valid by wrapping their array-valued bodies in an integer-valued cardinality mock.
### Release note
Fix BE crashes when ARRAY_SORT lambda comparators return nullable or wider integer values. NULL and non-integer comparator results report query errors.
### Check List (For Author)
- Test:
- Unit Test: 13 related ArrayMapFunctionTest array_sort tests passed under ASAN.
- Regression test: test_array_sort_lambda_comparator passed during output generation and normal verification; 7 existing and 9 new output sections verified.
- Manual test: the reported SQL returns [3, 2, 1]; a NULL comparator reports an error and the reported SQL succeeds afterward.
- Validation: build.sh --be (ASAN), clang-format v16, build hygiene, and clang-tidy passed.
- Behavior changed: Yes; valid integer comparators sort without crashing, and invalid comparator results fail the query.
- Does this need documentation: No
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6173914051
<details>
<summary>ClickBench: <b>Total hot run time: 24.5 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 831bba6e9a954ae95467e26c841af24b7b8b7759, data reload: false
query1 0.01 0.01 0.00
query2 0.10 0.05 0.05
query3 0.27 0.13 0.12
query4 1.61 0.13 0.13
query5 0.24 0.25 0.25
query6 1.17 0.68 0.67
query7 0.03 0.01 0.00
query8 0.05 0.04 0.03
query9 0.40 0.35 0.35
query10 0.60 0.57 0.56
query11 0.20 0.15 0.14
query12 0.18 0.15 0.14
query13 0.48 0.51 0.48
query14 1.04 1.02 1.00
query15 0.63 0.60 0.62
query16 0.36 0.36 0.36
query17 1.14 1.11 1.12
query18 0.22 0.20 0.20
query19 2.04 2.01 1.92
query20 0.02 0.01 0.01
query21 15.50 0.26 0.14
query22 4.49 0.06 0.05
query23 16.27 0.33 0.12
query24 3.12 0.66 0.39
query25 0.11 0.06 0.05
query26 1.32 0.24 0.15
query27 0.04 0.03 0.03
query28 2.21 0.61 0.42
query29 12.47 4.13 3.32
query30 0.29 0.17 0.13
query31 2.77 0.58 0.34
query32 3.22 0.61 0.53
query33 3.10 3.22 3.30
query34 15.56 4.03 3.38
query35 3.33 3.33 3.36
query36 0.55 0.46 0.42
query37 0.09 0.07 0.06
query38 0.05 0.04 0.03
query39 0.04 0.03 0.03
query40 0.18 0.15 0.15
query41 0.08 0.03 0.03
query42 0.03 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 95.65 s
Total hot run time: 24.5 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6172275363
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6194993276
Already fixed by https://github.com/apache/doris/pull/68612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6172276753
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6172278945
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6176569662
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/69078_831bba6e9a954ae95467e26c841af24b7b8b7759_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/69078_831bba6e9a954ae95467e26c841af24b7b8b7759_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 77.01% (35595/46219) |
| Line Coverage | 62.29% (403029/646991) |
| Region Coverage | 58.71% (340367/579780) |
| Branch Coverage | 59.60% (156760/263009) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg closed pull request #69078: [fix](be) Handle nullable integer array_sort comparators
URL: https://github.com/apache/doris/pull/69078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6175702299
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/831bba6e9a954ae95467e26c841af24b7b8b7759_831bba6e9a954ae95467e26c841af24b7b8b7759/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/831bba6e9a954ae95467e26c841af24b7b8b7759_831bba6e9a954ae95467e26c841af24b7b8b7759/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 66.14% (31581/47747) |
| Line Coverage | 50.95% (333409/654362) |
| Region Coverage | 46.26% (269364/582229) |
| Branch Coverage | 47.90% (126369/263838) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6173627150
<details>
<summary>TPC-H: <b>Total hot run time: 29630 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 831bba6e9a954ae95467e26c841af24b7b8b7759, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17577 5599 5527 5527
q2 2107 327 266 266
q3 10228 1393 857 857
q4 4687 479 343 343
q5 7438 801 538 538
q6 200 189 158 158
q7 771 799 624 624
q8 9252 1338 1089 1089
q9 5621 4273 4269 4269
q10 6816 1297 1027 1027
q11 434 253 240 240
q12 634 399 279 279
q13 18100 3529 3002 3002
q14 272 276 244 244
q15 q16 745 725 654 654
q17 1204 764 595 595
q18 7018 6217 6200 6200
q19 1277 954 601 601
q20 386 340 231 231
q21 5538 2572 2628 2572
q22 404 369 314 314
Total cold run time: 100709 ms
Total hot run time: 29630 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 6632 6617 6741 6617
q2 742 581 589 581
q3 6016 5765 5208 5208
q4 2054 2174 1559 1559
q5 5745 5711 5866 5711
q6 247 191 146 146
q7 2267 1972 1817 1817
q8 3112 2815 2701 2701
q9 8006 8008 7902 7902
q10 3649 3564 3268 3268
q11 588 403 372 372
q12 657 691 495 495
q13 3240 3621 3016 3016
q14 278 297 279 279
q15 q16 655 677 620 620
q17 7833 6961 6856 6856
q18 13025 12238 12981 12238
q19 926 809 795 795
q20 2203 2173 1947 1947
q21 5825 4787 4918 4787
q22 505 456 405 405
Total cold run time: 74205 ms
Total hot run time: 67320 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Handle nullable integer array_sort comparators [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69078:
URL: https://github.com/apache/doris/pull/69078#issuecomment-6173822208
<details>
<summary>TPC-DS: <b>Total hot run time: 152334 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 831bba6e9a954ae95467e26c841af24b7b8b7759, data reload: false
query5 4302 566 449 449
query6 420 199 187 187
query7 4799 447 220 220
query8 322 170 153 153
query9 8780 3954 3927 3927
query10 473 307 254 254
query11 5956 3781 3529 3529
query12 137 102 87 87
query13 1266 438 324 324
query14 6149 4041 3755 3755
query14_1 3523 3528 3501 3501
query15 206 199 181 181
query16 939 442 418 418
query17 843 658 544 544
query18 2442 425 314 314
query19 194 161 134 134
query20 86 89 83 83
query21 211 131 112 112
query22 12908 13035 12729 12729
query23 13586 12991 12547 12547
query23_1 12640 12679 12534 12534
query24 7113 842 475 475
query24_1 477 476 486 476
query25 534 425 368 368
query26 1283 265 141 141
query27 2761 449 278 278
query28 4599 1998 1979 1979
query29 1570 612 454 454
query30 291 226 188 188
query31 995 932 804 804
query32 146 104 91 91
query33 517 309 251 251
query34 900 871 517 517
query35 740 781 649 649
query36 815 821 737 737
query37 130 103 94 94
query38 1804 1775 1685 1685
query39 793 788 770 770
query39_1 737 726 746 726
query40 216 117 102 102
query41 73 73 70 70
query42 88 85 83 83
query43 367 384 332 332
query44 1322 711 730 711
query45 189 181 168 168
query46 845 977 576 576
query47 3000 3045 2962 2962
query48 299 309 215 215
query49 568 406 290 290
query50 691 301 214 214
query51 10567 10446 10707 10446
query52 75 81 73 73
query53 192 209 150 150
query54 224 204 188 188
query55 78 66 68 66
query56 242 213 220 213
query57 1621 1567 1506 1506
query58 288 259 254 254
query59 2122 2132 1992 1992
query60 272 240 221 221
query61 147 145 147 145
query62 397 338 291 291
query63 198 156 167 156
query64 2756 938 791 791
query65 3325 3228 3244 3228
query66 1772 404 295 295
query67 20303 20112 20144 20112
query68 3126 1017 622 622
query69 385 284 244 244
query70 851 794 749 749
query71 309 237 211 211
query72 2660 2540 2262 2262
query73 532 531 296 296
query74 4594 4455 4288 4288
query75 2319 2302 1937 1937
query76 2299 1025 616 616
query77 343 384 278 278
query78 9757 9752 9176 9176
query79 1008 848 543 543
query80 1410 422 356 356
query81 588 312 268 268
query82 572 129 103 103
query83 273 200 210 200
query84 331 131 97 97
query85 1093 413 360 360
query86 368 241 229 229
query87 1997 1926 1815 1815
query88 3659 2726 2669 2669
query89 344 300 261 261
query90 1757 186 183 183
query91 167 149 126 126
query92 101 91 81 81
query93 1030 1040 603 603
query94 585 301 245 245
query95 599 347 332 332
query96 691 535 240 240
query97 2469 2414 2325 2325
query98 168 151 147 147
query99 775 763 646 646
Total cold run time: 235564 ms
Total hot run time: 152334 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org