You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2026/01/21 09:30:47 UTC
[PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
cambyzju opened a new pull request, #60115:
URL: https://github.com/apache/doris/pull/60115
### What problem does this PR solve?
like https://github.com/apache/doris/pull/60003, When there are large of columns(such as 3W columns) in table, TupleDescriptor.getSlot function will be the main performance bottleneck.
**Before**
```
- Init Scan Node Time: 1s94ms
```
**After**
```
- Init Scan Node Time: 145ms
```
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3777594356
<details>
<summary>TPC-DS: <b>Total hot run time: 174448 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1ded16abe8c30621305d6e1affb6792137769472, data reload: false
query5 4400 628 483 483
query6 333 219 213 213
query7 4231 452 267 267
query8 324 244 242 242
query9 8666 2890 2873 2873
query10 546 380 341 341
query11 15179 15302 15132 15132
query12 177 124 116 116
query13 1259 479 405 405
query14 6164 3083 2827 2827
query14_1 2736 2671 2725 2671
query15 206 192 173 173
query16 985 479 466 466
query17 1102 666 555 555
query18 2451 441 339 339
query19 221 247 190 190
query20 119 116 112 112
query21 207 134 123 123
query22 3890 4080 3713 3713
query23 16084 15647 15192 15192
query23_1 15487 15572 15378 15378
query24 7121 1540 1158 1158
query24_1 1157 1179 1160 1160
query25 513 420 380 380
query26 1242 262 148 148
query27 2785 464 278 278
query28 4524 2146 2131 2131
query29 715 521 410 410
query30 323 245 205 205
query31 756 631 572 572
query32 82 79 74 74
query33 528 339 311 311
query34 867 878 526 526
query35 735 751 693 693
query36 874 893 846 846
query37 130 98 83 83
query38 2735 2722 2635 2635
query39 799 752 741 741
query39_1 697 720 714 714
query40 214 134 116 116
query41 74 61 62 61
query42 105 104 100 100
query43 433 451 435 435
query44 1330 745 737 737
query45 191 193 183 183
query46 823 945 568 568
query47 1419 1462 1379 1379
query48 321 330 243 243
query49 608 435 339 339
query50 618 275 205 205
query51 3756 3784 3771 3771
query52 101 105 98 98
query53 295 329 276 276
query54 297 293 283 283
query55 86 85 81 81
query56 317 336 322 322
query57 1033 1032 975 975
query58 276 266 266 266
query59 2185 2146 2077 2077
query60 349 345 316 316
query61 175 170 169 169
query62 404 369 323 323
query63 308 271 267 267
query64 4984 1374 1043 1043
query65 3863 3756 3767 3756
query66 1486 452 351 351
query67 15588 15556 15468 15468
query68 2405 1110 762 762
query69 523 361 322 322
query70 923 969 964 964
query71 328 312 289 289
query72 5345 3187 3250 3187
query73 606 742 315 315
query74 8741 8777 8527 8527
query75 2789 2807 2472 2472
query76 2276 1063 694 694
query77 341 388 300 300
query78 9828 9956 9236 9236
query79 1108 945 582 582
query80 1273 585 477 477
query81 557 265 236 236
query82 987 151 111 111
query83 335 252 242 242
query84 249 121 95 95
query85 901 477 432 432
query86 414 292 316 292
query87 2882 2907 2750 2750
query88 3517 2600 2572 2572
query89 396 360 321 321
query90 1954 175 171 171
query91 174 170 157 157
query92 79 77 66 66
query93 1030 899 546 546
query94 635 330 298 298
query95 579 395 324 324
query96 642 520 235 235
query97 2363 2379 2324 2324
query98 225 211 197 197
query99 598 571 506 506
Total cold run time: 247376 ms
Total hot run time: 174448 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3777614403
<details>
<summary>ClickBench: <b>Total hot run time: 26.68 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1ded16abe8c30621305d6e1affb6792137769472, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.05 0.04
query3 0.25 0.09 0.09
query4 1.60 0.12 0.11
query5 0.28 0.26 0.27
query6 1.15 0.67 0.65
query7 0.03 0.02 0.02
query8 0.06 0.04 0.04
query9 0.57 0.50 0.49
query10 0.56 0.55 0.55
query11 0.14 0.10 0.09
query12 0.15 0.11 0.10
query13 0.60 0.58 0.59
query14 0.94 0.92 0.95
query15 0.79 0.78 0.78
query16 0.40 0.40 0.42
query17 0.99 1.03 0.99
query18 0.23 0.22 0.22
query19 1.92 1.85 1.87
query20 0.02 0.02 0.01
query21 15.46 0.26 0.13
query22 5.43 0.05 0.05
query23 16.27 0.29 0.11
query24 0.92 0.84 0.27
query25 0.11 0.11 0.06
query26 0.16 0.14 0.13
query27 0.08 0.06 0.04
query28 3.78 1.08 0.88
query29 12.56 3.90 3.08
query30 0.28 0.14 0.12
query31 2.82 0.60 0.39
query32 3.23 0.56 0.46
query33 2.96 3.02 3.09
query34 16.13 5.13 4.44
query35 4.47 4.43 4.48
query36 0.67 0.51 0.49
query37 0.10 0.06 0.06
query38 0.06 0.04 0.03
query39 0.04 0.02 0.03
query40 0.16 0.14 0.14
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.05 0.04 0.03
Total cold run time: 96.7 s
Total hot run time: 26.68 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3782122259
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3777554257
<details>
<summary>TPC-H: <b>Total hot run time: 32143 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1ded16abe8c30621305d6e1affb6792137769472, data reload: false
------ Round 1 ----------------------------------
q1 17632 4269 4083 4083
q2 2057 392 256 256
q3 10096 1259 720 720
q4 10200 780 304 304
q5 7520 2152 1881 1881
q6 236 170 140 140
q7 930 819 656 656
q8 9308 1406 1175 1175
q9 5206 4745 4591 4591
q10 6822 1792 1400 1400
q11 529 292 286 286
q12 787 735 575 575
q13 17810 3925 3109 3109
q14 286 311 274 274
q15 603 514 516 514
q16 694 674 648 648
q17 785 848 506 506
q18 6940 6451 7151 6451
q19 1698 1112 720 720
q20 408 402 267 267
q21 3321 2666 2582 2582
q22 1127 1095 1005 1005
Total cold run time: 104995 ms
Total hot run time: 32143 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4437 4422 4384 4384
q2 345 413 341 341
q3 2416 2773 2419 2419
q4 1547 2031 1491 1491
q5 4515 4325 4341 4325
q6 212 177 129 129
q7 2143 1914 1812 1812
q8 2643 2381 2360 2360
q9 7118 7173 7122 7122
q10 2504 2651 2124 2124
q11 511 459 435 435
q12 699 734 559 559
q13 3408 3856 3123 3123
q14 269 281 260 260
q15 529 495 488 488
q16 622 673 616 616
q17 1123 1277 1311 1277
q18 7411 7275 7254 7254
q19 853 834 839 834
q20 1900 1953 1786 1786
q21 4733 4281 4265 4265
q22 1071 1073 968 968
Total cold run time: 51009 ms
Total hot run time: 48372 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3777070083
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3777070505
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3782428911
# FE Regression Coverage Report
Increment line coverage ` 75.00% (3/4)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60115_1ded16abe8c30621305d6e1affb6792137769472_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60115_1ded16abe8c30621305d6e1affb6792137769472_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3777437913
# FE UT Coverage Report
Increment line coverage ` 50.00% (2/4)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60115_1ded16abe8c30621305d6e1affb6792137769472/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60115_1ded16abe8c30621305d6e1affb6792137769472/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3789371761
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60115:
URL: https://github.com/apache/doris/pull/60115#issuecomment-3789371575
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](fe) opt the performance for TupleDescriptor.getSlot [doris]
Posted by "lide-reed (via GitHub)" <gi...@apache.org>.
lide-reed merged PR #60115:
URL: https://github.com/apache/doris/pull/60115
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org