You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "suxiaogang223 (via GitHub)" <gi...@apache.org> on 2025/11/12 08:31:50 UTC
[PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
suxiaogang223 opened a new pull request, #57950:
URL: https://github.com/apache/doris/pull/57950
### What problem does this PR solve?
Problem Summary:
Previously, PaimonScanNode.setPaimonParams() was called for each PaimonSplit and set hadoop conf (backendStorageProperties) in every split, which is redundant since these properties are the same at ScanNode level.
This commit:
1. Modifies PaimonJniReader to prefer hadoop conf from scan node level (range_params->properties) over split level, with fallback for backward compatibility
2. Removes redundant fileDesc.setHadoopConf() call in PaimonScanNode, since backendStorageProperties is already set at ScanNode level via params.properties in createScanRangeLocations()
This optimization:
- Reduces memory usage by avoiding duplicate configuration in each split
- Improves performance by eliminating redundant serialization
- Aligns with the implementation pattern used by Hudi and other data sources
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3521976254
# FE Regression Coverage Report
Increment line coverage ` 100.00% (4/4)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57950_0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57950_0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3526508950
<details>
<summary>ClickBench: <b>Total hot run time: 28.34 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9e94c75f46034e8e89d904927c7f056cb1181c3, data reload: false
query1 0.06 0.06 0.05
query2 0.12 0.07 0.07
query3 0.30 0.08 0.08
query4 1.60 0.09 0.10
query5 0.27 0.25 0.26
query6 1.19 0.66 0.65
query7 0.03 0.03 0.03
query8 0.08 0.06 0.06
query9 0.63 0.52 0.53
query10 0.58 0.59 0.59
query11 0.26 0.13 0.14
query12 0.27 0.14 0.16
query13 0.64 0.64 0.63
query14 1.03 1.03 1.02
query15 0.94 0.85 0.86
query16 0.39 0.38 0.39
query17 1.02 1.01 1.03
query18 0.23 0.22 0.22
query19 1.93 1.80 1.83
query20 0.02 0.02 0.02
query21 15.39 0.29 0.24
query22 4.94 0.11 0.10
query23 15.35 0.38 0.23
query24 2.71 0.52 0.34
query25 0.10 0.10 0.10
query26 0.18 0.18 0.18
query27 0.09 0.10 0.09
query28 3.65 1.25 1.07
query29 12.58 4.00 3.31
query30 0.37 0.16 0.13
query31 2.81 0.64 0.44
query32 3.23 0.59 0.51
query33 3.12 3.07 3.16
query34 16.56 5.20 4.40
query35 4.61 4.49 4.56
query36 0.65 0.52 0.50
query37 0.22 0.10 0.09
query38 0.19 0.06 0.06
query39 0.06 0.05 0.05
query40 0.21 0.18 0.18
query41 0.12 0.07 0.06
query42 0.07 0.06 0.05
query43 0.06 0.06 0.06
Total cold run time: 98.86 s
Total hot run time: 28.34 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3521406339
<details>
<summary>TPC-DS: <b>Total hot run time: 188290 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc, data reload: false
query1 1038 403 393 393
query2 6564 1685 1692 1685
query3 6761 228 224 224
query4 26721 23532 23481 23481
query5 4420 632 474 474
query6 349 242 232 232
query7 4645 509 306 306
query8 345 270 246 246
query9 8727 2648 2620 2620
query10 496 365 296 296
query11 15832 15028 14828 14828
query12 189 124 119 119
query13 1687 570 453 453
query14 11209 9223 9119 9119
query15 219 195 174 174
query16 7682 701 519 519
query17 1254 773 641 641
query18 2052 433 338 338
query19 216 212 183 183
query20 134 129 123 123
query21 218 133 115 115
query22 4126 4247 4171 4171
query23 33880 33120 33045 33045
query24 8414 2419 2405 2405
query25 596 501 431 431
query26 1250 264 155 155
query27 2759 508 355 355
query28 4315 2228 2212 2212
query29 800 627 497 497
query30 299 229 198 198
query31 919 796 711 711
query32 84 73 76 73
query33 626 373 321 321
query34 803 851 535 535
query35 805 817 741 741
query36 944 985 926 926
query37 114 111 88 88
query38 3564 3487 3509 3487
query39 1508 1406 1397 1397
query40 221 126 123 123
query41 64 62 59 59
query42 134 111 115 111
query43 493 503 463 463
query44 1234 757 734 734
query45 185 176 171 171
query46 876 988 644 644
query47 1748 1834 1727 1727
query48 392 424 313 313
query49 769 516 438 438
query50 651 692 399 399
query51 3902 3958 3927 3927
query52 109 109 104 104
query53 248 269 196 196
query54 302 295 276 276
query55 86 83 85 83
query56 316 321 300 300
query57 1185 1193 1100 1100
query58 295 272 267 267
query59 2548 2660 2562 2562
query60 340 354 331 331
query61 166 161 158 158
query62 799 723 657 657
query63 224 189 193 189
query64 4390 1154 868 868
query65 4044 3978 3976 3976
query66 1089 439 334 334
query67 15495 15212 15099 15099
query68 8301 924 603 603
query69 482 316 287 287
query70 1294 1262 1254 1254
query71 466 351 315 315
query72 5786 4918 4948 4918
query73 661 583 360 360
query74 9047 9015 8925 8925
query75 3577 3308 2819 2819
query76 3503 1143 731 731
query77 781 409 314 314
query78 9696 9734 8871 8871
query79 1928 825 596 596
query80 666 546 489 489
query81 514 262 229 229
query82 287 158 136 136
query83 266 268 256 256
query84 254 106 90 90
query85 892 483 438 438
query86 382 309 297 297
query87 3696 3665 3613 3613
query88 2864 2253 2226 2226
query89 388 323 297 297
query90 1966 215 214 214
query91 162 162 134 134
query92 83 70 63 63
query93 2313 993 650 650
query94 696 461 342 342
query95 408 311 299 299
query96 482 565 282 282
query97 2883 2959 2843 2843
query98 239 220 211 211
query99 1366 1382 1262 1262
Total cold run time: 275198 ms
Total hot run time: 188290 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3530624907
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3525722814
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3525851879
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c9e94c75f46034e8e89d904927c7f056cb1181c3_c9e94c75f46034e8e89d904927c7f056cb1181c3_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c9e94c75f46034e8e89d904927c7f056cb1181c3_c9e94c75f46034e8e89d904927c7f056cb1181c3_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.56% (1683/2089) |
| Line Coverage | 66.73% (29695/44498) |
| Region Coverage | 67.34% (14831/22024) |
| Branch Coverage | 57.57% (7889/13704) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3521428305
<details>
<summary>ClickBench: <b>Total hot run time: 28.37 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc, data reload: false
query1 0.05 0.05 0.05
query2 0.11 0.07 0.06
query3 0.30 0.07 0.07
query4 1.60 0.09 0.09
query5 0.26 0.24 0.25
query6 1.18 0.66 0.65
query7 0.03 0.03 0.02
query8 0.07 0.05 0.06
query9 0.63 0.54 0.52
query10 0.58 0.60 0.59
query11 0.25 0.14 0.13
query12 0.26 0.14 0.14
query13 0.64 0.63 0.61
query14 1.03 1.01 1.03
query15 0.94 0.85 0.85
query16 0.38 0.38 0.39
query17 1.04 1.05 1.06
query18 0.23 0.21 0.21
query19 1.92 1.82 1.84
query20 0.02 0.01 0.02
query21 15.40 0.28 0.24
query22 4.99 0.11 0.10
query23 15.36 0.38 0.23
query24 2.95 0.49 0.31
query25 0.09 0.09 0.09
query26 0.18 0.18 0.18
query27 0.09 0.10 0.09
query28 3.71 1.25 1.07
query29 12.58 3.96 3.28
query30 0.33 0.13 0.10
query31 2.83 0.63 0.44
query32 3.23 0.59 0.51
query33 3.02 3.07 3.06
query34 16.68 5.16 4.55
query35 4.50 4.53 4.60
query36 0.66 0.53 0.51
query37 0.22 0.09 0.08
query38 0.20 0.06 0.06
query39 0.06 0.05 0.05
query40 0.19 0.18 0.17
query41 0.13 0.07 0.06
query42 0.07 0.05 0.05
query43 0.05 0.06 0.06
Total cold run time: 99.04 s
Total hot run time: 28.37 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3526062715
# FE UT Coverage Report
Increment line coverage ` 0.00% (0/4)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57950_c9e94c75f46034e8e89d904927c7f056cb1181c3/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57950_c9e94c75f46034e8e89d904927c7f056cb1181c3/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3526481370
<details>
<summary>TPC-DS: <b>Total hot run time: 187691 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9e94c75f46034e8e89d904927c7f056cb1181c3, data reload: false
query1 1040 398 394 394
query2 6572 1711 1733 1711
query3 6750 224 222 222
query4 26343 23860 23176 23176
query5 4457 635 490 490
query6 351 250 246 246
query7 4664 495 301 301
query8 307 261 255 255
query9 8738 2605 2621 2605
query10 494 361 293 293
query11 15384 15018 14877 14877
query12 199 118 113 113
query13 1674 551 433 433
query14 10306 9432 9286 9286
query15 199 195 175 175
query16 7251 669 536 536
query17 1241 747 613 613
query18 1986 414 312 312
query19 205 203 174 174
query20 126 130 125 125
query21 222 136 117 117
query22 4018 4004 4050 4004
query23 34104 33033 32946 32946
query24 8482 2395 2391 2391
query25 641 542 468 468
query26 1242 272 163 163
query27 2755 487 360 360
query28 4384 2231 2189 2189
query29 871 632 528 528
query30 306 238 200 200
query31 902 790 772 772
query32 92 77 74 74
query33 616 388 339 339
query34 806 835 542 542
query35 803 849 760 760
query36 966 983 894 894
query37 117 112 91 91
query38 3515 3483 3401 3401
query39 1470 1431 1426 1426
query40 227 133 129 129
query41 74 64 65 64
query42 128 120 116 116
query43 494 514 478 478
query44 1224 752 754 752
query45 192 182 181 181
query46 888 985 645 645
query47 1753 1817 1711 1711
query48 388 434 324 324
query49 807 527 439 439
query50 644 716 411 411
query51 3934 3919 3832 3832
query52 109 107 98 98
query53 235 270 190 190
query54 311 283 272 272
query55 89 84 85 84
query56 313 308 310 308
query57 1168 1187 1100 1100
query58 306 272 271 271
query59 2575 2691 2552 2552
query60 340 350 324 324
query61 169 150 155 150
query62 794 717 673 673
query63 231 195 192 192
query64 4547 1152 913 913
query65 4062 3934 3916 3916
query66 1179 425 343 343
query67 15235 15199 14815 14815
query68 8200 919 588 588
query69 497 334 297 297
query70 1357 1322 1337 1322
query71 452 345 323 323
query72 6109 4849 4810 4810
query73 620 568 359 359
query74 8851 9075 8841 8841
query75 3552 3305 2790 2790
query76 3383 1147 743 743
query77 771 415 321 321
query78 9472 9457 8865 8865
query79 2457 826 607 607
query80 723 570 513 513
query81 517 266 233 233
query82 451 163 138 138
query83 308 266 254 254
query84 290 106 92 92
query85 918 469 457 457
query86 386 300 283 283
query87 3682 3709 3623 3623
query88 3814 2298 2278 2278
query89 393 326 300 300
query90 1850 226 232 226
query91 179 169 137 137
query92 77 70 66 66
query93 2080 958 649 649
query94 750 454 339 339
query95 410 324 316 316
query96 483 568 285 285
query97 2923 2967 2864 2864
query98 257 211 211 211
query99 1416 1404 1276 1276
Total cold run time: 274643 ms
Total hot run time: 187691 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3526847409
# BE UT Coverage Report
Increment line coverage `0.00% (0/10)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/c9e94c75f46034e8e89d904927c7f056cb1181c3_c9e94c75f46034e8e89d904927c7f056cb1181c3/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/c9e94c75f46034e8e89d904927c7f056cb1181c3_c9e94c75f46034e8e89d904927c7f056cb1181c3/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.80% (18280/34618) |
| Line Coverage | 38.17% (166194/435397) |
| Region Coverage | 33.16% (129185/389570) |
| Branch Coverage | 33.89% (55440/163568) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3520956105
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.56% (1683/2089) |
| Line Coverage | 66.67% (29669/44498) |
| Region Coverage | 67.25% (14811/22024) |
| Branch Coverage | 57.53% (7884/13704) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3521356475
<details>
<summary>TPC-H: <b>Total hot run time: 34534 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc, data reload: false
------ Round 1 ----------------------------------
q1 17430 5247 5035 5035
q2 2021 319 207 207
q3 10127 1273 704 704
q4 10232 905 385 385
q5 7501 2410 2408 2408
q6 186 169 135 135
q7 985 771 639 639
q8 9359 1386 1190 1190
q9 7402 5178 5164 5164
q10 6866 2228 1822 1822
q11 500 293 284 284
q12 381 371 240 240
q13 17822 3638 3045 3045
q14 231 233 219 219
q15 583 515 500 500
q16 1030 1000 945 945
q17 603 874 371 371
q18 7526 7107 7200 7107
q19 1102 961 548 548
q20 359 333 232 232
q21 3804 2662 2331 2331
q22 1132 1050 1023 1023
Total cold run time: 107182 ms
Total hot run time: 34534 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5172 5132 5084 5084
q2 262 354 231 231
q3 2143 2666 2300 2300
q4 1335 1773 1362 1362
q5 4199 4460 4389 4389
q6 228 175 137 137
q7 2070 1986 1821 1821
q8 2709 2674 2603 2603
q9 7341 7459 7376 7376
q10 3031 3327 2842 2842
q11 589 565 505 505
q12 723 817 629 629
q13 3662 3882 3338 3338
q14 286 327 263 263
q15 554 496 504 496
q16 1083 1208 1036 1036
q17 1207 1599 1451 1451
q18 7879 7681 7431 7431
q19 781 872 880 872
q20 1923 1979 1829 1829
q21 4762 4328 4411 4328
q22 1099 1086 992 992
Total cold run time: 53038 ms
Total hot run time: 51315 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #57950:
URL: https://github.com/apache/doris/pull/57950
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3527286754
# FE Regression Coverage Report
Increment line coverage ` 100.00% (4/4)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57950_c9e94c75f46034e8e89d904927c7f056cb1181c3_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57950_c9e94c75f46034e8e89d904927c7f056cb1181c3_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3520777563
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3521650192
# BE UT Coverage Report
Increment line coverage `0.00% (0/10)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc_0a047c8cb7a26f4b0b4c113e8bd6aaa3f676d8cc/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.81% (18279/34611) |
| Line Coverage | 38.17% (166147/435268) |
| Region Coverage | 33.14% (129074/389466) |
| Branch Coverage | 33.89% (55417/163503) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3530624959
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3526410266
<details>
<summary>TPC-H: <b>Total hot run time: 34379 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9e94c75f46034e8e89d904927c7f056cb1181c3, data reload: false
------ Round 1 ----------------------------------
q1 17493 5288 5031 5031
q2 2042 311 210 210
q3 10176 1300 735 735
q4 10242 981 382 382
q5 7554 2366 2412 2366
q6 185 172 140 140
q7 909 799 641 641
q8 9355 1372 1155 1155
q9 7173 5208 5255 5208
q10 6903 2232 1816 1816
q11 496 306 287 287
q12 362 380 229 229
q13 17800 3664 3013 3013
q14 238 237 212 212
q15 599 511 507 507
q16 1026 1003 936 936
q17 594 876 368 368
q18 7806 7227 7054 7054
q19 1089 944 559 559
q20 356 340 227 227
q21 3822 2556 2303 2303
q22 1058 1025 1000 1000
Total cold run time: 107278 ms
Total hot run time: 34379 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5139 5088 5112 5088
q2 259 330 238 238
q3 2178 2709 2302 2302
q4 1376 1773 1307 1307
q5 4209 4457 4583 4457
q6 215 172 136 136
q7 1980 2026 1886 1886
q8 2675 2570 2626 2570
q9 7482 7270 7436 7270
q10 3048 3288 2810 2810
q11 601 541 512 512
q12 680 775 649 649
q13 3533 3926 3310 3310
q14 308 295 301 295
q15 548 503 506 503
q16 1059 1104 1041 1041
q17 1243 1507 1450 1450
q18 8070 7656 7511 7511
q19 767 815 931 815
q20 1926 1943 1814 1814
q21 4868 4378 4683 4378
q22 1082 1056 997 997
Total cold run time: 53246 ms
Total hot run time: 51339 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [optimize](paimon): Avoid redundant hadoop conf setting in each split [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57950:
URL: https://github.com/apache/doris/pull/57950#issuecomment-3527180273
# BE Regression && UT Coverage Report
Increment line coverage `0.00% (0/10)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57950_c9e94c75f46034e8e89d904927c7f056cb1181c3_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57950_c9e94c75f46034e8e89d904927c7f056cb1181c3_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.53% (24333/34018) |
| Line Coverage | 58.01% (252978/436073) |
| Region Coverage | 53.37% (210850/395081) |
| Branch Coverage | 54.65% (89976/164643) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org