You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "suxiaogang223 (via GitHub)" <gi...@apache.org> on 2026/03/24 06:31:33 UTC
[PR] [refactor](fe) Extract Iceberg planning split producer [doris]
suxiaogang223 opened a new pull request, #61663:
URL: https://github.com/apache/doris/pull/61663
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
- Extract Iceberg batch split planning into a dedicated FE-side producer abstraction.
- Add PlanningSplitProducer and SplitSink so split production can evolve independently from SplitAssignment and SplitSource.
- Move the existing Iceberg local async planning path into LocalParallelPlanningSplitProducer without changing the downstream split consumption flow.
- Add FE unit coverage for producer behavior and SplitAssignment lifecycle delegation.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132598646
# FE UT Coverage Report
Increment line coverage ` 35.32% (178/504)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61663_cde10de764019768092d0ab7aacfb5dc97bad890/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61663_cde10de764019768092d0ab7aacfb5dc97bad890/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4128306148
<details>
<summary>TPC-H: <b>Total hot run time: 26628 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 717650219675b2da4548a5addf955827a5b7f44f, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17595 4440 4308 4308
q2 q3 10651 790 529 529
q4 4672 368 255 255
q5 7558 1216 1023 1023
q6 177 176 147 147
q7 783 831 677 677
q8 9310 1489 1304 1304
q9 4846 4752 4661 4661
q10 6266 1920 1632 1632
q11 459 268 259 259
q12 734 587 474 474
q13 18022 2676 1942 1942
q14 221 233 217 217
q15 q16 733 748 670 670
q17 729 859 443 443
q18 5988 5407 5359 5359
q19 1242 993 606 606
q20 535 481 371 371
q21 4934 1846 1435 1435
q22 446 459 316 316
Total cold run time: 95901 ms
Total hot run time: 26628 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4820 4687 4601 4601
q2 q3 3857 4321 3823 3823
q4 887 1195 768 768
q5 4106 4356 4382 4356
q6 187 180 140 140
q7 1762 1704 1549 1549
q8 2486 2679 2592 2592
q9 7712 7411 7486 7411
q10 3738 4015 3629 3629
q11 524 455 419 419
q12 495 577 458 458
q13 2445 2875 2075 2075
q14 275 293 285 285
q15 q16 707 774 770 770
q17 1213 1482 1363 1363
q18 7097 6799 6698 6698
q19 936 918 903 903
q20 2103 2228 2009 2009
q21 3952 3461 3411 3411
q22 466 422 379 379
Total cold run time: 49768 ms
Total hot run time: 47639 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4133611330
<details>
<summary>TPC-DS: <b>Total hot run time: 169458 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4559afcc163219692c584be26eb7a3e9410d8e5e, data reload: false
query5 4318 655 519 519
query6 327 228 212 212
query7 4230 460 264 264
query8 334 249 238 238
query9 8782 2770 2791 2770
query10 521 369 341 341
query11 6990 5097 4869 4869
query12 183 132 124 124
query13 1272 457 349 349
query14 5818 3741 3440 3440
query14_1 2884 2836 2821 2821
query15 213 201 174 174
query16 1004 472 371 371
query17 1120 725 640 640
query18 2450 455 358 358
query19 230 211 183 183
query20 141 126 127 126
query21 218 136 113 113
query22 13282 14418 14764 14418
query23 16699 16309 16202 16202
query23_1 15974 16480 15704 15704
query24 7155 1618 1217 1217
query24_1 1255 1243 1216 1216
query25 547 468 403 403
query26 1224 258 145 145
query27 2786 483 295 295
query28 4991 1905 1864 1864
query29 913 588 501 501
query30 308 226 191 191
query31 1036 969 920 920
query32 82 70 73 70
query33 530 340 296 296
query34 895 883 531 531
query35 662 693 628 628
query36 1124 1151 1021 1021
query37 132 96 86 86
query38 2983 2926 2857 2857
query39 863 839 812 812
query39_1 810 801 814 801
query40 234 153 132 132
query41 64 60 62 60
query42 259 257 258 257
query43 255 261 222 222
query44
query45 200 190 181 181
query46 877 1011 613 613
query47 2081 2125 2047 2047
query48 323 321 233 233
query49 636 470 390 390
query50 679 278 215 215
query51 4103 4092 3982 3982
query52 258 270 257 257
query53 289 343 280 280
query54 295 281 270 270
query55 87 84 85 84
query56 334 330 306 306
query57 1906 1824 1610 1610
query58 282 273 270 270
query59 2791 2914 2741 2741
query60 339 332 331 331
query61 159 156 151 151
query62 652 585 547 547
query63 307 275 271 271
query64 4989 1280 1055 1055
query65
query66 1457 450 361 361
query67 24202 24226 24200 24200
query68
query69 415 318 284 284
query70 980 915 978 915
query71 332 304 285 285
query72 2880 2709 2414 2414
query73 544 549 319 319
query74 9632 9611 9400 9400
query75 2893 2750 2425 2425
query76 2281 1032 677 677
query77 368 396 299 299
query78 10893 11103 10461 10461
query79 1107 771 563 563
query80 769 641 535 535
query81 489 258 224 224
query82 1342 157 123 123
query83 344 266 245 245
query84 252 120 97 97
query85 928 498 464 464
query86 385 337 301 301
query87 3148 3106 3087 3087
query88 3571 2685 2660 2660
query89 423 363 347 347
query90 1842 185 177 177
query91 180 167 147 147
query92 77 76 77 76
query93 917 834 507 507
query94 485 335 286 286
query95 589 341 384 341
query96 655 541 230 230
query97 2452 2464 2429 2429
query98 241 218 225 218
query99 1025 1000 906 906
Total cold run time: 250086 ms
Total hot run time: 169458 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4128376442
<details>
<summary>TPC-DS: <b>Total hot run time: 168903 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 717650219675b2da4548a5addf955827a5b7f44f, data reload: false
query5 4341 625 498 498
query6 325 224 203 203
query7 4218 462 263 263
query8 332 238 231 231
query9 8662 2695 2703 2695
query10 533 379 337 337
query11 6981 5094 4855 4855
query12 184 127 124 124
query13 1270 471 350 350
query14 5732 3644 3452 3452
query14_1 2910 2840 2819 2819
query15 202 196 177 177
query16 996 470 461 461
query17 1125 734 634 634
query18 2462 454 355 355
query19 218 210 185 185
query20 139 127 134 127
query21 217 138 110 110
query22 13253 14022 14758 14022
query23 16795 16093 16245 16093
query23_1 15654 15647 15617 15617
query24 7222 1592 1215 1215
query24_1 1212 1207 1230 1207
query25 550 474 419 419
query26 1220 261 147 147
query27 2787 477 289 289
query28 4482 1858 1826 1826
query29 825 556 472 472
query30 298 225 188 188
query31 1011 937 898 898
query32 85 68 69 68
query33 507 334 293 293
query34 885 865 509 509
query35 658 674 601 601
query36 1081 1147 992 992
query37 132 95 87 87
query38 2963 2855 2806 2806
query39 865 818 803 803
query39_1 806 784 803 784
query40 235 152 139 139
query41 64 58 61 58
query42 267 257 254 254
query43 236 247 212 212
query44
query45 198 186 180 180
query46 867 991 607 607
query47 2114 2150 2056 2056
query48 312 312 266 266
query49 626 465 376 376
query50 686 283 219 219
query51 4091 4124 4006 4006
query52 259 266 253 253
query53 288 338 278 278
query54 293 265 270 265
query55 89 86 79 79
query56 305 321 303 303
query57 1901 1720 1789 1720
query58 283 276 269 269
query59 2832 2940 2740 2740
query60 347 339 329 329
query61 159 149 156 149
query62 619 595 544 544
query63 310 275 283 275
query64 5056 1279 1009 1009
query65
query66 1466 444 355 355
query67 24333 24350 24237 24237
query68
query69 401 311 301 301
query70 950 964 954 954
query71 353 304 301 301
query72 2866 2865 2599 2599
query73 549 555 310 310
query74 9648 9540 9399 9399
query75 2886 2785 2463 2463
query76 2299 1035 664 664
query77 374 396 342 342
query78 11017 11081 10543 10543
query79 1122 801 567 567
query80 680 619 539 539
query81 488 258 228 228
query82 1359 149 117 117
query83 372 265 238 238
query84 252 118 104 104
query85 858 519 451 451
query86 382 310 324 310
query87 3161 3115 3012 3012
query88 3502 2627 2608 2608
query89 442 363 340 340
query90 1953 177 173 173
query91 180 160 145 145
query92 79 76 72 72
query93 901 840 489 489
query94 461 302 280 280
query95 585 337 392 337
query96 645 529 223 223
query97 2432 2505 2408 2408
query98 241 232 215 215
query99 1009 954 918 918
Total cold run time: 249323 ms
Total hot run time: 168903 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4125159344
<details>
<summary>TPC-DS: <b>Total hot run time: 168573 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 533cba96c354a22d67f2853a028698a2f84494e5, data reload: false
query5 4330 612 512 512
query6 331 217 200 200
query7 4223 459 280 280
query8 352 239 220 220
query9 8721 2685 2691 2685
query10 614 399 327 327
query11 7013 5082 4851 4851
query12 179 126 123 123
query13 1281 438 356 356
query14 5737 3666 3408 3408
query14_1 2826 2773 2758 2758
query15 202 192 171 171
query16 968 450 435 435
query17 857 699 595 595
query18 2433 432 341 341
query19 243 202 175 175
query20 138 128 128 128
query21 214 130 107 107
query22 13338 14285 14532 14285
query23 16572 16496 16106 16106
query23_1 16340 15951 15657 15657
query24 7267 1589 1214 1214
query24_1 1213 1202 1211 1202
query25 544 457 390 390
query26 1237 259 150 150
query27 2786 481 291 291
query28 4552 1828 1848 1828
query29 839 565 474 474
query30 304 212 187 187
query31 1014 966 867 867
query32 84 67 69 67
query33 513 323 275 275
query34 914 863 539 539
query35 637 678 590 590
query36 1063 1115 1036 1036
query37 141 99 89 89
query38 2900 2914 2855 2855
query39 863 825 814 814
query39_1 797 791 790 790
query40 242 159 172 159
query41 63 58 60 58
query42 260 255 251 251
query43 233 253 213 213
query44
query45 195 188 183 183
query46 887 978 592 592
query47 2138 2131 2065 2065
query48 308 316 228 228
query49 630 465 377 377
query50 683 276 205 205
query51 4028 4017 4044 4017
query52 273 268 255 255
query53 287 342 289 289
query54 301 262 257 257
query55 96 86 81 81
query56 319 323 302 302
query57 1908 1763 1573 1573
query58 280 280 283 280
query59 2772 2923 2734 2734
query60 341 339 317 317
query61 151 150 149 149
query62 634 594 521 521
query63 309 278 278 278
query64 5060 1271 981 981
query65
query66 1467 450 352 352
query67 24186 24282 24101 24101
query68
query69 395 310 284 284
query70 956 897 937 897
query71 345 302 298 298
query72 2752 2629 2511 2511
query73 548 539 320 320
query74 9626 9557 9390 9390
query75 2900 2767 2468 2468
query76 2291 1035 686 686
query77 369 386 319 319
query78 10927 11080 10452 10452
query79 1106 760 558 558
query80 727 653 569 569
query81 472 262 224 224
query82 1353 149 122 122
query83 370 270 248 248
query84 304 122 96 96
query85 943 562 513 513
query86 361 308 308 308
query87 3168 3124 2999 2999
query88 3551 2652 2646 2646
query89 441 364 345 345
query90 1937 181 170 170
query91 183 177 181 177
query92 73 74 70 70
query93 908 835 495 495
query94 454 333 305 305
query95 601 332 308 308
query96 652 531 232 232
query97 2496 2485 2381 2381
query98 236 222 219 219
query99 903 981 918 918
Total cold run time: 249182 ms
Total hot run time: 168573 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4133341978
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4128182987
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115947233
<details>
<summary>TPC-DS: <b>Total hot run time: 168724 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7dd02767bed05a619a228a99ff8e9df1be84faf4, data reload: false
query5 4344 632 494 494
query6 350 225 207 207
query7 4201 465 264 264
query8 354 245 232 232
query9 8726 2731 2705 2705
query10 521 374 336 336
query11 6954 5122 4897 4897
query12 180 130 124 124
query13 1278 471 345 345
query14 5736 3685 3465 3465
query14_1 2842 2841 2789 2789
query15 208 186 175 175
query16 967 465 432 432
query17 887 694 604 604
query18 2431 439 336 336
query19 217 202 180 180
query20 136 126 124 124
query21 222 144 113 113
query22 13317 13971 14402 13971
query23 16870 16204 15975 15975
query23_1 16313 15525 15687 15525
query24 7184 1616 1228 1228
query24_1 1208 1246 1218 1218
query25 544 457 398 398
query26 1233 263 149 149
query27 2764 487 293 293
query28 4517 1827 1853 1827
query29 856 567 473 473
query30 296 224 192 192
query31 993 963 861 861
query32 77 71 69 69
query33 520 339 283 283
query34 900 879 517 517
query35 629 686 597 597
query36 1064 1073 989 989
query37 134 95 85 85
query38 2972 2939 2875 2875
query39 865 834 816 816
query39_1 796 776 796 776
query40 235 151 138 138
query41 60 59 58 58
query42 258 262 257 257
query43 244 250 228 228
query44
query45 204 199 184 184
query46 892 1011 606 606
query47 2114 2156 2453 2156
query48 306 321 234 234
query49 637 452 383 383
query50 688 288 217 217
query51 4050 4169 3963 3963
query52 264 265 258 258
query53 301 334 294 294
query54 304 274 285 274
query55 91 85 85 85
query56 322 309 321 309
query57 1899 1722 1731 1722
query58 291 279 282 279
query59 2764 2921 2736 2736
query60 349 339 312 312
query61 163 155 160 155
query62 630 590 525 525
query63 311 288 285 285
query64 5134 1297 1019 1019
query65
query66 1468 463 373 373
query67 24249 24346 24219 24219
query68
query69 419 316 319 316
query70 1014 890 926 890
query71 339 312 309 309
query72 2950 2847 2646 2646
query73 542 545 332 332
query74 9654 9560 9362 9362
query75 2854 2748 2472 2472
query76 2276 1042 671 671
query77 359 387 300 300
query78 10904 10951 10496 10496
query79 3186 749 572 572
query80 1738 644 543 543
query81 572 269 229 229
query82 1011 155 119 119
query83 338 255 245 245
query84 303 122 97 97
query85 904 499 450 450
query86 493 310 327 310
query87 3123 3127 3012 3012
query88 3551 2652 2626 2626
query89 421 369 352 352
query90 2104 186 175 175
query91 168 162 142 142
query92 87 76 73 73
query93 2276 827 501 501
query94 646 312 290 290
query95 579 346 380 346
query96 646 515 227 227
query97 2468 2452 2357 2357
query98 241 223 220 220
query99 1006 996 934 934
Total cold run time: 253394 ms
Total hot run time: 168724 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115904014
<details>
<summary>TPC-H: <b>Total hot run time: 26655 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7dd02767bed05a619a228a99ff8e9df1be84faf4, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17636 4435 4274 4274
q2 q3 10646 791 524 524
q4 4678 351 245 245
q5 7559 1213 1021 1021
q6 184 173 146 146
q7 771 854 684 684
q8 9346 1490 1365 1365
q9 4880 4752 4714 4714
q10 6331 1928 1716 1716
q11 474 260 246 246
q12 737 595 457 457
q13 18046 2713 1916 1916
q14 223 234 210 210
q15 q16 745 731 670 670
q17 749 864 433 433
q18 5945 5311 5322 5311
q19 1377 968 618 618
q20 542 498 381 381
q21 4695 1862 1400 1400
q22 383 512 324 324
Total cold run time: 95947 ms
Total hot run time: 26655 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4766 4604 4562 4562
q2 q3 4007 4416 3850 3850
q4 918 1222 779 779
q5 4040 4411 4394 4394
q6 193 180 147 147
q7 1802 1635 1514 1514
q8 2494 2703 2545 2545
q9 7700 7530 7322 7322
q10 3728 4035 3659 3659
q11 515 432 434 432
q12 483 599 452 452
q13 2466 3066 2043 2043
q14 279 292 275 275
q15 q16 761 778 705 705
q17 1179 1385 1359 1359
q18 7082 6758 6925 6758
q19 968 929 937 929
q20 2086 2154 2026 2026
q21 4052 3556 3447 3447
q22 458 436 389 389
Total cold run time: 49977 ms
Total hot run time: 47587 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4125093750
<details>
<summary>TPC-H: <b>Total hot run time: 26553 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 533cba96c354a22d67f2853a028698a2f84494e5, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17624 4515 4291 4291
q2 q3 10644 791 520 520
q4 4678 351 249 249
q5 7558 1208 1003 1003
q6 175 173 151 151
q7 790 847 682 682
q8 9295 1454 1330 1330
q9 4914 4734 4668 4668
q10 6294 1903 1664 1664
q11 461 249 249 249
q12 754 567 459 459
q13 18023 2679 1918 1918
q14 237 233 216 216
q15 q16 722 715 659 659
q17 723 810 465 465
q18 5915 5367 5355 5355
q19 1125 989 625 625
q20 531 495 372 372
q21 4482 1816 1402 1402
q22 342 299 275 275
Total cold run time: 95287 ms
Total hot run time: 26553 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4856 4664 4780 4664
q2 q3 3855 4370 3822 3822
q4 883 1179 773 773
q5 4125 4406 4312 4312
q6 184 182 144 144
q7 1759 1680 1528 1528
q8 2496 2728 2556 2556
q9 7593 7479 7435 7435
q10 3769 4023 3583 3583
q11 503 428 419 419
q12 502 583 436 436
q13 2488 2963 2245 2245
q14 435 420 293 293
q15 q16 727 786 712 712
q17 1204 1397 1371 1371
q18 7212 6968 6595 6595
q19 940 896 1037 896
q20 2174 2191 1993 1993
q21 3969 3495 3359 3359
q22 442 433 395 395
Total cold run time: 50116 ms
Total hot run time: 47531 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4133545502
<details>
<summary>TPC-H: <b>Total hot run time: 26595 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4559afcc163219692c584be26eb7a3e9410d8e5e, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17650 4494 4314 4314
q2 q3 10647 759 504 504
q4 4682 352 252 252
q5 7562 1219 1031 1031
q6 179 178 151 151
q7 804 851 677 677
q8 9791 1482 1353 1353
q9 5363 4745 4714 4714
q10 6310 1904 1637 1637
q11 477 255 242 242
q12 756 579 461 461
q13 18048 2745 1957 1957
q14 225 228 208 208
q15 q16 734 720 660 660
q17 747 864 422 422
q18 5926 5447 5296 5296
q19 1449 975 634 634
q20 546 480 378 378
q21 4550 1869 1457 1457
q22 334 295 247 247
Total cold run time: 96780 ms
Total hot run time: 26595 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4435 4351 4346 4346
q2 q3 3829 4280 3773 3773
q4 849 1154 774 774
q5 4010 4315 4315 4315
q6 186 177 143 143
q7 1730 1591 1504 1504
q8 2439 2631 2526 2526
q9 7829 7633 7599 7599
q10 3783 3984 3595 3595
q11 510 441 425 425
q12 497 604 455 455
q13 2531 2913 2074 2074
q14 308 309 274 274
q15 q16 770 749 752 749
q17 1228 1350 1402 1350
q18 7246 6873 6728 6728
q19 967 948 959 948
q20 2113 2161 1985 1985
q21 4114 3495 3372 3372
q22 451 455 399 399
Total cold run time: 49825 ms
Total hot run time: 47334 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115758888
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4124943261
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4125459709
# FE UT Coverage Report
Increment line coverage ` 20.61% (101/490)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61663_533cba96c354a22d67f2853a028698a2f84494e5/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61663_533cba96c354a22d67f2853a028698a2f84494e5/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115765120
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 closed pull request #61663: [refactor](fe) Unify Iceberg split planning via producer
URL: https://github.com/apache/doris/pull/61663
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4139793953
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132195351
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132614375
<details>
<summary>TPC-H: <b>Total hot run time: 26779 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cde10de764019768092d0ab7aacfb5dc97bad890, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17627 4459 4288 4288
q2 q3 10640 775 519 519
q4 4670 369 248 248
q5 7554 1238 1044 1044
q6 175 174 145 145
q7 789 861 684 684
q8 9942 1514 1400 1400
q9 5360 4715 4716 4715
q10 6348 1941 1623 1623
q11 478 255 236 236
q12 750 575 458 458
q13 18028 2690 1929 1929
q14 237 235 209 209
q15 q16 771 728 673 673
q17 731 832 475 475
q18 5991 5370 5235 5235
q19 1233 994 661 661
q20 551 493 380 380
q21 4578 1833 1575 1575
q22 464 366 282 282
Total cold run time: 96917 ms
Total hot run time: 26779 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4772 4565 4562 4562
q2 q3 3900 4355 3996 3996
q4 906 1302 791 791
q5 4113 4422 4402 4402
q6 189 175 148 148
q7 1777 1657 1553 1553
q8 2517 3115 2563 2563
q9 7529 7559 7527 7527
q10 3778 3984 3618 3618
q11 496 438 418 418
q12 481 599 456 456
q13 3012 2914 2081 2081
q14 295 313 293 293
q15 q16 739 775 733 733
q17 1166 1397 1376 1376
q18 7305 6865 6772 6772
q19 934 1019 926 926
q20 2089 2166 2009 2009
q21 3947 3472 3351 3351
q22 474 432 375 375
Total cold run time: 50419 ms
Total hot run time: 47950 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132673908
<details>
<summary>TPC-DS: <b>Total hot run time: 169433 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cde10de764019768092d0ab7aacfb5dc97bad890, data reload: false
query5 4336 659 507 507
query6 321 229 212 212
query7 4225 464 267 267
query8 367 254 234 234
query9 8686 2735 2748 2735
query10 533 389 356 356
query11 6996 5109 4906 4906
query12 193 126 126 126
query13 1274 480 348 348
query14 5723 3714 3494 3494
query14_1 2875 2919 2775 2775
query15 200 187 174 174
query16 959 464 443 443
query17 862 701 605 605
query18 2430 437 346 346
query19 212 206 178 178
query20 139 126 122 122
query21 210 129 111 111
query22 13169 14148 14618 14148
query23 16635 16210 15979 15979
query23_1 16278 15773 15622 15622
query24 7205 1624 1219 1219
query24_1 1217 1256 1235 1235
query25 535 461 395 395
query26 1245 261 152 152
query27 2784 479 296 296
query28 4512 1847 1836 1836
query29 832 565 465 465
query30 299 221 187 187
query31 1024 934 873 873
query32 83 72 68 68
query33 509 330 280 280
query34 885 871 527 527
query35 651 707 606 606
query36 1050 1117 965 965
query37 128 103 80 80
query38 2906 2880 2903 2880
query39 864 827 804 804
query39_1 784 800 794 794
query40 229 155 137 137
query41 64 90 59 59
query42 269 257 257 257
query43 239 247 218 218
query44
query45 201 191 179 179
query46 877 980 609 609
query47 2121 2114 2606 2114
query48 321 312 245 245
query49 633 451 389 389
query50 685 278 210 210
query51 4170 4070 4057 4057
query52 268 263 258 258
query53 290 337 292 292
query54 307 281 269 269
query55 93 92 82 82
query56 321 327 320 320
query57 1938 1664 1685 1664
query58 283 268 265 265
query59 2773 2956 2751 2751
query60 349 341 325 325
query61 152 152 161 152
query62 609 583 538 538
query63 304 282 276 276
query64 5065 1308 1066 1066
query65
query66 1469 454 362 362
query67 24342 24490 24173 24173
query68
query69 412 311 279 279
query70 974 964 926 926
query71 339 324 297 297
query72 2842 2726 2578 2578
query73 552 533 323 323
query74 9650 9578 9439 9439
query75 2877 2780 2474 2474
query76 2290 1059 691 691
query77 363 392 324 324
query78 10857 11063 10446 10446
query79 1120 772 583 583
query80 738 643 580 580
query81 490 261 231 231
query82 1346 159 126 126
query83 334 291 249 249
query84 308 123 109 109
query85 944 595 557 557
query86 396 312 298 298
query87 3157 3110 3034 3034
query88 3524 2690 2665 2665
query89 435 372 355 355
query90 1944 193 183 183
query91 192 180 163 163
query92 79 76 77 76
query93 907 849 511 511
query94 479 329 296 296
query95 608 376 395 376
query96 643 525 230 230
query97 2512 2471 2419 2419
query98 250 223 221 221
query99 1003 993 917 917
Total cold run time: 248742 ms
Total hot run time: 169433 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org