You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/03/19 09:33:28 UTC
[PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
w41ter opened a new pull request, #32473:
URL: https://github.com/apache/doris/pull/32473
## Proposed changes
Issue Number: close #xxx
This PR fixes the spark read with an incorrect partition version in the cloud mode
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011718169
<details>
<summary>TPC-DS: <b>Total hot run time: 182117 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
query1 919 366 347 347
query2 7539 2069 1954 1954
query3 6705 224 214 214
query4 31049 21118 20725 20725
query5 4311 417 413 413
query6 267 195 171 171
query7 4629 290 287 287
query8 231 169 170 169
query9 9870 2278 2243 2243
query10 457 240 260 240
query11 17231 14347 14270 14270
query12 137 94 85 85
query13 1644 415 401 401
query14 11615 10580 10731 10580
query15 273 208 199 199
query16 8115 294 266 266
query17 1910 588 542 542
query18 2067 289 294 289
query19 243 163 155 155
query20 92 91 88 88
query21 200 130 129 129
query22 4682 4476 4386 4386
query23 31842 31390 31371 31371
query24 10706 2884 2801 2801
query25 624 388 381 381
query26 1183 160 159 159
query27 2465 347 348 347
query28 7615 1911 1905 1905
query29 877 658 629 629
query30 308 147 148 147
query31 966 756 736 736
query32 94 62 58 58
query33 784 277 342 277
query34 1011 477 482 477
query35 833 606 613 606
query36 976 856 902 856
query37 129 81 76 76
query38 3561 3480 3505 3480
query39 1412 1394 1380 1380
query40 221 120 112 112
query41 49 48 50 48
query42 105 99 96 96
query43 511 445 457 445
query44 1210 717 721 717
query45 280 265 254 254
query46 1084 675 683 675
query47 1659 1597 1591 1591
query48 431 359 359 359
query49 1127 332 334 332
query50 763 369 371 369
query51 6664 6547 6651 6547
query52 100 89 84 84
query53 337 272 274 272
query54 304 242 259 242
query55 82 75 77 75
query56 244 294 228 228
query57 1074 1016 1025 1016
query58 228 219 203 203
query59 2967 2703 2572 2572
query60 261 240 245 240
query61 99 94 98 94
query62 602 406 409 406
query63 300 278 276 276
query64 5624 3872 3936 3872
query65 3084 3004 3006 3004
query66 855 375 338 338
query67 14702 14607 14485 14485
query68 5328 505 506 505
query69 571 377 377 377
query70 1253 1146 1176 1146
query71 420 277 272 272
query72 6365 2794 2692 2692
query73 726 318 319 318
query74 6867 6533 6598 6533
query75 3605 2868 2898 2868
query76 3578 882 1001 882
query77 625 275 272 272
query78 10359 9574 9619 9574
query79 7746 516 521 516
query80 1490 420 404 404
query81 531 214 222 214
query82 1445 201 218 201
query83 209 153 148 148
query84 286 83 85 83
query85 1582 388 351 351
query86 469 307 307 307
query87 3740 3590 3575 3575
query88 5184 2287 2272 2272
query89 474 367 367 367
query90 2021 180 177 177
query91 165 157 149 149
query92 58 49 47 47
query93 5821 498 474 474
query94 1154 179 177 177
query95 438 338 330 330
query96 586 271 267 267
query97 3066 2886 2862 2862
query98 217 210 207 207
query99 1066 786 759 759
Total cold run time: 301984 ms
Total hot run time: 182117 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006507504
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006507601
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011403510
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32473:
URL: https://github.com/apache/doris/pull/32473
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006572467
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011664168
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2009380877
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011694260
<details>
<summary>TPC-H: <b>Total hot run time: 38055 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
------ Round 1 ----------------------------------
q1 17613 4335 4318 4318
q2 2032 148 145 145
q3 10606 1127 1186 1127
q4 10233 840 849 840
q5 7470 3015 2944 2944
q6 204 125 124 124
q7 1046 580 568 568
q8 9327 1994 2010 1994
q9 7113 6541 6491 6491
q10 8380 3444 3534 3444
q11 430 216 226 216
q12 415 203 198 198
q13 17807 2846 2850 2846
q14 226 206 209 206
q15 512 466 456 456
q16 472 353 358 353
q17 962 617 544 544
q18 7214 6633 6407 6407
q19 4479 1458 1489 1458
q20 546 251 282 251
q21 3620 2919 2834 2834
q22 334 301 291 291
Total cold run time: 111041 ms
Total hot run time: 38055 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4191 4085 4099 4085
q2 316 220 225 220
q3 2985 2892 2827 2827
q4 1858 1550 1596 1550
q5 5240 5296 5293 5293
q6 198 117 118 117
q7 2252 1857 1824 1824
q8 3140 3378 3277 3277
q9 8636 8581 8577 8577
q10 3735 3649 3718 3649
q11 551 457 442 442
q12 715 571 536 536
q13 16934 2878 2865 2865
q14 270 244 254 244
q15 486 443 451 443
q16 464 424 436 424
q17 1736 1481 1490 1481
q18 7371 7180 7214 7180
q19 1628 1584 1509 1509
q20 1894 1697 1687 1687
q21 4898 4730 4779 4730
q22 528 478 483 478
Total cold run time: 70026 ms
Total hot run time: 53438 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011787389
<details>
<summary>TPC-H: <b>Total hot run time: 38374 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
------ Round 1 ----------------------------------
q1 17775 4521 4436 4436
q2 2991 173 150 150
q3 11925 1159 1223 1159
q4 10934 776 755 755
q5 7633 3033 3025 3025
q6 208 128 126 126
q7 1061 619 586 586
q8 9624 2093 2049 2049
q9 7180 6633 6604 6604
q10 9442 3237 3524 3237
q11 430 225 214 214
q12 375 194 196 194
q13 17851 2841 2831 2831
q14 235 217 204 204
q15 497 466 457 457
q16 468 361 356 356
q17 973 574 604 574
q18 7262 6538 6512 6512
q19 1560 1473 1509 1473
q20 542 249 248 248
q21 3636 2972 2879 2879
q22 350 305 314 305
Total cold run time: 112952 ms
Total hot run time: 38374 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4144 4166 4219 4166
q2 331 244 226 226
q3 3046 2862 2857 2857
q4 1839 1547 1519 1519
q5 5239 5298 5277 5277
q6 194 119 119 119
q7 2260 1876 1920 1876
q8 3183 3307 3307 3307
q9 8619 8634 8621 8621
q10 3741 3662 3723 3662
q11 546 443 434 434
q12 741 546 564 546
q13 16874 2889 2842 2842
q14 274 246 251 246
q15 490 452 454 452
q16 468 410 414 410
q17 1741 1507 1476 1476
q18 7514 7168 7208 7168
q19 2963 1588 1524 1524
q20 1907 1694 1697 1694
q21 4837 4853 4698 4698
q22 532 437 467 437
Total cold run time: 71483 ms
Total hot run time: 53557 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006572348
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011813565
<details>
<summary>TPC-DS: <b>Total hot run time: 181315 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
query1 915 351 338 338
query2 7631 2040 1983 1983
query3 6704 212 215 212
query4 31255 20786 20757 20757
query5 4365 487 410 410
query6 267 179 190 179
query7 4621 291 295 291
query8 248 189 167 167
query9 9899 2273 2278 2273
query10 457 251 239 239
query11 14599 14263 14261 14261
query12 136 94 93 93
query13 1645 416 412 412
query14 11716 10636 11143 10636
query15 254 203 190 190
query16 8089 253 256 253
query17 1954 556 512 512
query18 2086 274 261 261
query19 216 145 164 145
query20 93 85 83 83
query21 197 125 132 125
query22 4669 4536 4432 4432
query23 31839 31324 31195 31195
query24 11622 2831 2812 2812
query25 642 381 385 381
query26 1788 149 163 149
query27 3077 342 340 340
query28 8032 1858 1906 1858
query29 1016 625 610 610
query30 300 145 146 145
query31 946 733 704 704
query32 94 57 52 52
query33 754 250 244 244
query34 1106 461 489 461
query35 828 604 594 594
query36 949 877 866 866
query37 115 77 73 73
query38 3560 3454 3454 3454
query39 1415 1359 1379 1359
query40 290 110 109 109
query41 52 45 43 43
query42 99 92 97 92
query43 496 477 479 477
query44 1210 730 717 717
query45 270 252 255 252
query46 1059 674 674 674
query47 1693 1592 1588 1588
query48 434 340 350 340
query49 1188 326 344 326
query50 768 363 368 363
query51 6655 6560 6617 6560
query52 104 98 88 88
query53 336 273 277 273
query54 317 235 254 235
query55 85 83 73 73
query56 261 241 237 237
query57 1089 1002 992 992
query58 237 207 206 206
query59 2877 2626 2694 2626
query60 289 250 256 250
query61 115 115 112 112
query62 603 395 393 393
query63 304 277 281 277
query64 6783 3890 3765 3765
query65 3064 3025 2988 2988
query66 1108 356 352 352
query67 14821 14433 14417 14417
query68 5392 510 526 510
query69 553 386 374 374
query70 1218 1176 1211 1176
query71 418 283 289 283
query72 6362 2789 2608 2608
query73 713 316 313 313
query74 6968 6531 6604 6531
query75 3476 2858 2869 2858
query76 3395 808 1043 808
query77 379 253 259 253
query78 10300 9664 9574 9574
query79 9453 505 516 505
query80 2267 372 375 372
query81 542 208 208 208
query82 1420 202 203 202
query83 316 141 147 141
query84 279 83 81 81
query85 2160 328 306 306
query86 503 266 309 266
query87 3680 3504 3534 3504
query88 5111 2282 2286 2282
query89 552 367 381 367
query90 2010 178 175 175
query91 167 129 132 129
query92 69 49 49 49
query93 7122 496 483 483
query94 1257 177 174 174
query95 433 323 326 323
query96 597 272 263 263
query97 3054 2872 2872 2872
query98 226 220 205 205
query99 1114 764 746 746
Total cold run time: 308212 ms
Total hot run time: 181315 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011037096
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org